Compare commits

..
Author SHA1 Message Date
Andrew Beveridge f99d6c1fb4 style: remove extraneous f-string prefixes
Address CodeRabbit nitpick comments - remove f-string prefixes from
strings without placeholders.
2025-12-19 18:42:03 -05:00
Andrew Beveridge f8a45bfbd2 fix: countdown padding synchronization for instrumental audio
When LyricsTranscriber adds 3-second countdown padding to vocals,
the instrumental audio must be padded to match. This was broken when
transcription was skipped due to existing files.

Changes:
- Add _detect_countdown_padding_from_lrc() to detect padding from existing LRC
- Update early returns in transcribe_lyrics() to include padding info
- Add _scan_directory_for_instrumentals() to find existing instrumentals
- Fix counting bug in apply_countdown_padding_to_instrumentals
- Add countdown_padding_seconds safety net to KaraokeFinalise
- Store padding metadata in lyrics_worker job state_data
- Pass padding info to video_worker for instrumental sync
- Add comprehensive unit tests for new functionality
2025-12-19 18:35:37 -05:00
613 changed files with 14751 additions and 135902 deletions

No files matched your search

-393
View File
@@ -1,393 +0,0 @@
# Plan: Admin Job Details Page
**Created:** 2026-01-09
**Branch:** feat/sess-20260109-1738-admin-job-details
**Status:** In Progress (Phase 1 Complete)
## Overview
The admin jobs list page links to `/admin/jobs/{job_id}` but that route doesn't exist - the detail page is at `/admin/jobs/detail` expecting `?id=` query params. This causes a 404 when clicking jobs.
We'll fix the routing and enhance the job detail page with comprehensive admin features: full job metadata display, file downloads with signed URLs, editable fields, and job stage reset capabilities.
**Note:** Due to Next.js static export constraints (the frontend deploys to Cloudflare Pages), we cannot use dynamic routes like `/admin/jobs/[jobId]`. Instead, we use query parameters: `/admin/jobs?id={job_id}`. The single page component handles both list and detail views based on the presence of the `id` query param.
## Requirements
### Must Have
- [x] Fix 404 - use query param routing at `/admin/jobs?id={job_id}` (dynamic routes not compatible with static export)
- [ ] Display all job metadata (source, user info, request_metadata, timeline)
- [ ] Stage-by-stage breakdown with timestamps and duration calculations
- [ ] Download links for all existing files (generate signed URLs)
- [ ] View worker logs with filtering by level/worker
- [ ] Restart job from beginning (reset to PENDING)
- [ ] Edit key job fields inline (artist, title, user_email, theme_id, etc.)
### Should Have
- [ ] Reset job to specific intermediate stages (audio selection, lyrics review)
- [ ] View/edit raw JSON fields (state_data, file_urls)
- [ ] Quick actions: send completion email, clear cache, impersonate user
### Nice to Have
- [ ] Job timeline visualization (Gantt-style or vertical timeline)
- [ ] Diff view when editing fields
- [ ] Audit log of admin changes
## Technical Approach
### Routing Fix (COMPLETED)
~~Create `/admin/jobs/[jobId]/page.tsx` as a dynamic route.~~ **UPDATE:** Dynamic routes are incompatible with Next.js `output: 'export'` (static site generation for Cloudflare Pages) without `generateStaticParams()`, which doesn't work for client components.
**Actual approach:** Enhanced `/admin/jobs/page.tsx` to handle both list and detail views:
- Uses `useSearchParams()` to detect `?id=` parameter
- Shows detail view when `selectedJobId` is present, list view otherwise
- Navigation uses `router.push(\`/admin/jobs?id=${job.job_id}\`)`
- Deleted old `/admin/jobs/detail/` directory
### Backend Enhancements
1. **New endpoint**: `GET /api/admin/jobs/{job_id}/files` - returns all files with signed download URLs
2. **New endpoint**: `PATCH /api/admin/jobs/{job_id}` - update editable fields (admin only)
3. **New endpoint**: `POST /api/admin/jobs/{job_id}/reset` - reset job to specific state
### Frontend Structure
The page will have collapsible sections:
1. **Header**: Job ID, status badge, artist/title, quick actions
2. **Overview Cards**: User, dates, progress, source info
3. **Stage Timeline**: Visual breakdown of each stage with times
4. **Files**: Downloadable links for all GCS files
5. **Logs**: Filterable worker logs
6. **Configuration**: Editable job settings
7. **Raw Data**: JSON views of state_data, file_urls, full job
### Job Reset Logic
Reset to specific states requires:
- Clearing downstream state_data keys
- Resetting status
- Optionally clearing associated files
- Adding timeline event for the reset
Valid reset targets:
| Target State | Clears | Use Case |
|--------------|--------|----------|
| PENDING | Everything except input | Full restart |
| AWAITING_AUDIO_SELECTION | Audio search results | Re-search for audio |
| AWAITING_REVIEW | Review data, screens, videos | Re-do lyrics review |
| AWAITING_INSTRUMENTAL_SELECTION | Instrumental selection, finals | Re-choose instrumental |
## Implementation Steps
Testing is integrated into each phase (following TDD/"write tests immediately after" guidance).
### Phase 1: Fix Routing (COMPLETED)
1. [x] ~~Create `/frontend/app/admin/jobs/[jobId]/page.tsx`~~ Enhanced `/admin/jobs/page.tsx` with query param routing
2. [x] Combined list and detail views in single component
3. [x] Use `useSearchParams()` to detect `?id=` parameter
4. [x] Delete `/frontend/app/admin/jobs/detail/` directory
5. [x] **E2E Test**: Added `admin-job-detail.spec.ts` with 9 tests (navigation, metadata, timeline, logs, error states, delete dialog, refresh)
6. [x] Verified: All 9 E2E tests pass, build succeeds
### Phase 2: Backend - File Downloads with Signed URLs
7. [ ] **Write tests first**: Add test file `backend/tests/test_admin_job_detail.py`
8. [ ] Add tests for `GET /api/admin/jobs/{job_id}/files` (see Testing Strategy)
9. [ ] Add `GET /api/admin/jobs/{job_id}/files` endpoint
10. [ ] Traverse job.file_urls recursively
11. [ ] Generate signed URLs for each GCS path
12. [ ] Return structured response with file metadata
13. [ ] Verify: `poetry run pytest backend/tests/test_admin_job_detail.py -v` passes
14. [ ] Add frontend API method `adminApi.getJobFiles(jobId)`
### Phase 3: Frontend - Enhanced File Display
15. [ ] Create "Files" accordion section with grouped downloads
16. [ ] Group by category (input, stems, lyrics, screens, videos, finals, packages)
17. [ ] Show file name, type, download button
18. [ ] Handle missing/unavailable files gracefully
19. [ ] **E2E Test**: Add files section test to `admin-job-detail.spec.ts`
20. [ ] Verify: `npm run test:e2e` passes
### Phase 4: Backend - Job Update Endpoint
21. [ ] **Write tests first**: Add tests for `PATCH /api/admin/jobs/{job_id}`
22. [ ] Add `PATCH /api/admin/jobs/{job_id}` endpoint
23. [ ] Define editable fields whitelist
24. [ ] Validate field values (e.g., status must be valid enum)
25. [ ] Log admin changes for audit trail
26. [ ] Verify: Backend tests pass
27. [ ] Add frontend API method `adminApi.updateJob(jobId, updates)`
### Phase 5: Frontend - Editable Fields
28. [ ] Create "Configuration" section with editable fields
29. [ ] Use inline edit pattern (click to edit, enter to save)
30. [ ] Fields: artist, title, user_email, theme_id, enable_cdg, enable_txt, enable_youtube_upload
31. [ ] Show validation errors inline
32. [ ] **E2E Test**: Add inline edit test to `admin-job-detail.spec.ts`
33. [ ] Verify: `npm run test:e2e` passes
### Phase 6: Backend - Job Reset Endpoint
34. [ ] **Write tests first**: Add tests for `POST /api/admin/jobs/{job_id}/reset`
35. [ ] Add `POST /api/admin/jobs/{job_id}/reset` endpoint
36. [ ] Accept `target_state` parameter
37. [ ] Implement reset logic for each target state
38. [ ] Clear appropriate state_data keys
39. [ ] Optionally trigger workers after reset
40. [ ] Verify: Backend tests pass
41. [ ] Add frontend API method `adminApi.resetJob(jobId, targetState)`
### Phase 7: Frontend - Reset Actions
42. [ ] Add "Reset Job" dropdown/menu in header
43. [ ] Show available reset targets based on current state
44. [ ] Confirm dialog explaining what will be cleared
45. [ ] Show success/error toast after reset
46. [ ] **E2E Test**: Add reset confirmation dialog test
47. [ ] Verify: `npm run test:e2e` passes
### Phase 8: Enhanced Stage Timeline
48. [ ] Parse timeline events to calculate stage durations
49. [ ] Display stages as cards or timeline items
50. [ ] Show duration for each stage
51. [ ] Highlight current/blocking stages
52. [ ] **E2E Test**: Add timeline display test
### Phase 9: Final Verification
53. [ ] Run full backend test suite: `make test-backend`
54. [ ] Run full frontend test suite: `cd frontend && npm run test:all`
55. [ ] Manual production testing checklist (see Testing Strategy)
56. [ ] Verify CI passes on PR
## Files to Create/Modify
| File | Action | Description |
|------|--------|-------------|
| `frontend/app/admin/jobs/[jobId]/page.tsx` | Create | Dynamic route for job detail |
| `frontend/app/admin/jobs/detail/` | Delete | Remove old query-param based route |
| `backend/api/routes/admin.py` | Modify | Add files, update, reset endpoints |
| `frontend/lib/api.ts` | Modify | Add adminApi methods for new endpoints |
| `backend/tests/test_admin_job_detail.py` | Create | Backend unit tests for new endpoints |
| `frontend/e2e/regression/admin-job-detail.spec.ts` | Create | Frontend E2E regression tests |
| `frontend/e2e/fixtures/test-helper.ts` | Modify | Add admin auth helper if needed |
## Testing Strategy
Based on `docs/TESTING.md` guidelines, this feature requires:
### Test Types Required
| Type | Location | Purpose |
|------|----------|---------|
| Backend Unit Tests | `backend/tests/test_admin_job_detail.py` | Test new API endpoints |
| Frontend E2E Regression | `frontend/e2e/regression/admin-job-detail.spec.ts` | Test UI flows with mocked API |
| Manual Production E2E | N/A | Final validation on staging/prod |
### Backend Unit Tests (`backend/tests/test_admin_job_detail.py`)
**Mocking approach:** Mock Firestore and GCS (following existing patterns in `backend/tests/conftest.py`). Do NOT mock our own code - test the real `JobManager` and `StorageService` logic.
**Test cases for `GET /api/admin/jobs/{job_id}/files`:**
```python
def test_get_job_files_returns_signed_urls_for_all_files():
"""Test that all files in file_urls get signed URLs."""
def test_get_job_files_handles_nested_file_urls():
"""Test recursive traversal of stems.instrumental_clean, etc."""
def test_get_job_files_handles_missing_files_gracefully():
"""Test files that don't exist in GCS are omitted or marked."""
def test_get_job_files_requires_admin():
"""Test non-admin gets 403."""
def test_get_job_files_returns_404_for_missing_job():
"""Test invalid job_id returns 404."""
```
**Test cases for `PATCH /api/admin/jobs/{job_id}`:**
```python
def test_update_job_allows_editable_fields():
"""Test artist, title, user_email, theme_id can be updated."""
def test_update_job_rejects_non_editable_fields():
"""Test job_id, created_at, etc. cannot be updated."""
def test_update_job_validates_status_enum():
"""Test invalid status value returns 422."""
def test_update_job_validates_theme_id_exists():
"""Test non-existent theme_id returns 422."""
def test_update_job_requires_admin():
"""Test non-admin gets 403."""
def test_update_job_returns_updated_job():
"""Test response includes updated job data."""
```
**Test cases for `POST /api/admin/jobs/{job_id}/reset`:**
```python
def test_reset_to_pending_clears_all_state_data():
"""Test full reset clears everything except input."""
def test_reset_to_audio_selection_clears_search_results():
"""Test audio selection reset clears audio_search_results."""
def test_reset_to_awaiting_review_clears_review_data():
"""Test review reset clears screens, videos, review data."""
def test_reset_to_instrumental_selection_clears_finals():
"""Test instrumental reset clears encoding results."""
def test_reset_adds_timeline_event():
"""Test reset adds admin action to timeline."""
def test_reset_invalid_target_returns_422():
"""Test invalid target_state returns validation error."""
def test_reset_requires_admin():
"""Test non-admin gets 403."""
def test_reset_from_complete_to_pending_works():
"""Test can reset completed job back to start."""
```
### Frontend E2E Regression Tests (`frontend/e2e/regression/admin-job-detail.spec.ts`)
**Mocking approach:** Use Playwright route interception to mock API responses (following patterns in `frontend/e2e/fixtures/test-helper.ts`).
```typescript
import { test, expect } from '@playwright/test';
import { setupApiFixtures, setAdminAuthToken } from '../fixtures/test-helper';
test.describe('Admin Job Detail Page', () => {
test.beforeEach(async ({ page }) => {
await setAdminAuthToken(page);
});
test('navigates from jobs list to detail page', async ({ page }) => {
// Mock jobs list and single job endpoints
await page.route('**/api/admin/jobs', route => route.fulfill({
json: [{ job_id: 'test123', status: 'complete', artist: 'Test', title: 'Song' }]
}));
await page.route('**/api/jobs/test123', route => route.fulfill({
json: { job_id: 'test123', status: 'complete', artist: 'Test', title: 'Song' }
}));
await page.goto('/admin/jobs');
await page.click('text=test123');
await expect(page).toHaveURL('/admin/jobs/test123');
await expect(page.getByText('Job test123')).toBeVisible();
});
test('displays all job metadata sections', async ({ page }) => {
// Test overview cards, timeline, logs accordion, etc.
});
test('files section shows downloadable links', async ({ page }) => {
// Mock files endpoint, verify download buttons appear
});
test('edit field inline and save', async ({ page }) => {
// Click edit, change value, press Enter, verify PATCH called
});
test('reset job shows confirmation dialog', async ({ page }) => {
// Click reset, verify dialog, confirm, verify POST called
});
test('handles API errors gracefully', async ({ page }) => {
// Mock 500 response, verify error toast shown
});
});
```
### Test File Organization
```
backend/tests/
├── test_admin_job_detail.py # NEW - all backend tests for this feature
├── test_admin_api.py # Existing admin tests (keep separate)
└── conftest.py # Shared fixtures
frontend/e2e/
├── fixtures/
│ ├── test-helper.ts # Add setAdminAuthToken helper
│ └── mock-responses.ts # Add admin job detail mocks
└── regression/
└── admin-job-detail.spec.ts # NEW - E2E regression tests
```
### Mocking Guidelines
**Backend - What to mock:**
- `StorageService.generate_signed_url()` - return predictable test URLs
- `StorageService.file_exists()` - control which files "exist"
- Firestore operations via existing `mock_firestore` fixture
**Backend - What NOT to mock:**
- `JobManager` logic - test the real reset/update logic
- Validation logic - test real Pydantic validation
- Authorization checks - test real `require_admin` dependency
**Frontend - What to mock:**
- All API responses via Playwright route interception
- Time (if testing token expiry)
**Frontend - What NOT to mock:**
- React component behavior
- User interactions (click, type, etc.)
### CI Requirements
This feature will be validated by existing CI checks:
- `backend-unit-tests` - Must pass (includes new `test_admin_job_detail.py`)
- `frontend-e2e-smoke` - Must pass (includes new `admin-job-detail.spec.ts`)
- `frontend-build` - Must pass (verifies new page compiles)
### Running Tests Locally
```bash
# Backend tests only (for this feature)
poetry run pytest backend/tests/test_admin_job_detail.py -v
# All backend tests
make test-backend
# Frontend E2E regression (includes new tests)
cd frontend && npm run test:e2e
# Single E2E test file
cd frontend && npx playwright test e2e/regression/admin-job-detail.spec.ts
# With browser visible for debugging
cd frontend && npx playwright test e2e/regression/admin-job-detail.spec.ts --headed
```
### Manual Production Testing Checklist
After deployment, manually verify on production:
- [ ] Navigate `/admin/jobs` → click job → lands on `/admin/jobs/{id}` (not 404)
- [ ] All file download links work (valid signed URLs)
- [ ] Edit artist/title → refresh → changes persisted
- [ ] Reset job to pending → job status shows "pending"
- [ ] Logs section loads and filters work
- [ ] Error states handled (invalid job ID shows friendly error)
## Open Questions
1. [ ] Should we keep the old `/admin/jobs/detail?id=` route for backwards compatibility, or delete it entirely?
- **Recommendation**: Delete it. No external links depend on it.
2. [ ] Should job reset delete GCS files or just clear state_data references?
- **Recommendation**: Clear state_data only. Files can be useful for debugging. Add a separate "Delete Files" action if needed.
3. [ ] Should edits be immediately saved or require explicit "Save" button?
- **Recommendation**: Inline edit with Enter to save, Escape to cancel. Show loading indicator during save.
## Rollback Plan
If issues arise:
1. Routing: Can revert to query-param routing by restoring `detail/page.tsx` and updating navigation
2. Backend: New endpoints are additive, won't affect existing functionality
3. Reset feature: Admin-only, can be disabled by removing endpoint if problematic
## Notes
- The existing detail page has good foundational code - we're enhancing rather than replacing
- Signed URLs expire (default 60-120 min) - consider caching or refresh mechanism for long sessions
- Job reset is powerful - consider adding confirmation dialogs and logging
@@ -1,218 +0,0 @@
# Plan: Job Email Notifications & Admin Message Tools
**Created:** 2026-01-06
**Branch:** feat/sess-20260106-0116-add-email-button
**Status:** Draft
## Overview
Add comprehensive email notification system for karaoke-gen that:
1. **Auto-sends emails** when jobs complete or need user input (after 5-minute idle threshold)
2. **Uses cloud-stored templates** for the job completion message
3. **Provides admin tools** to copy/send completion messages manually (for Fiverr, etc.)
## Requirements
### Functional Requirements
- [ ] Auto-send job completion email to user (CC: gen@nomadkaraoke.com)
- [ ] Auto-send "action needed" reminder emails when user is idle for 5+ minutes at blocking states (lyrics review, instrumental selection)
- [ ] Store email templates in GCS (editable without code deploy)
- [ ] Admin button: "Copy Message" - copies templated completion message to clipboard
- [ ] Admin button: "Send Email" - prompts for email address, then sends completion message
### Non-Functional Requirements
- [ ] Email sending must not block job completion (async/background)
- [ ] Idle detection via Cloud Tasks scheduled task (not polling)
- [ ] Template changes should take effect without restart (no long-term caching)
## Technical Approach
### 1. Template Storage (GCS)
Store templates in `gs://karaoke-gen-storage/templates/`:
- `job-completion.txt` - Plain text template for job completion
- `job-completion.html` - HTML version (optional, for rich emails)
- `action-needed-lyrics.txt` - Reminder for lyrics review
- `action-needed-instrumental.txt` - Reminder for instrumental selection
**Template Variables:**
- `{name}` - User's display name or "there" if unknown
- `{youtube_url}` - YouTube video URL
- `{dropbox_url}` - Dropbox folder URL
- `{artist}` - Artist name
- `{title}` - Song title
- `{job_id}` - Job ID
- `{review_url}` - Lyrics review URL (for action-needed emails)
- `{instrumental_url}` - Instrumental selection URL
- `{feedback_url}` - Feedback form URL (for completion emails)
### 2. Email Service Extensions
Extend `backend/services/email_service.py`:
- Add `send_job_completion()` method with CC support
- Add `send_action_reminder()` method
- Add `render_template()` helper to fetch & render GCS templates
### 3. Auto-Email Triggers
**Job Completion:**
- Trigger point: When job enters `COMPLETE` status in `job_manager.transition_to_state()`
- Implementation: Call email service async in background task
**Action Reminders (5-min idle detection):**
- When job enters `AWAITING_REVIEW` or `AWAITING_INSTRUMENTAL_SELECTION`:
- Record `blocking_state_entered_at` timestamp in `state_data`
- Schedule a Cloud Tasks task for 5 minutes later
- The scheduled task checks if job is still in blocking state and user hasn't started review
- If idle, send reminder email and set `reminder_sent` flag to prevent duplicates
- **Only one reminder per job** - if `reminder_sent` is true, skip sending
### 4. Admin UI (Frontend)
Add admin-only buttons to completed jobs in `OutputLinks.tsx`:
- "Copy Message" button - fetches rendered template, copies to clipboard
- "Send Email" button - opens dialog for email input, calls API to send
### 5. Backend API Endpoints
New endpoints in `backend/api/routes/admin.py`:
- `GET /api/admin/jobs/{job_id}/completion-message` - Returns rendered completion message
- `POST /api/admin/jobs/{job_id}/send-completion-email` - Sends email to specified address
New internal endpoint for scheduled reminder:
- `POST /api/internal/jobs/{job_id}/check-idle-reminder` - Called by Cloud Tasks
## Implementation Steps
### Phase 1: Template Infrastructure
1. [ ] Upload initial template to GCS from user's existing template file
2. [ ] Create `template_service.py` to fetch/render templates from GCS
3. [ ] Add unit tests for template rendering
### Phase 2: Email Extensions
4. [ ] Extend `email_service.py` with CC support in `send_email()`
5. [ ] Add `send_job_completion()` method
6. [ ] Add `send_action_reminder()` method
7. [ ] Add unit tests for new email methods
### Phase 3: Auto-Email on Completion
8. [ ] Modify `job_manager.py` to trigger completion email when entering COMPLETE
9. [ ] Add integration test for completion email trigger
### Phase 4: Idle Reminder System
10. [ ] Add `blocking_state_entered_at` field tracking to job state transitions
11. [ ] Create Cloud Tasks queue for idle reminders (`idle-reminder-queue`)
12. [ ] Schedule 5-minute delayed task when entering blocking states
13. [ ] Implement `/api/internal/jobs/{job_id}/check-idle-reminder` endpoint
14. [ ] Add tests for idle reminder flow
### Phase 5: Admin API
15. [ ] Add `GET /api/admin/jobs/{job_id}/completion-message` endpoint
16. [ ] Add `POST /api/admin/jobs/{job_id}/send-completion-email` endpoint
17. [ ] Add tests for admin endpoints
### Phase 6: Admin UI
18. [ ] Add `isAdmin` check to `OutputLinks.tsx` component
19. [ ] Add "Copy Message" button (admin-only) with loading state
20. [ ] Add "Send Email" button (admin-only) with email input dialog
21. [ ] Add E2E test for admin buttons
### Phase 7: Documentation
22. [ ] Document template management in docs/DEVELOPMENT.md
23. [ ] Update docs/API.md with new endpoints
## Files to Create/Modify
| File | Action | Description |
|------|--------|-------------|
| `backend/services/template_service.py` | Create | GCS template fetching and rendering |
| `backend/services/email_service.py` | Modify | Add CC support, completion/reminder methods |
| `backend/services/job_manager.py` | Modify | Trigger emails on state transitions |
| `backend/services/worker_service.py` | Modify | Add idle reminder task scheduling |
| `backend/api/routes/admin.py` | Modify | Add completion message/email endpoints |
| `backend/api/routes/internal.py` | Modify | Add idle reminder check endpoint |
| `backend/models/job.py` | Modify | Add `blocking_state_entered_at`, `reminder_sent_at` to state_data docs |
| `infrastructure/__main__.py` | Modify | Add idle-reminder-queue Cloud Tasks queue |
| `frontend/components/job/OutputLinks.tsx` | Modify | Add admin-only Copy/Send buttons |
| `frontend/lib/api.ts` | Modify | Add API methods for admin endpoints |
| `tests/unit/test_template_service.py` | Create | Template rendering tests |
| `tests/unit/test_email_service_completion.py` | Create | Completion email tests |
## Testing Strategy
**Unit Tests:**
- Template rendering with all variable combinations
- Email service CC support
- Completion/reminder email generation
- Idle detection logic
**Integration Tests:**
- Full flow: job completion → email sent
- Idle reminder: job enters blocking state → 5 min → reminder sent
- Admin endpoint authentication
**Manual Testing:**
- Verify emails render correctly in Gmail/Outlook
- Test admin buttons on completed job
- Test clipboard copy works across browsers
## Open Questions
- [x] Where to store templates? → **GCS bucket: `gs://karaoke-gen-storage/templates/`**
- [x] Should we rate-limit reminder emails? → **Max 1 reminder per job** (send once after 5 min idle, then stop)
- [x] Should completion emails also include a feedback request link? → **Yes, for all users**
## Rollback Plan
1. **Email sending failures** - All email sends are async/fire-and-forget; job completion unaffected
2. **Template fetch failures** - Fall back to hardcoded template in code
3. **Idle reminder failures** - Jobs continue normally; users can still access review UI manually
4. **Feature flag** - Add `ENABLE_AUTO_EMAILS=true/false` env var for quick disable
## Template Content
Initial template (from user's file, with feedback link added):
```
Hi {name},
Thanks for your order!
Here's the link for the karaoke video published to YouTube:
{youtube_url}
Here's the dropbox folder with all the finished files and source files, including:
- "(Final Karaoke Lossless).mkv": combined karaoke video in 4k H264 with lossless FLAC audio
- "(Final Karaoke).mp4": combined karaoke video with title/end screen in 4k H264/AAC
- "(Final Karaoke 720p).mp4": combined karaoke video in 720p H264/AAC (smaller file for older systems)
- "(With Vocals).mp4": sing along video in 4k H264/AAC with original vocals
- "(Karaoke).mov": karaoke video output from MidiCo (no title/end screen)
- "(Title).mov"/"(End).mov": title card and end screen videos
- "(Final Karaoke CDG).zip": CDG+MP3 format for older/commercial karaoke systems
- "(Final Karaoke TXT).zip": TXT+MP3 format for Power Karaoke
- stems/*.flac: various separated instrumental and vocal audio stems in lossless format
- lyrics/*.txt song lyrics from various sources in plain text format
{dropbox_url}
Let me know if anything isn't perfect and I'll happily tweak / fix, or if you need it in any other format I can probably convert it for you!
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
{feedback_url}
Thanks again and have a great day!
-Andrew
```
**Note:** The `{feedback_url}` placeholder will be populated with the feedback form URL. If no feedback URL is configured, this section can be omitted from the rendered output.
## Updating Templates
To update email templates after deployment:
```bash
# Upload new template
gsutil cp my-template.txt gs://karaoke-gen-storage/templates/job-completion.txt
# Templates are fetched fresh on each use (no caching in code)
# Changes take effect immediately
```
@@ -1,106 +0,0 @@
# Plan: Migrate from MailSlurp to testmail.app
**Created:** 2026-01-02
**Branch:** feat/sess-20260102-2109-migrate-mailslurp-testmail
**Status:** Complete
## Overview
Replace MailSlurp email testing service with testmail.app for E2E testing of magic link authentication. testmail.app uses a simpler API with namespace-based email addresses that require no inbox creation/deletion - emails just arrive at `{namespace}.{tag}@inbox.testmail.app`.
## Requirements
- [x] Remove all code referencing MailSlurp
- [x] Remove `mailslurp-client` npm dependency
- [x] Implement testmail.app API integration
- [x] Update environment variable names (`TESTMAIL_API_KEY`, `TESTMAIL_NAMESPACE`)
- [x] Maintain same `EmailHelper` interface for minimal test changes
- [x] Update archive docs to note migration (optional - historical accuracy)
## Technical Approach
### API Differences
| Feature | MailSlurp | testmail.app |
|---------|-----------|--------------|
| Email format | Random `xxx@mailslurp.com` | `{namespace}.{tag}@inbox.testmail.app` |
| Inbox creation | Required (`createInbox()`) | Not needed - use any tag |
| Inbox deletion | Required (`deleteInbox()`) | Not needed |
| Get emails | `waitForLatestEmail(inboxId)` | GET `/api/json?apikey=X&namespace=Y&tag=Z&livequery=true` |
| Auth | API key in constructor | API key as query param |
| Email body | `email.body` | `email.html` or `email.text` |
### Implementation Strategy
1. **Keep same `EmailHelper` interface** - `createInbox()`, `waitForEmail()`, `extractMagicLink()`, `deleteInbox()`
2. **Use testmail.app's live query feature** - `livequery=true` waits for email to arrive (up to timeout)
3. **Generate unique tags** - Use timestamp + random suffix for each test run to avoid collisions
4. **Inbox becomes a simple object** - `{ id: tag, emailAddress: '{namespace}.{tag}@inbox.testmail.app' }`
5. **Delete becomes no-op** - testmail.app emails auto-expire, no cleanup needed
### Environment Variables
| Old | New |
|-----|-----|
| `MAILSLURP_API_KEY` | `TESTMAIL_API_KEY` |
| (none) | `TESTMAIL_NAMESPACE` |
Values: Set via environment variables (see `frontend/.env.local.example`)
## Implementation Steps
1. [x] **Update `email-testing.ts`** - Replace MailSlurp implementation with testmail.app
- Remove `mailslurp-client` import
- Add testmail.app API fetch implementation
- Use native `fetch` (no new dependencies)
- Generate unique tag for each "inbox"
- Implement `livequery=true` for waiting
- Adapt email body extraction (`html`/`text` instead of `body`)
2. [x] **Update `full-user-journey.spec.ts`** - Change references
- Update doc comments (env var name)
- Update error messages
- Update console logs
3. [x] **Update `playwright.record.config.ts`** - Change env var comment
4. [x] **Update `package.json`** - Remove `mailslurp-client` dependency
5. [x] **Run `npm install`** - Update package-lock.json
6. [x] **Create `.env.local.example`** - Document required env vars for local testing
7. [x] **Update archive docs** - Skipped intentionally; historical docs preserved for accuracy
## Files to Create/Modify
| File | Action | Description |
|------|--------|-------------|
| `frontend/e2e/helpers/email-testing.ts` | Modify | Replace MailSlurp with testmail.app |
| `frontend/e2e/production/full-user-journey.spec.ts` | Modify | Update env var refs and messages |
| `frontend/playwright.record.config.ts` | Modify | Update env var comment |
| `frontend/package.json` | Modify | Remove `mailslurp-client` |
| `frontend/package-lock.json` | Modify | Auto-updated by npm install |
| `frontend/.env.local.example` | Create | Document env vars for local testing |
## Testing Strategy
- **Manual testing**: Run production E2E test with testmail.app credentials
```bash
cd frontend
TESTMAIL_API_KEY=your-api-key \
TESTMAIL_NAMESPACE=your-namespace \
npm run test:e2e:prod:headed
```
- **Verify**: Email received, magic link extracted, auth flow completes
- **CI testing**: Env vars already configured in GitHub Actions org secrets
## Open Questions
None - requirements are clear.
## Rollback Plan
1. Revert the commit
2. Run `npm install` to restore mailslurp-client
3. Re-add MAILSLURP_API_KEY to secrets if removed
@@ -1,401 +0,0 @@
# Plan: Video Worker Stage Refactor
**Created:** 2026-01-03
**Branch:** feat/sess-20260103-2250-investigate-youtube-upload
**Status:** COMPLETE - Ready for PR
**Last Updated:** 2026-01-04
## Overview
Refactor `video_worker.py` and `KaraokeFinalise` into separate, composable stages following SOLID principles. This eliminates the GCE vs KaraokeFinalise code path divergence that causes features like YouTube upload to be missing from the GCE encoding path.
## Three Execution Modes
Understanding the three ways this system runs is critical to the refactor:
### 1. Local CLI (`karaoke-gen`)
- Runs entirely on user's machine
- KaraokeFinalise.process() orchestrates everything
- Writes directly to local Dropbox sync folder (e.g., `~/Dropbox/Karaoke/Tracks-Organized/`)
- **Produces full output structure** with stems/, lyrics/, visualizations/, etc.
- **This is the source of truth for output format**
### 2. Remote CLI (`karaoke-gen-remote`)
- User's CLI talks to cloud backend (api.nomadkaraoke.com)
- Cloud does ALL processing (audio, lyrics, video encoding)
- CLI just monitors job status and downloads final outputs
- With Dropbox configured: cloud uploads full structure to user's Dropbox
- Without Dropbox: user downloads individual files
### 3. Web Frontend (`gen.nomadkaraoke.com`)
- Same cloud backend as remote CLI
- User sees:
- YouTube video link
- Dropbox folder link
- Direct download: final karaoke video (720p)
- Direct download: 4K video
- Direct download: "with vocals" video
- Direct download: CDG zip (if enabled)
## Problem Statement
Currently, `video_worker.py` has two completely divergent code paths:
```python
if use_gce_encoding:
# GCE PATH: Only encoding, missing YouTube/Discord/CDG/TXT
result = await _encode_via_gce(...)
# result DOES NOT include youtube_url!
else:
# STANDARD PATH: KaraokeFinalise does everything
finalise = KaraokeFinalise(...)
result = finalise.process()
# result includes youtube_url from upload
```
### What GCE Encoding Broke
| Feature | KaraokeFinalise Path | GCE Path |
|---------|---------------------|----------|
| Video encoding | ✅ 4 formats | ✅ 4 formats |
| YouTube upload | ✅ | ❌ **BROKEN** |
| Discord notification | ✅ | ❌ **BROKEN** |
| CDG/TXT packaging | ✅ | ❌ **BROKEN** |
| Brand code generation | ✅ | ✅ (via Dropbox service) |
| Dropbox upload | ✅ (rclone) | ✅ (native SDK) |
| Google Drive upload | ❌ | ✅ (native SDK) |
### Root Cause
`KaraokeFinalise.process()` is a monolith that handles:
1. CDG/TXT packaging
2. Video encoding (the CPU-intensive part)
3. YouTube upload
4. Discord notifications
5. Folder organization
The GCE encoding was added as a bypass around the entire class, rather than replacing just the encoding step.
## Requirements
### Functional Requirements
- [ ] YouTube upload works regardless of encoding backend (GCE or local)
- [ ] Discord notifications work regardless of encoding backend
- [ ] CDG/TXT packaging works regardless of encoding backend
- [ ] Brand code generation works consistently
- [ ] Dropbox/Google Drive upload works (already works in both paths)
- [ ] All existing CLI functionality continues to work
- [ ] Output structure matches local CLI exactly
### Output Structure Parity
Cloud backend must produce identical output to local CLI:
```
NOMAD-XXXX - Artist - Title/
├── Artist - Title (Title).mov
├── Artist - Title (Title).jpg
├── Artist - Title (End).mov
├── Artist - Title (End).jpg
├── Artist - Title (Final Karaoke Lossless 4k).mp4
├── Artist - Title (Final Karaoke Lossless 4k).mkv
├── Artist - Title (Final Karaoke Lossy 4k).mp4
├── Artist - Title (Final Karaoke Lossy 720p).mp4
├── Artist - Title (Final Karaoke CDG).zip
├── Artist - Title (Final Karaoke TXT).zip
├── Artist - Title (Instrumental {model}).flac
├── Artist - Title (Instrumental +BV {model}).flac
├── Artist - Title (Karaoke).mp4
├── Artist - Title (Karaoke).cdg
├── Artist - Title (Karaoke).mp3
├── Artist - Title (Karaoke).lrc
├── Artist - Title (With Vocals).mp4
├── stems/
│ ├── Artist - Title (Instrumental {model}).flac
│ ├── Artist - Title (Vocals {model}).flac
│ ├── Artist - Title (Lead Vocals {model}).flac
│ ├── Artist - Title (Backing Vocals {model}).flac
│ └── ... (other stems)
├── lyrics/
│ ├── Artist - Title (Karaoke).lrc
│ ├── Artist - Title (Karaoke).ass
│ ├── Artist - Title (Karaoke).srt
│ └── ... (corrections JSONs)
├── previews/
├── visualizations/
└── styles_assets/
```
### Non-Functional Requirements
- [ ] Single Responsibility: Each stage has one job
- [ ] Open/Closed: Easy to add new encoding backends without modifying stages
- [ ] Testable: Each stage can be unit tested in isolation
- [ ] No duplicate code between paths
- [ ] Backwards compatible with existing KaraokeFinalise CLI usage
## Technical Approach
### Architecture: Stage-Based Pipeline
```
video_worker.py (VideoWorkerOrchestrator)
│
├── 1. EncodingStage (interface)
│ ├── LocalEncodingService (wraps KaraokeFinalise encoding methods)
│ └── GCEEncodingService (existing encoding_service.py)
│
├── 2. PackagingStage
│ ├── CDGPackager (extracts from KaraokeFinalise)
│ └── TXTPackager (extracts from KaraokeFinalise)
│
├── 3. DistributionStage
│ ├── YouTubeUploadService (extracts from KaraokeFinalise)
│ ├── DropboxService (existing)
│ ├── GoogleDriveService (existing)
│ └── DiscordService (extracts from KaraokeFinalise)
│
└── 4. OrganizationStage
└── BrandCodeService (extracts from KaraokeFinalise + DropboxService)
```
### Key Design Decisions
1. **Extract, Don't Rewrite**: Pull methods out of KaraokeFinalise into services, don't rewrite from scratch
2. **Keep KaraokeFinalise for CLI**: Local CLI still uses KaraokeFinalise.process() - we extract services that it can also use
3. **Interface-Based Encoding**: EncodingStage interface allows swapping GCE/local without changing orchestration
4. **Existing Service Pattern**: Follow singleton + factory pattern already used in codebase
5. **Local CLI stays sync**: Local karaoke-gen CLI remains synchronous as it is today
### Data Flow
```
JobConfig (from Firestore)
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ VideoWorkerOrchestrator │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ 1. Setup: Download files, prepare directories ││
│ │ - Download title.mov, end.mov, with_vocals.mkv ││
│ │ - Download instrumental audio ││
│ │ - Prepare temp directory ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ 2. Packaging: CDG/TXT (if enabled) - runs BEFORE encoding ││
│ │ Reason: CDG depends on LRC file which is available ││
│ │ Input: LRC file, instrumental audio ││
│ │ Output: CDG.zip, TXT.zip, .cdg, .mp3 ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ 3. Encoding: GCE or Local (via interface) ││
│ │ Input: title.mov, karaoke.mov, end.mov ││
│ │ Output: 4K lossless, 4K lossy, 720p, MKV ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ 4. Organization: Brand code, folder structure ││
│ │ - Generate brand code (NOMAD-XXXX) ││
│ │ - Prepare full output directory structure ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ 5. Distribution: Upload to destinations ││
│ │ - YouTube upload (if enabled) ││
│ │ - Dropbox upload (if configured) ││
│ │ - Google Drive upload (if configured) ││
│ └─────────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ 6. Notification: Discord, Email ││
│ │ Input: YouTube URL, brand code, Dropbox link ││
│ └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
│
▼
Result (youtube_url, brand_code, dropbox_link, file_paths, etc.)
```
## Implementation Steps
### Phase 1: Extract Services from KaraokeFinalise (Non-Breaking) ✅ COMPLETE
These services can be used by both the cloud orchestrator AND KaraokeFinalise.
1. [x] **Create `backend/services/youtube_upload_service.py`** ✅
- Extract YouTube upload logic from KaraokeFinalise
- Methods: `upload_video()`, `check_duplicate()`, `delete_video()`, `truncate_title()`
- Supports pre-loaded credentials (server-side) or client secrets file (CLI)
- 23 unit tests with mocked YouTube API
2. [x] **Create `backend/services/discord_service.py`** ✅
- Extract Discord notification from KaraokeFinalise
- Methods: `post_message()`, `post_video_notification()`, `is_enabled()`
- Simple HTTP POST to webhook with validation
- 20 unit tests with mocked requests
3. [x] **Create `backend/services/packaging_service.py`** ✅
- Extract CDG/TXT generation from KaraokeFinalise
- Methods: `create_cdg_package()`, `create_txt_package()`
- Uses lyrics_converter and CDGGenerator (existing dependencies)
- 18 unit tests with temp files
4. [x] **Create `backend/services/local_encoding_service.py`** ✅
- Extract encoding methods from KaraokeFinalise
- Methods: `encode_all_formats()`, individual encoding methods
- Hardware acceleration detection with software fallback
- 28 unit tests with mocked subprocess
### Phase 2: Create Encoding Interface ✅ COMPLETE
5. [x] **Create `backend/services/encoding_interface.py`** ✅
- Define ABC `EncodingBackend` with methods: `encode()`, `is_available()`, `get_status()`
- `EncodingInput` and `EncodingOutput` dataclasses for config/results
- Factory function `get_encoding_backend()` with "local", "gce", "auto" options
6. [x] **`LocalEncodingBackend`** ✅
- Implements `EncodingBackend` interface
- Wraps `LocalEncodingService` with `asyncio.to_thread()`
- 17 unit tests
7. [x] **`GCEEncodingBackend`** ✅
- Implements `EncodingBackend` interface
- Wraps existing `EncodingService` (encoding_service.py)
- Requires GCS paths in options for cloud execution
- 6 additional unit tests
### Phase 3: Create Orchestrator ✅ COMPLETE
8. [x] **Create `backend/workers/video_worker_orchestrator.py`** ✅
- New class that coordinates all stages
- Replaces the branching logic in `generate_video()`
- Methods:
- `__init__(config, job_manager, storage, job_logger)`
- `async run() -> OrchestratorResult`
- `_run_packaging()` - CDG/TXT
- `_run_encoding()` - GCE or local via EncodingBackend interface
- `_run_organization()` - Brand code, folder structure
- `_run_distribution()` - YouTube, Dropbox, GDrive
- `_run_notifications()` - Discord
- 27 unit tests (test_video_worker_orchestrator.py)
9. [x] **Update `backend/workers/video_worker.py`** ✅
- Added `USE_NEW_ORCHESTRATOR` feature flag (default: true)
- `generate_video()` routes to orchestrator or legacy path
- `generate_video_orchestrated()` - new unified pipeline
- `generate_video_legacy()` - original code for rollback
- Legacy GCE and KaraokeFinalise paths preserved for safety
### Phase 4: Update KaraokeFinalise to Use Services (OPTIONAL)
10. [ ] **Update `karaoke_gen/karaoke_finalise/karaoke_finalise.py`**
- Import and use the new services
- Replace inline YouTube/Discord/CDG code with service calls
- Keep process() working for CLI backwards compatibility
- Services become the single source of truth
- **NOTE**: This is optional for MVP - the orchestrator already uses the extracted services directly
### Phase 5: Testing & Validation ✅ COMPLETE
11. [x] **Add unit tests for each service** ✅
- test_youtube_upload_service.py (23 tests)
- test_discord_service.py (20 tests)
- test_packaging_service.py (18 tests)
- test_local_encoding_service.py (28 tests)
- test_encoding_interface.py (23 tests)
- test_video_worker_orchestrator.py (27 tests)
- **Total: 139 new tests**
12. [x] **Full test suite passes** ✅
- 976 tests passed (86 skipped - emulator/integration tests)
- No regressions from changes
13. [ ] **Manual E2E testing** (After deployment)
- [ ] GCE encoding job with YouTube upload enabled
- [ ] Local encoding job with YouTube upload enabled
- [ ] CLI karaoke-gen with all features
- [ ] Job with CDG/TXT enabled
- [ ] Verify output structure matches local CLI
## Files Created/Modified
| File | Action | Description | Status |
|------|--------|-------------|--------|
| `backend/services/youtube_upload_service.py` | Create | YouTube upload logic (~446 lines) | ✅ |
| `backend/services/discord_service.py` | Create | Discord notification service (~167 lines) | ✅ |
| `backend/services/packaging_service.py` | Create | CDG/TXT package generation (~266 lines) | ✅ |
| `backend/services/local_encoding_service.py` | Create | Local FFmpeg encoding service (~512 lines) | ✅ |
| `backend/services/encoding_interface.py` | Create | Abstract interface + backends (~418 lines) | ✅ |
| `backend/workers/video_worker_orchestrator.py` | Create | Stage orchestration logic (~550 lines) | ✅ |
| `backend/workers/video_worker.py` | Modify | Added orchestrator path + feature flag | ✅ |
| `backend/tests/test_youtube_upload_service.py` | Create | 23 unit tests | ✅ |
| `backend/tests/test_discord_service.py` | Create | 20 unit tests | ✅ |
| `backend/tests/test_packaging_service.py` | Create | 18 unit tests | ✅ |
| `backend/tests/test_local_encoding_service.py` | Create | 28 unit tests | ✅ |
| `backend/tests/test_encoding_interface.py` | Create | 23 unit tests | ✅ |
| `backend/tests/test_video_worker_orchestrator.py` | Create | 27 unit tests | ✅ |
| `karaoke_gen/karaoke_finalise/karaoke_finalise.py` | Modify | Use extracted services (optional) | Deferred |
## Testing Strategy
### Unit Tests
- **youtube_upload_service**: Mock YouTube API, test upload/delete/duplicate check
- **discord_service**: Mock requests, test webhook POST
- **packaging_service**: Use temp files, test CDG/TXT generation
- **local_encoding_service**: Mock subprocess, test FFmpeg command construction
### Integration Tests
- **video_worker_orchestrator**: Mock all services, test stage sequencing
- **video_worker**: Test with mocked orchestrator
### E2E Verification
- [ ] GCE encoding job with YouTube upload enabled → YouTube link appears
- [ ] Local encoding job with YouTube upload enabled → Same behavior
- [ ] CLI `karaoke-gen` with all features → No regression
- [ ] Job with CDG/TXT enabled → Packages generated
- [ ] Dropbox output structure → Matches local CLI exactly
## Open Questions
- [x] Should we keep KaraokeFinalise.process() or deprecate it?
- **Decision**: Keep it for CLI backwards compatibility. It will use the new services internally.
- [x] Should local_encoding_service run FFmpeg sync or async?
- **Decision**: Sync implementation, async wrapper via `asyncio.to_thread()` for interface compatibility.
- [x] Order of packaging vs encoding?
- **Decision**: CDG/TXT runs BEFORE encoding (depends on LRC file, not videos). KaraokeFinalise does it this way too.
## Rollback Plan
1. The refactor is additive - new services don't replace old code until orchestrator is wired up
2. Feature flag: `USE_NEW_ORCHESTRATOR=true` to enable new code path
3. If issues found in production:
- Set `USE_NEW_ORCHESTRATOR=false` to revert to old behavior
- Old `generate_video()` code preserved behind feature flag
4. After 1 week stable, remove feature flag and old code
## Success Criteria
1. Job `5b78ad46` scenario (GCE + YouTube) would upload to YouTube
2. All 803+ existing tests pass
3. E2E happy path test passes
4. No regression in local CLI functionality
5. Output structure matches local CLI exactly
6. Code coverage maintained or improved
## Estimated Scope
- **Phase 1** (Extract Services): 4 new service files + tests (~400-600 lines each)
- **Phase 2** (Interface): 1 new file, 2 modifications (~100 lines)
- **Phase 3** (Orchestrator): 2 files (~300-400 lines)
- **Phase 4** (KaraokeFinalise): 1 modification (~100 lines of changes)
- **Phase 5** (Testing): Integration tests + manual verification
This is a medium-sized refactor that can be done incrementally. Each phase produces working, tested code.
-317
View File
@@ -1,317 +0,0 @@
# Plan: White Label B2B Karaoke Portals
**Created:** 2026-01-08
**Branch:** feat/sess-20260108-0358-white-label-portal
**Status:** Draft
## Overview
Add multi-tenant white label portal capability to karaoke-gen, enabling B2B customers like Vocal Star to access a customized version of the generator app on their own subdomain with their branding, theme, and restricted feature set.
**First customer: Vocal Star**
- Subdomain: `vocalstar.nomadkaraoke.com`
- Workflow: Upload original + instrumental audio (no audio search)
- Theme: Custom Vocal Star karaoke theme (already exists as JSON)
- Distribution: Download only (no Dropbox/GDrive/YouTube)
- Auth: Magic link for `@vocal-star.com` / `@vocalstarmusic.com` domains, plus fixed API token option
## Requirements
### Functional Requirements
- [ ] Tenant identification via subdomain (e.g., `vocalstar.nomadkaraoke.com`)
- [ ] Per-tenant frontend branding (logo, colors, metadata)
- [ ] Per-tenant feature flags (hide audio search, hide distribution options)
- [ ] Per-tenant default theme (always use Vocal Star theme)
- [ ] Per-tenant allowed email domains for magic link auth
- [ ] Per-tenant fixed API token support (for automation)
- [ ] Jobs scoped to tenant (Vocal Star users only see Vocal Star jobs)
- [ ] Users scoped to tenant
### Non-Functional Requirements
- [ ] Minimal code duplication - single codebase serves all tenants
- [ ] Easy to add new tenants via config, not code changes
- [ ] Graceful fallback - default to Nomad Karaoke if no tenant match
- [ ] No performance impact on existing single-tenant flow
## Technical Approach
### Architecture Decision: Config-Driven Multi-Tenancy
Rather than building a full tenant management system (overkill for ~1 customer/year), we'll use a **config-driven approach**:
1. **Tenant configs stored in GCS** at `tenants/{tenant_id}/config.json`
2. **Frontend detects subdomain** and fetches tenant config from API
3. **Backend middleware** extracts tenant from request and applies config
4. **Firestore collections** get `tenant_id` field for isolation
This approach:
- Requires no database migrations for new tenants
- Configs can be managed via simple JSON files
- Easy to test locally with query param override (`?tenant=vocalstar`)
- Can evolve to admin UI later if needed
### Tenant Config Schema
```json
{
"id": "vocalstar",
"name": "Vocal Star",
"subdomain": "vocalstar.nomadkaraoke.com",
"branding": {
"logo_url": "gs://karaoke-gen-storage/tenants/vocalstar/logo.png",
"primary_color": "#ffff00",
"secondary_color": "#006CF9",
"favicon_url": "gs://karaoke-gen-storage/tenants/vocalstar/favicon.ico",
"site_title": "Vocal Star Karaoke Generator"
},
"features": {
"audio_search": false,
"file_upload": true,
"youtube_upload": false,
"dropbox_upload": false,
"gdrive_upload": false,
"theme_selection": false
},
"defaults": {
"theme_id": "vocalstar",
"distribution_mode": "download_only"
},
"auth": {
"allowed_email_domains": ["vocal-star.com", "vocalstarmusic.com"],
"fixed_tokens": ["<will be in Secret Manager>"]
}
}
```
### Key Implementation Details
#### 1. Subdomain Detection (Frontend)
- Next.js middleware checks `request.headers.host`
- If subdomain matches tenant, store tenant ID in cookie/context
- Fetch `/api/tenant/config` to get branding/features
- Apply CSS variables dynamically for colors
- Swap logo component based on config
#### 2. Tenant Middleware (Backend)
- FastAPI middleware extracts tenant from:
- `X-Tenant-ID` header (set by frontend)
- Or subdomain from `Host` header
- Or query param `?tenant=` (dev only)
- Attaches `tenant_id` to request state
- Tenant config loaded from GCS (cached 5 min like themes)
#### 3. Auth Enhancement
- `AuthService.validate_token_full()` already returns `user_email`
- Add tenant validation: check if user's email domain is in tenant's allowed list
- Support fixed tokens per tenant (stored in `auth_tokens` with `tenant_id`)
- Magic links include tenant context for proper redirect
#### 4. Data Isolation
- Add `tenant_id: Optional[str]` to User and Job models
- Default to `None` for existing Nomad Karaoke users (backwards compatible)
- Query filters: `where('tenant_id', '==', tenant_id or None)`
- No schema migration needed - new field, optional
#### 5. Theme Integration
- Upload Vocal Star theme to `themes/vocalstar/` in GCS
- Upload assets (backgrounds, fonts) to `themes/vocalstar/assets/`
- Tenant config specifies `default_theme_id: "vocalstar"`
- When `theme_selection: false`, UI hides theme picker and uses default
## Implementation Steps
### Phase 1: Backend Tenant Foundation (Day 1-2)
1. [ ] **Create tenant model and config loader**
- Create `backend/models/tenant.py` with `TenantConfig`, `TenantBranding`, `TenantFeatures`
- Create `backend/services/tenant_service.py` with GCS config loading and caching
- Add `GET /api/tenant/config` endpoint (unauthenticated, returns public config)
- Add `GET /api/tenant/config/{tenant_id}` for explicit tenant fetch
2. [ ] **Add tenant middleware**
- Create `backend/api/middleware/tenant.py`
- Extract tenant from subdomain or header
- Attach to `request.state.tenant_id` and `request.state.tenant_config`
- Add `get_tenant()` dependency for routes
3. [ ] **Update data models**
- Add `tenant_id: Optional[str] = None` to `User` model
- Add `tenant_id: Optional[str] = None` to `Job` model
- Update Firestore indexes in Pulumi for tenant-scoped queries
4. [ ] **Update auth for tenant awareness**
- Modify `AuthService` to validate email domain against tenant config
- Add tenant context to magic link generation
- Support per-tenant fixed tokens in `auth_tokens` collection
### Phase 2: Upload Vocal Star Theme (Day 2)
5. [ ] **Prepare Vocal Star theme for GCS**
- Convert local paths in `karaoke-prep-styles-vocalstar.json` to GCS paths
- Upload background images to `themes/vocalstar/assets/`
- Upload Oswald font to `themes/vocalstar/assets/`
- Upload `style_params.json` to `themes/vocalstar/`
- Create preview image and upload to `themes/vocalstar/preview.png`
- Add Vocal Star to `themes/_metadata.json`
6. [ ] **Create Vocal Star tenant config**
- Create `tenants/vocalstar/config.json` in GCS
- Upload Vocal Star logo to `tenants/vocalstar/logo.png`
- Upload favicon to `tenants/vocalstar/favicon.ico`
### Phase 3: Frontend Tenant Support (Day 3-4)
7. [ ] **Add tenant context provider**
- Create `frontend/lib/tenant.ts` with tenant config types
- Create `frontend/hooks/useTenant.ts` for tenant state
- Create `TenantProvider` component that fetches config on mount
- Detect subdomain in `_app.tsx` or middleware
8. [ ] **Dynamic branding**
- Create `TenantLogo` component that renders tenant or default logo
- Update `globals.css` to support CSS variable overrides
- Add `applyTenantColors()` function to set CSS variables from config
- Update page metadata (title, favicon) from tenant config
9. [ ] **Feature-flagged UI**
- Wrap audio search in `{tenant.features.audio_search && ...}`
- Wrap theme selector in `{tenant.features.theme_selection && ...}`
- Wrap distribution options based on tenant features
- Show simplified form for Vocal Star (just file upload fields)
10. [ ] **Update magic link flow**
- Pass tenant context when requesting magic link
- Magic link redirect goes to tenant subdomain
- Verify endpoint extracts tenant from link context
### Phase 4: Backend Feature Enforcement (Day 4-5)
11. [ ] **Enforce tenant features in API**
- Audio search endpoint: return 403 if `audio_search: false`
- File upload: check `file_upload: true`
- Distribution settings: ignore/error if disabled features requested
- Theme selection: use default if `theme_selection: false`
12. [ ] **Job creation with tenant context**
- `JobCreate` handler sets `tenant_id` from request state
- Distribution config respects tenant defaults
- Theme defaults to tenant's default theme
13. [ ] **Query scoping**
- Update `JobManager.get_jobs()` to filter by tenant_id
- Update `UserService` queries to include tenant_id
- Admin endpoints bypass tenant filter (see all tenants)
### Phase 5: Infrastructure & Deployment (Day 5-6)
14. [ ] **DNS setup**
- Add `vocalstar.nomadkaraoke.com` CNAME to Cloudflare
- Configure Cloudflare Pages to accept wildcard subdomain
- Add SSL certificate for subdomain
15. [ ] **Backend CORS update**
- Add `vocalstar.nomadkaraoke.com` to allowed origins
- Or use pattern matching for `*.nomadkaraoke.com`
16. [ ] **Pulumi updates**
- Add Firestore indexes for tenant_id queries
- Add any new secrets (tenant fixed tokens)
### Phase 6: Testing & Documentation (Day 6-7)
17. [ ] **Testing**
- Unit tests for TenantService config loading
- Unit tests for tenant middleware
- E2E test for Vocal Star flow (upload audio, generate, download)
- Test magic link works for @vocal-star.com email
18. [ ] **Documentation**
- Update CLAUDE.md with tenant info
- Create `docs/WHITE-LABEL.md` with tenant setup guide
- Document how to add new tenants
## Files to Create/Modify
| File | Action | Description |
|------|--------|-------------|
| `backend/models/tenant.py` | Create | Tenant config models |
| `backend/services/tenant_service.py` | Create | Tenant config loading/caching |
| `backend/api/routes/tenant.py` | Create | Tenant config API endpoint |
| `backend/api/middleware/tenant.py` | Create | Tenant extraction middleware |
| `backend/api/dependencies.py` | Modify | Add `get_tenant()` dependency |
| `backend/models/user.py` | Modify | Add `tenant_id` field |
| `backend/models/job.py` | Modify | Add `tenant_id` field |
| `backend/services/auth_service.py` | Modify | Tenant email domain validation |
| `backend/services/user_service.py` | Modify | Tenant-aware user queries |
| `backend/services/job_manager.py` | Modify | Tenant-aware job queries |
| `backend/api/routes/file_upload.py` | Modify | Tenant feature enforcement |
| `backend/api/routes/audio_search.py` | Modify | Tenant feature enforcement |
| `backend/main.py` | Modify | Add tenant middleware |
| `frontend/lib/tenant.ts` | Create | Tenant types and API |
| `frontend/hooks/useTenant.ts` | Create | Tenant context hook |
| `frontend/components/TenantProvider.tsx` | Create | Tenant context provider |
| `frontend/components/TenantLogo.tsx` | Create | Dynamic logo component |
| `frontend/app/layout.tsx` | Modify | Add TenantProvider |
| `frontend/app/globals.css` | Modify | CSS variable support for tenant colors |
| `frontend/app/app/page.tsx` | Modify | Feature-flag UI sections |
| `infrastructure/__main__.py` | Modify | Add tenant Firestore indexes |
| `docs/WHITE-LABEL.md` | Create | Tenant setup documentation |
## Testing Strategy
### Unit Tests
- `TenantService`: Config loading, caching, validation
- `TenantMiddleware`: Subdomain extraction, header extraction
- `AuthService`: Tenant email domain validation
- `JobManager`: Tenant-scoped queries
### Integration Tests (Emulator)
- Create job with tenant context
- Query jobs filtered by tenant
- Auth with tenant-allowed email domain
- Auth rejection for wrong email domain
### E2E Tests
- Vocal Star happy path: Upload audio files, review lyrics, download video
- Auth flow: Magic link for @vocal-star.com, verify redirect to subdomain
- Feature hiding: Verify audio search not shown on Vocal Star portal
## Open Questions
- [x] **Q: Should tenant configs be in GCS or Firestore?**
A: GCS - simpler, matches theme pattern, easy to edit JSON directly
- [x] **Q: How to handle localhost development for specific tenant?**
A: Query param override `?tenant=vocalstar` for local testing
- [x] **Q: Should admin portal be tenant-aware?**
A: No - admins see all tenants. Admin routes bypass tenant filter.
- [x] **Q: Do we need tenant-specific email sender identity?**
A: Yes - use `{tenant}@nomadkaraoke.com` (e.g., `vocalstar@nomadkaraoke.com`). Catchall is configured.
- [x] **Q: Vocal Star logo and favicon - do you have these assets?**
A: Logo provided as `vocalstar-logo.jpg` - yellow star, white text on black. Will convert to PNG with transparency for web use.
## Rollback Plan
Since this is additive (new optional `tenant_id` field), rollback is straightforward:
1. Revert frontend to not check tenant
2. Revert backend middleware to not extract tenant
3. Existing data unaffected (tenant_id = null = Nomad Karaoke)
4. No schema migration to reverse
For DNS: Simply remove CNAME record if needed.
## Success Criteria
- [ ] Vocal Star user can access `vocalstar.nomadkaraoke.com`
- [ ] Portal shows Vocal Star branding (logo, colors)
- [ ] Audio search is hidden; only file upload available
- [ ] Theme is locked to Vocal Star style
- [ ] Distribution options hidden; download only
- [ ] Magic link works for @vocal-star.com emails
- [ ] Fixed API token works for automation
- [ ] Generated videos use Vocal Star theme
- [ ] Nomad Karaoke portal unchanged at gen.nomadkaraoke.com
@@ -1,222 +0,0 @@
# Plan: Worker Logs Rearchitecture
**Created:** 2026-01-04
**Branch:** `feat/sess-20260104-1931-logs-research`
**Status:** Implementation Complete - Pending PR Merge & Deployment
## Overview
Move `worker_logs` from embedded array in job documents to a Firestore subcollection to avoid the 1MB document size limit. The current approach caused job 501258e1 to fail when logs reached 1.26 MB (98.6% of document size).
## Problem Statement
- Firestore documents have a 1 MB size limit
- `worker_logs` is stored as an array field within job documents
- Long-running jobs (especially video encoding) can generate thousands of log entries
- When the limit is exceeded, job updates fail with: `Document cannot be written because its size exceeds the maximum allowed size`
## Requirements
- [x] Logs must remain associated with their job
- [x] Logs should be queryable by job_id, worker type, timestamp
- [x] Old logs should auto-cleanup (TTL: 30 days)
- [x] Existing API endpoint (`GET /api/jobs/{id}/logs`) must continue to work
- [x] Frontend must continue to display logs without changes
- [x] Migration path for existing jobs (optional - can just start fresh)
## Technical Approach
### Option A: Subcollection (Recommended)
Store logs in `jobs/{job_id}/logs/{log_id}` subcollection.
**Pros:**
- Natural association with job (delete job = delete logs with cascading)
- Each log entry is its own document (no size limit concerns)
- Can add TTL via Firestore TTL policies
- Efficient queries with composite indexes
**Cons:**
- More Firestore read operations for paginated log fetching
- Slightly more complex write logic
### Option B: Separate Collection
Store logs in `job_logs` collection with `job_id` field.
**Pros:**
- Can query across all jobs
- Simpler TTL management
**Cons:**
- Orphaned logs if job deleted (need cleanup logic)
- Extra index needed for job_id queries
### Decision: Option A (Subcollection)
## Implementation Steps
### Phase 1: Backend Changes
1. [x] **Create log entry model** - `backend/models/worker_log.py`
- `WorkerLogEntry` dataclass with: id, job_id, timestamp, level, worker, message, metadata, ttl_expiry
- Factory method `create()` for easy instantiation
- `to_dict()` for Firestore storage
- `to_legacy_dict()` for API compatibility
- `from_dict()` for Firestore retrieval
2. [x] **Update FirestoreService** - `backend/services/firestore_service.py`
- Added `append_log_to_subcollection(job_id, log_entry)` method
- Added `get_logs_from_subcollection(job_id, limit, since_timestamp, worker, offset)` method
- Added `get_logs_count_from_subcollection(job_id)` method
- Added `delete_logs_subcollection(job_id, batch_size)` method
- Old `append_worker_log` marked as deprecated
3. [x] **Update JobManager** - `backend/services/job_manager.py`
- Modified `append_worker_log()` to use subcollection when `USE_LOG_SUBCOLLECTION=true`
- Modified `get_worker_logs()` to read from subcollection with fallback to embedded array
- Added `get_worker_logs_count()` for total count
- Updated `delete_job()` to clean up logs subcollection
4. [x] **Update jobs API** - `backend/api/routes/jobs.py`
- `GET /api/jobs/{id}/logs` - Updated to use new methods
- Response format stays the same (backward compatible)
- Fixed `next_index` calculation
5. [x] **Add feature flag** - `backend/config.py`
- Added `USE_LOG_SUBCOLLECTION` environment variable (default: true)
6. [x] **Add TTL configuration** - `infrastructure/__main__.py`
- Added Firestore TTL policy for logs subcollection (30 days)
- Added composite index for worker + timestamp queries
### Phase 2: Testing
7. [x] **Unit tests** - `backend/tests/test_worker_log_subcollection.py`
- 24 tests covering:
- WorkerLogEntry model creation and conversion
- FirestoreService subcollection methods
- JobManager logging with feature flag
- Edge cases (unicode, long messages, empty messages)
8. [x] **Integration tests** - `backend/tests/emulator/test_worker_logs_subcollection.py`
- 11 tests using real Firestore emulator:
- Single and sequential writes
- TTL field verification
- Concurrent writes (no race conditions)
- Large volume (500 logs)
- Worker filtering
- Timestamp ordering
- Subcollection deletion
- Concurrent workers interleaved
- Job deletion cleanup
- Comparison with embedded array
### Phase 3: Deployment & Migration
9. [ ] **Deploy with feature flag**
- Add `USE_LOG_SUBCOLLECTION=true` env var (default is true)
- Deploy to production
10. [ ] **Monitor**
- Watch for errors in Cloud Logging
- Verify logs appear in frontend
11. [x] **Backward compatibility**
- Old jobs with embedded arrays still work (fallback in get_worker_logs)
- No migration needed - new jobs use subcollection, old jobs use array
### Phase 4: Cleanup (Future)
12. [ ] **Remove old code path** (after confidence period)
- Remove `worker_logs` field from job model
- Remove old `append_worker_log` method
- Remove feature flag
## Files Created/Modified
| File | Action | Description |
|------|--------|-------------|
| `backend/models/worker_log.py` | Created | WorkerLogEntry dataclass |
| `backend/services/firestore_service.py` | Modified | Added subcollection methods |
| `backend/services/job_manager.py` | Modified | Switch to subcollection with fallback |
| `backend/api/routes/jobs.py` | Modified | Updated logs endpoint |
| `backend/config.py` | Modified | Added USE_LOG_SUBCOLLECTION setting |
| `backend/tests/test_worker_log_subcollection.py` | Created | Unit tests (24 tests) |
| `backend/tests/emulator/test_worker_logs_subcollection.py` | Created | Integration tests (11 tests) |
| `infrastructure/__main__.py` | Modified | Added TTL policy and index |
## Log Entry Schema
```python
@dataclass
class WorkerLogEntry:
timestamp: datetime
level: str # "DEBUG", "INFO", "WARNING", "ERROR"
worker: str # "audio", "lyrics", "video", "render", "screens", "distribution"
message: str
id: str # Auto-generated UUID
job_id: str
ttl_expiry: datetime # timestamp + 30 days
metadata: Optional[Dict[str, Any]] = None
```
## Firestore Structure
```
jobs/
{job_id}/
... (job fields, worker_logs still present for old jobs)
logs/ # subcollection for new jobs
{log_id}/
id: string
job_id: string
timestamp: datetime
level: string
worker: string
message: string
metadata: map (optional)
ttl_expiry: datetime
```
## API Response (Unchanged)
```json
{
"logs": [
{
"timestamp": "2026-01-04T12:00:00Z",
"level": "INFO",
"worker": "video",
"message": "Starting video generation"
}
],
"next_index": 42,
"total_logs": 150
}
```
## Testing Results
All tests pass:
- 24 unit tests in `test_worker_log_subcollection.py`
- 11 emulator integration tests in `test_worker_logs_subcollection.py`
- 68 total tests in emulator suite pass
## Rollback Plan
1. Set `USE_LOG_SUBCOLLECTION=false` env var
2. Redeploy backend
3. New logs will go back to embedded array
4. Subcollection logs will remain but won't be read
## Answers to Open Questions
- [x] **Should we paginate logs in the API?** - Yes, added `offset` parameter support
- [x] **What's the optimal TTL period?** - 30 days (configurable via DEFAULT_LOG_TTL_DAYS)
- [x] **Should we index by timestamp for efficient range queries?** - Yes, added composite index (worker + timestamp)
## References
- [Firestore TTL](https://cloud.google.com/firestore/docs/ttl)
- [Firestore Subcollections](https://firebase.google.com/docs/firestore/data-model#subcollections)
- [Firestore Limits](https://firebase.google.com/docs/firestore/quotas#limits)
- [Pulumi Firestore Field](https://www.pulumi.com/registry/packages/gcp/api-docs/firestore/field/)
+21
View File
@@ -0,0 +1,21 @@
---
description: Git branching and PR workflow
alwaysApply: true
---
# Branching Workflow
**Never commit directly to `main`.** Always work on a feature branch.
Create descriptive branch names like `feature/add-discord-notifications`, `fix/audio-sync-issue`, or `refactor/cleanup-api-routes`.
# Creating a Pull Request
Once all changes are complete, tests pass, and commits are pushed, create a PR with:
1. **Summary** - Brief overview of the changes
2. **Changes Made** - List of specific modifications
3. **Testing** - How changes were tested (unit tests, manual testing)
4. **Related Issues** - Reference any GitHub issues (e.g., "Fixes #123")
Only merge after review and CI checks pass.
+14
View File
@@ -0,0 +1,14 @@
# Infrastructure Changes
All Google Cloud infrastructure (IAM, Cloud Storage, Cloud Run, Secret Manager, Artifact Registry, etc.) must be managed via Pulumi in the `infrastructure/` folder.
## Rules
- **Reading/querying** with `gcloud` CLI is fine
- **Creating or modifying** resources must go through IaC — never use `gcloud` or the Console to make changes
## Workflow
1. Make changes in `infrastructure/`
2. Run `pulumi preview` and verify the output
3. Run `pulumi up` to apply
+27
View File
@@ -0,0 +1,27 @@
---
description: Version bumping rule for commits
alwaysApply: true
---
# Version Management
When committing and pushing changes, always bump the patch version in `pyproject.toml`.
The version is located at `tool.poetry.version` in `pyproject.toml`. For example, if the current version is `0.70.3`, bump it to `0.70.4`.
This ensures every commit pushed to the repository has a unique version number for tracking and deployment purposes.
# Testing Requirements
Before committing and pushing changes, ensure all tests pass:
```bash
python -m pytest tests/ backend/tests/ -v -x --ignore=tests/integration
```
Ensure all code touched is thoroughly covered by unit tests, we have a 70% coverage minimum in CI:
```bash
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=xml --cov-report=term-missing --cov-fail-under=70
```
Do not commit or push code that fails tests. Fix any test failures before proceeding.
-83
View File
@@ -1,83 +0,0 @@
# .dockerignore - Reduce Docker build context size
# This is critical for fast builds as it reduces upload time and cache invalidation
# Git
.git/
.gitignore
.gitmodules
# Node.js / Frontend (not needed for backend image)
node_modules/
frontend/
frontend-react/
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# Python virtual environments (use Docker's Python instead)
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
.venv/
venv/
backend/venv/
ENV/
env/
*.egg-info/
.eggs/
dist/
build/
*.egg
# IDE
.idea/
.vscode/
*.swp
*.swo
*~
.DS_Store
# Testing (not needed in production image)
.pytest_cache/
.coverage
htmlcov/
.tox/
.nox/
coverage.xml
*.cover
tests/
backend/tests/
# Documentation (not needed in Docker image)
docs/
*.md
!README.md
# Infrastructure (not needed for Docker build)
infrastructure/
# CI/CD configs
.github/
# Cloud Build configs (keep for reference but not in image)
cloudbuild*.yaml
# Local development files
lyrics_transcriber_local/
*.local
.env
.env.*
*.bak
*.tmp
Thumbs.db
# Large test/sample files
tests/data/
*.mp3
*.wav
*.mp4
*.mkv
*.flac
@@ -1,115 +0,0 @@
# Build Encoding Worker GCE Image
#
# Rebuilds the custom Packer image when provisioning changes.
# The image includes Python 3.13, FFmpeg, fonts, and dependencies
# pre-installed to reduce VM startup time from ~10 min to ~30 sec.
#
# Triggers:
# - Push to main with changes to infrastructure/packer/**
# - Manual dispatch (workflow_dispatch)
#
# After image build, the encoding-worker VM will use the new image
# on next recreation (e.g., via pulumi up or manual restart).
name: Build Encoding Worker Image
on:
push:
branches: [main]
paths:
- 'infrastructure/packer/**'
workflow_dispatch:
inputs:
python_version:
description: 'Python version to install'
required: false
default: '3.13.1'
concurrency:
group: build-encoding-worker-image
cancel-in-progress: false # Don't cancel in-progress builds
jobs:
build-image:
name: Build Packer Image
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # Required for Workload Identity Federation
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Authenticate to Google Cloud (Workload Identity)
uses: google-github-actions/auth@v2
with:
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: "latest"
- name: Install Packer
run: |
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt-get update && sudo apt-get install -y packer
- name: Initialize Packer
working-directory: infrastructure/packer
run: packer init encoding-worker.pkr.hcl
- name: Validate Packer Template
working-directory: infrastructure/packer
run: packer validate encoding-worker.pkr.hcl
- name: Build Image
working-directory: infrastructure/packer
env:
PYTHON_VERSION: ${{ github.event.inputs.python_version || '3.13.1' }}
run: |
echo "Building encoding worker image with Python ${PYTHON_VERSION}..."
packer build \
-var "python_version=${PYTHON_VERSION}" \
encoding-worker.pkr.hcl
- name: Get Latest Image Info
id: image-info
run: |
IMAGE_NAME=$(gcloud compute images list \
--project=nomadkaraoke \
--filter="family:encoding-worker" \
--sort-by="~creationTimestamp" \
--limit=1 \
--format="value(name)")
echo "image_name=${IMAGE_NAME}" >> $GITHUB_OUTPUT
echo "Built image: ${IMAGE_NAME}"
- name: Summary
run: |
echo "## Encoding Worker Image Built" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Image:** \`${{ steps.image-info.outputs.image_name }}\`" >> $GITHUB_STEP_SUMMARY
echo "**Family:** \`encoding-worker\`" >> $GITHUB_STEP_SUMMARY
echo "**Project:** \`nomadkaraoke\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "The encoding-worker VM will use this image on next recreation." >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "To force VM recreation now:" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`bash" >> $GITHUB_STEP_SUMMARY
echo "gcloud compute instances delete encoding-worker --zone=us-central1-a --project=nomadkaraoke" >> $GITHUB_STEP_SUMMARY
echo "# Then run pulumi up or wait for next infrastructure deploy" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
notify-on-failure:
name: Notify on Failure
runs-on: ubuntu-latest
needs: build-image
if: failure()
steps:
- name: Log Failure
run: |
echo "::error::Encoding worker image build failed. Check the build-image job logs."
+148 -951
View File
File diff suppressed because it is too large. Load diff
-173
View File
@@ -1,173 +0,0 @@
name: E2E Happy Path Test
# Comprehensive end-to-end test that validates the full karaoke generation journey
# using ONLY browser interactions (no API shortcuts)
#
# See docs/E2E-HAPPY-PATH-TEST-SPEC.md for full requirements
on:
# Daily run at 6 AM UTC
schedule:
- cron: '0 6 * * *'
# Manual trigger with optional parameters
workflow_dispatch:
inputs:
debug_mode:
description: 'Enable debug mode (slower, more screenshots)'
required: false
default: 'false'
type: boolean
browser:
description: 'Browser to test with'
required: false
default: 'chromium'
type: choice
options:
- chromium
- firefox
- webkit
use_test_token:
description: 'Skip signup and use E2E_TEST_TOKEN (for iterations)'
required: false
default: 'false'
type: boolean
permissions:
contents: read
jobs:
e2e-happy-path:
name: "Full Happy Path (Real User)"
runs-on: ubuntu-latest
timeout-minutes: 90 # 1.5 hours max (test itself is 60 min)
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Install dependencies
working-directory: frontend
run: npm ci --legacy-peer-deps
- name: Install Playwright browsers
working-directory: frontend
run: |
BROWSER=${{ inputs.browser || 'chromium' }}
npx playwright install --with-deps $BROWSER
- name: Create test-results directory
working-directory: frontend
run: mkdir -p test-results
- name: Run E2E Happy Path Test
working-directory: frontend
env:
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
TESTMAIL_NAMESPACE: ${{ secrets.TESTMAIL_NAMESPACE }}
E2E_TEST_TOKEN: ${{ inputs.use_test_token && secrets.E2E_TEST_TOKEN || '' }}
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
DEBUG_MODE: ${{ inputs.debug_mode || 'false' }}
run: |
echo "Starting E2E Happy Path Test"
echo "Browser: ${{ inputs.browser || 'chromium' }}"
echo "Debug mode: ${{ inputs.debug_mode || 'false' }}"
echo "Use test token: ${{ inputs.use_test_token || 'false' }}"
echo "Time: $(date)"
echo ""
# Run the real user happy path test
npx playwright test happy-path-real-user.spec.ts \
--config=playwright.production.config.ts \
--project=${{ inputs.browser || 'chromium' }} \
--reporter=list \
--timeout=3600000 \
2>&1 | tee test-output.log
continue-on-error: true
- name: Check test result
id: test_result
working-directory: frontend
run: |
if grep -q "passed" test-output.log && ! grep -q "failed" test-output.log; then
echo "result=success" >> $GITHUB_OUTPUT
echo "Test PASSED"
else
echo "result=failure" >> $GITHUB_OUTPUT
echo "Test FAILED - see artifacts for details"
fi
- name: Upload test artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: e2e-happy-path-results-${{ github.run_number }}
path: |
frontend/test-results/
frontend/playwright-report/
frontend/test-output.log
retention-days: 30
- name: Upload video recordings
if: always()
uses: actions/upload-artifact@v4
with:
name: e2e-happy-path-videos-${{ github.run_number }}
path: frontend/test-results/**/*.webm
retention-days: 30
if-no-files-found: ignore
- name: Create summary
if: always()
run: |
echo "## E2E Happy Path Test Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "| Property | Value |" >> $GITHUB_STEP_SUMMARY
echo "|----------|-------|" >> $GITHUB_STEP_SUMMARY
echo "| Run | #${{ github.run_number }} |" >> $GITHUB_STEP_SUMMARY
echo "| Trigger | ${{ github.event_name }} |" >> $GITHUB_STEP_SUMMARY
echo "| Browser | ${{ inputs.browser || 'chromium' }} |" >> $GITHUB_STEP_SUMMARY
echo "| Result | ${{ steps.test_result.outputs.result || 'unknown' }} |" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ -f frontend/test-output.log ]; then
echo "### Test Output (last 100 lines)" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
tail -100 frontend/test-output.log >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
fi
- name: Send email notification
if: always() && github.event_name == 'schedule'
uses: dawidd6/action-send-mail@v3
with:
server_address: smtp.sendgrid.net
server_port: 587
username: apikey
password: ${{ secrets.SENDGRID_API_KEY }}
subject: "E2E Happy Path Test ${{ steps.test_result.outputs.result == 'success' && 'PASSED' || 'FAILED' }} - Run #${{ github.run_number }}"
to: andrew@nomadkaraoke.com
from: "Karaoke Gen CI <noreply@nomadkaraoke.com>"
body: |
E2E Happy Path Test Results
===========================
Status: ${{ steps.test_result.outputs.result == 'success' && 'PASSED ✅' || 'FAILED ❌' }}
Run: #${{ github.run_number }}
Trigger: ${{ github.event_name }}
Time: ${{ github.event.repository.updated_at }}
View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
This is an automated notification from the daily E2E happy path test.
- name: Fail if test failed
if: steps.test_result.outputs.result == 'failure'
run: exit 1
-15
View File
@@ -12,9 +12,6 @@ tests/outputs
/input
/output/
# Cloud Infra
.emulator-logs
# Next.js / React Frontend (./frontend)
node_modules/
frontend/node_modules/
@@ -44,10 +41,6 @@ frontend/.pnpm-debug.log*
# Testing
frontend/coverage/
frontend/test-results/
frontend/playwright-report/
frontend/playwright/.cache/
frontend/e2e/screenshots/
# TypeScript
frontend/*.tsbuildinfo
@@ -80,12 +73,8 @@ dist/
downloads/
eggs/
.eggs/
# Python lib directories (not frontend)
lib/
lib64/
# But allow frontend lib directories
!frontend/lib/
!buy-site/lib/
parts/
sdist/
var/
@@ -228,7 +217,3 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
flacfetch
# Test output directories
Rick Astley*/
-163
View File
@@ -1,163 +0,0 @@
# Karaoke-Gen AI Assistant Guidelines
## Project Overview
**Karaoke video generation platform** - CLI tool and web service that creates professional karaoke videos with synchronized lyrics.
- **Production**: <https://gen.nomadkaraoke.com> (frontend), <https://api.nomadkaraoke.com> (backend)
- **CLI**: `karaoke-gen` (local), `karaoke-gen-remote` (cloud)
- **Repo**: <https://github.com/nomadkaraoke/karaoke-gen>
## Quick Reference
| What | Where |
|------|-------|
| Product vision & goals | `docs/PRODUCT-VISION.md` |
| Current status | `docs/README.md` |
| Architecture | `docs/ARCHITECTURE.md` |
| Dev setup & testing | `docs/DEVELOPMENT.md` |
| **Testing & code quality** | `docs/TESTING.md` |
| API reference | `docs/API.md` |
| Past learnings | `docs/LESSONS-LEARNED.md` |
| **Product communication** | `docs/PRODUCT-COMMUNICATION-GUIDE.md` |
| Brand style guide | `docs/BRAND-STYLE-GUIDE.md` |
## Tech Stack
- **Backend**: FastAPI on Cloud Run, Firestore, GCS, Secret Manager
- **Frontend**: Next.js on Cloudflare Pages
- **Processing**: Modal (GPU audio separation), AudioShake/Whisper (transcription)
- **Infra**: Pulumi IaC, GitHub Actions CI/CD
## Essential Rules
### Git Workflow
- **Never commit directly to `main`** - use `/new-worktree <description>` to start
- **Follow global workflow** - see `~/.claude/CLAUDE.md` for command sequence
- Create PR with summary, changes, testing info
- Merge only after CI passes
### Testing (Required)
**Before planning any implementation work**, read `docs/TESTING.md` and ensure your plan includes a thorough testing strategy that follows the guidance there. This includes:
- Which test types are needed (unit, integration, E2E)
- Where tests should be placed
- What mocking approach to use
- Coverage expectations
- **Production E2E tests for user-facing features** - see below
**Critical:** If your plan includes "Manual Testing" steps, convert them to automated Playwright E2E tests. Production E2E tests (in `frontend/e2e/production/`) are valuable even if only run once after deployment - they codify expected behavior and verify the deployed code works. See `docs/TESTING.md` for details.
**Before committing**, run all tests:
```bash
make test 2>&1 | tail -n 500
```
This single command:
- Installs dependencies automatically (backend + frontend)
- Runs all backend tests (unit, integration, emulator)
- Runs all frontend tests (Jest unit + Playwright E2E)
- Takes ~5-10 minutes
**All tests must pass.** Don't dismiss failures as "pre-existing" - investigate and fix them.
### Version Bumping
- Bump `tool.poetry.version` in `pyproject.toml` for code changes
- Skip for docs-only changes
### Infrastructure
- **All GCP changes via Pulumi PRs** - changes in `infrastructure/` deploy automatically on merge to main
- Never modify GCP resources directly via console or `gcloud` CLI
- `gcloud` CLI for reading/debugging only (e.g., checking logs, SSH to VMs)
- Local `pulumi preview` works but `pulumi up` requires broader permissions than CI
- Stop and notify user on auth issues
## API Authentication (for Agents)
When you need to call backend APIs programmatically:
### Internal Worker Endpoints
Use the `X-Admin-Token` header with the token from Secret Manager:
```bash
# Trigger video worker
curl -X POST "https://api.nomadkaraoke.com/api/internal/workers/video" \
-H "X-Admin-Token: $(gcloud secrets versions access latest --secret=admin-tokens --project=nomadkaraoke)" \
-H "Content-Type: application/json" \
-d '{"job_id": "YOUR_JOB_ID"}'
# Other internal endpoints follow the same pattern
```
### Firestore Direct Access
```bash
# Set the correct project
export GOOGLE_CLOUD_PROJECT=nomadkaraoke
# Use Python with google-cloud-firestore
python3 << 'EOF'
import os
os.environ['GOOGLE_CLOUD_PROJECT'] = 'nomadkaraoke'
from google.cloud import firestore
db = firestore.Client(project='nomadkaraoke')
# Query/update documents...
EOF
```
### GCE Encoding Worker
```bash
# SSH to restart service (clears in-memory job queue)
gcloud compute ssh encoding-worker --zone=us-central1-a --project=nomadkaraoke \
--command="sudo systemctl restart encoding-worker"
# Check health
gcloud compute ssh encoding-worker --zone=us-central1-a --project=nomadkaraoke \
--command="curl -s http://localhost:8080/health"
```
### What Doesn't Work
- `gcloud auth print-identity-token` - Wrong token type for internal endpoints
- Cloud Tasks without proper service account - Gets 401 from Cloud Run
## Documentation Maintenance
### Structure
```text
docs/
├── README.md # Current status + navigation (UPDATE THIS)
├── ARCHITECTURE.md # System design
├── DEVELOPMENT.md # Dev setup, testing, deployment
├── API.md # Backend API reference
├── LESSONS-LEARNED.md # Key insights for future agents
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
```
### Rules
1. **Before merging PRs**: Run `/docs-review` to check if docs need updating
2. **Periodically**: Run `/docs-maintain` to verify docs are organized
3. **For significant work**: Create `docs/archive/YYYY-MM-DD-topic.md`
4. **Add learnings**: Update `docs/LESSONS-LEARNED.md` with insights
5. **Keep status current**: Update `docs/README.md` when project state changes
### What Goes Where
- **README.md**: Current status, known issues, quick links
- **ARCHITECTURE.md**: System design, data flow, tech decisions
- **DEVELOPMENT.md**: Setup, testing, deployment, CI/CD
- **API.md**: Endpoints, request/response formats
- **LESSONS-LEARNED.md**: What worked, what didn't, gotchas
- **archive/**: Completed work, session summaries, old plans
## Related Projects
### flacfetch
Location: `/Users/andrew/Projects/flacfetch`
Workflow: Make changes on `main`, bump version, push, wait for CI, then `poetry update flacfetch` in karaoke-gen.
## Pre-PR Checklist
Follow global workflow (`~/.claude/CLAUDE.md`):
- [ ] Tests pass: `/test` (or `make test` / `npm run test:all`)
- [ ] Version bumped (if code changed)
- [ ] Docs checked: `/docs-review`
- [ ] Code reviewed: `/review` (CodeRabbit CLI)
- [ ] PR created: `/pr` (adds @coderabbitai ignore)
-103
View File
@@ -1,103 +0,0 @@
# Makefile for karaoke-gen project
# Run `make help` to see available commands
.PHONY: help install install-backend install-frontend test test-unit test-backend test-e2e test-frontend test-all lint clean emulators-start emulators-stop
# Default target
help:
@echo "Available commands:"
@echo " make test - Run ALL tests (backend + frontend, installs deps automatically)"
@echo " make test-backend - Run backend tests only (unit + emulator)"
@echo " make test-frontend - Run frontend tests only (unit + E2E)"
@echo " make test-unit - Run unit tests only (karaoke_gen package)"
@echo " make test-e2e - Run emulator tests with auto-start/stop"
@echo " make install - Install all dependencies (backend + frontend)"
@echo " make lint - Run linter checks"
@echo " make emulators-start - Start GCP emulators for local development"
@echo " make emulators-stop - Stop GCP emulators"
@echo ""
@echo "Before committing, run: make test"
# Install dependencies (only if needed)
install-backend:
@if [ ! -d "$$(poetry env info --path 2>/dev/null)" ] || ! poetry run python -c "import fastapi" 2>/dev/null; then \
echo "=== Installing backend dependencies ==="; \
poetry install; \
fi
install-frontend:
@if [ ! -d "frontend/node_modules" ]; then \
echo "=== Installing frontend dependencies ==="; \
cd frontend && npm install; \
fi
install: install-backend install-frontend
# Run unit tests for karaoke_gen package
test-unit: install-backend
@echo "=== Running karaoke_gen unit tests ==="
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=term-missing --cov-fail-under=69
# Run backend unit tests (excludes emulator tests)
test-backend-unit: install-backend
@echo "=== Running backend unit tests ==="
poetry run pytest backend/tests/ --ignore=backend/tests/emulator -v
# Run E2E integration tests with emulators
test-e2e: install-backend
@echo "=== Running E2E integration tests with emulators ==="
@./scripts/start-emulators.sh || (echo "Failed to start emulators" && exit 1)
@export FIRESTORE_EMULATOR_HOST=127.0.0.1:8080 && \
export STORAGE_EMULATOR_HOST=http://127.0.0.1:4443 && \
export GOOGLE_CLOUD_PROJECT=test-project && \
export GCS_BUCKET_NAME=test-bucket && \
export ADMIN_TOKENS=test-admin-token && \
export ENVIRONMENT=test && \
poetry run pytest backend/tests/emulator/ -v; \
TEST_RESULT=$$?; \
./scripts/stop-emulators.sh; \
exit $$TEST_RESULT
# Run all backend tests (unit + emulator)
test-backend: test-unit test-backend-unit test-e2e
@echo ""
@echo "✅ Backend tests passed!"
# Run frontend tests (unit + E2E)
test-frontend: install-frontend
@echo "=== Running frontend tests ==="
cd frontend && npm run test:all
# Run ALL tests (backend + frontend) - use this before committing!
test: test-backend test-frontend
@echo ""
@echo "✅ All tests passed! Ready to commit."
# Alias for test
test-all: test
# Lint checks
lint:
@echo "=== Running lint checks ==="
poetry run ruff check karaoke_gen/ backend/
poetry run ruff format --check karaoke_gen/ backend/
# Start emulators for local development
emulators-start:
@echo "=== Starting GCP emulators ==="
./scripts/start-emulators.sh
# Stop emulators
emulators-stop:
@echo "=== Stopping GCP emulators ==="
./scripts/stop-emulators.sh
# Clean up temporary files
clean:
find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
find . -type d -name ".pytest_cache" -exec rm -rf {} + 2>/dev/null || true
find . -type d -name ".coverage" -exec rm -rf {} + 2>/dev/null || true
find . -type f -name "*.pyc" -delete 2>/dev/null || true
rm -rf htmlcov/ .coverage coverage.xml 2>/dev/null || true
@echo "Cleaned up temporary files"
+3 -176
View File
@@ -1,7 +1,7 @@
# Karaoke Generator 🎶 🎥 🚀
![PyPI - Version](https://img.shields.io/pypi/v/karaoke-gen)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/karaoke-gen)
![Python Version](https://img.shields.io/badge/python-3.10+-blue)
![Tests](https://github.com/nomadkaraoke/karaoke-gen/workflows/Test%20and%20Publish/badge.svg)
![Test Coverage](https://codecov.io/gh/nomadkaraoke/karaoke-gen/branch/main/graph/badge.svg)
@@ -54,76 +54,10 @@ pip install karaoke-gen
This installs both `karaoke-gen` (local) and `karaoke-gen-remote` (cloud) CLIs.
### Requirements
- Python 3.10-3.13
- Python 3.10+
- FFmpeg
- For local processing: CUDA-capable GPU or Apple Silicon CPU recommended
### Transcription Provider Setup
**Transcription is required** for creating karaoke videos with synchronized lyrics. The system needs word-level timing data to display lyrics in sync with the music.
#### Option 1: AudioShake (Recommended)
Commercial service with high-quality transcription. Best for production use.
```bash
export AUDIOSHAKE_API_TOKEN="your_audioshake_token"
```
Get an API key at [https://www.audioshake.ai/](https://www.audioshake.ai/) - business only, at time of writing this.
#### Option 2: Local Whisper (No Cloud Required)
Run Whisper directly on your local machine using whisper-timestamped. Works on CPU, NVIDIA GPU (CUDA), or Apple Silicon.
```bash
# Install with local Whisper support
pip install "karaoke-gen[local-whisper]"
# Optional: Configure model size (tiny, base, small, medium, large)
export WHISPER_MODEL_SIZE="medium"
# Optional: Force specific device (cpu, cuda, mps)
export WHISPER_DEVICE="cpu"
```
**Model Size Guide:**
| Model | VRAM | Speed | Quality |
|-------|------|-------|---------|
| tiny | ~1GB | Fast | Lower |
| base | ~1GB | Fast | Basic |
| small | ~2GB | Medium | Good |
| medium | ~5GB | Slower | Better |
| large | ~10GB | Slowest | Best |
**CPU-Only Installation** (no GPU required):
```bash
# Pre-install CPU-only PyTorch first
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install "karaoke-gen[local-whisper]"
```
Local Whisper runs automatically as a fallback when no cloud transcription services are configured.
#### Option 3: Whisper via RunPod
Cloud-based alternative using OpenAI's Whisper model on RunPod infrastructure.
```bash
export RUNPOD_API_KEY="your_runpod_key"
export WHISPER_RUNPOD_ID="your_whisper_endpoint_id"
```
Set up a Whisper endpoint at [https://www.runpod.io/](https://www.runpod.io/)
#### Without Transcription (Instrumental Only)
If you don't need synchronized lyrics, use the `--skip-lyrics` flag:
```bash
karaoke-gen --skip-lyrics "Artist" "Title"
```
This creates an instrumental-only karaoke video without lyrics overlay.
> **Note:** See `lyrics_transcriber_temp/README.md` for detailed transcription provider configuration options.
---
## 🖥️ Local CLI (`karaoke-gen`)
@@ -246,11 +180,9 @@ karaoke-gen-remote \
|----------|-------------|---------|
| `KARAOKE_GEN_URL` | Backend service URL | Required |
| `KARAOKE_GEN_AUTH_TOKEN` | Admin auth token (for protected endpoints) | Optional |
| `REVIEW_UI_URL` | Lyrics review UI URL | `https://gen.nomadkaraoke.com/lyrics/` |
| `REVIEW_UI_URL` | Lyrics review UI URL | `https://lyrics.nomadkaraoke.com` |
| `POLL_INTERVAL` | Seconds between status polls | `5` |
**Note:** The `REVIEW_UI_URL` defaults to the hosted lyrics review UI. For local development, set it to `http://localhost:5173` if you're running the frontend dev server.
### Authentication
The backend uses token-based authentication for admin operations (bulk delete, internal worker triggers). For basic job submission and monitoring, authentication is optional.
@@ -541,111 +473,6 @@ Check backend health status.
---
## 🔧 Troubleshooting
### "No suitable files found for processing"
This error occurs during the finalisation step when the `(With Vocals).mkv` file is missing. This file is created during lyrics transcription.
**Most common cause:** No transcription provider configured.
**Quick fix:**
1. Check if transcription providers are configured:
```bash
echo $AUDIOSHAKE_API_TOKEN
echo $RUNPOD_API_KEY
```
2. If both are empty, set up a provider (see [Transcription Provider Setup](#transcription-provider-setup))
3. Or use `--skip-lyrics` for instrumental-only karaoke:
```bash
karaoke-gen --skip-lyrics "Artist" "Title"
```
**Other causes:**
- Invalid API credentials - verify your tokens are correct and active
- API service unavailable - check service status pages
- Network connectivity issues - ensure you can reach the API endpoints
- Transcription timeout - try again or use a different provider
### Transcription Fails Silently
If karaoke-gen runs without errors but produces no synchronized lyrics:
1. **Check logs** - Run with `--log_level debug` for detailed output:
```bash
karaoke-gen --log_level debug "Artist" "Title"
```
2. **Verify environment variables** - Ensure API tokens are exported in your shell:
```bash
# Check if set
printenv | grep -E "(AUDIOSHAKE|RUNPOD|WHISPER)"
# Set in current session
export AUDIOSHAKE_API_TOKEN="your_token"
```
3. **Test API connectivity** - Verify you can reach the transcription service
### "No lyrics found from any source"
This warning means no reference lyrics were fetched from online sources (Genius, Spotify, Musixmatch). The transcription will still work, but auto-correction may be less accurate.
**To fix:**
- Set `GENIUS_API_TOKEN` for Genius lyrics
- Set `SPOTIFY_COOKIE_SP_DC` for Spotify lyrics
- Set `RAPIDAPI_KEY` for Musixmatch lyrics
- Or provide lyrics manually with `--lyrics_file /path/to/lyrics.txt`
### Video Quality Issues
If the output video has quality problems:
- Ensure FFmpeg is properly installed: `ffmpeg -version`
- Check available codecs: `ffmpeg -codecs`
- For 4K output, ensure sufficient disk space (10GB+ per track)
### Local Whisper Issues
#### GPU Out of Memory
If you get CUDA out of memory errors:
```bash
# Use a smaller model
export WHISPER_MODEL_SIZE="small" # or "tiny"
# Or force CPU mode
export WHISPER_DEVICE="cpu"
```
#### Slow Transcription on CPU
CPU transcription is significantly slower than GPU. For faster processing:
- Use a smaller model (`tiny` or `base`)
- Consider using cloud transcription (AudioShake or RunPod)
- On Apple Silicon, the `small` model offers good speed/quality balance
#### Model Download Issues
Whisper models are downloaded on first use (~1-3GB depending on size). If downloads fail:
- Check your internet connection
- Set a custom cache directory: `export WHISPER_CACHE_DIR="/path/with/space"`
- Models are cached in `~/.cache/whisper/` by default
#### whisper-timestamped Not Found
If you get "whisper-timestamped is not installed":
```bash
pip install "karaoke-gen[local-whisper]"
# Or install directly:
pip install whisper-timestamped
```
#### Disabling Local Whisper
If you want to disable local Whisper (e.g., to force cloud transcription):
```bash
export ENABLE_LOCAL_WHISPER="false"
```
---
## 🧪 Development
### Running Tests
-75
View File
@@ -1,75 +0,0 @@
[
{
"test_name": "Sound-alike error (wurld -> world)",
"success": true,
"proposals": [
{
"action": "ReplaceWord",
"word_id": null,
"word_ids": [
"w1"
],
"replacement_text": "world",
"confidence": 0.75,
"reason": "Sound-alike error. Reference suggests: 'world'. The transcription 'wurld' is a phonetic misspelling or sound-alike of the word 'world' which appears in both reference sources (Genius and LRCLIB).",
"gap_category": "GapCategory.SOUND_ALIKE",
"requires_human_review": false
}
]
},
{
"test_name": "Punctuation difference",
"success": true,
"proposals": [
{
"action": "NoAction",
"word_id": null,
"word_ids": [
"w2"
],
"replacement_text": null,
"confidence": 0.99,
"reason": "Transcription matches at least one reference source. The gap text 'Hello' followed by the context 'is it me you're looking for' matches the Genius reference lyrics exactly, excluding punctuation. According to the guidelines, if any reference source matches the transcription text (ignoring punctuation), it should be classified as NO_ERROR.",
"gap_category": "GapCategory.NO_ERROR",
"requires_human_review": false
}
]
},
{
"test_name": "Extra filler word",
"success": true,
"proposals": [
{
"action": "DeleteWord",
"word_id": "w3",
"word_ids": null,
"replacement_text": null,
"confidence": 0.8,
"reason": "Extra filler word 'And' at sentence start not in reference. The transcription includes the filler word 'And' at the beginning of the phrase, which is not present in the reference lyrics. This matches the definition of EXTRA_WORDS.",
"gap_category": "GapCategory.EXTRA_WORDS",
"requires_human_review": false
}
]
},
{
"test_name": "Complex multi-word error",
"success": true,
"proposals": [
{
"action": "NoAction",
"word_id": null,
"word_ids": [
"w7",
"w8",
"w9",
"w10"
],
"replacement_text": null,
"confidence": 0.99,
"reason": "Transcription matches at least one reference source. The transcription text 'gonna give you up' matches the Genius reference lyrics exactly when ignoring punctuation and capitalization. In context, the full phrase 'never gonna give you up' is correctly transcribed and aligns perfectly with the known lyrics and title of the song.",
"gap_category": "GapCategory.NO_ERROR",
"requires_human_review": false
}
]
}
]
+3 -8
View File
@@ -11,9 +11,7 @@
# gcloud builds submit --config=cloudbuild-base.yaml --project=nomadkaraoke
# Use pre-built base image with all dependencies
# Override with --build-arg BASE_IMAGE=karaoke-backend-base:local for local testing
ARG BASE_IMAGE=us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest
FROM ${BASE_IMAGE}
FROM us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest
# Set working directory (should already exist from base)
WORKDIR /app
@@ -36,8 +34,5 @@ ENV PORT=8080
# Expose port
EXPOSE 8080
# Run uvicorn directly
# Note: Transmission daemon removed - torrent downloads now handled by
# dedicated flacfetch VM service (see docs/00-current-plan/TORRENT-SERVICE-PLAN.md)
# Using --loop asyncio (instead of uvloop) to support nest_asyncio for remote flacfetch calls
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8080", "--loop", "asyncio"]
# Run the application
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8080"]
+4 -37
View File
@@ -7,46 +7,14 @@
FROM python:3.11-slim
# Install system dependencies required by karaoke-gen
# Note: Transmission daemon removed - torrent downloads now handled by
# dedicated flacfetch VM service (see docs/00-current-plan/TORRENT-SERVICE-PLAN.md)
RUN apt-get update && apt-get install -y \
ffmpeg \
libsndfile1 \
libsox-dev \
sox \
build-essential \
curl \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
# Install comprehensive font support for international lyrics and symbols
# Noto fonts provide full Unicode coverage including:
# - Latin, Greek, Cyrillic (core)
# - CJK: Chinese, Japanese, Korean (~150MB)
# - Arabic, Hebrew, Thai, Hindi, and other scripts (extra)
# - Color emoji (limited support in libass, but available)
# - Musical symbols (♪ ♫ etc.)
# Note: This adds ~200-300MB to the image but ensures all languages render correctly
RUN apt-get update && apt-get install -y \
fonts-noto-core \
fonts-noto-cjk \
fonts-noto-extra \
fonts-noto-color-emoji \
fontconfig \
&& fc-cache -f \
&& rm -rf /var/lib/apt/lists/*
# Install optimized static FFmpeg build (7.0.2) instead of Debian package (5.1.8)
# John Van Sickle's builds are compiled with better optimizations for x86_64
# This provides ~20-40% performance improvement for encoding tasks
# Source: https://johnvansickle.com/ffmpeg/
RUN curl -L https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz -o /tmp/ffmpeg.tar.xz && \
tar -xf /tmp/ffmpeg.tar.xz -C /tmp && \
cp /tmp/ffmpeg-*-amd64-static/ffmpeg /usr/local/bin/ && \
cp /tmp/ffmpeg-*-amd64-static/ffprobe /usr/local/bin/ && \
chmod +x /usr/local/bin/ffmpeg /usr/local/bin/ffprobe && \
rm -rf /tmp/ffmpeg* && \
ffmpeg -version
# Set working directory
WORKDIR /app
@@ -54,7 +22,6 @@ WORKDIR /app
COPY pyproject.toml README.md LICENSE /app/
COPY karaoke_gen /app/karaoke_gen
COPY lyrics_transcriber_temp /app/lyrics_transcriber_temp
COPY backend /app/backend
# Install all Python dependencies (the slow part - cached in base image)
RUN pip install --no-cache-dir --upgrade pip && \
@@ -67,9 +34,9 @@ RUN python -m spacy download en_core_web_sm
# Label with build info
ARG BUILD_DATE
ARG BUILD_ID
ARG BASE_CONTENT_HASH
ARG POETRY_LOCK_HASH
LABEL org.opencontainers.image.created="${BUILD_DATE}"
LABEL org.opencontainers.image.revision="${BUILD_ID}"
LABEL org.opencontainers.image.title="karaoke-backend-base"
LABEL org.opencontainers.image.description="Base image with Python dependencies and system tools for karaoke-backend"
LABEL base.content.hash="${BASE_CONTENT_HASH}"
LABEL org.opencontainers.image.description="Base image with Python dependencies for karaoke-backend"
LABEL poetry.lock.hash="${POETRY_LOCK_HASH}"
+30 -349
View File
@@ -2,28 +2,16 @@
FastAPI dependencies for authentication and authorization.
"""
import logging
import secrets
from datetime import datetime, timedelta, UTC
from typing import Optional, Tuple, Callable, Union
from fastapi import Depends, HTTPException, Header, Query, Request, Path
from typing import Optional, Tuple
from fastapi import Depends, HTTPException, Header, Query, Request
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from backend.services.auth_service import get_auth_service, UserType, AuthService, AuthResult
from backend.services.auth_service import get_auth_service, UserType, AuthService
logger = logging.getLogger(__name__)
def generate_review_token() -> str:
"""Generate a cryptographically secure review token."""
return secrets.token_urlsafe(32)
def get_review_token_expiry(hours: int = 24) -> datetime:
"""Get expiry time for a review token."""
return datetime.now(UTC) + timedelta(hours=hours)
# HTTP Bearer security scheme (Authorization: Bearer <token>)
security = HTTPBearer(auto_error=False)
@@ -36,21 +24,14 @@ async def get_token_from_request(
"""
Extract access token from request.
Supports three methods (in priority order):
1. X-Admin-Token header: Used by Cloud Tasks (since OIDC overwrites Authorization)
2. Authorization header: Authorization: Bearer <token>
3. Query parameter: ?token=<token> (for download links)
Supports two methods:
1. Authorization header: Authorization: Bearer <token>
2. Query parameter: ?token=<token> (for download links)
Returns:
Token string or None
"""
# Check X-Admin-Token header first (used by Cloud Tasks)
# Cloud Tasks OIDC token overwrites Authorization header, so we use a custom header
admin_token = request.headers.get("X-Admin-Token")
if admin_token:
return admin_token
# Try Authorization header
# Try Authorization header first
if credentials:
return credentials.credentials
@@ -66,130 +47,62 @@ async def require_auth(
auth_service: AuthService = Depends(get_auth_service),
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
token: Optional[str] = Query(None)
) -> AuthResult:
) -> Tuple[str, UserType, int]:
"""
Require authentication for an endpoint.
Returns:
AuthResult with full authentication context including:
- is_valid, user_type, remaining_uses, message (backward compatible via tuple unpacking)
- user_email: Email of authenticated user (if session/API key auth)
- is_admin: Whether user has admin privileges
(token, user_type, remaining_uses)
Raises:
HTTPException: 401 if authentication fails
"""
# Get token from request
token_str = await get_token_from_request(request, credentials, token)
if not token_str:
raise HTTPException(
status_code=401,
detail="Authentication required. Provide token via Authorization header or ?token= parameter"
)
# Validate token using the full method
auth_result = auth_service.validate_token_full(token_str)
# Get request_id from middleware for correlation
request_id = getattr(request.state, "request_id", None)
if not auth_result.is_valid:
# Log auth failure with request_id for correlation
auth_header = request.headers.get("Authorization", "")
logger.warning(
"auth_failed",
extra={
"request_id": request_id,
"audit_type": "auth_event",
"auth_message": auth_result.message,
"token_provided": bool(token_str),
"token_length": len(token_str) if token_str else 0,
"auth_header_present": bool(auth_header),
}
)
raise HTTPException(
status_code=401,
detail=f"Authentication failed: {auth_result.message}"
)
# Log successful auth with request_id for correlation with request audit
logger.info(
"auth_success",
extra={
"request_id": request_id,
"user_email": auth_result.user_email,
"user_type": auth_result.user_type.value if auth_result.user_type else None,
"is_admin": auth_result.is_admin,
"remaining_uses": auth_result.remaining_uses,
"audit_type": "auth_event",
}
)
return auth_result
async def require_auth_legacy(
request: Request,
auth_service: AuthService = Depends(get_auth_service),
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
token: Optional[str] = Query(None)
) -> Tuple[str, UserType, int]:
"""
Legacy authentication dependency that returns tuple.
DEPRECATED: Use require_auth which returns AuthResult instead.
Returns:
(token, user_type, remaining_uses)
"""
token_str = await get_token_from_request(request, credentials, token)
if not token_str:
raise HTTPException(
status_code=401,
detail="Authentication required. Provide token via Authorization header or ?token= parameter"
)
# Validate token
is_valid, user_type, remaining_uses, message = auth_service.validate_token(token_str)
if not is_valid:
logger.warning(f"Authentication failed: {message}")
raise HTTPException(
status_code=401,
detail=f"Authentication failed: {message}"
)
logger.info(f"Authenticated as {user_type} (remaining: {remaining_uses})")
return token_str, user_type, remaining_uses
async def require_admin(
auth_result: AuthResult = Depends(require_auth)
) -> AuthResult:
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
) -> Tuple[str, UserType, int]:
"""
Require admin access for an endpoint.
Admin access is granted if:
- Using an admin token from ADMIN_TOKENS env var
- User email is from admin domain (e.g., @nomadkaraoke.com)
- User role is set to ADMIN in database
Returns:
AuthResult with admin privileges
(token, user_type, remaining_uses)
Raises:
HTTPException: 403 if user is not admin
"""
if not auth_result.is_admin:
logger.warning(
f"Admin access denied for {auth_result.user_type} user "
f"(email: {auth_result.user_email or 'unknown'})"
)
token, user_type, remaining_uses = auth_data
if user_type != UserType.ADMIN:
logger.warning(f"Admin access denied for {user_type} user")
raise HTTPException(
status_code=403,
detail="Admin access required"
)
return auth_result
return token, user_type, remaining_uses
def optional_auth(
@@ -223,235 +136,3 @@ def optional_auth(
return None
return token_str, user_type, remaining_uses
def require_review_auth_factory(job_id_param: str = "job_id"):
"""
Factory to create a review authentication dependency.
Accepts either:
1. Full user authentication (admin/user token) - also validates job ownership
2. Job-specific review token (only valid for the specific job)
Args:
job_id_param: Name of the path parameter containing the job ID
Returns:
Dependency function that validates review access
"""
async def require_review_auth(
request: Request,
job_id: str = Path(...),
review_token: Optional[str] = Query(None, description="Job-specific review token"),
auth_service: AuthService = Depends(get_auth_service),
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
token: Optional[str] = Query(None, alias="token", description="Full access token")
) -> Tuple[str, str]:
"""
Validate review access for a job.
Returns:
(job_id, auth_type) where auth_type is "full" or "review_token"
Raises:
HTTPException: 401 if authentication fails
HTTPException: 403 if user doesn't own the job (for full auth)
"""
# Import here to avoid circular dependency
from backend.services.job_manager import JobManager
job_manager = JobManager()
# Try full authentication first
full_token = None
if credentials:
full_token = credentials.credentials
elif token:
full_token = token
if full_token:
auth_result = auth_service.validate_token_full(full_token)
if auth_result.is_valid:
# For full auth, also verify job ownership
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Check ownership: admin can access all, users only their own jobs
if not auth_result.is_admin:
if auth_result.user_email and job.user_email:
if auth_result.user_email.lower() != job.user_email.lower():
logger.warning(
f"Review access denied: user {auth_result.user_email} "
f"tried to access job {job_id} owned by {job.user_email}"
)
raise HTTPException(
status_code=403,
detail="You don't have permission to access this job's review"
)
elif job.user_email:
# Token auth without email trying to access a job with owner
raise HTTPException(
status_code=403,
detail="You don't have permission to access this job's review"
)
logger.info(f"Review access granted via full auth ({auth_result.user_type}) for job {job_id}")
return job_id, "full"
# Try review token
if review_token:
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Validate review token matches
if job.review_token and secrets.compare_digest(job.review_token, review_token):
# Check expiry if set
if job.review_token_expires_at:
now = datetime.now(UTC)
# Handle timezone-naive datetimes from Firestore
expiry = job.review_token_expires_at
if expiry.tzinfo is None:
expiry = expiry.replace(tzinfo=UTC)
if now > expiry:
logger.warning(f"Review token expired for job {job_id}")
raise HTTPException(
status_code=401,
detail="Review token has expired. Please request a new review link."
)
logger.info(f"Review access granted via review_token for job {job_id}")
return job_id, "review_token"
else:
logger.warning(f"Invalid review token for job {job_id}")
# No valid authentication
raise HTTPException(
status_code=401,
detail="Authentication required. Provide either a full access token or a valid review_token for this job."
)
return require_review_auth
# Default instance for most review endpoints
require_review_auth = require_review_auth_factory()
def require_instrumental_auth_factory(job_id_param: str = "job_id"):
"""
Factory to create an instrumental review authentication dependency.
Accepts either:
1. Full user authentication (admin/user token) - also validates job ownership
2. Job-specific instrumental token (only valid for the specific job)
Args:
job_id_param: Name of the path parameter containing the job ID
Returns:
Dependency function that validates instrumental review access
"""
async def require_instrumental_auth(
request: Request,
job_id: str = Path(...),
instrumental_token: Optional[str] = Query(None, description="Job-specific instrumental review token"),
auth_service: AuthService = Depends(get_auth_service),
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
token: Optional[str] = Query(None, alias="token", description="Full access token")
) -> Tuple[str, str]:
"""
Validate instrumental review access for a job.
Returns:
(job_id, auth_type) where auth_type is "full" or "instrumental_token"
Raises:
HTTPException: 401 if authentication fails
HTTPException: 403 if user doesn't own the job (for full auth)
"""
# Import here to avoid circular dependency
from backend.services.job_manager import JobManager
job_manager = JobManager()
# Try full authentication first
full_token = None
if credentials:
full_token = credentials.credentials
elif token:
full_token = token
if full_token:
auth_result = auth_service.validate_token_full(full_token)
if auth_result.is_valid:
# For full auth, also verify job ownership
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Check ownership: admin can access all, users only their own jobs
if not auth_result.is_admin:
if auth_result.user_email and job.user_email:
if auth_result.user_email.lower() != job.user_email.lower():
logger.warning(
f"Instrumental access denied: user {auth_result.user_email} "
f"tried to access job {job_id} owned by {job.user_email}"
)
raise HTTPException(
status_code=403,
detail="You don't have permission to access this job's instrumental review"
)
elif job.user_email:
# Token auth without email trying to access a job with owner
raise HTTPException(
status_code=403,
detail="You don't have permission to access this job's instrumental review"
)
logger.info(f"Instrumental access granted via full auth ({auth_result.user_type}) for job {job_id}")
return job_id, "full"
# Try instrumental token
if instrumental_token:
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Validate instrumental token matches
if job.instrumental_token and secrets.compare_digest(job.instrumental_token, instrumental_token):
# Check expiry if set
if job.instrumental_token_expires_at:
now = datetime.now(UTC)
# Handle timezone-naive datetimes from Firestore
expiry = job.instrumental_token_expires_at
if expiry.tzinfo is None:
expiry = expiry.replace(tzinfo=UTC)
if now > expiry:
logger.warning(f"Instrumental token expired for job {job_id}")
raise HTTPException(
status_code=401,
detail="Instrumental review token has expired. Please request a new review link."
)
logger.info(f"Instrumental access granted via instrumental_token for job {job_id}")
return job_id, "instrumental_token"
else:
logger.warning(f"Invalid instrumental token for job {job_id}")
# No valid authentication
raise HTTPException(
status_code=401,
detail="Authentication required. Provide either a full access token or a valid instrumental_token for this job."
)
return require_instrumental_auth
# Default instance for most instrumental review endpoints
require_instrumental_auth = require_instrumental_auth_factory()
File diff suppressed because it is too large. Load diff
+91 -444
View File
@@ -17,11 +17,10 @@ import os
import tempfile
from typing import Optional, List, Dict, Any
from fastapi import APIRouter, HTTPException, BackgroundTasks, Request, Depends
from pydantic import BaseModel, Field, validator
from fastapi import APIRouter, HTTPException, BackgroundTasks, Request
from pydantic import BaseModel, Field
from backend.models.job import JobCreate, JobStatus
from karaoke_gen.utils import normalize_text
from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.services.worker_service import get_worker_service
@@ -33,28 +32,10 @@ from backend.services.audio_search_service import (
NoResultsError,
DownloadError,
)
from backend.services.theme_service import get_theme_service
from backend.services.job_defaults_service import resolve_cdg_txt_defaults
from backend.config import get_settings
from backend.version import VERSION
from backend.api.dependencies import require_auth
from backend.services.auth_service import UserType, AuthResult
from backend.middleware.tenant import get_tenant_config_from_request
from pathlib import Path
logger = logging.getLogger(__name__)
# Valid style file types (from file_upload.py)
STYLE_FILE_TYPES = {
'style_params': {'.json'},
'style_intro_background': {'.jpg', '.jpeg', '.png'},
'style_karaoke_background': {'.jpg', '.jpeg', '.png'},
'style_end_background': {'.jpg', '.jpeg', '.png'},
'style_font': {'.ttf', '.otf'},
'style_cdg_instrumental_background': {'.jpg', '.jpeg', '.png'},
'style_cdg_title_background': {'.jpg', '.jpeg', '.png'},
'style_cdg_outro_background': {'.jpg', '.jpeg', '.png'},
}
router = APIRouter(tags=["audio-search"])
# Initialize services
@@ -67,39 +48,21 @@ worker_service = get_worker_service()
# Pydantic models
# ============================================================================
class StyleFileRequest(BaseModel):
"""Information about a style file to be uploaded."""
filename: str = Field(..., description="Original filename with extension")
content_type: str = Field(..., description="MIME type of the file")
file_type: str = Field(..., description="Type of file: 'style_params', 'style_intro_background', 'style_karaoke_background', 'style_end_background', 'style_font', 'style_cdg_instrumental_background', 'style_cdg_title_background', 'style_cdg_outro_background'")
class StyleUploadUrl(BaseModel):
"""Signed URL for uploading a style file."""
file_type: str = Field(..., description="Type of file being uploaded")
gcs_path: str = Field(..., description="Destination path in GCS")
upload_url: str = Field(..., description="Signed URL to PUT the file to")
class AudioSearchRequest(BaseModel):
"""Request to search for audio by artist and title."""
artist: str = Field(..., description="Artist name to search for")
title: str = Field(..., description="Song title to search for")
# Auto-download mode
auto_download: bool = Field(False, description="Automatically select best result and download")
# Theme configuration
theme_id: Optional[str] = Field(None, description="Theme ID to use (e.g., 'nomad', 'default'). If set, CDG/TXT are enabled by default.")
color_overrides: Optional[Dict[str, str]] = Field(None, description="Color overrides: artist_color, title_color, sung_lyrics_color, unsung_lyrics_color (hex #RRGGBB)")
# Processing options (CDG/TXT require style config or theme, disabled by default unless theme is set)
enable_cdg: Optional[bool] = Field(None, description="Generate CDG+MP3 package. Default: True if theme_id set, False otherwise")
enable_txt: Optional[bool] = Field(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise")
# Processing options
enable_cdg: bool = Field(True, description="Generate CDG+MP3 package")
enable_txt: bool = Field(True, description="Generate TXT+MP3 package")
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
enable_youtube_upload: Optional[bool] = Field(None, description="Upload to YouTube. None = use server default")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
@@ -111,66 +74,23 @@ class AudioSearchRequest(BaseModel):
lyrics_artist: Optional[str] = Field(None, description="Override artist name for lyrics search")
lyrics_title: Optional[str] = Field(None, description="Override title for lyrics search")
subtitle_offset_ms: int = Field(0, description="Subtitle timing offset in milliseconds")
# Display overrides (optional - if empty, search values are used for display)
display_artist: Optional[str] = Field(None, description="Artist name for title screens/filenames. If empty, uses search artist.")
display_title: Optional[str] = Field(None, description="Title for title screens/filenames. If empty, uses search title.")
# Audio separation model configuration
clean_instrumental_model: Optional[str] = Field(None, description="Model for clean instrumental separation")
backing_vocals_models: Optional[List[str]] = Field(None, description="Models for backing vocals separation")
other_stems_models: Optional[List[str]] = Field(None, description="Models for other stems")
# Style file uploads (optional)
# Style params JSON and related assets (background images, fonts)
style_files: Optional[List[StyleFileRequest]] = Field(None, description="Style files to upload (style_params JSON, backgrounds, fonts)")
# Non-interactive mode
non_interactive: bool = Field(False, description="Skip interactive steps (lyrics review, instrumental selection)")
@validator('artist', 'title', 'lyrics_artist', 'lyrics_title', 'display_artist', 'display_title')
def normalize_text_fields(cls, v):
"""Normalize text fields to standardize Unicode characters."""
if v is not None and isinstance(v, str):
return normalize_text(v)
return v
class AudioSearchResultResponse(BaseModel):
"""A single audio search result.
Contains all fields needed for rich display using flacfetch's
shared formatting functions.
"""
"""A single audio search result."""
index: int
title: str
artist: str
provider: str # Maps to source_name in Release
url: Optional[str] = None # Maps to download_url in Release (may be None for remote)
duration: Optional[int] = None # Maps to duration_seconds in Release
quality: Optional[str] = None # Stringified quality
source_id: Optional[str] = None # Maps to info_hash in Release
seeders: Optional[int] = None
target_file: Optional[str] = None
# Additional fields for rich display (from Release.to_dict())
year: Optional[int] = None
label: Optional[str] = None
edition_info: Optional[str] = None
release_type: Optional[str] = None
channel: Optional[str] = None # For YouTube
view_count: Optional[int] = None # For YouTube
size_bytes: Optional[int] = None
target_file_size: Optional[int] = None
track_pattern: Optional[str] = None
match_score: Optional[float] = None
# Pre-computed display fields
formatted_size: Optional[str] = None
formatted_duration: Optional[str] = None
formatted_views: Optional[str] = None
is_lossless: Optional[bool] = None
quality_str: Optional[str] = None
# Full quality object for Release.from_dict()
quality_data: Optional[Dict[str, Any]] = None
provider: str
url: str
duration: Optional[int] = None
quality: Optional[str] = None
source_id: Optional[str] = None
class AudioSearchResponse(BaseModel):
@@ -182,8 +102,6 @@ class AudioSearchResponse(BaseModel):
results_count: int = 0
auto_download: bool = False
server_version: str
# Style file upload URLs (when style_files are specified in request)
style_upload_urls: Optional[List[StyleUploadUrl]] = None
class AudioSelectRequest(BaseModel):
@@ -250,10 +168,6 @@ async def _download_and_start_processing(
This is called either:
- Immediately after search if auto_download=True
- When user calls the select endpoint
For remote flacfetch downloads (torrent sources), the file is downloaded
on the flacfetch VM and uploaded directly to GCS. For local downloads
(YouTube), the file is downloaded locally and then uploaded to GCS.
"""
job = job_manager.get_job(job_id)
if not job:
@@ -263,17 +177,14 @@ async def _download_and_start_processing(
search_results = job.state_data.get('audio_search_results', [])
if not search_results:
raise HTTPException(status_code=400, detail="No search results available")
if selection_index < 0 or selection_index >= len(search_results):
raise HTTPException(
status_code=400,
detail=f"Invalid selection index {selection_index}. Valid range: 0-{len(search_results)-1}"
)
selected = search_results[selection_index]
# Get remote_search_id from state_data (stored during initial search)
remote_search_id = job.state_data.get('remote_search_id')
# Transition to downloading state
job_manager.transition_to_state(
@@ -288,129 +199,33 @@ async def _download_and_start_processing(
)
try:
# Determine if this is a remote torrent download
is_torrent_source = selected.get('provider') in ['RED', 'OPS']
is_remote_enabled = audio_search_service.is_remote_enabled()
# Check if we can use download_by_id (preferred - avoids re-searching)
source_id = selected.get('source_id')
source_name = selected.get('provider')
target_file = selected.get('target_file')
download_url = selected.get('url')
# For remote torrent downloads, have flacfetch VM upload directly to GCS
if is_torrent_source and is_remote_enabled:
# Generate GCS path for remote upload
gcs_destination = f"uploads/{job_id}/audio/"
# Use download_by_id if we have source_id (preferred - no re-search needed)
if source_id and source_name:
logger.info(f"Using download_by_id for {source_name} ID={source_id} with GCS upload to: {gcs_destination}")
result = audio_search_service.download_by_id(
source_name=source_name,
source_id=source_id,
output_dir="", # Not used for remote
target_file=target_file,
download_url=download_url,
gcs_path=gcs_destination,
)
else:
# Fallback to search-based download (requires re-search)
logger.info(f"No source_id available, falling back to search-based download to: {gcs_destination}")
result = audio_search_service.download(
result_index=selection_index,
output_dir="", # Not used for remote
gcs_path=gcs_destination,
remote_search_id=remote_search_id,
)
# For remote downloads, filepath is already the GCS path
if result.filepath.startswith("gs://"):
# Extract the path portion after the bucket name
# Format: gs://bucket/uploads/job_id/audio/filename.flac
parts = result.filepath.replace("gs://", "").split("/", 1)
if len(parts) == 2:
audio_gcs_path = parts[1]
else:
audio_gcs_path = result.filepath
filename = os.path.basename(result.filepath)
else:
# Fallback: treat as local path (shouldn't happen for remote)
filename = os.path.basename(result.filepath)
audio_gcs_path = f"uploads/{job_id}/audio/{filename}"
logger.warning(f"Remote download returned local path: {result.filepath}, uploading manually")
with open(result.filepath, 'rb') as f:
storage_service.upload_fileobj(f, audio_gcs_path, content_type='audio/flac')
logger.info(f"Remote download complete, GCS path: {audio_gcs_path}")
else:
# Local download (YouTube or fallback)
temp_dir = tempfile.mkdtemp(prefix=f"audio_download_{job_id}_")
# Use download_by_id if we have source_id and remote is enabled
if source_id and source_name and is_remote_enabled:
logger.info(f"Using download_by_id for local download: {source_name} ID={source_id}")
result = audio_search_service.download_by_id(
source_name=source_name,
source_id=source_id,
output_dir=temp_dir,
target_file=target_file,
download_url=download_url,
)
elif source_name == 'YouTube' and download_url:
# For YouTube without remote, download directly using URL from state_data
# This avoids relying on in-memory cache which doesn't persist across instances
logger.info(f"Downloading YouTube audio directly from URL: {download_url}")
from backend.workers.audio_worker import download_from_url
local_path = await download_from_url(
download_url,
temp_dir,
selected.get('artist'),
selected.get('title')
)
if not local_path or not os.path.exists(local_path):
raise DownloadError(f"Failed to download from YouTube: {download_url}")
# Create a result-like object
class DownloadResult:
def __init__(self, filepath):
self.filepath = filepath
result = DownloadResult(local_path)
else:
# Fallback to search-based download (may fail if cache is empty)
logger.warning(f"Falling back to cache-based download for {source_name} - this may fail on multi-instance deployments")
result = audio_search_service.download(
result_index=selection_index,
output_dir=temp_dir,
remote_search_id=remote_search_id, # Pass for potential fallback scenarios
)
# Upload to GCS
filename = os.path.basename(result.filepath)
audio_gcs_path = f"uploads/{job_id}/audio/{filename}"
with open(result.filepath, 'rb') as f:
storage_service.upload_fileobj(
f,
audio_gcs_path,
content_type='audio/flac' # flacfetch typically returns FLAC
)
logger.info(f"Uploaded audio to GCS: {audio_gcs_path}")
# Clean up temp file
try:
os.remove(result.filepath)
os.rmdir(temp_dir)
except Exception as e:
logger.warning(f"Failed to clean up temp files: {e}")
# Download to temp directory
temp_dir = tempfile.mkdtemp(prefix=f"audio_download_{job_id}_")
result = audio_search_service.download(
result_index=selection_index,
output_dir=temp_dir,
)
# Upload to GCS
filename = os.path.basename(result.filepath)
audio_gcs_path = f"uploads/{job_id}/audio/{filename}"
with open(result.filepath, 'rb') as f:
storage_service.upload_fileobj(
f,
audio_gcs_path,
content_type='audio/flac' # flacfetch typically returns FLAC
)
logger.info(f"Uploaded audio to GCS: {audio_gcs_path}")
# Clean up temp file
try:
os.remove(result.filepath)
os.rmdir(temp_dir)
except Exception as e:
logger.warning(f"Failed to clean up temp files: {e}")
# Update job with GCS path and transition to DOWNLOADING
job_manager.update_job(job_id, {
@@ -448,47 +263,32 @@ async def search_audio(
request: Request,
background_tasks: BackgroundTasks,
body: AudioSearchRequest,
auth_result: AuthResult = Depends(require_auth)
):
"""
Search for audio by artist and title, creating a new job.
This endpoint:
1. Creates a job in PENDING state
2. Searches for audio using flacfetch
3. Either returns search results for user selection, or
4. If auto_download=True, automatically selects best and starts processing
Use cases:
- Interactive mode (default): Returns results, user calls /select endpoint
- Auto mode (auto_download=True): Automatically selects and downloads best
"""
# Check tenant feature flag
tenant_config = get_tenant_config_from_request(request)
if tenant_config and not tenant_config.features.audio_search:
raise HTTPException(
status_code=403,
detail="Audio search is not available for this portal"
)
try:
# Apply default distribution settings
settings = get_settings()
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
# Apply defaults for YouTube/Dropbox distribution (for web service)
# Use explicit value if provided, otherwise fall back to server default
effective_enable_youtube_upload = body.enable_youtube_upload if body.enable_youtube_upload is not None else settings.default_enable_youtube_upload
effective_brand_prefix = body.brand_prefix or settings.default_brand_prefix
effective_youtube_description = body.youtube_description or settings.default_youtube_description
# Validate credentials if distribution services are requested
invalid_services = []
credential_manager = get_credential_manager()
if effective_enable_youtube_upload:
if body.enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
@@ -515,46 +315,19 @@ async def search_audio(
# Extract request metadata
request_metadata = extract_request_metadata(request, created_from="audio_search")
# Apply default theme if none specified
# This ensures all karaoke videos use the Nomad theme by default
effective_theme_id = body.theme_id
if effective_theme_id is None:
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if effective_theme_id:
logger.info(f"Applying default theme: {effective_theme_id}")
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = resolve_cdg_txt_defaults(
effective_theme_id, body.enable_cdg, body.enable_txt
)
# Use authenticated user's email
effective_user_email = auth_result.user_email
# Determine display values - use display_* if provided, otherwise fall back to search values
# Display values are used for title screens, filenames, YouTube, etc.
# Search values (body.artist, body.title) are used for audio search
effective_display_artist = body.display_artist.strip() if body.display_artist else body.artist
effective_display_title = body.display_title.strip() if body.display_title else body.title
# Create job
job_create = JobCreate(
artist=effective_display_artist, # Display value for title screens, filenames
title=effective_display_title, # Display value for title screens, filenames
theme_id=effective_theme_id,
color_overrides=body.color_overrides or {},
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
brand_prefix=effective_brand_prefix,
enable_youtube_upload=effective_enable_youtube_upload,
youtube_description=effective_youtube_description,
youtube_description_template=effective_youtube_description, # video_worker reads this field
artist=body.artist,
title=body.title,
enable_cdg=body.enable_cdg,
enable_txt=body.enable_txt,
brand_prefix=body.brand_prefix,
enable_youtube_upload=body.enable_youtube_upload,
youtube_description=body.youtube_description,
discord_webhook_url=effective_discord_webhook_url,
dropbox_path=effective_dropbox_path,
gdrive_folder_id=effective_gdrive_folder_id,
user_email=effective_user_email,
lyrics_artist=body.lyrics_artist,
lyrics_title=body.lyrics_title,
subtitle_offset_ms=body.subtitle_offset_ms,
@@ -565,13 +338,10 @@ async def search_audio(
audio_search_title=body.title,
auto_download=body.auto_download,
request_metadata=request_metadata,
non_interactive=body.non_interactive,
# Tenant scoping
tenant_id=tenant_config.id if tenant_config else None,
)
job = job_manager.create_job(job_create, is_admin=auth_result.is_admin)
job = job_manager.create_job(job_create)
job_id = job.job_id
logger.info(f"Created job {job_id} for audio search: {body.artist} - {body.title}")
# Update job with audio search fields
@@ -580,87 +350,6 @@ async def search_audio(
'audio_search_title': body.title,
'auto_download': body.auto_download,
})
# If theme is set and no custom style files are being uploaded, prepare theme style now
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
# can access the style configuration for preview videos
if effective_theme_id and not body.style_files:
from backend.api.routes.file_upload import _prepare_theme_for_job
try:
style_params_path, theme_style_assets, youtube_desc = _prepare_theme_for_job(
job_id, effective_theme_id, body.color_overrides
)
theme_update = {
'style_params_gcs_path': style_params_path,
'style_assets': theme_style_assets,
}
if youtube_desc and not effective_youtube_description:
theme_update['youtube_description_template'] = youtube_desc
job_manager.update_job(job_id, theme_update)
logger.info(f"Applied theme '{effective_theme_id}' to job {job_id}")
except Exception as e:
logger.warning(f"Failed to prepare theme '{effective_theme_id}' for job {job_id}: {e}")
# Continue without theme - job can still be processed with defaults
# Handle style file uploads if provided
style_upload_urls: List[StyleUploadUrl] = []
style_assets = {}
if body.style_files:
logger.info(f"Processing {len(body.style_files)} style file uploads for job {job_id}")
for file_info in body.style_files:
# Validate file type
if file_info.file_type not in STYLE_FILE_TYPES:
raise HTTPException(
status_code=400,
detail=f"Invalid file type '{file_info.file_type}'. Must be one of: {', '.join(STYLE_FILE_TYPES.keys())}"
)
# Validate extension
ext = Path(file_info.filename).suffix.lower()
allowed_exts = STYLE_FILE_TYPES[file_info.file_type]
if ext not in allowed_exts:
raise HTTPException(
status_code=400,
detail=f"Invalid extension '{ext}' for {file_info.file_type}. Allowed: {', '.join(allowed_exts)}"
)
# Generate GCS path
if file_info.file_type == 'style_params':
gcs_path = f"uploads/{job_id}/style/style_params.json"
else:
# style_intro_background -> intro_background, etc.
asset_key = file_info.file_type.replace('style_', '')
gcs_path = f"uploads/{job_id}/style/{asset_key}{ext}"
# Generate signed upload URL
signed_url = storage_service.generate_signed_upload_url(
gcs_path,
content_type=file_info.content_type,
expiration_minutes=60
)
style_upload_urls.append(StyleUploadUrl(
file_type=file_info.file_type,
gcs_path=gcs_path,
upload_url=signed_url
))
# Track the expected asset paths
if file_info.file_type == 'style_params':
style_assets['style_params'] = gcs_path
else:
asset_key = file_info.file_type.replace('style_', '')
style_assets[asset_key] = gcs_path
# Update job with style asset expectations
style_update = {'style_assets': style_assets}
if 'style_params' in style_assets:
style_update['style_params_gcs_path'] = style_assets['style_params']
job_manager.update_job(job_id, style_update)
logger.info(f"Generated {len(style_upload_urls)} style upload URLs for job {job_id}")
# Transition to searching state
job_manager.transition_to_state(
@@ -689,17 +378,13 @@ async def search_audio(
job_manager.fail_job(job_id, f"Audio search failed: {e}")
raise HTTPException(status_code=500, detail=f"Search failed: {e}")
# Store results in job state_data, including remote_search_id if available
# Store results in job state_data
results_dicts = [r.to_dict() for r in search_results]
state_data_update = {
'audio_search_results': results_dicts,
'audio_search_count': len(results_dicts),
}
# Store remote_search_id for use during download (important for concurrent requests)
if audio_search_service.last_remote_search_id:
state_data_update['remote_search_id'] = audio_search_service.last_remote_search_id
job_manager.update_job(job_id, {
'state_data': state_data_update
'state_data': {
'audio_search_results': results_dicts,
'audio_search_count': len(results_dicts),
}
})
# If auto_download, select best and start processing
@@ -723,7 +408,6 @@ async def search_audio(
results_count=len(search_results),
auto_download=True,
server_version=VERSION,
style_upload_urls=style_upload_urls if style_upload_urls else None,
)
# Interactive mode: return results for user selection
@@ -734,59 +418,20 @@ async def search_audio(
message=f"Found {len(search_results)} audio sources. Waiting for selection."
)
# Convert to response format with full Release data for rich display
result_responses = []
for r in search_results:
# Get full serialized data from raw_result if available
# raw_result can be either:
# - A dict (from remote flacfetch API)
# - A Release object (from local flacfetch)
raw_dict = {}
if r.raw_result:
if isinstance(r.raw_result, dict):
# Remote flacfetch API returns dicts directly
raw_dict = r.raw_result
else:
# Local flacfetch returns Release objects
try:
raw_dict = r.raw_result.to_dict()
except AttributeError:
pass # Not a Release object
result_responses.append(
AudioSearchResultResponse(
index=r.index,
title=r.title,
artist=r.artist,
provider=r.provider,
url=r.url,
duration=r.duration,
quality=r.quality,
source_id=r.source_id,
seeders=raw_dict.get('seeders') or r.seeders,
target_file=raw_dict.get('target_file') or r.target_file,
# Additional fields for rich display
year=raw_dict.get('year'),
label=raw_dict.get('label'),
edition_info=raw_dict.get('edition_info'),
release_type=raw_dict.get('release_type'),
channel=raw_dict.get('channel'),
view_count=raw_dict.get('view_count'),
size_bytes=raw_dict.get('size_bytes'),
target_file_size=raw_dict.get('target_file_size'),
track_pattern=raw_dict.get('track_pattern'),
match_score=raw_dict.get('match_score'),
# Pre-computed display fields
formatted_size=raw_dict.get('formatted_size'),
formatted_duration=raw_dict.get('formatted_duration'),
formatted_views=raw_dict.get('formatted_views'),
is_lossless=raw_dict.get('is_lossless'),
quality_str=raw_dict.get('quality_str'),
# Full quality object for Release.from_dict()
# Remote API uses 'quality_data', local uses 'quality' for the dict
quality_data=raw_dict.get('quality_data') or raw_dict.get('quality'),
)
# Convert to response format
result_responses = [
AudioSearchResultResponse(
index=r.index,
title=r.title,
artist=r.artist,
provider=r.provider,
url=r.url,
duration=r.duration,
quality=r.quality,
source_id=r.source_id,
)
for r in search_results
]
return AudioSearchResponse(
status="awaiting_selection",
@@ -796,7 +441,6 @@ async def search_audio(
results_count=len(search_results),
auto_download=False,
server_version=VERSION,
style_upload_urls=style_upload_urls if style_upload_urls else None,
)
except HTTPException:
@@ -807,10 +451,7 @@ async def search_audio(
@router.get("/audio-search/{job_id}/results")
async def get_audio_search_results(
job_id: str,
auth_result: AuthResult = Depends(require_auth)
):
async def get_audio_search_results(job_id: str):
"""
Get audio search results for a job.
@@ -832,8 +473,6 @@ async def get_audio_search_results(
"status": "success",
"job_id": job_id,
"job_status": job.status,
"artist": job.audio_search_artist or job.artist,
"title": job.audio_search_title or job.title,
"results": search_results,
"results_count": len(search_results),
}
@@ -844,7 +483,6 @@ async def select_audio_source(
job_id: str,
background_tasks: BackgroundTasks,
body: AudioSelectRequest,
auth_result: AuthResult = Depends(require_auth)
):
"""
Select an audio source and start job processing.
@@ -866,16 +504,25 @@ async def select_audio_source(
detail=f"Job is not awaiting audio selection (status: {job.status})"
)
# Get search service instance
# Note: With download_by_id, we no longer need to re-search to populate the cache.
# The source_id stored in job.state_data['audio_search_results'] is sufficient.
# Re-instantiate the audio search service (it caches raw results from search)
# For production, we'd need to re-search or store raw results differently
audio_search_service = get_audio_search_service()
# Validate search results exist in job state_data
# Re-run search to populate the cache
# This is necessary because the service caches raw results in memory
search_results = job.state_data.get('audio_search_results', [])
if not search_results:
raise HTTPException(status_code=400, detail="No search results cached for this job")
artist = job.audio_search_artist or job.artist
title = job.audio_search_title or job.title
try:
# Re-search to populate cache
audio_search_service.search(artist, title)
except Exception as e:
logger.warning(f"Re-search failed, trying direct download: {e}")
selection_info = await _download_and_start_processing(
job_id=job_id,
selection_index=body.selection_index,
File diff suppressed because it is too large. Load diff
+1 -384
View File
@@ -1,174 +1,10 @@
"""
Health check routes.
"""
import os
import logging
import subprocess
import platform
from fastapi import APIRouter
from typing import Dict, Any, Optional
from backend.version import VERSION
from backend.services.flacfetch_client import get_flacfetch_client
from backend.services.email_service import get_email_service
from backend.services.stripe_service import get_stripe_service
from backend.services.encoding_service import get_encoding_service
from backend.services.spacy_preloader import get_preloaded_model, is_model_preloaded
from backend.services.nltk_preloader import get_preloaded_cmudict, is_cmudict_preloaded
from backend.services.langfuse_preloader import get_preloaded_langfuse_handler, is_langfuse_preloaded, is_langfuse_configured
from typing import Dict
router = APIRouter()
logger = logging.getLogger(__name__)
def get_system_info() -> Dict[str, Any]:
"""Get system information for performance analysis."""
info = {
"platform": platform.system(),
"platform_release": platform.release(),
"python_version": platform.python_version(),
}
# Get CPU info
try:
cpu_count = os.cpu_count() or 0
info["cpu_count"] = cpu_count
# Try to get more detailed CPU info on Linux
if platform.system() == "Linux":
try:
with open("/proc/cpuinfo", "r") as f:
for line in f:
if "model name" in line:
info["cpu_model"] = line.split(":")[1].strip()
break
except Exception:
pass
except Exception as e:
info["cpu_error"] = str(e)
# Get memory info
try:
if platform.system() == "Linux":
with open("/proc/meminfo", "r") as f:
for line in f:
if line.startswith("MemTotal:"):
mem_kb = int(line.split()[1])
info["memory_gb"] = round(mem_kb / (1024 * 1024), 1)
break
except Exception as e:
info["memory_error"] = str(e)
return info
def get_ffmpeg_info() -> Dict[str, Any]:
"""Get FFmpeg version and configuration for debugging encoding performance."""
try:
# Get FFmpeg version
result = subprocess.run(
["ffmpeg", "-version"],
capture_output=True,
text=True,
timeout=10
)
if result.returncode == 0:
lines = result.stdout.split("\n")
version_line = lines[0] if lines else "unknown"
# Check for hardware acceleration support
hw_accel = {}
config_result = subprocess.run(
["ffmpeg", "-hwaccels"],
capture_output=True,
text=True,
timeout=10
)
if config_result.returncode == 0:
hw_lines = config_result.stdout.strip().split("\n")
hw_accel["available"] = [h.strip() for h in hw_lines[1:] if h.strip()]
# Check encoder availability
encoders = {}
encoder_result = subprocess.run(
["ffmpeg", "-encoders"],
capture_output=True,
text=True,
timeout=10
)
if encoder_result.returncode == 0:
encoder_output = encoder_result.stdout
# Check for specific encoders we care about
encoders["libx264"] = "libx264" in encoder_output
encoders["h264_nvenc"] = "h264_nvenc" in encoder_output
encoders["h264_vaapi"] = "h264_vaapi" in encoder_output
encoders["hevc_nvenc"] = "hevc_nvenc" in encoder_output
return {
"available": True,
"version": version_line,
"hw_acceleration": hw_accel,
"encoders": encoders,
}
else:
return {
"available": False,
"error": result.stderr[:500] if result.stderr else "Unknown error",
}
except subprocess.TimeoutExpired:
return {"available": False, "error": "Timeout getting FFmpeg info"}
except FileNotFoundError:
return {"available": False, "error": "FFmpeg not found"}
except Exception as e:
return {"available": False, "error": str(e)}
def check_transmission_status() -> Dict[str, Any]:
"""Check if Transmission daemon is available and responsive."""
try:
import transmission_rpc
host = os.environ.get("TRANSMISSION_HOST", "localhost")
port = int(os.environ.get("TRANSMISSION_PORT", "9091"))
client = transmission_rpc.Client(host=host, port=port, timeout=5)
stats = client.session_stats()
# Get detailed torrent info
torrents = client.get_torrents()
torrent_details = []
for t in torrents:
torrent_info = {
"name": t.name,
"progress": round(t.progress, 1),
"status": str(t.status),
"peers": t.peers_connected if hasattr(t, 'peers_connected') else 0,
"download_speed": round(t.rate_download / 1024, 1) if hasattr(t, 'rate_download') else 0, # KB/s
}
# Check if stalled (downloading but no progress and no peers)
if t.status.downloading and t.progress < 100 and torrent_info['peers'] == 0:
torrent_info['stalled'] = True
torrent_details.append(torrent_info)
return {
"available": True,
"host": host,
"port": port,
"download_dir": stats.download_dir if hasattr(stats, 'download_dir') else None,
"active_torrent_count": stats.active_torrent_count if hasattr(stats, 'active_torrent_count') else 0,
"torrents": torrent_details,
}
except ImportError as e:
return {
"available": False,
"error": f"transmission_rpc not installed: {e}",
}
except Exception as e:
return {
"available": False,
"host": os.environ.get("TRANSMISSION_HOST", "localhost"),
"port": int(os.environ.get("TRANSMISSION_PORT", "9091")),
"error": str(e),
}
@router.get("/health")
@@ -180,225 +16,6 @@ async def health_check() -> Dict[str, str]:
}
async def check_flacfetch_service_status() -> Dict[str, Any]:
"""Check if remote flacfetch service is available and healthy."""
client = get_flacfetch_client()
if not client:
return {
"configured": False,
"message": "Remote flacfetch service not configured (FLACFETCH_API_URL not set)",
}
try:
health = await client.health_check()
return {
"configured": True,
"available": True,
"status": health.get("status"),
"version": health.get("version"),
"transmission": health.get("transmission", {}),
"disk": health.get("disk", {}),
"providers": health.get("providers", {}),
}
except Exception as e:
return {
"configured": True,
"available": False,
"error": str(e),
}
async def check_encoding_worker_status() -> Dict[str, Any]:
"""Check if GCE encoding worker is available and healthy."""
encoding_service = get_encoding_service()
if not encoding_service.is_configured:
return {
"configured": False,
"enabled": encoding_service.settings.use_gce_encoding,
"message": "GCE encoding worker not configured (ENCODING_WORKER_URL or API key not set)",
}
try:
health = await encoding_service.health_check()
return {
"configured": True,
"enabled": encoding_service.is_enabled,
"available": health.get("status") == "ok",
"status": health.get("status"),
"active_jobs": health.get("active_jobs", 0),
"queue_length": health.get("queue_length", 0),
"ffmpeg_version": health.get("ffmpeg_version"),
"wheel_version": health.get("wheel_version"),
}
except Exception as e:
return {
"configured": True,
"enabled": encoding_service.is_enabled,
"available": False,
"error": str(e),
}
@router.get("/health/encoding-worker")
async def encoding_worker_health() -> Dict[str, Any]:
"""
Lightweight endpoint to check encoding worker status.
Returns minimal info for frontend footer display.
No authentication required.
"""
status = await check_encoding_worker_status()
# Return simplified response for frontend
if not status.get("configured"):
return {
"available": False,
"status": "not_configured",
}
if not status.get("available"):
return {
"available": False,
"status": "offline",
"error": status.get("error"),
}
return {
"available": True,
"status": "ok",
"version": status.get("wheel_version"),
"active_jobs": status.get("active_jobs", 0),
"queue_length": status.get("queue_length", 0),
}
@router.get("/health/detailed")
async def detailed_health_check() -> Dict[str, Any]:
"""
Detailed health check including dependencies.
Use this to debug issues with Transmission, flacfetch service, etc.
"""
transmission_status = check_transmission_status()
flacfetch_status = await check_flacfetch_service_status()
encoding_status = await check_encoding_worker_status()
# Check email service
email_service = get_email_service()
email_status = {
"configured": email_service.is_configured(),
"provider": type(email_service.provider).__name__,
}
# Check Stripe service
stripe_service = get_stripe_service()
stripe_status = {
"configured": stripe_service.is_configured(),
}
# Log for debugging
if not transmission_status.get("available"):
logger.warning(f"Local Transmission not available: {transmission_status.get('error')}")
else:
logger.info(f"Local Transmission available at {transmission_status.get('host')}:{transmission_status.get('port')}")
if flacfetch_status.get("configured") and not flacfetch_status.get("available"):
logger.warning(f"Remote flacfetch service not available: {flacfetch_status.get('error')}")
elif flacfetch_status.get("available"):
logger.info(f"Remote flacfetch service healthy: {flacfetch_status.get('status')}")
if not email_status["configured"]:
logger.warning("Email service not configured - magic links will not work")
if not stripe_status["configured"]:
logger.warning("Stripe service not configured - payments will not work")
# Get system and FFmpeg info for performance analysis
system_info = get_system_info()
ffmpeg_info = get_ffmpeg_info()
return {
"status": "healthy",
"service": "karaoke-gen-backend",
"version": VERSION,
"system": system_info,
"ffmpeg": ffmpeg_info,
"dependencies": {
"transmission_local": transmission_status,
"flacfetch_remote": flacfetch_status,
"encoding_worker": encoding_status,
},
"services": {
"email": email_status,
"stripe": stripe_status,
}
}
@router.get("/health/preload-status")
async def preload_status() -> Dict[str, Any]:
"""
Check status of preloaded resources for performance optimization.
Use this endpoint to verify that NLTK, SpaCy, and Langfuse resources
were successfully preloaded at container startup. If any show as
not preloaded, check Cloud Run startup logs for errors.
Expected state after successful deployment:
- spacy.preloaded: true
- nltk.preloaded: true
- langfuse.preloaded: true (if configured) or configured: false
"""
# SpaCy status
spacy_model = get_preloaded_model("en_core_web_sm")
spacy_status = {
"preloaded": is_model_preloaded("en_core_web_sm"),
"model": "en_core_web_sm",
}
if spacy_model:
spacy_status["vocab_size"] = len(spacy_model.vocab)
# NLTK status
cmudict = get_preloaded_cmudict()
nltk_status = {
"preloaded": is_cmudict_preloaded(),
"resource": "cmudict",
}
if cmudict:
nltk_status["entries"] = len(cmudict)
# Langfuse status
langfuse_handler = get_preloaded_langfuse_handler()
langfuse_status = {
"configured": is_langfuse_configured(),
"preloaded": is_langfuse_preloaded(),
}
if langfuse_handler:
langfuse_status["handler_type"] = type(langfuse_handler).__name__
# Overall status
all_preloaded = (
spacy_status["preloaded"]
and nltk_status["preloaded"]
and (langfuse_status["preloaded"] or not langfuse_status["configured"])
)
return {
"status": "ok" if all_preloaded else "degraded",
"message": "All resources preloaded" if all_preloaded else "Some resources not preloaded - check startup logs",
"spacy": spacy_status,
"nltk": nltk_status,
"langfuse": langfuse_status,
"performance_impact": {
"spacy_preload": "Saves ~60s on first lyrics correction",
"nltk_preload": "Saves ~100-150s on SyllablesMatchHandler init",
"langfuse_preload": "Saves ~200s on AgenticCorrector init",
}
}
@router.get("/readiness")
async def readiness_check() -> Dict[str, str]:
"""Readiness check endpoint for Cloud Run."""
+10 -178
View File
@@ -6,17 +6,11 @@ They are protected by admin authentication.
With Cloud Tasks integration, these endpoints may be called multiple times
(retry on failure). Idempotency checks prevent duplicate processing.
Observability:
- Extracts trace context from incoming requests (propagated via Cloud Tasks)
- Creates worker spans linked to the original request trace
- All logs include job_id for easy filtering in Cloud Logging
"""
import logging
import asyncio
import time
from typing import Tuple, Optional
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Request
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends
from pydantic import BaseModel
from backend.workers.audio_worker import process_audio_separation
@@ -27,12 +21,6 @@ from backend.workers.render_video_worker import process_render_video
from backend.api.dependencies import require_admin
from backend.services.auth_service import UserType
from backend.services.job_manager import JobManager
from backend.services.tracing import (
extract_trace_context,
start_span_with_context,
add_span_attribute,
add_span_event,
)
logger = logging.getLogger(__name__)
@@ -69,7 +57,7 @@ def _check_worker_idempotency(job_id: str, worker_name: str) -> Optional[WorkerR
job = job_manager.get_job(job_id)
if not job:
logger.warning(f"[job:{job_id}] Job not found for {worker_name} worker")
logger.warning(f"Job {job_id} not found for {worker_name} worker")
return WorkerResponse(
status="not_found",
job_id=job_id,
@@ -82,7 +70,7 @@ def _check_worker_idempotency(job_id: str, worker_name: str) -> Optional[WorkerR
stage = worker_progress.get('stage')
if stage == 'running':
logger.info(f"[job:{job_id}] {worker_name.capitalize()} worker already running, skipping")
logger.info(f"{worker_name.capitalize()} worker already running for job {job_id}, skipping")
return WorkerResponse(
status="already_running",
job_id=job_id,
@@ -90,7 +78,7 @@ def _check_worker_idempotency(job_id: str, worker_name: str) -> Optional[WorkerR
)
if stage == 'complete':
logger.info(f"[job:{job_id}] {worker_name.capitalize()} worker already complete, skipping")
logger.info(f"{worker_name.capitalize()} worker already complete for job {job_id}, skipping")
return WorkerResponse(
status="already_complete",
job_id=job_id,
@@ -105,7 +93,6 @@ def _check_worker_idempotency(job_id: str, worker_name: str) -> Optional[WorkerR
@router.post("/workers/audio", response_model=WorkerResponse)
async def trigger_audio_worker(
request: WorkerRequest,
http_request: Request,
background_tasks: BackgroundTasks,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
@@ -120,18 +107,11 @@ async def trigger_audio_worker(
The worker runs in the background and updates job state as it progresses.
"""
job_id = request.job_id
# Extract trace context from incoming request (propagated via Cloud Tasks)
trace_context = extract_trace_context(dict(http_request.headers))
logger.info(f"[job:{job_id}] WORKER_TRIGGER worker=audio")
add_span_attribute("job_id", job_id)
add_span_attribute("worker", "audio")
logger.info(f"Triggering audio worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "audio")
if skip_response:
add_span_event("worker_skipped", {"reason": skip_response.status})
return skip_response
# Add task to background tasks
@@ -139,7 +119,6 @@ async def trigger_audio_worker(
# while the worker continues processing
background_tasks.add_task(process_audio_separation, job_id)
add_span_event("worker_started")
return WorkerResponse(
status="started",
job_id=job_id,
@@ -150,7 +129,6 @@ async def trigger_audio_worker(
@router.post("/workers/lyrics", response_model=WorkerResponse)
async def trigger_lyrics_worker(
request: WorkerRequest,
http_request: Request,
background_tasks: BackgroundTasks,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
@@ -165,24 +143,16 @@ async def trigger_lyrics_worker(
The worker runs in the background and updates job state as it progresses.
"""
job_id = request.job_id
# Extract trace context from incoming request
trace_context = extract_trace_context(dict(http_request.headers))
logger.info(f"[job:{job_id}] WORKER_TRIGGER worker=lyrics")
add_span_attribute("job_id", job_id)
add_span_attribute("worker", "lyrics")
logger.info(f"Triggering lyrics worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "lyrics")
if skip_response:
add_span_event("worker_skipped", {"reason": skip_response.status})
return skip_response
# Add task to background tasks
background_tasks.add_task(process_lyrics_transcription, job_id)
add_span_event("worker_started")
return WorkerResponse(
status="started",
job_id=job_id,
@@ -193,7 +163,6 @@ async def trigger_lyrics_worker(
@router.post("/workers/screens", response_model=WorkerResponse)
async def trigger_screens_worker(
request: WorkerRequest,
http_request: Request,
background_tasks: BackgroundTasks,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
@@ -205,24 +174,16 @@ async def trigger_screens_worker(
Idempotency: If worker is already running or complete, returns early.
"""
job_id = request.job_id
# Extract trace context from incoming request
trace_context = extract_trace_context(dict(http_request.headers))
logger.info(f"[job:{job_id}] WORKER_TRIGGER worker=screens")
add_span_attribute("job_id", job_id)
add_span_attribute("worker", "screens")
logger.info(f"Triggering screens worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "screens")
if skip_response:
add_span_event("worker_skipped", {"reason": skip_response.status})
return skip_response
# Add task to background tasks
background_tasks.add_task(generate_screens, job_id)
add_span_event("worker_started")
return WorkerResponse(
status="started",
job_id=job_id,
@@ -233,7 +194,6 @@ async def trigger_screens_worker(
@router.post("/workers/video", response_model=WorkerResponse)
async def trigger_video_worker(
request: WorkerRequest,
http_request: Request,
background_tasks: BackgroundTasks,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
@@ -246,24 +206,16 @@ async def trigger_video_worker(
Idempotency: If worker is already running or complete, returns early.
"""
job_id = request.job_id
# Extract trace context from incoming request
trace_context = extract_trace_context(dict(http_request.headers))
logger.info(f"[job:{job_id}] WORKER_TRIGGER worker=video")
add_span_attribute("job_id", job_id)
add_span_attribute("worker", "video")
logger.info(f"Triggering video worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "video")
if skip_response:
add_span_event("worker_skipped", {"reason": skip_response.status})
return skip_response
# Add task to background tasks
background_tasks.add_task(generate_video, job_id)
add_span_event("worker_started")
return WorkerResponse(
status="started",
job_id=job_id,
@@ -274,7 +226,6 @@ async def trigger_video_worker(
@router.post("/workers/render-video", response_model=WorkerResponse)
async def trigger_render_video_worker(
request: WorkerRequest,
http_request: Request,
background_tasks: BackgroundTasks,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
@@ -291,24 +242,16 @@ async def trigger_render_video_worker(
Next state: AWAITING_INSTRUMENTAL_SELECTION
"""
job_id = request.job_id
# Extract trace context from incoming request
trace_context = extract_trace_context(dict(http_request.headers))
logger.info(f"[job:{job_id}] WORKER_TRIGGER worker=render-video")
add_span_attribute("job_id", job_id)
add_span_attribute("worker", "render-video")
logger.info(f"Triggering render-video worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "render")
if skip_response:
add_span_event("worker_skipped", {"reason": skip_response.status})
return skip_response
# Add task to background tasks
background_tasks.add_task(process_render_video, job_id)
add_span_event("worker_started")
return WorkerResponse(
status="started",
job_id=job_id,
@@ -316,124 +259,13 @@ async def trigger_render_video_worker(
)
@router.post("/jobs/{job_id}/check-idle-reminder")
async def check_idle_reminder(
job_id: str,
http_request: Request,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
"""
Check if a job needs an idle reminder email.
This endpoint is called by a Cloud Tasks scheduled task 5 minutes after
a job enters a blocking state (AWAITING_REVIEW or AWAITING_INSTRUMENTAL_SELECTION).
If the job is still in the blocking state and no reminder has been sent yet,
sends a reminder email to the user.
Idempotency: Only one reminder per job (tracked via reminder_sent flag).
"""
from backend.models.job import JobStatus
from backend.services.job_notification_service import get_job_notification_service
# Extract trace context from incoming request
trace_context = extract_trace_context(dict(http_request.headers))
logger.info(f"[job:{job_id}] IDLE_REMINDER_CHECK starting")
add_span_attribute("job_id", job_id)
add_span_attribute("operation", "idle_reminder_check")
job_manager = JobManager()
job = job_manager.get_job(job_id)
if not job:
logger.warning(f"[job:{job_id}] Job not found for idle reminder check")
add_span_event("job_not_found")
return {"status": "not_found", "job_id": job_id, "message": "Job not found"}
# Check if job is still in a blocking state
blocking_states = [JobStatus.AWAITING_REVIEW, JobStatus.AWAITING_INSTRUMENTAL_SELECTION]
if job.status not in [s.value for s in blocking_states]:
logger.info(f"[job:{job_id}] Job no longer in blocking state ({job.status}), skipping reminder")
add_span_event("not_blocking", {"current_status": job.status})
return {
"status": "skipped",
"job_id": job_id,
"message": f"Job not in blocking state (current: {job.status})"
}
# Normalize state_data to prevent None errors
state_data = job.state_data or {}
# Check if reminder was already sent (idempotency)
if state_data.get('reminder_sent'):
logger.info(f"[job:{job_id}] Reminder already sent, skipping")
add_span_event("already_sent")
return {"status": "already_sent", "job_id": job_id, "message": "Reminder already sent"}
# Skip reminders for made-for-you jobs (admin handles these directly, no intermediate customer emails)
if getattr(job, 'made_for_you', False):
logger.info(f"[job:{job_id}] Made-for-you job, skipping customer reminder (admin handles)")
add_span_event("made_for_you_skip")
return {"status": "skipped", "job_id": job_id, "message": "Made-for-you job - admin handles directly"}
# Check if user has an email
if not job.user_email:
logger.warning(f"[job:{job_id}] No user email, cannot send reminder")
add_span_event("no_email")
return {"status": "no_email", "job_id": job_id, "message": "No user email configured"}
# Determine action type
action_type = state_data.get('blocking_action_type')
if not action_type:
action_type = "lyrics" if job.status == JobStatus.AWAITING_REVIEW.value else "instrumental"
# Send the reminder email
try:
notification_service = get_job_notification_service()
success = await notification_service.send_action_reminder_email(
job_id=job.job_id,
user_email=job.user_email,
action_type=action_type,
user_name=None, # Could fetch from user service if needed
artist=job.artist,
title=job.title,
audio_hash=job.audio_hash,
review_token=job.review_token,
instrumental_token=job.instrumental_token,
)
if success:
# Mark reminder as sent (prevents duplicate sends)
job_manager.firestore.update_job(job_id, {
'state_data': {**state_data, 'reminder_sent': True}
})
logger.info(f"[job:{job_id}] Sent {action_type} reminder email to {job.user_email}")
add_span_event("reminder_sent", {"action_type": action_type})
return {
"status": "sent",
"job_id": job_id,
"message": f"Sent {action_type} reminder to {job.user_email}"
}
else:
logger.error(f"[job:{job_id}] Failed to send reminder email")
add_span_event("send_failed")
return {"status": "failed", "job_id": job_id, "message": "Failed to send reminder"}
except Exception as e:
logger.exception(f"[job:{job_id}] Error sending reminder: {e}")
add_span_event("error", {"error": str(e)})
return {"status": "error", "job_id": job_id, "message": str(e)}
@router.get("/health")
async def internal_health(
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
"""
Internal health check endpoint.
Used to verify the internal API is responsive.
Requires admin authentication.
"""
+86 -503
View File
@@ -11,7 +11,7 @@ import asyncio
import logging
import httpx
from typing import List, Optional, Dict, Any, Tuple
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Request
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends
from backend.models.job import Job, JobCreate, JobResponse, JobStatus
from backend.models.requests import (
@@ -25,13 +25,9 @@ from backend.models.requests import (
from backend.services.job_manager import JobManager
from backend.services.worker_service import get_worker_service
from backend.services.storage_service import StorageService
from backend.services.theme_service import get_theme_service
from backend.config import get_settings
from backend.api.dependencies import require_admin, require_auth, require_instrumental_auth
from backend.services.auth_service import UserType, AuthResult
from backend.services.metrics import metrics
from backend.middleware.tenant import get_tenant_from_request
from backend.utils.test_data import is_test_email
from backend.api.dependencies import require_admin
from backend.services.auth_service import UserType
logger = logging.getLogger(__name__)
@@ -59,8 +55,7 @@ async def _trigger_workers_parallel(job_id: str) -> None:
@router.post("", response_model=JobResponse)
async def create_job(
request: URLSubmissionRequest,
background_tasks: BackgroundTasks,
auth_result: AuthResult = Depends(require_auth)
background_tasks: BackgroundTasks
) -> JobResponse:
"""
Create a new karaoke generation job from a URL.
@@ -72,56 +67,19 @@ async def create_job(
4. When both complete, job transitions to AWAITING_REVIEW
"""
try:
# Determine job owner email:
# All authentication methods must provide a user_email for job ownership
if auth_result.user_email:
# Use authenticated user's email (standard case)
user_email = auth_result.user_email
else:
# This should never happen - all auth methods now require user_email
logger.error("Authentication succeeded but no user_email provided")
raise HTTPException(
status_code=500,
detail="Authentication error: no user identity available"
)
# Admins can optionally create jobs on behalf of other users
if request.user_email and auth_result.is_admin and request.user_email != auth_result.user_email:
user_email = request.user_email
logger.info(f"Admin {auth_result.user_email} creating job on behalf of {user_email}")
# Apply YouTube upload default from settings
# Use explicit value if provided, otherwise fall back to server default
settings = get_settings()
effective_enable_youtube_upload = request.enable_youtube_upload if request.enable_youtube_upload is not None else settings.default_enable_youtube_upload
# Apply default theme - all jobs require a theme
theme_service = get_theme_service()
effective_theme_id = theme_service.get_default_theme_id()
if not effective_theme_id:
raise HTTPException(
status_code=422,
detail="No default theme configured. Please contact support or specify a theme_id."
)
logger.info(f"Applying default theme: {effective_theme_id}")
# Create job with all preferences
job_create = JobCreate(
url=str(request.url),
artist=request.artist,
title=request.title,
theme_id=effective_theme_id, # Required - all jobs must have a theme
enable_cdg=request.enable_cdg,
enable_txt=request.enable_txt,
enable_youtube_upload=effective_enable_youtube_upload,
enable_youtube_upload=request.enable_youtube_upload,
youtube_description=request.youtube_description,
webhook_url=request.webhook_url,
user_email=user_email
user_email=request.user_email
)
job = job_manager.create_job(job_create, is_admin=auth_result.is_admin)
# Record job creation metric
metrics.record_job_created(job.job_id, source="url")
job = job_manager.create_job(job_create)
# Trigger both workers in parallel using asyncio.gather
# (FastAPI's BackgroundTasks runs async tasks sequentially)
@@ -144,132 +102,69 @@ async def create_job(
@router.get("/{job_id}", response_model=Job)
async def get_job(
job_id: str,
auth_result: AuthResult = Depends(require_auth)
) -> Job:
async def get_job(job_id: str) -> Job:
"""Get job status and details."""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership - users can only see their own jobs, admins can see all
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
# If job is complete, include download URLs
if job.status == JobStatus.COMPLETE:
job.download_urls = job_manager.get_output_urls(job_id)
return job
def _check_job_ownership(job: Job, auth_result: AuthResult) -> bool:
"""
Check if the authenticated user owns the job or has admin access.
Returns:
True if user can access the job, False otherwise
"""
# Admins can access all jobs
if auth_result.is_admin:
return True
# Check if user owns the job
if auth_result.user_email and job.user_email:
return auth_result.user_email.lower() == job.user_email.lower()
# If no user_email on auth (token auth without email), deny access to jobs with user_email
# This prevents token-based auth from accessing user jobs
if job.user_email:
return False
# Legacy jobs without user_email - allow access for backward compatibility
# TODO: Consider restricting this in the future
return True
@router.get("", response_model=List[Job])
async def list_jobs(
request: Request,
status: Optional[JobStatus] = None,
environment: Optional[str] = None,
client_id: Optional[str] = None,
created_after: Optional[str] = None,
created_before: Optional[str] = None,
exclude_test: bool = True,
limit: int = 100,
auth_result: AuthResult = Depends(require_auth)
limit: int = 100
) -> List[Job]:
"""
List jobs with optional filters.
Regular users only see their own jobs. Admins see all jobs.
Users on tenant portals only see jobs from their tenant.
Args:
status: Filter by job status (pending, complete, failed, etc.)
environment: Filter by request_metadata.environment (test/production/development)
client_id: Filter by request_metadata.client_id (customer identifier)
created_after: Filter jobs created after this ISO datetime (e.g., 2024-01-01T00:00:00Z)
created_before: Filter jobs created before this ISO datetime
exclude_test: If True (default), exclude jobs from test users (admin only)
limit: Maximum number of jobs to return (default 100)
Returns:
List of jobs matching filters, ordered by created_at descending
"""
from datetime import datetime
try:
# Parse datetime strings if provided
created_after_dt = None
created_before_dt = None
if created_after:
try:
created_after_dt = datetime.fromisoformat(created_after.replace('Z', '+00:00'))
except ValueError as e:
raise HTTPException(status_code=400, detail=f"Invalid created_after format: {created_after}") from e
except ValueError:
raise HTTPException(status_code=400, detail=f"Invalid created_after format: {created_after}")
if created_before:
try:
created_before_dt = datetime.fromisoformat(created_before.replace('Z', '+00:00'))
except ValueError as e:
raise HTTPException(status_code=400, detail=f"Invalid created_before format: {created_before}") from e
# Determine user_email filter based on admin status
# Admins see all jobs, regular users only see their own
user_email_filter = None
if not auth_result.is_admin:
if auth_result.user_email:
user_email_filter = auth_result.user_email
logger.debug(f"Filtering jobs for user: {user_email_filter}")
else:
# Token-based auth without user email - show no jobs for security
logger.warning("Non-admin auth without user_email, returning empty job list")
return []
# Get tenant_id from request for portal scoping
# Tenant users only see jobs from their tenant
tenant_id = get_tenant_from_request(request)
except ValueError:
raise HTTPException(status_code=400, detail=f"Invalid created_before format: {created_before}")
jobs = job_manager.list_jobs(
status=status,
environment=environment,
client_id=client_id,
created_after=created_after_dt,
created_before=created_before_dt,
user_email=user_email_filter,
tenant_id=tenant_id,
limit=limit
)
# Filter out test user jobs if exclude_test is True (admin only)
if exclude_test and auth_result.is_admin:
jobs = [j for j in jobs if not is_test_email(j.user_email or "")]
logger.debug(f"Listed {len(jobs)} jobs for user={auth_result.user_email}, admin={auth_result.is_admin}")
return jobs
except HTTPException:
raise
@@ -279,23 +174,15 @@ async def list_jobs(
@router.delete("/{job_id}")
async def delete_job(
job_id: str,
delete_files: bool = True,
auth_result: AuthResult = Depends(require_auth)
) -> dict:
async def delete_job(job_id: str, delete_files: bool = True) -> dict:
"""Delete a job and optionally its output files."""
try:
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership - users can only delete their own jobs
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to delete this job")
job_manager.delete_job(job_id, delete_files=delete_files)
return {"status": "success", "message": f"Job {job_id} deleted"}
except HTTPException:
raise
@@ -312,7 +199,7 @@ async def bulk_delete_jobs(
created_before: Optional[str] = None,
delete_files: bool = True,
confirm: bool = False,
_auth_result: AuthResult = Depends(require_admin)
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
) -> dict:
"""
Delete multiple jobs matching filter criteria.
@@ -414,24 +301,17 @@ async def bulk_delete_jobs(
# ============================================================================
@router.get("/{job_id}/review-data")
async def get_review_data(
job_id: str,
auth_result: AuthResult = Depends(require_auth)
) -> Dict[str, Any]:
async def get_review_data(job_id: str) -> Dict[str, Any]:
"""
Get data needed for lyrics review interface.
Returns corrections JSON URL and audio URL.
Frontend loads these to render the review UI.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
raise HTTPException(
status_code=400,
@@ -440,27 +320,12 @@ async def get_review_data(
# Get URLs from file_urls
corrections_url = job.file_urls.get('lyrics', {}).get('corrections')
# For audio, try multiple sources in order of preference:
# 1. Explicit lyrics audio (if worker uploaded it)
# 2. Lead vocals stem (best for reviewing lyrics sync)
# 3. Input media (original audio)
audio_url = (
job.file_urls.get('lyrics', {}).get('audio') or
job.file_urls.get('stems', {}).get('lead_vocals') or
job.input_media_gcs_path
)
if not corrections_url:
audio_url = job.file_urls.get('lyrics', {}).get('audio')
if not corrections_url or not audio_url:
raise HTTPException(
status_code=500,
detail="Corrections data not available"
)
if not audio_url:
raise HTTPException(
status_code=500,
detail="Audio not available for review"
detail="Review data not available"
)
# Generate signed URLs for direct access
@@ -477,24 +342,12 @@ async def get_review_data(
@router.post("/{job_id}/start-review")
async def start_review(
job_id: str,
request: StartReviewRequest,
auth_result: AuthResult = Depends(require_auth)
) -> dict:
async def start_review(job_id: str, request: StartReviewRequest) -> dict:
"""
Mark job as IN_REVIEW (user opened review interface).
This helps track that the user is actively working on the review.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
success = job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.IN_REVIEW,
@@ -511,25 +364,20 @@ async def start_review(
async def submit_corrections(
job_id: str,
submission: CorrectionsSubmission,
background_tasks: BackgroundTasks,
auth_result: AuthResult = Depends(require_auth)
background_tasks: BackgroundTasks
) -> dict:
"""
Save corrected lyrics during human review.
This endpoint saves review progress but does NOT complete the review.
Call POST /{job_id}/complete-review to finish and trigger video rendering.
Can be called multiple times to save progress.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to modify this job")
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
raise HTTPException(
status_code=400,
@@ -574,12 +422,11 @@ async def submit_corrections(
@router.post("/{job_id}/complete-review")
async def complete_review(
job_id: str,
background_tasks: BackgroundTasks,
auth_result: AuthResult = Depends(require_auth)
background_tasks: BackgroundTasks
) -> dict:
"""
Complete the human review and trigger video rendering.
This is the FIRST critical human-in-the-loop completion point.
After this:
1. Job transitions to REVIEW_COMPLETE
@@ -590,11 +437,7 @@ async def complete_review(
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to modify this job")
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
raise HTTPException(
status_code=400,
@@ -627,18 +470,13 @@ async def complete_review(
@router.get("/{job_id}/instrumental-options")
async def get_instrumental_options(
job_id: str,
auth_info: Tuple[str, str] = Depends(require_instrumental_auth)
) -> Dict[str, Any]:
async def get_instrumental_options(job_id: str) -> Dict[str, Any]:
"""
Get instrumental audio options for user selection.
Returns signed URLs for both options:
1. Clean instrumental (no backing vocals)
2. Instrumental with backing vocals
Accepts either full auth token or job-specific instrumental_token.
"""
job = job_manager.get_job(job_id)
if not job:
@@ -687,10 +525,7 @@ async def get_instrumental_options(
@router.get("/{job_id}/instrumental-analysis")
async def get_instrumental_analysis(
job_id: str,
auth_info: Tuple[str, str] = Depends(require_instrumental_auth)
) -> Dict[str, Any]:
async def get_instrumental_analysis(job_id: str) -> Dict[str, Any]:
"""
Get audio analysis data for instrumental selection.
@@ -701,8 +536,6 @@ async def get_instrumental_analysis(
This endpoint enables intelligent instrumental selection by providing
detailed analysis of the backing vocals track.
Accepts either full auth token or job-specific instrumental_token.
"""
job = job_manager.get_job(job_id)
if not job:
@@ -766,11 +599,7 @@ async def get_instrumental_analysis(
@router.get("/{job_id}/audio-stream/{stem_type}")
async def stream_audio(
job_id: str,
stem_type: str,
auth_info: Tuple[str, str] = Depends(require_instrumental_auth)
):
async def stream_audio(job_id: str, stem_type: str):
"""
Stream an audio file for playback in the browser.
@@ -782,8 +611,6 @@ async def stream_audio(
Returns audio as a streaming response with proper headers for
browser audio playback.
Accepts either full auth token or job-specific instrumental_token.
"""
from fastapi.responses import StreamingResponse
import tempfile
@@ -860,7 +687,6 @@ async def stream_audio(
async def create_custom_instrumental(
job_id: str,
request: CreateCustomInstrumentalRequest,
auth_info: Tuple[str, str] = Depends(require_instrumental_auth)
) -> Dict[str, Any]:
"""
Create a custom instrumental by muting regions of backing vocals.
@@ -872,8 +698,6 @@ async def create_custom_instrumental(
After calling this endpoint, the user can select "custom" in the
select-instrumental endpoint.
Accepts either full auth token or job-specific instrumental_token.
"""
from backend.services.audio_editing_service import AudioEditingService
from karaoke_gen.instrumental_review import MuteRegion
@@ -952,11 +776,7 @@ async def create_custom_instrumental(
@router.get("/{job_id}/waveform-data")
async def get_waveform_data(
job_id: str,
num_points: int = 500,
auth_info: Tuple[str, str] = Depends(require_instrumental_auth)
) -> Dict[str, Any]:
async def get_waveform_data(job_id: str, num_points: int = 500) -> Dict[str, Any]:
"""
Get waveform amplitude data for client-side rendering.
@@ -973,8 +793,6 @@ async def get_waveform_data(
"duration_seconds": 180.5,
"num_points": 500
}
Accepts either full auth token or job-specific instrumental_token.
"""
from backend.services.audio_analysis_service import AudioAnalysisService
@@ -1015,16 +833,13 @@ async def get_waveform_data(
async def select_instrumental(
job_id: str,
selection: InstrumentalSelection,
background_tasks: BackgroundTasks,
auth_info: Tuple[str, str] = Depends(require_instrumental_auth)
background_tasks: BackgroundTasks
) -> dict:
"""
Submit instrumental selection.
This is the SECOND critical human-in-the-loop interaction point.
After selection, the job proceeds to video generation.
Accepts either full auth token or job-specific instrumental_token.
"""
job = job_manager.get_job(job_id)
if not job:
@@ -1066,24 +881,17 @@ async def select_instrumental(
@router.get("/{job_id}/download-urls")
async def get_download_urls(
job_id: str,
auth_result: AuthResult = Depends(require_auth)
) -> dict:
async def get_download_urls(job_id: str) -> dict:
"""
Get download URLs for all job output files.
Returns a dictionary mapping file types to download URLs.
Uses the streaming download endpoint which proxies through the backend.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
if job.status != JobStatus.COMPLETE:
raise HTTPException(
status_code=400,
@@ -1114,42 +922,21 @@ async def get_download_urls(
}
# Map file keys to human-readable suffixes (matching Dropbox naming from karaoke_finalise.py)
DOWNLOAD_FILENAME_SUFFIXES = {
"lossless_4k_mp4": " (Final Karaoke Lossless 4k).mp4",
"lossless_4k_mkv": " (Final Karaoke Lossless 4k).mkv",
"lossy_4k_mp4": " (Final Karaoke Lossy 4k).mp4",
"lossy_720p_mp4": " (Final Karaoke Lossy 720p).mp4",
"cdg_zip": " (Final Karaoke CDG).zip",
"txt_zip": " (Final Karaoke TXT).zip",
"with_vocals": " (With Vocals).mkv",
}
@router.get("/{job_id}/download/{category}/{file_key}")
async def download_file(
job_id: str,
category: str,
file_key: str,
auth_result: AuthResult = Depends(require_auth)
):
async def download_file(job_id: str, category: str, file_key: str):
"""
Stream download a specific file from a completed job.
This endpoint proxies the file from GCS through the backend,
so no client-side authentication is required.
"""
from fastapi.responses import StreamingResponse
import tempfile
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
if job.status != JobStatus.COMPLETE:
raise HTTPException(
status_code=400,
@@ -1190,18 +977,8 @@ async def download_file(
}
content_type = content_types.get(ext, 'application/octet-stream')
# Build proper filename: "Artist - Title (Final Karaoke Lossy 4k).mp4"
# Use sanitize_filename to handle Unicode characters (curly quotes, em dashes, etc.)
# that cause HTTP header encoding issues (Content-Disposition uses latin-1)
from karaoke_gen.utils import sanitize_filename
artist_clean = sanitize_filename(job.artist) if job.artist else None
title_clean = sanitize_filename(job.title) if job.title else None
base_name = f"{artist_clean} - {title_clean}" if artist_clean and title_clean else None
if base_name and file_key in DOWNLOAD_FILENAME_SUFFIXES:
filename = f"{base_name}{DOWNLOAD_FILENAME_SUFFIXES[file_key]}"
else:
filename = gcs_path.split('/')[-1] # Fallback to original
# Get filename for Content-Disposition
filename = gcs_path.split('/')[-1]
try:
# Download to temp file and stream
@@ -1233,24 +1010,12 @@ async def download_file(
@router.post("/{job_id}/cancel")
async def cancel_job(
job_id: str,
request: CancelJobRequest,
auth_result: AuthResult = Depends(require_auth)
) -> dict:
async def cancel_job(job_id: str, request: CancelJobRequest) -> dict:
"""
Cancel a job.
Jobs can be cancelled at any stage before completion.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to cancel this job")
success = job_manager.cancel_job(job_id, reason=request.reason)
if not success:
@@ -1266,42 +1031,35 @@ async def cancel_job(
@router.post("/{job_id}/retry")
async def retry_job(
job_id: str,
background_tasks: BackgroundTasks,
auth_result: AuthResult = Depends(require_auth)
background_tasks: BackgroundTasks
) -> dict:
"""
Retry a failed or cancelled job from the last successful checkpoint.
This endpoint allows resuming jobs that failed or were cancelled during:
- Audio processing (re-runs from beginning if input audio exists)
Retry a failed job from the last successful checkpoint.
This endpoint allows resuming jobs that failed during:
- Video generation (re-runs video worker)
- Encoding (re-runs video worker)
- Packaging (re-runs video worker)
The retry logic determines the appropriate stage to resume from
based on what files/state already exist.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to retry this job")
if job.status not in [JobStatus.FAILED, JobStatus.CANCELLED]:
if job.status != JobStatus.FAILED:
raise HTTPException(
status_code=400,
detail=f"Only failed or cancelled jobs can be retried (current status: {job.status})"
detail=f"Only failed jobs can be retried (current status: {job.status})"
)
try:
# Determine retry point based on what's already complete
error_details = job.error_details or {}
error_stage = error_details.get('stage', 'unknown')
original_status = job.status
logger.info(f"Job {job_id}: Retrying from {original_status} state (error stage: '{error_stage}')")
logger.info(f"Job {job_id}: Retrying from failed stage '{error_stage}'")
# Check what state exists to determine retry point
file_urls = job.file_urls or {}
@@ -1313,13 +1071,12 @@ async def retry_job(
logger.info(f"Job {job_id}: Has rendered video and instrumental selection, retrying video generation")
# Clear error state and reset worker progress for idempotency
# Clear error state
job_manager.update_job(job_id, {
'error_message': None,
'error_details': None,
})
job_manager.update_state_data(job_id, 'video_progress', {'stage': 'pending'})
# Reset to INSTRUMENTAL_SELECTED and trigger video worker
if not job_manager.transition_to_state(
job_id=job_id,
@@ -1347,14 +1104,13 @@ async def retry_job(
file_urls.get('screens', {}).get('title')):
logger.info(f"Job {job_id}: Has corrections and screens, retrying from render stage")
# Clear error state and reset worker progress for idempotency
# Clear error state
job_manager.update_job(job_id, {
'error_message': None,
'error_details': None,
})
job_manager.update_state_data(job_id, 'render_progress', {'stage': 'pending'})
# Reset to REVIEW_COMPLETE and trigger render worker
if not job_manager.transition_to_state(
job_id=job_id,
@@ -1382,14 +1138,13 @@ async def retry_job(
file_urls.get('lyrics', {}).get('corrections')):
logger.info(f"Job {job_id}: Has stems and corrections, retrying from screens stage")
# Clear error state and reset worker progress for idempotency
# Clear error state
job_manager.update_job(job_id, {
'error_message': None,
'error_details': None,
})
job_manager.update_state_data(job_id, 'screens_progress', {'stage': 'pending'})
# Reset to a state before screens and trigger screens worker
if not job_manager.transition_to_state(
job_id=job_id,
@@ -1412,46 +1167,11 @@ async def retry_job(
"retry_stage": "screens_generation"
}
# If we have input audio (uploaded or from URL), restart from beginning
elif job.input_media_gcs_path or job.url:
logger.info(f"Job {job_id}: Has input audio, restarting from beginning")
# Clear error state and any partial progress
job_manager.update_job(job_id, {
'error_message': None,
'error_details': None,
'state_data': {}, # Clear parallel worker progress
})
# Reset to DOWNLOADING and trigger audio worker
if not job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.DOWNLOADING,
progress=5,
message=f"Restarting job from {original_status} state"
):
raise HTTPException(
status_code=500,
detail="Failed to transition job status for retry"
)
# Trigger audio worker (which kicks off parallel audio + lyrics processing)
background_tasks.add_task(worker_service.trigger_audio_worker, job_id)
background_tasks.add_task(worker_service.trigger_lyrics_worker, job_id)
return {
"status": "success",
"job_id": job_id,
"job_status": "downloading",
"message": f"Job restarted from {original_status} state",
"retry_stage": "from_beginning"
}
else:
# No input audio available - job needs to be resubmitted
# Can't determine a safe retry point, need to restart from beginning
raise HTTPException(
status_code=400,
detail="Cannot retry: no input audio available. Job must be resubmitted."
detail="Cannot determine safe retry point. Job may need to be resubmitted."
)
except HTTPException:
@@ -1465,24 +1185,19 @@ async def retry_job(
async def get_worker_logs(
job_id: str,
since_index: int = 0,
worker: Optional[str] = None,
auth_result: AuthResult = Depends(require_auth)
worker: Optional[str] = None
) -> Dict[str, Any]:
"""
Get worker logs for debugging.
This endpoint returns worker logs stored in Firestore.
Use `since_index` for efficient polling (returns only new logs).
Logs are stored in a subcollection (jobs/{job_id}/logs) to avoid
the 1MB document size limit. Older jobs may have logs in an embedded
array (worker_logs field) - this endpoint handles both transparently.
Args:
job_id: Job ID
since_index: Return only logs after this index (for pagination/polling)
worker: Filter by worker name (audio, lyrics, screens, video, render, distribution)
worker: Filter by worker name (audio, lyrics, screens, video, render)
Returns:
{
"logs": [{"timestamp": "...", "level": "INFO", "worker": "audio", "message": "..."}],
@@ -1493,145 +1208,13 @@ async def get_worker_logs(
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Check ownership - users can only see logs for their own jobs
if not _check_job_ownership(job, auth_result):
raise HTTPException(status_code=403, detail="You don't have permission to access logs for this job")
logs = job_manager.get_worker_logs(job_id, since_index=since_index, worker=worker)
total = job_manager.get_worker_logs_count(job_id)
total = len(job.worker_logs) if job.worker_logs else 0
return {
"logs": logs,
"next_index": since_index + len(logs),
"next_index": total,
"total_logs": total
}
@router.post("/{job_id}/cleanup-distribution")
async def cleanup_distribution(
job_id: str,
delete_job: bool = True,
auth_result: AuthResult = Depends(require_admin)
) -> dict:
"""
Clean up all distributed content for a job (YouTube, Dropbox, Google Drive).
This admin-only endpoint is designed for E2E test cleanup. It:
1. Deletes YouTube video (if uploaded)
2. Deletes Dropbox folder (if uploaded)
3. Deletes Google Drive files (if uploaded)
4. Optionally deletes the job itself
Args:
job_id: Job ID to clean up
delete_job: If True, also delete the job after cleaning up distribution
Returns:
Cleanup results for each service
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
state_data = job.state_data or {}
results = {
"job_id": job_id,
"youtube": {"status": "skipped", "reason": "no youtube_url in state_data"},
"dropbox": {"status": "skipped", "reason": "no brand_code or dropbox_path"},
"gdrive": {"status": "skipped", "reason": "no gdrive_files in state_data"},
"job_deleted": False
}
# Clean up YouTube
youtube_url = state_data.get('youtube_url')
if youtube_url:
try:
# Extract video ID from URL (format: https://youtu.be/VIDEO_ID or https://www.youtube.com/watch?v=VIDEO_ID)
import re
video_id_match = re.search(r'(?:youtu\.be/|youtube\.com/watch\?v=)([^&\s]+)', youtube_url)
if video_id_match:
video_id = video_id_match.group(1)
# Import and use karaoke_finalise for YouTube deletion
from karaoke_gen.karaoke_finalise.karaoke_finalise import KaraokeFinalise
from backend.services.youtube_service import get_youtube_service
youtube_service = get_youtube_service()
if youtube_service.is_configured:
# Create minimal KaraokeFinalise instance for deletion
finalise = KaraokeFinalise(
dry_run=False,
non_interactive=True,
enable_youtube=True,
user_youtube_credentials=youtube_service.get_credentials_dict()
)
success = finalise.delete_youtube_video(video_id)
results["youtube"] = {
"status": "success" if success else "failed",
"video_id": video_id
}
else:
results["youtube"] = {"status": "failed", "reason": "YouTube credentials not configured"}
else:
results["youtube"] = {"status": "failed", "reason": f"Could not extract video ID from {youtube_url}"}
except Exception as e:
logger.error(f"Error cleaning up YouTube for job {job_id}: {e}", exc_info=True)
results["youtube"] = {"status": "error", "error": str(e)}
# Clean up Dropbox
brand_code = state_data.get('brand_code')
dropbox_path = getattr(job, 'dropbox_path', None)
if brand_code and dropbox_path:
try:
from backend.services.dropbox_service import get_dropbox_service
dropbox = get_dropbox_service()
if dropbox.is_configured:
base_name = f"{job.artist} - {job.title}"
folder_name = f"{brand_code} - {base_name}"
full_path = f"{dropbox_path}/{folder_name}"
success = dropbox.delete_folder(full_path)
results["dropbox"] = {
"status": "success" if success else "failed",
"path": full_path
}
else:
results["dropbox"] = {"status": "failed", "reason": "Dropbox credentials not configured"}
except Exception as e:
logger.error(f"Error cleaning up Dropbox for job {job_id}: {e}", exc_info=True)
results["dropbox"] = {"status": "error", "error": str(e)}
# Clean up Google Drive
gdrive_files = state_data.get('gdrive_files')
if gdrive_files:
try:
from backend.services.gdrive_service import get_gdrive_service
gdrive = get_gdrive_service()
if gdrive.is_configured:
# gdrive_files is a dict like {"mp4": "file_id", "mp4_720p": "file_id", "cdg": "file_id"}
file_ids = list(gdrive_files.values()) if isinstance(gdrive_files, dict) else []
delete_results = gdrive.delete_files(file_ids)
all_success = all(delete_results.values())
results["gdrive"] = {
"status": "success" if all_success else "partial",
"files": delete_results
}
else:
results["gdrive"] = {"status": "failed", "reason": "Google Drive credentials not configured"}
except Exception as e:
logger.error(f"Error cleaning up Google Drive for job {job_id}: {e}", exc_info=True)
results["gdrive"] = {"status": "error", "error": str(e)}
# Delete the job if requested
if delete_job:
try:
job_manager.delete_job(job_id, delete_files=True)
results["job_deleted"] = True
logger.info(f"Deleted job {job_id} after distribution cleanup")
except Exception as e:
logger.error(f"Error deleting job {job_id}: {e}", exc_info=True)
results["job_deleted"] = False
results["job_delete_error"] = str(e)
return results
-238
View File
@@ -1,238 +0,0 @@
"""
Push Notification API routes.
Provides endpoints for managing Web Push notification subscriptions:
- GET /api/push/vapid-public-key: Get VAPID public key for client-side subscription
- POST /api/push/subscribe: Register a push subscription
- POST /api/push/unsubscribe: Remove a push subscription
- GET /api/push/subscriptions: List user's subscriptions
- POST /api/push/test: Send a test notification (admin only)
"""
import logging
from typing import Optional, Dict, List
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from backend.config import get_settings
from backend.api.dependencies import require_auth, require_admin
from backend.services.auth_service import AuthResult
from backend.services.push_notification_service import get_push_notification_service
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/push", tags=["push"])
# Request/Response Models
class VapidPublicKeyResponse(BaseModel):
"""Response containing VAPID public key."""
enabled: bool
vapid_public_key: Optional[str] = None
class SubscribeRequest(BaseModel):
"""Request to subscribe to push notifications."""
endpoint: str
keys: Dict[str, str] # p256dh and auth
device_name: Optional[str] = None
class SubscribeResponse(BaseModel):
"""Response after subscribing."""
status: str
message: str
class UnsubscribeRequest(BaseModel):
"""Request to unsubscribe from push notifications."""
endpoint: str
class UnsubscribeResponse(BaseModel):
"""Response after unsubscribing."""
status: str
message: str
class SubscriptionInfo(BaseModel):
"""Information about a push subscription."""
endpoint: str
device_name: Optional[str] = None
created_at: Optional[str] = None
last_used_at: Optional[str] = None
class SubscriptionsListResponse(BaseModel):
"""Response containing user's subscriptions."""
subscriptions: List[SubscriptionInfo]
count: int
class TestNotificationRequest(BaseModel):
"""Request to send a test notification."""
title: Optional[str] = "Test Notification"
body: Optional[str] = "This is a test push notification from Karaoke Generator"
class TestNotificationResponse(BaseModel):
"""Response after sending test notification."""
status: str
sent_count: int
message: str
# Routes
@router.get("/vapid-public-key", response_model=VapidPublicKeyResponse)
async def get_vapid_public_key():
"""
Get the VAPID public key for push subscription.
This endpoint is public - no authentication required.
Returns the public key needed for client-side PushManager.subscribe().
"""
settings = get_settings()
push_service = get_push_notification_service()
if not settings.enable_push_notifications:
return VapidPublicKeyResponse(enabled=False)
public_key = push_service.get_public_key()
if not public_key:
return VapidPublicKeyResponse(enabled=False)
return VapidPublicKeyResponse(
enabled=True,
vapid_public_key=public_key
)
@router.post("/subscribe", response_model=SubscribeResponse)
async def subscribe_push(
request: SubscribeRequest,
auth_result: AuthResult = Depends(require_auth)
):
"""
Register a push notification subscription for the current user.
Requires authentication. Users can have up to 5 subscriptions
(configurable via MAX_PUSH_SUBSCRIPTIONS_PER_USER).
"""
settings = get_settings()
if not settings.enable_push_notifications:
raise HTTPException(status_code=503, detail="Push notifications are not enabled")
if not auth_result.user_email:
raise HTTPException(status_code=401, detail="User email not available")
push_service = get_push_notification_service()
# Validate keys
if "p256dh" not in request.keys or "auth" not in request.keys:
raise HTTPException(status_code=400, detail="Missing required keys (p256dh, auth)")
success = await push_service.add_subscription(
user_email=auth_result.user_email,
endpoint=request.endpoint,
keys=request.keys,
device_name=request.device_name
)
if not success:
raise HTTPException(status_code=500, detail="Failed to save subscription")
return SubscribeResponse(
status="success",
message="Push subscription registered successfully"
)
@router.post("/unsubscribe", response_model=UnsubscribeResponse)
async def unsubscribe_push(
request: UnsubscribeRequest,
auth_result: AuthResult = Depends(require_auth)
):
"""
Remove a push notification subscription.
Requires authentication. Users can only remove their own subscriptions.
"""
if not auth_result.user_email:
raise HTTPException(status_code=401, detail="User email not available")
push_service = get_push_notification_service()
success = await push_service.remove_subscription(
user_email=auth_result.user_email,
endpoint=request.endpoint
)
if not success:
# Don't error if subscription wasn't found - might already be removed
return UnsubscribeResponse(
status="success",
message="Subscription removed (or was not found)"
)
return UnsubscribeResponse(
status="success",
message="Push subscription removed successfully"
)
@router.get("/subscriptions", response_model=SubscriptionsListResponse)
async def list_subscriptions(
auth_result: AuthResult = Depends(require_auth)
):
"""
List all push notification subscriptions for the current user.
Requires authentication.
"""
if not auth_result.user_email:
raise HTTPException(status_code=401, detail="User email not available")
push_service = get_push_notification_service()
subscriptions = await push_service.list_subscriptions(auth_result.user_email)
return SubscriptionsListResponse(
subscriptions=[SubscriptionInfo(**s) for s in subscriptions],
count=len(subscriptions)
)
@router.post("/test", response_model=TestNotificationResponse)
async def send_test_notification(
request: TestNotificationRequest,
auth_result: AuthResult = Depends(require_admin)
):
"""
Send a test push notification to the current user's devices.
Admin only. Useful for testing push notification setup.
"""
settings = get_settings()
if not settings.enable_push_notifications:
raise HTTPException(status_code=503, detail="Push notifications are not enabled")
if not auth_result.user_email:
raise HTTPException(status_code=401, detail="User email not available")
push_service = get_push_notification_service()
sent_count = await push_service.send_push(
user_email=auth_result.user_email,
title=request.title or "Test Notification",
body=request.body or "This is a test push notification",
url="/app/",
tag="test"
)
return TestNotificationResponse(
status="success",
sent_count=sent_count,
message=f"Test notification sent to {sent_count} device(s)"
)
-428
View File
@@ -1,428 +0,0 @@
"""
Admin API routes for rate limit management.
Handles:
- Rate limit statistics and monitoring
- Blocklist management (disposable domains, blocked emails, blocked IPs)
- User override management (whitelist/bypass permissions)
"""
import logging
from datetime import datetime, timezone
from typing import List, Optional
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, EmailStr
from backend.api.dependencies import require_admin
from backend.services.auth_service import AuthResult
from backend.services.rate_limit_service import get_rate_limit_service, RateLimitService
from backend.services.email_validation_service import get_email_validation_service, EmailValidationService
from backend.config import settings
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/admin/rate-limits", tags=["admin", "rate-limits"])
# =============================================================================
# Response Models
# =============================================================================
class RateLimitStatsResponse(BaseModel):
"""Current rate limit statistics."""
# Configuration
jobs_per_day_limit: int
youtube_uploads_per_day_limit: int
beta_ip_per_day_limit: int
rate_limiting_enabled: bool
# Current usage
youtube_uploads_today: int
youtube_uploads_remaining: int
# Blocklist stats
disposable_domains_count: int
blocked_emails_count: int
blocked_ips_count: int
# User override stats
total_overrides: int
class UserRateLimitStatusResponse(BaseModel):
"""Rate limit status for a specific user."""
email: str
jobs_today: int
jobs_limit: int
jobs_remaining: int
has_bypass: bool
custom_limit: Optional[int]
bypass_reason: Optional[str]
class BlocklistsResponse(BaseModel):
"""All blocklist data."""
disposable_domains: List[str]
blocked_emails: List[str]
blocked_ips: List[str]
updated_at: Optional[datetime]
updated_by: Optional[str]
class DomainRequest(BaseModel):
"""Request to add/remove a domain."""
domain: str
class EmailRequest(BaseModel):
"""Request to add/remove an email."""
email: str
class IPRequest(BaseModel):
"""Request to add/remove an IP."""
ip_address: str
class UserOverride(BaseModel):
"""User override configuration."""
email: str
bypass_job_limit: bool
custom_daily_job_limit: Optional[int]
reason: str
created_by: str
created_at: datetime
class UserOverrideRequest(BaseModel):
"""Request to set/update a user override."""
bypass_job_limit: bool = False
custom_daily_job_limit: Optional[int] = None
reason: str
class UserOverridesListResponse(BaseModel):
"""List of all user overrides."""
overrides: List[UserOverride]
total: int
class SuccessResponse(BaseModel):
"""Generic success response."""
success: bool
message: str
# =============================================================================
# Statistics Endpoints
# =============================================================================
@router.get("/stats", response_model=RateLimitStatsResponse)
async def get_rate_limit_stats(
auth_result: AuthResult = Depends(require_admin),
):
"""
Get current rate limit statistics.
Returns configuration values, current usage counts, and blocklist stats.
"""
rate_limit_service = get_rate_limit_service()
email_validation = get_email_validation_service()
# Get YouTube uploads today
youtube_today = rate_limit_service.get_youtube_uploads_today()
youtube_limit = settings.rate_limit_youtube_uploads_per_day
youtube_remaining = max(0, youtube_limit - youtube_today) if youtube_limit > 0 else -1
# Get blocklist stats
blocklist_stats = email_validation.get_blocklist_stats()
# Get override count
overrides = rate_limit_service.get_all_overrides()
return RateLimitStatsResponse(
jobs_per_day_limit=settings.rate_limit_jobs_per_day,
youtube_uploads_per_day_limit=settings.rate_limit_youtube_uploads_per_day,
beta_ip_per_day_limit=settings.rate_limit_beta_ip_per_day,
rate_limiting_enabled=settings.enable_rate_limiting,
youtube_uploads_today=youtube_today,
youtube_uploads_remaining=youtube_remaining,
disposable_domains_count=blocklist_stats["disposable_domains_count"],
blocked_emails_count=blocklist_stats["blocked_emails_count"],
blocked_ips_count=blocklist_stats["blocked_ips_count"],
total_overrides=len(overrides),
)
@router.get("/users/{email}", response_model=UserRateLimitStatusResponse)
async def get_user_rate_limit_status(
email: str,
auth_result: AuthResult = Depends(require_admin),
):
"""
Get rate limit status for a specific user.
Returns their current usage and any override settings.
"""
rate_limit_service = get_rate_limit_service()
jobs_today = rate_limit_service.get_user_job_count_today(email)
override = rate_limit_service.get_user_override(email)
if override:
has_bypass = override.get("bypass_job_limit", False)
custom_limit = override.get("custom_daily_job_limit")
bypass_reason = override.get("reason")
jobs_limit = custom_limit if custom_limit is not None else settings.rate_limit_jobs_per_day
else:
has_bypass = False
custom_limit = None
bypass_reason = None
jobs_limit = settings.rate_limit_jobs_per_day
jobs_remaining = max(0, jobs_limit - jobs_today) if not has_bypass else -1
return UserRateLimitStatusResponse(
email=email,
jobs_today=jobs_today,
jobs_limit=jobs_limit,
jobs_remaining=jobs_remaining,
has_bypass=has_bypass,
custom_limit=custom_limit,
bypass_reason=bypass_reason,
)
# =============================================================================
# Blocklist Management Endpoints
# =============================================================================
@router.get("/blocklists", response_model=BlocklistsResponse)
async def get_blocklists(
auth_result: AuthResult = Depends(require_admin),
):
"""
Get all blocklist data.
Returns disposable domains, blocked emails, and blocked IPs.
"""
email_validation = get_email_validation_service()
# Force refresh to get latest data
config = email_validation.get_blocklist_config(force_refresh=True)
# Get the raw document for metadata
from google.cloud import firestore
from backend.services.firestore_service import get_firestore_client
from backend.services.email_validation_service import BLOCKLISTS_COLLECTION, BLOCKLIST_CONFIG_DOC
db = get_firestore_client()
doc = db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC).get()
updated_at = None
updated_by = None
if doc.exists:
data = doc.to_dict()
updated_at = data.get("updated_at")
updated_by = data.get("updated_by")
return BlocklistsResponse(
disposable_domains=sorted(list(config["disposable_domains"])),
blocked_emails=sorted(list(config["blocked_emails"])),
blocked_ips=sorted(list(config["blocked_ips"])),
updated_at=updated_at,
updated_by=updated_by,
)
@router.post("/blocklists/disposable-domains", response_model=SuccessResponse)
async def add_disposable_domain(
request: DomainRequest,
auth_result: AuthResult = Depends(require_admin),
):
"""Add a domain to the disposable domains blocklist."""
email_validation = get_email_validation_service()
domain = request.domain.lower().strip()
if not domain or "." not in domain:
raise HTTPException(status_code=400, detail="Invalid domain format")
email_validation.add_disposable_domain(domain, auth_result.user_email)
return SuccessResponse(
success=True,
message=f"Domain '{domain}' added to disposable domains blocklist"
)
@router.delete("/blocklists/disposable-domains/{domain}", response_model=SuccessResponse)
async def remove_disposable_domain(
domain: str,
auth_result: AuthResult = Depends(require_admin),
):
"""Remove a domain from the disposable domains blocklist."""
email_validation = get_email_validation_service()
domain = domain.lower().strip()
if not email_validation.remove_disposable_domain(domain, auth_result.user_email):
raise HTTPException(status_code=404, detail="Domain not found in blocklist")
return SuccessResponse(
success=True,
message=f"Domain '{domain}' removed from disposable domains blocklist"
)
@router.post("/blocklists/blocked-emails", response_model=SuccessResponse)
async def add_blocked_email(
request: EmailRequest,
auth_result: AuthResult = Depends(require_admin),
):
"""Add an email to the blocked emails list."""
email_validation = get_email_validation_service()
email = request.email.lower().strip()
if not email or "@" not in email:
raise HTTPException(status_code=400, detail="Invalid email format")
email_validation.add_blocked_email(email, auth_result.user_email)
return SuccessResponse(
success=True,
message=f"Email '{email}' added to blocked emails list"
)
@router.delete("/blocklists/blocked-emails/{email}", response_model=SuccessResponse)
async def remove_blocked_email(
email: str,
auth_result: AuthResult = Depends(require_admin),
):
"""Remove an email from the blocked emails list."""
email_validation = get_email_validation_service()
email = email.lower().strip()
if not email_validation.remove_blocked_email(email, auth_result.user_email):
raise HTTPException(status_code=404, detail="Email not found in blocklist")
return SuccessResponse(
success=True,
message=f"Email '{email}' removed from blocked emails list"
)
@router.post("/blocklists/blocked-ips", response_model=SuccessResponse)
async def add_blocked_ip(
request: IPRequest,
auth_result: AuthResult = Depends(require_admin),
):
"""Add an IP address to the blocked IPs list."""
email_validation = get_email_validation_service()
ip_address = request.ip_address.strip()
if not ip_address:
raise HTTPException(status_code=400, detail="Invalid IP address")
email_validation.add_blocked_ip(ip_address, auth_result.user_email)
return SuccessResponse(
success=True,
message=f"IP '{ip_address}' added to blocked IPs list"
)
@router.delete("/blocklists/blocked-ips/{ip_address}", response_model=SuccessResponse)
async def remove_blocked_ip(
ip_address: str,
auth_result: AuthResult = Depends(require_admin),
):
"""Remove an IP address from the blocked IPs list."""
email_validation = get_email_validation_service()
ip_address = ip_address.strip()
if not email_validation.remove_blocked_ip(ip_address, auth_result.user_email):
raise HTTPException(status_code=404, detail="IP not found in blocklist")
return SuccessResponse(
success=True,
message=f"IP '{ip_address}' removed from blocked IPs list"
)
# =============================================================================
# User Override Management Endpoints
# =============================================================================
@router.get("/overrides", response_model=UserOverridesListResponse)
async def get_all_overrides(
auth_result: AuthResult = Depends(require_admin),
):
"""Get all user overrides."""
rate_limit_service = get_rate_limit_service()
overrides_data = rate_limit_service.get_all_overrides()
overrides = [
UserOverride(
email=email,
bypass_job_limit=data.get("bypass_job_limit", False),
custom_daily_job_limit=data.get("custom_daily_job_limit"),
reason=data.get("reason", ""),
created_by=data.get("created_by", "unknown"),
created_at=data.get("created_at", datetime.now(timezone.utc)),
)
for email, data in overrides_data.items()
]
return UserOverridesListResponse(
overrides=overrides,
total=len(overrides),
)
@router.put("/overrides/{email}", response_model=SuccessResponse)
async def set_user_override(
email: str,
request: UserOverrideRequest,
auth_result: AuthResult = Depends(require_admin),
):
"""Set or update a user override."""
rate_limit_service = get_rate_limit_service()
email = email.lower().strip()
if not email or "@" not in email:
raise HTTPException(status_code=400, detail="Invalid email format")
if not request.reason or len(request.reason.strip()) < 3:
raise HTTPException(status_code=400, detail="Reason is required (min 3 characters)")
rate_limit_service.set_user_override(
user_email=email,
bypass_job_limit=request.bypass_job_limit,
custom_daily_job_limit=request.custom_daily_job_limit,
reason=request.reason,
admin_email=auth_result.user_email,
)
return SuccessResponse(
success=True,
message=f"Override set for user '{email}'"
)
@router.delete("/overrides/{email}", response_model=SuccessResponse)
async def remove_user_override(
email: str,
auth_result: AuthResult = Depends(require_admin),
):
"""Remove a user override."""
rate_limit_service = get_rate_limit_service()
email = email.lower().strip()
if not rate_limit_service.remove_user_override(email, auth_result.user_email):
raise HTTPException(status_code=404, detail="Override not found for user")
return SuccessResponse(
success=True,
message=f"Override removed for user '{email}'"
)
+52 -180
View File
@@ -16,9 +16,9 @@ import json
import os
import tempfile
from pathlib import Path
from typing import Dict, Any, Set, Tuple
from typing import Dict, Any, Set
from fastapi import APIRouter, HTTPException, Request, Depends
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import FileResponse, StreamingResponse
from starlette.background import BackgroundTask
@@ -27,10 +27,6 @@ from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.services.job_logging import job_log_context, JobLogger
from backend.services.tracing import create_span, add_span_attribute, add_span_event
from backend.services.encoding_service import get_encoding_service
from backend.api.dependencies import require_auth, require_review_auth
from backend.services.auth_service import UserType
from backend.config import get_settings
# LyricsTranscriber imports for preview generation
from lyrics_transcriber.types import CorrectionResult
@@ -67,10 +63,7 @@ async def ping(job_id: str):
@router.get("/{job_id}/correction-data")
async def get_correction_data(
job_id: str,
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def get_correction_data(job_id: str):
"""
Get correction data for the review interface.
@@ -136,21 +129,14 @@ async def get_correction_data(
@router.get("/{job_id}/audio/{audio_hash}")
async def get_audio_with_hash(
job_id: str,
audio_hash: str,
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def get_audio_with_hash(job_id: str, audio_hash: str):
"""Stream the audio file for playback (with hash parameter)."""
return await _stream_audio(job_id)
@router.get("/{job_id}/audio/")
@router.get("/{job_id}/audio")
async def get_audio_no_hash(
job_id: str,
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def get_audio_no_hash(job_id: str):
"""Stream the audio file for playback (without hash parameter)."""
return await _stream_audio(job_id)
@@ -203,11 +189,7 @@ async def _stream_audio(job_id: str):
@router.post("/{job_id}/complete")
async def complete_review(
job_id: str,
updated_data: Dict[str, Any],
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def complete_review(job_id: str, updated_data: Dict[str, Any]):
"""
Complete the review and save corrected lyrics.
@@ -262,11 +244,7 @@ async def complete_review(
@router.post("/{job_id}/handlers")
async def update_handlers(
job_id: str,
enabled_handlers: list,
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def update_handlers(job_id: str, enabled_handlers: list):
"""
Update enabled correction handlers (optional feature).
@@ -278,11 +256,7 @@ async def update_handlers(
@router.post("/{job_id}/add-lyrics")
async def add_lyrics(
job_id: str,
data: Dict[str, str],
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def add_lyrics(job_id: str, data: Dict[str, str]):
"""
Add custom lyrics source and rerun correction.
@@ -380,45 +354,31 @@ async def add_lyrics(
@router.post("/{job_id}/preview-video")
async def generate_preview_video(
job_id: str,
updated_data: Dict[str, Any],
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def generate_preview_video(job_id: str, updated_data: Dict[str, Any]):
"""
Generate a preview video with the current corrections.
Uses the LyricsTranscriber's CorrectionOperations to generate a 360p preview
video with the user's current corrections applied.
When USE_GCE_PREVIEW_ENCODING is enabled, video encoding is offloaded to
the high-performance GCE worker for faster generation (15-20s vs 60+s).
"""
job_manager = JobManager()
storage = StorageService()
settings = get_settings()
encoding_service = get_encoding_service()
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Job must be in review state to generate preview
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
raise HTTPException(
status_code=400,
detail=f"Job not in review state (current status: {job.status})"
)
# Check if GCE preview encoding is enabled
use_gce_preview = encoding_service.is_preview_enabled
# Check if user wants theme background image (slower) or black background (faster, default)
use_background_image = updated_data.get("use_background_image", False)
logger.info(f"Job {job_id}: Generating preview video (GCE: {use_gce_preview}, background image: {use_background_image})")
logger.info(f"Job {job_id}: Generating preview video")
# Use tracing and job_log_context for full observability
with create_span("generate-preview-video", {"job_id": job_id, "use_gce": use_gce_preview}) as span:
with create_span("generate-preview-video", {"job_id": job_id}) as span:
with job_log_context(job_id, worker="preview"):
try:
# Create temp directory for this preview operation
@@ -428,19 +388,19 @@ async def generate_preview_video(
corrections_gcs = f"jobs/{job_id}/lyrics/corrections.json"
corrections_path = os.path.join(temp_dir, "corrections.json")
storage.download_file(corrections_gcs, corrections_path)
with open(corrections_path, 'r', encoding='utf-8') as f:
original_data = json.load(f)
# 2. Download input audio
audio_path = os.path.join(temp_dir, "audio.flac")
storage.download_file(job.input_media_gcs_path, audio_path)
download_span.set_attribute("audio_gcs_path", job.input_media_gcs_path)
# 3. Load original as CorrectionResult
correction_result = CorrectionResult.from_dict(original_data)
add_span_event("corrections_loaded")
# 4. Get or create styles file for preview using unified style loader
with create_span("load-styles") as styles_span:
styles_path, _ = load_styles_from_gcs(
@@ -451,131 +411,54 @@ async def generate_preview_video(
logger=logger,
)
styles_span.set_attribute("styles_path", styles_path)
# 5. Set up output config for preview
output_dir = os.path.join(temp_dir, "output")
cache_dir = os.path.join(temp_dir, "cache")
os.makedirs(output_dir, exist_ok=True)
os.makedirs(cache_dir, exist_ok=True)
output_config = OutputConfig(
output_styles_json=styles_path,
output_dir=output_dir,
cache_dir=cache_dir,
video_resolution="360p",
)
# 6. Generate preview (ASS-only if using GCE, or full video if local)
preview_gcs_path = None
if use_gce_preview:
# GCE path: Generate ASS only, then offload encoding to GCE
try:
with create_span("generate-ass-subtitles") as ass_span:
result = CorrectionOperations.generate_preview_video(
correction_result=correction_result,
updated_data=updated_data,
output_config=output_config,
audio_filepath=audio_path,
artist=job.artist,
title=job.title,
logger=logger,
ass_only=True, # Only generate ASS, skip video encoding
)
preview_hash = result["preview_hash"]
ass_path = result["ass_path"]
ass_span.set_attribute("ass_path", ass_path)
add_span_event("ass_generated")
# Upload ASS to GCS
with create_span("upload-ass-to-gcs") as upload_ass_span:
ass_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.ass"
storage.upload_file(ass_path, ass_gcs_path)
upload_ass_span.set_attribute("ass_gcs_path", ass_gcs_path)
# Call GCE encoding service
with create_span("gce-preview-encoding") as gce_span:
bucket_name = settings.gcs_bucket_name
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
# Get background image and font from style assets if available
style_assets = job.style_assets or {}
# Only use background image if user explicitly requested it
# Default is black background for faster preview generation (~10s vs ~30-60s)
background_image_gcs_path = None
if use_background_image:
for key in ["karaoke_background", "style_karaoke_background"]:
if key in style_assets:
background_image_gcs_path = f"gs://{bucket_name}/{style_assets[key]}"
gce_span.set_attribute("background_image", background_image_gcs_path)
break
gce_span.set_attribute("use_background_image", use_background_image)
font_gcs_path = None
for key in ["font", "style_font"]:
if key in style_assets:
font_gcs_path = f"gs://{bucket_name}/{style_assets[key]}"
gce_span.set_attribute("font", font_gcs_path)
break
gce_result = await encoding_service.encode_preview_video(
job_id=f"preview_{job_id}_{preview_hash}",
ass_gcs_path=f"gs://{bucket_name}/{ass_gcs_path}",
audio_gcs_path=f"gs://{bucket_name}/{job.input_media_gcs_path}",
output_gcs_path=f"gs://{bucket_name}/{preview_gcs_path}",
background_color="black",
background_image_gcs_path=background_image_gcs_path,
font_gcs_path=font_gcs_path,
)
gce_span.set_attribute("gce_status", gce_result.get("status"))
add_span_event("gce_encoding_complete")
logger.info(f"Job {job_id}: Preview generated via GCE: {preview_hash}")
except Exception as gce_error:
# Fall back to local encoding if GCE fails
logger.warning(
f"Job {job_id}: GCE preview encoding failed, falling back to local: {gce_error}"
)
span.set_attribute("gce_fallback", True)
use_gce_preview = False # Fall through to local encoding below
if not use_gce_preview:
# Local path: Generate full preview video locally
with create_span("render-preview-video-local") as render_span:
render_span.set_attribute("resolution", "360p")
result = CorrectionOperations.generate_preview_video(
correction_result=correction_result,
updated_data=updated_data,
output_config=output_config,
audio_filepath=audio_path,
artist=job.artist,
title=job.title,
logger=logger,
ass_only=False, # Generate full video locally
)
preview_hash = result["preview_hash"]
video_path = result["video_path"]
add_span_event("render_complete")
# Upload preview video to GCS
with create_span("upload-preview-video") as upload_span:
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
storage.upload_file(video_path, preview_gcs_path)
upload_span.set_attribute("gcs_path", preview_gcs_path)
# 6. Generate preview video using CorrectionOperations (heavy operation)
with create_span("render-preview-video") as render_span:
render_span.set_attribute("resolution", "360p")
result = CorrectionOperations.generate_preview_video(
correction_result=correction_result,
updated_data=updated_data,
output_config=output_config,
audio_filepath=audio_path,
artist=job.artist,
title=job.title,
logger=logger
)
add_span_event("render_complete")
# 7. Upload preview video to GCS for serving
preview_hash = result["preview_hash"]
video_path = result["video_path"]
with create_span("upload-preview-video") as upload_span:
preview_gcs_path = f"jobs/{job_id}/previews/{preview_hash}.mp4"
storage.upload_file(video_path, preview_gcs_path)
upload_span.set_attribute("gcs_path", preview_gcs_path)
# Store the GCS path for serving
if job_id not in _preview_videos:
_preview_videos[job_id] = {}
_preview_videos[job_id][preview_hash] = preview_gcs_path
logger.info(f"Job {job_id}: Preview video generated: {preview_hash}")
span.set_attribute("preview_hash", preview_hash)
span.set_attribute("success", True)
return {"status": "success", "preview_hash": preview_hash}
except Exception as e:
logger.error(f"Job {job_id}: Failed to generate preview video: {e}", exc_info=True)
span.set_attribute("error", str(e))
@@ -586,11 +469,7 @@ async def generate_preview_video(
@router.get("/{job_id}/preview-video/{preview_hash}")
async def get_preview_video(
job_id: str,
preview_hash: str,
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def get_preview_video(job_id: str, preview_hash: str):
"""Stream the generated preview video."""
storage = StorageService()
@@ -631,11 +510,7 @@ async def get_preview_video(
@router.post("/{job_id}/v1/annotations")
async def submit_annotation(
job_id: str,
annotation: Dict[str, Any],
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def submit_annotation(job_id: str, annotation: Dict[str, Any]):
"""
Submit a correction annotation for ML training data.
@@ -647,10 +522,7 @@ async def submit_annotation(
@router.get("/{job_id}/v1/annotations/stats")
async def get_annotation_stats(
job_id: str,
auth_info: Tuple[str, str] = Depends(require_review_auth)
):
async def get_annotation_stats(job_id: str):
"""Get annotation statistics."""
return {
"total_annotations": 0,
-120
View File
@@ -1,120 +0,0 @@
"""
Tenant API routes for white-label portal configuration.
These endpoints are public (no auth required) since the frontend needs
to fetch tenant branding before the user logs in.
"""
import logging
from typing import Optional
from fastapi import APIRouter, Query, Request
from backend.models.tenant import TenantConfigResponse, TenantPublicConfig
from backend.services.tenant_service import get_tenant_service
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/tenant", tags=["tenant"])
@router.get("/config", response_model=TenantConfigResponse)
async def get_tenant_config(
request: Request,
tenant: Optional[str] = Query(
None,
description="Tenant ID override (for development). If not provided, detected from subdomain.",
),
):
"""
Get tenant configuration for the current portal.
This endpoint detects the tenant from:
1. Query parameter `tenant` (for development/testing)
2. X-Tenant-ID header (set by frontend)
3. Host header subdomain
Returns the public tenant configuration (branding, features, defaults)
or indicates this is the default Nomad Karaoke portal.
"""
tenant_service = get_tenant_service()
# Priority 1: Query parameter (for dev/testing)
tenant_id = tenant
# Priority 2: X-Tenant-ID header
if not tenant_id:
tenant_id = request.headers.get("X-Tenant-ID")
# Priority 3: Detect from Host header
if not tenant_id:
host = request.headers.get("Host", "")
# Check if this is a tenant subdomain
# Pattern: {tenant}.nomadkaraoke.com or {tenant}.gen.nomadkaraoke.com
if host and "nomadkaraoke.com" in host.lower():
parts = host.lower().split(".")
# Skip known non-tenant subdomains
if parts[0] not in ["gen", "api", "www", "buy", "admin"]:
potential_tenant = parts[0]
if tenant_service.tenant_exists(potential_tenant):
tenant_id = potential_tenant
# If no tenant detected, return default config
if not tenant_id:
logger.debug("No tenant detected, returning default config")
return TenantConfigResponse(tenant=None, is_default=True)
# Load tenant config
public_config = tenant_service.get_public_config(tenant_id)
if not public_config:
logger.warning(f"Tenant not found: {tenant_id}")
return TenantConfigResponse(tenant=None, is_default=True)
if not public_config.is_active:
logger.warning(f"Tenant is inactive: {tenant_id}")
return TenantConfigResponse(tenant=None, is_default=True)
logger.info(f"Returning config for tenant: {tenant_id}")
return TenantConfigResponse(tenant=public_config, is_default=False)
@router.get("/config/{tenant_id}", response_model=TenantConfigResponse)
async def get_tenant_config_by_id(tenant_id: str):
"""
Get tenant configuration by explicit tenant ID.
This is useful for admin tools or debugging.
"""
tenant_service = get_tenant_service()
public_config = tenant_service.get_public_config(tenant_id)
if not public_config:
return TenantConfigResponse(tenant=None, is_default=True)
# Check if tenant is active
if not public_config.is_active:
logger.warning(f"Tenant is inactive: {tenant_id}")
return TenantConfigResponse(tenant=None, is_default=True)
return TenantConfigResponse(tenant=public_config, is_default=False)
@router.get("/asset/{tenant_id}/{asset_name}")
async def get_tenant_asset(tenant_id: str, asset_name: str):
"""
Get a signed URL for a tenant asset (logo, favicon, etc.).
This redirects to the signed GCS URL for the asset.
"""
from fastapi.responses import RedirectResponse
tenant_service = get_tenant_service()
url = tenant_service.get_asset_url(tenant_id, asset_name)
if not url:
from fastapi import HTTPException
raise HTTPException(status_code=404, detail="Asset not found")
return RedirectResponse(url=url)
-162
View File
@@ -1,162 +0,0 @@
"""
Theme API routes for listing and retrieving theme information.
Themes are pre-created style configurations that users can select when creating jobs.
These endpoints are public (no authentication required) to allow theme selection
before job creation.
"""
import logging
from typing import Optional
from fastapi import APIRouter, HTTPException
from backend.models.theme import ThemeDetailResponse, ThemesListResponse
from backend.services.theme_service import get_theme_service
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/themes", tags=["themes"])
@router.get("", response_model=ThemesListResponse)
async def list_themes() -> ThemesListResponse:
"""
List all available themes.
Returns a list of themes with their metadata and preview image URLs.
Preview URLs are signed and valid for 60 minutes.
This endpoint is public and does not require authentication.
"""
try:
theme_service = get_theme_service()
themes = theme_service.list_themes()
return ThemesListResponse(themes=themes)
except Exception as e:
logger.error(f"Error listing themes: {e}")
raise HTTPException(
status_code=500,
detail="Failed to load themes"
)
@router.get("/{theme_id}", response_model=ThemeDetailResponse)
async def get_theme(theme_id: str) -> ThemeDetailResponse:
"""
Get detailed information about a specific theme.
Returns the full theme details including style parameters.
This can be used to preview theme settings or inspect configuration.
Args:
theme_id: The unique identifier of the theme
Returns:
ThemeDetailResponse with full theme details
Raises:
404: Theme not found
"""
try:
theme_service = get_theme_service()
theme = theme_service.get_theme(theme_id)
if theme is None:
raise HTTPException(
status_code=404,
detail=f"Theme not found: {theme_id}"
)
return ThemeDetailResponse(theme=theme)
except HTTPException:
raise
except Exception as e:
logger.error(f"Error getting theme {theme_id}: {e}")
raise HTTPException(
status_code=500,
detail="Failed to load theme"
)
@router.get("/{theme_id}/preview")
async def get_theme_preview(theme_id: str) -> dict:
"""
Get a signed URL for the theme's preview image.
This endpoint is useful when you need just the preview URL
without loading the full theme details.
Args:
theme_id: The unique identifier of the theme
Returns:
Object with preview_url field
Raises:
404: Theme not found or no preview available
"""
try:
theme_service = get_theme_service()
if not theme_service.theme_exists(theme_id):
raise HTTPException(
status_code=404,
detail=f"Theme not found: {theme_id}"
)
theme = theme_service.get_theme(theme_id)
if theme is None or theme.preview_url is None:
raise HTTPException(
status_code=404,
detail=f"No preview available for theme: {theme_id}"
)
return {"preview_url": theme.preview_url}
except HTTPException:
raise
except Exception as e:
logger.error(f"Error getting preview for theme {theme_id}: {e}")
raise HTTPException(
status_code=500,
detail="Failed to load theme preview"
)
@router.get("/{theme_id}/youtube-description")
async def get_theme_youtube_description(theme_id: str) -> dict:
"""
Get the YouTube description template for a theme.
Some themes include a default YouTube video description template
that can be used when uploading videos.
Args:
theme_id: The unique identifier of the theme
Returns:
Object with description field containing the template text,
or null if no template is available
Raises:
404: Theme not found
"""
try:
theme_service = get_theme_service()
if not theme_service.theme_exists(theme_id):
raise HTTPException(
status_code=404,
detail=f"Theme not found: {theme_id}"
)
description = theme_service.get_youtube_description(theme_id)
return {"description": description}
except HTTPException:
raise
except Exception as e:
logger.error(f"Error getting YouTube description for theme {theme_id}: {e}")
raise HTTPException(
status_code=500,
detail="Failed to load YouTube description"
)
File diff suppressed because it is too large. Load diff
+2 -70
View File
@@ -40,15 +40,7 @@ class Settings(BaseSettings):
# Processing
max_concurrent_jobs: int = int(os.getenv("MAX_CONCURRENT_JOBS", "5"))
job_timeout_seconds: int = int(os.getenv("JOB_TIMEOUT_SECONDS", "3600"))
# Agentic AI Correction (for lyrics correction via LLM)
# When enabled, uses Gemini via Vertex AI for intelligent lyrics correction
use_agentic_ai: bool = os.getenv("USE_AGENTIC_AI", "true").lower() in ("true", "1", "yes")
agentic_ai_model: str = os.getenv("AGENTIC_AI_MODEL", "vertexai/gemini-3-flash-preview")
# Timeout for agentic correction in seconds. If correction takes longer, abort and
# use uncorrected transcription - human review will fix any issues.
agentic_correction_timeout_seconds: int = int(os.getenv("AGENTIC_CORRECTION_TIMEOUT_SECONDS", "180"))
# Cloud Tasks (for scalable worker coordination)
# When enabled, workers are triggered via Cloud Tasks for guaranteed delivery
# When disabled (default), workers are triggered via direct HTTP (for development)
@@ -60,35 +52,9 @@ class Settings(BaseSettings):
# instead of Cloud Tasks. This supports encoding times >30 minutes (up to 24 hours).
# Default is false - Cloud Tasks is sufficient for most videos (15-20 min).
use_cloud_run_jobs_for_video: bool = os.getenv("USE_CLOUD_RUN_JOBS_FOR_VIDEO", "false").lower() in ("true", "1", "yes")
# GCE Encoding Worker (for high-performance video encoding)
# When enabled, video encoding is offloaded to a dedicated C4 GCE instance
# with faster CPU (Intel Granite Rapids 3.9 GHz) instead of Cloud Run.
# This provides 2-3x faster encoding times for CPU-bound FFmpeg libx264 encoding.
use_gce_encoding: bool = os.getenv("USE_GCE_ENCODING", "false").lower() in ("true", "1", "yes")
encoding_worker_url: Optional[str] = os.getenv("ENCODING_WORKER_URL") # e.g., http://136.119.50.148:8080
encoding_worker_api_key: Optional[str] = os.getenv("ENCODING_WORKER_API_KEY")
# GCE Preview Encoding (for faster preview video generation)
# When enabled, preview video encoding during lyrics review is offloaded to the GCE worker.
# This reduces preview generation time from 60+ seconds to ~15-20 seconds.
# Requires use_gce_encoding to be enabled and the GCE worker to support /encode-preview endpoint.
use_gce_preview_encoding: bool = os.getenv("USE_GCE_PREVIEW_ENCODING", "false").lower() in ("true", "1", "yes")
# Storage paths
temp_dir: str = os.getenv("TEMP_DIR", "/tmp/karaoke-gen")
# Worker logs storage mode
# When enabled, worker logs are stored in a Firestore subcollection (jobs/{job_id}/logs)
# instead of an embedded array. This avoids the 1MB document size limit.
# Default is true for new deployments.
use_log_subcollection: bool = os.getenv("USE_LOG_SUBCOLLECTION", "true").lower() in ("true", "1", "yes")
# Flacfetch remote service (for torrent downloads)
# When configured, audio search uses the remote flacfetch HTTP API instead of local flacfetch.
# This is required for torrent downloads since Cloud Run doesn't support BitTorrent.
flacfetch_api_url: Optional[str] = os.getenv("FLACFETCH_API_URL") # e.g., http://10.0.0.5:8080
flacfetch_api_key: Optional[str] = os.getenv("FLACFETCH_API_KEY")
# Default distribution settings (can be overridden per-request)
default_dropbox_path: Optional[str] = os.getenv("DEFAULT_DROPBOX_PATH")
@@ -97,41 +63,7 @@ class Settings(BaseSettings):
default_discord_webhook_url: Optional[str] = (
os.getenv("DEFAULT_DISCORD_WEBHOOK_URL", "").strip() or None
)
# Default values for web service jobs (YouTube/Dropbox distribution)
default_enable_youtube_upload: bool = os.getenv("DEFAULT_ENABLE_YOUTUBE_UPLOAD", "false").lower() in ("true", "1", "yes")
default_brand_prefix: Optional[str] = os.getenv("DEFAULT_BRAND_PREFIX")
# Rate Limiting Configuration
# Enable/disable rate limiting system-wide (useful for development)
enable_rate_limiting: bool = os.getenv("ENABLE_RATE_LIMITING", "true").lower() in ("true", "1", "yes")
# Maximum jobs a user can create per day (0 = unlimited)
rate_limit_jobs_per_day: int = int(os.getenv("RATE_LIMIT_JOBS_PER_DAY", "5"))
# Maximum YouTube uploads system-wide per day (0 = unlimited)
rate_limit_youtube_uploads_per_day: int = int(os.getenv("RATE_LIMIT_YOUTUBE_UPLOADS_PER_DAY", "10"))
# Maximum beta enrollments from same IP per day (0 = unlimited)
rate_limit_beta_ip_per_day: int = int(os.getenv("RATE_LIMIT_BETA_IP_PER_DAY", "1"))
default_youtube_description: str = os.getenv(
"DEFAULT_YOUTUBE_DESCRIPTION",
"Karaoke video created with Nomad Karaoke (https://nomadkaraoke.com)\n\n"
"AI-powered vocal separation and synchronized lyrics.\n\n"
"#karaoke #music #singing #instrumental #lyrics"
)
# Default CDG/TXT generation settings
# When True, CDG and TXT packages are generated by default (when a theme is set)
# These can be overridden per-request via explicit enable_cdg/enable_txt parameters
default_enable_cdg: bool = os.getenv("DEFAULT_ENABLE_CDG", "true").lower() in ("true", "1", "yes")
default_enable_txt: bool = os.getenv("DEFAULT_ENABLE_TXT", "true").lower() in ("true", "1", "yes")
# Push Notifications Configuration
# When enabled, users can subscribe to push notifications for job status updates
enable_push_notifications: bool = os.getenv("ENABLE_PUSH_NOTIFICATIONS", "false").lower() in ("true", "1", "yes")
# Maximum number of push subscriptions per user (oldest removed when exceeded)
max_push_subscriptions_per_user: int = int(os.getenv("MAX_PUSH_SUBSCRIPTIONS_PER_USER", "5"))
# VAPID subject (email or URL for push service to contact)
vapid_subject: str = os.getenv("VAPID_SUBJECT", "mailto:gen@nomadkaraoke.com")
# Secret Manager cache
_secret_cache: Dict[str, str] = {}
-66
View File
@@ -1,66 +0,0 @@
"""
Custom exceptions for the karaoke generation backend.
These exceptions are used for structured error handling across the application,
particularly for rate limiting and validation errors.
"""
class RateLimitExceededError(Exception):
"""
Raised when a rate limit is exceeded.
Includes information about when the limit will reset for client retry logic.
Attributes:
message: Human-readable error message
limit_type: Type of limit exceeded (e.g., "jobs_per_day", "youtube_uploads")
remaining_seconds: Seconds until the limit resets (for Retry-After header)
current_count: Current usage count
limit_value: The limit that was exceeded
"""
def __init__(
self,
message: str,
limit_type: str = "unknown",
remaining_seconds: int = 0,
current_count: int = 0,
limit_value: int = 0
):
self.message = message
self.limit_type = limit_type
self.remaining_seconds = remaining_seconds
self.current_count = current_count
self.limit_value = limit_value
super().__init__(message)
class EmailValidationError(Exception):
"""
Raised when email validation fails.
Attributes:
message: Human-readable error message
reason: Specific reason for failure (e.g., "disposable", "blocked", "invalid")
"""
def __init__(self, message: str, reason: str = "invalid"):
self.message = message
self.reason = reason
super().__init__(message)
class IPBlockedError(Exception):
"""
Raised when a request comes from a blocked IP address.
Attributes:
message: Human-readable error message
ip_address: The blocked IP address (may be partially masked)
"""
def __init__(self, message: str, ip_address: str = ""):
self.message = message
self.ip_address = ip_address
super().__init__(message)
+7 -67
View File
@@ -7,22 +7,18 @@ from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from backend.config import settings
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users, admin, tenant, rate_limits, push
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
from backend.services.structured_logging import setup_structured_logging
from backend.services.spacy_preloader import preload_spacy_model
from backend.services.nltk_preloader import preload_all_nltk_resources
from backend.services.langfuse_preloader import preload_langfuse_handler
from backend.middleware.audit_logging import AuditLoggingMiddleware
from backend.middleware.tenant import TenantMiddleware
from backend.version import VERSION
# Configure structured logging (JSON in Cloud Run, human-readable locally)
# This must happen before any logging calls
setup_structured_logging()
# Configure logging with trace ID support
logging.basicConfig(
level=getattr(logging, settings.log_level.upper()),
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
# Initialize OpenTelemetry tracing (must happen before app creation)
@@ -71,28 +67,7 @@ async def lifespan(app: FastAPI):
logger.info(f"Environment: {settings.environment}")
logger.info(f"GCS Bucket: {settings.gcs_bucket_name}")
logger.info(f"Tracing enabled: {tracing_enabled}")
# Preload NLP models and resources to avoid cold start delays
# See docs/archive/2026-01-08-performance-investigation.md for background
# 1. SpaCy model (60+ second delay without preload)
try:
preload_spacy_model("en_core_web_sm")
except Exception as e:
logger.warning(f"SpaCy preload failed (will load lazily): {e}")
# 2. NLTK cmudict (50-100+ second delay without preload)
try:
preload_all_nltk_resources()
except Exception as e:
logger.warning(f"NLTK preload failed (will load lazily): {e}")
# 3. Langfuse callback handler (200+ second delay without preload)
try:
preload_langfuse_handler()
except Exception as e:
logger.warning(f"Langfuse preload failed (will initialize lazily): {e}")
# Validate OAuth credentials (non-blocking)
try:
await validate_credentials_on_startup()
@@ -126,12 +101,6 @@ app.add_middleware(
allow_headers=["*"],
)
# Add audit logging middleware (captures all requests with request_id for correlation)
app.add_middleware(AuditLoggingMiddleware)
# Add tenant detection middleware (extracts tenant from subdomain/headers)
app.add_middleware(TenantMiddleware)
# Include routers
app.include_router(health.router, prefix="/api")
app.include_router(jobs.router, prefix="/api")
@@ -140,35 +109,6 @@ app.include_router(internal.router, prefix="/api") # Internal worker endpoints
app.include_router(review.router, prefix="/api") # Review UI compatibility endpoints
app.include_router(auth.router, prefix="/api") # OAuth credential management
app.include_router(audio_search.router, prefix="/api") # Audio search (artist+title mode)
app.include_router(themes.router, prefix="/api") # Theme selection for styles
app.include_router(users.router, prefix="/api") # User auth, credits, and Stripe webhooks
app.include_router(admin.router, prefix="/api") # Admin dashboard and management
app.include_router(rate_limits.router, prefix="/api") # Rate limits admin management
app.include_router(push.router, prefix="/api") # Push notification subscription management
app.include_router(tenant.router) # Tenant/white-label configuration (no /api prefix, router has it)
# Exception handler for rate limiting
from fastapi import Request
from fastapi.responses import JSONResponse
from backend.exceptions import RateLimitExceededError
@app.exception_handler(RateLimitExceededError)
async def rate_limit_exception_handler(request: Request, exc: RateLimitExceededError):
"""Handle rate limit exceeded errors with 429 status."""
return JSONResponse(
status_code=429,
content={
"detail": exc.message,
"limit_type": exc.limit_type,
"current_count": exc.current_count,
"limit_value": exc.limit_value,
},
headers={
"Retry-After": str(exc.remaining_seconds),
} if exc.remaining_seconds > 0 else None,
)
@app.get("/")
-11
View File
@@ -1,11 +0,0 @@
"""Middleware package for FastAPI application."""
from backend.middleware.audit_logging import AuditLoggingMiddleware
from backend.middleware.tenant import TenantMiddleware, get_tenant_from_request, get_tenant_config_from_request
__all__ = [
"AuditLoggingMiddleware",
"TenantMiddleware",
"get_tenant_from_request",
"get_tenant_config_from_request",
]
-124
View File
@@ -1,124 +0,0 @@
"""
Audit logging middleware for tracking all HTTP requests.
This middleware captures request metadata and logs it to Cloud Logging
for audit trail and user activity investigation purposes.
"""
import logging
import time
import uuid
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response
logger = logging.getLogger("audit")
# Endpoints to exclude from audit logging (high-frequency health checks)
EXCLUDED_PATHS = {
"/",
"/api/health",
"/api/health/detailed",
"/api/readiness",
"/healthz",
"/ready",
}
class AuditLoggingMiddleware(BaseHTTPMiddleware):
"""
Middleware that logs all HTTP requests for audit purposes.
Captures:
- request_id: UUID for correlating with auth logs
- method: HTTP method
- path: Request path
- status_code: Response status
- latency_ms: Request duration
- client_ip: Client IP (from X-Forwarded-For for proxied requests)
- user_agent: Browser/client identifier
The request_id is stored in request.state and added to response headers,
allowing correlation with auth logs that capture user_email.
"""
async def dispatch(self, request: Request, call_next) -> Response:
# Skip excluded paths (health checks)
if request.url.path in EXCLUDED_PATHS:
return await call_next(request)
# Generate unique request ID for correlation
request_id = str(uuid.uuid4())
request.state.request_id = request_id
# Capture start time
start_time = time.time()
# Extract client info
client_ip = self._get_client_ip(request)
user_agent = request.headers.get("user-agent", "")
# Process request
try:
response = await call_next(request)
except Exception:
# Log failed requests too (exception() auto-includes stack trace)
latency_ms = int((time.time() - start_time) * 1000)
logger.exception(
"request_audit_error",
extra={
"request_id": request_id,
"method": request.method,
"path": request.url.path,
"query_string": str(request.query_params) if request.query_params else None,
"latency_ms": latency_ms,
"client_ip": client_ip,
"user_agent": user_agent[:200] if user_agent else None,
"audit_type": "http_request",
}
)
raise
# Calculate latency
latency_ms = int((time.time() - start_time) * 1000)
# Log audit entry (INFO level for successful requests)
log_level = logging.WARNING if response.status_code >= 400 else logging.INFO
logger.log(
log_level,
"request_audit",
extra={
"request_id": request_id,
"method": request.method,
"path": request.url.path,
"query_string": str(request.query_params) if request.query_params else None,
"status_code": response.status_code,
"latency_ms": latency_ms,
"client_ip": client_ip,
"user_agent": user_agent[:200] if user_agent else None,
"audit_type": "http_request",
}
)
# Add request_id to response headers for debugging/correlation
response.headers["X-Request-ID"] = request_id
return response
def _get_client_ip(self, request: Request) -> str:
"""
Extract client IP address, handling proxy scenarios.
Cloud Run and other proxies set X-Forwarded-For header.
"""
# Check X-Forwarded-For for proxy scenarios (Cloud Run, load balancers)
forwarded = request.headers.get("x-forwarded-for", "")
if forwarded:
# First IP is the original client
return forwarded.split(",")[0].strip()
# Fall back to direct connection
if request.client:
return request.client.host
return "unknown"
-192
View File
@@ -1,192 +0,0 @@
"""
Tenant detection middleware for white-label B2B portals.
This middleware extracts the tenant ID from incoming requests and attaches
the tenant configuration to the request state for use by downstream handlers.
Tenant detection priority:
1. X-Tenant-ID header (explicitly set by frontend)
2. `tenant` query parameter (for development/testing)
3. Host header subdomain detection (production flow)
The middleware is non-blocking - if no tenant is detected, the request
proceeds as a default Nomad Karaoke request (tenant_id = None).
"""
import logging
import os
from typing import Optional
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response
from backend.services.tenant_service import get_tenant_service
from backend.models.tenant import TenantConfig
logger = logging.getLogger(__name__)
# Only allow query param tenant override in non-production environments
IS_PRODUCTION = os.environ.get("ENV", "").lower() == "production" or \
os.environ.get("ENVIRONMENT", "").lower() == "production"
# Known non-tenant subdomains that should be treated as default Nomad Karaoke
NON_TENANT_SUBDOMAINS = {"gen", "api", "www", "buy", "admin", "app", "beta"}
class TenantMiddleware(BaseHTTPMiddleware):
"""
Middleware that detects tenant from request and attaches config to state.
After this middleware runs, routes can access:
- request.state.tenant_id: str or None
- request.state.tenant_config: TenantConfig or None
Usage in route handlers:
```python
@router.get("/something")
async def handler(request: Request):
tenant_id = getattr(request.state, "tenant_id", None)
tenant_config = getattr(request.state, "tenant_config", None)
```
"""
async def dispatch(self, request: Request, call_next) -> Response:
# Extract tenant ID from request
tenant_id = self._extract_tenant_id(request)
# Load tenant config if tenant detected
tenant_config: Optional[TenantConfig] = None
if tenant_id:
tenant_service = get_tenant_service()
tenant_config = tenant_service.get_tenant_config(tenant_id)
if tenant_config and not tenant_config.is_active:
# Tenant exists but is inactive - treat as default
logger.warning(f"Inactive tenant requested: {tenant_id}")
tenant_id = None
tenant_config = None
# Attach to request state
request.state.tenant_id = tenant_id
request.state.tenant_config = tenant_config
# Log tenant detection for debugging
if tenant_id:
logger.debug(f"Request tenant: {tenant_id} (path: {request.url.path})")
# Process request
response = await call_next(request)
# Add tenant ID to response headers for debugging
if tenant_id:
response.headers["X-Tenant-ID"] = tenant_id
return response
def _extract_tenant_id(self, request: Request) -> Optional[str]:
"""
Extract tenant ID from request using priority-based detection.
Returns:
Tenant ID if detected, None for default Nomad Karaoke
"""
# Priority 1: X-Tenant-ID header (explicitly set by frontend)
header_tenant = request.headers.get("X-Tenant-ID")
if header_tenant:
return header_tenant.lower().strip()
# Priority 2: Query parameter (for development/testing only)
# Disabled in production to prevent tenant spoofing
if not IS_PRODUCTION:
query_tenant = request.query_params.get("tenant")
if query_tenant:
return query_tenant.lower().strip()
# Priority 3: Host header subdomain detection
host = request.headers.get("Host", "")
return self._extract_tenant_from_host(host)
def _extract_tenant_from_host(self, host: str) -> Optional[str]:
"""
Extract tenant ID from Host header subdomain.
Patterns handled:
- vocalstar.nomadkaraoke.com -> vocalstar
- vocalstar.gen.nomadkaraoke.com -> vocalstar
- localhost:3000 -> None (local dev)
- gen.nomadkaraoke.com -> None (main app)
- api.nomadkaraoke.com -> None (API)
Returns:
Tenant ID if subdomain matches tenant, None otherwise
"""
if not host:
return None
# Normalize
host_lower = host.lower()
# Remove port if present
if ":" in host_lower:
host_lower = host_lower.split(":")[0]
# Check if this is a nomadkaraoke.com domain
if "nomadkaraoke.com" not in host_lower:
return None
# Split into parts
parts = host_lower.split(".")
# Need at least 3 parts: subdomain.nomadkaraoke.com
if len(parts) < 3:
return None
# First part is the potential tenant ID
potential_tenant = parts[0]
# Skip known non-tenant subdomains
if potential_tenant in NON_TENANT_SUBDOMAINS:
return None
# Verify tenant exists in GCS
tenant_service = get_tenant_service()
if tenant_service.tenant_exists(potential_tenant):
return potential_tenant
return None
def get_tenant_from_request(request: Request) -> Optional[str]:
"""
Helper function to get tenant ID from request state.
Usage in route handlers:
```python
from backend.middleware.tenant import get_tenant_from_request
@router.get("/something")
async def handler(request: Request):
tenant_id = get_tenant_from_request(request)
```
"""
return getattr(request.state, "tenant_id", None)
def get_tenant_config_from_request(request: Request) -> Optional[TenantConfig]:
"""
Helper function to get tenant config from request state.
Usage in route handlers:
```python
from backend.middleware.tenant import get_tenant_config_from_request
@router.get("/something")
async def handler(request: Request):
config = get_tenant_config_from_request(request)
if config and not config.features.audio_search:
raise HTTPException(403, "Audio search not available")
```
"""
return getattr(request.state, "tenant_config", None)
+15 -89
View File
@@ -9,8 +9,6 @@ from datetime import datetime
from enum import Enum
from pydantic import BaseModel, Field, validator
from karaoke_gen.utils import normalize_text
class JobStatus(str, Enum):
"""
@@ -136,23 +134,17 @@ STATE_TRANSITIONS = {
JobStatus.UPLOADING: [JobStatus.NOTIFYING, JobStatus.COMPLETE, JobStatus.FAILED],
JobStatus.NOTIFYING: [JobStatus.COMPLETE, JobStatus.FAILED],
# Terminal states - COMPLETE, PREP_COMPLETE have no transitions
# FAILED and CANCELLED allow retry transitions to resume from checkpoints
# Terminal states - COMPLETE, PREP_COMPLETE, and CANCELLED have no transitions
# FAILED allows retry transitions to resume from checkpoints
# PREP_COMPLETE allows finalise-only continuation
JobStatus.COMPLETE: [],
JobStatus.PREP_COMPLETE: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED], # Finalise-only continues from here
JobStatus.FAILED: [
JobStatus.DOWNLOADING, # Retry from beginning (if input audio exists)
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
JobStatus.REVIEW_COMPLETE, # Retry from render stage
JobStatus.LYRICS_COMPLETE, # Retry from screens generation
],
JobStatus.CANCELLED: [
JobStatus.DOWNLOADING, # Retry from beginning (if input audio exists)
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
JobStatus.REVIEW_COMPLETE, # Retry from render stage
JobStatus.LYRICS_COMPLETE, # Retry from screens generation
],
JobStatus.CANCELLED: [],
# Legacy states (for backward compatibility)
JobStatus.QUEUED: [JobStatus.PENDING],
@@ -200,29 +192,14 @@ class Job(BaseModel):
input_media_gcs_path: Optional[str] = None # GCS path to uploaded file
# User preferences
enable_cdg: bool = False # Generate CDG+MP3 package (requires style config)
enable_txt: bool = False # Generate TXT+MP3 package (requires style config)
enable_cdg: bool = True # Generate CDG+MP3 package
enable_txt: bool = True # Generate TXT+MP3 package
enable_youtube_upload: bool = False # Upload to YouTube
youtube_description: Optional[str] = None # YouTube video description
webhook_url: Optional[str] = None # Webhook for notifications
user_email: Optional[str] = None # Email for notifications
non_interactive: bool = False # Skip interactive steps (lyrics review, instrumental selection)
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None # Tenant ID for white-label portal scoping
# Theme configuration (pre-made themes from GCS)
theme_id: Optional[str] = None # Theme identifier (e.g., "nomad", "default")
color_overrides: Dict[str, str] = Field(default_factory=dict)
"""
User color overrides applied on top of theme. Keys:
- artist_color: Hex color for artist name (#RRGGBB)
- title_color: Hex color for song title
- sung_lyrics_color: Hex color for highlighted lyrics
- unsung_lyrics_color: Hex color for unhighlighted lyrics
"""
# Style configuration (uploaded files - used when theme_id is not set)
# Style configuration (uploaded files)
style_params_gcs_path: Optional[str] = None # GCS path to style_params.json
style_assets: Dict[str, str] = Field(default_factory=dict)
"""
@@ -274,21 +251,6 @@ class Job(BaseModel):
finalise_only: bool = False # Skip prep, run only finalisation (requires uploaded prep outputs)
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Review authentication (Batch 7)
review_token: Optional[str] = None # Job-scoped token for lyrics review UI access (generated when entering AWAITING_REVIEW)
review_token_expires_at: Optional[datetime] = None # Token expiry time (optional, for extra security)
instrumental_token: Optional[str] = None # Job-scoped token for instrumental review UI access (generated when entering AWAITING_INSTRUMENTAL_SELECTION)
instrumental_token_expires_at: Optional[datetime] = None # Token expiry time
# Made-for-you order tracking
made_for_you: bool = False # Flag indicating this is a made-for-you customer order
customer_email: Optional[str] = None # Customer email for final delivery (job owned by admin during processing)
customer_notes: Optional[str] = None # Notes provided by customer with their order
# Output deletion tracking (for admin cleanup without deleting job)
outputs_deleted_at: Optional[datetime] = None # Timestamp when outputs were deleted by admin
outputs_deleted_by: Optional[str] = None # Admin email who deleted outputs
# Processing state
track_output_dir: Optional[str] = None # Local output directory (temp)
audio_hash: Optional[str] = None # Hash for deduplication
@@ -426,28 +388,16 @@ class JobCreate(BaseModel):
artist: Optional[str] = None
title: Optional[str] = None
filename: Optional[str] = None # Original uploaded filename
# Optional preferences
enable_cdg: bool = False # Requires style config
enable_txt: bool = False # Requires style config
enable_cdg: bool = True
enable_txt: bool = True
enable_youtube_upload: bool = False
youtube_description: Optional[str] = None
webhook_url: Optional[str] = None
user_email: Optional[str] = None
non_interactive: bool = False # Skip interactive steps (lyrics review, instrumental selection)
# Theme configuration (pre-made themes from GCS)
theme_id: Optional[str] = None # Theme identifier (e.g., "nomad", "default")
color_overrides: Dict[str, str] = Field(default_factory=dict)
"""
User color overrides applied on top of theme. Keys:
- artist_color: Hex color for artist name (#RRGGBB)
- title_color: Hex color for song title
- sung_lyrics_color: Hex color for highlighted lyrics
- unsung_lyrics_color: Hex color for unhighlighted lyrics
"""
# Style configuration (will be populated after file upload, or from theme)
# Style configuration (will be populated after file upload)
style_params_gcs_path: Optional[str] = None
style_assets: Dict[str, str] = Field(default_factory=dict)
@@ -486,12 +436,7 @@ class JobCreate(BaseModel):
prep_only: bool = False # Stop after review, don't run finalisation
finalise_only: bool = False # Skip prep, run only finalisation
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Made-for-you order tracking
made_for_you: bool = False # Flag indicating this is a made-for-you customer order
customer_email: Optional[str] = None # Customer email for final delivery (job owned by admin during processing)
customer_notes: Optional[str] = None # Notes provided by customer with their order
# Request metadata (set by API endpoint from request headers)
request_metadata: Dict[str, Any] = Field(default_factory=dict)
"""
@@ -503,33 +448,14 @@ class JobCreate(BaseModel):
- server_version: Current server version
- custom_headers: All X-* headers
"""
# Tenant scoping for white-label portals
tenant_id: Optional[str] = None # Tenant ID for job scoping
@validator('url')
def validate_url(cls, v):
"""Validate URL is not empty."""
@validator('url', 'artist', 'title')
def validate_inputs(cls, v):
"""Validate string inputs are not empty."""
if v is not None and isinstance(v, str) and not v.strip():
raise ValueError("Field cannot be empty string")
return v.strip() if isinstance(v, str) else v
@validator('artist', 'title')
def normalize_artist_title(cls, v):
"""Normalize artist/title text to standardize Unicode characters.
This ensures consistent data storage by converting:
- Curly quotes -> straight quotes
- Various dashes -> hyphen
- Unusual whitespace -> regular space
"""
if v is not None and isinstance(v, str):
if not v.strip():
raise ValueError("Field cannot be empty string")
# normalize_text handles stripping and Unicode normalization
return normalize_text(v)
return v
class JobResponse(BaseModel):
"""Job response model."""
+7 -7
View File
@@ -12,9 +12,9 @@ class URLSubmissionRequest(BaseModel):
title: Optional[str] = None # Auto-detected if not provided
# Optional preferences
enable_cdg: bool = False # Requires style config
enable_txt: bool = False # Requires style config
enable_youtube_upload: Optional[bool] = None # None = use server default
enable_cdg: bool = True
enable_txt: bool = True
enable_youtube_upload: bool = False
youtube_description: Optional[str] = None
webhook_url: Optional[str] = None
user_email: Optional[str] = None
@@ -24,11 +24,11 @@ class UploadSubmissionRequest(BaseModel):
"""Request to submit a job from an uploaded file."""
artist: str
title: str
# Optional preferences
enable_cdg: bool = False # Requires style config
enable_txt: bool = False # Requires style config
enable_youtube_upload: Optional[bool] = None # None = use server default
enable_cdg: bool = True
enable_txt: bool = True
enable_youtube_upload: bool = False
youtube_description: Optional[str] = None
webhook_url: Optional[str] = None
user_email: Optional[str] = None
-208
View File
@@ -1,208 +0,0 @@
"""
Tenant data models for white-label B2B portals.
Tenants are B2B customers who get their own branded portal with custom
configuration. Each tenant has:
- Custom branding (logo, colors, site title)
- Feature flags (enable/disable audio search, distribution, etc.)
- Default settings (theme, distribution mode)
- Auth configuration (allowed email domains, fixed tokens)
Tenant configs are stored in GCS at tenants/{tenant_id}/config.json
"""
from datetime import datetime
from typing import List, Optional
from pydantic import BaseModel, Field
class TenantBranding(BaseModel):
"""Branding configuration for a tenant's portal."""
logo_url: Optional[str] = Field(
None, description="URL to tenant's logo image (PNG with transparency preferred)"
)
logo_height: int = Field(40, description="Logo height in pixels for header display")
primary_color: str = Field(
"#ff5bb8", description="Primary brand color (hex format)"
)
secondary_color: str = Field(
"#8b5cf6", description="Secondary brand color (hex format)"
)
accent_color: Optional[str] = Field(
None, description="Accent color for highlights (hex format)"
)
background_color: Optional[str] = Field(
None, description="Custom background color (defaults to dark theme)"
)
favicon_url: Optional[str] = Field(None, description="URL to custom favicon")
site_title: str = Field(
"Karaoke Generator", description="Browser tab title"
)
tagline: Optional[str] = Field(
None, description="Optional tagline shown below logo"
)
class TenantFeatures(BaseModel):
"""Feature flags controlling what's available in the tenant's portal."""
# Audio input methods
audio_search: bool = Field(
True, description="Enable audio search (flacfetch integration)"
)
file_upload: bool = Field(True, description="Enable direct file upload")
youtube_url: bool = Field(True, description="Enable YouTube URL input")
# Distribution options
youtube_upload: bool = Field(True, description="Enable YouTube upload option")
dropbox_upload: bool = Field(True, description="Enable Dropbox upload option")
gdrive_upload: bool = Field(True, description="Enable Google Drive upload option")
# Customization options
theme_selection: bool = Field(
True, description="Allow user to select theme (false = use default)"
)
color_overrides: bool = Field(
True, description="Allow user to customize colors"
)
# Output formats
enable_cdg: bool = Field(True, description="Generate CDG format output")
enable_4k: bool = Field(True, description="Generate 4K video output")
# Advanced features
admin_access: bool = Field(
False, description="Allow access to admin dashboard"
)
class TenantDefaults(BaseModel):
"""Default settings applied to all jobs for this tenant."""
theme_id: Optional[str] = Field(
None, description="Default theme ID (required if theme_selection=false)"
)
locked_theme: Optional[str] = Field(
None,
description="If set, users cannot change theme - always uses this theme ID"
)
distribution_mode: str = Field(
"all",
description="Distribution mode: 'all', 'download_only', or 'cloud_only'"
)
brand_prefix: Optional[str] = Field(
None, description="Prefix for output filenames (e.g., 'VSTAR')"
)
youtube_description_template: Optional[str] = Field(
None, description="Default YouTube description template"
)
class TenantAuth(BaseModel):
"""Authentication configuration for a tenant."""
allowed_email_domains: List[str] = Field(
default_factory=list,
description="Email domains allowed for magic link auth (e.g., ['vocal-star.com'])"
)
require_email_domain: bool = Field(
True,
description="If true, only allowed domains can sign up. If false, domains get auto-approved."
)
fixed_token_ids: List[str] = Field(
default_factory=list,
description="IDs of fixed API tokens for this tenant (tokens stored in auth_tokens)"
)
sender_email: Optional[str] = Field(
None,
description="Email sender address for this tenant (e.g., 'vocalstar@nomadkaraoke.com')"
)
class TenantConfig(BaseModel):
"""Complete configuration for a white-label tenant."""
id: str = Field(..., description="Unique tenant identifier (e.g., 'vocalstar')")
name: str = Field(..., description="Display name (e.g., 'Vocal Star')")
subdomain: str = Field(
..., description="Full subdomain (e.g., 'vocalstar.nomadkaraoke.com')"
)
is_active: bool = Field(True, description="Whether tenant portal is active")
branding: TenantBranding = Field(default_factory=TenantBranding)
features: TenantFeatures = Field(default_factory=TenantFeatures)
defaults: TenantDefaults = Field(default_factory=TenantDefaults)
auth: TenantAuth = Field(default_factory=TenantAuth)
created_at: Optional[datetime] = None
updated_at: Optional[datetime] = None
def get_sender_email(self) -> str:
"""Get the email sender address for this tenant."""
if self.auth.sender_email:
return self.auth.sender_email
# Default pattern: {tenant_id}@nomadkaraoke.com
return f"{self.id}@nomadkaraoke.com"
def is_email_allowed(self, email: str) -> bool:
"""Check if an email address is allowed for this tenant."""
if not self.auth.allowed_email_domains:
# No domain restrictions
return True
email_lower = email.lower()
for domain in self.auth.allowed_email_domains:
if email_lower.endswith(f"@{domain.lower()}"):
return True
return not self.auth.require_email_domain
class TenantPublicConfig(BaseModel):
"""
Public tenant configuration returned to frontend.
This excludes sensitive auth details like token IDs.
"""
id: str
name: str
subdomain: str
is_active: bool
branding: TenantBranding
features: TenantFeatures
defaults: TenantDefaults
# Auth info limited to what frontend needs
allowed_email_domains: List[str] = Field(default_factory=list)
@classmethod
def from_config(cls, config: TenantConfig) -> "TenantPublicConfig":
"""Create public config from full config."""
return cls(
id=config.id,
name=config.name,
subdomain=config.subdomain,
is_active=config.is_active,
branding=config.branding,
features=config.features,
defaults=TenantDefaults(
theme_id=config.defaults.theme_id,
locked_theme=config.defaults.locked_theme,
distribution_mode=config.defaults.distribution_mode,
# Don't expose brand_prefix or youtube_description_template
),
allowed_email_domains=config.auth.allowed_email_domains,
)
class TenantConfigResponse(BaseModel):
"""Response from GET /api/tenant/config endpoint."""
tenant: Optional[TenantPublicConfig] = Field(
None, description="Tenant config if found, null for default Nomad Karaoke"
)
is_default: bool = Field(
True, description="True if using default Nomad Karaoke config"
)
-153
View File
@@ -1,153 +0,0 @@
"""
Theme data models for karaoke-gen.
Themes are pre-created style configurations stored in GCS that users can select
when creating karaoke jobs. Each theme includes:
- Style parameters (fonts, colors, backgrounds, layouts)
- Asset files (images, fonts)
- Preview images
- Optional YouTube description template
"""
from datetime import datetime
from typing import Any, Dict, List, Optional
from pydantic import BaseModel, Field
class ThemeSummary(BaseModel):
"""Summary of a theme for listing purposes."""
id: str = Field(..., description="Unique theme identifier (e.g., 'nomad', 'default')")
name: str = Field(..., description="Human-readable theme name")
description: str = Field(..., description="Brief description of the theme style")
preview_url: Optional[str] = Field(None, description="Signed URL for preview image")
thumbnail_url: Optional[str] = Field(None, description="Signed URL for smaller thumbnail")
is_default: bool = Field(False, description="Whether this is the default theme")
class ThemeMetadata(BaseModel):
"""Metadata for a single theme in the registry."""
id: str
name: str
description: str
is_default: bool = False
created_at: Optional[datetime] = None
class ThemeRegistry(BaseModel):
"""Registry of all available themes, stored as _metadata.json in GCS."""
version: int = 1
themes: List[ThemeMetadata] = Field(default_factory=list)
class ThemeDetail(BaseModel):
"""Full theme details including style parameters."""
id: str
name: str
description: str
preview_url: Optional[str] = None
is_default: bool = False
style_params: Dict[str, Any] = Field(default_factory=dict)
has_youtube_description: bool = False
class ColorOverrides(BaseModel):
"""
User-customizable color overrides that can be applied on top of a theme.
All colors are hex format (#RRGGBB). When applied:
- artist_color: Applied to intro.artist_color, end.artist_color, cdg.artist_color
- title_color: Applied to intro.title_color, end.title_color, cdg.title_color
- sung_lyrics_color: Applied to karaoke.primary_color (converted to RGBA), cdg.active_fill
- unsung_lyrics_color: Applied to karaoke.secondary_color (converted to RGBA), cdg.inactive_fill
"""
artist_color: Optional[str] = Field(
None,
pattern=r"^#[0-9A-Fa-f]{6}$",
description="Hex color for artist name on intro/end screens",
)
title_color: Optional[str] = Field(
None,
pattern=r"^#[0-9A-Fa-f]{6}$",
description="Hex color for song title on intro/end screens",
)
sung_lyrics_color: Optional[str] = Field(
None,
pattern=r"^#[0-9A-Fa-f]{6}$",
description="Hex color for lyrics being sung (highlighted)",
)
unsung_lyrics_color: Optional[str] = Field(
None,
pattern=r"^#[0-9A-Fa-f]{6}$",
description="Hex color for lyrics not yet sung",
)
def has_overrides(self) -> bool:
"""Check if any color overrides are set."""
return any(
[
self.artist_color,
self.title_color,
self.sung_lyrics_color,
self.unsung_lyrics_color,
]
)
def to_dict(self) -> Dict[str, str]:
"""Convert to dict, excluding None values."""
return {k: v for k, v in self.model_dump().items() if v is not None}
class ThemesListResponse(BaseModel):
"""Response from GET /api/themes endpoint."""
themes: List[ThemeSummary]
class ThemeDetailResponse(BaseModel):
"""Response from GET /api/themes/{theme_id} endpoint."""
theme: ThemeDetail
# Utility functions for color conversion
def hex_to_rgba(hex_color: str, alpha: int = 255) -> str:
"""
Convert hex color (#RRGGBB) to RGBA format for ASS subtitles.
ASS subtitle format uses "R, G, B, A" string format.
Args:
hex_color: Hex color string like "#7070F7"
alpha: Alpha value 0-255 (default 255 = opaque)
Returns:
RGBA string like "112, 112, 247, 255"
"""
hex_color = hex_color.lstrip("#")
r = int(hex_color[0:2], 16)
g = int(hex_color[2:4], 16)
b = int(hex_color[4:6], 16)
return f"{r}, {g}, {b}, {alpha}"
def rgba_to_hex(rgba_str: str) -> str:
"""
Convert RGBA format to hex color.
Args:
rgba_str: RGBA string like "112, 112, 247, 255"
Returns:
Hex color string like "#7070F7"
"""
parts = [int(x.strip()) for x in rgba_str.split(",")]
r, g, b = parts[0], parts[1], parts[2]
return f"#{r:02x}{g:02x}{b:02x}"
-290
View File
@@ -1,290 +0,0 @@
"""
User model for authentication and credits.
Supports:
- Magic link authentication (email-based)
- Credit system for karaoke generation
- Role-based access control (user/admin)
- Stripe integration for payments
- Beta tester program with feedback collection
"""
from datetime import datetime, timezone
from enum import Enum
from typing import Optional, List, Dict
from pydantic import BaseModel, Field
class UserRole(str, Enum):
"""User roles for access control."""
USER = "user"
ADMIN = "admin"
class BetaTesterStatus(str, Enum):
"""Status of beta tester participation."""
ACTIVE = "active" # Enrolled, free credits available
PENDING_FEEDBACK = "pending_feedback" # Job completed, awaiting feedback
COMPLETED = "completed" # Feedback submitted
EXPIRED = "expired" # 24hr deadline passed without feedback
class CreditTransaction(BaseModel):
"""Record of a credit transaction."""
id: str
amount: int # Positive = add, negative = deduct
reason: str # e.g., "purchase", "refund", "admin_grant", "job_creation"
job_id: Optional[str] = None
stripe_session_id: Optional[str] = None
created_at: datetime = Field(default_factory=datetime.utcnow)
created_by: Optional[str] = None # Admin email if granted by admin
class PushSubscription(BaseModel):
"""
Web Push subscription for a user's device.
Stores the push subscription endpoint and encryption keys needed
to send push notifications to the user's browser/device.
"""
endpoint: str # Push service endpoint URL
keys: Dict[str, str] # p256dh and auth keys for encryption
device_name: Optional[str] = None # e.g., "iPhone", "Chrome on Windows"
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
last_used_at: Optional[datetime] = None # Last time a notification was sent
class User(BaseModel):
"""
User model stored in Firestore.
Users are identified by email address. Authentication is via magic links
(no passwords). Credits are consumed when creating karaoke jobs.
For multi-tenant white-label portals, users are scoped to a tenant.
Users with tenant_id=None are default Nomad Karaoke users.
"""
email: str # Primary identifier
role: UserRole = UserRole.USER
credits: int = 0
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None
# Stripe integration
stripe_customer_id: Optional[str] = None
# Account state
is_active: bool = True
email_verified: bool = False
# Timestamps
created_at: datetime = Field(default_factory=datetime.utcnow)
updated_at: datetime = Field(default_factory=datetime.utcnow)
last_login_at: Optional[datetime] = None
# Credit history (last 100 transactions)
credit_transactions: List[CreditTransaction] = Field(default_factory=list)
# Job tracking
total_jobs_created: int = 0
total_jobs_completed: int = 0
# Optional profile fields for future use
display_name: Optional[str] = None
# Beta tester program
is_beta_tester: bool = False
beta_tester_status: Optional[BetaTesterStatus] = None
beta_enrolled_at: Optional[datetime] = None
beta_promise_text: Optional[str] = None # User's promise statement
beta_feedback_due_at: Optional[datetime] = None # 24hr after job completion
beta_feedback_email_sent: bool = False
# Push notification subscriptions (Web Push API)
# Users can subscribe from multiple devices/browsers
push_subscriptions: List[PushSubscription] = Field(default_factory=list)
class MagicLinkToken(BaseModel):
"""
Magic link token for passwordless authentication.
Tokens are short-lived (15 minutes) and single-use.
Stored in Firestore with automatic TTL.
For multi-tenant portals, the tenant_id determines which portal
the user will be redirected to after verification.
"""
token: str # Secure random token
email: str
created_at: datetime = Field(default_factory=datetime.utcnow)
expires_at: datetime
used: bool = False
used_at: Optional[datetime] = None
ip_address: Optional[str] = None
user_agent: Optional[str] = None
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None
class Session(BaseModel):
"""
User session for authenticated requests.
Sessions are created after successful magic link verification.
Sessions expire after 7 days of inactivity or 30 days absolute.
For multi-tenant portals, sessions are scoped to a tenant.
"""
token: str # Secure random session token
user_email: str
created_at: datetime = Field(default_factory=datetime.utcnow)
expires_at: datetime
last_activity_at: datetime = Field(default_factory=datetime.utcnow)
ip_address: Optional[str] = None
user_agent: Optional[str] = None
is_active: bool = True
# Multi-tenant support (None = default Nomad Karaoke)
tenant_id: Optional[str] = None
# Pydantic models for API requests/responses
class SendMagicLinkRequest(BaseModel):
"""Request to send a magic link email."""
email: str
class SendMagicLinkResponse(BaseModel):
"""Response after sending magic link."""
status: str
message: str
class VerifyMagicLinkRequest(BaseModel):
"""Request to verify a magic link token."""
token: str
class VerifyMagicLinkResponse(BaseModel):
"""Response after verifying magic link."""
status: str
session_token: str
user: "UserPublic"
message: str
class UserPublic(BaseModel):
"""Public user information (safe to expose to frontend)."""
email: str
role: UserRole
credits: int
display_name: Optional[str] = None
total_jobs_created: int = 0
total_jobs_completed: int = 0
tenant_id: Optional[str] = None
class AddCreditsRequest(BaseModel):
"""Admin request to add credits to a user."""
email: str
amount: int
reason: str = "admin_grant"
class AddCreditsResponse(BaseModel):
"""Response after adding credits."""
status: str
email: str
credits_added: int
new_balance: int
message: str
class UserListResponse(BaseModel):
"""Response for listing users (admin only)."""
users: List[UserPublic]
total: int
# ============================================================================
# Beta Tester Program Models
# ============================================================================
class BetaTesterFeedback(BaseModel):
"""
Feedback from a beta tester after using the service.
Stored in Firestore 'beta_feedback' collection.
"""
id: str # Unique feedback ID
user_email: str
job_id: Optional[str] = None # The job they're providing feedback on
# Ratings (1-5 scale)
overall_rating: int = Field(ge=1, le=5)
ease_of_use_rating: int = Field(ge=1, le=5)
lyrics_accuracy_rating: int = Field(ge=1, le=5)
correction_experience_rating: int = Field(ge=1, le=5)
# Open-ended feedback
what_went_well: Optional[str] = None
what_could_improve: Optional[str] = None
additional_comments: Optional[str] = None
# Would they recommend / use again?
would_recommend: bool = True
would_use_again: bool = True
# Metadata
created_at: datetime = Field(default_factory=datetime.utcnow)
submitted_via: str = "web" # 'web' or 'email'
class BetaTesterEnrollRequest(BaseModel):
"""Request to enroll as a beta tester."""
email: str
promise_text: str = Field(
min_length=10,
description="User's promise to provide feedback (min 10 chars)"
)
accept_corrections_work: bool = Field(
description="User accepts they may need to review/correct lyrics"
)
class BetaTesterEnrollResponse(BaseModel):
"""Response after enrolling as beta tester."""
status: str
message: str
credits_granted: int
session_token: Optional[str] = None # If new user, provide session
class BetaFeedbackRequest(BaseModel):
"""Request to submit beta tester feedback."""
job_id: Optional[str] = None
# Ratings (1-5 scale)
overall_rating: int = Field(ge=1, le=5)
ease_of_use_rating: int = Field(ge=1, le=5)
lyrics_accuracy_rating: int = Field(ge=1, le=5)
correction_experience_rating: int = Field(ge=1, le=5)
# Open-ended feedback
what_went_well: Optional[str] = None
what_could_improve: Optional[str] = None
additional_comments: Optional[str] = None
# Would they recommend / use again?
would_recommend: bool = True
would_use_again: bool = True
class BetaFeedbackResponse(BaseModel):
"""Response after submitting feedback."""
status: str
message: str
bonus_credits: int = 0 # Bonus credits for great feedback
-164
View File
@@ -1,164 +0,0 @@
"""
Worker log entry model for subcollection storage.
This module defines the LogEntry model for storing worker logs in a Firestore
subcollection instead of an embedded array. This avoids the 1MB document size
limit that caused job 501258e1 to fail when logs reached 1.26 MB.
"""
from dataclasses import dataclass, field
from datetime import datetime, timezone, timedelta
from typing import Optional, Dict, Any
import uuid
# Default TTL for log entries (30 days)
DEFAULT_LOG_TTL_DAYS = 30
@dataclass
class WorkerLogEntry:
"""
Worker log entry for Firestore subcollection storage.
Stored at: jobs/{job_id}/logs/{log_id}
This model is separate from the legacy LogEntry (in job.py) which is
stored as an embedded array in the job document. This new model supports
the subcollection approach with TTL and richer metadata.
"""
# Core fields (required)
timestamp: datetime
level: str # "DEBUG", "INFO", "WARNING", "ERROR"
worker: str # "audio", "lyrics", "screens", "video", "render", "distribution"
message: str
# Identifiers
id: str = field(default_factory=lambda: str(uuid.uuid4()))
job_id: str = "" # Set when writing to Firestore
# TTL for automatic cleanup
ttl_expiry: datetime = field(
default_factory=lambda: datetime.now(timezone.utc) + timedelta(days=DEFAULT_LOG_TTL_DAYS)
)
# Optional metadata for debugging
metadata: Optional[Dict[str, Any]] = None
@classmethod
def create(
cls,
job_id: str,
worker: str,
level: str,
message: str,
metadata: Optional[Dict[str, Any]] = None,
ttl_days: int = DEFAULT_LOG_TTL_DAYS
) -> "WorkerLogEntry":
"""
Factory method to create a new log entry.
Args:
job_id: Job ID this log belongs to
worker: Worker name (audio, lyrics, screens, video, render, distribution)
level: Log level (DEBUG, INFO, WARNING, ERROR)
message: Log message (truncated to 1000 chars)
metadata: Optional additional metadata
ttl_days: Days until log expires (default 30)
Returns:
New WorkerLogEntry instance
"""
now = datetime.now(timezone.utc)
return cls(
id=str(uuid.uuid4()),
job_id=job_id,
timestamp=now,
level=level.upper(),
worker=worker,
message=message[:1000], # Truncate long messages
metadata=metadata,
ttl_expiry=now + timedelta(days=ttl_days)
)
def to_dict(self) -> Dict[str, Any]:
"""
Convert to dictionary for Firestore storage.
Returns:
Dictionary representation for Firestore
"""
result = {
"id": self.id,
"job_id": self.job_id,
"timestamp": self.timestamp,
"level": self.level,
"worker": self.worker,
"message": self.message,
"ttl_expiry": self.ttl_expiry
}
if self.metadata:
result["metadata"] = self.metadata
return result
def to_legacy_dict(self) -> Dict[str, Any]:
"""
Convert to legacy format for backward compatibility with API responses.
The existing API returns logs in this format:
{"timestamp": "...", "level": "INFO", "worker": "audio", "message": "..."}
Returns:
Dictionary in legacy format
"""
# Format timestamp: use "Z" suffix for UTC, isoformat() for others
if self.timestamp.tzinfo is None:
# Naive datetime - use as-is
timestamp_str = self.timestamp.isoformat()
elif self.timestamp.utcoffset() == timedelta(0):
# UTC datetime - replace +00:00 with Z for cleaner format
timestamp_str = self.timestamp.replace(tzinfo=None).isoformat() + "Z"
else:
# Non-UTC timezone-aware - use full isoformat with offset
timestamp_str = self.timestamp.isoformat()
return {
"timestamp": timestamp_str,
"level": self.level,
"worker": self.worker,
"message": self.message
}
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "WorkerLogEntry":
"""
Create from Firestore document data.
Args:
data: Dictionary from Firestore document
Returns:
WorkerLogEntry instance
"""
# Handle timestamp conversion (Firestore returns datetime objects)
timestamp = data.get("timestamp")
if isinstance(timestamp, str):
# Parse ISO format string
timestamp = datetime.fromisoformat(timestamp.rstrip("Z")).replace(tzinfo=timezone.utc)
ttl_expiry = data.get("ttl_expiry")
if isinstance(ttl_expiry, str):
ttl_expiry = datetime.fromisoformat(ttl_expiry.rstrip("Z")).replace(tzinfo=timezone.utc)
elif ttl_expiry is None:
# Default TTL if not set
ttl_expiry = datetime.now(timezone.utc) + timedelta(days=DEFAULT_LOG_TTL_DAYS)
return cls(
id=data.get("id", str(uuid.uuid4())),
job_id=data.get("job_id", ""),
timestamp=timestamp or datetime.now(timezone.utc),
level=data.get("level", "INFO"),
worker=data.get("worker", "unknown"),
message=data.get("message", ""),
metadata=data.get("metadata"),
ttl_expiry=ttl_expiry
)
+208 -562
View File
@@ -1,69 +1,79 @@
"""
Audio search service for finding and downloading audio files.
This service integrates with karaoke_gen.audio_fetcher (which wraps flacfetch)
to search for audio from various sources (YouTube, music trackers, etc.)
and download the selected audio file.
This is a thin wrapper that adds backend-specific functionality:
- Caching raw results for API-based selection flow
- Firestore-compatible serialization
- Singleton pattern for service lifecycle
- Remote flacfetch service integration for torrent downloads
When FLACFETCH_API_URL is configured, this service uses the remote flacfetch
HTTP API for search and download operations. This is required for torrent
downloads since Cloud Run doesn't support BitTorrent peer connections.
This service integrates with flacfetch to search for audio from various sources
(YouTube, music trackers, etc.) and download the selected audio file.
"""
import asyncio
import logging
import os
from typing import List, Optional
import nest_asyncio
# Import shared classes from karaoke_gen.audio_fetcher - single source of truth
# Note: Import directly from audio_fetcher module, not karaoke_gen.__init__
# to avoid pulling in heavier dependencies like lyrics_transcriber
from karaoke_gen.audio_fetcher import (
FlacFetcher,
AudioSearchResult,
AudioFetchResult,
AudioFetcherError,
NoResultsError,
DownloadError,
)
from .flacfetch_client import get_flacfetch_client, FlacfetchServiceError
import tempfile
from dataclasses import dataclass, asdict
from typing import List, Optional, Dict, Any
logger = logging.getLogger(__name__)
# Re-export exception under backend's naming convention for compatibility
AudioSearchError = AudioFetcherError
@dataclass
class AudioSearchResult:
"""
Represents a single search result for audio.
This is a serializable representation of a search result that can be
stored in Firestore and sent to clients.
"""
title: str
artist: str
provider: str
url: str
duration: Optional[int] = None # Duration in seconds
quality: Optional[str] = None # e.g., "FLAC", "320kbps", etc.
source_id: Optional[str] = None # Unique ID from the source
index: int = 0 # Index in the results list (for selection)
def to_dict(self) -> Dict[str, Any]:
"""Convert to dict for JSON serialization."""
return asdict(self)
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AudioSearchResult":
"""Create from dict."""
return cls(**data)
# Also alias AudioFetchResult as AudioDownloadResult for backwards compatibility
AudioDownloadResult = AudioFetchResult
@dataclass
class AudioDownloadResult:
"""Result of downloading an audio file."""
filepath: str
artist: str
title: str
provider: str
duration: Optional[int] = None
quality: Optional[str] = None
class AudioSearchError(Exception):
"""Base exception for audio search errors."""
pass
class NoResultsError(AudioSearchError):
"""Raised when no search results are found."""
pass
class DownloadError(AudioSearchError):
"""Raised when download fails."""
pass
class AudioSearchService:
"""
Service for searching and downloading audio files.
Service for searching and downloading audio files using flacfetch.
This is a thin wrapper around karaoke_gen.audio_fetcher.FlacFetcher that adds
backend-specific functionality:
- Caches raw results for API-based selection (client picks by index)
- Provides Firestore-compatible serialization
- Manages service lifecycle as singleton
- Remote flacfetch service integration for torrent downloads
When FLACFETCH_API_URL is configured, this service uses the remote flacfetch
HTTP API for search and download operations. This enables torrent downloads
which are not possible in Cloud Run due to network restrictions.
The actual flacfetch integration is in karaoke_gen.audio_fetcher.FlacFetcher,
which is shared between local CLI and cloud backend.
This service is used by the backend to:
1. Search for audio by artist/title
2. Return results for user selection
3. Download the selected audio to a local file
"""
# Sentinel value to indicate "use environment variable"
@@ -71,64 +81,72 @@ class AudioSearchService:
def __init__(
self,
red_api_key: Optional[str] = _USE_ENV,
red_api_url: Optional[str] = _USE_ENV,
redacted_api_key: Optional[str] = _USE_ENV,
ops_api_key: Optional[str] = _USE_ENV,
ops_api_url: Optional[str] = _USE_ENV,
):
"""
Initialize the audio search service.
Args:
red_api_key: API key for RED tracker (optional, uses env if not provided)
red_api_url: Base URL for RED tracker API (optional, uses env if not provided)
redacted_api_key: API key for Redacted tracker (optional, uses env if not provided)
ops_api_key: API key for OPS tracker (optional, uses env if not provided)
ops_api_url: Base URL for OPS tracker API (optional, uses env if not provided)
"""
# Use environment variables if not explicitly provided
if red_api_key is self._USE_ENV:
red_api_key = os.environ.get("RED_API_KEY")
if red_api_url is self._USE_ENV:
red_api_url = os.environ.get("RED_API_URL")
if redacted_api_key is self._USE_ENV:
self._redacted_api_key = os.environ.get("REDACTED_API_KEY")
else:
self._redacted_api_key = redacted_api_key
if ops_api_key is self._USE_ENV:
ops_api_key = os.environ.get("OPS_API_KEY")
if ops_api_url is self._USE_ENV:
ops_api_url = os.environ.get("OPS_API_URL")
# Check for remote flacfetch client
self._remote_client = get_flacfetch_client()
# Log which mode we're using
if self._remote_client:
logger.info("AudioSearchService using REMOTE flacfetch service (torrent downloads enabled)")
self._ops_api_key = os.environ.get("OPS_API_KEY")
else:
logger.info("AudioSearchService using LOCAL flacfetch (YouTube only in Cloud Run)")
self._ops_api_key = ops_api_key
self._manager = None
self._raw_results = {} # Cache raw results for download
def _get_manager(self):
"""Lazily initialize and return the FetchManager."""
if self._manager is None:
try:
from flacfetch.core.manager import FetchManager
from flacfetch.providers.youtube import YouTubeProvider
except ImportError as e:
logger.error(f"flacfetch not installed: {e}")
raise AudioSearchError(
"flacfetch library is not installed. "
"Install with: pip install flacfetch"
) from e
self._manager = FetchManager()
# Add providers based on available API keys
if self._redacted_api_key:
try:
from flacfetch.providers.redacted import RedactedProvider
self._manager.add_provider(RedactedProvider(api_key=self._redacted_api_key))
logger.debug("Added Redacted provider")
except ImportError:
logger.warning("Redacted provider not available")
if self._ops_api_key:
try:
from flacfetch.providers.ops import OPSProvider
self._manager.add_provider(OPSProvider(api_key=self._ops_api_key))
logger.debug("Added OPS provider")
except ImportError:
logger.warning("OPS provider not available")
# Always add YouTube as a fallback provider
self._manager.add_provider(YouTubeProvider())
logger.debug("Added YouTube provider")
# Delegate to shared FlacFetcher implementation (for local mode or fallback)
self._fetcher = FlacFetcher(
red_api_key=red_api_key,
red_api_url=red_api_url,
ops_api_key=ops_api_key,
ops_api_url=ops_api_url,
)
# Cache search results for API-based selection flow
# Key: index, Value: AudioSearchResult (with raw_result)
self._cached_results: List[AudioSearchResult] = []
# Cache for remote search results (search_id -> results)
self._remote_search_id: Optional[str] = None
return self._manager
def search(self, artist: str, title: str) -> List[AudioSearchResult]:
"""
Search for audio matching the given artist and title.
Results are cached internally for later download via download().
If a remote flacfetch service is configured (FLACFETCH_API_URL), uses that
for better torrent download support. Otherwise falls back to local flacfetch.
Args:
artist: The artist name to search for
title: The track title to search for
@@ -138,94 +156,49 @@ class AudioSearchService:
Raises:
NoResultsError: If no results are found
AudioSearchError: For other errors (e.g., flacfetch not installed)
AudioSearchError: For other errors
"""
try:
# Try remote flacfetch service first if configured
if self._remote_client:
return self._search_remote(artist, title)
# Fallback to local flacfetch
results = self._fetcher.search(artist, title)
# Cache results for later download
self._cached_results = results
self._remote_search_id = None
logger.info(f"Found {len(results)} results for: {artist} - {title}")
return results
except (NoResultsError, AudioFetcherError):
# Re-raise these as-is
raise
from flacfetch.core.models import TrackQuery
except ImportError as e:
raise AudioSearchError("flacfetch not installed") from e
manager = self._get_manager()
query = TrackQuery(artist=artist, title=title)
logger.info(f"Searching for audio: {artist} - {title}")
try:
results = manager.search(query)
except Exception as e:
logger.error(f"Search failed: {e}")
raise AudioSearchError(f"Search failed: {e}") from e
def _search_remote(self, artist: str, title: str) -> List[AudioSearchResult]:
"""
Search using the remote flacfetch service.
Runs async code in a sync context for compatibility with existing API.
"""
try:
# Enable nested event loops (needed when called from FastAPI async context)
# Apply to the current loop if one exists, otherwise it's a no-op
try:
loop = asyncio.get_running_loop()
nest_asyncio.apply(loop)
except RuntimeError:
pass # No running loop, nest_asyncio not needed
if not results:
raise NoResultsError(f"No results found for: {artist} - {title}")
# Convert to our AudioSearchResult format and cache raw results
search_results = []
self._raw_results.clear()
for i, result in enumerate(results):
search_result = AudioSearchResult(
title=getattr(result, "title", title),
artist=getattr(result, "artist", artist),
provider=getattr(result, "provider", "Unknown"),
url=getattr(result, "url", ""),
duration=getattr(result, "duration", None),
quality=getattr(result, "quality", None),
source_id=getattr(result, "id", str(i)),
index=i,
)
search_results.append(search_result)
# Run async search in sync context
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
response = loop.run_until_complete(
self._remote_client.search(artist, title)
)
finally:
loop.close()
# Check for empty results
if not response.get("results"):
raise NoResultsError(f"No results found for: {artist} - {title}")
# Store search_id for download
self._remote_search_id = response.get("search_id")
# Convert remote results to AudioSearchResult objects
results = []
for item in response.get("results", []):
result = AudioSearchResult(
title=item.get("title", "Unknown"),
artist=item.get("artist", "Unknown"),
url=item.get("download_url", ""), # May be empty for remote
provider=item.get("provider", "Unknown"),
duration=item.get("duration_seconds"),
quality=item.get("quality", ""),
source_id=item.get("info_hash"),
index=item.get("index", 0),
seeders=item.get("seeders"),
target_file=item.get("target_file"),
# Store full remote data in raw_result for rich display
raw_result=item,
)
results.append(result)
# Cache results
self._cached_results = results
logger.info(f"Found {len(results)} results from remote flacfetch for: {artist} - {title}")
return results
except FlacfetchServiceError as e:
logger.warning(f"Remote flacfetch search failed, falling back to local: {e}")
# Fallback to local search
self._remote_search_id = None
results = self._fetcher.search(artist, title)
self._cached_results = results
return results
# Cache raw result for later download
self._raw_results[i] = result
logger.info(f"Found {len(search_results)} results for: {artist} - {title}")
return search_results
def select_best(self, results: List[AudioSearchResult]) -> int:
"""
@@ -239,331 +212,90 @@ class AudioSearchService:
Returns:
Index of the best result
"""
return self._fetcher.select_best(results)
if not results:
return 0
manager = self._get_manager()
# Get raw results for the indices we have
raw_results = [self._raw_results.get(r.index) for r in results if r.index in self._raw_results]
if raw_results:
try:
best = manager.select_best(raw_results)
# Find index of best result
for i, raw in enumerate(raw_results):
if raw == best:
return results[i].index
except Exception as e:
logger.warning(f"select_best failed, using first result: {e}")
# Fallback: return first result
return results[0].index if results else 0
def download(
self,
result_index: int,
output_dir: str,
output_filename: Optional[str] = None,
gcs_path: Optional[str] = None,
remote_search_id: Optional[str] = None,
) -> AudioDownloadResult:
"""
Download audio from a cached search result.
This method uses the cached results from the last search() call.
The API flow is:
1. Client calls search() -> gets list of results
2. Client picks an index
3. Client calls download(index) -> gets downloaded file
If a remote flacfetch service is configured and the search was performed
remotely (for torrent sources), uses the remote service for download.
Download audio from a search result.
Args:
result_index: Index of the result to download (from search results)
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
gcs_path: Optional GCS path for remote uploads (e.g., "uploads/job123/audio/")
remote_search_id: Optional search_id for remote downloads. If provided,
uses this instead of the cached _remote_search_id. This is important
for concurrent requests where the service-level cache may be stale.
Returns:
AudioDownloadResult with the downloaded file path
Raises:
DownloadError: If download fails or no cached result for index
"""
# Use provided search_id or fall back to service-level cache
effective_search_id = remote_search_id or self._remote_search_id
# Check if we have local cached results
has_local_cache = result_index >= 0 and result_index < len(self._cached_results)
# If no local cache but we have remote search_id, use remote download
# This handles multi-instance deployments where the cache doesn't persist
if not has_local_cache:
if effective_search_id and self._remote_client:
logger.info(f"No local cache, using remote download with search_id={effective_search_id}, index={result_index}")
return self._download_remote(result_index, output_dir, output_filename, gcs_path, effective_search_id)
else:
# Provide clear error message based on whether cache is empty or index out of bounds
if len(self._cached_results) == 0:
raise DownloadError(
f"No cached result for index {result_index}. "
"No cached results available. Run search() first."
)
raise DownloadError(
f"No cached result for index {result_index}. "
f"Available indices: 0-{len(self._cached_results) - 1}. "
"Run search() first."
)
result = self._cached_results[result_index]
logger.info(f"Downloading: {result.artist} - {result.title} from {result.provider}")
# Check if this was a remote search (torrent sources need remote download)
if effective_search_id and self._remote_client:
# Check if this is a torrent source that needs remote handling
is_torrent = result.provider in ["RED", "OPS"]
if is_torrent:
return self._download_remote(result_index, output_dir, output_filename, gcs_path, effective_search_id)
# Delegate to shared FlacFetcher (local download)
fetch_result = self._fetcher.download(result, output_dir, output_filename)
logger.info(f"Downloaded to: {fetch_result.filepath}")
return fetch_result
def download_by_id(
self,
source_name: str,
source_id: str,
output_dir: str,
output_filename: Optional[str] = None,
target_file: Optional[str] = None,
download_url: Optional[str] = None,
gcs_path: Optional[str] = None,
) -> AudioDownloadResult:
"""
Download audio directly by source ID (no prior search required).
This is the preferred method when you have stored the source_id from a
previous search and want to download later without re-searching. This
avoids unnecessary API calls to private trackers.
Args:
source_name: Provider name (RED, OPS, YouTube, Spotify)
source_id: Source-specific ID (torrent ID, video ID, track ID)
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
target_file: For torrents, specific file to extract from the torrent
download_url: For YouTube/Spotify, direct URL (optional)
gcs_path: Optional GCS path for remote uploads
Returns:
AudioDownloadResult with the downloaded file path
Raises:
DownloadError: If download fails
"""
logger.info(f"Download by ID: {source_name} ID={source_id}")
# For torrent sources, must use remote client
if source_name in ["RED", "OPS"]:
if not self._remote_client:
raise DownloadError(
f"Cannot download from {source_name} without remote flacfetch service"
)
return self._download_by_id_remote(
source_name=source_name,
source_id=source_id,
output_dir=output_dir,
output_filename=output_filename,
target_file=target_file,
download_url=download_url,
gcs_path=gcs_path,
)
# For YouTube/Spotify, can use local flacfetch via FetchManager.download_by_id
# But we currently only have remote support, so use remote if available
if self._remote_client:
return self._download_by_id_remote(
source_name=source_name,
source_id=source_id,
output_dir=output_dir,
output_filename=output_filename,
target_file=target_file,
download_url=download_url,
gcs_path=gcs_path,
)
# Local download for YouTube/Spotify (fallback)
# Use the local fetcher's download_by_id if available
raise DownloadError(
f"Local download_by_id not yet implemented for {source_name}. "
"Configure FLACFETCH_API_URL for remote downloads."
)
def _download_by_id_remote(
self,
source_name: str,
source_id: str,
output_dir: str, # Not used for remote downloads (files go to GCS or remote server)
output_filename: Optional[str] = None,
target_file: Optional[str] = None,
download_url: Optional[str] = None,
gcs_path: Optional[str] = None,
) -> AudioDownloadResult:
"""
Download by ID using the remote flacfetch service.
Note: output_dir is accepted for API compatibility but ignored for remote
downloads. Remote downloads either go to GCS (if gcs_path is set) or to
the remote server's download directory.
"""
# output_dir is intentionally unused - remote downloads don't use local paths
_ = output_dir
if result_index not in self._raw_results:
raise DownloadError(f"No cached result for index {result_index}. Run search() first.")
raw_result = self._raw_results[result_index]
manager = self._get_manager()
# Ensure output directory exists
os.makedirs(output_dir, exist_ok=True)
# Get metadata from raw result
artist = getattr(raw_result, "artist", "Unknown")
title = getattr(raw_result, "title", "Unknown")
provider = getattr(raw_result, "provider", "Unknown")
# Generate filename if not provided
if output_filename is None:
output_filename = f"{artist} - {title}"
logger.info(f"Downloading: {artist} - {title} from {provider}")
try:
# Enable nested event loops
try:
running_loop = asyncio.get_running_loop()
nest_asyncio.apply(running_loop)
except RuntimeError:
pass
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
# Start download
download_id = loop.run_until_complete(
self._remote_client.download_by_id(
source_name=source_name,
source_id=source_id,
output_filename=output_filename,
target_file=target_file,
download_url=download_url,
gcs_path=gcs_path,
)
)
logger.info(f"Remote download by ID started: {download_id}")
# Wait for completion
def log_progress(status):
progress = status.get("progress", 0)
speed = status.get("download_speed_kbps", 0)
logger.debug(f"Download progress: {progress:.1f}% ({speed:.1f} KB/s)")
final_status = loop.run_until_complete(
self._remote_client.wait_for_download(
download_id,
timeout=600,
progress_callback=log_progress,
)
)
finally:
loop.close()
# Determine file path
filepath = final_status.get("gcs_path") or final_status.get("output_path")
if not filepath:
raise DownloadError("Remote download completed but no file path returned")
logger.info(f"Remote download by ID complete: {filepath}")
filepath = manager.download(
raw_result,
output_path=output_dir,
output_filename=output_filename,
)
if filepath is None:
raise DownloadError(f"Download returned no file path for: {artist} - {title}")
logger.info(f"Downloaded to: {filepath}")
return AudioDownloadResult(
filepath=filepath,
artist="", # Not available without search
title="", # Not available without search
provider=source_name,
quality="", # Not available without search
)
except FlacfetchServiceError as e:
raise DownloadError(f"Remote download by ID failed: {e}") from e
except Exception as e:
logger.error(f"Remote download by ID error: {e}")
raise DownloadError(f"Remote download by ID failed: {e}") from e
def _download_remote(
self,
result_index: int,
output_dir: str,
output_filename: Optional[str] = None,
gcs_path: Optional[str] = None,
search_id: Optional[str] = None,
) -> AudioDownloadResult:
"""
Download using the remote flacfetch service.
The remote service downloads via torrent and optionally uploads to GCS.
Args:
result_index: Index of the result to download
output_dir: Directory to save downloaded file
output_filename: Optional filename
gcs_path: Optional GCS path for remote uploads
search_id: Remote search ID to use (overrides self._remote_search_id)
"""
effective_search_id = search_id or self._remote_search_id
if not effective_search_id:
raise DownloadError("No remote search ID - run search() first")
# Try to get local result for metadata, but it's optional
# The remote service maintains its own cache by search_id
result = None
if result_index >= 0 and result_index < len(self._cached_results):
result = self._cached_results[result_index]
try:
# Enable nested event loops (needed when called from FastAPI async context)
try:
running_loop = asyncio.get_running_loop()
nest_asyncio.apply(running_loop)
except RuntimeError:
pass # No running loop, nest_asyncio not needed
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
# Start download
download_id = loop.run_until_complete(
self._remote_client.download(
search_id=effective_search_id,
result_index=result_index,
output_filename=output_filename,
gcs_path=gcs_path,
)
)
logger.info(f"Remote download started: {download_id}")
# Wait for completion with progress logging
def log_progress(status):
progress = status.get("progress", 0)
speed = status.get("download_speed_kbps", 0)
logger.debug(f"Download progress: {progress:.1f}% ({speed:.1f} KB/s)")
final_status = loop.run_until_complete(
self._remote_client.wait_for_download(
download_id,
timeout=600, # 10 minute timeout
progress_callback=log_progress,
)
)
finally:
loop.close()
# Determine file path
filepath = final_status.get("gcs_path") or final_status.get("output_path")
if not filepath:
raise DownloadError("Remote download completed but no file path returned")
logger.info(f"Remote download complete: {filepath}")
# Use local result metadata if available, otherwise use empty strings
# (the actual download is handled by the remote service using search_id)
return AudioDownloadResult(
filepath=filepath,
artist=result.artist if result else "",
title=result.title if result else "",
provider=result.provider if result else "remote",
quality=result.quality if result else "",
artist=artist,
title=title,
provider=provider,
duration=getattr(raw_result, "duration", None),
quality=getattr(raw_result, "quality", None),
)
except FlacfetchServiceError as e:
raise DownloadError(f"Remote download failed: {e}") from e
except Exception as e:
logger.error(f"Remote download error: {e}")
raise DownloadError(f"Remote download failed: {e}") from e
raise DownloadError(f"Failed to download {artist} - {title}: {e}") from e
def search_and_download_auto(
self,
@@ -571,7 +303,6 @@ class AudioSearchService:
title: str,
output_dir: str,
output_filename: Optional[str] = None,
gcs_path: Optional[str] = None,
) -> AudioDownloadResult:
"""
Search for audio and automatically download the best result.
@@ -584,7 +315,6 @@ class AudioSearchService:
title: The track title to search for
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
gcs_path: Optional GCS path for remote uploads
Returns:
AudioDownloadResult with the downloaded file path
@@ -598,96 +328,10 @@ class AudioSearchService:
# Select best
best_index = self.select_best(results)
best_result = results[best_index]
logger.info(
f"Auto-selected result {best_index}: "
f"{best_result.artist} - {best_result.title} ({best_result.provider})"
)
logger.info(f"Auto-selected result {best_index}: {results[best_index].artist} - {results[best_index].title} ({results[best_index].provider})")
# Download
return self.download(best_index, output_dir, output_filename, gcs_path)
async def search_async(self, artist: str, title: str) -> List[AudioSearchResult]:
"""
Async version of search for use in async routes.
Note: Currently wraps sync search in executor. Future optimization
could make this fully async.
"""
loop = asyncio.get_event_loop()
return await loop.run_in_executor(None, self.search, artist, title)
async def download_async(
self,
result_index: int,
output_dir: str,
output_filename: Optional[str] = None,
gcs_path: Optional[str] = None,
remote_search_id: Optional[str] = None,
) -> AudioDownloadResult:
"""
Async version of download for use in async routes.
Note: Currently wraps sync download in executor. Future optimization
could make this fully async.
"""
loop = asyncio.get_event_loop()
return await loop.run_in_executor(
None,
lambda: self.download(result_index, output_dir, output_filename, gcs_path, remote_search_id)
)
async def download_by_id_async(
self,
source_name: str,
source_id: str,
output_dir: str,
output_filename: Optional[str] = None,
target_file: Optional[str] = None,
download_url: Optional[str] = None,
gcs_path: Optional[str] = None,
) -> AudioDownloadResult:
"""
Async version of download_by_id for use in async routes.
"""
loop = asyncio.get_event_loop()
return await loop.run_in_executor(
None,
lambda: self.download_by_id(
source_name, source_id, output_dir, output_filename,
target_file, download_url, gcs_path
)
)
def is_remote_enabled(self) -> bool:
"""Check if remote flacfetch service is configured."""
return self._remote_client is not None
@property
def last_remote_search_id(self) -> Optional[str]:
"""Get the search_id from the last remote search.
This should be stored in job state_data after search and passed
back to download() to ensure correct remote download handling.
"""
return self._remote_search_id
async def check_remote_health(self) -> Optional[dict]:
"""
Check health of remote flacfetch service.
Returns:
Health status dict if remote service is configured and healthy,
None if not configured or unhealthy.
"""
if not self._remote_client:
return None
try:
return await self._remote_client.health_check()
except Exception as e:
logger.warning(f"Remote flacfetch health check failed: {e}")
return None
return self.download(best_index, output_dir, output_filename)
# Singleton instance
@@ -700,3 +344,5 @@ def get_audio_search_service() -> AudioSearchService:
if _audio_search_service is None:
_audio_search_service = AudioSearchService()
return _audio_search_service
+19 -364
View File
@@ -3,81 +3,50 @@ Authentication service for karaoke backend.
Provides token-based authentication with support for:
- Admin tokens (hardcoded in environment)
- Admin by email domain (@nomadkaraoke.com)
- User tokens (stored in Firestore auth_tokens collection)
- Session tokens (stored in Firestore sessions collection, from magic link auth)
- API keys for business users
- User tokens (stored in Firestore)
- Usage tracking and limits
- Token management API
This is designed to be extensible for future Stripe integration
and admin dashboard functionality.
"""
import logging
import hashlib
import time
import secrets
from dataclasses import dataclass
from enum import Enum
from typing import Optional, Tuple, Dict, Any, List
from typing import Optional, Tuple, Dict, Any
from datetime import datetime
from backend.services.firestore_service import FirestoreService
from backend.config import get_settings
from backend.models.user import UserRole
logger = logging.getLogger(__name__)
# Admin email domains - users with these domains are automatically admin
ADMIN_EMAIL_DOMAINS = ["nomadkaraoke.com"]
class UserType(str, Enum):
"""User access levels."""
ADMIN = "admin" # Unlimited access, can manage tokens
UNLIMITED = "unlimited" # Unlimited karaoke generation
LIMITED = "limited" # Limited number of uses
STRIPE = "stripe" # Paid access via Stripe
API_KEY = "api_key" # Business API key access
@dataclass
class AuthResult:
"""Result of authentication validation."""
is_valid: bool
user_type: UserType
remaining_uses: int # -1 = unlimited, 0 = exhausted, >0 = remaining
message: str
user_email: Optional[str] = None # Email if authenticated via session/API key
is_admin: bool = False # True if admin token or admin email domain
api_key_id: Optional[str] = None # API key ID if authenticated via API key
def __iter__(self):
"""Allow unpacking as tuple for backward compatibility."""
return iter((self.is_valid, self.user_type, self.remaining_uses, self.message))
def is_admin_email(email: str) -> bool:
"""Check if an email belongs to an admin domain."""
if not email:
return False
email_lower = email.lower()
return any(email_lower.endswith(f"@{domain}") for domain in ADMIN_EMAIL_DOMAINS)
STRIPE = "stripe" # Paid access via Stripe (future)
class AuthService:
"""Service for authentication and token management."""
def __init__(self):
"""Initialize auth service."""
self.firestore = FirestoreService()
self.settings = get_settings()
self._init_admin_tokens()
def _init_admin_tokens(self):
"""Load admin tokens from environment variables."""
# Get admin tokens from secret manager or env vars
admin_tokens_str = self.settings.admin_tokens or ""
self.admin_tokens = [t.strip() for t in admin_tokens_str.split(",") if t.strip()]
if self.admin_tokens:
logger.info(f"Loaded {len(self.admin_tokens)} admin token(s)")
else:
@@ -99,40 +68,10 @@ class AuthService:
logger.info("Admin token validated")
return True, UserType.ADMIN, -1, "Admin access granted"
# Debug: Log token comparison info (only first/last few chars for security)
if self.admin_tokens:
expected_prefix = self.admin_tokens[0][:8] if len(self.admin_tokens[0]) >= 8 else self.admin_tokens[0]
provided_prefix = token[:8] if len(token) >= 8 else token
logger.debug(
f"Token mismatch: expected prefix '{expected_prefix}...', "
f"got '{provided_prefix}...', "
f"expected len={len(self.admin_tokens[0])}, got len={len(token)}"
)
# Check stored tokens in Firestore (auth_tokens collection)
# Check stored tokens in Firestore
token_data = self.firestore.get_token(token)
if not token_data:
# Fall back to checking session tokens (from magic link auth)
# Import here to avoid circular dependency
from backend.services.user_service import get_user_service
user_service = get_user_service()
is_valid, user, message = user_service.validate_session(token)
if is_valid and user:
# Session is valid - user is authenticated via magic link
# Return their credits as remaining uses
credits = user.credits
logger.info(f"Session token validated for user {user.email} (credits: {credits})")
if credits <= 0:
# User has no credits but is authenticated
return True, UserType.STRIPE, 0, f"Authenticated but no credits remaining"
return True, UserType.STRIPE, credits, f"Session valid: {credits} credits"
# Neither auth_token nor session found
return False, UserType.LIMITED, 0, "Invalid token"
# Check if token is active
@@ -180,319 +119,35 @@ class AuthService:
return True, token_type, -1, "Stripe access granted"
return False, UserType.LIMITED, 0, "Unknown token type"
def validate_token_full(self, token: str) -> AuthResult:
"""
Validate an access token and return full authentication details.
This is the preferred method for new code as it returns complete
authentication context including user email and admin status.
Returns:
AuthResult with full authentication details
"""
if not token:
return AuthResult(
is_valid=False,
user_type=UserType.LIMITED,
remaining_uses=0,
message="No token provided"
)
# Check for admin tokens (highest priority)
# Admin tokens from env var are associated with the system admin email
if token in self.admin_tokens:
logger.info("Admin token validated")
return AuthResult(
is_valid=True,
user_type=UserType.ADMIN,
remaining_uses=-1,
message="Admin access granted",
user_email="admin@nomadkaraoke.com",
is_admin=True
)
# Check stored tokens in Firestore (auth_tokens collection)
token_data = self.firestore.get_token(token)
if not token_data:
# Fall back to checking session tokens (from magic link auth)
from backend.services.user_service import get_user_service
user_service = get_user_service()
is_valid, user, _message = user_service.validate_session(token)
if is_valid and user:
# Session is valid - user is authenticated via magic link
credits = user.credits
user_email = user.email
# Check if user is admin by email domain or role
user_is_admin = is_admin_email(user_email) or user.role == UserRole.ADMIN
if user_is_admin:
logger.info(f"Admin session validated for {user_email}")
return AuthResult(
is_valid=True,
user_type=UserType.ADMIN,
remaining_uses=-1,
message="Admin access granted (domain)",
user_email=user_email,
is_admin=True
)
logger.info(f"Session token validated for user {user_email} (credits: {credits})")
if credits <= 0:
return AuthResult(
is_valid=True,
user_type=UserType.STRIPE,
remaining_uses=0,
message="Authenticated but no credits remaining",
user_email=user_email,
is_admin=False
)
return AuthResult(
is_valid=True,
user_type=UserType.STRIPE,
remaining_uses=credits,
message=f"Session valid: {credits} credits",
user_email=user_email,
is_admin=False
)
# Neither auth_token nor session found
return AuthResult(
is_valid=False,
user_type=UserType.LIMITED,
remaining_uses=0,
message="Invalid token"
)
# Check if token is active
if not token_data.get("active", True):
return AuthResult(
is_valid=False,
user_type=UserType(token_data["type"]),
remaining_uses=0,
message="Token has been revoked"
)
token_type = UserType(token_data["type"])
max_uses = token_data.get("max_uses", -1)
# All auth_tokens must have an associated user_email for job ownership
token_user_email = token_data.get("user_email")
api_key_id = token_data.get("api_key_id")
# Require user_email on all auth_tokens (no anonymous token auth)
if not token_user_email:
logger.warning("Auth token missing required user_email field")
return AuthResult(
is_valid=False,
user_type=token_type,
remaining_uses=0,
message="Token configuration error: missing user_email. Please contact support."
)
# Check if token's user is an admin (by email domain)
token_is_admin = is_admin_email(token_user_email)
# UNLIMITED tokens: no usage limits
if token_type == UserType.UNLIMITED:
return AuthResult(
is_valid=True,
user_type=token_type,
remaining_uses=-1,
message="Unlimited access granted",
user_email=token_user_email,
is_admin=token_is_admin,
api_key_id=api_key_id
)
# LIMITED tokens: check usage count
if token_type == UserType.LIMITED:
if max_uses <= 0: # -1 means unlimited
return AuthResult(
is_valid=True,
user_type=token_type,
remaining_uses=-1,
message="Limited token with unlimited uses",
user_email=token_user_email,
is_admin=token_is_admin,
api_key_id=api_key_id
)
current_uses = token_data.get("usage_count", 0)
remaining = max_uses - current_uses
if remaining <= 0:
return AuthResult(
is_valid=False,
user_type=token_type,
remaining_uses=0,
message="Token usage limit exceeded",
user_email=token_user_email,
api_key_id=api_key_id
)
return AuthResult(
is_valid=True,
user_type=token_type,
remaining_uses=remaining,
message=f"Limited token: {remaining} uses remaining",
user_email=token_user_email,
is_admin=token_is_admin,
api_key_id=api_key_id
)
# STRIPE tokens: check expiration and usage
if token_type == UserType.STRIPE:
expires_at = token_data.get("expires_at")
if expires_at and time.time() > expires_at:
return AuthResult(
is_valid=False,
user_type=token_type,
remaining_uses=0,
message="Token has expired",
user_email=token_user_email,
api_key_id=api_key_id
)
if max_uses > 0:
current_uses = token_data.get("usage_count", 0)
remaining = max_uses - current_uses
if remaining <= 0:
return AuthResult(
is_valid=False,
user_type=token_type,
remaining_uses=0,
message="Token usage limit exceeded",
user_email=token_user_email,
api_key_id=api_key_id
)
return AuthResult(
is_valid=True,
user_type=token_type,
remaining_uses=remaining,
message=f"Stripe token: {remaining} uses remaining",
user_email=token_user_email,
is_admin=token_is_admin,
api_key_id=api_key_id
)
return AuthResult(
is_valid=True,
user_type=token_type,
remaining_uses=-1,
message="Stripe access granted",
user_email=token_user_email,
is_admin=token_is_admin,
api_key_id=api_key_id
)
# API_KEY tokens
if token_type == UserType.API_KEY:
if max_uses > 0:
current_uses = token_data.get("usage_count", 0)
remaining = max_uses - current_uses
if remaining <= 0:
return AuthResult(
is_valid=False,
user_type=token_type,
remaining_uses=0,
message="API key usage limit exceeded",
user_email=token_user_email,
api_key_id=api_key_id
)
return AuthResult(
is_valid=True,
user_type=token_type,
remaining_uses=remaining,
message=f"API key valid: {remaining} uses remaining",
user_email=token_user_email,
is_admin=token_is_admin,
api_key_id=api_key_id
)
return AuthResult(
is_valid=True,
user_type=token_type,
remaining_uses=-1,
message="API key access granted",
user_email=token_user_email,
is_admin=token_is_admin,
api_key_id=api_key_id
)
return AuthResult(
is_valid=False,
user_type=UserType.LIMITED,
remaining_uses=0,
message="Unknown token type"
)
def increment_token_usage(self, token: str, job_id: str) -> bool:
"""
Increment usage count for a token and track the job.
For session tokens (magic link auth), this deducts a credit from the user.
For auth_tokens, this increments the usage count.
Args:
token: The access token
job_id: The job ID being created
Returns:
True if usage was tracked, False otherwise
"""
# Validate token first
is_valid, user_type, remaining_uses, message = self.validate_token(token)
if not is_valid:
logger.warning(f"Cannot increment usage for invalid token: {message}")
return False
# Don't track usage for admin or unlimited tokens
if user_type in [UserType.ADMIN, UserType.UNLIMITED]:
logger.debug(f"Skipping usage tracking for {user_type} token")
return True
# For admin tokens (not in Firestore), no tracking needed
if token in self.admin_tokens:
return True
# Check if this is a session token (STRIPE type from validate_token means it's a session)
# Session tokens are not in auth_tokens collection
token_data = self.firestore.get_token(token)
if not token_data:
# This is a session token - deduct credit from user
from backend.services.user_service import get_user_service
user_service = get_user_service()
# Get the user email from the session
is_valid, user, _ = user_service.validate_session(token)
if not is_valid or not user:
logger.error(f"Session token validation failed during usage increment")
return False
# Deduct one credit
success, new_balance, deduct_message = user_service.deduct_credit(
user.email, job_id, reason="job_creation"
)
if success:
logger.info(f"Deducted credit for user {user.email} (remaining: {new_balance})")
return True
else:
logger.error(f"Failed to deduct credit for user {user.email}: {deduct_message}")
return False
# Regular auth_token - increment usage in Firestore
# Increment usage in Firestore
try:
self.firestore.increment_token_usage(token, job_id)
logger.info(f"Incremented usage for token (remaining: {remaining_uses - 1})")
-172
View File
@@ -1,172 +0,0 @@
"""
Discord Notification Service.
Provides Discord webhook notification functionality, extracted from KaraokeFinalise
for use by both the cloud backend (video_worker) and local CLI.
This service handles:
- Posting messages to Discord webhooks
- Video upload notifications
- Validation of webhook URLs
"""
import logging
import re
from typing import Optional
import requests
logger = logging.getLogger(__name__)
class DiscordNotificationService:
"""
Service for sending Discord webhook notifications.
Supports posting messages to Discord channels via webhooks,
commonly used to notify when new karaoke videos are uploaded.
"""
# Discord webhook URL pattern
WEBHOOK_URL_PATTERN = re.compile(
r"^https://discord\.com/api/webhooks/\d+/[a-zA-Z0-9_-]+$"
)
def __init__(
self,
webhook_url: Optional[str] = None,
dry_run: bool = False,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the Discord notification service.
Args:
webhook_url: Discord webhook URL for posting notifications
dry_run: If True, log actions without performing them
logger: Optional logger instance
"""
self.webhook_url = webhook_url
self.dry_run = dry_run
self.logger = logger or logging.getLogger(__name__)
# Validate webhook URL if provided
if webhook_url:
self._validate_webhook_url(webhook_url)
def _validate_webhook_url(self, url: str) -> None:
"""
Validate that the webhook URL is a valid Discord webhook URL.
Args:
url: URL to validate
Raises:
ValueError: If URL is not a valid Discord webhook URL
"""
url = url.strip()
if not url.startswith("https://discord.com/api/webhooks/"):
raise ValueError(
f"Invalid Discord webhook URL: {url}. "
"URL must start with 'https://discord.com/api/webhooks/'"
)
def post_message(
self,
message: str,
webhook_url: Optional[str] = None,
) -> bool:
"""
Post a message to a Discord channel via webhook.
Args:
message: The message content to post
webhook_url: Optional webhook URL (uses instance URL if not provided)
Returns:
True if message was posted successfully, False otherwise
Raises:
ValueError: If no webhook URL is available
requests.HTTPError: If the webhook request fails
"""
url = webhook_url or self.webhook_url
if not url:
raise ValueError("No Discord webhook URL provided")
url = url.strip()
self._validate_webhook_url(url)
if self.dry_run:
self.logger.info(
f"DRY RUN: Would post Discord message: '{message}' to {url}"
)
return True
self.logger.info(f"Posting message to Discord webhook...")
data = {"content": message}
response = requests.post(url, json=data, timeout=30)
response.raise_for_status()
self.logger.info("Message posted to Discord successfully")
return True
def post_video_notification(
self,
youtube_url: str,
webhook_url: Optional[str] = None,
) -> bool:
"""
Post a notification about a new video upload.
Args:
youtube_url: The YouTube URL of the uploaded video
webhook_url: Optional webhook URL (uses instance URL if not provided)
Returns:
True if notification was posted successfully, False otherwise
"""
if not youtube_url:
self.logger.info("Skipping Discord notification - no YouTube URL available")
return False
message = f"New upload: {youtube_url}"
return self.post_message(message, webhook_url)
def is_enabled(self) -> bool:
"""
Check if Discord notifications are enabled.
Returns:
True if a valid webhook URL is configured
"""
return bool(self.webhook_url)
# Singleton instance and factory function (following existing service pattern)
_discord_notification_service: Optional[DiscordNotificationService] = None
def get_discord_notification_service(
webhook_url: Optional[str] = None,
**kwargs
) -> DiscordNotificationService:
"""
Get a Discord notification service instance.
Args:
webhook_url: Discord webhook URL for notifications
**kwargs: Additional arguments passed to DiscordNotificationService
Returns:
DiscordNotificationService instance
"""
global _discord_notification_service
# Create new instance if webhook URL changed or not yet created
if _discord_notification_service is None or webhook_url:
_discord_notification_service = DiscordNotificationService(
webhook_url=webhook_url,
**kwargs
)
return _discord_notification_service
+5 -44
View File
@@ -199,7 +199,7 @@ class DropboxService:
def upload_folder(self, local_dir: str, remote_path: str) -> None:
"""
Recursively upload all files and subdirectories to Dropbox folder.
Upload all files in a directory to Dropbox folder.
Args:
local_dir: Local directory to upload
@@ -212,17 +212,10 @@ class DropboxService:
logger.info(f"Uploading folder {local_dir} to {remote_path}")
uploaded_count = 0
for root, _dirs, files in os.walk(local_dir):
# Calculate the relative path from local_dir to current root
rel_root = os.path.relpath(root, local_dir)
if rel_root == ".":
current_remote = remote_path
else:
current_remote = f"{remote_path}/{rel_root}"
for filename in files:
local_file = os.path.join(root, filename)
remote_file = f"{current_remote}/{filename}"
for filename in os.listdir(local_dir):
local_file = os.path.join(local_dir, filename)
if os.path.isfile(local_file):
remote_file = f"{remote_path}/{filename}"
self.upload_file(local_file, remote_file)
uploaded_count += 1
@@ -263,38 +256,6 @@ class DropboxService:
return links.links[0].url
raise
def delete_folder(self, path: str) -> bool:
"""
Delete a folder and all its contents from Dropbox.
Args:
path: Dropbox path to delete (e.g., "/Karaoke/Tracks-Organized/NOMAD-1234 - Artist - Title")
Returns:
True if deleted successfully, False otherwise
"""
from dropbox.exceptions import ApiError
# Ensure path starts with /
if not path.startswith("/"):
path = f"/{path}"
logger.info(f"Deleting Dropbox folder: {path}")
try:
self.client.files_delete_v2(path)
logger.info(f"Successfully deleted: {path}")
return True
except ApiError as e:
if e.error.is_path_lookup() and e.error.get_path_lookup().is_not_found():
logger.warning(f"Folder not found (already deleted?): {path}")
return True # Consider it success if already gone
logger.error(f"Failed to delete Dropbox folder: {e}")
return False
except Exception as e:
logger.error(f"Error deleting Dropbox folder: {e}")
return False
def get_dropbox_service() -> DropboxService:
"""Factory function to get a DropboxService instance."""
File diff suppressed because it is too large. Load diff
@@ -1,646 +0,0 @@
"""
Email Validation Service.
Provides email validation, normalization, and blocklist checking
to prevent abuse during beta enrollment and other flows.
"""
import hashlib
import logging
import re
from datetime import datetime, timezone
from typing import Optional, Tuple, Set
from google.cloud import firestore
from backend.config import settings
logger = logging.getLogger(__name__)
def _mask_email(email: str) -> str:
"""Mask an email address for privacy-safe logging."""
if not email or "@" not in email:
return "***"
local, domain = email.split("@", 1)
if len(local) <= 2:
masked_local = "*" * len(local)
else:
masked_local = local[0] + "*" * (len(local) - 2) + local[-1]
return f"{masked_local}@{domain}"
# Firestore collection for blocklist data
BLOCKLISTS_COLLECTION = "blocklists"
BLOCKLIST_CONFIG_DOC = "config"
# Default disposable email domains (can be extended via admin UI)
DEFAULT_DISPOSABLE_DOMAINS = {
# Popular temporary email services
"10minutemail.com",
"10minutemail.net",
"guerrillamail.com",
"guerrillamail.org",
"guerrillamail.net",
"guerrillamail.biz",
"guerrillamailblock.com",
"tempmail.com",
"tempmail.net",
"temp-mail.org",
"temp-mail.io",
"throwawaymail.com",
"mailinator.com",
"mailinator.net",
"mailinator.org",
"mailinator.info",
"maildrop.cc",
"yopmail.com",
"yopmail.fr",
"yopmail.net",
"fakeinbox.com",
"fakemailgenerator.com",
"dispostable.com",
"getairmail.com",
"getnada.com",
"mohmal.com",
"trashmail.com",
"trashmail.net",
"trashmail.org",
"sharklasers.com",
"grr.la",
"guerrillamail.de",
"pokemail.net",
"spam4.me",
"spamgourmet.com",
"mytrashmail.com",
"mailnesia.com",
"mailcatch.com",
"mintemail.com",
"tempr.email",
"tempail.com",
"emailondeck.com",
"incognitomail.com",
"inboxalias.com",
"33mail.com",
"spamex.com",
"spamfree24.org",
"spamspot.com",
"mailnull.com",
"mailsac.com",
"emailfake.com",
"fakemail.fr",
"tempinbox.com",
"throwaway.email",
"burnermail.io",
"anonbox.net",
"anonymbox.com",
"discard.email",
"discardmail.com",
"discardmail.de",
"mailexpire.com",
"mailforspam.com",
"meltmail.com",
"mt2009.com",
"mt2014.com",
"nospam.ze.tc",
"nospamfor.us",
"nowmymail.com",
"receiveee.com",
"safe-mail.net",
"spamavert.com",
"spambob.com",
"spambog.com",
"spambox.us",
"spamcannon.com",
"spamcannon.net",
"spamcon.org",
"spamcorptastic.com",
"spamday.com",
"spamfree.eu",
"spamherelots.com",
"spamhereplease.com",
"spamhole.com",
"spamify.com",
"spaminator.de",
"spamkill.info",
"spaml.com",
"spaml.de",
"spamoff.de",
"spamobox.com",
"spamslicer.com",
"spamstack.net",
"spamthis.co.uk",
"spamthisplease.com",
"supergreatmail.com",
"suremail.info",
"teleworm.us",
"tempemail.co.za",
"tempemail.net",
"tempmailaddress.com",
"tempmailo.com",
"thankyou2010.com",
"thisisnotmyrealemail.com",
"tm.slsrs.ru",
"tmpeml.info",
"trash-mail.at",
"trash-mail.de",
"trash2009.com",
"trashemail.de",
"trashmail.at",
"trashmailer.com",
"wegwerfmail.de",
"wegwerfmail.net",
"wegwerfmail.org",
"wh4f.org",
"willhackforfood.biz",
"willselfdestruct.com",
"xmaily.com",
"xyzfree.net",
"yep.it",
"yogamaven.com",
"yuurok.com",
"zehnminutenmail.de",
"zippymail.info",
# Additional common ones
"mailnator.com",
"bugmenot.com",
"dodgeit.com",
"dodgit.com",
"e4ward.com",
"emailsensei.com",
"hushmail.com",
"jetable.org",
"kasmail.com",
"mailblock.net",
"mailcatch.com",
"mymailoasis.com",
"nervmich.net",
"nervtmansen.de",
"oneoffemail.com",
"pookmail.com",
"shortmail.net",
"sneakemail.com",
"sogetthis.com",
"tempemailaddress.com",
"tempomail.fr",
"temporaryemail.net",
"temporaryforwarding.com",
"temporaryinbox.com",
"thankyou2010.com",
"tyldd.com",
"uggsrock.com",
"veryrealemail.com",
"yourewronghereswhy.com",
}
# Gmail-like domains that support alias normalization
GMAIL_LIKE_DOMAINS = {
"gmail.com",
"googlemail.com",
}
class EmailValidationService:
"""
Service for email validation and abuse prevention.
Features:
- Email normalization (Gmail alias stripping)
- Disposable domain detection
- Blocklist checking (emails, IPs)
- IP-based enrollment rate limiting
"""
_instance: Optional["EmailValidationService"] = None
_blocklist_cache: Optional[dict] = None
_blocklist_cache_time: Optional[datetime] = None
_cache_ttl_seconds: int = 300 # 5 minutes
def __init__(self, db: Optional[firestore.Client] = None):
"""Initialize the email validation service."""
if db is None:
from backend.services.firestore_service import get_firestore_client
db = get_firestore_client()
self.db = db
@classmethod
def get_instance(cls, db: Optional[firestore.Client] = None) -> "EmailValidationService":
"""Get singleton instance of the service."""
if cls._instance is None:
cls._instance = cls(db)
return cls._instance
def normalize_email(self, email: str) -> str:
"""
Normalize an email address.
For Gmail-like domains:
- Removes dots from local part
- Removes everything after + in local part
- Converts to lowercase
For other domains:
- Only converts to lowercase
Args:
email: Email address to normalize
Returns:
Normalized email address
Examples:
j.o.h.n+spam@gmail.com -> john@gmail.com
John.Doe@example.com -> john.doe@example.com
"""
if not email or "@" not in email:
return email.lower().strip() if email else ""
email = email.lower().strip()
local, domain = email.rsplit("@", 1)
if domain in GMAIL_LIKE_DOMAINS:
# Remove dots from local part
local = local.replace(".", "")
# Remove everything after +
if "+" in local:
local = local.split("+")[0]
return f"{local}@{domain}"
def get_blocklist_config(self, force_refresh: bool = False) -> dict:
"""
Get blocklist configuration from Firestore.
Uses caching to reduce Firestore reads.
Returns:
Dict with disposable_domains, blocked_emails, blocked_ips sets
"""
now = datetime.now(timezone.utc)
# Check cache validity
if (
not force_refresh
and self._blocklist_cache is not None
and self._blocklist_cache_time is not None
and (now - self._blocklist_cache_time).total_seconds() < self._cache_ttl_seconds
):
return self._blocklist_cache
# Fetch from Firestore
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC)
doc = doc_ref.get()
if doc.exists:
data = doc.to_dict()
config = {
"disposable_domains": set(data.get("disposable_domains", [])) | DEFAULT_DISPOSABLE_DOMAINS,
"blocked_emails": set(data.get("blocked_emails", [])),
"blocked_ips": set(data.get("blocked_ips", [])),
}
else:
# Use defaults if no config exists
config = {
"disposable_domains": DEFAULT_DISPOSABLE_DOMAINS.copy(),
"blocked_emails": set(),
"blocked_ips": set(),
}
# Update cache
EmailValidationService._blocklist_cache = config
EmailValidationService._blocklist_cache_time = now
return config
def is_disposable_domain(self, email: str) -> bool:
"""
Check if an email uses a disposable domain.
Args:
email: Email address to check
Returns:
True if the domain is known to be disposable
"""
if not email or "@" not in email:
return False
domain = email.lower().split("@")[-1]
config = self.get_blocklist_config()
return domain in config["disposable_domains"]
def is_email_blocked(self, email: str) -> bool:
"""
Check if an email is explicitly blocked.
Checks both the raw email and normalized version.
Args:
email: Email address to check
Returns:
True if the email is blocked
"""
if not email:
return False
config = self.get_blocklist_config()
email_lower = email.lower().strip()
email_normalized = self.normalize_email(email)
return (
email_lower in config["blocked_emails"]
or email_normalized in config["blocked_emails"]
)
def is_ip_blocked(self, ip_address: str) -> bool:
"""
Check if an IP address is blocked.
Args:
ip_address: IP address to check
Returns:
True if the IP is blocked
"""
if not ip_address:
return False
config = self.get_blocklist_config()
return ip_address in config["blocked_ips"]
def validate_email_for_beta(self, email: str) -> Tuple[bool, str]:
"""
Validate an email for beta enrollment.
Performs all validation checks:
1. Basic format validation
2. Disposable domain check
3. Blocked email check
Args:
email: Email address to validate
Returns:
Tuple of (is_valid, error_message)
"""
if not email or "@" not in email:
return False, "Invalid email format"
# Check disposable domain
if self.is_disposable_domain(email):
logger.warning(f"Beta enrollment blocked - disposable domain: {_mask_email(email)}")
return False, "Disposable email addresses are not allowed"
# Check blocked email
if self.is_email_blocked(email):
logger.warning(f"Beta enrollment blocked - email blocked: {_mask_email(email)}")
return False, "This email address is not allowed"
return True, ""
def hash_ip(self, ip_address: str) -> str:
"""
Hash an IP address for privacy-preserving storage.
Args:
ip_address: IP address to hash
Returns:
SHA-256 hash of the IP
"""
return hashlib.sha256(ip_address.encode()).hexdigest()
# -------------------------------------------------------------------------
# Blocklist Management (Admin)
# -------------------------------------------------------------------------
def add_disposable_domain(self, domain: str, admin_email: str) -> bool:
"""Add a domain to the disposable domains blocklist."""
domain = domain.lower().strip()
if not domain:
return False
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC)
@firestore.transactional
def update_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if doc.exists:
data = doc.to_dict()
domains = set(data.get("disposable_domains", []))
else:
data = {}
domains = set()
domains.add(domain)
data["disposable_domains"] = list(domains)
data["updated_at"] = datetime.now(timezone.utc)
data["updated_by"] = admin_email
transaction.set(doc_ref, data, merge=True)
transaction = self.db.transaction()
update_in_transaction(transaction, doc_ref)
# Invalidate cache
EmailValidationService._blocklist_cache = None
logger.info(f"Added disposable domain: {domain} by {admin_email}")
return True
def remove_disposable_domain(self, domain: str, admin_email: str) -> bool:
"""Remove a domain from the disposable domains blocklist."""
domain = domain.lower().strip()
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC)
result = {"found": False}
@firestore.transactional
def remove_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if not doc.exists:
return
data = doc.to_dict()
domains = set(data.get("disposable_domains", []))
if domain not in domains:
return
result["found"] = True
domains.discard(domain)
data["disposable_domains"] = list(domains)
data["updated_at"] = datetime.now(timezone.utc)
data["updated_by"] = admin_email
transaction.set(doc_ref, data, merge=True)
transaction = self.db.transaction()
remove_in_transaction(transaction, doc_ref)
if not result["found"]:
return False
# Invalidate cache
EmailValidationService._blocklist_cache = None
logger.info(f"Removed disposable domain: {domain} by {admin_email}")
return True
def add_blocked_email(self, email: str, admin_email: str) -> bool:
"""Add an email to the blocked emails list."""
email = email.lower().strip()
if not email:
return False
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC)
@firestore.transactional
def update_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if doc.exists:
data = doc.to_dict()
emails = set(data.get("blocked_emails", []))
else:
data = {}
emails = set()
emails.add(email)
data["blocked_emails"] = list(emails)
data["updated_at"] = datetime.now(timezone.utc)
data["updated_by"] = admin_email
transaction.set(doc_ref, data, merge=True)
transaction = self.db.transaction()
update_in_transaction(transaction, doc_ref)
# Invalidate cache
EmailValidationService._blocklist_cache = None
logger.info(f"Added blocked email: {email} by {admin_email}")
return True
def remove_blocked_email(self, email: str, admin_email: str) -> bool:
"""Remove an email from the blocked emails list."""
email = email.lower().strip()
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC)
result = {"found": False}
@firestore.transactional
def remove_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if not doc.exists:
return
data = doc.to_dict()
emails = set(data.get("blocked_emails", []))
if email not in emails:
return
result["found"] = True
emails.discard(email)
data["blocked_emails"] = list(emails)
data["updated_at"] = datetime.now(timezone.utc)
data["updated_by"] = admin_email
transaction.set(doc_ref, data, merge=True)
transaction = self.db.transaction()
remove_in_transaction(transaction, doc_ref)
if not result["found"]:
return False
# Invalidate cache
EmailValidationService._blocklist_cache = None
logger.info(f"Removed blocked email: {email} by {admin_email}")
return True
def add_blocked_ip(self, ip_address: str, admin_email: str) -> bool:
"""Add an IP address to the blocked IPs list."""
ip_address = ip_address.strip()
if not ip_address:
return False
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC)
@firestore.transactional
def update_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if doc.exists:
data = doc.to_dict()
ips = set(data.get("blocked_ips", []))
else:
data = {}
ips = set()
ips.add(ip_address)
data["blocked_ips"] = list(ips)
data["updated_at"] = datetime.now(timezone.utc)
data["updated_by"] = admin_email
transaction.set(doc_ref, data, merge=True)
transaction = self.db.transaction()
update_in_transaction(transaction, doc_ref)
# Invalidate cache
EmailValidationService._blocklist_cache = None
logger.info(f"Added blocked IP: {ip_address} by {admin_email}")
return True
def remove_blocked_ip(self, ip_address: str, admin_email: str) -> bool:
"""Remove an IP address from the blocked IPs list."""
ip_address = ip_address.strip()
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC)
result = {"found": False}
@firestore.transactional
def remove_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if not doc.exists:
return
data = doc.to_dict()
ips = set(data.get("blocked_ips", []))
if ip_address not in ips:
return
result["found"] = True
ips.discard(ip_address)
data["blocked_ips"] = list(ips)
data["updated_at"] = datetime.now(timezone.utc)
data["updated_by"] = admin_email
transaction.set(doc_ref, data, merge=True)
transaction = self.db.transaction()
remove_in_transaction(transaction, doc_ref)
if not result["found"]:
return False
# Invalidate cache
EmailValidationService._blocklist_cache = None
logger.info(f"Removed blocked IP: {ip_address} by {admin_email}")
return True
def get_blocklist_stats(self) -> dict:
"""Get statistics about current blocklists."""
config = self.get_blocklist_config(force_refresh=True)
return {
"disposable_domains_count": len(config["disposable_domains"]),
"blocked_emails_count": len(config["blocked_emails"]),
"blocked_ips_count": len(config["blocked_ips"]),
"default_disposable_domains_count": len(DEFAULT_DISPOSABLE_DOMAINS),
}
def get_email_validation_service(
db: Optional[firestore.Client] = None,
) -> EmailValidationService:
"""Get the singleton EmailValidationService instance."""
return EmailValidationService.get_instance(db)
-454
View File
@@ -1,454 +0,0 @@
"""
Encoding Interface.
Defines the abstract interface for video encoding backends, allowing
the video worker orchestrator to use either GCE or local encoding
interchangeably.
This follows the Strategy pattern - different encoding implementations
can be swapped without changing the orchestration logic.
"""
import logging
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from typing import Dict, List, Optional, Any
logger = logging.getLogger(__name__)
@dataclass
class EncodingInput:
"""
Input configuration for video encoding.
Contains all the paths and metadata needed to encode a karaoke video.
"""
# Required input files
title_video_path: str # Title card video (MOV)
karaoke_video_path: str # Main karaoke video with vocals (MOV/MKV)
instrumental_audio_path: str # Instrumental audio track (FLAC)
# Optional input files
end_video_path: Optional[str] = None # End credits video (MOV)
# Metadata for output naming
artist: str = ""
title: str = ""
brand_code: Optional[str] = None
# Output directory
output_dir: str = ""
# Additional options
options: Dict[str, Any] = field(default_factory=dict)
@dataclass
class EncodingOutput:
"""
Output from video encoding.
Contains paths to all generated video files and status information.
"""
success: bool
error_message: Optional[str] = None
# Output file paths (relative to output_dir or absolute)
karaoke_mp4_path: Optional[str] = None # Karaoke video with instrumental audio
with_vocals_mp4_path: Optional[str] = None # With vocals MP4 version
lossless_4k_mp4_path: Optional[str] = None # Final lossless 4K MP4
lossy_4k_mp4_path: Optional[str] = None # Final lossy 4K MP4
lossless_mkv_path: Optional[str] = None # MKV with FLAC (for YouTube)
lossy_720p_mp4_path: Optional[str] = None # 720p web version
# All output files as a dict for convenience
output_files: Dict[str, str] = field(default_factory=dict)
# Encoding metadata
encoding_time_seconds: Optional[float] = None
encoding_backend: Optional[str] = None # "gce" or "local"
class EncodingBackend(ABC):
"""
Abstract base class for video encoding backends.
Implementations:
- GCEEncodingBackend: Cloud-based encoding using Google Compute Engine
- LocalEncodingBackend: Local FFmpeg-based encoding
"""
@property
@abstractmethod
def name(self) -> str:
"""Return the name of this encoding backend."""
pass
@abstractmethod
async def encode(self, input_config: EncodingInput) -> EncodingOutput:
"""
Encode video files according to the input configuration.
This method should:
1. Create karaoke video with instrumental audio
2. Convert with-vocals video to MP4 if needed
3. Encode lossless 4K MP4 (concatenating title + karaoke + end)
4. Encode lossy 4K MP4 with AAC audio
5. Create MKV with FLAC audio for YouTube
6. Encode 720p version for web
Args:
input_config: EncodingInput with all required paths and options
Returns:
EncodingOutput with paths to generated files and status
"""
pass
@abstractmethod
async def is_available(self) -> bool:
"""
Check if this encoding backend is available and configured.
Returns:
True if the backend can be used, False otherwise
"""
pass
@abstractmethod
async def get_status(self) -> Dict[str, Any]:
"""
Get the current status of the encoding backend.
Returns:
Dict with status information (availability, queue length, etc.)
"""
pass
class LocalEncodingBackend(EncodingBackend):
"""
Local FFmpeg-based encoding backend.
Wraps the LocalEncodingService to implement the EncodingBackend interface.
Uses asyncio.to_thread() to run synchronous FFmpeg operations asynchronously.
"""
def __init__(
self,
dry_run: bool = False,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the local encoding backend.
Args:
dry_run: If True, log operations without executing
logger: Optional logger instance
"""
self.dry_run = dry_run
self.logger = logger or logging.getLogger(__name__)
self._service = None
@property
def name(self) -> str:
return "local"
def _get_service(self):
"""Lazy-load the local encoding service."""
if self._service is None:
from backend.services.local_encoding_service import LocalEncodingService
self._service = LocalEncodingService(
dry_run=self.dry_run,
logger=self.logger
)
return self._service
async def encode(self, input_config: EncodingInput) -> EncodingOutput:
"""
Encode video using local FFmpeg.
Args:
input_config: Encoding input configuration
Returns:
EncodingOutput with results
"""
import asyncio
import time
from backend.services.local_encoding_service import EncodingConfig
start_time = time.time()
# Build output file paths
base_name = f"{input_config.artist} - {input_config.title}"
output_dir = input_config.output_dir or "."
import os
config = EncodingConfig(
title_video=input_config.title_video_path,
karaoke_video=input_config.karaoke_video_path,
instrumental_audio=input_config.instrumental_audio_path,
end_video=input_config.end_video_path,
output_karaoke_mp4=os.path.join(output_dir, f"{base_name} (Karaoke).mp4"),
output_with_vocals_mp4=os.path.join(output_dir, f"{base_name} (With Vocals).mp4"),
output_lossless_4k_mp4=os.path.join(output_dir, f"{base_name} (Final Karaoke Lossless 4k).mp4"),
output_lossy_4k_mp4=os.path.join(output_dir, f"{base_name} (Final Karaoke Lossy 4k).mp4"),
output_lossless_mkv=os.path.join(output_dir, f"{base_name} (Final Karaoke Lossless 4k).mkv"),
output_720p_mp4=os.path.join(output_dir, f"{base_name} (Final Karaoke Lossy 720p).mp4"),
)
# Run encoding in thread pool to avoid blocking
service = self._get_service()
result = await asyncio.to_thread(service.encode_all_formats, config)
encoding_time = time.time() - start_time
if result.success:
return EncodingOutput(
success=True,
karaoke_mp4_path=config.output_karaoke_mp4,
with_vocals_mp4_path=config.output_with_vocals_mp4,
lossless_4k_mp4_path=config.output_lossless_4k_mp4,
lossy_4k_mp4_path=config.output_lossy_4k_mp4,
lossless_mkv_path=config.output_lossless_mkv,
lossy_720p_mp4_path=config.output_720p_mp4,
output_files=result.output_files,
encoding_time_seconds=encoding_time,
encoding_backend=self.name
)
else:
return EncodingOutput(
success=False,
error_message=result.error,
output_files=result.output_files,
encoding_time_seconds=encoding_time,
encoding_backend=self.name
)
async def is_available(self) -> bool:
"""Check if local encoding is available (FFmpeg installed)."""
import subprocess
import asyncio
try:
result = await asyncio.to_thread(
subprocess.run,
["ffmpeg", "-version"],
capture_output=True,
timeout=10
)
return result.returncode == 0
except (FileNotFoundError, subprocess.TimeoutExpired):
return False
async def get_status(self) -> Dict[str, Any]:
"""Get local encoding status."""
available = await self.is_available()
service = self._get_service()
return {
"backend": self.name,
"available": available,
"hwaccel_available": service.hwaccel_available if available else False,
"video_encoder": service.video_encoder if available else None,
}
class GCEEncodingBackend(EncodingBackend):
"""
GCE-based encoding backend.
Wraps the existing EncodingService to implement the EncodingBackend interface.
Submits jobs to a remote GCE worker for high-performance encoding.
"""
def __init__(
self,
dry_run: bool = False,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the GCE encoding backend.
Args:
dry_run: Ignored for GCE backend (remote service doesn't support dry run)
logger: Optional logger instance
"""
self.dry_run = dry_run # Stored but not used (GCE is remote)
self.logger = logger or logging.getLogger(__name__)
self._service = None
@property
def name(self) -> str:
return "gce"
def _get_service(self):
"""Lazy-load the GCE encoding service."""
if self._service is None:
from backend.services.encoding_service import get_encoding_service
self._service = get_encoding_service()
return self._service
async def encode(self, input_config: EncodingInput) -> EncodingOutput:
"""
Encode video using GCE worker.
Note: GCE encoding requires files to be in GCS, so this method
expects input_config.options to contain 'input_gcs_path' and
'output_gcs_path'.
Args:
input_config: Encoding input configuration
Returns:
EncodingOutput with results
"""
import time
start_time = time.time()
service = self._get_service()
# GCE requires GCS paths
input_gcs_path = input_config.options.get("input_gcs_path")
output_gcs_path = input_config.options.get("output_gcs_path")
job_id = input_config.options.get("job_id", input_config.brand_code or "unknown")
if not input_gcs_path or not output_gcs_path:
return EncodingOutput(
success=False,
error_message="GCE encoding requires input_gcs_path and output_gcs_path in options",
encoding_backend=self.name
)
try:
# Build encoding config
encoding_config = {
"formats": ["mp4_4k_lossless", "mp4_4k_lossy", "mkv_4k", "mp4_720p"],
"artist": input_config.artist,
"title": input_config.title,
}
# Submit and wait for completion
result = await service.encode_videos(
job_id=job_id,
input_gcs_path=input_gcs_path,
output_gcs_path=output_gcs_path,
encoding_config=encoding_config,
)
encoding_time = time.time() - start_time
# Extract output file paths from result
# Handle case where GCE worker returns a list or unexpected format
if isinstance(result, list):
# If result is a list, try to find the output_files in the first dict
self.logger.warning(f"GCE returned list instead of dict: {result}")
result = result[0] if result and isinstance(result[0], dict) else {}
if not isinstance(result, dict):
self.logger.error(f"Unexpected GCE result type: {type(result)}")
result = {}
raw_output_files = result.get("output_files", {})
# Convert output_files from list of paths to dict
# GCE worker returns list like: ["path/Artist - Title (Final Karaoke Lossless 4k).mp4", ...]
# We need dict like: {"mp4_4k_lossless": "path/...", "mp4_720p": "path/..."}
if isinstance(raw_output_files, list):
output_files = {}
for path in raw_output_files:
if not isinstance(path, str):
continue
filename = path.split("/")[-1] if "/" in path else path
filename_lower = filename.lower()
# Map filename patterns to output format keys
# Files are named like "Artist - Title (Final Karaoke Lossless 4k).mp4"
if "lossless 4k" in filename_lower:
if filename.endswith(".mkv"):
output_files["mkv_4k"] = path
else:
output_files["mp4_4k_lossless"] = path
elif "lossy 4k" in filename_lower:
output_files["mp4_4k_lossy"] = path
elif "720p" in filename_lower:
output_files["mp4_720p"] = path
self.logger.info(f"Converted output_files list to dict: {output_files}")
else:
output_files = raw_output_files if isinstance(raw_output_files, dict) else {}
return EncodingOutput(
success=True,
lossless_4k_mp4_path=output_files.get("mp4_4k_lossless"),
lossy_4k_mp4_path=output_files.get("mp4_4k_lossy"),
lossless_mkv_path=output_files.get("mkv_4k"),
lossy_720p_mp4_path=output_files.get("mp4_720p"),
output_files=output_files,
encoding_time_seconds=encoding_time,
encoding_backend=self.name
)
except Exception as e:
encoding_time = time.time() - start_time
self.logger.error(f"GCE encoding failed: {e}")
return EncodingOutput(
success=False,
error_message=str(e),
encoding_time_seconds=encoding_time,
encoding_backend=self.name
)
async def is_available(self) -> bool:
"""Check if GCE encoding is available and configured."""
service = self._get_service()
return service.is_enabled
async def get_status(self) -> Dict[str, Any]:
"""Get GCE encoding status."""
service = self._get_service()
status = {
"backend": self.name,
"available": service.is_enabled,
"configured": service.is_configured,
}
if service.is_configured:
try:
health = await service.health_check()
status["health"] = health
except Exception as e:
status["health_error"] = str(e)
return status
# Factory function to get an encoding backend
def get_encoding_backend(
backend_type: str = "auto",
**kwargs
) -> EncodingBackend:
"""
Get an encoding backend instance.
Args:
backend_type: Type of backend - "local", "gce", or "auto"
**kwargs: Additional arguments for the backend
Returns:
EncodingBackend instance
Raises:
ValueError: If backend_type is unknown
"""
if backend_type == "local":
return LocalEncodingBackend(**kwargs)
elif backend_type == "gce":
return GCEEncodingBackend(**kwargs)
elif backend_type == "auto":
# Check if GCE is available, otherwise use local
gce_backend = GCEEncodingBackend(**kwargs)
if gce_backend._get_service().is_enabled:
return gce_backend
return LocalEncodingBackend(**kwargs)
else:
raise ValueError(f"Unknown encoding backend type: {backend_type}")
-502
View File
@@ -1,502 +0,0 @@
"""
GCE Encoding Worker Service.
This service dispatches video encoding jobs to a dedicated high-performance
GCE instance (C4-standard with Intel Granite Rapids CPU) for faster encoding.
The GCE worker provides:
- 3.9 GHz all-core frequency (vs 3.7 GHz on Cloud Run)
- Dedicated vCPUs (no contention)
- 2-3x faster FFmpeg libx264 encoding
Usage:
encoding_service = get_encoding_service()
if encoding_service.is_configured:
result = await encoding_service.encode_videos(job_id, input_gcs_path, config)
"""
import asyncio
import logging
from typing import Optional, Dict, Any
import aiohttp
from backend.config import get_settings
logger = logging.getLogger(__name__)
# Retry configuration for handling transient failures (e.g., worker restarts)
MAX_RETRIES = 3
INITIAL_BACKOFF_SECONDS = 2.0
MAX_BACKOFF_SECONDS = 10.0
class EncodingService:
"""Service for dispatching encoding jobs to GCE worker."""
def __init__(self):
self.settings = get_settings()
self._url = None
self._api_key = None
self._initialized = False
def _load_credentials(self):
"""Load encoding worker URL and API key from config/secrets."""
if self._initialized:
return
# Try environment variables first, then Secret Manager
self._url = self.settings.encoding_worker_url
self._api_key = self.settings.encoding_worker_api_key
# Fall back to Secret Manager
if not self._url:
self._url = self.settings.get_secret("encoding-worker-url")
if not self._api_key:
self._api_key = self.settings.get_secret("encoding-worker-api-key")
self._initialized = True
@property
def is_configured(self) -> bool:
"""Check if encoding service is configured with URL and API key."""
self._load_credentials()
return bool(self._url and self._api_key)
@property
def is_enabled(self) -> bool:
"""Check if GCE encoding is enabled and configured."""
return self.settings.use_gce_encoding and self.is_configured
@property
def is_preview_enabled(self) -> bool:
"""Check if GCE preview encoding is enabled and configured."""
return self.settings.use_gce_preview_encoding and self.is_configured
async def _request_with_retry(
self,
method: str,
url: str,
headers: Dict[str, str],
json_payload: Optional[Dict[str, Any]] = None,
timeout: float = 30.0,
job_id: str = "unknown",
) -> Dict[str, Any]:
"""
Make an HTTP request with retry logic for transient failures.
This handles connection errors that occur when the GCE worker is
restarting (e.g., during deployments) by retrying with exponential backoff.
Args:
method: HTTP method (GET, POST)
url: Request URL
headers: Request headers
json_payload: JSON body for POST requests
timeout: Request timeout in seconds
job_id: Job ID for logging
Returns:
Dict with keys:
- status (int): HTTP status code
- json (Any): Parsed JSON response body (if status 200, else None)
- text (str): Raw response text (if status != 200, else None)
Raises:
aiohttp.ClientConnectorError: If all retries fail due to connection errors
aiohttp.ServerDisconnectedError: If all retries fail due to server disconnect
asyncio.TimeoutError: If all retries fail due to timeout
"""
last_exception = None
backoff = INITIAL_BACKOFF_SECONDS
for attempt in range(MAX_RETRIES + 1):
try:
async with aiohttp.ClientSession() as session:
if method.upper() == "POST":
async with session.post(
url, json=json_payload, headers=headers, timeout=timeout
) as resp:
# Return a copy of the response data since we exit the context
return {
"status": resp.status,
"json": await resp.json() if resp.status == 200 else None,
"text": await resp.text() if resp.status != 200 else None,
}
else: # GET
async with session.get(
url, headers=headers, timeout=timeout
) as resp:
return {
"status": resp.status,
"json": await resp.json() if resp.status == 200 else None,
"text": await resp.text() if resp.status != 200 else None,
}
except (aiohttp.ClientConnectorError, aiohttp.ServerDisconnectedError, asyncio.TimeoutError) as e:
last_exception = e
if attempt < MAX_RETRIES:
logger.warning(
f"[job:{job_id}] GCE worker connection failed (attempt {attempt + 1}/{MAX_RETRIES + 1}): {e}. "
f"Retrying in {backoff:.1f}s..."
)
await asyncio.sleep(backoff)
backoff = min(backoff * 2, MAX_BACKOFF_SECONDS)
else:
logger.error(
f"[job:{job_id}] GCE worker connection failed after {MAX_RETRIES + 1} attempts: {e}"
)
raise last_exception
async def submit_encoding_job(
self,
job_id: str,
input_gcs_path: str,
output_gcs_path: str,
encoding_config: Dict[str, Any],
) -> Dict[str, Any]:
"""
Submit an encoding job to the GCE worker.
Args:
job_id: Unique job identifier
input_gcs_path: GCS path to input files (gs://bucket/path/)
output_gcs_path: GCS path for output files (gs://bucket/path/)
encoding_config: Configuration for encoding (formats, quality, etc.)
Returns:
Response from the encoding worker
Raises:
Exception: If submission fails
"""
self._load_credentials()
if not self.is_configured:
raise RuntimeError("Encoding service not configured")
url = f"{self._url}/encode"
headers = {"X-API-Key": self._api_key, "Content-Type": "application/json"}
payload = {
"job_id": job_id,
"input_gcs_path": input_gcs_path,
"output_gcs_path": output_gcs_path,
"encoding_config": encoding_config,
}
logger.info(f"[job:{job_id}] Submitting encoding job to GCE worker: {url}")
resp = await self._request_with_retry(
method="POST",
url=url,
headers=headers,
json_payload=payload,
timeout=30.0,
job_id=job_id,
)
if resp["status"] == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp["status"] == 409:
raise RuntimeError(f"Encoding job {job_id} already exists")
if resp["status"] != 200:
raise RuntimeError(f"Failed to submit encoding job: {resp['status']} - {resp['text']}")
return resp["json"]
async def get_job_status(self, job_id: str) -> Dict[str, Any]:
"""
Get the status of an encoding job.
Args:
job_id: Job identifier
Returns:
Job status including: status, progress, error, output_files
"""
self._load_credentials()
if not self.is_configured:
raise RuntimeError("Encoding service not configured")
url = f"{self._url}/status/{job_id}"
headers = {"X-API-Key": self._api_key}
resp = await self._request_with_retry(
method="GET",
url=url,
headers=headers,
timeout=30.0,
job_id=job_id,
)
if resp["status"] == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp["status"] == 404:
raise RuntimeError(f"Encoding job {job_id} not found")
if resp["status"] != 200:
raise RuntimeError(f"Failed to get job status: {resp['status']} - {resp['text']}")
return resp["json"]
async def wait_for_completion(
self,
job_id: str,
poll_interval: float = 10.0,
timeout: float = 3600.0,
progress_callback=None,
) -> Dict[str, Any]:
"""
Poll for encoding job completion.
Args:
job_id: Job identifier
poll_interval: Seconds between status checks
timeout: Maximum time to wait (default 1 hour)
progress_callback: Optional callback(progress: int) for progress updates
Returns:
Final job status with output files
Raises:
TimeoutError: If job doesn't complete within timeout
RuntimeError: If job fails
"""
logger.info(f"[job:{job_id}] Waiting for GCE encoding to complete...")
start_time = asyncio.get_event_loop().time()
last_progress = 0
while True:
elapsed = asyncio.get_event_loop().time() - start_time
if elapsed > timeout:
raise TimeoutError(f"Encoding job {job_id} timed out after {timeout}s")
status = await self.get_job_status(job_id)
# Handle case where GCE worker returns a list instead of dict
if isinstance(status, list):
logger.warning(f"[job:{job_id}] GCE returned list instead of dict: {status}")
status = status[0] if status and isinstance(status[0], dict) else {}
if not isinstance(status, dict):
logger.error(f"[job:{job_id}] Unexpected status type: {type(status)}")
status = {}
job_status = status.get("status", "unknown")
progress = status.get("progress", 0)
# Report progress
if progress != last_progress:
logger.info(f"[job:{job_id}] Encoding progress: {progress}%")
last_progress = progress
if progress_callback:
try:
progress_callback(progress)
except Exception as e:
logger.warning(f"Progress callback failed: {e}")
if job_status == "complete":
logger.info(f"[job:{job_id}] GCE encoding complete in {elapsed:.1f}s")
return status
if job_status == "failed":
error = status.get("error", "Unknown error")
raise RuntimeError(f"Encoding job {job_id} failed: {error}")
await asyncio.sleep(poll_interval)
async def encode_videos(
self,
job_id: str,
input_gcs_path: str,
output_gcs_path: str,
encoding_config: Optional[Dict[str, Any]] = None,
progress_callback=None,
) -> Dict[str, Any]:
"""
Submit encoding job and wait for completion.
This is a convenience method that combines submit + wait.
Args:
job_id: Unique job identifier
input_gcs_path: GCS path to input files
output_gcs_path: GCS path for output files
encoding_config: Optional encoding configuration
progress_callback: Optional callback for progress updates
Returns:
Final job status with output files
"""
config = encoding_config or {"formats": ["mp4_4k", "mp4_720p"]}
# Submit the job
await self.submit_encoding_job(job_id, input_gcs_path, output_gcs_path, config)
# Wait for completion
return await self.wait_for_completion(
job_id, progress_callback=progress_callback
)
async def submit_preview_encoding_job(
self,
job_id: str,
ass_gcs_path: str,
audio_gcs_path: str,
output_gcs_path: str,
background_color: str = "black",
background_image_gcs_path: Optional[str] = None,
font_gcs_path: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a preview video encoding job to the GCE worker.
Args:
job_id: Unique job identifier
ass_gcs_path: GCS path to ASS subtitles file (gs://bucket/path/file.ass)
audio_gcs_path: GCS path to audio file
output_gcs_path: GCS path for output video
background_color: Background color (default: black)
background_image_gcs_path: Optional GCS path to background image
font_gcs_path: Optional GCS path to custom font file
Returns:
Response from the encoding worker
Raises:
Exception: If submission fails
"""
self._load_credentials()
if not self.is_configured:
raise RuntimeError("Encoding service not configured")
url = f"{self._url}/encode-preview"
headers = {"X-API-Key": self._api_key, "Content-Type": "application/json"}
payload = {
"job_id": job_id,
"ass_gcs_path": ass_gcs_path,
"audio_gcs_path": audio_gcs_path,
"output_gcs_path": output_gcs_path,
"background_color": background_color,
}
if background_image_gcs_path:
payload["background_image_gcs_path"] = background_image_gcs_path
if font_gcs_path:
payload["font_gcs_path"] = font_gcs_path
logger.info(f"[job:{job_id}] Submitting preview encoding job to GCE worker: {url}")
resp = await self._request_with_retry(
method="POST",
url=url,
headers=headers,
json_payload=payload,
timeout=30.0,
job_id=job_id,
)
if resp["status"] == 401:
raise RuntimeError("Invalid API key for encoding worker")
if resp["status"] == 409:
raise RuntimeError(f"Preview encoding job {job_id} already exists")
if resp["status"] != 200:
raise RuntimeError(f"Failed to submit preview encoding job: {resp['status']} - {resp['text']}")
return resp["json"]
async def encode_preview_video(
self,
job_id: str,
ass_gcs_path: str,
audio_gcs_path: str,
output_gcs_path: str,
background_color: str = "black",
background_image_gcs_path: Optional[str] = None,
font_gcs_path: Optional[str] = None,
timeout: float = 90.0,
poll_interval: float = 2.0,
) -> Dict[str, Any]:
"""
Submit preview encoding job and wait for completion.
This is a convenience method that combines submit + wait with shorter
timeouts suitable for preview videos.
Args:
job_id: Unique job identifier
ass_gcs_path: GCS path to ASS subtitles file
audio_gcs_path: GCS path to audio file
output_gcs_path: GCS path for output video
background_color: Background color (default: black)
background_image_gcs_path: Optional GCS path to background image
font_gcs_path: Optional GCS path to custom font file
timeout: Maximum time to wait (default 90s for preview)
poll_interval: Seconds between status checks (default 2s)
Returns:
Final job status with output files
"""
# Submit the job
submit_result = await self.submit_preview_encoding_job(
job_id=job_id,
ass_gcs_path=ass_gcs_path,
audio_gcs_path=audio_gcs_path,
output_gcs_path=output_gcs_path,
background_color=background_color,
background_image_gcs_path=background_image_gcs_path,
font_gcs_path=font_gcs_path,
)
# If cached, return immediately - video already exists in GCS
submit_status = submit_result.get("status")
if submit_status == "cached":
logger.info(f"[job:{job_id}] Preview already cached, returning immediately")
return {"status": "complete", "output_path": submit_result.get("output_path")}
# If in_progress, another request is encoding it - just wait for that
if submit_status == "in_progress":
logger.info(f"[job:{job_id}] Preview encoding already in progress, waiting")
# Wait for completion with shorter timeout
return await self.wait_for_completion(
job_id=job_id,
poll_interval=poll_interval,
timeout=timeout,
)
async def health_check(self) -> Dict[str, Any]:
"""
Check the health of the encoding worker.
Returns:
Health status including active jobs and FFmpeg version
"""
self._load_credentials()
if not self.is_configured:
return {"status": "not_configured"}
url = f"{self._url}/health"
headers = {"X-API-Key": self._api_key}
try:
async with aiohttp.ClientSession() as session:
async with session.get(url, headers=headers, timeout=10) as resp:
if resp.status == 200:
return await resp.json()
return {"status": "error", "code": resp.status}
except Exception as e:
return {"status": "error", "error": str(e)}
# Singleton instance
_encoding_service: Optional[EncodingService] = None
def get_encoding_service() -> EncodingService:
"""Get the singleton encoding service instance."""
global _encoding_service
if _encoding_service is None:
_encoding_service = EncodingService()
return _encoding_service
+12 -208
View File
@@ -9,7 +9,6 @@ from google.cloud.firestore_v1 import FieldFilter
from backend.config import settings
from backend.models.job import Job, JobStatus, TimelineEvent
from backend.models.worker_log import WorkerLogEntry
logger = logging.getLogger(__name__)
@@ -115,59 +114,47 @@ class FirestoreService:
client_id: Optional[str] = None,
created_after: Optional[datetime] = None,
created_before: Optional[datetime] = None,
user_email: Optional[str] = None,
tenant_id: Optional[str] = None,
limit: int = 100
) -> List[Job]:
"""
List jobs with optional filters.
Args:
status: Filter by job status
environment: Filter by request_metadata.environment (test/production/development)
client_id: Filter by request_metadata.client_id
created_after: Filter jobs created after this datetime
created_before: Filter jobs created before this datetime
user_email: Filter by user_email (owner of the job)
tenant_id: Filter by tenant_id (white-label portal scoping)
limit: Maximum number of jobs to return
Returns:
List of Job objects matching filters, ordered by created_at descending
"""
try:
query = self.db.collection(self.collection)
if status:
query = query.where(filter=FieldFilter('status', '==', status.value))
# Filter by request_metadata fields using dot notation
if environment:
query = query.where(filter=FieldFilter('request_metadata.environment', '==', environment))
if client_id:
query = query.where(filter=FieldFilter('request_metadata.client_id', '==', client_id))
# Filter by user_email (job owner)
if user_email:
query = query.where(filter=FieldFilter('user_email', '==', user_email.lower()))
# Filter by tenant_id (white-label portal scoping)
if tenant_id:
query = query.where(filter=FieldFilter('tenant_id', '==', tenant_id))
# Date range filters
if created_after:
query = query.where(filter=FieldFilter('created_at', '>=', created_after))
if created_before:
query = query.where(filter=FieldFilter('created_at', '<=', created_before))
query = query.order_by('created_at', direction=firestore.Query.DESCENDING).limit(limit)
docs = query.stream()
jobs = [Job(**doc.to_dict()) for doc in docs]
return jobs
except Exception as e:
logger.error(f"Error listing jobs: {e}")
@@ -252,13 +239,10 @@ class FirestoreService:
def append_worker_log(self, job_id: str, log_entry: Dict[str, Any]) -> None:
"""
Atomically append a log entry to worker_logs using ArrayUnion.
This avoids the race condition of read-modify-write when multiple
workers are logging concurrently.
DEPRECATED: Use append_log_to_subcollection() instead to avoid
the 1MB document size limit.
Args:
job_id: Job ID
log_entry: Log entry dict with timestamp, level, worker, message
@@ -273,165 +257,6 @@ class FirestoreService:
except Exception as e:
# Log but don't raise - logging shouldn't break workers
logger.debug(f"Error appending worker log for job {job_id}: {e}")
# ============================================
# Worker Log Subcollection Methods
# ============================================
# These methods store logs in a subcollection (jobs/{job_id}/logs)
# instead of an embedded array to avoid the 1MB document size limit.
def append_log_to_subcollection(self, job_id: str, log_entry: WorkerLogEntry) -> None:
"""
Append a log entry to the logs subcollection.
Stores logs at: jobs/{job_id}/logs/{log_id}
This approach avoids the 1MB document size limit by storing each
log entry as a separate document in a subcollection.
Args:
job_id: Job ID
log_entry: WorkerLogEntry instance
"""
try:
# Ensure job_id is set on the log entry
log_entry.job_id = job_id
# Get subcollection reference
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
# Add document with auto-generated ID or use log_entry.id
doc_ref = logs_ref.document(log_entry.id)
doc_ref.set(log_entry.to_dict())
# Don't log every append - too spammy
except Exception as e:
# Log but don't raise - logging shouldn't break workers
logger.debug(f"Error appending log to subcollection for job {job_id}: {e}")
def get_logs_from_subcollection(
self,
job_id: str,
limit: int = 500,
since_timestamp: Optional[datetime] = None,
worker: Optional[str] = None,
offset: int = 0
) -> List[WorkerLogEntry]:
"""
Get log entries from the logs subcollection.
Args:
job_id: Job ID
limit: Maximum number of logs to return (default 500)
since_timestamp: Return only logs after this timestamp
worker: Filter by worker name (optional)
offset: Number of logs to skip (for pagination)
Returns:
List of WorkerLogEntry instances, ordered by timestamp ascending
"""
try:
# Get subcollection reference
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
# Build query
query = logs_ref.order_by("timestamp", direction=firestore.Query.ASCENDING)
if since_timestamp:
query = query.where(filter=FieldFilter("timestamp", ">", since_timestamp))
if worker:
query = query.where(filter=FieldFilter("worker", "==", worker))
# Apply offset and limit
if offset > 0:
query = query.offset(offset)
query = query.limit(limit)
# Execute query
docs = query.stream()
logs = [WorkerLogEntry.from_dict(doc.to_dict()) for doc in docs]
return logs
except Exception as e:
logger.error(f"Error getting logs from subcollection for job {job_id}: {e}")
return []
def get_logs_count_from_subcollection(self, job_id: str) -> int:
"""
Get the total count of log entries in the subcollection.
Args:
job_id: Job ID
Returns:
Total count of log entries
"""
try:
# Get subcollection reference
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
# Use aggregation query for efficient counting
count_query = logs_ref.count()
result = count_query.get()
# Result is a list of AggregationResult, we want the first one's count
if result and len(result) > 0:
return result[0][0].value
return 0
except Exception as e:
logger.error(f"Error counting logs for job {job_id}: {e}")
return 0
def delete_logs_subcollection(self, job_id: str, batch_size: int = 500) -> int:
"""
Delete all log entries in the logs subcollection.
This is used when deleting a job to clean up its logs.
Args:
job_id: Job ID
batch_size: Number of documents to delete per batch
Returns:
Number of logs deleted
"""
try:
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
deleted_count = 0
while True:
# Get a batch of documents
docs = logs_ref.limit(batch_size).stream()
deleted_in_batch = 0
# Delete in a batch
batch = self.db.batch()
for doc in docs:
batch.delete(doc.reference)
deleted_in_batch += 1
if deleted_in_batch == 0:
break
batch.commit()
deleted_count += deleted_in_batch
# If we deleted less than batch_size, we're done
if deleted_in_batch < batch_size:
break
if deleted_count > 0:
logger.info(f"Deleted {deleted_count} logs for job {job_id}")
return deleted_count
except Exception as e:
logger.error(f"Error deleting logs subcollection for job {job_id}: {e}")
return 0
# ============================================
# Token Management Methods
@@ -516,24 +341,3 @@ class FirestoreService:
logger.error(f"Error listing tokens: {e}")
return []
# Singleton client instance
_firestore_client: Optional[firestore.Client] = None
def get_firestore_client() -> firestore.Client:
"""
Get a shared Firestore client instance.
This returns a raw Firestore client (not the FirestoreService) for use
in services that need direct Firestore access without the job-specific
abstractions provided by FirestoreService.
Returns:
Firestore client instance
"""
global _firestore_client
if _firestore_client is None:
_firestore_client = firestore.Client(project=settings.google_cloud_project)
return _firestore_client
-573
View File
@@ -1,573 +0,0 @@
"""
Client for the remote flacfetch HTTP API service.
This client communicates with a dedicated flacfetch VM that handles:
- BitTorrent downloads from private trackers (RED, OPS)
- YouTube downloads
- GCS uploads of downloaded files
The flacfetch service provides:
- Full peer connectivity for torrents (not possible in Cloud Run)
- Indefinite seeding of completed torrents
- Automatic disk cleanup
Usage:
client = get_flacfetch_client()
if client:
# Search for audio
search_result = await client.search("ABBA", "Waterloo")
# Download the best result
download_id = await client.download(
search_id=search_result["search_id"],
result_index=0,
gcs_path="uploads/job123/audio/",
)
# Wait for completion
result = await client.wait_for_download(download_id)
print(f"Downloaded to: {result['gcs_path']}")
"""
import asyncio
import logging
from typing import Any, Dict, List, Optional
import httpx
logger = logging.getLogger(__name__)
class FlacfetchServiceError(Exception):
"""Error communicating with or returned from flacfetch service."""
pass
class FlacfetchClient:
"""
Client for remote flacfetch HTTP API.
All endpoints require authentication via X-API-Key header.
"""
def __init__(
self,
base_url: str,
api_key: str,
timeout: int = 60,
):
"""
Initialize flacfetch client.
Args:
base_url: Base URL of flacfetch service (e.g., http://10.0.0.5:8080)
api_key: API key for authentication
timeout: Default request timeout in seconds
"""
self.base_url = base_url.rstrip('/')
self.api_key = api_key
self.timeout = timeout
logger.info(f"FlacfetchClient initialized with base_url={self.base_url}")
def _headers(self) -> Dict[str, str]:
"""Get request headers with authentication."""
return {
"X-API-Key": self.api_key,
"Content-Type": "application/json",
}
async def health_check(self) -> Dict[str, Any]:
"""
Check if flacfetch service is healthy.
Returns:
Health status dict with transmission, disk, and provider info
Raises:
FlacfetchServiceError: If service is unhealthy or unreachable
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.get(
f"{self.base_url}/health",
headers=self._headers(),
timeout=10,
)
resp.raise_for_status()
data = resp.json()
if data.get("status") not in ["healthy", "degraded"]:
raise FlacfetchServiceError(f"Service unhealthy: {data}")
return data
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Cannot reach flacfetch service: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(f"Health check failed: {e.response.status_code}")
async def search(
self,
artist: str,
title: str,
) -> Dict[str, Any]:
"""
Search for audio matching artist and title.
Args:
artist: Artist name
title: Track title
Returns:
Search response dict with search_id and results list
Raises:
FlacfetchServiceError: On search failure
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
f"{self.base_url}/search",
headers=self._headers(),
json={"artist": artist, "title": title},
timeout=self.timeout,
)
if resp.status_code == 404:
# No results found - return empty results
return {
"search_id": None,
"artist": artist,
"title": title,
"results": [],
"results_count": 0,
}
resp.raise_for_status()
return resp.json()
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Search request failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(f"Search failed: {e.response.status_code} - {e.response.text}")
async def download(
self,
search_id: str,
result_index: int,
output_filename: Optional[str] = None,
gcs_path: Optional[str] = None,
) -> str:
"""
Start downloading an audio file.
Args:
search_id: Search ID from previous search
result_index: Index of result to download
output_filename: Optional custom filename (without extension)
gcs_path: GCS path for upload (e.g., "uploads/job123/audio/")
Returns:
Download ID for tracking progress
Raises:
FlacfetchServiceError: On download start failure
"""
try:
async with httpx.AsyncClient() as client:
payload = {
"search_id": search_id,
"result_index": result_index,
}
if output_filename:
payload["output_filename"] = output_filename
if gcs_path:
payload["upload_to_gcs"] = True
payload["gcs_path"] = gcs_path
resp = await client.post(
f"{self.base_url}/download",
headers=self._headers(),
json=payload,
timeout=self.timeout,
)
resp.raise_for_status()
data = resp.json()
return data["download_id"]
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Download request failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(f"Download start failed: {e.response.status_code} - {e.response.text}")
async def download_by_id(
self,
source_name: str,
source_id: str,
output_filename: Optional[str] = None,
target_file: Optional[str] = None,
download_url: Optional[str] = None,
gcs_path: Optional[str] = None,
) -> str:
"""
Start downloading directly by source ID (no prior search required).
This is useful when you have stored the source_id from a previous search
and want to download later without re-searching.
Args:
source_name: Provider name (RED, OPS, YouTube, Spotify)
source_id: Source-specific ID (torrent ID, video ID, track ID)
output_filename: Optional custom filename (without extension)
target_file: For torrents, specific file to extract
download_url: For YouTube/Spotify, direct URL (optional)
gcs_path: GCS path for upload (e.g., "uploads/job123/audio/")
Returns:
Download ID for tracking progress
Raises:
FlacfetchServiceError: On download start failure
"""
try:
async with httpx.AsyncClient() as client:
payload = {
"source_name": source_name,
"source_id": source_id,
}
if output_filename:
payload["output_filename"] = output_filename
if target_file:
payload["target_file"] = target_file
if download_url:
payload["download_url"] = download_url
if gcs_path:
payload["upload_to_gcs"] = True
payload["gcs_path"] = gcs_path
resp = await client.post(
f"{self.base_url}/download-by-id",
headers=self._headers(),
json=payload,
timeout=self.timeout,
)
resp.raise_for_status()
data = resp.json()
return data["download_id"]
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Download by ID request failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(f"Download by ID failed: {e.response.status_code} - {e.response.text}")
async def get_download_status(self, download_id: str) -> Dict[str, Any]:
"""
Get the current status of a download.
Args:
download_id: Download ID from start_download
Returns:
Status dict with progress, speed, path info
Raises:
FlacfetchServiceError: On status check failure
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.get(
f"{self.base_url}/download/{download_id}/status",
headers=self._headers(),
timeout=10,
)
resp.raise_for_status()
return resp.json()
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Status request failed: {e}")
except httpx.HTTPStatusError as e:
if e.response.status_code == 404:
raise FlacfetchServiceError(f"Download not found: {download_id}")
raise FlacfetchServiceError(f"Status check failed: {e.response.status_code}")
async def wait_for_download(
self,
download_id: str,
timeout: int = 600,
poll_interval: float = 2.0,
progress_callback: Optional[callable] = None,
) -> Dict[str, Any]:
"""
Wait for a download to complete.
Args:
download_id: Download ID to wait for
timeout: Maximum wait time in seconds
poll_interval: Time between status checks
progress_callback: Optional callback(status_dict) for progress updates
Returns:
Final status dict with output_path or gcs_path
Raises:
FlacfetchServiceError: On download failure or timeout
"""
elapsed = 0.0
while elapsed < timeout:
status = await self.get_download_status(download_id)
if progress_callback:
try:
progress_callback(status)
except Exception as e:
logger.warning(f"Progress callback error: {e}")
download_status = status.get("status")
if download_status in ["complete", "seeding"]:
logger.info(f"Download {download_id} complete: {status.get('gcs_path') or status.get('output_path')}")
return status
elif download_status == "failed":
error = status.get("error", "Unknown error")
raise FlacfetchServiceError(f"Download failed: {error}")
elif download_status == "cancelled":
raise FlacfetchServiceError("Download was cancelled")
# Log progress
progress = status.get("progress", 0)
speed = status.get("download_speed_kbps", 0)
peers = status.get("peers", 0)
logger.debug(
f"Download {download_id}: {progress:.1f}% "
f"({speed:.1f} KB/s, {peers} peers)"
)
await asyncio.sleep(poll_interval)
elapsed += poll_interval
raise FlacfetchServiceError(f"Download timed out after {timeout}s")
async def list_torrents(self) -> Dict[str, Any]:
"""
List all torrents in Transmission.
Returns:
Torrent list response with torrent info and totals
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.get(
f"{self.base_url}/torrents",
headers=self._headers(),
timeout=30,
)
resp.raise_for_status()
return resp.json()
except httpx.RequestError as e:
raise FlacfetchServiceError(f"List torrents failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(f"List torrents failed: {e.response.status_code}")
async def delete_torrent(
self,
torrent_id: int,
delete_data: bool = True,
) -> Dict[str, Any]:
"""
Delete a torrent from Transmission.
Args:
torrent_id: Transmission torrent ID
delete_data: Also delete downloaded files
Returns:
Delete response
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.delete(
f"{self.base_url}/torrents/{torrent_id}",
headers=self._headers(),
params={"delete_data": str(delete_data).lower()},
timeout=30,
)
resp.raise_for_status()
return resp.json()
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Delete torrent failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(f"Delete torrent failed: {e.response.status_code}")
async def cleanup_torrents(
self,
strategy: str = "oldest",
target_free_gb: float = 10.0,
) -> Dict[str, Any]:
"""
Trigger disk cleanup.
Args:
strategy: Cleanup strategy (oldest, largest, lowest_ratio)
target_free_gb: Target free space to achieve
Returns:
Cleanup response with removed count and freed space
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
f"{self.base_url}/torrents/cleanup",
headers=self._headers(),
json={
"strategy": strategy,
"target_free_gb": target_free_gb,
},
timeout=60,
)
resp.raise_for_status()
return resp.json()
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Cleanup failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(f"Cleanup failed: {e.response.status_code}")
# =========================================================================
# Cache Management
# =========================================================================
async def clear_search_cache(self, artist: str, title: str) -> bool:
"""
Clear a specific cached search result by artist and title.
Args:
artist: Artist name
title: Track title
Returns:
True if a cache entry was deleted, False if no entry existed
Raises:
FlacfetchServiceError: On request failure
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.request(
"DELETE",
f"{self.base_url}/cache/search",
headers=self._headers(),
json={"artist": artist, "title": title},
timeout=30,
)
resp.raise_for_status()
data = resp.json()
deleted = data.get("deleted", False)
logger.info(
f"Cleared flacfetch cache for '{artist}' - '{title}': "
f"{'deleted' if deleted else 'no entry found'}"
)
return deleted
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Clear search cache request failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(
f"Clear search cache failed: {e.response.status_code} - {e.response.text}"
)
async def clear_all_cache(self) -> int:
"""
Clear all cached search results.
Returns:
Number of cache entries deleted
Raises:
FlacfetchServiceError: On request failure
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.delete(
f"{self.base_url}/cache",
headers=self._headers(),
timeout=60,
)
resp.raise_for_status()
data = resp.json()
deleted_count = data.get("deleted_count", 0)
logger.info(f"Cleared all flacfetch cache: {deleted_count} entries deleted")
return deleted_count
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Clear all cache request failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(
f"Clear all cache failed: {e.response.status_code} - {e.response.text}"
)
async def get_cache_stats(self) -> Dict[str, Any]:
"""
Get statistics about the cache.
Returns:
Dict with count, total_size_bytes, oldest_entry, newest_entry, configured
Raises:
FlacfetchServiceError: On request failure
"""
try:
async with httpx.AsyncClient() as client:
resp = await client.get(
f"{self.base_url}/cache/stats",
headers=self._headers(),
timeout=30,
)
resp.raise_for_status()
return resp.json()
except httpx.RequestError as e:
raise FlacfetchServiceError(f"Get cache stats request failed: {e}")
except httpx.HTTPStatusError as e:
raise FlacfetchServiceError(
f"Get cache stats failed: {e.response.status_code} - {e.response.text}"
)
# Singleton client instance
_client: Optional[FlacfetchClient] = None
def get_flacfetch_client() -> Optional[FlacfetchClient]:
"""
Get the flacfetch client if configured.
Returns:
FlacfetchClient instance if FLACFETCH_API_URL and FLACFETCH_API_KEY are set,
None otherwise (indicating local-only mode).
"""
global _client
if _client is None:
from backend.config import get_settings
settings = get_settings()
if settings.flacfetch_api_url and settings.flacfetch_api_key:
_client = FlacfetchClient(
base_url=settings.flacfetch_api_url,
api_key=settings.flacfetch_api_key,
)
logger.info("FlacfetchClient initialized for remote service")
else:
logger.debug("FlacfetchClient not configured (missing URL or API key)")
return _client
def reset_flacfetch_client():
"""Reset the singleton client (for testing)."""
global _client
_client = None
-72
View File
@@ -1,72 +0,0 @@
# GCE Encoding Worker
HTTP API service that runs on the GCE encoding worker VM for video encoding jobs.
## Overview
This service provides HTTP endpoints for submitting and monitoring video encoding jobs. It runs as a systemd service on the `encoding-worker` GCE VM instance.
## Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/encode` | POST | Submit a full encoding job |
| `/encode-preview` | POST | Submit a preview encoding job (480x270, fast) |
| `/status/{job_id}` | GET | Get job status and progress |
| `/health` | GET | Health check |
## Authentication
All endpoints (except `/health`) require an API key via the `X-API-Key` header. The key is stored in Secret Manager (`encoding-worker-api-key`).
## Encoding Process
### Full Encoding (`/encode`)
1. Downloads input files from GCS (title screen, karaoke video, end screen, instrumental audio)
2. Uses `LocalEncodingService` from the karaoke-gen wheel to produce:
- Lossless 4K MP4
- Lossy 4K MP4
- Lossless MKV
- 720p MP4
3. Uploads results to GCS
4. Reports progress via status endpoint
### Preview Encoding (`/encode-preview`)
1. Downloads ASS subtitle and audio files from GCS
2. Runs FFmpeg to create a quick 480x270 preview video
3. Uploads result to GCS
4. Used for real-time preview in the web UI
## Dependencies
- **FFmpeg**: Static build from John Van Sickle
- **Python 3.13**: Built from source (required for karaoke-gen)
- **karaoke-gen wheel**: Downloaded from GCS at job start (enables hot updates)
## Deployment
This service is deployed via the VM's startup script. The script:
1. Installs system dependencies (FFmpeg, fonts, Python build deps)
2. Builds Python 3.13 from source
3. Creates venv and installs dependencies
4. Writes `main.py` to `/opt/encoding-worker/`
5. Creates systemd service
6. Starts the service on port 8080
## Local Development
This code is extracted from the infrastructure for maintainability. To test locally:
```bash
cd backend/services/gce_encoding
pip install -r requirements.txt
pip install ../../../ # Install karaoke-gen wheel
ENCODING_API_KEY=test python main.py
```
## Architecture Notes
- **Hot code updates**: The karaoke-gen wheel is re-downloaded at the start of each job, allowing code updates without VM restart
- **In-memory job tracking**: Job state is stored in memory. Restart clears queue.
- **Parallel encoding**: ThreadPoolExecutor with 4 workers
- **GCS integration**: Direct download/upload via google-cloud-storage client
-22
View File
@@ -1,22 +0,0 @@
"""
GCE Encoding Worker Service.
This package contains the FastAPI application that runs on the GCE encoding worker VM.
It provides HTTP endpoints for video encoding jobs.
The service:
- Downloads input files from GCS
- Runs FFmpeg encoding via LocalEncodingService
- Uploads results back to GCS
- Provides job status tracking
Endpoints:
- POST /encode - Submit a full encoding job
- POST /encode-preview - Submit a preview encoding job (fast, low-res)
- GET /status/{job_id} - Get job status
- GET /health - Health check
"""
from .main import app
__all__ = ["app"]
-603
View File
@@ -1,603 +0,0 @@
import asyncio
import glob as glob_module
import json
import logging
import os
import subprocess
import sys
import tempfile
import uuid
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Optional
from fastapi import FastAPI, BackgroundTasks, HTTPException, Header, Depends
from google.cloud import storage
from pydantic import BaseModel
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
app = FastAPI(title="Encoding Worker", version="1.0.0")
# API key authentication
API_KEY = os.environ.get("ENCODING_API_KEY", "")
async def verify_api_key(x_api_key: str = Header(None)):
# Verify API key for authentication
if not API_KEY:
logger.warning("No API key configured - authentication disabled")
return True
if x_api_key != API_KEY:
raise HTTPException(status_code=401, detail="Invalid API key")
return True
# Job tracking
jobs: dict[str, dict] = {}
executor = ThreadPoolExecutor(max_workers=4) # 4 parallel encoding jobs
# GCS client
storage_client = storage.Client()
class EncodeRequest(BaseModel):
job_id: str
input_gcs_path: str # gs://bucket/path/to/inputs/
output_gcs_path: str # gs://bucket/path/to/outputs/
encoding_config: dict # Video formats to generate
class EncodePreviewRequest(BaseModel):
job_id: str
ass_gcs_path: str # gs://bucket/path/to/subtitles.ass
audio_gcs_path: str # gs://bucket/path/to/audio.flac
output_gcs_path: str # gs://bucket/path/to/output.mp4
background_color: str = "black"
background_image_gcs_path: Optional[str] = None
font_gcs_path: Optional[str] = None # gs://bucket/path/to/custom-font.ttf
class JobStatus(BaseModel):
job_id: str
status: str # pending, running, complete, failed
progress: int # 0-100
error: Optional[str] = None
output_files: Optional[list[str]] = None
def download_from_gcs(gcs_uri: str, local_path: Path):
# Download a file or folder from GCS
# Parse gs://bucket/path
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
prefix = parts[1] if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
blobs = list(bucket.list_blobs(prefix=prefix))
for blob in blobs:
# Get relative path from prefix
rel_path = blob.name[len(prefix):].lstrip("/")
if not rel_path:
continue
dest = local_path / rel_path
dest.parent.mkdir(parents=True, exist_ok=True)
blob.download_to_filename(str(dest))
logger.info(f"Downloaded: {blob.name} -> {dest}")
def upload_to_gcs(local_path: Path, gcs_uri: str):
# Upload a file or folder to GCS
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
prefix = parts[1].rstrip("/") if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
if local_path.is_file():
blob_name = f"{prefix}/{local_path.name}" if prefix else local_path.name
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(local_path))
logger.info(f"Uploaded: {local_path} -> gs://{bucket_name}/{blob_name}")
else:
for file in local_path.rglob("*"):
if file.is_file():
rel_path = file.relative_to(local_path)
blob_name = f"{prefix}/{rel_path}" if prefix else str(rel_path)
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(file))
logger.info(f"Uploaded: {file} -> gs://{bucket_name}/{blob_name}")
def download_single_file_from_gcs(gcs_uri: str, local_path: Path):
# Download a single file from GCS
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
blob_name = parts[1] if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(blob_name)
local_path.parent.mkdir(parents=True, exist_ok=True)
blob.download_to_filename(str(local_path))
logger.info(f"Downloaded: {gcs_uri} -> {local_path}")
def upload_single_file_to_gcs(local_path: Path, gcs_uri: str):
# Upload a single file to a specific GCS path
parts = gcs_uri.replace("gs://", "").split("/", 1)
bucket_name = parts[0]
blob_name = parts[1] if len(parts) > 1 else ""
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.upload_from_filename(str(local_path))
logger.info(f"Uploaded: {local_path} -> {gcs_uri}")
def run_preview_encoding(job_id: str, work_dir: Path, request: "EncodePreviewRequest"):
# Run FFmpeg encoding for preview video (480x270, fast settings)
jobs[job_id]["status"] = "running"
jobs[job_id]["progress"] = 10
try:
# Download input files
ass_path = work_dir / "subtitles.ass"
audio_path = work_dir / "audio.flac"
download_single_file_from_gcs(request.ass_gcs_path, ass_path)
jobs[job_id]["progress"] = 20
download_single_file_from_gcs(request.audio_gcs_path, audio_path)
jobs[job_id]["progress"] = 30
# Download background image if provided
bg_image_path = None
if request.background_image_gcs_path:
bg_image_path = work_dir / "background.png"
download_single_file_from_gcs(request.background_image_gcs_path, bg_image_path)
# Download custom font if provided and register with fontconfig
if request.font_gcs_path:
# Use standard fontconfig location that's already in the search path
fonts_dir = Path("/usr/local/share/fonts/custom")
fonts_dir.mkdir(parents=True, exist_ok=True)
font_filename = request.font_gcs_path.split("/")[-1]
font_path = fonts_dir / font_filename
download_single_file_from_gcs(request.font_gcs_path, font_path)
logger.info(f"Downloaded custom font: {font_path}")
# Update fontconfig cache so libass can find the font
subprocess.run(["fc-cache", "-fv"], capture_output=True)
logger.info(f"Updated fontconfig cache with custom font: {font_filename}")
# Build FFmpeg command
output_path = work_dir / "preview.mp4"
# Escape special characters in path for FFmpeg filter syntax
# FFmpeg filter parsing requires escaping: \ : , [ ] ;
def escape_ffmpeg_filter_path(path: str) -> str:
# Note: Extra escaping needed since this is inside a triple-quoted string in Pulumi
return path.replace("\\", "\\\\").replace(":", "\\:").replace(",", "\\,").replace("[", "\\[").replace("]", "\\]").replace(";", "\\;")
escaped_ass_path = escape_ffmpeg_filter_path(str(ass_path))
# Base command with frame rate
cmd = ["ffmpeg", "-y", "-r", "24"]
# Video input: background image or solid color
if bg_image_path and bg_image_path.exists():
cmd.extend(["-loop", "1", "-i", str(bg_image_path)])
# Scale and pad background to 480x270
vf = f"scale=480:270:force_original_aspect_ratio=decrease,pad=480:270:(ow-iw)/2:(oh-ih)/2,ass={escaped_ass_path}"
else:
# Solid color background
color = request.background_color or "black"
cmd.extend(["-f", "lavfi", "-i", f"color=c={color}:s=480x270:r=24"])
vf = f"ass={escaped_ass_path}"
# Audio input
cmd.extend(["-i", str(audio_path)])
# Video filter and encoding settings
cmd.extend([
"-vf", vf,
"-c:a", "aac", "-b:a", "96k",
"-c:v", "libx264",
"-preset", "superfast",
"-crf", "28",
"-pix_fmt", "yuv420p",
"-movflags", "+faststart",
"-threads", "8",
"-shortest",
str(output_path)
])
jobs[job_id]["progress"] = 40
logger.info(f"Running preview encoding: {' '.join(cmd)}")
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
logger.error(f"FFmpeg failed: {result.stderr}")
raise RuntimeError(f"FFmpeg preview encoding failed: {result.stderr[-500:]}")
jobs[job_id]["progress"] = 80
logger.info(f"Preview encoded: {output_path}")
# Upload output to GCS
upload_single_file_to_gcs(output_path, request.output_gcs_path)
jobs[job_id]["progress"] = 95
jobs[job_id]["output_files"] = [request.output_gcs_path]
return output_path
except Exception as e:
logger.error(f"Preview encoding failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
raise
def ensure_latest_wheel():
'''Download and install latest karaoke-gen wheel from GCS.
Called at the start of each job to enable hot code updates without restart.
In-progress jobs continue with their version, new jobs get latest code.
'''
try:
logger.info("Checking for latest karaoke-gen wheel in GCS...")
# Download latest wheel
result = subprocess.run(
["gsutil", "cp", "gs://karaoke-gen-storage-nomadkaraoke/wheels/karaoke_gen-*.whl", "/tmp/"],
capture_output=True, text=True, timeout=60
)
# Find the downloaded wheel (get the latest by version sorting)
wheels = glob_module.glob("/tmp/karaoke_gen-*.whl")
if not wheels:
logger.warning("No wheel found in GCS, using fallback encoding logic")
return False
# Sort to get latest version
wheel_path = sorted(wheels)[-1]
logger.info(f"Installing wheel: {wheel_path}")
# Install (or upgrade) the wheel
# Use 5-minute timeout - first install at job start may need to resolve dependencies
# Subsequent installs are faster since dependencies are cached
install_result = subprocess.run(
[sys.executable, "-m", "pip", "install", "--upgrade", "--quiet", wheel_path],
capture_output=True, text=True, timeout=300
)
if install_result.returncode != 0:
logger.warning(f"Wheel installation failed: {install_result.stderr}")
return False
logger.info(f"Successfully installed wheel: {wheel_path}")
return True
except subprocess.TimeoutExpired:
logger.warning("Wheel download/install timed out, using fallback")
return False
except Exception as e:
logger.warning(f"Failed to ensure latest wheel: {e}")
return False
def find_file(work_dir: Path, *patterns):
'''Find a file matching any of the given glob patterns.'''
for pattern in patterns:
matches = list(work_dir.glob(f"**/{pattern}"))
if matches:
return matches[0]
return None
def run_encoding(job_id: str, work_dir: Path, config: dict):
'''Run encoding using LocalEncodingService (single source of truth).
Uses LocalEncodingService from the installed karaoke-gen wheel to ensure
output files match local CLI exactly:
- Proper names like "Artist - Title (Final Karaoke Lossless 4k).mp4"
- Concatenated title + karaoke + end screens
- All formats: lossless 4K MP4, lossy 4K MP4, lossless MKV, 720p MP4
Requires the karaoke-gen wheel to be installed (done by ensure_latest_wheel).
'''
jobs[job_id]["status"] = "running"
jobs[job_id]["progress"] = 10
try:
# Import LocalEncodingService from installed wheel (required, no fallback)
from backend.services.local_encoding_service import LocalEncodingService, EncodingConfig
logger.info("Using LocalEncodingService from installed wheel")
# Get artist/title from config for proper naming
artist = config.get("artist", "Unknown Artist")
title = config.get("title", "Unknown Title")
base_name = f"{artist} - {title}"
logger.info(f"Encoding for: {base_name}")
# Find input files in work_dir
# Title/end screens are in screens/ subdirectory
title_video = find_file(work_dir, "screens/title.mov", "*Title*.mov", "*title*.mov")
end_video = find_file(work_dir, "screens/end.mov", "*End*.mov", "*end*.mov")
# Karaoke video - search for With Vocals or main karaoke video
karaoke_video = find_file(
work_dir,
"*With Vocals*.mov", "*With Vocals*.mkv",
"*Vocals*.mov", "*Vocals*.mkv",
"*.mkv", "*.mov"
)
# Exclude title/end/output videos
if karaoke_video:
name_lower = karaoke_video.name.lower()
if "title" in name_lower or "end" in name_lower or "outputs" in str(karaoke_video):
# Search more specifically for karaoke video
karaoke_video = find_file(work_dir, "*Karaoke*.mkv", "*Karaoke*.mov", "*vocals*.mkv")
# Instrumental audio - respect user's selection from encoding config
instrumental_selection = config.get("instrumental_selection", "clean")
logger.info(f"Instrumental selection from config: {instrumental_selection}")
if instrumental_selection == "with_backing":
# User selected instrumental with backing vocals
instrumental = find_file(
work_dir,
"*instrumental_with_backing*.flac", "*Instrumental Backing*.flac",
"*with_backing*.flac", "*Backing*.flac",
"*instrumental*.flac", "*Instrumental*.flac",
"*instrumental*.wav"
)
else:
# Default to clean instrumental
instrumental = find_file(
work_dir,
"*instrumental_clean*.flac", "*Instrumental Clean*.flac",
"*instrumental*.flac", "*Instrumental*.flac",
"*instrumental*.wav"
)
logger.info(f"Found files:")
logger.info(f" Title video: {title_video}")
logger.info(f" Karaoke video: {karaoke_video}")
logger.info(f" End video: {end_video}")
logger.info(f" Instrumental ({instrumental_selection}): {instrumental}")
# Validate required files
if not title_video:
raise ValueError(f"No title video found in {work_dir}. Check screens/ subdirectory.")
if not karaoke_video:
raise ValueError(f"No karaoke video found in {work_dir}")
if not instrumental:
raise ValueError(f"No instrumental audio found in {work_dir}")
output_dir = work_dir / "outputs"
output_dir.mkdir(exist_ok=True)
jobs[job_id]["progress"] = 20
# Build encoding config with proper file names
encoding_config = EncodingConfig(
title_video=str(title_video),
karaoke_video=str(karaoke_video),
instrumental_audio=str(instrumental),
end_video=str(end_video) if end_video else None,
output_karaoke_mp4=str(output_dir / f"{base_name} (Karaoke).mp4"),
output_with_vocals_mp4=str(output_dir / f"{base_name} (With Vocals).mp4"),
output_lossless_4k_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossless 4k).mp4"),
output_lossy_4k_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossy 4k).mp4"),
output_lossless_mkv=str(output_dir / f"{base_name} (Final Karaoke Lossless 4k).mkv"),
output_720p_mp4=str(output_dir / f"{base_name} (Final Karaoke Lossy 720p).mp4"),
)
# Create service and run encoding
service = LocalEncodingService(logger=logger)
jobs[job_id]["progress"] = 30
logger.info("Starting LocalEncodingService.encode_all_formats()")
result = service.encode_all_formats(encoding_config)
if not result.success:
raise RuntimeError(f"Encoding failed: {result.error}")
jobs[job_id]["progress"] = 90
# Collect output files
output_files = [str(f) for f in output_dir.glob("*") if f.is_file()]
jobs[job_id]["output_files"] = output_files
logger.info(f"Encoding complete. Output files: {output_files}")
return output_dir
except ImportError as e:
# No fallback - wheel must be installed
error_msg = (
f"LocalEncodingService not available: {e}. "
"The karaoke-gen wheel must be installed. "
"Check that ensure_latest_wheel() succeeded and wheel exists in GCS."
)
logger.error(error_msg)
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = error_msg
raise RuntimeError(error_msg) from e
except Exception as e:
logger.error(f"Encoding failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
raise
async def process_job(job_id: str, request: EncodeRequest):
# Process an encoding job asynchronously
try:
# Download and install latest wheel at job start (allows hot updates without restart)
# This means in-progress jobs continue with their version, new jobs get latest code
ensure_latest_wheel()
with tempfile.TemporaryDirectory() as temp_dir:
work_dir = Path(temp_dir) / "work"
work_dir.mkdir()
# Download input files
jobs[job_id]["progress"] = 5
logger.info(f"Downloading from {request.input_gcs_path}")
download_from_gcs(request.input_gcs_path, work_dir)
# Run encoding in thread pool (CPU-bound)
loop = asyncio.get_event_loop()
output_dir = await loop.run_in_executor(
executor,
run_encoding,
job_id,
work_dir,
request.encoding_config
)
# Upload outputs
jobs[job_id]["progress"] = 95
logger.info(f"Uploading to {request.output_gcs_path}")
upload_to_gcs(output_dir, request.output_gcs_path)
# Convert local paths to blob paths (backend expects blob paths, not full gs:// URIs)
# output_gcs_path is like "gs://bucket/jobs/id/encoded/"
# We need paths like "jobs/id/encoded/Artist - Title (Final Karaoke Lossless 4k).mp4"
gcs_path = request.output_gcs_path.replace("gs://", "")
parts = gcs_path.split("/", 1)
prefix = parts[1].rstrip("/") if len(parts) > 1 else ""
local_output_files = jobs[job_id].get("output_files", [])
blob_paths = []
for local_path in local_output_files:
filename = Path(local_path).name
blob_path = f"{prefix}/{filename}" if prefix else filename
blob_paths.append(blob_path)
jobs[job_id]["output_files"] = blob_paths
logger.info(f"Output files (blob paths): {blob_paths}")
jobs[job_id]["status"] = "complete"
jobs[job_id]["progress"] = 100
logger.info(f"Job {job_id} complete")
except Exception as e:
logger.error(f"Job {job_id} failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
async def process_preview_job(job_id: str, request: EncodePreviewRequest):
# Process a preview encoding job asynchronously
try:
with tempfile.TemporaryDirectory() as temp_dir:
work_dir = Path(temp_dir) / "work"
work_dir.mkdir()
# Run preview encoding in thread pool (CPU-bound)
# Note: run_preview_encoding handles download/upload internally
loop = asyncio.get_event_loop()
await loop.run_in_executor(
executor,
run_preview_encoding,
job_id,
work_dir,
request
)
jobs[job_id]["status"] = "complete"
jobs[job_id]["progress"] = 100
logger.info(f"Preview job {job_id} complete")
except Exception as e:
logger.error(f"Preview job {job_id} failed: {e}")
jobs[job_id]["status"] = "failed"
jobs[job_id]["error"] = str(e)
@app.post("/encode-preview")
async def submit_preview_encode_job(request: EncodePreviewRequest, background_tasks: BackgroundTasks, _auth: bool = Depends(verify_api_key)):
# Submit a preview encoding job
job_id = request.job_id
# If job already exists, return cached result or current status
if job_id in jobs:
existing_job = jobs[job_id]
if existing_job["status"] == "complete":
# Return cached result - preview already encoded
return {"status": "cached", "job_id": job_id, "output_path": existing_job.get("output_path")}
elif existing_job["status"] == "failed":
# Previous attempt failed, allow retry by replacing the job
pass
else:
# Job is still in progress
return {"status": "in_progress", "job_id": job_id}
jobs[job_id] = {
"job_id": job_id,
"status": "pending",
"progress": 0,
"error": None,
"output_files": None,
}
background_tasks.add_task(process_preview_job, job_id, request)
return {"status": "accepted", "job_id": job_id}
@app.post("/encode")
async def submit_encode_job(request: EncodeRequest, background_tasks: BackgroundTasks, _auth: bool = Depends(verify_api_key)):
# Submit an encoding job
job_id = request.job_id
if job_id in jobs:
raise HTTPException(status_code=409, detail=f"Job {job_id} already exists")
jobs[job_id] = {
"job_id": job_id,
"status": "pending",
"progress": 0,
"error": None,
"output_files": None,
}
background_tasks.add_task(process_job, job_id, request)
return {"status": "accepted", "job_id": job_id}
@app.get("/status/{job_id}")
async def get_job_status(job_id: str, _auth: bool = Depends(verify_api_key)) -> JobStatus:
# Get the status of an encoding job
if job_id not in jobs:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
return JobStatus(**jobs[job_id])
@app.get("/health")
async def health_check():
# Health check endpoint
active_jobs = sum(1 for j in jobs.values() if j["status"] == "running")
# Get karaoke-gen wheel version if installed
wheel_version = None
try:
from importlib.metadata import version as get_version
wheel_version = get_version("karaoke-gen")
except Exception:
pass
return {
"status": "ok",
"active_jobs": active_jobs,
"queue_length": sum(1 for j in jobs.values() if j["status"] == "pending"),
"ffmpeg_version": subprocess.run(["ffmpeg", "-version"], capture_output=True, text=True).stdout.split("\n")[0],
"wheel_version": wheel_version,
}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8080)
@@ -1,16 +0,0 @@
# GCE Encoding Worker dependencies
# These are installed in the VM's Python virtual environment
# Web framework
fastapi>=0.109.0
uvicorn>=0.27.0
# GCS client
google-cloud-storage>=2.14.0
# Async utilities (optional, for future improvements)
aiofiles>=23.2.0
aiohttp>=3.9.0
# Note: The karaoke-gen wheel is installed separately from GCS.
# It provides LocalEncodingService which does the actual encoding.
+3 -50
View File
@@ -15,7 +15,6 @@ import os
from typing import Any, Dict, Optional
from backend.config import get_settings
from karaoke_gen.utils import sanitize_filename
logger = logging.getLogger(__name__)
@@ -137,10 +136,8 @@ class GoogleDriveService:
logger.info(f"Looking for folder '{folder_name}' in parent {parent_id}")
# Search for existing folder
# Escape single quotes in folder name for Google Drive API query syntax
escaped_folder_name = folder_name.replace("'", "\\'")
query = (
f"name='{escaped_folder_name}' and '{parent_id}' in parents "
f"name='{folder_name}' and '{parent_id}' in parents "
f"and mimeType='application/vnd.google-apps.folder' and trashed=false"
)
results = self.service.files().list(q=query, fields="files(id, name)").execute()
@@ -206,10 +203,8 @@ class GoogleDriveService:
# Check for existing file with same name
if replace_existing:
# Escape single quotes in filename for Google Drive API query syntax
escaped_filename = filename.replace("'", "\\'")
query = (
f"name='{escaped_filename}' and '{parent_id}' in parents and trashed=false"
f"name='{filename}' and '{parent_id}' in parents and trashed=false"
)
results = self.service.files().list(q=query, fields="files(id)").execute()
for existing_file in results.get("files", []):
@@ -254,10 +249,7 @@ class GoogleDriveService:
Returns:
Dictionary mapping category to uploaded file ID
"""
# Sanitize base_name to handle Unicode characters (curly quotes, em dashes, etc.)
# that could cause issues with Google Drive API queries and file naming
safe_base_name = sanitize_filename(base_name) if base_name else base_name
filename_base = f"{brand_code} - {safe_base_name}"
filename_base = f"{brand_code} - {base_name}"
uploaded_files = {}
logger.info(
@@ -304,45 +296,6 @@ class GoogleDriveService:
logger.info(f"Public share upload complete: {len(uploaded_files)} files uploaded")
return uploaded_files
def delete_file(self, file_id: str) -> bool:
"""
Delete a file from Google Drive.
Args:
file_id: Google Drive file ID to delete
Returns:
True if deleted successfully, False otherwise
"""
logger.info(f"Deleting Google Drive file: {file_id}")
try:
self.service.files().delete(fileId=file_id).execute()
logger.info(f"Successfully deleted file: {file_id}")
return True
except Exception as e:
# Check if it's a 404 (already deleted)
if hasattr(e, 'resp') and e.resp.status == 404:
logger.warning(f"File not found (already deleted?): {file_id}")
return True
logger.error(f"Failed to delete Google Drive file: {e}")
return False
def delete_files(self, file_ids: list[str]) -> dict[str, bool]:
"""
Delete multiple files from Google Drive.
Args:
file_ids: List of Google Drive file IDs to delete
Returns:
Dictionary mapping file_id to success status
"""
results = {}
for file_id in file_ids:
results[file_id] = self.delete_file(file_id)
return results
# Singleton instance
_gdrive_service: Optional[GoogleDriveService] = None
-113
View File
@@ -1,113 +0,0 @@
"""
Centralized job defaults service.
This module provides consistent handling of job creation defaults across all
endpoints (file_upload, audio_search, made-for-you webhook, etc.).
Centralizing these defaults prevents divergence between code paths and ensures
all jobs receive the same default configuration.
"""
from dataclasses import dataclass
from typing import Optional, Tuple
from backend.config import get_settings
@dataclass
class EffectiveDistributionSettings:
"""Distribution settings with defaults applied from environment variables."""
dropbox_path: Optional[str]
gdrive_folder_id: Optional[str]
discord_webhook_url: Optional[str]
brand_prefix: Optional[str]
enable_youtube_upload: bool
youtube_description: Optional[str]
def get_effective_distribution_settings(
dropbox_path: Optional[str] = None,
gdrive_folder_id: Optional[str] = None,
discord_webhook_url: Optional[str] = None,
brand_prefix: Optional[str] = None,
enable_youtube_upload: Optional[bool] = None,
youtube_description: Optional[str] = None,
) -> EffectiveDistributionSettings:
"""
Get distribution settings with defaults applied from environment variables.
This ensures consistent handling of defaults across all job creation endpoints.
Each parameter, if not provided (None), falls back to the corresponding
environment variable configured in settings.
Args:
dropbox_path: Explicit Dropbox path or None for default
gdrive_folder_id: Explicit Google Drive folder ID or None for default
discord_webhook_url: Explicit Discord webhook URL or None for default
brand_prefix: Explicit brand prefix or None for default
enable_youtube_upload: Explicit flag or None for default
youtube_description: Explicit description or None for default
Returns:
EffectiveDistributionSettings with defaults applied
"""
settings = get_settings()
return EffectiveDistributionSettings(
dropbox_path=dropbox_path or settings.default_dropbox_path,
gdrive_folder_id=gdrive_folder_id or settings.default_gdrive_folder_id,
discord_webhook_url=discord_webhook_url or settings.default_discord_webhook_url,
brand_prefix=brand_prefix or settings.default_brand_prefix,
enable_youtube_upload=enable_youtube_upload if enable_youtube_upload is not None else settings.default_enable_youtube_upload,
youtube_description=youtube_description or settings.default_youtube_description,
)
def resolve_cdg_txt_defaults(
theme_id: Optional[str],
enable_cdg: Optional[bool] = None,
enable_txt: Optional[bool] = None,
) -> Tuple[bool, bool]:
"""
Resolve CDG/TXT settings based on theme and explicit settings.
The resolution logic is:
1. If explicit True/False is provided, use that value
2. Otherwise, if a theme is set, use the server defaults (settings.default_enable_cdg/txt)
3. If no theme is set, default to False (CDG/TXT require style configuration)
This ensures CDG/TXT are only enabled when:
- A theme is configured (provides necessary style params), AND
- The server defaults allow it (DEFAULT_ENABLE_CDG=true by default)
Args:
theme_id: Theme identifier (if any)
enable_cdg: Explicit CDG setting (None means use default)
enable_txt: Explicit TXT setting (None means use default)
Returns:
Tuple of (resolved_enable_cdg, resolved_enable_txt)
"""
settings = get_settings()
# Default based on whether theme is set AND server defaults
# Theme is required because CDG/TXT need style configuration
theme_is_set = theme_id is not None
default_cdg = theme_is_set and settings.default_enable_cdg
default_txt = theme_is_set and settings.default_enable_txt
# Explicit values override defaults, None uses computed default
resolved_cdg = enable_cdg if enable_cdg is not None else default_cdg
resolved_txt = enable_txt if enable_txt is not None else default_txt
return resolved_cdg, resolved_txt
# Singleton instance (optional, for convenience)
_service_instance = None
def get_job_defaults_service():
"""Get the job defaults service (module-level functions work fine, this is for consistency)."""
return {
'get_effective_distribution_settings': get_effective_distribution_settings,
'resolve_cdg_txt_defaults': resolve_cdg_txt_defaults,
}
+34 -418
View File
@@ -13,10 +13,7 @@ import uuid
from datetime import datetime
from typing import Optional, Dict, Any, List
from backend.config import settings
from backend.exceptions import RateLimitExceededError
from backend.models.job import Job, JobStatus, JobCreate, STATE_TRANSITIONS
from backend.models.worker_log import WorkerLogEntry
from backend.services.firestore_service import FirestoreService
from backend.services.storage_service import StorageService
@@ -24,16 +21,6 @@ from backend.services.storage_service import StorageService
logger = logging.getLogger(__name__)
def _mask_email(email: str) -> str:
"""Mask email for logging to protect PII. Shows first char + domain."""
if not email or "@" not in email:
return "***"
local, domain = email.split("@", 1)
if len(local) <= 1:
return f"*@{domain}"
return f"{local[0]}***@{domain}"
class JobManager:
"""Manager for job lifecycle and state."""
@@ -42,52 +29,13 @@ class JobManager:
self.firestore = FirestoreService()
self.storage = StorageService()
def create_job(self, job_create: JobCreate, is_admin: bool = False) -> Job:
def create_job(self, job_create: JobCreate) -> Job:
"""
Create a new job with initial state PENDING.
Jobs start in PENDING state and transition to DOWNLOADING
when a worker picks them up.
Args:
job_create: Job creation parameters
is_admin: Whether the requesting user is an admin (bypasses rate limits)
Raises:
ValueError: If theme_id is not provided (all jobs require a theme)
RateLimitExceededError: If user has exceeded their daily job limit
"""
# Check rate limit FIRST (before any other validation)
# This prevents wasted work if user is rate limited
if job_create.user_email:
from backend.services.rate_limit_service import get_rate_limit_service
rate_limit_service = get_rate_limit_service()
allowed, remaining, message = rate_limit_service.check_user_job_limit(
user_email=job_create.user_email,
is_admin=is_admin
)
if not allowed:
from backend.services.rate_limit_service import _seconds_until_midnight_utc
# Get actual current count - remaining is clamped to 0 which loses info
current_count = rate_limit_service.get_user_job_count_today(job_create.user_email)
raise RateLimitExceededError(
message=message,
limit_type="jobs_per_day",
remaining_seconds=_seconds_until_midnight_utc(),
current_count=current_count,
limit_value=settings.rate_limit_jobs_per_day
)
# Enforce theme requirement - all jobs must have a theme
# This prevents unstyled videos from ever being generated
if not job_create.theme_id:
raise ValueError(
"theme_id is required for all jobs. "
"Use get_theme_service().get_default_theme_id() to get the default theme."
)
job_id = str(uuid.uuid4())[:8]
now = datetime.utcnow()
@@ -105,46 +53,19 @@ class JobManager:
enable_txt=job_create.enable_txt,
enable_youtube_upload=job_create.enable_youtube_upload,
youtube_description=job_create.youtube_description,
youtube_description_template=job_create.youtube_description_template, # video_worker reads this
webhook_url=job_create.webhook_url,
user_email=job_create.user_email,
# Distribution settings
brand_prefix=job_create.brand_prefix,
discord_webhook_url=job_create.discord_webhook_url,
dropbox_path=job_create.dropbox_path,
gdrive_folder_id=job_create.gdrive_folder_id,
# Theme configuration
theme_id=job_create.theme_id,
color_overrides=job_create.color_overrides,
style_params_gcs_path=job_create.style_params_gcs_path,
style_assets=job_create.style_assets,
# Two-phase workflow (Batch 6)
prep_only=job_create.prep_only,
finalise_only=job_create.finalise_only,
keep_brand_code=job_create.keep_brand_code,
# Request metadata (for tracking and filtering)
request_metadata=job_create.request_metadata,
# Tenant scoping
tenant_id=job_create.tenant_id,
# Made-for-you order fields
made_for_you=job_create.made_for_you,
customer_email=job_create.customer_email,
customer_notes=job_create.customer_notes,
)
self.firestore.create_job(job)
logger.info(f"Created new job {job_id} with status PENDING")
# Record job creation for rate limiting (after successful persistence)
if job_create.user_email:
try:
from backend.services.rate_limit_service import get_rate_limit_service
rate_limit_service = get_rate_limit_service()
rate_limit_service.record_job_creation(job_create.user_email, job_id)
except Exception as e:
# Don't fail job creation if rate limit recording fails
logger.warning(f"Failed to record job creation for rate limiting: {e}")
return job
def get_job(self, job_id: str) -> Optional[Job]:
@@ -179,23 +100,19 @@ class JobManager:
client_id: Optional[str] = None,
created_after: Optional[datetime] = None,
created_before: Optional[datetime] = None,
user_email: Optional[str] = None,
tenant_id: Optional[str] = None,
limit: int = 100
) -> List[Job]:
"""
List jobs with optional filtering.
Args:
status: Filter by job status
environment: Filter by request_metadata.environment (test/production/development)
client_id: Filter by request_metadata.client_id (customer identifier)
created_after: Filter jobs created after this datetime
created_before: Filter jobs created before this datetime
user_email: Filter by user_email (job owner)
tenant_id: Filter by tenant_id (white-label portal scoping)
limit: Maximum number of jobs to return
Returns:
List of Job objects matching filters
"""
@@ -205,8 +122,6 @@ class JobManager:
client_id=client_id,
created_after=created_after,
created_before=created_before,
user_email=user_email,
tenant_id=tenant_id,
limit=limit
)
@@ -295,7 +210,7 @@ class JobManager:
return urls
def delete_job(self, job_id: str, delete_files: bool = True) -> None:
"""Delete a job, its files, and its logs subcollection."""
"""Delete a job and optionally its files."""
if delete_files:
job = self.get_job(job_id)
if job and job.output_files:
@@ -304,15 +219,7 @@ class JobManager:
self.storage.delete_file(gcs_path)
except Exception as e:
logger.error(f"Error deleting file {gcs_path}: {e}")
# Delete logs subcollection first (must be done before deleting parent doc)
try:
deleted_logs = self.firestore.delete_logs_subcollection(job_id)
if deleted_logs > 0:
logger.info(f"Deleted {deleted_logs} log entries for job {job_id}")
except Exception as e:
logger.warning(f"Error deleting logs subcollection for job {job_id}: {e}")
self.firestore.delete_job(job_id)
logger.info(f"Deleted job {job_id}")
@@ -373,24 +280,6 @@ class JobManager:
if progress is not None:
updates['progress'] = progress
# Generate review token when entering AWAITING_REVIEW state
if new_status == JobStatus.AWAITING_REVIEW:
from backend.api.dependencies import generate_review_token, get_review_token_expiry
review_token = generate_review_token()
review_token_expires = get_review_token_expiry(hours=48) # 48 hour expiry
updates['review_token'] = review_token
updates['review_token_expires_at'] = review_token_expires
logger.info(f"Generated review token for job {job_id}, expires in 48 hours")
# Generate instrumental token when entering AWAITING_INSTRUMENTAL_SELECTION state
if new_status == JobStatus.AWAITING_INSTRUMENTAL_SELECTION:
from backend.api.dependencies import generate_review_token, get_review_token_expiry
instrumental_token = generate_review_token() # Reuse same token generator
instrumental_token_expires = get_review_token_expiry(hours=48) # 48 hour expiry
updates['instrumental_token'] = instrumental_token
updates['instrumental_token_expires_at'] = instrumental_token_expires
logger.info(f"Generated instrumental token for job {job_id}, expires in 48 hours")
# If we have state_data_updates, merge them with existing state_data
merged_state_data = None
if state_data_updates:
@@ -415,230 +304,9 @@ class JobManager:
message=message
)
# Apply review token update separately if generated
if new_status == JobStatus.AWAITING_REVIEW and 'review_token' in updates:
self.firestore.update_job(job_id, {
'review_token': updates['review_token'],
'review_token_expires_at': updates['review_token_expires_at']
})
# Apply instrumental token update separately if generated
if new_status == JobStatus.AWAITING_INSTRUMENTAL_SELECTION and 'instrumental_token' in updates:
self.firestore.update_job(job_id, {
'instrumental_token': updates['instrumental_token'],
'instrumental_token_expires_at': updates['instrumental_token_expires_at']
})
logger.info(f"Job {job_id} transitioned to {new_status}")
# Trigger notifications asynchronously (fire-and-forget)
self._trigger_state_notifications(job_id, new_status)
return True
def _trigger_state_notifications(self, job_id: str, new_status: JobStatus) -> None:
"""
Trigger email and push notifications based on state transitions.
This is fire-and-forget - notification failures don't affect job processing.
Args:
job_id: Job ID
new_status: New job status
"""
import asyncio
try:
# Get the job to access user info
job = self.get_job(job_id)
if not job or not job.user_email:
logger.debug(f"No user email for job {job_id}, skipping notifications")
return
# Job completion notification
if new_status == JobStatus.COMPLETE:
self._schedule_completion_email(job)
self._send_push_notification(job, "complete")
# Idle reminder scheduling for blocking states
elif new_status in [JobStatus.AWAITING_REVIEW, JobStatus.AWAITING_INSTRUMENTAL_SELECTION]:
self._schedule_idle_reminder(job, new_status)
# Send push notification for blocking states
action_type = "lyrics" if new_status == JobStatus.AWAITING_REVIEW else "instrumental"
self._send_push_notification(job, action_type)
except Exception as e:
# Never let notification failures affect job processing
logger.error(f"Error triggering notifications for job {job_id}: {e}")
def _schedule_completion_email(self, job: Job) -> None:
"""
Schedule sending a job completion email.
For made-for-you orders, also transfers ownership from admin to customer.
Uses asyncio to fire-and-forget the email sending.
"""
import asyncio
import threading
try:
from backend.services.job_notification_service import get_job_notification_service
notification_service = get_job_notification_service()
# Get youtube, dropbox URLs, and brand_code from state_data (may be None)
state_data = job.state_data or {}
youtube_url = state_data.get('youtube_url')
dropbox_url = state_data.get('dropbox_link')
brand_code = state_data.get('brand_code')
# For made-for-you orders, send to customer and transfer ownership
recipient_email = job.user_email
if job.made_for_you and job.customer_email:
recipient_email = job.customer_email
# Transfer ownership from admin to customer (non-blocking - email still goes out if this fails)
try:
self.update_job(job.job_id, {'user_email': job.customer_email})
logger.info(f"Transferred ownership of made-for-you job {job.job_id} to {_mask_email(job.customer_email)}")
except Exception as e:
logger.error(f"Failed to transfer ownership for job {job.job_id}: {e}")
# Create async task (fire-and-forget)
async def send_email():
await notification_service.send_job_completion_email(
job_id=job.job_id,
user_email=recipient_email,
user_name=None, # Could fetch from user service if needed
artist=job.artist,
title=job.title,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
brand_code=brand_code,
)
# Try to get existing event loop, create new one if none exists
try:
loop = asyncio.get_running_loop()
# If we're in an async context, create a task
loop.create_task(send_email())
except RuntimeError:
# No event loop - we're likely in a sync context
# Use daemon thread to avoid blocking job completion
def run_in_thread():
asyncio.run(send_email())
thread = threading.Thread(target=run_in_thread, daemon=True)
thread.start()
logger.info(f"Scheduled completion email for job {job.job_id}")
except Exception as e:
logger.error(f"Failed to schedule completion email for job {job.job_id}: {e}")
def _schedule_idle_reminder(self, job: Job, new_status: JobStatus) -> None:
"""
Schedule an idle reminder for a blocking state.
Records the timestamp when the blocking state was entered and
schedules a Cloud Tasks task for 5 minutes later.
"""
import asyncio
import threading
try:
# Record when we entered blocking state (for idle detection)
blocking_entered_at = datetime.utcnow().isoformat()
action_type = "lyrics" if new_status == JobStatus.AWAITING_REVIEW else "instrumental"
# Update state_data with blocking state info (handle None state_data)
existing_state_data = job.state_data or {}
state_data_update = {
'blocking_state_entered_at': blocking_entered_at,
'blocking_action_type': action_type,
'reminder_sent': False, # Will be set to True after reminder is sent
}
self.firestore.update_job(job.job_id, {
'state_data': {**existing_state_data, **state_data_update}
})
# Schedule the idle reminder check via worker service (5 min delay)
from backend.services.worker_service import get_worker_service
async def schedule_reminder():
worker_service = get_worker_service()
await worker_service.schedule_idle_reminder(job.job_id)
# Try to get existing event loop, create new one if none exists
try:
loop = asyncio.get_running_loop()
loop.create_task(schedule_reminder())
except RuntimeError:
# No event loop - we're in a sync context
# Use daemon thread to avoid blocking job processing
def run_in_thread():
asyncio.run(schedule_reminder())
thread = threading.Thread(target=run_in_thread, daemon=True)
thread.start()
logger.info(f"Scheduled idle reminder for job {job.job_id} ({action_type})")
except Exception as e:
logger.error(f"Failed to schedule idle reminder for job {job.job_id}: {e}")
def _send_push_notification(self, job: Job, action_type: str) -> None:
"""
Send a push notification for job state changes.
Fire-and-forget - failures don't affect job processing.
Args:
job: Job object
action_type: Type of notification ("lyrics", "instrumental", or "complete")
"""
import asyncio
import threading
try:
from backend.services.push_notification_service import get_push_notification_service
push_service = get_push_notification_service()
# Skip if push notifications not enabled
if not push_service.is_enabled():
logger.debug("Push notifications not enabled, skipping")
return
# Build job dict for notification service
job_dict = {
"job_id": job.job_id,
"user_email": job.user_email,
"artist": job.artist,
"title": job.title,
}
async def send_notification():
if action_type == "complete":
await push_service.send_completion_notification(job_dict)
else:
await push_service.send_blocking_notification(job_dict, action_type)
# Try to get existing event loop, create new one if none exists
try:
loop = asyncio.get_running_loop()
loop.create_task(send_notification())
except RuntimeError:
# No event loop - we're in a sync context
def run_in_thread():
asyncio.run(send_notification())
thread = threading.Thread(target=run_in_thread, daemon=True)
thread.start()
logger.debug(f"Scheduled push notification for job {job.job_id} ({action_type})")
except Exception as e:
logger.error(f"Failed to send push notification for job {job.job_id}: {e}")
def update_state_data(self, job_id: str, key: str, value: Any) -> None:
"""
Update a specific key in the job's state_data field.
@@ -865,45 +533,33 @@ class JobManager:
worker: str,
level: str,
message: str,
max_logs: int = 500,
metadata: Optional[Dict[str, Any]] = None
max_logs: int = 500
) -> None:
"""
Append a log entry to the job's logs.
By default (USE_LOG_SUBCOLLECTION=true), logs are stored in a Firestore
subcollection (jobs/{job_id}/logs) to avoid the 1MB document size limit.
For backward compatibility (USE_LOG_SUBCOLLECTION=false), logs are stored
in the embedded worker_logs array using atomic ArrayUnion.
Append a log entry to the job's worker_logs atomically.
Uses Firestore ArrayUnion for atomic append to avoid race conditions
when multiple workers (audio + lyrics) are logging concurrently.
Note: max_logs is not enforced per-append (would require transactions).
Logs may grow beyond max_logs; trim periodically if needed.
Args:
job_id: Job ID
worker: Worker name (audio, lyrics, screens, video, render, distribution)
worker: Worker name (audio, lyrics, screens, video, render)
level: Log level (DEBUG, INFO, WARNING, ERROR)
message: Log message
max_logs: Not used (kept for API compatibility)
metadata: Optional additional metadata dict
"""
if settings.use_log_subcollection:
# New subcollection approach - avoids 1MB limit
log_entry = WorkerLogEntry.create(
job_id=job_id,
worker=worker,
level=level,
message=message,
metadata=metadata
)
self.firestore.append_log_to_subcollection(job_id, log_entry)
else:
# Legacy embedded array approach
log_entry = {
'timestamp': datetime.utcnow().isoformat() + 'Z',
'level': level,
'worker': worker,
'message': message[:1000] # Truncate long messages
}
self.firestore.append_worker_log(job_id, log_entry)
log_entry = {
'timestamp': datetime.utcnow().isoformat() + 'Z',
'level': level,
'worker': worker,
'message': message[:1000] # Truncate long messages
}
# Use atomic ArrayUnion to avoid race conditions
self.firestore.append_worker_log(job_id, log_entry)
def get_worker_logs(
self,
@@ -913,68 +569,28 @@ class JobManager:
) -> List[Dict[str, Any]]:
"""
Get worker logs for a job, optionally filtered by worker and index.
By default (USE_LOG_SUBCOLLECTION=true), logs are read from the
subcollection. Falls back to embedded array for older jobs.
Args:
job_id: Job ID
since_index: Return only logs after this index (for pagination)
worker: Filter by worker name (optional)
Returns:
List of log entries as dicts (in legacy format for API compatibility)
List of log entries as dicts
"""
if settings.use_log_subcollection:
# Try subcollection first
subcollection_logs = self.firestore.get_logs_from_subcollection(
job_id=job_id,
offset=since_index,
worker=worker,
limit=500
)
if subcollection_logs:
# Convert to legacy format for API compatibility
return [log.to_legacy_dict() for log in subcollection_logs]
# Fall through to check embedded array for older jobs
# Embedded array approach (legacy jobs or fallback)
job = self.get_job(job_id)
if not job or not job.worker_logs:
return []
logs = [log.dict() if hasattr(log, 'dict') else log for log in job.worker_logs]
# Filter by index
if since_index > 0:
logs = logs[since_index:]
# Filter by worker
if worker:
logs = [log for log in logs if log.get('worker') == worker]
return logs
def get_worker_logs_count(self, job_id: str) -> int:
"""
Get the total count of worker logs for a job.
Args:
job_id: Job ID
Returns:
Total count of logs
"""
if settings.use_log_subcollection:
# Try subcollection first
count = self.firestore.get_logs_count_from_subcollection(job_id)
if count > 0:
return count
# Fall through to check embedded array
# Embedded array (legacy jobs or fallback)
job = self.get_job(job_id)
if not job or not job.worker_logs:
return 0
return len(job.worker_logs)
@@ -1,271 +0,0 @@
"""
Job notification service for sending emails on job state changes.
Handles:
- Job completion emails (when job enters COMPLETE status)
- Action reminder emails (when user is idle at blocking states)
This service orchestrates between the template service and email service,
building the complete notification flow for jobs.
"""
import logging
import os
from typing import Optional, Dict, Any
from backend.services.email_service import get_email_service
from backend.services.template_service import get_template_service
logger = logging.getLogger(__name__)
# Environment variable to enable/disable auto emails
ENABLE_AUTO_EMAILS = os.getenv("ENABLE_AUTO_EMAILS", "true").lower() == "true"
# Feedback form URL (configured per environment, empty by default to avoid placeholder in emails)
FEEDBACK_FORM_URL = os.getenv("FEEDBACK_FORM_URL", "")
def _mask_email(email: str) -> str:
"""Mask email for logging to protect PII. Shows first char + domain."""
if not email or "@" not in email:
return "***"
local, domain = email.split("@", 1)
if len(local) <= 1:
return f"*@{domain}"
return f"{local[0]}***@{domain}"
class JobNotificationService:
"""
Service for sending job-related email notifications.
Coordinates between template service (for message rendering) and
email service (for sending).
"""
def __init__(self):
"""Initialize notification service."""
self.email_service = get_email_service()
self.template_service = get_template_service()
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
self.backend_url = os.getenv("BACKEND_URL", "https://api.nomadkaraoke.com")
def _build_review_url(self, job_id: str, audio_hash: Optional[str] = None, review_token: Optional[str] = None) -> str:
"""Build the lyrics review URL for a job."""
import urllib.parse
review_ui_url = os.getenv("REVIEW_UI_URL", f"{self.frontend_url}/lyrics/")
base_api_url = f"{self.backend_url}/api/review/{job_id}"
encoded_api_url = urllib.parse.quote(base_api_url, safe='')
url = f"{review_ui_url}?baseApiUrl={encoded_api_url}"
if audio_hash:
url += f"&audioHash={urllib.parse.quote(audio_hash, safe='')}"
if review_token:
url += f"&reviewToken={urllib.parse.quote(review_token, safe='')}"
return url
def _build_instrumental_url(self, job_id: str, instrumental_token: Optional[str] = None) -> str:
"""Build the instrumental selection URL for a job."""
base_api_url = f"{self.backend_url}/api/jobs/{job_id}"
import urllib.parse
encoded_api_url = urllib.parse.quote(base_api_url, safe='')
url = f"{self.frontend_url}/instrumental/?baseApiUrl={encoded_api_url}"
if instrumental_token:
url += f"&instrumentalToken={urllib.parse.quote(instrumental_token, safe='')}"
return url
async def send_job_completion_email(
self,
job_id: str,
user_email: str,
user_name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
youtube_url: Optional[str] = None,
dropbox_url: Optional[str] = None,
brand_code: Optional[str] = None,
) -> bool:
"""
Send job completion email to user.
Args:
job_id: Job ID
user_email: User's email address
user_name: User's display name (optional)
artist: Artist name
title: Song title
youtube_url: YouTube video URL
dropbox_url: Dropbox folder URL
brand_code: Release ID (e.g., "NOMAD-1178")
Returns:
True if email was sent successfully
"""
if not ENABLE_AUTO_EMAILS:
logger.info(f"Auto emails disabled, skipping completion email for job {job_id}")
return False
if not user_email:
logger.warning(f"No user email for job {job_id}, skipping completion email")
return False
try:
# Render the completion message using template service
message_content = self.template_service.render_job_completion(
name=user_name,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
artist=artist,
title=title,
job_id=job_id,
feedback_url=FEEDBACK_FORM_URL,
)
# Send the email with CC to admin
success = self.email_service.send_job_completion(
to_email=user_email,
message_content=message_content,
artist=artist,
title=title,
brand_code=brand_code,
cc_admin=True,
)
if success:
logger.info(f"Sent completion email for job {job_id} to {_mask_email(user_email)}")
else:
logger.error(f"Failed to send completion email for job {job_id}")
return success
except Exception as e:
logger.exception(f"Error sending completion email for job {job_id}: {e}")
return False
async def send_action_reminder_email(
self,
job_id: str,
user_email: str,
action_type: str,
user_name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
audio_hash: Optional[str] = None,
review_token: Optional[str] = None,
instrumental_token: Optional[str] = None,
) -> bool:
"""
Send action-needed reminder email to user.
Args:
job_id: Job ID
user_email: User's email address
action_type: Type of action needed ("lyrics" or "instrumental")
user_name: User's display name
artist: Artist name
title: Song title
audio_hash: Audio hash for review URL
review_token: Review token for URL
instrumental_token: Instrumental token for URL
Returns:
True if email was sent successfully
"""
if not ENABLE_AUTO_EMAILS:
logger.info(f"Auto emails disabled, skipping reminder for job {job_id}")
return False
if not user_email:
logger.warning(f"No user email for job {job_id}, skipping reminder")
return False
try:
# Render the appropriate template
if action_type == "lyrics":
review_url = self._build_review_url(job_id, audio_hash, review_token)
message_content = self.template_service.render_action_needed_lyrics(
name=user_name,
artist=artist,
title=title,
review_url=review_url,
)
elif action_type == "instrumental":
instrumental_url = self._build_instrumental_url(job_id, instrumental_token)
message_content = self.template_service.render_action_needed_instrumental(
name=user_name,
artist=artist,
title=title,
instrumental_url=instrumental_url,
)
else:
logger.error(f"Unknown action type: {action_type}")
return False
# Send the reminder email (no CC)
success = self.email_service.send_action_reminder(
to_email=user_email,
message_content=message_content,
action_type=action_type,
artist=artist,
title=title,
)
if success:
logger.info(f"Sent {action_type} reminder for job {job_id} to {_mask_email(user_email)}")
else:
logger.error(f"Failed to send {action_type} reminder for job {job_id}")
return success
except Exception as e:
logger.exception(f"Error sending {action_type} reminder for job {job_id}: {e}")
return False
def get_completion_message(
self,
job_id: str,
user_name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
youtube_url: Optional[str] = None,
dropbox_url: Optional[str] = None,
) -> str:
"""
Get the rendered completion message for a job (for admin copy functionality).
Args:
job_id: Job ID
user_name: User's display name
artist: Artist name
title: Song title
youtube_url: YouTube video URL
dropbox_url: Dropbox folder URL
Returns:
Rendered message content as plain text
"""
return self.template_service.render_job_completion(
name=user_name,
youtube_url=youtube_url,
dropbox_url=dropbox_url,
artist=artist,
title=title,
job_id=job_id,
feedback_url=FEEDBACK_FORM_URL,
)
# Global instance
_job_notification_service: Optional[JobNotificationService] = None
def get_job_notification_service() -> JobNotificationService:
"""Get the global job notification service instance."""
global _job_notification_service
if _job_notification_service is None:
_job_notification_service = JobNotificationService()
return _job_notification_service
-98
View File
@@ -1,98 +0,0 @@
"""Langfuse callback handler preloader for container startup.
Initializes the Langfuse callback handler at container startup to avoid
slow initialization during request processing. The CallbackHandler()
constructor makes blocking network calls to the Langfuse API, which can
take 3+ minutes on Cloud Run cold starts.
See docs/archive/2026-01-08-performance-investigation.md for background.
"""
import logging
import os
import time
from typing import Optional, Any
logger = logging.getLogger(__name__)
# Singleton storage for preloaded handler
_preloaded_handler: Optional[Any] = None
_initialization_attempted: bool = False
def preload_langfuse_handler() -> Optional[Any]:
"""Preload Langfuse callback handler at startup.
Only initializes if LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY
environment variables are set.
Returns:
The preloaded CallbackHandler, or None if not configured
"""
global _preloaded_handler, _initialization_attempted
if _initialization_attempted:
logger.debug("Langfuse initialization already attempted")
return _preloaded_handler
_initialization_attempted = True
# Check if Langfuse is configured
public_key = os.getenv("LANGFUSE_PUBLIC_KEY")
secret_key = os.getenv("LANGFUSE_SECRET_KEY")
if not (public_key and secret_key):
logger.info("Langfuse not configured (missing keys), skipping preload")
return None
logger.info("Preloading Langfuse callback handler...")
start_time = time.time()
try:
from langfuse.langchain import CallbackHandler
# Initialize the handler - this is the slow part that makes network calls
_preloaded_handler = CallbackHandler()
elapsed = time.time() - start_time
host = os.getenv("LANGFUSE_HOST", "cloud.langfuse.com")
logger.info(f"Langfuse handler preloaded in {elapsed:.2f}s (host: {host})")
return _preloaded_handler
except ImportError:
logger.warning("langfuse package not installed, skipping preload")
return None
except Exception as e:
elapsed = time.time() - start_time
logger.error(f"Failed to preload Langfuse handler after {elapsed:.2f}s: {e}")
# Don't raise - Langfuse is optional
return None
def get_preloaded_langfuse_handler() -> Optional[Any]:
"""Get the preloaded Langfuse callback handler if available.
Returns:
The preloaded CallbackHandler, or None if not preloaded/configured
"""
return _preloaded_handler
def is_langfuse_preloaded() -> bool:
"""Check if Langfuse handler has been preloaded."""
return _preloaded_handler is not None
def is_langfuse_configured() -> bool:
"""Check if Langfuse environment variables are configured."""
return bool(
os.getenv("LANGFUSE_PUBLIC_KEY") and os.getenv("LANGFUSE_SECRET_KEY")
)
def clear_preloaded_handler() -> None:
"""Clear preloaded handler. Useful for testing."""
global _preloaded_handler, _initialization_attempted
_preloaded_handler = None
_initialization_attempted = False
-590
View File
@@ -1,590 +0,0 @@
"""
Local Encoding Service.
Provides local FFmpeg-based video encoding functionality, extracted from KaraokeFinalise
for use by both the cloud backend (video_worker as a fallback) and local CLI.
This service handles:
- Concatenating title, karaoke, and end videos
- Encoding to multiple output formats (4K lossless, 4K lossy, 720p, MKV)
- Hardware acceleration detection and fallback to software encoding
- Remuxing video with instrumental audio
"""
import logging
import os
import shlex
import subprocess
import sys
from dataclasses import dataclass
from typing import Optional, List, Dict, Any, Tuple
logger = logging.getLogger(__name__)
@dataclass
class EncodingConfig:
"""Configuration for video encoding."""
title_video: str # Path to title video
karaoke_video: str # Path to karaoke video (with vocals)
instrumental_audio: str # Path to instrumental audio
end_video: Optional[str] = None # Optional path to end credits video
# Output paths
output_karaoke_mp4: Optional[str] = None
output_with_vocals_mp4: Optional[str] = None
output_lossless_4k_mp4: Optional[str] = None
output_lossy_4k_mp4: Optional[str] = None
output_lossless_mkv: Optional[str] = None
output_720p_mp4: Optional[str] = None
@dataclass
class EncodingResult:
"""Result of video encoding operation."""
success: bool
output_files: Dict[str, str]
error: Optional[str] = None
class LocalEncodingService:
"""
Service for local FFmpeg-based video encoding.
Supports hardware acceleration (NVENC) with automatic fallback
to software encoding (libx264) when unavailable.
"""
# MP4 flags for better compatibility
MP4_FLAGS = "-movflags +faststart"
def __init__(
self,
dry_run: bool = False,
log_level: int = logging.INFO,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the local encoding service.
Args:
dry_run: If True, log commands without executing them
log_level: Logging level (affects FFmpeg verbosity)
logger: Optional logger instance
"""
self.dry_run = dry_run
self.log_level = log_level
self.logger = logger or logging.getLogger(__name__)
# Hardware acceleration settings (detected on first use)
self._hwaccel_available: Optional[bool] = None
self._video_encoder: Optional[str] = None
self._scale_filter: Optional[str] = None
self._hwaccel_decode_flags: Optional[str] = None
self._aac_codec: Optional[str] = None
# Build FFmpeg base command
self._ffmpeg_base_command = self._build_ffmpeg_base_command()
def _build_ffmpeg_base_command(self) -> str:
"""Build the FFmpeg base command with appropriate flags."""
# Use bundled FFmpeg for frozen builds
ffmpeg_path = os.path.join(sys._MEIPASS, "ffmpeg.exe") if getattr(sys, "frozen", False) else "ffmpeg"
base_cmd = f"{ffmpeg_path} -hide_banner -nostats -y"
if self.log_level == logging.DEBUG:
base_cmd += " -loglevel verbose"
else:
base_cmd += " -loglevel fatal"
return base_cmd
def _detect_hardware_acceleration(self) -> Tuple[bool, str, str, str, str]:
"""
Detect available hardware acceleration.
Returns:
Tuple of (hwaccel_available, video_encoder, scale_filter,
hwaccel_decode_flags, aac_codec)
"""
self.logger.info("Detecting hardware acceleration capabilities...")
# Try NVENC (NVIDIA)
try:
test_cmd = f"{self._ffmpeg_base_command} -hide_banner -loglevel error " \
f"-f lavfi -i testsrc=duration=1:size=320x240:rate=1 " \
f"-c:v h264_nvenc -f null -"
subprocess.run(
test_cmd, shell=True, check=True,
capture_output=True, timeout=30
)
self.logger.info("NVIDIA NVENC hardware acceleration available")
return (
True,
"h264_nvenc",
"scale_cuda",
"-hwaccel cuda -hwaccel_output_format cuda",
"aac"
)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired, FileNotFoundError):
pass
# No hardware acceleration available
self.logger.info("No hardware acceleration available, using software encoding")
return (False, "libx264", "scale", "", "aac")
@property
def hwaccel_available(self) -> bool:
"""Check if hardware acceleration is available."""
if self._hwaccel_available is None:
self._detect_and_set_hwaccel()
return self._hwaccel_available
@property
def video_encoder(self) -> str:
"""Get the video encoder to use."""
if self._video_encoder is None:
self._detect_and_set_hwaccel()
return self._video_encoder
@property
def scale_filter(self) -> str:
"""Get the scale filter to use."""
if self._scale_filter is None:
self._detect_and_set_hwaccel()
return self._scale_filter
@property
def hwaccel_decode_flags(self) -> str:
"""Get hardware acceleration decode flags."""
if self._hwaccel_decode_flags is None:
self._detect_and_set_hwaccel()
return self._hwaccel_decode_flags
@property
def aac_codec(self) -> str:
"""Get the AAC codec to use."""
if self._aac_codec is None:
self._detect_and_set_hwaccel()
return self._aac_codec
def _detect_and_set_hwaccel(self) -> None:
"""Detect and set hardware acceleration settings."""
(
self._hwaccel_available,
self._video_encoder,
self._scale_filter,
self._hwaccel_decode_flags,
self._aac_codec
) = self._detect_hardware_acceleration()
def _get_nvenc_quality_settings(self, preset: str = "medium") -> str:
"""Get NVENC quality settings for different presets."""
if not self.hwaccel_available:
return ""
settings = {
"lossless": "-preset p7 -tune hq -rc vbr -cq 0 -qmin 0 -qmax 0",
"high": "-preset p7 -tune hq -rc vbr -cq 19 -b:v 0",
"medium": "-preset p4 -tune hq -rc vbr -cq 23 -b:v 0",
"fast": "-preset p2 -tune ll -rc vbr -cq 28 -b:v 0",
}
return settings.get(preset, settings["medium"])
def _execute_command(
self,
command: str,
description: str,
timeout: int = 3600, # 1 hour default
) -> bool:
"""
Execute an FFmpeg command.
Args:
command: The FFmpeg command to execute
description: Human-readable description of the operation
timeout: Command timeout in seconds
Returns:
True if successful, False otherwise
"""
self.logger.info(f"Executing: {description}")
self.logger.debug(f"Command: {command}")
if self.dry_run:
self.logger.info(f"DRY RUN: Would execute: {command}")
return True
try:
result = subprocess.run(
command,
shell=True,
check=True,
capture_output=True,
text=True,
timeout=timeout
)
self.logger.info(f"Completed: {description}")
return True
except subprocess.CalledProcessError as e:
self.logger.error(f"Failed: {description}")
self.logger.error(f"Error: {e.stderr}")
return False
except subprocess.TimeoutExpired:
self.logger.error(f"Timeout: {description}")
return False
def _execute_command_with_fallback(
self,
gpu_command: str,
cpu_command: str,
description: str,
) -> bool:
"""
Execute a command with GPU, falling back to CPU if it fails.
Args:
gpu_command: Hardware-accelerated command
cpu_command: Software fallback command
description: Human-readable description
Returns:
True if successful, False otherwise
"""
if self.hwaccel_available:
self.logger.info(f"Trying hardware-accelerated encoding for: {description}")
if self._execute_command(gpu_command, description):
return True
self.logger.warning(f"Hardware encoding failed, falling back to software")
return self._execute_command(cpu_command, description)
def remux_with_instrumental(
self,
input_video: str,
instrumental_audio: str,
output_file: str,
) -> bool:
"""
Remux video with instrumental audio track.
Args:
input_video: Path to input video file
instrumental_audio: Path to instrumental audio file
output_file: Path for output file
Returns:
True if successful, False otherwise
"""
command = (
f'{self._ffmpeg_base_command} -i "{input_video}" '
f'-i "{instrumental_audio}" -map 0:v -map 1:a -c copy '
f'{self.MP4_FLAGS} "{output_file}"'
)
return self._execute_command(command, "Remuxing with instrumental audio")
def convert_mov_to_mp4(
self,
input_file: str,
output_file: str,
) -> bool:
"""
Convert MOV to MP4 format.
Args:
input_file: Path to input MOV file
output_file: Path for output MP4 file
Returns:
True if successful, False otherwise
"""
gpu_command = (
f'{self._ffmpeg_base_command} {self.hwaccel_decode_flags} -i "{input_file}" '
f'-c:v {self.video_encoder} -c:a copy {self.MP4_FLAGS} "{output_file}"'
)
cpu_command = (
f'{self._ffmpeg_base_command} -i "{input_file}" '
f'-c:v libx264 -c:a copy {self.MP4_FLAGS} "{output_file}"'
)
return self._execute_command_with_fallback(
gpu_command, cpu_command, "Converting MOV to MP4"
)
def encode_lossless_mp4(
self,
title_video: str,
karaoke_video: str,
output_file: str,
end_video: Optional[str] = None,
) -> bool:
"""
Create lossless 4K MP4 by concatenating title, karaoke, and optionally end videos.
Args:
title_video: Path to title video
karaoke_video: Path to karaoke video
output_file: Path for output file
end_video: Optional path to end credits video
Returns:
True if successful, False otherwise
"""
# Quote file paths
title_quoted = shlex.quote(os.path.abspath(title_video))
karaoke_quoted = shlex.quote(os.path.abspath(karaoke_video))
# Build filter and inputs for concatenation
if end_video and os.path.isfile(end_video):
end_quoted = shlex.quote(os.path.abspath(end_video))
extra_input = f"-i {end_quoted}"
concat_filter = (
'-filter_complex "[0:v:0][0:a:0][1:v:0][1:a:0][2:v:0][2:a:0]'
'concat=n=3:v=1:a=1[outv][outa]"'
)
else:
extra_input = ""
concat_filter = (
'-filter_complex "[0:v:0][0:a:0][1:v:0][1:a:0]'
'concat=n=2:v=1:a=1[outv][outa]"'
)
gpu_command = (
f"{self._ffmpeg_base_command} {self.hwaccel_decode_flags} -i {title_quoted} "
f"{self.hwaccel_decode_flags} -i {karaoke_quoted} {extra_input} "
f'{concat_filter} -map "[outv]" -map "[outa]" -c:v {self.video_encoder} '
f'{self._get_nvenc_quality_settings("lossless")} -c:a pcm_s16le '
f'{self.MP4_FLAGS} "{output_file}"'
)
cpu_command = (
f"{self._ffmpeg_base_command} -i {title_quoted} -i {karaoke_quoted} {extra_input} "
f'{concat_filter} -map "[outv]" -map "[outa]" -c:v libx264 -c:a pcm_s16le '
f'{self.MP4_FLAGS} "{output_file}"'
)
return self._execute_command_with_fallback(
gpu_command, cpu_command, "Encoding lossless 4K MP4"
)
def encode_lossy_mp4(
self,
input_file: str,
output_file: str,
) -> bool:
"""
Create lossy 4K MP4 with AAC audio.
Args:
input_file: Path to input file (typically lossless MP4)
output_file: Path for output file
Returns:
True if successful, False otherwise
"""
command = (
f'{self._ffmpeg_base_command} -i "{input_file}" '
f'-c:v copy -c:a {self.aac_codec} -ar 48000 -b:a 320k '
f'{self.MP4_FLAGS} "{output_file}"'
)
return self._execute_command(command, "Encoding lossy 4K MP4 with AAC")
def encode_lossless_mkv(
self,
input_file: str,
output_file: str,
) -> bool:
"""
Create MKV with FLAC audio (for YouTube upload).
Args:
input_file: Path to input file
output_file: Path for output file
Returns:
True if successful, False otherwise
"""
command = (
f'{self._ffmpeg_base_command} -i "{input_file}" '
f'-c:v copy -c:a flac "{output_file}"'
)
return self._execute_command(command, "Creating MKV with FLAC for YouTube")
def encode_720p(
self,
input_file: str,
output_file: str,
) -> bool:
"""
Create 720p MP4 with AAC audio.
Args:
input_file: Path to input file
output_file: Path for output file
Returns:
True if successful, False otherwise
"""
gpu_command = (
f'{self._ffmpeg_base_command} {self.hwaccel_decode_flags} -i "{input_file}" '
f'-c:v {self.video_encoder} -vf "{self.scale_filter}=1280:720" '
f'{self._get_nvenc_quality_settings("medium")} -b:v 2000k '
f'-c:a {self.aac_codec} -ar 48000 -b:a 128k '
f'{self.MP4_FLAGS} "{output_file}"'
)
cpu_command = (
f'{self._ffmpeg_base_command} -i "{input_file}" '
f'-c:v libx264 -vf "scale=1280:720" -b:v 2000k -preset medium -tune animation '
f'-c:a {self.aac_codec} -ar 48000 -b:a 128k '
f'{self.MP4_FLAGS} "{output_file}"'
)
return self._execute_command_with_fallback(
gpu_command, cpu_command, "Encoding 720p MP4"
)
def encode_all_formats(
self,
config: EncodingConfig,
) -> EncodingResult:
"""
Encode video to all output formats.
This performs the full encoding pipeline:
1. Remux with instrumental audio
2. Convert to MP4 if needed
3. Encode lossless 4K MP4 (concatenated)
4. Encode lossy 4K MP4
5. Encode lossless MKV (for YouTube)
6. Encode 720p MP4
Args:
config: Encoding configuration with input/output paths
Returns:
EncodingResult with success status and output file paths
"""
output_files = {}
try:
# Step 1: Remux with instrumental audio
if config.output_karaoke_mp4:
self.logger.info("[Step 1/6] Remuxing video with instrumental audio...")
if not self.remux_with_instrumental(
config.karaoke_video,
config.instrumental_audio,
config.output_karaoke_mp4
):
return EncodingResult(
success=False,
output_files=output_files,
error="Failed to remux with instrumental audio"
)
output_files["karaoke_mp4"] = config.output_karaoke_mp4
# Step 2: Convert to MP4 if needed
if config.output_with_vocals_mp4:
if not config.karaoke_video.endswith(".mp4"):
self.logger.info("[Step 2/6] Converting karaoke video to MP4...")
if not self.convert_mov_to_mp4(
config.karaoke_video,
config.output_with_vocals_mp4
):
return EncodingResult(
success=False,
output_files=output_files,
error="Failed to convert to MP4"
)
output_files["with_vocals_mp4"] = config.output_with_vocals_mp4
else:
self.logger.info("[Step 2/6] Skipped - video already MP4")
# Step 3: Encode lossless 4K MP4
if config.output_lossless_4k_mp4:
self.logger.info("[Step 3/6] Encoding lossless 4K MP4...")
karaoke_for_concat = config.output_karaoke_mp4 or config.karaoke_video
if not self.encode_lossless_mp4(
config.title_video,
karaoke_for_concat,
config.output_lossless_4k_mp4,
config.end_video
):
return EncodingResult(
success=False,
output_files=output_files,
error="Failed to encode lossless 4K MP4"
)
output_files["lossless_4k_mp4"] = config.output_lossless_4k_mp4
# Step 4: Encode lossy 4K MP4
if config.output_lossy_4k_mp4 and config.output_lossless_4k_mp4:
self.logger.info("[Step 4/6] Encoding lossy 4K MP4...")
if not self.encode_lossy_mp4(
config.output_lossless_4k_mp4,
config.output_lossy_4k_mp4
):
return EncodingResult(
success=False,
output_files=output_files,
error="Failed to encode lossy 4K MP4"
)
output_files["lossy_4k_mp4"] = config.output_lossy_4k_mp4
# Step 5: Create MKV with FLAC audio
if config.output_lossless_mkv and config.output_lossless_4k_mp4:
self.logger.info("[Step 5/6] Creating MKV with FLAC audio...")
if not self.encode_lossless_mkv(
config.output_lossless_4k_mp4,
config.output_lossless_mkv
):
return EncodingResult(
success=False,
output_files=output_files,
error="Failed to create MKV"
)
output_files["lossless_mkv"] = config.output_lossless_mkv
# Step 6: Encode 720p version
if config.output_720p_mp4 and config.output_lossless_4k_mp4:
self.logger.info("[Step 6/6] Encoding 720p MP4...")
if not self.encode_720p(
config.output_lossless_4k_mp4,
config.output_720p_mp4
):
return EncodingResult(
success=False,
output_files=output_files,
error="Failed to encode 720p"
)
output_files["720p_mp4"] = config.output_720p_mp4
self.logger.info("All encoding steps completed successfully")
return EncodingResult(success=True, output_files=output_files)
except Exception as e:
self.logger.error(f"Encoding failed with exception: {e}")
return EncodingResult(
success=False,
output_files=output_files,
error=str(e)
)
# Singleton instance and factory function (following existing service pattern)
_local_encoding_service: Optional[LocalEncodingService] = None
def get_local_encoding_service(**kwargs) -> LocalEncodingService:
"""
Get a local encoding service instance.
Args:
**kwargs: Arguments passed to LocalEncodingService
Returns:
LocalEncodingService instance
"""
global _local_encoding_service
if _local_encoding_service is None:
_local_encoding_service = LocalEncodingService(**kwargs)
return _local_encoding_service
@@ -1,407 +0,0 @@
"""
Local Preview Encoding Service.
Provides local FFmpeg-based preview video encoding functionality, extracted from
VideoGenerator for use by the GCE worker. This ensures the same encoding logic
is used across local CLI, Cloud Run, and GCE worker environments.
This service handles:
- Generating preview videos with ASS subtitle overlay
- Hardware acceleration detection (NVENC) with fallback to software encoding
- Background image/color support
- Custom font support
- Optimized settings for fast preview generation
"""
import logging
import os
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Optional, List
logger = logging.getLogger(__name__)
@dataclass
class PreviewEncodingConfig:
"""Configuration for preview video encoding."""
ass_path: str # Path to ASS subtitles file
audio_path: str # Path to audio file
output_path: str # Path for output video file
# Optional background settings
background_image_path: Optional[str] = None # Path to background image
background_color: str = "black" # Fallback background color
# Optional font settings
font_path: Optional[str] = None # Path to custom font file
@dataclass
class PreviewEncodingResult:
"""Result of preview encoding operation."""
success: bool
output_path: Optional[str] = None
error: Optional[str] = None
class LocalPreviewEncodingService:
"""
Service for local FFmpeg-based preview video encoding.
This is the single source of truth for preview encoding logic, used by:
- Local CLI (via VideoGenerator which delegates here)
- Cloud Run (when GCE is unavailable)
- GCE worker (via installed wheel)
Supports hardware acceleration (NVENC) with automatic fallback
to software encoding (libx264) when unavailable.
"""
# Preview video settings - these are the canonical values
PREVIEW_WIDTH = 480
PREVIEW_HEIGHT = 270
PREVIEW_FPS = 24
PREVIEW_AUDIO_BITRATE = "96k"
def __init__(
self,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the local preview encoding service.
Args:
logger: Optional logger instance
"""
self.logger = logger or logging.getLogger(__name__)
# Hardware acceleration settings (detected on first use)
self._nvenc_available: Optional[bool] = None
self._video_encoder: Optional[str] = None
self._hwaccel_flags: Optional[List[str]] = None
def _detect_nvenc_support(self) -> bool:
"""
Detect if NVENC hardware encoding is available.
Returns:
True if NVENC is available, False otherwise
"""
try:
self.logger.info("Detecting NVENC hardware acceleration...")
# Test h264_nvenc encoder directly
test_cmd = [
"ffmpeg", "-hide_banner", "-loglevel", "error",
"-f", "lavfi", "-i", "testsrc=duration=1:size=320x240:rate=1",
"-c:v", "h264_nvenc", "-f", "null", "-"
]
result = subprocess.run(
test_cmd,
capture_output=True,
text=True,
timeout=30
)
if result.returncode == 0:
self.logger.info("NVENC hardware acceleration available")
return True
# Try alternative test with different source
alt_test_cmd = [
"ffmpeg", "-hide_banner", "-loglevel", "error",
"-f", "lavfi", "-i", "color=red:size=320x240:duration=0.1",
"-c:v", "h264_nvenc", "-preset", "fast", "-f", "null", "-"
]
alt_result = subprocess.run(
alt_test_cmd,
capture_output=True,
text=True,
timeout=30
)
if alt_result.returncode == 0:
self.logger.info("NVENC hardware acceleration available")
return True
self.logger.info("NVENC not available, using software encoding")
return False
except subprocess.TimeoutExpired:
self.logger.debug("NVENC detection timed out")
return False
except Exception as e:
self.logger.debug(f"NVENC detection failed: {e}")
return False
def _configure_hardware_acceleration(self) -> None:
"""Configure hardware acceleration settings based on detected capabilities."""
if self._nvenc_available is None:
self._nvenc_available = self._detect_nvenc_support()
if self._nvenc_available:
self._video_encoder = "h264_nvenc"
self._hwaccel_flags = ["-hwaccel", "cuda", "-hwaccel_output_format", "cuda"]
self.logger.info("Using NVENC hardware acceleration for preview encoding")
else:
self._video_encoder = "libx264"
self._hwaccel_flags = []
self.logger.info("Using software encoding (libx264) for preview")
@property
def nvenc_available(self) -> bool:
"""Check if NVENC hardware acceleration is available."""
if self._nvenc_available is None:
self._configure_hardware_acceleration()
return self._nvenc_available
@property
def video_encoder(self) -> str:
"""Get the video encoder to use."""
if self._video_encoder is None:
self._configure_hardware_acceleration()
return self._video_encoder
@property
def hwaccel_flags(self) -> List[str]:
"""Get hardware acceleration flags."""
if self._hwaccel_flags is None:
self._configure_hardware_acceleration()
return self._hwaccel_flags
def _escape_ffmpeg_filter_path(self, path: str) -> str:
"""
Escape a path for FFmpeg filter expressions (for subprocess without shell).
When using subprocess with a command list (no shell), FFmpeg receives the
filter string directly. FFmpeg's filter parser requires escaping:
- Backslashes: double them (\\ -> \\\\)
- Single quotes/apostrophes: escape with three backslashes (' -> \\\\')
- Spaces: escape with backslash ( -> \\ )
- Special characters: :,[];
Example: "I'm With You" becomes "I\\\\'m\\ With\\ You"
"""
# First escape existing backslashes
escaped = path.replace("\\", "\\\\")
# Escape single quotes
escaped = escaped.replace("'", "\\\\\\'")
# Escape spaces
escaped = escaped.replace(" ", "\\ ")
# Escape FFmpeg filter special characters
escaped = escaped.replace(":", "\\:")
escaped = escaped.replace(",", "\\,")
escaped = escaped.replace("[", "\\[")
escaped = escaped.replace("]", "\\]")
escaped = escaped.replace(";", "\\;")
return escaped
def _build_ass_filter(self, ass_path: str, font_path: Optional[str] = None) -> str:
"""
Build ASS filter with optional font directory support.
Args:
ass_path: Path to ASS subtitles file
font_path: Optional path to custom font file
Returns:
FFmpeg ASS filter string
"""
escaped_ass_path = self._escape_ffmpeg_filter_path(ass_path)
ass_filter = f"ass={escaped_ass_path}"
if font_path and os.path.isfile(font_path):
font_dir = os.path.dirname(font_path)
escaped_font_dir = self._escape_ffmpeg_filter_path(font_dir)
ass_filter += f":fontsdir={escaped_font_dir}"
self.logger.debug(f"Using font directory: {font_dir}")
return ass_filter
def _build_preview_ffmpeg_command(self, config: PreviewEncodingConfig) -> List[str]:
"""
Build FFmpeg command for preview video generation.
This is the canonical preview encoding command, ensuring consistency
across all environments (local CLI, Cloud Run, GCE worker).
Args:
config: Preview encoding configuration
Returns:
FFmpeg command as a list of arguments
"""
width, height = self.PREVIEW_WIDTH, self.PREVIEW_HEIGHT
cmd = [
"ffmpeg",
"-hide_banner",
"-loglevel", "error",
"-r", str(self.PREVIEW_FPS),
]
# Add hardware acceleration flags if available
cmd.extend(self.hwaccel_flags)
# Input source (background image or solid color)
if config.background_image_path and os.path.isfile(config.background_image_path):
self.logger.debug(f"Using background image: {config.background_image_path}")
cmd.extend([
"-loop", "1",
"-i", config.background_image_path,
])
# Build video filter with scaling and ASS subtitles
video_filter = (
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2,"
f"{self._build_ass_filter(config.ass_path, config.font_path)}"
)
else:
self.logger.debug(f"Using solid {config.background_color} background")
cmd.extend([
"-f", "lavfi",
"-i", f"color=c={config.background_color}:s={width}x{height}:r={self.PREVIEW_FPS}",
])
# Just ASS subtitles, no scaling needed
video_filter = self._build_ass_filter(config.ass_path, config.font_path)
cmd.extend([
"-i", config.audio_path,
"-vf", video_filter,
"-c:a", "aac",
"-b:a", self.PREVIEW_AUDIO_BITRATE,
"-c:v", self.video_encoder,
])
# Add encoder-specific settings optimized for speed
if self.nvenc_available:
cmd.extend([
"-preset", "p1", # Fastest NVENC preset
"-tune", "ll", # Low latency
"-rc", "cbr", # Constant bitrate for speed
"-b:v", "800k", # Lower bitrate for speed
"-profile:v", "baseline", # Most compatible profile
"-level", "3.1", # Lower level for speed
])
self.logger.debug("Using NVENC with maximum speed settings")
else:
cmd.extend([
"-profile:v", "baseline",
"-level", "3.0",
"-preset", "superfast",
"-tune", "fastdecode",
"-b:v", "600k",
"-maxrate", "800k",
"-bufsize", "1200k",
"-crf", "28",
])
self.logger.debug("Using software encoding with maximum speed settings")
cmd.extend([
"-pix_fmt", "yuv420p", # Required for browser compatibility
"-movflags", "+faststart+frag_keyframe+empty_moov+dash",
"-g", "48", # Keyframe every 48 frames (2 seconds at 24fps)
"-keyint_min", "48",
"-sc_threshold", "0", # Disable scene change detection for speed
"-threads", "0", # Use all available CPU threads
"-shortest",
"-y",
config.output_path,
])
return cmd
def encode_preview(self, config: PreviewEncodingConfig) -> PreviewEncodingResult:
"""
Encode a preview video.
Args:
config: Preview encoding configuration
Returns:
PreviewEncodingResult with success status and output path
"""
self.logger.info(f"Encoding preview video: {config.output_path}")
# Validate input files
if not os.path.isfile(config.ass_path):
return PreviewEncodingResult(
success=False,
error=f"ASS subtitles file not found: {config.ass_path}"
)
if not os.path.isfile(config.audio_path):
return PreviewEncodingResult(
success=False,
error=f"Audio file not found: {config.audio_path}"
)
# Ensure output directory exists
output_dir = os.path.dirname(config.output_path)
if output_dir:
os.makedirs(output_dir, exist_ok=True)
try:
# Build and execute FFmpeg command
cmd = self._build_preview_ffmpeg_command(config)
self.logger.debug(f"FFmpeg command: {' '.join(cmd)}")
result = subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=300 # 5 minute timeout for preview encoding
)
if result.returncode != 0:
error_msg = result.stderr[-500:] if result.stderr else "Unknown error"
self.logger.error(f"FFmpeg failed: {error_msg}")
return PreviewEncodingResult(
success=False,
error=f"FFmpeg preview encoding failed: {error_msg}"
)
self.logger.info(f"Preview encoded successfully: {config.output_path}")
return PreviewEncodingResult(
success=True,
output_path=config.output_path
)
except subprocess.TimeoutExpired:
self.logger.error("Preview encoding timed out")
return PreviewEncodingResult(
success=False,
error="Preview encoding timed out after 5 minutes"
)
except Exception as e:
self.logger.error(f"Preview encoding failed: {e}")
return PreviewEncodingResult(
success=False,
error=str(e)
)
# Singleton instance and factory function
_local_preview_encoding_service: Optional[LocalPreviewEncodingService] = None
def get_local_preview_encoding_service(**kwargs) -> LocalPreviewEncodingService:
"""
Get a local preview encoding service instance.
Args:
**kwargs: Arguments passed to LocalPreviewEncodingService
Returns:
LocalPreviewEncodingService instance
"""
global _local_preview_encoding_service
if _local_preview_encoding_service is None:
_local_preview_encoding_service = LocalPreviewEncodingService(**kwargs)
return _local_preview_encoding_service
-216
View File
@@ -1,216 +0,0 @@
"""
LyricsTranscriber cache synchronization with GCS.
This service persists LyricsTranscriber's cache files to GCS so that
cloud workers (Cloud Run instances) can share cache across containers.
Cache files are stored flat in GCS under lyrics-transcriber-cache/ prefix:
- Transcription: {provider}_{audio_hash}_raw.json, {provider}_{audio_hash}_converted.json
- Lyrics: {provider}_{artist_title_hash}_raw.json, {provider}_{artist_title_hash}_converted.json
Hash computation matches LyricsTranscriber's implementation exactly:
- Audio hash: MD5 of audio file bytes
- Lyrics hash: MD5 of "{artist.lower()}_{title.lower()}"
"""
import hashlib
import logging
import os
from typing import Dict, List, Optional
from backend.services.storage_service import StorageService
logger = logging.getLogger(__name__)
# Providers that use audio file hash as cache key
TRANSCRIPTION_PROVIDERS = ["audioshake", "whisper", "localwhisper"]
# Providers that use artist+title hash as cache key
LYRICS_PROVIDERS = ["genius", "spotify", "lrclib", "musixmatch"]
# Cache file suffixes
CACHE_SUFFIXES = ["raw", "converted"]
class LyricsCacheService:
"""Service to sync LyricsTranscriber cache with GCS."""
GCS_CACHE_PREFIX = "lyrics-transcriber-cache/"
def __init__(self, storage: Optional[StorageService] = None):
"""Initialize the cache service.
Args:
storage: StorageService instance. If None, creates a new one.
"""
self.storage = storage or StorageService()
def compute_audio_hash(self, audio_path: str) -> str:
"""Compute MD5 hash of audio file bytes.
This matches LyricsTranscriber's _get_file_hash() method exactly.
Args:
audio_path: Path to the audio file.
Returns:
MD5 hex digest of the file contents.
"""
md5_hash = hashlib.md5()
with open(audio_path, "rb") as f:
for chunk in iter(lambda: f.read(4096), b""):
md5_hash.update(chunk)
return md5_hash.hexdigest()
def compute_lyrics_hash(self, artist: str, title: str) -> str:
"""Compute MD5 hash of artist and title.
This matches LyricsTranscriber's _get_artist_title_hash() method exactly.
Args:
artist: Artist name.
title: Track title.
Returns:
MD5 hex digest of "{artist.lower()}_{title.lower()}".
"""
combined = f"{artist.lower()}_{title.lower()}"
return hashlib.md5(combined.encode()).hexdigest()
def _get_cache_filenames(
self, providers: List[str], hash_value: str
) -> List[str]:
"""Generate list of possible cache filenames for given providers and hash.
Args:
providers: List of provider names (e.g., ["audioshake", "whisper"]).
hash_value: The hash to use in filenames.
Returns:
List of filenames like ["audioshake_abc123_raw.json", ...].
"""
filenames = []
for provider in providers:
for suffix in CACHE_SUFFIXES:
filenames.append(f"{provider}_{hash_value}_{suffix}.json")
return filenames
def sync_cache_from_gcs(
self,
local_cache_dir: str,
audio_hash: str,
lyrics_hash: str,
) -> Dict[str, int]:
"""Download relevant cache files from GCS to local directory.
Downloads cache files for both transcription (audio hash) and
lyrics (artist+title hash) providers.
Args:
local_cache_dir: Local directory to download cache files to.
audio_hash: MD5 hash of audio file.
lyrics_hash: MD5 hash of artist+title.
Returns:
Dict with counts: {"downloaded": N, "not_found": M, "errors": E}
"""
os.makedirs(local_cache_dir, exist_ok=True)
stats = {"downloaded": 0, "not_found": 0, "errors": 0}
# Get all possible cache filenames
transcription_files = self._get_cache_filenames(
TRANSCRIPTION_PROVIDERS, audio_hash
)
lyrics_files = self._get_cache_filenames(LYRICS_PROVIDERS, lyrics_hash)
all_files = transcription_files + lyrics_files
for filename in all_files:
gcs_path = f"{self.GCS_CACHE_PREFIX}{filename}"
local_path = os.path.join(local_cache_dir, filename)
try:
if self.storage.file_exists(gcs_path):
self.storage.download_file(gcs_path, local_path)
logger.info(f"Cache hit: downloaded {filename}")
stats["downloaded"] += 1
else:
logger.debug(f"Cache miss: {filename} not in GCS")
stats["not_found"] += 1
except Exception as e:
logger.warning(f"Error downloading cache file {filename}: {e}")
stats["errors"] += 1
logger.info(
f"Cache sync from GCS complete: "
f"{stats['downloaded']} downloaded, "
f"{stats['not_found']} not found, "
f"{stats['errors']} errors"
)
return stats
def sync_cache_to_gcs(
self,
local_cache_dir: str,
audio_hash: str,
lyrics_hash: str,
) -> Dict[str, int]:
"""Upload new cache files from local directory to GCS.
Only uploads files that match expected cache patterns for the given
hashes and don't already exist in GCS.
Args:
local_cache_dir: Local directory with cache files.
audio_hash: MD5 hash of audio file.
lyrics_hash: MD5 hash of artist+title.
Returns:
Dict with counts: {"uploaded": N, "skipped": M, "errors": E}
"""
stats = {"uploaded": 0, "skipped": 0, "errors": 0}
if not os.path.exists(local_cache_dir):
logger.warning(f"Local cache dir does not exist: {local_cache_dir}")
return stats
# Get all possible cache filenames we're interested in
transcription_files = self._get_cache_filenames(
TRANSCRIPTION_PROVIDERS, audio_hash
)
lyrics_files = self._get_cache_filenames(LYRICS_PROVIDERS, lyrics_hash)
expected_files = set(transcription_files + lyrics_files)
# Check each file in local cache dir
for filename in os.listdir(local_cache_dir):
# Only process files we expect (matching our hash patterns)
if filename not in expected_files:
continue
local_path = os.path.join(local_cache_dir, filename)
if not os.path.isfile(local_path):
continue
gcs_path = f"{self.GCS_CACHE_PREFIX}{filename}"
try:
# Skip if already exists in GCS (same hash = same content)
if self.storage.file_exists(gcs_path):
logger.debug(f"Cache file already in GCS: {filename}")
stats["skipped"] += 1
continue
self.storage.upload_file(local_path, gcs_path)
logger.info(f"Uploaded cache file: {filename}")
stats["uploaded"] += 1
except Exception as e:
logger.warning(f"Error uploading cache file {filename}: {e}")
stats["errors"] += 1
logger.info(
f"Cache sync to GCS complete: "
f"{stats['uploaded']} uploaded, "
f"{stats['skipped']} skipped (already exist), "
f"{stats['errors']} errors"
)
return stats
-413
View File
@@ -1,413 +0,0 @@
"""
Custom metrics for Cloud Monitoring.
This module provides application-level metrics for tracking job processing,
worker performance, and external API usage. Metrics can be viewed in:
1. Cloud Logging (via structured log entries)
2. Cloud Monitoring (via log-based metrics or OpenTelemetry)
The metrics service uses a pragmatic approach:
- Always emits metrics as structured log entries (works immediately)
- Uses the same JSON format as Cloud Logging
- Can be enhanced with OpenTelemetry metrics exporters when available
Usage:
from backend.services.metrics import metrics
# Record a job completion
metrics.record_job_completed("abc123", source="upload")
# Record worker duration
metrics.record_worker_duration("audio", 45.2, success=True)
# Record external API call
with metrics.time_external_api("modal"):
response = await modal_client.separate_audio(...)
"""
import logging
import time
from contextlib import contextmanager
from dataclasses import dataclass, field
from typing import Any, Dict, Optional
from backend.services.tracing import get_current_trace_id, get_current_span_id
logger = logging.getLogger("metrics")
@dataclass
class MetricLabels:
"""Common metric labels."""
job_id: Optional[str] = None
worker: Optional[str] = None
status: Optional[str] = None
source: Optional[str] = None
api: Optional[str] = None
operation: Optional[str] = None
bucket: Optional[str] = None
def to_dict(self) -> Dict[str, Any]:
"""Convert to dict, excluding None values."""
return {k: v for k, v in self.__dict__.items() if v is not None}
class MetricsService:
"""
Application metrics service.
Emits metrics as structured log entries that can be:
1. Queried directly in Cloud Logging
2. Converted to Cloud Monitoring metrics via log-based metrics
3. Exported via OpenTelemetry (when configured)
"""
def __init__(self):
"""Initialize the metrics service."""
self._logger = logging.getLogger("metrics")
# Ensure metrics logger outputs at INFO level
self._logger.setLevel(logging.INFO)
def _emit_metric(self, metric_name: str, metric_type: str, value: float, labels: Dict[str, Any]) -> None:
"""
Emit a metric as a structured log entry.
The log format is designed to be easily parsed by Cloud Logging
and converted to log-based metrics.
Args:
metric_name: Name of the metric (e.g., "jobs_total")
metric_type: Type of metric (counter, histogram, gauge)
value: Metric value
labels: Metric labels/dimensions
"""
# Add trace context if available
trace_id = get_current_trace_id()
span_id = get_current_span_id()
# Build metric entry
metric_entry = {
"metric_name": metric_name,
"metric_type": metric_type,
"metric_value": value,
**labels,
}
if trace_id:
metric_entry["trace_id"] = trace_id
if span_id:
metric_entry["span_id"] = span_id
# Emit as structured log entry
# Use INFO level so metrics always show up
self._logger.info(
f"METRIC {metric_name}={value}",
extra=metric_entry
)
# =========================================
# Job Metrics
# =========================================
def record_job_created(self, job_id: str, source: str = "unknown") -> None:
"""
Record a new job creation.
Args:
job_id: Job ID
source: Job source (upload, url, search)
"""
self._emit_metric(
metric_name="jobs_total",
metric_type="counter",
value=1,
labels={"job_id": job_id, "status": "created", "source": source}
)
def record_job_completed(self, job_id: str, source: str = "unknown") -> None:
"""
Record a job completion.
Args:
job_id: Job ID
source: Job source (upload, url, search)
"""
self._emit_metric(
metric_name="jobs_total",
metric_type="counter",
value=1,
labels={"job_id": job_id, "status": "completed", "source": source}
)
def record_job_failed(self, job_id: str, source: str = "unknown", error: Optional[str] = None) -> None:
"""
Record a job failure.
Args:
job_id: Job ID
source: Job source (upload, url, search)
error: Optional error message
"""
labels = {"job_id": job_id, "status": "failed", "source": source}
if error:
labels["error"] = error[:200] # Truncate long errors
self._emit_metric(
metric_name="jobs_total",
metric_type="counter",
value=1,
labels=labels
)
def record_job_duration(self, job_id: str, duration_seconds: float, source: str = "unknown") -> None:
"""
Record total job processing duration.
Args:
job_id: Job ID
duration_seconds: Total processing time in seconds
source: Job source (upload, url, search)
"""
self._emit_metric(
metric_name="job_duration_seconds",
metric_type="histogram",
value=duration_seconds,
labels={"job_id": job_id, "source": source}
)
# =========================================
# Worker Metrics
# =========================================
def record_worker_started(self, worker: str, job_id: str) -> None:
"""
Record a worker invocation start.
Args:
worker: Worker name (audio, lyrics, screens, video, render_video)
job_id: Job ID
"""
self._emit_metric(
metric_name="worker_invocations_total",
metric_type="counter",
value=1,
labels={"worker": worker, "job_id": job_id, "status": "started"}
)
def record_worker_duration(self, worker: str, duration_seconds: float, success: bool, job_id: Optional[str] = None) -> None:
"""
Record worker processing duration.
Args:
worker: Worker name
duration_seconds: Processing time in seconds
success: Whether worker completed successfully
job_id: Optional job ID
"""
labels = {
"worker": worker,
"success": str(success).lower(),
}
if job_id:
labels["job_id"] = job_id
self._emit_metric(
metric_name="job_stage_duration_seconds",
metric_type="histogram",
value=duration_seconds,
labels=labels
)
# Also emit a counter for success/failure tracking
self._emit_metric(
metric_name="worker_invocations_total",
metric_type="counter",
value=1,
labels={"worker": worker, "success": str(success).lower(), "job_id": job_id or "unknown"}
)
# =========================================
# GCS Metrics
# =========================================
def record_gcs_operation(
self,
operation: str,
bucket: str,
success: bool,
size_bytes: Optional[int] = None,
duration_seconds: Optional[float] = None,
job_id: Optional[str] = None,
) -> None:
"""
Record a GCS operation.
Args:
operation: Operation type (upload, download, delete)
bucket: GCS bucket name
success: Whether operation succeeded
size_bytes: Optional file size in bytes
duration_seconds: Optional operation duration
job_id: Optional job ID
"""
labels = {
"operation": operation,
"bucket": bucket,
"success": str(success).lower(),
}
if job_id:
labels["job_id"] = job_id
self._emit_metric(
metric_name="gcs_operations_total",
metric_type="counter",
value=1,
labels=labels
)
if size_bytes is not None:
self._emit_metric(
metric_name="gcs_operation_bytes",
metric_type="histogram",
value=size_bytes,
labels={**labels, "operation": operation}
)
if duration_seconds is not None:
self._emit_metric(
metric_name="gcs_operation_duration_seconds",
metric_type="histogram",
value=duration_seconds,
labels={**labels, "operation": operation}
)
# =========================================
# External API Metrics
# =========================================
def record_external_api_call(
self,
api: str,
success: bool,
duration_seconds: float,
job_id: Optional[str] = None,
error: Optional[str] = None,
) -> None:
"""
Record an external API call.
Args:
api: API name (modal, audioshake, genius, spotify)
success: Whether call succeeded
duration_seconds: API call duration
job_id: Optional job ID
error: Optional error message
"""
labels = {
"api": api,
"success": str(success).lower(),
}
if job_id:
labels["job_id"] = job_id
if error:
labels["error"] = error[:100] # Truncate
self._emit_metric(
metric_name="external_api_calls_total",
metric_type="counter",
value=1,
labels=labels
)
self._emit_metric(
metric_name="external_api_duration_seconds",
metric_type="histogram",
value=duration_seconds,
labels={"api": api, "success": str(success).lower()}
)
@contextmanager
def time_external_api(self, api: str, job_id: Optional[str] = None):
"""
Context manager to time an external API call.
Usage:
with metrics.time_external_api("modal", job_id) as timer:
response = await client.call_api()
timer.set_success(True)
Args:
api: API name
job_id: Optional job ID
Yields:
Timer object with set_success() method
"""
timer = _ApiTimer()
start_time = time.time()
try:
yield timer
except Exception as e:
timer.set_success(False)
timer.error = str(e)
raise
finally:
duration = time.time() - start_time
self.record_external_api_call(
api=api,
success=timer.success,
duration_seconds=duration,
job_id=job_id,
error=timer.error,
)
@contextmanager
def time_worker(self, worker: str, job_id: str):
"""
Context manager to time a worker execution.
Usage:
with metrics.time_worker("audio", job_id) as timer:
await process_audio()
timer.set_success(True)
Args:
worker: Worker name
job_id: Job ID
Yields:
Timer object with set_success() method
"""
timer = _ApiTimer()
self.record_worker_started(worker, job_id)
start_time = time.time()
try:
yield timer
except Exception as e:
timer.set_success(False)
raise
finally:
duration = time.time() - start_time
self.record_worker_duration(
worker=worker,
duration_seconds=duration,
success=timer.success,
job_id=job_id,
)
class _ApiTimer:
"""Helper class for tracking API call success state."""
def __init__(self):
self.success = True # Assume success unless set otherwise
self.error: Optional[str] = None
def set_success(self, success: bool) -> None:
self.success = success
# Global metrics instance
metrics = MetricsService()
-122
View File
@@ -1,122 +0,0 @@
"""NLTK resource preloader for container startup.
Loads NLTK data at container startup to avoid slow downloads during request processing.
Cloud Run's ephemeral filesystem means NLTK data must be re-downloaded on each cold start,
which can take 30-100+ seconds for cmudict.
See docs/archive/2026-01-08-performance-investigation.md for background.
"""
import logging
import time
from typing import Optional, Dict, Any
logger = logging.getLogger(__name__)
# Singleton storage for preloaded resources
_preloaded_resources: Dict[str, Any] = {}
def preload_nltk_cmudict() -> None:
"""Preload NLTK's CMU Pronouncing Dictionary at startup.
The cmudict is used by SyllablesMatchHandler for syllable counting.
Without preloading, each SyllablesMatchHandler init downloads ~30MB,
which took 50-100+ seconds in Cloud Run.
"""
global _preloaded_resources
if "cmudict" in _preloaded_resources:
logger.info("NLTK cmudict already preloaded")
return
logger.info("Preloading NLTK cmudict...")
start_time = time.time()
try:
import nltk
# Ensure the data is downloaded
try:
from nltk.corpus import cmudict
# Try to access it - will raise LookupError if not downloaded
_ = cmudict.dict()
except LookupError:
logger.info("Downloading NLTK cmudict data...")
nltk.download("cmudict", quiet=True)
from nltk.corpus import cmudict
# Load into memory
cmu_dict = cmudict.dict()
_preloaded_resources["cmudict"] = cmu_dict
elapsed = time.time() - start_time
logger.info(f"NLTK cmudict preloaded in {elapsed:.2f}s ({len(cmu_dict)} entries)")
except Exception as e:
logger.error(f"Failed to preload NLTK cmudict: {e}")
raise
def get_preloaded_cmudict() -> Optional[Dict]:
"""Get the preloaded CMU dictionary if available.
Returns:
The preloaded cmudict dictionary, or None if not preloaded
"""
return _preloaded_resources.get("cmudict")
def is_cmudict_preloaded() -> bool:
"""Check if cmudict has been preloaded."""
return "cmudict" in _preloaded_resources
def preload_nltk_punkt() -> None:
"""Preload NLTK's punkt tokenizer (optional, used for sentence tokenization)."""
global _preloaded_resources
if "punkt" in _preloaded_resources:
logger.info("NLTK punkt already preloaded")
return
logger.info("Preloading NLTK punkt tokenizer...")
start_time = time.time()
try:
import nltk
try:
from nltk.tokenize import word_tokenize
# Test it works
_ = word_tokenize("test")
except LookupError:
logger.info("Downloading NLTK punkt data...")
nltk.download("punkt", quiet=True)
nltk.download("punkt_tab", quiet=True)
_preloaded_resources["punkt"] = True
elapsed = time.time() - start_time
logger.info(f"NLTK punkt preloaded in {elapsed:.2f}s")
except Exception as e:
logger.warning(f"Failed to preload NLTK punkt (non-critical): {e}")
def preload_all_nltk_resources() -> None:
"""Preload all NLTK resources used by the application."""
preload_nltk_cmudict()
# punkt is optional and less critical
try:
preload_nltk_punkt()
except Exception:
pass # Non-critical
def clear_preloaded_resources() -> None:
"""Clear all preloaded resources. Useful for testing."""
global _preloaded_resources
_preloaded_resources.clear()
-287
View File
@@ -1,287 +0,0 @@
"""
Packaging Service.
Provides CDG and TXT package generation functionality, extracted from KaraokeFinalise
for use by both the cloud backend (video_worker) and local CLI.
This service handles:
- CDG (CD+G) file generation from LRC files
- TXT lyric file generation from LRC files
- ZIP packaging of CDG/MP3 and TXT/MP3 pairs
"""
import logging
import os
import zipfile
from typing import Optional, Dict, Any, Tuple
logger = logging.getLogger(__name__)
class PackagingService:
"""
Service for creating CDG and TXT karaoke packages.
CDG (CD+Graphics) is a format used by karaoke machines.
TXT packages are used by software karaoke players.
Both formats are packaged as ZIP files with an MP3 audio track.
"""
def __init__(
self,
cdg_styles: Optional[Dict[str, Any]] = None,
dry_run: bool = False,
non_interactive: bool = False,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the packaging service.
Args:
cdg_styles: CDG style configuration for CDG generation
dry_run: If True, log actions without performing them
non_interactive: If True, skip interactive prompts
logger: Optional logger instance
"""
self.cdg_styles = cdg_styles
self.dry_run = dry_run
self.non_interactive = non_interactive
self.logger = logger or logging.getLogger(__name__)
def create_cdg_package(
self,
lrc_file: str,
audio_file: str,
output_zip_path: str,
artist: str,
title: str,
output_mp3_path: Optional[str] = None,
output_cdg_path: Optional[str] = None,
) -> Tuple[str, Optional[str], Optional[str]]:
"""
Create a CDG package (ZIP containing CDG and MP3 files).
Args:
lrc_file: Path to the LRC lyrics file
audio_file: Path to the instrumental audio file
output_zip_path: Path for the output ZIP file
artist: Artist name for metadata
title: Song title for metadata
output_mp3_path: Optional path for the extracted MP3 file
output_cdg_path: Optional path for the extracted CDG file
Returns:
Tuple of (zip_path, mp3_path, cdg_path)
Raises:
ValueError: If CDG styles are not configured
FileNotFoundError: If input files are missing
Exception: If CDG generation fails
"""
self.logger.info(f"Creating CDG package for {artist} - {title}")
# Validate inputs
if not os.path.isfile(lrc_file):
raise FileNotFoundError(f"LRC file not found: {lrc_file}")
if not os.path.isfile(audio_file):
raise FileNotFoundError(f"Audio file not found: {audio_file}")
# Check if ZIP already exists
if os.path.isfile(output_zip_path):
if self.non_interactive:
self.logger.info(
f"CDG ZIP exists, will be overwritten: {output_zip_path}"
)
else:
self.logger.info(f"CDG ZIP already exists: {output_zip_path}")
# Check if individual files exist (allows skipping generation)
if output_mp3_path and output_cdg_path:
if os.path.isfile(output_mp3_path) and os.path.isfile(output_cdg_path):
self.logger.info("Found existing MP3 and CDG files, creating ZIP directly")
if not self.dry_run:
self._create_zip_from_files(
output_zip_path,
[(output_mp3_path, os.path.basename(output_mp3_path)),
(output_cdg_path, os.path.basename(output_cdg_path))]
)
return output_zip_path, output_mp3_path, output_cdg_path
if self.dry_run:
self.logger.info(
f"DRY RUN: Would generate CDG package: {output_zip_path}"
)
return output_zip_path, output_mp3_path, output_cdg_path
# Generate CDG files
if self.cdg_styles is None:
raise ValueError(
"CDG styles configuration is required for CDG generation"
)
self.logger.info("Generating CDG and MP3 files")
from lyrics_transcriber.output.cdg import CDGGenerator
output_dir = os.path.dirname(output_zip_path) or os.getcwd()
generator = CDGGenerator(output_dir=output_dir, logger=self.logger)
cdg_file, mp3_file, zip_file = generator.generate_cdg_from_lrc(
lrc_file=lrc_file,
audio_file=audio_file,
title=title,
artist=artist,
cdg_styles=self.cdg_styles,
)
# Rename ZIP to expected output path if different
if os.path.isfile(zip_file) and zip_file != output_zip_path:
os.rename(zip_file, output_zip_path)
self.logger.info(f"Renamed CDG ZIP: {zip_file} -> {output_zip_path}")
if not os.path.isfile(output_zip_path):
raise Exception(f"Failed to create CDG ZIP file: {output_zip_path}")
# Extract the ZIP to get individual files if paths provided
extracted_mp3 = None
extracted_cdg = None
if output_mp3_path or output_cdg_path:
self.logger.info(f"Extracting CDG ZIP file: {output_zip_path}")
with zipfile.ZipFile(output_zip_path, "r") as zip_ref:
zip_ref.extractall(output_dir)
# Find extracted files
if output_mp3_path and os.path.isfile(output_mp3_path):
extracted_mp3 = output_mp3_path
self.logger.info(f"Extracted MP3: {extracted_mp3}")
if output_cdg_path and os.path.isfile(output_cdg_path):
extracted_cdg = output_cdg_path
self.logger.info(f"Extracted CDG: {extracted_cdg}")
self.logger.info(f"CDG package created: {output_zip_path}")
return output_zip_path, extracted_mp3, extracted_cdg
def create_txt_package(
self,
lrc_file: str,
mp3_file: str,
output_zip_path: str,
output_txt_path: Optional[str] = None,
) -> Tuple[str, Optional[str]]:
"""
Create a TXT package (ZIP containing TXT lyrics and MP3 files).
Args:
lrc_file: Path to the LRC lyrics file
mp3_file: Path to the MP3 audio file
output_zip_path: Path for the output ZIP file
output_txt_path: Optional path for the generated TXT file
Returns:
Tuple of (zip_path, txt_path)
Raises:
FileNotFoundError: If input files are missing
Exception: If TXT generation fails
"""
self.logger.info(f"Creating TXT package from {lrc_file}")
# Validate inputs
if not os.path.isfile(lrc_file):
raise FileNotFoundError(f"LRC file not found: {lrc_file}")
if not os.path.isfile(mp3_file):
raise FileNotFoundError(f"MP3 file not found: {mp3_file}")
# Check if ZIP already exists
if os.path.isfile(output_zip_path):
if self.non_interactive:
self.logger.info(
f"TXT ZIP exists, will be overwritten: {output_zip_path}"
)
else:
self.logger.info(f"TXT ZIP already exists: {output_zip_path}")
if self.dry_run:
self.logger.info(
f"DRY RUN: Would create TXT package: {output_zip_path}"
)
return output_zip_path, output_txt_path
# Generate TXT from LRC
self.logger.info(f"Converting LRC to TXT format: {lrc_file}")
from lyrics_converter import LyricsConverter
txt_converter = LyricsConverter(output_format="txt", filepath=lrc_file)
converted_txt = txt_converter.convert_file()
# Write TXT file
if output_txt_path is None:
# Default to same name as ZIP but with .txt extension
output_txt_path = output_zip_path.replace(".zip", ".txt")
with open(output_txt_path, "w") as txt_file:
txt_file.write(converted_txt)
self.logger.info(f"TXT file written: {output_txt_path}")
# Create ZIP containing MP3 and TXT
self.logger.info(f"Creating TXT ZIP: {output_zip_path}")
self._create_zip_from_files(
output_zip_path,
[(mp3_file, os.path.basename(mp3_file)),
(output_txt_path, os.path.basename(output_txt_path))]
)
if not os.path.isfile(output_zip_path):
raise Exception(f"Failed to create TXT ZIP file: {output_zip_path}")
self.logger.info(f"TXT package created: {output_zip_path}")
return output_zip_path, output_txt_path
def _create_zip_from_files(
self,
zip_path: str,
files: list,
) -> None:
"""
Create a ZIP file from a list of files.
Args:
zip_path: Path for the output ZIP file
files: List of (file_path, archive_name) tuples
"""
with zipfile.ZipFile(zip_path, "w") as zipf:
for file_path, archive_name in files:
if os.path.isfile(file_path):
zipf.write(file_path, archive_name)
self.logger.debug(f"Added to ZIP: {archive_name}")
else:
self.logger.warning(f"File not found for ZIP: {file_path}")
# Singleton instance and factory function (following existing service pattern)
_packaging_service: Optional[PackagingService] = None
def get_packaging_service(
cdg_styles: Optional[Dict[str, Any]] = None,
**kwargs
) -> PackagingService:
"""
Get a packaging service instance.
Args:
cdg_styles: CDG style configuration
**kwargs: Additional arguments passed to PackagingService
Returns:
PackagingService instance
"""
global _packaging_service
# Create new instance if settings changed
if _packaging_service is None or cdg_styles:
_packaging_service = PackagingService(
cdg_styles=cdg_styles,
**kwargs
)
return _packaging_service
@@ -1,409 +0,0 @@
"""
Push Notification Service for Web Push Notifications.
Handles sending push notifications to users' browsers/devices using the Web Push protocol.
Uses pywebpush library for VAPID authentication and push message encryption.
"""
import logging
from datetime import datetime, timezone
from typing import Optional, List
from pywebpush import webpush, WebPushException
from backend.config import get_settings
from backend.models.user import User, PushSubscription
logger = logging.getLogger(__name__)
class SubscriptionGoneError(Exception):
"""Raised when a push subscription is no longer valid (410 Gone)."""
pass
class PushNotificationService:
"""
Service for sending Web Push notifications.
Requires VAPID keys to be configured in Secret Manager:
- vapid-public-key: Base64-encoded public key
- vapid-private-key: Base64-encoded private key
"""
def __init__(self, db=None):
"""
Initialize the push notification service.
Args:
db: Firestore client (optional, will use singleton if not provided)
"""
self.settings = get_settings()
self._vapid_private_key: Optional[str] = None
self._vapid_public_key: Optional[str] = None
self._db = db
@property
def db(self):
"""Get Firestore client (lazy initialization)."""
if self._db is None:
from google.cloud import firestore
self._db = firestore.Client()
return self._db
@property
def vapid_private_key(self) -> Optional[str]:
"""Get VAPID private key from Secret Manager (cached)."""
if self._vapid_private_key is None:
self._vapid_private_key = self.settings.get_secret("vapid-private-key")
return self._vapid_private_key
@property
def vapid_public_key(self) -> Optional[str]:
"""Get VAPID public key from Secret Manager (cached)."""
if self._vapid_public_key is None:
self._vapid_public_key = self.settings.get_secret("vapid-public-key")
return self._vapid_public_key
def is_enabled(self) -> bool:
"""Check if push notifications are enabled and properly configured."""
return (
self.settings.enable_push_notifications and
self.vapid_private_key is not None and
self.vapid_public_key is not None
)
def get_public_key(self) -> Optional[str]:
"""Get the VAPID public key for client-side subscription."""
if not self.settings.enable_push_notifications:
return None
return self.vapid_public_key
async def send_push(
self,
user_email: str,
title: str,
body: str,
url: str = "/app/",
tag: Optional[str] = None
) -> int:
"""
Send a push notification to all of a user's subscribed devices.
Args:
user_email: The user's email address
title: Notification title
body: Notification body text
url: URL to open when notification is clicked
tag: Optional tag for notification grouping (replaces notifications with same tag)
Returns:
Number of notifications successfully sent
"""
if not self.is_enabled():
logger.debug("Push notifications not enabled, skipping")
return 0
# Get user from Firestore
user_doc = self.db.collection("gen_users").document(user_email).get()
if not user_doc.exists:
logger.warning(f"User {user_email} not found for push notification")
return 0
user_data = user_doc.to_dict()
subscriptions = user_data.get("push_subscriptions", [])
if not subscriptions:
logger.debug(f"User {user_email} has no push subscriptions")
return 0
# Build notification payload
import json
payload = json.dumps({
"title": title,
"body": body,
"url": url,
"tag": tag or "default",
"icon": "/nomad-logo.png"
})
# Send to each subscription
success_count = 0
invalid_subscriptions = []
for sub in subscriptions:
try:
webpush(
subscription_info={
"endpoint": sub["endpoint"],
"keys": sub["keys"]
},
data=payload,
vapid_private_key=self.vapid_private_key,
vapid_claims={
"sub": self.settings.vapid_subject
}
)
success_count += 1
logger.debug(f"Push sent to {sub.get('device_name', 'unknown device')}")
except WebPushException as e:
logger.warning(f"Push failed: {e}")
# Check if subscription is gone (410) or unauthorized (401/403)
if e.response and e.response.status_code in (404, 410):
logger.info(f"Subscription gone, will remove: {sub['endpoint'][:50]}...")
invalid_subscriptions.append(sub["endpoint"])
elif e.response and e.response.status_code in (401, 403):
logger.warning(f"Push auth failed for {sub['endpoint'][:50]}...")
except Exception as e:
logger.error(f"Unexpected error sending push: {e}")
# Remove invalid subscriptions
if invalid_subscriptions:
await self._remove_invalid_subscriptions(user_email, invalid_subscriptions)
logger.info(f"Push notifications sent to {success_count}/{len(subscriptions)} devices for {user_email}")
return success_count
async def _remove_invalid_subscriptions(self, user_email: str, endpoints: List[str]) -> None:
"""Remove invalid subscriptions from user's list."""
try:
user_ref = self.db.collection("gen_users").document(user_email)
user_doc = user_ref.get()
if not user_doc.exists:
return
user_data = user_doc.to_dict()
current_subs = user_data.get("push_subscriptions", [])
# Filter out invalid subscriptions
valid_subs = [s for s in current_subs if s["endpoint"] not in endpoints]
if len(valid_subs) < len(current_subs):
user_ref.update({"push_subscriptions": valid_subs})
logger.info(f"Removed {len(current_subs) - len(valid_subs)} invalid subscriptions for {user_email}")
except Exception as e:
logger.error(f"Failed to remove invalid subscriptions: {e}")
async def send_blocking_notification(
self,
job: dict,
action_type: str # "lyrics" or "instrumental"
) -> int:
"""
Send notification when a job enters a blocking state requiring user action.
Args:
job: Job dictionary with job_id, user_email, artist, title
action_type: Type of action needed ("lyrics" or "instrumental")
Returns:
Number of notifications sent
"""
user_email = job.get("user_email")
if not user_email:
return 0
job_id = job.get("job_id", "unknown")
artist = job.get("artist", "Unknown Artist")
title = job.get("title", "Unknown Title")
if action_type == "lyrics":
notif_title = "Review Lyrics"
notif_body = f'"{title}" by {artist} needs lyrics review'
url = f"/review/{job_id}"
tag = f"lyrics-{job_id}"
else: # instrumental
notif_title = "Select Instrumental"
notif_body = f'"{title}" by {artist} needs instrumental selection'
url = f"/instrumental/{job_id}"
tag = f"instrumental-{job_id}"
return await self.send_push(
user_email=user_email,
title=notif_title,
body=notif_body,
url=url,
tag=tag
)
async def send_completion_notification(self, job: dict) -> int:
"""
Send notification when a job completes successfully.
Args:
job: Job dictionary with job_id, user_email, artist, title
Returns:
Number of notifications sent
"""
user_email = job.get("user_email")
if not user_email:
return 0
job_id = job.get("job_id", "unknown")
artist = job.get("artist", "Unknown Artist")
title = job.get("title", "Unknown Title")
return await self.send_push(
user_email=user_email,
title="Video Ready!",
body=f'Your karaoke video for "{title}" by {artist} is ready to download',
url=f"/app/?job={job_id}",
tag=f"complete-{job_id}"
)
async def add_subscription(
self,
user_email: str,
endpoint: str,
keys: dict,
device_name: Optional[str] = None
) -> bool:
"""
Add a push subscription for a user.
Enforces max subscriptions per user - oldest removed if limit exceeded.
Args:
user_email: User's email address
endpoint: Push service endpoint URL
keys: Encryption keys (p256dh, auth)
device_name: Optional device identifier
Returns:
True if subscription was added successfully
"""
try:
user_ref = self.db.collection("gen_users").document(user_email)
user_doc = user_ref.get()
if not user_doc.exists:
logger.warning(f"User {user_email} not found")
return False
user_data = user_doc.to_dict()
subscriptions = user_data.get("push_subscriptions", [])
# Check if this endpoint already exists (update it)
existing_idx = None
for i, sub in enumerate(subscriptions):
if sub["endpoint"] == endpoint:
existing_idx = i
break
new_sub = {
"endpoint": endpoint,
"keys": keys,
"device_name": device_name,
"created_at": datetime.now(timezone.utc).isoformat(),
"last_used_at": None
}
if existing_idx is not None:
# Update existing subscription
subscriptions[existing_idx] = new_sub
logger.info(f"Updated existing push subscription for {user_email}")
else:
# Add new subscription
subscriptions.append(new_sub)
logger.info(f"Added new push subscription for {user_email}")
# Enforce max subscriptions (remove oldest)
max_subs = self.settings.max_push_subscriptions_per_user
if len(subscriptions) > max_subs:
# Sort by created_at and keep only the newest
subscriptions.sort(key=lambda s: s.get("created_at", ""), reverse=True)
removed = subscriptions[max_subs:]
subscriptions = subscriptions[:max_subs]
logger.info(f"Removed {len(removed)} old subscriptions for {user_email} (max {max_subs})")
user_ref.update({"push_subscriptions": subscriptions})
return True
except Exception as e:
logger.error(f"Failed to add subscription for {user_email}: {e}")
return False
async def remove_subscription(self, user_email: str, endpoint: str) -> bool:
"""
Remove a push subscription for a user.
Args:
user_email: User's email address
endpoint: Push service endpoint URL to remove
Returns:
True if subscription was removed
"""
try:
user_ref = self.db.collection("gen_users").document(user_email)
user_doc = user_ref.get()
if not user_doc.exists:
return False
user_data = user_doc.to_dict()
subscriptions = user_data.get("push_subscriptions", [])
# Filter out the subscription
new_subs = [s for s in subscriptions if s["endpoint"] != endpoint]
if len(new_subs) < len(subscriptions):
user_ref.update({"push_subscriptions": new_subs})
logger.info(f"Removed push subscription for {user_email}")
return True
return False
except Exception as e:
logger.error(f"Failed to remove subscription for {user_email}: {e}")
return False
async def list_subscriptions(self, user_email: str) -> List[dict]:
"""
List all push subscriptions for a user.
Args:
user_email: User's email address
Returns:
List of subscription info (endpoint, device_name, created_at, last_used_at)
"""
try:
user_doc = self.db.collection("gen_users").document(user_email).get()
if not user_doc.exists:
return []
user_data = user_doc.to_dict()
subscriptions = user_data.get("push_subscriptions", [])
# Return safe subset of subscription data
return [
{
"endpoint": s["endpoint"],
"device_name": s.get("device_name"),
"created_at": s.get("created_at"),
"last_used_at": s.get("last_used_at")
}
for s in subscriptions
]
except Exception as e:
logger.error(f"Failed to list subscriptions for {user_email}: {e}")
return []
# Singleton instance
_push_service: Optional[PushNotificationService] = None
def get_push_notification_service(db=None) -> PushNotificationService:
"""Get the push notification service singleton."""
global _push_service
if _push_service is None:
_push_service = PushNotificationService(db)
return _push_service
-641
View File
@@ -1,641 +0,0 @@
"""
Rate limiting service for job creation, YouTube uploads, and beta enrollment.
Uses Firestore for distributed rate limit tracking with date-based document IDs
that automatically reset at UTC midnight.
"""
import hashlib
import logging
from datetime import datetime, timezone
from typing import Tuple, Optional, List, Dict, Any
from google.cloud import firestore
from backend.config import settings
logger = logging.getLogger(__name__)
# Firestore collection names
RATE_LIMITS_COLLECTION = "rate_limits"
BLOCKLISTS_COLLECTION = "blocklists"
OVERRIDES_COLLECTION = "overrides"
def _get_today_date_str() -> str:
"""Get today's date in UTC as YYYY-MM-DD string."""
return datetime.now(timezone.utc).strftime("%Y-%m-%d")
def _seconds_until_midnight_utc() -> int:
"""Calculate seconds until UTC midnight (when rate limits reset)."""
now = datetime.now(timezone.utc)
midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
# Add 1 day to get next midnight
from datetime import timedelta
next_midnight = midnight + timedelta(days=1)
return int((next_midnight - now).total_seconds())
def _hash_ip(ip_address: str) -> str:
"""Hash an IP address for storage (privacy-preserving)."""
return hashlib.sha256(ip_address.encode()).hexdigest()[:16]
class RateLimitService:
"""
Service for managing rate limits across the platform.
Rate limits use Firestore for distributed tracking with automatic
reset at UTC midnight via date-based document IDs.
"""
def __init__(self, db: Optional[firestore.Client] = None):
"""Initialize rate limit service with Firestore client."""
if db is None:
self.db = firestore.Client(project=settings.google_cloud_project)
else:
self.db = db
self._blocklist_cache: Optional[Dict[str, Any]] = None
self._blocklist_cache_time: Optional[datetime] = None
self._cache_ttl_seconds = 300 # 5 minute cache
# -------------------------------------------------------------------------
# Job Rate Limiting
# -------------------------------------------------------------------------
def check_user_job_limit(self, user_email: str, is_admin: bool = False) -> Tuple[bool, int, str]:
"""
Check if a user can create a new job.
Args:
user_email: User's email address
is_admin: Whether the user is an admin (admins bypass limits)
Returns:
Tuple of (allowed, remaining, message)
- allowed: True if user can create job
- remaining: Number of jobs remaining today
- message: Human-readable status message
"""
if not settings.enable_rate_limiting:
return True, -1, "Rate limiting disabled"
if is_admin:
return True, -1, "Admin users bypass rate limits"
limit = settings.rate_limit_jobs_per_day
if limit == 0:
return True, -1, "No job limit configured"
# Check for user override
override = self.get_user_override(user_email)
if override and override.get("bypass_job_limit"):
return True, -1, "User has rate limit bypass"
if override and override.get("custom_daily_job_limit"):
limit = override["custom_daily_job_limit"]
# Get current count
current_count = self.get_user_job_count_today(user_email)
remaining = max(0, limit - current_count)
if current_count >= limit:
seconds = _seconds_until_midnight_utc()
logger.warning(
f"Rate limit exceeded for {user_email}: {current_count}/{limit} jobs today"
)
return (
False,
0,
f"Daily job limit reached ({limit} jobs per day). Resets in {seconds // 3600}h {(seconds % 3600) // 60}m."
)
return True, remaining, f"{remaining} jobs remaining today"
def get_user_job_count_today(self, user_email: str) -> int:
"""Get the number of jobs created by a user today."""
date_str = _get_today_date_str()
doc_ref = self.db.collection(RATE_LIMITS_COLLECTION).document(
f"user_jobs_{user_email}_{date_str}"
)
doc = doc_ref.get()
if not doc.exists:
return 0
return doc.to_dict().get("count", 0)
def record_job_creation(self, user_email: str, job_id: str) -> None:
"""
Record a job creation for rate limiting.
Uses Firestore transactions for atomic increment.
"""
if not settings.enable_rate_limiting:
return
date_str = _get_today_date_str()
doc_ref = self.db.collection(RATE_LIMITS_COLLECTION).document(
f"user_jobs_{user_email}_{date_str}"
)
@firestore.transactional
def update_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if doc.exists:
data = doc.to_dict()
count = data.get("count", 0) + 1
job_ids = data.get("job_ids", [])
job_ids.append(job_id)
else:
count = 1
job_ids = [job_id]
transaction.set(doc_ref, {
"user_email": user_email,
"date": date_str,
"count": count,
"job_ids": job_ids,
"updated_at": datetime.now(timezone.utc),
})
transaction = self.db.transaction()
update_in_transaction(transaction, doc_ref)
logger.info(f"Recorded job {job_id} for user {user_email} rate limiting")
# -------------------------------------------------------------------------
# YouTube Upload Rate Limiting
# -------------------------------------------------------------------------
def check_youtube_upload_limit(self) -> Tuple[bool, int, str]:
"""
Check if the system can perform a YouTube upload.
YouTube uploads are limited system-wide due to API quota constraints.
Returns:
Tuple of (allowed, remaining, message)
"""
if not settings.enable_rate_limiting:
return True, -1, "Rate limiting disabled"
limit = settings.rate_limit_youtube_uploads_per_day
if limit == 0:
return True, -1, "No YouTube upload limit configured"
current_count = self.get_youtube_uploads_today()
remaining = max(0, limit - current_count)
if current_count >= limit:
seconds = _seconds_until_midnight_utc()
logger.warning(
f"YouTube upload limit exceeded: {current_count}/{limit} uploads today"
)
return (
False,
0,
f"Daily YouTube upload limit reached ({limit} uploads per day). Resets in {seconds // 3600}h {(seconds % 3600) // 60}m."
)
return True, remaining, f"{remaining} YouTube uploads remaining today"
def get_youtube_uploads_today(self) -> int:
"""Get the number of YouTube uploads performed today (system-wide)."""
date_str = _get_today_date_str()
doc_ref = self.db.collection(RATE_LIMITS_COLLECTION).document(
f"youtube_uploads_{date_str}"
)
doc = doc_ref.get()
if not doc.exists:
return 0
return doc.to_dict().get("count", 0)
def record_youtube_upload(self, job_id: str, user_email: str) -> None:
"""
Record a YouTube upload for rate limiting.
Uses Firestore transactions for atomic increment.
"""
if not settings.enable_rate_limiting:
return
date_str = _get_today_date_str()
doc_ref = self.db.collection(RATE_LIMITS_COLLECTION).document(
f"youtube_uploads_{date_str}"
)
@firestore.transactional
def update_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if doc.exists:
data = doc.to_dict()
count = data.get("count", 0) + 1
uploads = data.get("uploads", [])
else:
count = 1
uploads = []
uploads.append({
"job_id": job_id,
"user_email": user_email,
"timestamp": datetime.now(timezone.utc),
})
transaction.set(doc_ref, {
"date": date_str,
"count": count,
"uploads": uploads,
"updated_at": datetime.now(timezone.utc),
})
transaction = self.db.transaction()
update_in_transaction(transaction, doc_ref)
logger.info(f"Recorded YouTube upload for job {job_id}")
# -------------------------------------------------------------------------
# Beta Enrollment IP Rate Limiting
# -------------------------------------------------------------------------
def check_beta_ip_limit(self, ip_address: str) -> Tuple[bool, int, str]:
"""
Check if an IP address can enroll in the beta program.
Args:
ip_address: Client IP address
Returns:
Tuple of (allowed, remaining, message)
"""
if not settings.enable_rate_limiting:
return True, -1, "Rate limiting disabled"
limit = settings.rate_limit_beta_ip_per_day
if limit == 0:
return True, -1, "No beta IP limit configured"
# Check today's enrollment count for this IP
ip_hash = _hash_ip(ip_address)
date_str = _get_today_date_str()
doc_ref = self.db.collection(RATE_LIMITS_COLLECTION).document(
f"beta_ip_{ip_hash}_{date_str}"
)
doc = doc_ref.get()
current_count = 0
if doc.exists:
current_count = doc.to_dict().get("count", 0)
remaining = max(0, limit - current_count)
if current_count >= limit:
seconds = _seconds_until_midnight_utc()
logger.warning(
f"Beta enrollment IP limit exceeded for {ip_hash}: {current_count}/{limit} enrollments today"
)
return (
False,
0,
f"Too many beta enrollments from this network today. Please try again tomorrow."
)
return True, remaining, f"{remaining} beta enrollments remaining from this IP today"
def record_beta_enrollment(self, ip_address: str, email: str) -> None:
"""
Record a beta enrollment for IP rate limiting.
Uses Firestore transactions for atomic increment.
"""
if not settings.enable_rate_limiting:
return
ip_hash = _hash_ip(ip_address)
date_str = _get_today_date_str()
doc_ref = self.db.collection(RATE_LIMITS_COLLECTION).document(
f"beta_ip_{ip_hash}_{date_str}"
)
@firestore.transactional
def update_in_transaction(transaction, doc_ref):
doc = doc_ref.get(transaction=transaction)
if doc.exists:
data = doc.to_dict()
count = data.get("count", 0) + 1
enrollments = data.get("enrollments", [])
else:
count = 1
enrollments = []
enrollments.append({
"email": email,
"timestamp": datetime.now(timezone.utc),
})
transaction.set(doc_ref, {
"ip_hash": ip_hash,
"date": date_str,
"count": count,
"enrollments": enrollments,
"updated_at": datetime.now(timezone.utc),
})
transaction = self.db.transaction()
update_in_transaction(transaction, doc_ref)
logger.info(f"Recorded beta enrollment from IP {ip_hash} for {email}")
# -------------------------------------------------------------------------
# User Overrides (Whitelist)
# -------------------------------------------------------------------------
def get_user_override(self, user_email: str) -> Optional[Dict[str, Any]]:
"""
Get rate limit override settings for a user.
Returns:
Override settings dict or None if no override exists
"""
doc_ref = self.db.collection(OVERRIDES_COLLECTION).document(user_email.lower())
doc = doc_ref.get()
if not doc.exists:
return None
return doc.to_dict()
def set_user_override(
self,
user_email: str,
bypass_job_limit: bool = False,
custom_daily_job_limit: Optional[int] = None,
reason: str = "",
admin_email: str = ""
) -> None:
"""
Set rate limit override for a user.
Args:
user_email: User to override
bypass_job_limit: If True, user bypasses all job limits
custom_daily_job_limit: Custom limit (None = use default)
reason: Reason for override
admin_email: Admin who set the override
"""
doc_ref = self.db.collection(OVERRIDES_COLLECTION).document(user_email.lower())
doc_ref.set({
"email": user_email.lower(),
"bypass_job_limit": bypass_job_limit,
"custom_daily_job_limit": custom_daily_job_limit,
"reason": reason,
"created_by": admin_email,
"created_at": datetime.now(timezone.utc),
})
logger.info(f"Set rate limit override for {user_email} by {admin_email}")
def remove_user_override(self, user_email: str, admin_email: str = "") -> bool:
"""Remove rate limit override for a user."""
doc_ref = self.db.collection(OVERRIDES_COLLECTION).document(user_email.lower())
doc = doc_ref.get()
if not doc.exists:
return False
doc_ref.delete()
logger.info(f"Removed rate limit override for {user_email} by {admin_email}")
return True
def list_user_overrides(self) -> List[Dict[str, Any]]:
"""List all user rate limit overrides."""
docs = self.db.collection(OVERRIDES_COLLECTION).stream()
return [doc.to_dict() for doc in docs]
def get_all_overrides(self) -> Dict[str, Dict[str, Any]]:
"""Get all user rate limit overrides as a dict keyed by email."""
docs = self.db.collection(OVERRIDES_COLLECTION).stream()
return {doc.id: doc.to_dict() for doc in docs}
# -------------------------------------------------------------------------
# Blocklist Management
# -------------------------------------------------------------------------
def _load_blocklist(self, force_refresh: bool = False) -> Dict[str, Any]:
"""
Load blocklist from Firestore with caching.
Returns:
Dict with keys: disposable_domains, blocked_emails, blocked_ips
"""
now = datetime.now(timezone.utc)
# Check cache
if not force_refresh and self._blocklist_cache and self._blocklist_cache_time:
cache_age = (now - self._blocklist_cache_time).total_seconds()
if cache_age < self._cache_ttl_seconds:
return self._blocklist_cache
# Load from Firestore
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document("config")
doc = doc_ref.get()
if not doc.exists:
# Initialize with empty blocklist
self._blocklist_cache = {
"disposable_domains": [],
"blocked_emails": [],
"blocked_ips": [],
}
else:
self._blocklist_cache = doc.to_dict()
self._blocklist_cache_time = now
return self._blocklist_cache
def is_disposable_domain(self, domain: str) -> bool:
"""Check if a domain is in the disposable domains blocklist."""
blocklist = self._load_blocklist()
return domain.lower() in [d.lower() for d in blocklist.get("disposable_domains", [])]
def is_blocked_email(self, email: str) -> bool:
"""Check if an email is explicitly blocked."""
blocklist = self._load_blocklist()
return email.lower() in [e.lower() for e in blocklist.get("blocked_emails", [])]
def is_blocked_ip(self, ip_address: str) -> bool:
"""Check if an IP address is blocked."""
blocklist = self._load_blocklist()
return ip_address in blocklist.get("blocked_ips", [])
def get_blocklist(self) -> Dict[str, Any]:
"""Get all blocklist data."""
return self._load_blocklist(force_refresh=True)
def add_disposable_domain(self, domain: str, admin_email: str) -> None:
"""Add a domain to the disposable domains blocklist."""
blocklist = self._load_blocklist(force_refresh=True)
domains = set(blocklist.get("disposable_domains", []))
domains.add(domain.lower())
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document("config")
doc_ref.set({
**blocklist,
"disposable_domains": list(domains),
"updated_at": datetime.now(timezone.utc),
"updated_by": admin_email,
}, merge=True)
self._blocklist_cache = None # Invalidate cache
logger.info(f"Added disposable domain {domain} by {admin_email}")
def remove_disposable_domain(self, domain: str) -> bool:
"""Remove a domain from the disposable domains blocklist."""
blocklist = self._load_blocklist(force_refresh=True)
domains = set(blocklist.get("disposable_domains", []))
if domain.lower() not in [d.lower() for d in domains]:
return False
domains = {d for d in domains if d.lower() != domain.lower()}
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document("config")
doc_ref.set({
**blocklist,
"disposable_domains": list(domains),
"updated_at": datetime.now(timezone.utc),
}, merge=True)
self._blocklist_cache = None
logger.info(f"Removed disposable domain {domain}")
return True
def add_blocked_email(self, email: str, admin_email: str) -> None:
"""Add an email to the blocked emails list."""
blocklist = self._load_blocklist(force_refresh=True)
emails = set(blocklist.get("blocked_emails", []))
emails.add(email.lower())
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document("config")
doc_ref.set({
**blocklist,
"blocked_emails": list(emails),
"updated_at": datetime.now(timezone.utc),
"updated_by": admin_email,
}, merge=True)
self._blocklist_cache = None
logger.info(f"Added blocked email {email} by {admin_email}")
def remove_blocked_email(self, email: str) -> bool:
"""Remove an email from the blocked emails list."""
blocklist = self._load_blocklist(force_refresh=True)
emails = set(blocklist.get("blocked_emails", []))
if email.lower() not in [e.lower() for e in emails]:
return False
emails = {e for e in emails if e.lower() != email.lower()}
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document("config")
doc_ref.set({
**blocklist,
"blocked_emails": list(emails),
"updated_at": datetime.now(timezone.utc),
}, merge=True)
self._blocklist_cache = None
logger.info(f"Removed blocked email {email}")
return True
def add_blocked_ip(self, ip_address: str, admin_email: str) -> None:
"""Add an IP address to the blocked IPs list."""
blocklist = self._load_blocklist(force_refresh=True)
ips = set(blocklist.get("blocked_ips", []))
ips.add(ip_address)
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document("config")
doc_ref.set({
**blocklist,
"blocked_ips": list(ips),
"updated_at": datetime.now(timezone.utc),
"updated_by": admin_email,
}, merge=True)
self._blocklist_cache = None
logger.info(f"Added blocked IP {ip_address} by {admin_email}")
def remove_blocked_ip(self, ip_address: str) -> bool:
"""Remove an IP address from the blocked IPs list."""
blocklist = self._load_blocklist(force_refresh=True)
ips = set(blocklist.get("blocked_ips", []))
if ip_address not in ips:
return False
ips.discard(ip_address)
doc_ref = self.db.collection(BLOCKLISTS_COLLECTION).document("config")
doc_ref.set({
**blocklist,
"blocked_ips": list(ips),
"updated_at": datetime.now(timezone.utc),
}, merge=True)
self._blocklist_cache = None
logger.info(f"Removed blocked IP {ip_address}")
return True
# -------------------------------------------------------------------------
# Stats
# -------------------------------------------------------------------------
def get_rate_limit_stats(self) -> Dict[str, Any]:
"""
Get current rate limit statistics.
Returns:
Dict with current usage stats
"""
date_str = _get_today_date_str()
# Get YouTube upload count
youtube_count = self.get_youtube_uploads_today()
# Count unique users with jobs today
users_with_jobs = set()
total_jobs_today = 0
# Query all user job documents for today
docs = self.db.collection(RATE_LIMITS_COLLECTION).where(
"date", "==", date_str
).stream()
for doc in docs:
data = doc.to_dict()
if data.get("user_email"):
users_with_jobs.add(data["user_email"])
total_jobs_today += data.get("count", 0)
return {
"date": date_str,
"youtube_uploads_today": youtube_count,
"youtube_uploads_limit": settings.rate_limit_youtube_uploads_per_day,
"total_jobs_today": total_jobs_today,
"users_with_jobs_today": len(users_with_jobs),
"job_limit_per_user": settings.rate_limit_jobs_per_day,
"beta_ip_limit_per_day": settings.rate_limit_beta_ip_per_day,
"rate_limiting_enabled": settings.enable_rate_limiting,
"seconds_until_reset": _seconds_until_midnight_utc(),
}
# Singleton instance
_rate_limit_service: Optional[RateLimitService] = None
def get_rate_limit_service() -> RateLimitService:
"""Get the singleton RateLimitService instance."""
global _rate_limit_service
if _rate_limit_service is None:
_rate_limit_service = RateLimitService()
return _rate_limit_service
-65
View File
@@ -1,65 +0,0 @@
"""SpaCy model preloader for container startup.
Loads SpaCy models at container startup to avoid slow loading during request processing.
Cloud Run filesystem I/O can cause 60+ second delays when loading SpaCy models lazily.
"""
import logging
import time
from typing import Optional
logger = logging.getLogger(__name__)
# Singleton storage for preloaded models
_preloaded_models: dict = {}
def preload_spacy_model(model_name: str = "en_core_web_sm") -> None:
"""Preload a SpaCy model at startup.
Args:
model_name: The SpaCy model to load (default: en_core_web_sm)
"""
global _preloaded_models
if model_name in _preloaded_models:
logger.info(f"SpaCy model '{model_name}' already preloaded")
return
logger.info(f"Preloading SpaCy model '{model_name}'...")
start_time = time.time()
try:
import spacy
nlp = spacy.load(model_name)
_preloaded_models[model_name] = nlp
elapsed = time.time() - start_time
logger.info(f"SpaCy model '{model_name}' preloaded in {elapsed:.2f}s")
except Exception as e:
logger.error(f"Failed to preload SpaCy model '{model_name}': {e}")
raise
def get_preloaded_model(model_name: str = "en_core_web_sm") -> Optional[object]:
"""Get a preloaded SpaCy model if available.
Args:
model_name: The SpaCy model name
Returns:
The preloaded SpaCy Language object, or None if not preloaded
"""
return _preloaded_models.get(model_name)
def is_model_preloaded(model_name: str = "en_core_web_sm") -> bool:
"""Check if a SpaCy model has been preloaded."""
return model_name in _preloaded_models
def clear_preloaded_models() -> None:
"""Clear all preloaded models. Useful for testing."""
global _preloaded_models
_preloaded_models.clear()
-397
View File
@@ -1,397 +0,0 @@
"""
Stripe service for payment processing.
Handles:
- Creating checkout sessions for credit purchases
- Processing webhook events
- Managing customer records
"""
import logging
import os
from typing import Optional, Dict, Any, Tuple
import stripe
from backend.config import get_settings
logger = logging.getLogger(__name__)
# Credit packages available for purchase
CREDIT_PACKAGES = {
"1_credit": {
"credits": 1,
"price_cents": 500, # $5.00
"name": "1 Karaoke Credit",
"description": "Create 1 professional karaoke video",
},
"3_credits": {
"credits": 3,
"price_cents": 1200, # $12.00 (20% discount)
"name": "3 Karaoke Credits",
"description": "Create 3 professional karaoke videos (Save 20%)",
},
"5_credits": {
"credits": 5,
"price_cents": 1750, # $17.50 (30% discount)
"name": "5 Karaoke Credits",
"description": "Create 5 professional karaoke videos (Save 30%)",
},
"10_credits": {
"credits": 10,
"price_cents": 3000, # $30.00 (40% discount)
"name": "10 Karaoke Credits",
"description": "Create 10 professional karaoke videos (Save 40%)",
},
}
# Made-for-you service package (not a credit package - creates a job directly)
MADE_FOR_YOU_PACKAGE = {
"price_cents": 1500, # $15.00
"name": "Made For You Karaoke Video",
"description": "Professional 4K karaoke video with perfectly synced lyrics, delivered to your email within 24 hours",
"images": ["https://gen.nomadkaraoke.com/nomad-logo.png"],
}
class StripeService:
"""Service for Stripe payment processing."""
def __init__(self):
"""Initialize Stripe with API key."""
self.settings = get_settings()
self.secret_key = os.getenv("STRIPE_SECRET_KEY")
self.webhook_secret = os.getenv("STRIPE_WEBHOOK_SECRET")
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
# After consolidation, buy URL is the same as frontend URL
self.buy_url = os.getenv("BUY_URL", self.frontend_url)
# Payment method configuration ID from Stripe Dashboard
# Enables Google Pay, Apple Pay, Link, and other wallet methods
# Get this from: Dashboard > Settings > Payment methods > [Your Config] > Copy ID
self.payment_method_config = os.getenv("STRIPE_PAYMENT_METHOD_CONFIG")
if self.secret_key:
stripe.api_key = self.secret_key
logger.info("Stripe initialized with API key")
else:
logger.warning("STRIPE_SECRET_KEY not set - payments disabled")
if self.payment_method_config:
logger.info(f"Using payment method config: {self.payment_method_config}")
else:
logger.info("No STRIPE_PAYMENT_METHOD_CONFIG set - using Stripe defaults")
def is_configured(self) -> bool:
"""Check if Stripe is properly configured."""
return bool(self.secret_key)
def get_credit_packages(self) -> Dict[str, Dict[str, Any]]:
"""Get available credit packages."""
return CREDIT_PACKAGES
def create_checkout_session(
self,
package_id: str,
user_email: str,
success_url: Optional[str] = None,
cancel_url: Optional[str] = None,
) -> Tuple[bool, Optional[str], str]:
"""
Create a Stripe Checkout session for purchasing credits.
Args:
package_id: ID of the credit package to purchase
user_email: Email of the purchasing user
success_url: URL to redirect to on success (optional)
cancel_url: URL to redirect to on cancel (optional)
Returns:
(success, checkout_url, message)
"""
if not self.is_configured():
return False, None, "Payment processing is not configured"
package = CREDIT_PACKAGES.get(package_id)
if not package:
return False, None, f"Invalid package: {package_id}"
try:
# Default URLs
if not success_url:
success_url = f"{self.frontend_url}/payment/success?session_id={{CHECKOUT_SESSION_ID}}"
if not cancel_url:
cancel_url = f"{self.buy_url}?cancelled=true"
# Build checkout session params
session_params = {
# Omit payment_method_types to auto-enable Apple Pay, Google Pay, Link, etc.
'line_items': [{
'price_data': {
'currency': 'usd',
'product_data': {
'name': package['name'],
'description': package['description'],
'images': ['https://gen.nomadkaraoke.com/nomad-logo.png'],
},
'unit_amount': package['price_cents'],
},
'quantity': 1,
}],
'mode': 'payment',
'success_url': success_url,
'cancel_url': cancel_url,
'customer_email': user_email,
'metadata': {
'package_id': package_id,
'credits': str(package['credits']),
'user_email': user_email,
},
# Allow promotion codes
'allow_promotion_codes': True,
}
# Add payment method configuration if set (enables Google Pay, Link, etc.)
if self.payment_method_config:
session_params['payment_method_configuration'] = self.payment_method_config
# Create checkout session
session = stripe.checkout.Session.create(**session_params)
logger.info(f"Created checkout session {session.id} for {user_email}, package {package_id}")
return True, session.url, "Checkout session created"
except stripe.error.StripeError as e:
logger.error(f"Stripe error creating checkout session: {e}")
return False, None, f"Payment error: {str(e)}"
except Exception as e:
logger.error(f"Error creating checkout session: {e}")
return False, None, "Failed to create checkout session"
def create_made_for_you_checkout_session(
self,
customer_email: str,
artist: str,
title: str,
source_type: str = "search",
youtube_url: Optional[str] = None,
notes: Optional[str] = None,
success_url: Optional[str] = None,
cancel_url: Optional[str] = None,
) -> Tuple[bool, Optional[str], str]:
"""
Create a Stripe Checkout session for a made-for-you order.
This is for the full-service karaoke video creation where Nomad Karaoke
handles all the work (lyrics review, instrumental selection, etc.).
Args:
customer_email: Customer's email for delivery
artist: Song artist
title: Song title
source_type: Audio source type (search, youtube, upload)
youtube_url: YouTube URL if source_type is youtube
notes: Any special requests from customer
success_url: URL to redirect to on success (optional)
cancel_url: URL to redirect to on cancel (optional)
Returns:
(success, checkout_url, message)
"""
if not self.is_configured():
return False, None, "Payment processing is not configured"
try:
# Default URLs - redirect to gen app success page
if not success_url:
success_url = f"{self.frontend_url}/order/success?session_id={{CHECKOUT_SESSION_ID}}"
if not cancel_url:
cancel_url = "https://nomadkaraoke.com/#do-it-for-me"
# Build metadata for job creation after payment
metadata = {
'order_type': 'made_for_you',
'customer_email': customer_email,
'artist': artist,
'title': title,
'source_type': source_type,
}
if youtube_url:
metadata['youtube_url'] = youtube_url
if notes:
# Truncate notes to fit Stripe's 500 char limit per metadata value
metadata['notes'] = notes[:500] if len(notes) > 500 else notes
# Build checkout session params
session_params = {
# Omit payment_method_types to auto-enable Apple Pay, Google Pay, Link, etc.
'line_items': [{
'price_data': {
'currency': 'usd',
'product_data': {
'name': f"Karaoke Video: {artist} - {title}",
'description': MADE_FOR_YOU_PACKAGE['description'],
'images': MADE_FOR_YOU_PACKAGE['images'],
},
'unit_amount': MADE_FOR_YOU_PACKAGE['price_cents'],
},
'quantity': 1,
}],
'mode': 'payment',
'success_url': success_url,
'cancel_url': cancel_url,
'customer_email': customer_email,
'metadata': metadata,
'allow_promotion_codes': True,
}
# Add payment method configuration if set (enables Google Pay, Link, etc.)
if self.payment_method_config:
session_params['payment_method_configuration'] = self.payment_method_config
# Create checkout session
session = stripe.checkout.Session.create(**session_params)
logger.info(
f"Created made-for-you checkout session {session.id} for {customer_email}, "
f"song: {artist} - {title}"
)
return True, session.url, "Checkout session created"
except stripe.error.StripeError as e:
logger.error(f"Stripe error creating made-for-you checkout session: {e}")
return False, None, f"Payment error: {str(e)}"
except Exception as e:
logger.error(f"Error creating made-for-you checkout session: {e}")
return False, None, "Failed to create checkout session"
def verify_webhook_signature(self, payload: bytes, signature: str) -> Tuple[bool, Optional[Dict], str]:
"""
Verify a Stripe webhook signature.
Args:
payload: Raw request body
signature: Stripe-Signature header value
Returns:
(valid, event_data, message)
"""
if not self.webhook_secret:
logger.error("STRIPE_WEBHOOK_SECRET not configured")
return False, None, "Webhook secret not configured"
try:
event = stripe.Webhook.construct_event(
payload, signature, self.webhook_secret
)
return True, event, "Webhook verified"
except stripe.error.SignatureVerificationError as e:
logger.error(f"Invalid webhook signature: {e}")
return False, None, "Invalid signature"
except Exception as e:
logger.error(f"Error verifying webhook: {e}")
return False, None, str(e)
def handle_checkout_completed(self, session: Dict) -> Tuple[bool, Optional[str], int, str]:
"""
Handle a completed checkout session.
Args:
session: Stripe checkout session object
Returns:
(success, user_email, credits_to_add, message)
"""
try:
metadata = session.get('metadata', {})
user_email = metadata.get('user_email') or session.get('customer_email')
try:
credits = int(metadata.get('credits', 0))
except (ValueError, TypeError):
logger.error(f"Invalid credits metadata in session {session.get('id')}: {metadata.get('credits')}")
return False, user_email, 0, "Invalid credit amount in session metadata"
package_id = metadata.get('package_id')
if not user_email:
logger.error(f"No user email in checkout session {session.get('id')}")
return False, None, 0, "No user email found"
if credits <= 0:
logger.error(f"Invalid credits in checkout session {session.get('id')}")
return False, user_email, 0, "Invalid credit amount"
logger.info(
f"Checkout completed: {user_email} purchased {credits} credits "
f"(package: {package_id}, session: {session.get('id')})"
)
return True, user_email, credits, f"Successfully purchased {credits} credits"
except Exception as e:
logger.error(f"Error handling checkout completed: {e}")
return False, None, 0, str(e)
def get_session(self, session_id: str) -> Optional[Dict]:
"""
Get a checkout session by ID.
Args:
session_id: Stripe checkout session ID
Returns:
Session data or None
"""
if not self.is_configured():
return None
try:
session = stripe.checkout.Session.retrieve(session_id)
return dict(session)
except Exception as e:
logger.error(f"Error retrieving session {session_id}: {e}")
return None
def create_customer(self, email: str, name: Optional[str] = None) -> Optional[str]:
"""
Create or get a Stripe customer for a user.
Args:
email: User's email
name: User's display name (optional)
Returns:
Stripe customer ID or None
"""
if not self.is_configured():
return None
try:
# Check if customer already exists
customers = stripe.Customer.list(email=email, limit=1)
if customers.data:
return customers.data[0].id
# Create new customer
customer = stripe.Customer.create(
email=email,
name=name,
metadata={'source': 'nomad_karaoke'},
)
logger.info(f"Created Stripe customer {customer.id} for {email}")
return customer.id
except Exception as e:
logger.error(f"Error creating Stripe customer: {e}")
return None
# Global instance
_stripe_service = None
def get_stripe_service() -> StripeService:
"""Get the global Stripe service instance."""
global _stripe_service
if _stripe_service is None:
_stripe_service = StripeService()
return _stripe_service
-254
View File
@@ -1,254 +0,0 @@
"""
Structured logging with trace correlation for Cloud Logging.
This module provides JSON-formatted logging that integrates with Google Cloud Logging
and correlates logs with OpenTelemetry traces in Cloud Trace.
When running in Cloud Run:
- Logs are output as JSON for Cloud Logging to parse
- Each log entry includes trace ID and span ID for correlation
- Custom fields (job_id, worker) are preserved in the log structure
Usage:
# In main.py, before any logging:
from backend.services.structured_logging import setup_structured_logging
setup_structured_logging()
# Then use standard logging:
logger = logging.getLogger(__name__)
logger.info("Processing started", extra={"job_id": "abc123"})
"""
import json
import logging
import os
import sys
from datetime import datetime
from typing import Any, Dict, Optional
# Cloud Logging severity mapping (compatible with GCP)
# https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#LogSeverity
SEVERITY_MAP = {
"DEBUG": "DEBUG",
"INFO": "INFO",
"WARNING": "WARNING",
"ERROR": "ERROR",
"CRITICAL": "CRITICAL",
}
class StructuredFormatter(logging.Formatter):
"""
JSON formatter with trace correlation for Google Cloud Logging.
Produces log entries in the format expected by Cloud Logging, including:
- logging.googleapis.com/trace: Links to Cloud Trace
- logging.googleapis.com/spanId: Current span ID
- severity: Cloud Logging severity level
- Custom fields passed via 'extra' dict
"""
def __init__(self, project_id: Optional[str] = None):
"""
Initialize the structured formatter.
Args:
project_id: GCP project ID for trace URLs (auto-detected if not provided)
"""
super().__init__()
self.project_id = project_id or os.environ.get("GOOGLE_CLOUD_PROJECT") or os.environ.get("GCP_PROJECT")
def format(self, record: logging.LogRecord) -> str:
"""
Format a log record as JSON.
Args:
record: The log record to format
Returns:
JSON string for Cloud Logging
"""
from backend.services.tracing import get_current_trace_id, get_current_span_id
# Build base log entry
log_entry: Dict[str, Any] = {
"timestamp": datetime.utcfromtimestamp(record.created).isoformat() + "Z",
"severity": SEVERITY_MAP.get(record.levelname, record.levelname),
"message": record.getMessage(),
"logger": record.name,
}
# Add trace correlation if available
trace_id = get_current_trace_id()
span_id = get_current_span_id()
if trace_id and self.project_id:
# Format for Cloud Logging trace correlation
log_entry["logging.googleapis.com/trace"] = f"projects/{self.project_id}/traces/{trace_id}"
if span_id:
log_entry["logging.googleapis.com/spanId"] = span_id
# Add custom fields from 'extra' dict
# Common fields we want to extract from log records
custom_fields = [
# Job-related fields
"job_id", "worker", "operation", "duration", "status", "error",
# Audit logging fields (from middleware and auth)
"request_id", "user_email", "client_ip", "latency_ms",
"audit_type", "method", "path", "status_code", "query_string",
"user_agent", "user_type", "is_admin", "remaining_uses",
"auth_message", "token_provided", "token_length", "auth_header_present",
]
for field in custom_fields:
value = getattr(record, field, None)
if value is not None:
log_entry[field] = value
# Add source location for debugging
if record.pathname and record.lineno:
log_entry["logging.googleapis.com/sourceLocation"] = {
"file": record.pathname,
"line": record.lineno,
"function": record.funcName,
}
# Add exception info if present
if record.exc_info:
log_entry["exception"] = self.formatException(record.exc_info)
# Remove None values for cleaner output
return json.dumps({k: v for k, v in log_entry.items() if v is not None})
class HumanReadableFormatter(logging.Formatter):
"""
Human-readable formatter for local development.
Includes trace context when available but outputs in traditional format.
"""
def format(self, record: logging.LogRecord) -> str:
"""Format a log record for human readability."""
from backend.services.tracing import get_current_trace_id
# Build base message
timestamp = datetime.utcfromtimestamp(record.created).strftime("%Y-%m-%d %H:%M:%S")
# Add job_id prefix if available
job_id = getattr(record, "job_id", None)
job_prefix = f"[job:{job_id[:8]}] " if job_id else ""
# Add trace ID suffix if available
trace_id = get_current_trace_id()
trace_suffix = f" [trace:{trace_id[:8]}]" if trace_id else ""
message = f"{timestamp} - {record.name} - {record.levelname} - {job_prefix}{record.getMessage()}{trace_suffix}"
# Add exception info if present
if record.exc_info:
message += "\n" + self.formatException(record.exc_info)
return message
def is_cloud_run() -> bool:
"""Check if we're running in Cloud Run."""
return os.environ.get("K_SERVICE") is not None
def setup_structured_logging(force_json: bool = False, log_level: Optional[str] = None) -> None:
"""
Configure structured logging for the application.
In Cloud Run: Uses JSON format with trace correlation for Cloud Logging
In development: Uses human-readable format with optional trace context
Args:
force_json: Force JSON output even in development
log_level: Override log level (default from settings or INFO)
"""
from backend.config import settings
# Determine log level
level = log_level or getattr(settings, "log_level", "INFO")
level = getattr(logging, level.upper(), logging.INFO)
# Choose formatter based on environment
if is_cloud_run() or force_json:
formatter = StructuredFormatter()
else:
formatter = HumanReadableFormatter()
# Create handler
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(formatter)
handler.setLevel(level)
# Configure root logger
root_logger = logging.getLogger()
root_logger.setLevel(level)
# Remove existing handlers and add our handler
root_logger.handlers = [handler]
# Also configure uvicorn loggers to use our format
for logger_name in ["uvicorn", "uvicorn.access", "uvicorn.error"]:
logger = logging.getLogger(logger_name)
logger.handlers = [handler]
logger.propagate = False
class JobLogAdapter(logging.LoggerAdapter):
"""
Logger adapter that automatically adds job context to log records.
Usage:
logger = logging.getLogger(__name__)
job_logger = JobLogAdapter(logger, job_id="abc123", worker="audio")
job_logger.info("Processing started") # Automatically includes job_id and worker
"""
def __init__(self, logger: logging.Logger, job_id: str, worker: Optional[str] = None, **extra):
"""
Initialize the adapter.
Args:
logger: Base logger instance
job_id: Job ID to include in all log records
worker: Optional worker name
**extra: Additional fields to include in all log records
"""
super().__init__(logger, {"job_id": job_id, "worker": worker, **extra})
def process(self, msg: str, kwargs: Dict[str, Any]) -> tuple:
"""Process a logging call to add extra context."""
# Merge extra dict from adapter with any extra passed to the log call
extra = kwargs.get("extra", {})
extra.update(self.extra)
kwargs["extra"] = extra
return msg, kwargs
def get_job_logger(job_id: str, worker: Optional[str] = None, name: Optional[str] = None) -> JobLogAdapter:
"""
Get a logger adapter configured for a specific job.
This is a convenience function for creating job-specific loggers.
Args:
job_id: Job ID to include in all log records
worker: Optional worker name
name: Logger name (defaults to "job")
Returns:
JobLogAdapter configured for the job
Usage:
logger = get_job_logger("abc123", "audio")
logger.info("Starting audio separation")
logger.error("Failed to process", extra={"error": str(e)})
"""
base_logger = logging.getLogger(name or "job")
return JobLogAdapter(base_logger, job_id=job_id, worker=worker)
-330
View File
@@ -1,330 +0,0 @@
"""
Template service for managing email templates stored in GCS.
Templates are stored in GCS bucket and rendered with job-specific variables.
This allows updating email content without code deployment.
Template locations:
- gs://{bucket}/templates/job-completion.txt - Job completion email (plain text)
- gs://{bucket}/templates/action-needed-lyrics.txt - Lyrics review reminder
- gs://{bucket}/templates/action-needed-instrumental.txt - Instrumental selection reminder
"""
import logging
import re
from typing import Optional, Dict, Any
from backend.config import get_settings
logger = logging.getLogger(__name__)
# Default templates (fallback if GCS fetch fails)
DEFAULT_JOB_COMPLETION_TEMPLATE = """Hi {name},
Thanks for your order!
Here's the link for the karaoke video published to YouTube:
{youtube_url}
Here's the dropbox folder with all the finished files and source files, including:
- "(Final Karaoke Lossless).mkv": combined karaoke video in 4k H264 with lossless FLAC audio
- "(Final Karaoke).mp4": combined karaoke video with title/end screen in 4k H264/AAC
- "(Final Karaoke 720p).mp4": combined karaoke video in 720p H264/AAC (smaller file for older systems)
- "(With Vocals).mp4": sing along video in 4k H264/AAC with original vocals
- "(Karaoke).mov": karaoke video output from MidiCo (no title/end screen)
- "(Title).mov"/"(End).mov": title card and end screen videos
- "(Final Karaoke CDG).zip": CDG+MP3 format for older/commercial karaoke systems
- "(Final Karaoke TXT).zip": TXT+MP3 format for Power Karaoke
- stems/*.flac: various separated instrumental and vocal audio stems in lossless format
- lyrics/*.txt song lyrics from various sources in plain text format
{dropbox_url}
Let me know if anything isn't perfect and I'll happily tweak / fix, or if you need it in any other format I can probably convert it for you!
If you have a moment, I'd really appreciate your feedback (takes 2 minutes):
{feedback_url}
Thanks again and have a great day!
-Andrew
"""
DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE = """Hi {name},
Your karaoke video for "{artist} - {title}" is ready for lyrics review!
Our system has transcribed and synchronized the lyrics, but they may need some corrections. Please review and make any corrections necessary so we can finish generating your video.
Review your lyrics here:
{review_url}
This usually takes just a few minutes.
Thanks!
-Andrew
"""
DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE = """Hi {name},
Your karaoke video for "{artist} - {title}" is almost done!
We've separated the audio into different versions. Please select which instrumental track you'd like to use for the final video.
Select your instrumental here:
{instrumental_url}
Thanks!
-Andrew
"""
class TemplateService:
"""
Service for fetching and rendering email templates from GCS.
Templates support the following variables:
- {name} - User's display name or "there" if unknown
- {youtube_url} - YouTube video URL
- {dropbox_url} - Dropbox folder URL
- {artist} - Artist name
- {title} - Song title
- {job_id} - Job ID
- {review_url} - Lyrics review URL
- {instrumental_url} - Instrumental selection URL
- {feedback_url} - Feedback form URL
"""
TEMPLATE_PREFIX = "templates/"
def __init__(self):
"""Initialize template service."""
self.settings = get_settings()
self._storage_client = None
self._bucket = None
@property
def storage_client(self):
"""Lazy-initialize storage client."""
if self._storage_client is None:
from google.cloud import storage
self._storage_client = storage.Client(project=self.settings.google_cloud_project)
return self._storage_client
@property
def bucket(self):
"""Get the GCS bucket for templates."""
if self._bucket is None:
self._bucket = self.storage_client.bucket(self.settings.gcs_bucket_name)
return self._bucket
def _fetch_template_from_gcs(self, template_name: str) -> Optional[str]:
"""
Fetch a template from GCS.
Args:
template_name: Name of template file (e.g., "job-completion.txt")
Returns:
Template content as string, or None if not found
"""
blob_path = f"{self.TEMPLATE_PREFIX}{template_name}"
try:
blob = self.bucket.blob(blob_path)
if blob.exists():
content = blob.download_as_text()
logger.debug(f"Fetched template from GCS: {blob_path}")
return content
else:
logger.warning(f"Template not found in GCS: {blob_path}")
return None
except Exception as e:
logger.error(f"Failed to fetch template {blob_path}: {e}")
return None
def get_job_completion_template(self) -> str:
"""Get the job completion email template."""
template = self._fetch_template_from_gcs("job-completion.txt")
if template is None:
logger.info("Using default job completion template")
return DEFAULT_JOB_COMPLETION_TEMPLATE
return template
def get_action_needed_lyrics_template(self) -> str:
"""Get the lyrics review reminder template."""
template = self._fetch_template_from_gcs("action-needed-lyrics.txt")
if template is None:
logger.info("Using default lyrics reminder template")
return DEFAULT_ACTION_NEEDED_LYRICS_TEMPLATE
return template
def get_action_needed_instrumental_template(self) -> str:
"""Get the instrumental selection reminder template."""
template = self._fetch_template_from_gcs("action-needed-instrumental.txt")
if template is None:
logger.info("Using default instrumental reminder template")
return DEFAULT_ACTION_NEEDED_INSTRUMENTAL_TEMPLATE
return template
def render_template(self, template: str, variables: Dict[str, Any]) -> str:
"""
Render a template with the given variables.
Missing variables are replaced with empty strings.
Handles conditional sections like feedback URL.
Args:
template: Template string with {variable} placeholders
variables: Dictionary of variable values
Returns:
Rendered template string
"""
result = template
# Handle feedback URL section - remove if not provided
if not variables.get("feedback_url"):
# Remove feedback section (lines containing feedback_url placeholder and surrounding text)
result = re.sub(
r'\n*If you have a moment.*?\{feedback_url\}\n*',
'\n',
result,
flags=re.DOTALL
)
variables["feedback_url"] = ""
# Replace all variables
for key, value in variables.items():
placeholder = "{" + key + "}"
result = result.replace(placeholder, str(value) if value else "")
# Clean up any remaining unreplaced placeholders
result = re.sub(r'\{[a-z_]+\}', '', result)
return result.strip()
def render_job_completion(
self,
name: Optional[str] = None,
youtube_url: Optional[str] = None,
dropbox_url: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
job_id: Optional[str] = None,
feedback_url: Optional[str] = None,
) -> str:
"""
Render the job completion email template.
Args:
name: User's display name (defaults to "there")
youtube_url: YouTube video URL
dropbox_url: Dropbox folder URL
artist: Artist name
title: Song title
job_id: Job ID
feedback_url: Feedback form URL (optional)
Returns:
Rendered email content
"""
template = self.get_job_completion_template()
variables = {
"name": name or "there",
"youtube_url": youtube_url or "[YouTube URL not available]",
"dropbox_url": dropbox_url or "[Dropbox URL not available]",
"artist": artist or "Unknown Artist",
"title": title or "Unknown Title",
"job_id": job_id or "",
"feedback_url": feedback_url,
}
return self.render_template(template, variables)
def render_action_needed_lyrics(
self,
name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
review_url: str = "",
) -> str:
"""
Render the lyrics review reminder template.
Args:
name: User's display name
artist: Artist name
title: Song title
review_url: Lyrics review URL
Returns:
Rendered email content
"""
template = self.get_action_needed_lyrics_template()
variables = {
"name": name or "there",
"artist": artist or "Unknown Artist",
"title": title or "Unknown Title",
"review_url": review_url,
}
return self.render_template(template, variables)
def render_action_needed_instrumental(
self,
name: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
instrumental_url: str = "",
) -> str:
"""
Render the instrumental selection reminder template.
Args:
name: User's display name
artist: Artist name
title: Song title
instrumental_url: Instrumental selection URL
Returns:
Rendered email content
"""
template = self.get_action_needed_instrumental_template()
variables = {
"name": name or "there",
"artist": artist or "Unknown Artist",
"title": title or "Unknown Title",
"instrumental_url": instrumental_url,
}
return self.render_template(template, variables)
def upload_template(self, template_name: str, content: str) -> bool:
"""
Upload a template to GCS.
Args:
template_name: Name of template file (e.g., "job-completion.txt")
content: Template content
Returns:
True if upload successful
"""
blob_path = f"{self.TEMPLATE_PREFIX}{template_name}"
try:
blob = self.bucket.blob(blob_path)
blob.upload_from_string(content, content_type="text/plain")
logger.info(f"Uploaded template to GCS: {blob_path}")
return True
except Exception as e:
logger.error(f"Failed to upload template {blob_path}: {e}")
return False
# Global instance
_template_service: Optional[TemplateService] = None
def get_template_service() -> TemplateService:
"""Get the global template service instance."""
global _template_service
if _template_service is None:
_template_service = TemplateService()
return _template_service
-285
View File
@@ -1,285 +0,0 @@
"""
Tenant service for managing white-label B2B portal configurations.
Tenant configs are stored in GCS at tenants/{tenant_id}/config.json.
The service provides:
- Tenant config loading with caching
- Subdomain to tenant ID resolution
- Tenant validation
Similar to ThemeService, configs are cached for 5 minutes to reduce GCS reads.
"""
import logging
import threading
from datetime import datetime
from typing import Dict, Optional
from backend.models.tenant import TenantConfig, TenantPublicConfig
from backend.services.storage_service import StorageService
logger = logging.getLogger(__name__)
# GCS paths for tenant configs
TENANTS_PREFIX = "tenants"
# Default sender email (shared with EmailService for consistency)
DEFAULT_SENDER_EMAIL = "gen@nomadkaraoke.com"
class TenantService:
"""Service for managing tenant configurations from GCS."""
def __init__(self, storage: Optional[StorageService] = None):
"""
Initialize the tenant service.
Args:
storage: StorageService instance (creates new one if not provided)
"""
self.storage = storage or StorageService()
self._config_cache: Dict[str, TenantConfig] = {}
self._cache_times: Dict[str, datetime] = {}
self._subdomain_map: Dict[str, str] = {} # subdomain -> tenant_id
self._subdomain_map_time: Optional[datetime] = None
self.CACHE_TTL_SECONDS = 300 # 5 minute cache
def _get_config_path(self, tenant_id: str) -> str:
"""Get the GCS path for a tenant's config file."""
return f"{TENANTS_PREFIX}/{tenant_id}/config.json"
def _is_cache_valid(self, tenant_id: str) -> bool:
"""Check if the config cache for a tenant is still valid."""
if tenant_id not in self._config_cache or tenant_id not in self._cache_times:
return False
age = datetime.now() - self._cache_times[tenant_id]
return age.total_seconds() < self.CACHE_TTL_SECONDS
def _is_subdomain_map_valid(self) -> bool:
"""Check if the subdomain map cache is still valid."""
if not self._subdomain_map or self._subdomain_map_time is None:
return False
age = datetime.now() - self._subdomain_map_time
return age.total_seconds() < self.CACHE_TTL_SECONDS
def get_tenant_config(
self, tenant_id: str, force_refresh: bool = False
) -> Optional[TenantConfig]:
"""
Load tenant configuration from GCS with caching.
Args:
tenant_id: The tenant identifier
force_refresh: Force reload from GCS even if cache is valid
Returns:
TenantConfig if found, None otherwise
"""
if not force_refresh and self._is_cache_valid(tenant_id):
return self._config_cache[tenant_id]
try:
config_path = self._get_config_path(tenant_id)
if not self.storage.file_exists(config_path):
logger.debug(f"Tenant config not found: {tenant_id}")
return None
data = self.storage.download_json(config_path)
config = TenantConfig(**data)
# Update cache
self._config_cache[tenant_id] = config
self._cache_times[tenant_id] = datetime.now()
# Update subdomain map and its timestamp
self._subdomain_map[config.subdomain.lower()] = tenant_id
self._subdomain_map_time = datetime.now()
logger.info(f"Loaded tenant config: {tenant_id}")
return config
except Exception as e:
logger.error(f"Failed to load tenant config {tenant_id}: {e}")
return None
def get_tenant_by_subdomain(self, subdomain: str) -> Optional[TenantConfig]:
"""
Get tenant config by subdomain.
Args:
subdomain: Full subdomain (e.g., 'vocalstar.nomadkaraoke.com')
Returns:
TenantConfig if found, None otherwise
"""
subdomain_lower = subdomain.lower()
# Check cache first
if subdomain_lower in self._subdomain_map:
tenant_id = self._subdomain_map[subdomain_lower]
return self.get_tenant_config(tenant_id)
# Try to find tenant by listing all configs
# This is a fallback for when we don't have the subdomain cached
tenant_id = self._resolve_subdomain_to_tenant_id(subdomain_lower)
if tenant_id:
return self.get_tenant_config(tenant_id)
return None
def _resolve_subdomain_to_tenant_id(self, subdomain: str) -> Optional[str]:
"""
Resolve a subdomain to tenant ID by checking GCS.
This extracts the tenant ID from the subdomain pattern:
- 'vocalstar.nomadkaraoke.com' -> 'vocalstar'
- 'vocalstar.gen.nomadkaraoke.com' -> 'vocalstar'
"""
# Extract potential tenant ID from subdomain
# Pattern: {tenant}.nomadkaraoke.com or {tenant}.gen.nomadkaraoke.com
parts = subdomain.split(".")
if len(parts) >= 3:
# First part is likely the tenant ID
potential_tenant_id = parts[0]
# Check if config exists
config_path = self._get_config_path(potential_tenant_id)
if self.storage.file_exists(config_path):
return potential_tenant_id
return None
def get_public_config(self, tenant_id: str) -> Optional[TenantPublicConfig]:
"""
Get the public (frontend-safe) tenant configuration.
Args:
tenant_id: The tenant identifier
Returns:
TenantPublicConfig if found, None otherwise
"""
config = self.get_tenant_config(tenant_id)
if config:
return TenantPublicConfig.from_config(config)
return None
def get_public_config_by_subdomain(
self, subdomain: str
) -> Optional[TenantPublicConfig]:
"""
Get the public tenant configuration by subdomain.
Args:
subdomain: Full subdomain
Returns:
TenantPublicConfig if found, None otherwise
"""
config = self.get_tenant_by_subdomain(subdomain)
if config:
return TenantPublicConfig.from_config(config)
return None
def tenant_exists(self, tenant_id: str) -> bool:
"""
Check if a tenant exists.
Args:
tenant_id: The tenant identifier
Returns:
True if tenant config exists, False otherwise
"""
config_path = self._get_config_path(tenant_id)
return self.storage.file_exists(config_path)
def is_email_allowed_for_tenant(self, tenant_id: str, email: str) -> bool:
"""
Check if an email is allowed for a specific tenant.
Args:
tenant_id: The tenant identifier
email: Email address to check
Returns:
True if email is allowed, False otherwise
"""
config = self.get_tenant_config(tenant_id)
if not config:
return False
return config.is_email_allowed(email)
def get_tenant_sender_email(self, tenant_id: str) -> str:
"""
Get the email sender address for a tenant.
Args:
tenant_id: The tenant identifier
Returns:
Sender email address
"""
config = self.get_tenant_config(tenant_id)
if config:
return config.get_sender_email()
# Default fallback (consistent with EmailService)
return DEFAULT_SENDER_EMAIL
def invalidate_cache(self, tenant_id: Optional[str] = None) -> None:
"""
Invalidate tenant config cache.
Args:
tenant_id: Specific tenant to invalidate, or None for all
"""
if tenant_id:
self._config_cache.pop(tenant_id, None)
self._cache_times.pop(tenant_id, None)
# Also remove any subdomain map entries pointing to this tenant
subdomains_to_remove = [
subdomain
for subdomain, tid in self._subdomain_map.items()
if tid == tenant_id
]
for subdomain in subdomains_to_remove:
self._subdomain_map.pop(subdomain, None)
logger.info(f"Tenant config cache invalidated: {tenant_id}")
else:
self._config_cache.clear()
self._cache_times.clear()
self._subdomain_map.clear()
self._subdomain_map_time = None
logger.info("All tenant config caches invalidated")
def get_asset_url(self, tenant_id: str, asset_name: str) -> Optional[str]:
"""
Get a signed URL for a tenant asset (logo, favicon, etc.).
Args:
tenant_id: The tenant identifier
asset_name: Asset filename (e.g., 'logo.png')
Returns:
Signed URL if asset exists, None otherwise
"""
asset_path = f"{TENANTS_PREFIX}/{tenant_id}/{asset_name}"
if self.storage.file_exists(asset_path):
return self.storage.generate_signed_url(asset_path, expiration_minutes=60)
return None
# Singleton instance with thread-safe initialization
_tenant_service: Optional[TenantService] = None
_tenant_service_lock = threading.Lock()
def get_tenant_service() -> TenantService:
"""Get or create the singleton TenantService instance (thread-safe)."""
global _tenant_service
if _tenant_service is None:
with _tenant_service_lock:
# Double-check after acquiring lock
if _tenant_service is None:
_tenant_service = TenantService()
return _tenant_service
-469
View File
@@ -1,469 +0,0 @@
"""
Theme service for managing pre-created style themes.
Themes are stored in GCS at themes/{theme_id}/ with:
- style_params.json: Complete style configuration
- preview.png: Preview image for theme selection UI
- youtube_description.txt: Optional YouTube description template
- assets/: Fonts, backgrounds, and other assets
The service provides:
- Theme listing with signed preview URLs
- Theme detail retrieval
- Color override application
- Job style preparation (copying theme to job folder)
"""
import copy
import json
import logging
import os
import tempfile
import threading
from datetime import datetime, timedelta
from typing import Any, Dict, List, Optional, Tuple
from backend.models.theme import (
ColorOverrides,
ThemeDetail,
ThemeMetadata,
ThemeRegistry,
ThemeSummary,
hex_to_rgba,
)
from backend.services.storage_service import StorageService
logger = logging.getLogger(__name__)
# GCS paths for themes
THEMES_PREFIX = "themes"
METADATA_FILE = f"{THEMES_PREFIX}/_metadata.json"
class ThemeService:
"""Service for managing themes from GCS."""
def __init__(self, storage: Optional[StorageService] = None):
"""
Initialize the theme service.
Args:
storage: StorageService instance (creates new one if not provided)
"""
self.storage = storage or StorageService()
self._metadata_cache: Optional[ThemeRegistry] = None
self._cache_time: Optional[datetime] = None
self.CACHE_TTL_SECONDS = 300 # 5 minute cache for theme metadata
def _get_theme_path(self, theme_id: str, filename: str = "") -> str:
"""Get the GCS path for a theme file."""
if filename:
return f"{THEMES_PREFIX}/{theme_id}/{filename}"
return f"{THEMES_PREFIX}/{theme_id}"
def _is_cache_valid(self) -> bool:
"""Check if the metadata cache is still valid."""
if self._metadata_cache is None or self._cache_time is None:
return False
age = datetime.now() - self._cache_time
return age.total_seconds() < self.CACHE_TTL_SECONDS
def _load_metadata(self, force_refresh: bool = False) -> ThemeRegistry:
"""
Load theme metadata from GCS with caching.
Args:
force_refresh: Force reload from GCS even if cache is valid
Returns:
ThemeRegistry containing all theme metadata
"""
if not force_refresh and self._is_cache_valid():
return self._metadata_cache # type: ignore
try:
data = self.storage.download_json(METADATA_FILE)
self._metadata_cache = ThemeRegistry(**data)
self._cache_time = datetime.now()
logger.info(f"Loaded theme metadata: {len(self._metadata_cache.themes)} themes")
return self._metadata_cache
except Exception as e:
logger.error(f"Failed to load theme metadata: {e}")
# Return empty registry on error
return ThemeRegistry(version=1, themes=[])
def list_themes(self) -> List[ThemeSummary]:
"""
List all available themes with signed preview URLs.
Returns:
List of ThemeSummary objects with preview URLs
"""
metadata = self._load_metadata()
summaries = []
for theme in metadata.themes:
try:
# Generate signed URLs for preview images
preview_path = self._get_theme_path(theme.id, "preview.png")
thumbnail_path = self._get_theme_path(theme.id, "preview_thumbnail.png")
preview_url = None
thumbnail_url = None
if self.storage.file_exists(preview_path):
preview_url = self.storage.generate_signed_url(preview_path, expiration_minutes=60)
# Try thumbnail, fall back to main preview
if self.storage.file_exists(thumbnail_path):
thumbnail_url = self.storage.generate_signed_url(thumbnail_path, expiration_minutes=60)
elif preview_url:
thumbnail_url = preview_url
summaries.append(
ThemeSummary(
id=theme.id,
name=theme.name,
description=theme.description,
preview_url=preview_url,
thumbnail_url=thumbnail_url,
is_default=theme.is_default,
)
)
except Exception as e:
logger.warning(f"Error processing theme {theme.id}: {e}")
# Still include theme but without preview URLs
summaries.append(
ThemeSummary(
id=theme.id,
name=theme.name,
description=theme.description,
is_default=theme.is_default,
)
)
return summaries
def get_theme(self, theme_id: str) -> Optional[ThemeDetail]:
"""
Get full theme details including style parameters.
Args:
theme_id: The theme identifier
Returns:
ThemeDetail if found, None otherwise
"""
metadata = self._load_metadata()
# Find theme in metadata
theme_meta = next((t for t in metadata.themes if t.id == theme_id), None)
if not theme_meta:
logger.warning(f"Theme not found: {theme_id}")
return None
try:
# Load style_params.json
style_params_path = self._get_theme_path(theme_id, "style_params.json")
style_params = self.storage.download_json(style_params_path)
# Check for YouTube description
youtube_desc_path = self._get_theme_path(theme_id, "youtube_description.txt")
has_youtube_desc = self.storage.file_exists(youtube_desc_path)
# Generate preview URL
preview_path = self._get_theme_path(theme_id, "preview.png")
preview_url = None
if self.storage.file_exists(preview_path):
preview_url = self.storage.generate_signed_url(preview_path, expiration_minutes=60)
return ThemeDetail(
id=theme_id,
name=theme_meta.name,
description=theme_meta.description,
preview_url=preview_url,
is_default=theme_meta.is_default,
style_params=style_params,
has_youtube_description=has_youtube_desc,
)
except Exception as e:
logger.error(f"Error loading theme {theme_id}: {e}")
return None
def get_theme_style_params(self, theme_id: str) -> Optional[Dict[str, Any]]:
"""
Get just the style_params.json for a theme.
Args:
theme_id: The theme identifier
Returns:
Style parameters dict if found, None otherwise
"""
try:
style_params_path = self._get_theme_path(theme_id, "style_params.json")
return self.storage.download_json(style_params_path)
except Exception as e:
logger.error(f"Error loading style params for theme {theme_id}: {e}")
return None
def get_youtube_description(self, theme_id: str) -> Optional[str]:
"""
Get the YouTube description template for a theme.
Args:
theme_id: The theme identifier
Returns:
YouTube description text if found, None otherwise
"""
try:
youtube_desc_path = self._get_theme_path(theme_id, "youtube_description.txt")
if not self.storage.file_exists(youtube_desc_path):
return None
# Download to temp file and read
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as tmp:
tmp_path = tmp.name
try:
self.storage.download_file(youtube_desc_path, tmp_path)
with open(tmp_path, "r") as f:
return f.read()
finally:
if os.path.exists(tmp_path):
os.remove(tmp_path)
except Exception as e:
logger.error(f"Error loading YouTube description for theme {theme_id}: {e}")
return None
def apply_color_overrides(
self, style_params: Dict[str, Any], overrides: ColorOverrides
) -> Dict[str, Any]:
"""
Apply user color overrides to theme style parameters.
Args:
style_params: Base style parameters from theme
overrides: User color overrides
Returns:
Modified style parameters with overrides applied
"""
if not overrides.has_overrides():
return style_params
# Deep copy to avoid modifying original
result = copy.deepcopy(style_params)
# Apply artist_color to intro, end, and CDG sections
if overrides.artist_color:
if "intro" in result:
result["intro"]["artist_color"] = overrides.artist_color
if "end" in result:
result["end"]["artist_color"] = overrides.artist_color
if "cdg" in result:
result["cdg"]["artist_color"] = overrides.artist_color
# Apply title_color to intro, end, and CDG sections
if overrides.title_color:
if "intro" in result:
result["intro"]["title_color"] = overrides.title_color
if "end" in result:
result["end"]["title_color"] = overrides.title_color
if "cdg" in result:
result["cdg"]["title_color"] = overrides.title_color
# Apply sung_lyrics_color (karaoke uses RGBA, CDG uses hex)
if overrides.sung_lyrics_color:
if "karaoke" in result:
result["karaoke"]["primary_color"] = hex_to_rgba(overrides.sung_lyrics_color)
if "cdg" in result:
result["cdg"]["active_fill"] = overrides.sung_lyrics_color
# Apply unsung_lyrics_color (karaoke uses RGBA, CDG uses hex)
if overrides.unsung_lyrics_color:
if "karaoke" in result:
result["karaoke"]["secondary_color"] = hex_to_rgba(overrides.unsung_lyrics_color)
if "cdg" in result:
result["cdg"]["inactive_fill"] = overrides.unsung_lyrics_color
return result
def prepare_job_style(
self,
job_id: str,
theme_id: str,
color_overrides: Optional[ColorOverrides] = None,
) -> Tuple[str, Dict[str, str]]:
"""
Prepare style files for a job by copying theme to job folder.
This method:
1. Loads the theme's style_params.json
2. Applies any color overrides
3. Updates asset paths to point to theme assets (shared, not copied)
4. Uploads modified style_params.json to job folder
5. Returns the GCS path and style_assets mapping
Args:
job_id: The job ID
theme_id: The theme to use
color_overrides: Optional color overrides
Returns:
Tuple of (style_params_gcs_path, style_assets dict)
Raises:
ValueError: If theme not found
"""
# Load theme style params
style_params = self.get_theme_style_params(theme_id)
if style_params is None:
raise ValueError(f"Theme not found: {theme_id}")
# Apply color overrides if provided
if color_overrides:
style_params = self.apply_color_overrides(style_params, color_overrides)
# Update asset paths to point to theme's shared assets
# Theme assets stay in themes/{theme_id}/assets/ - they're shared
style_assets = self._build_style_assets_mapping(theme_id, style_params)
# Update style_params to use theme asset paths (not job-specific)
style_params = self._update_asset_paths_in_style(theme_id, style_params)
# Upload modified style_params.json to job's style folder
job_style_path = f"uploads/{job_id}/style/style_params.json"
self.storage.upload_json(job_style_path, style_params)
logger.info(f"Prepared job {job_id} style from theme {theme_id}")
return job_style_path, style_assets
def _build_style_assets_mapping(
self, theme_id: str, style_params: Dict[str, Any]
) -> Dict[str, str]:
"""
Build the style_assets mapping for a theme.
Maps asset keys to their GCS paths in the theme folder.
"""
style_assets = {}
theme_assets_prefix = f"{THEMES_PREFIX}/{theme_id}/assets"
# Check each potential asset type
asset_checks = [
("intro_background", "intro", "background_image"),
("karaoke_background", "karaoke", "background_image"),
("end_background", "end", "background_image"),
("font", "intro", "font"), # Font is typically shared across sections
("cdg_instrumental_background", "cdg", "instrumental_background"),
("cdg_title_background", "cdg", "title_screen_background"),
("cdg_outro_background", "cdg", "outro_background"),
]
for asset_key, section, field in asset_checks:
if section in style_params and field in style_params[section]:
value = style_params[section][field]
if value and isinstance(value, str):
# Check if it's a theme asset path or already has the full path
if value.startswith(f"{THEMES_PREFIX}/"):
# Already a full theme path
style_assets[asset_key] = value
elif not value.startswith("/") and not value.startswith("gs://"):
# Relative path - prepend theme assets prefix
asset_path = f"{theme_assets_prefix}/{os.path.basename(value)}"
if self.storage.file_exists(asset_path):
style_assets[asset_key] = asset_path
return style_assets
def _update_asset_paths_in_style(
self, theme_id: str, style_params: Dict[str, Any]
) -> Dict[str, Any]:
"""
Update asset paths in style_params to use full GCS theme paths.
This ensures the style_loader can find assets when processing.
"""
result = copy.deepcopy(style_params)
theme_assets_prefix = f"{THEMES_PREFIX}/{theme_id}/assets"
# Fields that contain asset paths
path_fields = {
"intro": ["background_image", "font"],
"end": ["background_image", "font"],
"karaoke": ["background_image", "font_path"],
"cdg": [
"font_path",
"instrumental_background",
"title_screen_background",
"outro_background",
],
}
for section, fields in path_fields.items():
if section not in result:
continue
for field in fields:
if field not in result[section]:
continue
value = result[section][field]
if value and isinstance(value, str):
# Skip if already a full path or URL
if value.startswith(f"{THEMES_PREFIX}/") or value.startswith("gs://"):
continue
# Skip absolute local paths (shouldn't happen but be safe)
if value.startswith("/"):
continue
# Update to full theme asset path
result[section][field] = f"{theme_assets_prefix}/{os.path.basename(value)}"
return result
def theme_exists(self, theme_id: str) -> bool:
"""
Check if a theme exists.
Args:
theme_id: The theme identifier
Returns:
True if theme exists, False otherwise
"""
metadata = self._load_metadata()
return any(t.id == theme_id for t in metadata.themes)
def get_default_theme_id(self) -> Optional[str]:
"""
Get the ID of the default theme.
Returns:
Default theme ID if one exists, None otherwise
"""
metadata = self._load_metadata()
default_theme = next((t for t in metadata.themes if t.is_default), None)
return default_theme.id if default_theme else None
def invalidate_cache(self) -> None:
"""Force invalidation of the metadata cache."""
self._metadata_cache = None
self._cache_time = None
logger.info("Theme metadata cache invalidated")
# Singleton instance with thread-safe initialization
_theme_service: Optional[ThemeService] = None
_theme_service_lock = threading.Lock()
def get_theme_service() -> ThemeService:
"""Get or create the singleton ThemeService instance (thread-safe)."""
global _theme_service
if _theme_service is None:
with _theme_service_lock:
# Double-check after acquiring lock
if _theme_service is None:
_theme_service = ThemeService()
return _theme_service
-161
View File
@@ -19,25 +19,6 @@ Usage:
with tracer.start_as_current_span("my-operation") as span:
span.set_attribute("job_id", job_id)
# ... do work ...
# For job-specific spans with automatic job_id attribute
from backend.services.tracing import job_span
with job_span("audio-worker", job_id) as span:
# job_id is automatically set as an attribute
span.set_attribute("stage", "separation")
# ... do work ...
# For propagating trace context through Cloud Tasks
from backend.services.tracing import inject_trace_context, extract_trace_context
# In worker_service.py (when creating task):
headers = inject_trace_context({})
# In internal.py (when receiving task):
context = extract_trace_context(request.headers)
with tracer.start_as_current_span("worker", context=context):
...
"""
import os
import logging
@@ -399,145 +380,3 @@ def get_current_span_id() -> Optional[str]:
if ctx.is_valid:
return format(ctx.span_id, '016x')
return None
# =====================================================
# Job-Aware Tracing Utilities
# =====================================================
@contextmanager
def job_span(
name: str,
job_id: str,
attributes: Optional[Dict[str, Any]] = None,
kind: trace.SpanKind = trace.SpanKind.INTERNAL,
):
"""
Create a traced span with job_id automatically set as an attribute.
This is the preferred way to create spans in job processing workers.
The job_id attribute enables filtering spans by job in Cloud Trace.
Usage:
with job_span("audio-separation", job_id) as span:
span.set_attribute("stage", "clean_instrumental")
# ... do work ...
Args:
name: Span name (appears in Cloud Trace)
job_id: Job ID to attach to the span
attributes: Additional span attributes
kind: Span kind (INTERNAL, SERVER, CLIENT, PRODUCER, CONSUMER)
Yields:
The active span
"""
all_attributes = {
"job_id": job_id,
"service.operation": name,
}
if attributes:
all_attributes.update(attributes)
with create_span(name, all_attributes, kind) as span:
yield span
def inject_trace_context(headers: Dict[str, str]) -> Dict[str, str]:
"""
Inject current trace context into HTTP headers.
Use this when creating Cloud Tasks or making HTTP calls to propagate
the trace context so child spans link to the parent trace.
Usage:
headers = inject_trace_context({
"Content-Type": "application/json",
"Authorization": "Bearer token",
})
# Use headers in HTTP request or Cloud Task
Args:
headers: Existing headers dict to add trace context to
Returns:
Headers dict with traceparent/tracestate headers added
"""
propagator = TraceContextTextMapPropagator()
carrier = dict(headers) # Make a copy
propagator.inject(carrier)
return carrier
def extract_trace_context(headers: Dict[str, str]) -> Optional[trace.Context]:
"""
Extract trace context from HTTP headers.
Use this when receiving Cloud Tasks or HTTP requests to continue
the trace from the parent span.
Usage:
context = extract_trace_context(request.headers)
with tracer.start_as_current_span("worker", context=context):
# ... processing ...
Args:
headers: HTTP headers containing traceparent/tracestate
Returns:
Context object for use with start_as_current_span, or None
"""
propagator = TraceContextTextMapPropagator()
return propagator.extract(headers)
def start_span_with_context(
name: str,
context: Optional[trace.Context],
job_id: Optional[str] = None,
attributes: Optional[Dict[str, Any]] = None,
):
"""
Start a span as child of the given context (or current context if None).
This is useful for starting worker spans that should link to the
trace from the original API request.
Usage:
context = extract_trace_context(request.headers)
with start_span_with_context("audio-worker", context, job_id) as span:
# ... processing ...
Args:
name: Span name
context: Parent context (from extract_trace_context) or None
job_id: Optional job ID to set as attribute
attributes: Additional span attributes
Returns:
Context manager yielding the span
"""
tracer = get_tracer()
all_attributes = attributes or {}
if job_id:
all_attributes["job_id"] = job_id
all_attributes["service.operation"] = name
@contextmanager
def _span():
kwargs = {"kind": trace.SpanKind.INTERNAL}
if context is not None:
kwargs["context"] = context
with tracer.start_as_current_span(name, **kwargs) as span:
for key, value in all_attributes.items():
span.set_attribute(key, value)
try:
yield span
except Exception as e:
span.set_status(Status(StatusCode.ERROR, str(e)))
span.record_exception(e)
raise
return _span()
-799
View File
@@ -1,799 +0,0 @@
"""
User service for authentication, credits, and user management.
Handles:
- Magic link authentication (send, verify)
- Session management
- Credit operations (add, deduct, refund)
- User CRUD operations
"""
import logging
import secrets
import uuid
from datetime import datetime, timedelta, timezone
from typing import Optional, List, Tuple
from google.cloud import firestore
from google.cloud.firestore_v1 import FieldFilter
from google.cloud.firestore_v1 import Increment
from backend.config import get_settings
from backend.models.user import (
User,
UserRole,
UserPublic,
MagicLinkToken,
Session,
CreditTransaction,
)
logger = logging.getLogger(__name__)
# Collection names
USERS_COLLECTION = "gen_users"
MAGIC_LINKS_COLLECTION = "magic_links"
SESSIONS_COLLECTION = "sessions"
PROCESSED_STRIPE_SESSIONS_COLLECTION = "processed_stripe_sessions"
# Token/session configuration
MAGIC_LINK_EXPIRY_MINUTES = 15
SESSION_EXPIRY_DAYS = 7
SESSION_ABSOLUTE_EXPIRY_DAYS = 30
MAX_CREDIT_TRANSACTIONS = 100 # Keep last N transactions
class UserService:
"""Service for user management and authentication."""
# Number of free credits granted to new users
NEW_USER_FREE_CREDITS = 1
def __init__(self):
"""Initialize user service with Firestore client."""
self.settings = get_settings()
self.db = firestore.Client(project=self.settings.google_cloud_project)
# =========================================================================
# User CRUD Operations
# =========================================================================
def get_user(self, email: str) -> Optional[User]:
"""Get a user by email."""
try:
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
doc = doc_ref.get()
if not doc.exists:
return None
return User(**doc.to_dict())
except Exception:
logger.exception(f"Error getting user {email}")
return None
def get_or_create_user(self, email: str, tenant_id: Optional[str] = None) -> User:
"""
Get existing user or create a new one.
New users receive a welcome credit to try the service.
Args:
email: User's email address
tenant_id: Tenant ID for white-label portals (None = default Nomad Karaoke)
Note: If user exists but has a different tenant_id, the existing user is returned.
Users are uniquely identified by email, not email+tenant.
"""
email = email.lower()
user = self.get_user(email)
if user:
# If user exists but tenant_id differs, we still return the user
# This allows users to access multiple tenants with one account
# (though features may be restricted per-tenant)
return user
# Create new user with welcome credit
welcome_credit = self.NEW_USER_FREE_CREDITS
welcome_transaction = CreditTransaction(
id=str(uuid.uuid4()),
amount=welcome_credit,
reason="welcome_credit",
)
user = User(
email=email,
credits=welcome_credit,
credit_transactions=[welcome_transaction],
tenant_id=tenant_id, # Associate with tenant on creation
)
self._save_user(user)
logger.info(f"Created new user: {email} with {welcome_credit} welcome credit(s) (tenant: {tenant_id or 'default'})")
return user
def _save_user(self, user: User) -> None:
"""Save user to Firestore."""
try:
user.updated_at = datetime.utcnow()
doc_ref = self.db.collection(USERS_COLLECTION).document(user.email.lower())
doc_ref.set(user.model_dump(mode='json'))
except Exception:
logger.exception(f"Error saving user {user.email}")
raise
def update_user(self, email: str, **updates) -> Optional[User]:
"""Update user fields."""
try:
updates['updated_at'] = datetime.utcnow()
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
doc_ref.update(updates)
return self.get_user(email)
except Exception:
logger.exception(f"Error updating user {email}")
return None
def list_users(self, limit: int = 100, include_inactive: bool = False) -> List[User]:
"""List all users (admin only)."""
try:
query = self.db.collection(USERS_COLLECTION)
if not include_inactive:
query = query.where(filter=FieldFilter('is_active', '==', True))
query = query.order_by('created_at', direction=firestore.Query.DESCENDING)
query = query.limit(limit)
docs = query.stream()
return [User(**doc.to_dict()) for doc in docs]
except Exception:
logger.exception("Error listing users")
return []
def get_user_public(self, email: str) -> Optional[UserPublic]:
"""Get public user info."""
user = self.get_user(email)
if not user:
return None
return UserPublic(
email=user.email,
role=user.role,
credits=user.credits,
display_name=user.display_name,
total_jobs_created=user.total_jobs_created,
total_jobs_completed=user.total_jobs_completed,
tenant_id=user.tenant_id,
)
# =========================================================================
# Magic Link Authentication
# =========================================================================
def create_magic_link(
self,
email: str,
ip_address: Optional[str] = None,
user_agent: Optional[str] = None,
tenant_id: Optional[str] = None
) -> MagicLinkToken:
"""
Create a magic link token for email authentication.
Returns the token object. The actual sending of the email
should be handled by the caller (or an email service).
Args:
email: User's email address
ip_address: Client IP for auditing
user_agent: Client user agent for auditing
tenant_id: Tenant ID for white-label portals (None = default Nomad Karaoke)
"""
email = email.lower()
# Ensure user exists (with tenant association)
self.get_or_create_user(email, tenant_id=tenant_id)
# Generate secure token
token = secrets.token_urlsafe(32)
magic_link = MagicLinkToken(
token=token,
email=email,
expires_at=datetime.utcnow() + timedelta(minutes=MAGIC_LINK_EXPIRY_MINUTES),
ip_address=ip_address,
user_agent=user_agent,
tenant_id=tenant_id,
)
# Save to Firestore
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
doc_ref.set(magic_link.model_dump(mode='json'))
logger.info(f"Created magic link for {email}")
return magic_link
def create_admin_login_token(
self,
email: str,
expiry_hours: int = 24,
) -> MagicLinkToken:
"""
Create an admin login token for email-embedded authentication links.
Similar to magic links but with configurable expiry (default 24 hours).
Used for made-for-you order notification emails to allow admin one-click login.
Args:
email: Admin's email address to authenticate as
expiry_hours: Hours until token expires (default: 24, max: 168)
Returns:
MagicLinkToken object containing the token
Raises:
ValueError: If expiry_hours is out of valid range (1-168)
"""
# Validate expiry_hours (1 hour to 7 days)
if not 1 <= expiry_hours <= 168:
raise ValueError(f"expiry_hours must be between 1 and 168, got {expiry_hours}")
email = email.lower()
# Ensure user exists
self.get_or_create_user(email)
# Generate secure token
token = secrets.token_urlsafe(32)
admin_login = MagicLinkToken(
token=token,
email=email,
expires_at=datetime.utcnow() + timedelta(hours=expiry_hours),
)
# Save to Firestore (same collection as magic links for unified verification)
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
doc_ref.set(admin_login.model_dump(mode='json'))
# Log with redacted email (show only domain) for PII protection
domain = email.split('@')[-1] if '@' in email else 'unknown'
logger.info(f"Created admin login token for ***@{domain} (expires in {expiry_hours}h)")
return admin_login
def verify_magic_link(self, token: str) -> Tuple[bool, Optional[User], str]:
"""
Verify a magic link token using a Firestore transaction to prevent race conditions.
Returns:
(success, user, message)
"""
try:
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
@firestore.transactional
def verify_in_transaction(transaction):
"""Atomically verify and mark magic link as used."""
doc = doc_ref.get(transaction=transaction)
if not doc.exists:
return False, None, "Invalid or expired link"
magic_link = MagicLinkToken(**doc.to_dict())
# Check if already used
if magic_link.used:
return False, None, "This link has already been used"
# Check expiry
if datetime.utcnow() > magic_link.expires_at:
return False, None, "This link has expired"
# Mark as used atomically within transaction
transaction.update(doc_ref, {
'used': True,
'used_at': datetime.utcnow()
})
return True, magic_link.email, "Success"
# Execute the transaction
transaction = self.db.transaction()
success, email_or_error, message = verify_in_transaction(transaction)
if not success:
return False, None, message
# Get user and mark email as verified (outside transaction)
user = self.get_user(email_or_error)
if user:
self.update_user(
email_or_error,
email_verified=True,
last_login_at=datetime.utcnow()
)
user = self.get_user(email_or_error) # Refresh
logger.info(f"Magic link verified for {email_or_error}")
return True, user, "Successfully authenticated"
except Exception:
logger.exception("Error verifying magic link")
return False, None, "An error occurred during verification"
# =========================================================================
# Session Management
# =========================================================================
def create_session(
self,
user_email: str,
ip_address: Optional[str] = None,
user_agent: Optional[str] = None,
tenant_id: Optional[str] = None
) -> Session:
"""
Create a new session for an authenticated user.
Args:
user_email: User's email address
ip_address: Client IP for auditing
user_agent: Client user agent for auditing
tenant_id: Tenant ID for white-label portals (None = default Nomad Karaoke)
"""
token = secrets.token_urlsafe(32)
token_prefix = token[:12]
session = Session(
token=token,
user_email=user_email.lower(),
expires_at=datetime.utcnow() + timedelta(days=SESSION_ABSOLUTE_EXPIRY_DAYS),
ip_address=ip_address,
user_agent=user_agent,
tenant_id=tenant_id,
)
# Serialize and write to Firestore
session_data = session.model_dump(mode='json')
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
doc_ref.set(session_data)
# Verify the write succeeded by reading back
verify_doc = doc_ref.get()
if not verify_doc.exists:
logger.error(f"Session write verification FAILED for {user_email}: {token_prefix}...")
raise RuntimeError(f"Failed to persist session for {user_email}")
logger.info(f"Created and verified session for {user_email}: {token_prefix}... (expires: {session.expires_at})")
return session
def validate_session(self, token: str) -> Tuple[bool, Optional[User], str]:
"""
Validate a session token.
Returns:
(valid, user, message)
"""
# Log token info for debugging (only prefix for security)
token_prefix = token[:12] if token and len(token) >= 12 else token
logger.debug(f"Validating session token: {token_prefix}... (len={len(token) if token else 0})")
try:
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
doc = doc_ref.get()
if not doc.exists:
logger.warning(f"Session not found in Firestore: {token_prefix}...")
return False, None, "Invalid session"
raw_data = doc.to_dict()
logger.debug(f"Session data found for {token_prefix}...: user_email={raw_data.get('user_email')}, is_active={raw_data.get('is_active')}")
session = Session(**raw_data)
# Check if active
if not session.is_active:
logger.warning(f"Session revoked for {token_prefix}... (user: {session.user_email})")
return False, None, "Session has been revoked"
# Use timezone-aware datetime for all comparisons
# Firestore returns timezone-aware datetimes, so we must use aware datetimes
now = datetime.now(timezone.utc)
# Normalize session datetimes to be timezone-aware for comparison
# (handles legacy naive datetimes that may exist in Firestore)
expires_at = session.expires_at
if expires_at.tzinfo is None:
expires_at = expires_at.replace(tzinfo=timezone.utc)
last_activity = session.last_activity_at
if last_activity.tzinfo is None:
last_activity = last_activity.replace(tzinfo=timezone.utc)
# Check expiry
if now > expires_at:
logger.warning(f"Session expired for {token_prefix}... (user: {session.user_email}, expired_at: {expires_at}, now: {now})")
return False, None, "Session has expired"
# Check inactivity (7 days)
inactivity_limit = now - timedelta(days=SESSION_EXPIRY_DAYS)
if last_activity < inactivity_limit:
logger.warning(f"Session inactive for {token_prefix}... (user: {session.user_email}, last_activity: {last_activity}, limit: {inactivity_limit})")
return False, None, "Session expired due to inactivity"
# Update last activity (use timezone-aware datetime)
doc_ref.update({'last_activity_at': now})
# Get user
user = self.get_user(session.user_email)
if not user:
logger.warning(f"User not found for session {token_prefix}...: {session.user_email}")
return False, None, "User not found"
if not user.is_active:
logger.warning(f"User account disabled for session {token_prefix}...: {session.user_email}")
return False, None, "User account is disabled"
logger.debug(f"Session valid for {token_prefix}... (user: {user.email}, credits: {user.credits})")
return True, user, "Valid session"
except Exception:
logger.exception(f"Error validating session {token_prefix}...")
return False, None, "An error occurred during validation"
def revoke_session(self, token: str) -> bool:
"""Revoke a session (logout)."""
try:
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
doc_ref.update({'is_active': False})
logger.info("Session revoked")
return True
except Exception:
logger.exception("Error revoking session")
return False
def revoke_all_sessions(self, user_email: str) -> int:
"""Revoke all sessions for a user."""
try:
query = self.db.collection(SESSIONS_COLLECTION).where(
filter=FieldFilter('user_email', '==', user_email.lower())
).where(
filter=FieldFilter('is_active', '==', True)
)
count = 0
for doc in query.stream():
doc.reference.update({'is_active': False})
count += 1
logger.info(f"Revoked {count} sessions for {user_email}")
return count
except Exception:
logger.exception("Error revoking sessions")
return 0
def list_sessions_for_user(
self,
user_email: str,
include_revoked: bool = False,
limit: int = 50
) -> List[Session]:
"""
List all sessions for a user.
Args:
user_email: User's email address
include_revoked: If True, include revoked/inactive sessions
limit: Maximum number of sessions to return
Returns:
List of Session objects, ordered by created_at descending
"""
try:
query = self.db.collection(SESSIONS_COLLECTION).where(
filter=FieldFilter('user_email', '==', user_email.lower())
)
if not include_revoked:
query = query.where(filter=FieldFilter('is_active', '==', True))
query = query.order_by('created_at', direction=firestore.Query.DESCENDING)
query = query.limit(limit)
sessions = []
for doc in query.stream():
try:
sessions.append(Session(**doc.to_dict()))
except Exception as e:
logger.warning(f"Failed to parse session document: {e}")
logger.debug(f"Found {len(sessions)} sessions for {user_email}")
return sessions
except Exception:
logger.exception(f"Error listing sessions for {user_email}")
return []
# =========================================================================
# Credit Operations
# =========================================================================
def add_credits(
self,
email: str,
amount: int,
reason: str,
job_id: Optional[str] = None,
stripe_session_id: Optional[str] = None,
admin_email: Optional[str] = None
) -> Tuple[bool, int, str]:
"""
Add credits to a user account.
Returns:
(success, new_balance, message)
"""
try:
# For Stripe sessions, atomically check and mark as processed
# Uses create() which fails if document exists - ensures idempotency
if stripe_session_id:
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
try:
doc_ref.create({
'stripe_session_id': stripe_session_id,
'email': email.lower(),
'amount': amount,
'processed_at': datetime.utcnow()
})
except Exception:
# Document already exists - this session was already processed
logger.info(f"Stripe session {stripe_session_id} already processed (idempotent skip)")
return False, 0, "Session already processed"
email = email.lower()
user = self.get_or_create_user(email)
# Create transaction record
transaction = CreditTransaction(
id=str(uuid.uuid4()),
amount=amount,
reason=reason,
job_id=job_id,
stripe_session_id=stripe_session_id,
created_by=admin_email,
)
# Add to transaction history (keep last N)
transactions = user.credit_transactions[-MAX_CREDIT_TRANSACTIONS + 1:]
transactions.append(transaction)
# Update user
new_balance = user.credits + amount
self.update_user(
email,
credits=new_balance,
credit_transactions=[t.model_dump(mode='json') for t in transactions]
)
logger.info(f"Added {amount} credits to {email} ({reason}). New balance: {new_balance}")
return True, new_balance, f"Added {amount} credits"
except Exception as e:
logger.exception(f"Error adding credits to {email}")
return False, 0, f"Failed to add credits: {e}"
def deduct_credit(
self,
email: str,
job_id: str,
reason: str = "job_creation"
) -> Tuple[bool, int, str]:
"""
Deduct one credit from a user account.
Uses Firestore transaction to prevent race conditions.
Returns:
(success, remaining_credits, message)
"""
try:
email = email.lower()
doc_ref = self.db.collection(USERS_COLLECTION).document(email)
@firestore.transactional
def deduct_in_transaction(transaction):
"""Atomically check and deduct credit."""
doc = doc_ref.get(transaction=transaction)
if not doc.exists:
return False, 0, "User not found"
user_data = doc.to_dict()
current_credits = user_data.get('credits', 0)
if current_credits <= 0:
return False, 0, "Insufficient credits"
# Create transaction record
credit_txn = CreditTransaction(
id=str(uuid.uuid4()),
amount=-1,
reason=reason,
job_id=job_id,
)
# Get existing transactions and add new one
existing_transactions = user_data.get('credit_transactions', [])
# Keep last N-1 transactions to make room for new one
transactions = existing_transactions[-(MAX_CREDIT_TRANSACTIONS - 1):]
transactions.append(credit_txn.model_dump(mode='json'))
# Calculate new values
new_balance = current_credits - 1
total_jobs = user_data.get('total_jobs_created', 0) + 1
# Update atomically within transaction
transaction.update(doc_ref, {
'credits': new_balance,
'credit_transactions': transactions,
'total_jobs_created': total_jobs,
'updated_at': datetime.utcnow()
})
return True, new_balance, f"Credit deducted. {new_balance} remaining"
# Execute the transaction
fs_transaction = self.db.transaction()
success, new_balance, message = deduct_in_transaction(fs_transaction)
if success:
logger.info(f"Deducted 1 credit from {email} for job {job_id}. Remaining: {new_balance}")
return success, new_balance, message
except Exception as e:
logger.exception(f"Error deducting credit from {email}")
return False, 0, f"Failed to deduct credit: {e}"
def refund_credit(
self,
email: str,
job_id: str,
reason: str = "job_failed"
) -> Tuple[bool, int, str]:
"""
Refund one credit to a user account (e.g., after job failure).
Returns:
(success, new_balance, message)
"""
return self.add_credits(email, 1, reason, job_id=job_id)
def check_credits(self, email: str) -> int:
"""Check user's credit balance."""
user = self.get_user(email)
return user.credits if user else 0
def has_credits(self, email: str) -> bool:
"""Check if user has at least one credit."""
return self.check_credits(email) > 0
def is_stripe_session_processed(self, stripe_session_id: str) -> bool:
"""
Check if a Stripe session ID has already been processed.
Used to ensure idempotency of webhook processing - prevents
duplicate credit additions if Stripe sends the same webhook twice.
Uses a dedicated collection for O(1) lookup instead of scanning all users.
Args:
stripe_session_id: The Stripe checkout session ID
Returns:
True if this session was already processed
"""
try:
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
doc = doc_ref.get()
if doc.exists:
logger.info(f"Stripe session {stripe_session_id} already processed")
return True
return False
except Exception:
logger.exception(f"Error checking if stripe session {stripe_session_id} was processed")
# On error, return False to allow processing (better to risk duplicate than block)
return False
def _mark_stripe_session_processed(self, stripe_session_id: str, email: str, amount: int) -> None:
"""
Mark a Stripe session as processed.
Called after successfully adding credits from a Stripe webhook.
Args:
stripe_session_id: The Stripe checkout session ID
email: User email who received the credits
amount: Number of credits added
"""
try:
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
doc_ref.set({
'stripe_session_id': stripe_session_id,
'email': email,
'amount': amount,
'processed_at': datetime.utcnow()
})
except Exception:
# Log but don't fail - the credits were already added
logger.exception(f"Error marking stripe session {stripe_session_id} as processed")
# =========================================================================
# Admin Operations
# =========================================================================
def set_user_role(self, email: str, role: UserRole, admin_email: str) -> bool:
"""Set a user's role (admin only)."""
try:
user = self.get_user(email)
if not user:
return False
self.update_user(email, role=role.value)
logger.info(f"Admin {admin_email} set role for {email} to {role.value}")
return True
except Exception:
logger.exception("Error setting user role")
return False
def disable_user(self, email: str, admin_email: str) -> bool:
"""Disable a user account (admin only)."""
try:
self.update_user(email, is_active=False)
self.revoke_all_sessions(email)
logger.info(f"Admin {admin_email} disabled user {email}")
return True
except Exception:
logger.exception("Error disabling user")
return False
def enable_user(self, email: str, admin_email: str) -> bool:
"""Enable a user account (admin only)."""
try:
self.update_user(email, is_active=True)
logger.info(f"Admin {admin_email} enabled user {email}")
return True
except Exception:
logger.exception("Error enabling user")
return False
def increment_jobs_completed(self, email: str) -> bool:
"""Increment the completed jobs counter for a user using atomic increment."""
try:
# Check user exists first
if not self.get_user(email):
return False
# Use atomic increment to prevent race conditions
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
doc_ref.update({
'total_jobs_completed': Increment(1),
'updated_at': datetime.utcnow()
})
return True
except Exception:
logger.exception(f"Error incrementing jobs completed for {email}")
return False
# Global instance
_user_service = None
def get_user_service() -> UserService:
"""Get the global user service instance."""
global _user_service
if _user_service is None:
_user_service = UserService()
return _user_service
+14 -161
View File
@@ -10,20 +10,14 @@ SOLID Principles:
- Single Responsibility: Only handles worker coordination
- Dependency Inversion: Depends on HTTP abstraction, not implementation
- Open/Closed: Can add new workers without modifying existing code
Observability:
- Propagates trace context through Cloud Tasks for distributed tracing
- All worker invocations create spans linked to original request trace
"""
import logging
import os
import json
from typing import Optional
import httpx
from google.protobuf import duration_pb2
from backend.config import get_settings
from backend.services.tracing import inject_trace_context
logger = logging.getLogger(__name__)
@@ -37,25 +31,8 @@ WORKER_QUEUES = {
"screens": "screens-worker-queue",
"render-video": "render-worker-queue",
"video": "video-worker-queue",
"idle-reminder": "idle-reminder-queue", # For delayed idle reminder checks
}
# Dispatch deadlines for each worker type (in seconds)
# Cloud Tasks max is 1800s (30 min). These set how long Cloud Tasks waits
# for the HTTP handler to respond before considering it failed.
# Must be >= actual worker timeout + buffer for startup/upload.
WORKER_DISPATCH_DEADLINES = {
"audio": 1800, # 30 min - Modal separation can take 15-20 min
"lyrics": 1500, # 25 min - TRANSCRIPTION_TIMEOUT_SECONDS is 1200s (20 min)
"screens": 600, # 10 min - Screen generation is fast
"render-video": 1800, # 30 min - Video encoding can be slow
"video": 1800, # 30 min - Video encoding can be slow
"idle-reminder": 60, # 1 min - Quick check and potential email send
}
# Default delay for idle reminders (seconds)
IDLE_REMINDER_DELAY_SECONDS = 5 * 60 # 5 minutes
class WorkerService:
"""
@@ -203,9 +180,6 @@ class WorkerService:
- Deduplication (via task name if needed)
- OIDC authentication for Cloud Run
Observability:
- Injects trace context headers so worker spans link to parent trace
Args:
worker_type: Worker type
job_id: Job ID to process
@@ -231,36 +205,14 @@ class WorkerService:
# Build queue path
parent = self.tasks_client.queue_path(project, location, queue_name)
# Build base headers
headers = {
"Content-Type": "application/json",
}
# Add admin auth via custom header
# NOTE: We use X-Admin-Token instead of Authorization because Cloud Tasks
# OIDC token overwrites the Authorization header when oidc_token is specified.
# The OIDC token handles Cloud Run authentication (allows Cloud Tasks to invoke
# the service), while X-Admin-Token handles application-level authentication.
if self._admin_token:
headers["X-Admin-Token"] = self._admin_token
logger.debug(
f"[job:{job_id}] Using admin token for Cloud Task auth via X-Admin-Token, "
f"token prefix: {self._admin_token[:8]}..., len={len(self._admin_token)}"
)
# Inject trace context for distributed tracing
# This allows worker spans to link back to the original request trace
headers = inject_trace_context(headers)
# Get dispatch deadline for this worker type (how long Cloud Tasks waits for response)
dispatch_deadline_seconds = WORKER_DISPATCH_DEADLINES.get(worker_type, 600)
# Build task payload
task = {
"http_request": {
"http_method": tasks_v2.HttpMethod.POST,
"url": f"{self._base_url}/api/internal/workers/{worker_type}",
"headers": headers,
"headers": {
"Content-Type": "application/json",
},
"body": json.dumps({"job_id": job_id}).encode(),
# Use OIDC token for Cloud Run authentication
# This allows Cloud Tasks to invoke the Cloud Run service
@@ -268,25 +220,25 @@ class WorkerService:
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
},
},
# Set dispatch_deadline - how long Cloud Tasks waits for the handler to respond
# Default is 10 min, but audio separation can take 30+ min
# Max is 1800s (30 min) for Cloud Tasks
"dispatch_deadline": duration_pb2.Duration(seconds=dispatch_deadline_seconds),
}
# Add admin auth header if available
# This is needed for the internal endpoint's require_admin dependency
if self._admin_token:
task["http_request"]["headers"]["Authorization"] = f"Bearer {self._admin_token}"
# Create task
# Note: We don't set a task name, allowing Cloud Tasks to generate one
# This prevents duplicate task errors on retries
response = self.tasks_client.create_task(parent=parent, task=task)
logger.info(
f"[job:{job_id}] Created Cloud Task for {worker_type} worker: {response.name} "
f"(dispatch_deadline={dispatch_deadline_seconds}s)"
f"Created Cloud Task for {worker_type} worker, job {job_id}: {response.name}"
)
return True
except Exception as e:
logger.error(
f"[job:{job_id}] Failed to enqueue Cloud Task for {worker_type}: {e}",
f"Failed to enqueue Cloud Task for {worker_type}/{job_id}: {e}",
exc_info=True
)
return False
@@ -303,9 +255,6 @@ class WorkerService:
This is the original implementation - direct HTTP POST to internal endpoint.
The endpoint adds the worker function to FastAPI BackgroundTasks.
Observability:
- Injects trace context headers so worker spans link to parent trace
Args:
worker_type: Worker type
job_id: Job ID to process
@@ -320,9 +269,6 @@ class WorkerService:
if self._admin_token:
headers["Authorization"] = f"Bearer {self._admin_token}"
# Inject trace context for distributed tracing
headers = inject_trace_context(headers)
async with httpx.AsyncClient(timeout=timeout_seconds) as client:
url = f"{self._base_url}/api/internal/workers/{worker_type}"
@@ -333,22 +279,22 @@ class WorkerService:
)
if response.status_code == 200:
logger.info(f"[job:{job_id}] Successfully triggered {worker_type} worker")
logger.info(f"Successfully triggered {worker_type} worker for job {job_id}")
return True
else:
logger.error(
f"[job:{job_id}] Failed to trigger {worker_type} worker: "
f"Failed to trigger {worker_type} worker for job {job_id}: "
f"HTTP {response.status_code} - {response.text}"
)
return False
except httpx.TimeoutException:
logger.error(f"[job:{job_id}] Timeout triggering {worker_type} worker")
logger.error(f"Timeout triggering {worker_type} worker for job {job_id}")
return False
except Exception as e:
logger.error(
f"[job:{job_id}] Error triggering {worker_type} worker: {e}",
f"Error triggering {worker_type} worker for job {job_id}: {e}",
exc_info=True
)
return False
@@ -435,99 +381,6 @@ class WorkerService:
"""Trigger render video worker (post-review)."""
return await self.trigger_worker("render-video", job_id)
async def schedule_idle_reminder(
self,
job_id: str,
delay_seconds: int = IDLE_REMINDER_DELAY_SECONDS
) -> bool:
"""
Schedule an idle reminder check for a job.
The reminder task will be delivered after the specified delay.
When the task executes, it checks if the job is still in a blocking
state and sends a reminder email if the user hasn't taken action.
Args:
job_id: Job ID to check
delay_seconds: Delay before executing the check (default: 5 minutes)
Returns:
True if task was scheduled successfully, False otherwise
"""
if not self._use_cloud_tasks:
# In development mode, log and skip (no delayed execution support)
logger.info(
f"[job:{job_id}] Idle reminder not scheduled (Cloud Tasks disabled). "
f"Would have fired in {delay_seconds}s."
)
return True
try:
from google.cloud import tasks_v2
from google.protobuf import timestamp_pb2
import time
queue_name = WORKER_QUEUES.get("idle-reminder")
if not queue_name:
logger.error("Idle reminder queue not configured")
return False
project = self.settings.google_cloud_project
if not project:
logger.error("GOOGLE_CLOUD_PROJECT not set")
return False
location = self.settings.gcp_region
# Build queue path
parent = self.tasks_client.queue_path(project, location, queue_name)
# Build headers
headers = {
"Content-Type": "application/json",
}
if self._admin_token:
headers["X-Admin-Token"] = self._admin_token
# Inject trace context
headers = inject_trace_context(headers)
# Calculate schedule time
schedule_time = timestamp_pb2.Timestamp()
schedule_time.FromSeconds(int(time.time()) + delay_seconds)
dispatch_deadline_seconds = WORKER_DISPATCH_DEADLINES.get("idle-reminder", 60)
# Build task payload
task = {
"http_request": {
"http_method": tasks_v2.HttpMethod.POST,
"url": f"{self._base_url}/api/internal/jobs/{job_id}/check-idle-reminder",
"headers": headers,
"body": json.dumps({"job_id": job_id}).encode(),
"oidc_token": {
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
},
},
"schedule_time": schedule_time,
"dispatch_deadline": duration_pb2.Duration(seconds=dispatch_deadline_seconds),
}
# Create task
response = self.tasks_client.create_task(parent=parent, task=task)
logger.info(
f"[job:{job_id}] Scheduled idle reminder check in {delay_seconds}s: {response.name}"
)
return True
except Exception as e:
logger.error(
f"[job:{job_id}] Failed to schedule idle reminder: {e}",
exc_info=True
)
return False
# Global worker service instance
_worker_service: Optional[WorkerService] = None
-445
View File
@@ -1,445 +0,0 @@
"""
YouTube Upload Service.
Provides video upload functionality for YouTube, extracted from KaraokeFinalise
for use by both the cloud backend (video_worker) and local CLI.
This service handles:
- Video uploads with metadata (title, description, tags)
- Thumbnail uploads
- Duplicate video detection and replacement
- Authentication via pre-stored credentials or client secrets file
"""
import logging
import os
from typing import Optional, Dict, Any, Tuple
from thefuzz import fuzz
logger = logging.getLogger(__name__)
class YouTubeUploadService:
"""
Service for uploading videos to YouTube.
Supports two authentication modes:
1. Pre-stored credentials (for server-side/non-interactive use)
2. Client secrets file (for interactive CLI use with browser OAuth)
"""
YOUTUBE_URL_PREFIX = "https://www.youtube.com/watch?v="
SCOPES = ["https://www.googleapis.com/auth/youtube"]
def __init__(
self,
credentials: Optional[Dict[str, Any]] = None,
client_secrets_file: Optional[str] = None,
non_interactive: bool = False,
server_side_mode: bool = False,
dry_run: bool = False,
logger: Optional[logging.Logger] = None,
):
"""
Initialize the YouTube upload service.
Args:
credentials: Pre-stored OAuth credentials dict with keys:
- token: Access token
- refresh_token: Refresh token
- token_uri: Token endpoint URL
- client_id: OAuth client ID
- client_secret: OAuth client secret
- scopes: List of OAuth scopes
client_secrets_file: Path to OAuth client secrets JSON file
(for interactive authentication)
non_interactive: If True, skip interactive prompts
server_side_mode: If True, use exact matching for duplicate detection
dry_run: If True, log actions without performing them
logger: Optional logger instance
"""
self.credentials = credentials
self.client_secrets_file = client_secrets_file
self.non_interactive = non_interactive
self.server_side_mode = server_side_mode
self.dry_run = dry_run
self.logger = logger or logging.getLogger(__name__)
self._youtube_service = None
self._channel_id = None
def _authenticate(self):
"""
Authenticate with YouTube API and return service object.
Returns:
YouTube API service object
Raises:
Exception: If authentication fails or is required in non-interactive mode
"""
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
import pickle
# Check if we have pre-stored credentials (for non-interactive mode)
if self.credentials and self.non_interactive:
try:
creds = Credentials(
token=self.credentials.get('token'),
refresh_token=self.credentials.get('refresh_token'),
token_uri=self.credentials.get('token_uri'),
client_id=self.credentials.get('client_id'),
client_secret=self.credentials.get('client_secret'),
scopes=self.credentials.get('scopes', self.SCOPES)
)
# Refresh token if needed
if creds.expired and creds.refresh_token:
creds.refresh(Request())
youtube = build('youtube', 'v3', credentials=creds)
self.logger.info("Successfully authenticated with YouTube using pre-stored credentials")
return youtube
except Exception as e:
self.logger.error(f"Failed to authenticate with pre-stored credentials: {str(e)}")
# Fall through to original authentication if pre-stored credentials fail
# For non-interactive mode without pre-stored credentials, raise error
if self.non_interactive:
raise Exception(
"YouTube authentication required but running in non-interactive mode. "
"Please pre-authenticate or disable YouTube upload."
)
# Interactive authentication using client secrets file
if not self.client_secrets_file:
raise Exception("No YouTube credentials or client secrets file provided")
# Token file stores the user's access and refresh tokens
youtube_token_file = "/tmp/karaoke-finalise-youtube-token.pickle"
creds = None
# Check if we have saved credentials
if os.path.exists(youtube_token_file):
with open(youtube_token_file, "rb") as token:
creds = pickle.load(token)
# If there are no valid credentials, let the user log in
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
self.client_secrets_file, scopes=self.SCOPES
)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open(youtube_token_file, "wb") as token:
pickle.dump(creds, token)
return build("youtube", "v3", credentials=creds)
@property
def youtube_service(self):
"""Lazy-load YouTube service on first access."""
if self._youtube_service is None:
self._youtube_service = self._authenticate()
return self._youtube_service
def get_channel_id(self) -> Optional[str]:
"""
Get the authenticated user's YouTube channel ID.
Returns:
Channel ID string, or None if not found
"""
if self._channel_id:
return self._channel_id
request = self.youtube_service.channels().list(part="snippet", mine=True)
response = request.execute()
if "items" in response and len(response["items"]) > 0:
self._channel_id = response["items"][0]["id"]
return self._channel_id
return None
def check_duplicate(
self,
title: str,
exact_match: Optional[bool] = None
) -> Tuple[bool, Optional[str], Optional[str]]:
"""
Check if a video with the given title already exists on the channel.
Args:
title: Video title to search for
exact_match: If True, require exact title match. If None, uses
server_side_mode setting (exact in server mode, fuzzy in CLI)
Returns:
Tuple of (exists: bool, video_id: Optional[str], video_url: Optional[str])
"""
channel_id = self.get_channel_id()
if not channel_id:
self.logger.warning("Could not get channel ID, skipping duplicate check")
return False, None, None
use_exact_match = exact_match if exact_match is not None else self.server_side_mode
self.logger.info(f"Searching YouTube channel {channel_id} for title: {title}")
request = self.youtube_service.search().list(
part="snippet",
channelId=channel_id,
q=title,
type="video",
maxResults=10
)
response = request.execute()
if "items" not in response or len(response["items"]) == 0:
self.logger.info(f"No matching video found with title: {title}")
return False, None, None
for item in response["items"]:
# Verify the video actually belongs to our channel
result_channel_id = item["snippet"]["channelId"]
if result_channel_id != channel_id:
self.logger.debug(
f"Skipping video from different channel: {item['snippet']['title']} "
f"(channel: {result_channel_id})"
)
continue
found_title = item["snippet"]["title"]
# Determine if this is a match
if use_exact_match:
is_match = title.lower() == found_title.lower()
similarity_score = 100 if is_match else 0
else:
similarity_score = fuzz.ratio(title.lower(), found_title.lower())
is_match = similarity_score >= 70
if is_match:
video_id = item["id"]["videoId"]
video_url = f"{self.YOUTUBE_URL_PREFIX}{video_id}"
self.logger.info(
f"Potential match found on YouTube channel with ID: {video_id} "
f"and title: {found_title} (similarity: {similarity_score}%)"
)
# In non-interactive mode, return the match directly
if self.non_interactive:
self.logger.info("Non-interactive mode, found a match.")
return True, video_id, video_url
# Interactive confirmation
confirmation = input(
f"Is '{found_title}' the video you are finalising? (y/n): "
).strip().lower()
if confirmation == "y":
return True, video_id, video_url
self.logger.info(f"No matching video found with title: {title}")
return False, None, None
def delete_video(self, video_id: str) -> bool:
"""
Delete a YouTube video by its ID.
Args:
video_id: The YouTube video ID to delete
Returns:
True if successful, False otherwise
"""
self.logger.info(f"Deleting YouTube video with ID: {video_id}")
if self.dry_run:
self.logger.info(f"DRY RUN: Would delete YouTube video with ID: {video_id}")
return True
try:
self.youtube_service.videos().delete(id=video_id).execute()
self.logger.info(f"Successfully deleted YouTube video with ID: {video_id}")
return True
except Exception as e:
self.logger.error(f"Failed to delete YouTube video with ID {video_id}: {e}")
return False
@staticmethod
def truncate_title(title: str, max_length: int = 95) -> str:
"""
Truncate title to the nearest whole word within max_length.
Args:
title: Title to truncate
max_length: Maximum length (default 95 for YouTube)
Returns:
Truncated title with "..." if needed
"""
if len(title) <= max_length:
return title
truncated_title = title[:max_length].rsplit(" ", 1)[0]
if len(truncated_title) < max_length:
truncated_title += " ..."
return truncated_title
def upload_video(
self,
video_path: str,
title: str,
description: str,
thumbnail_path: Optional[str] = None,
tags: Optional[list] = None,
category_id: str = "10", # Music category
privacy_status: str = "public",
replace_existing: bool = False,
) -> Tuple[Optional[str], Optional[str]]:
"""
Upload a video to YouTube with metadata and optional thumbnail.
Args:
video_path: Path to the video file to upload
title: Video title (will be truncated to 95 chars if needed)
description: Video description
thumbnail_path: Optional path to thumbnail image
tags: Optional list of tags/keywords
category_id: YouTube category ID (default "10" for Music)
privacy_status: "public", "private", or "unlisted"
replace_existing: If True, delete existing video with same title
Returns:
Tuple of (video_id, video_url) or (None, None) if upload failed/skipped
"""
from googleapiclient.http import MediaFileUpload
# Truncate title if needed
youtube_title = self.truncate_title(title)
self.logger.info(f"Uploading video to YouTube: {youtube_title}")
if self.dry_run:
self.logger.info(
f"DRY RUN: Would upload {video_path} to YouTube with title: {youtube_title}"
)
return "dry_run_video_id", f"{self.YOUTUBE_URL_PREFIX}dry_run_video_id"
# Check for existing video
should_replace = True if self.server_side_mode else replace_existing
exists, existing_id, existing_url = self.check_duplicate(youtube_title)
if exists:
if should_replace:
self.logger.info(f"Video already exists on YouTube, deleting before re-upload: {existing_url}")
if self.delete_video(existing_id):
self.logger.info("Successfully deleted existing video, proceeding with upload")
else:
self.logger.error("Failed to delete existing video, aborting upload")
return None, None
else:
self.logger.warning(f"Video already exists on YouTube, skipping upload: {existing_url}")
return existing_id, existing_url
# Prepare video metadata
body = {
"snippet": {
"title": youtube_title,
"description": description,
"tags": tags or [],
"categoryId": category_id,
},
"status": {"privacyStatus": privacy_status},
}
# Determine MIME type based on file extension
ext = os.path.splitext(video_path)[1].lower()
mime_type = {
".mkv": "video/x-matroska",
".mp4": "video/mp4",
".mov": "video/quicktime",
".avi": "video/x-msvideo",
}.get(ext, "video/*")
# Upload video
self.logger.info(f"Authenticating with YouTube...")
media_file = MediaFileUpload(video_path, mimetype=mime_type, resumable=True)
self.logger.info(f"Uploading video to YouTube...")
request = self.youtube_service.videos().insert(
part="snippet,status",
body=body,
media_body=media_file
)
response = request.execute()
video_id = response.get("id")
video_url = f"{self.YOUTUBE_URL_PREFIX}{video_id}"
self.logger.info(f"Uploaded video to YouTube: {video_url}")
# Upload thumbnail if provided
if thumbnail_path and os.path.isfile(thumbnail_path):
try:
self.logger.info(f"Uploading thumbnail from: {thumbnail_path}")
media_thumbnail = MediaFileUpload(thumbnail_path, mimetype="image/jpeg")
self.youtube_service.thumbnails().set(
videoId=video_id,
media_body=media_thumbnail
).execute()
self.logger.info(f"Uploaded thumbnail for video ID {video_id}")
except Exception as e:
self.logger.error(f"Failed to upload thumbnail: {e}")
self.logger.warning(
"Video uploaded but thumbnail not set. "
"You may need to set it manually on YouTube."
)
elif thumbnail_path:
self.logger.warning(f"Thumbnail file not found, skipping: {thumbnail_path}")
return video_id, video_url
# Singleton instance and factory function (following existing service pattern)
_youtube_upload_service: Optional[YouTubeUploadService] = None
def get_youtube_upload_service(
credentials: Optional[Dict[str, Any]] = None,
client_secrets_file: Optional[str] = None,
**kwargs
) -> YouTubeUploadService:
"""
Get a YouTube upload service instance.
For server-side use, pass credentials from YouTubeService.
For CLI use, pass client_secrets_file.
Args:
credentials: Pre-stored OAuth credentials dict
client_secrets_file: Path to OAuth client secrets JSON file
**kwargs: Additional arguments passed to YouTubeUploadService
Returns:
YouTubeUploadService instance
"""
global _youtube_upload_service
# Create new instance if credentials/settings changed
if _youtube_upload_service is None or credentials or client_secrets_file:
_youtube_upload_service = YouTubeUploadService(
credentials=credentials,
client_secrets_file=client_secrets_file,
**kwargs
)
return _youtube_upload_service
+4 -117
View File
@@ -6,31 +6,19 @@ Provides common mocks and test utilities across all test modules.
NOTE: Module-level mocks are NOT applied here to allow emulator tests to work.
Individual tests must mock dependencies as needed.
"""
# IMPORTANT: Set environment variables FIRST, before ANY imports that might
# trigger backend module loading and Settings singleton creation.
import os
# Set up test environment variables BEFORE importing any backend modules
# This must happen before Settings is instantiated, which occurs on first import
if 'FIRESTORE_EMULATOR_HOST' not in os.environ:
os.environ.setdefault('ADMIN_TOKENS', 'test-admin-token')
os.environ.setdefault('GOOGLE_CLOUD_PROJECT', 'test-project')
os.environ.setdefault('GCS_BUCKET_NAME', 'test-bucket')
os.environ.setdefault('FIRESTORE_COLLECTION', 'jobs')
os.environ.setdefault('ENVIRONMENT', 'test')
import pytest
from unittest.mock import Mock, MagicMock, AsyncMock, patch
from datetime import datetime, UTC
from fastapi.testclient import TestClient
# Mock google.auth.default AND firestore for unit tests if not using emulator
# This prevents DefaultCredentialsError and FirestoreClient initialization during imports
# Mock google.auth.default for unit tests if not using emulator
# This prevents DefaultCredentialsError during imports
if 'FIRESTORE_EMULATOR_HOST' not in os.environ:
from unittest.mock import MagicMock
# Mock google.auth.default - prevents credential errors
# Only mock the default() function, not the entire module
# This allows other google.auth imports to work normally
try:
import google.auth
mock_credentials = MagicMock()
@@ -42,111 +30,10 @@ if 'FIRESTORE_EMULATOR_HOST' not in os.environ:
except ImportError:
# If google.auth not installed, that's ok for some tests
pass
# Mock google.cloud.firestore.Client - prevents Firestore initialization
# This is CRITICAL: when AuthService is instantiated, it creates FirestoreService
# which tries to create a real Firestore client. The client returns MagicMocks
# which fail when used as enum values (e.g., UserType).
try:
import google.cloud.firestore as firestore_module
# Create a mock client that returns proper values for get_token
mock_firestore_client = MagicMock()
mock_collection = MagicMock()
mock_doc = MagicMock()
mock_snapshot = MagicMock()
# For auth token lookups, return None (token not found in Firestore)
# This forces validation to use admin tokens from environment
mock_snapshot.exists = False
mock_snapshot.to_dict.return_value = None
mock_doc.get.return_value = mock_snapshot
mock_collection.document.return_value = mock_doc
mock_firestore_client.collection.return_value = mock_collection
# Replace the Client class
original_client = firestore_module.Client
firestore_module.Client = MagicMock(return_value=mock_firestore_client)
except ImportError:
pass
# Mock google.cloud.storage.Client - prevents GCS initialization
try:
import google.cloud.storage as storage_module
mock_storage_client = MagicMock()
mock_bucket = MagicMock()
mock_storage_client.bucket.return_value = mock_bucket
storage_module.Client = MagicMock(return_value=mock_storage_client)
except ImportError:
pass
from backend.models.job import Job, JobStatus, JobCreate
@pytest.fixture(autouse=True)
def mock_auth_dependency(request):
"""Mock the require_auth dependency for all tests using FastAPI's dependency override system."""
# Skip for emulator tests and integration tests which use real auth
test_path = str(request.fspath)
if 'emulator' in test_path or 'integration' in test_path:
yield
return
# Skip for service-only unit tests that don't need the FastAPI app
# Also skip tests that manage their own auth mocks (e.g., push tests use AuthResult objects)
service_only_tests = ['test_rate_limit_service', 'test_email_validation_service', 'test_rate_limits_api', 'test_gce_encoding_worker', 'test_push_routes', 'test_push_notification_service']
if any(test_name in test_path for test_name in service_only_tests):
yield
return
# Skip if FIRESTORE_EMULATOR_HOST is set (running in emulator environment)
import os
if os.environ.get('FIRESTORE_EMULATOR_HOST'):
yield
return
from backend.services.auth_service import UserType
from backend.api.dependencies import require_auth, require_admin, require_review_auth, require_instrumental_auth
from backend.main import app
# Create mock auth functions
async def mock_require_auth():
"""Mock require_auth to always return valid admin credentials."""
return ("test-admin-token", UserType.ADMIN, 999)
async def mock_require_admin():
"""Mock require_admin to always return valid admin credentials."""
return ("test-admin-token", UserType.ADMIN, 999)
async def mock_require_review_auth(job_id: str = "test123"):
"""Mock require_review_auth to always return valid review access."""
return (job_id, "full")
async def mock_require_instrumental_auth(job_id: str = "test123"):
"""Mock require_instrumental_auth to always return valid instrumental review access."""
return (job_id, "full")
# Use FastAPI's dependency override system
app.dependency_overrides[require_auth] = mock_require_auth
app.dependency_overrides[require_admin] = mock_require_admin
app.dependency_overrides[require_review_auth] = mock_require_review_auth
app.dependency_overrides[require_instrumental_auth] = mock_require_instrumental_auth
yield
# Clean up after test
app.dependency_overrides.clear()
@pytest.fixture
def auth_headers():
"""Get authentication headers for test requests."""
return {
"Authorization": "Bearer test-admin-token",
"Content-Type": "application/json"
}
@pytest.fixture
def mock_firestore():
"""Mock Firestore client for unit tests."""
+1 -22
View File
@@ -39,24 +39,6 @@ os.environ["ADMIN_TOKENS"] = "test-admin-token"
# Only import app if emulators are running
if emulators_running():
from unittest.mock import Mock
# Mock theme service BEFORE importing app to ensure all jobs get theme_id="nomad"
_mock_theme_service = Mock()
_mock_theme_service.get_default_theme_id.return_value = "nomad"
_mock_theme_service.get_theme.return_value = None
# Apply patch at module load time (before app imports the service)
_theme_patches = [
patch("backend.services.theme_service.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.audio_search.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.file_upload.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.users.get_theme_service", return_value=_mock_theme_service),
patch("backend.api.routes.jobs.get_theme_service", return_value=_mock_theme_service),
]
for p in _theme_patches:
p.start()
from fastapi.testclient import TestClient
from backend.main import app
else:
@@ -94,10 +76,7 @@ def mock_worker_service():
@pytest.fixture(scope="session")
def client(mock_worker_service):
"""Create FastAPI test client with mocked workers.
Theme service is mocked at module load time (see above).
"""
"""Create FastAPI test client with mocked workers."""
with patch("backend.api.routes.file_upload.worker_service", mock_worker_service):
return TestClient(app)
File diff suppressed because it is too large. Load diff
@@ -27,15 +27,15 @@ pytestmark = pytest.mark.skipif(
class TestEmulatorBasics:
"""Basic emulator connectivity tests."""
def test_health_endpoint(self, client, auth_headers):
def test_health_endpoint(self, client):
"""Test health endpoint works."""
response = client.get("/api/health", )
response = client.get("/api/health")
assert response.status_code == 200
assert response.json()["status"] == "healthy"
def test_root_endpoint(self, client, auth_headers):
def test_root_endpoint(self, client):
"""Test root endpoint works."""
response = client.get("/", )
response = client.get("/")
assert response.status_code == 200
assert response.json()["service"] == "karaoke-gen-backend"
@@ -281,13 +281,13 @@ class TestReviewEndpoints:
time.sleep(0.2)
# Should fail - job not in AWAITING_REVIEW status
response = client.get(f"/api/review/{job_id}/correction-data", headers=auth_headers)
response = client.get(f"/api/review/{job_id}/correction-data")
assert response.status_code == 400
assert "not ready for review" in response.json()["detail"].lower()
def test_review_audio_no_job(self, client, auth_headers):
def test_review_audio_no_job(self, client):
"""Test audio endpoint returns 404 for nonexistent job."""
response = client.get("/api/review/nonexistent-job/audio/", headers=auth_headers)
response = client.get("/api/review/nonexistent-job/audio/")
assert response.status_code == 404
def test_review_preview_video_stub(self, client, auth_headers):
@@ -303,7 +303,6 @@ class TestReviewEndpoints:
# Preview video should return error since job not ready
response = client.post(
f"/api/review/{job_id}/preview-video",
headers=auth_headers,
json={"corrections": [], "corrected_segments": []}
)
# Should return 400 (job not in AWAITING_REVIEW state)
@@ -321,7 +320,6 @@ class TestReviewEndpoints:
# Annotations endpoint should accept data (even if it just logs it)
response = client.post(
f"/api/review/{job_id}/v1/annotations",
headers=auth_headers,
json={"type": "test", "data": "test annotation"}
)
assert response.status_code == 200
@@ -330,7 +328,7 @@ class TestReviewEndpoints:
class TestRenderVideoWorker:
"""Test render video worker endpoint."""
def test_render_video_worker_endpoint_exists(self, client, auth_headers):
"""Test that render-video worker endpoint exists."""
# Create a job
@@ -340,9 +338,9 @@ class TestRenderVideoWorker:
json={"url": "https://youtube.com/watch?v=render-test"}
)
job_id = create_resp.json()["job_id"]
time.sleep(0.2)
# Test render-video worker endpoint exists
response = client.post(
"/api/internal/workers/render-video",
@@ -1,167 +0,0 @@
"""
Emulator integration tests for made-for-you order features.
Tests with real Firestore emulator via API endpoints to verify:
1. Admin login token creation and verification
2. Made-for-you job fields are properly stored
Run with: scripts/run-emulator-tests.sh
"""
import pytest
import time
from .conftest import emulators_running
# Skip all tests if emulators not running
pytestmark = pytest.mark.skipif(
not emulators_running(),
reason="GCP emulators not running. Start with: scripts/start-emulators.sh"
)
class TestAdminLoginTokenIntegration:
"""
Integration tests for admin login token feature via API.
These tests verify that admin login tokens can be verified
through the actual API endpoint.
"""
def test_verify_invalid_token_returns_401(self, client):
"""Invalid token returns 401 unauthorized."""
response = client.get("/api/users/auth/verify?token=invalid-token-xyz")
assert response.status_code == 401
def test_verify_missing_token_returns_422(self, client):
"""Missing token parameter returns 422 validation error."""
response = client.get("/api/users/auth/verify")
assert response.status_code == 422
def test_verify_empty_token_returns_401(self, client):
"""Empty token returns 401."""
response = client.get("/api/users/auth/verify?token=")
assert response.status_code == 401
class TestMadeForYouJobModel:
"""
Tests for made-for-you job model fields.
Verifies that the Job model properly supports made_for_you fields.
"""
def test_job_model_has_made_for_you_fields(self):
"""Job model includes made_for_you and customer_email fields."""
from backend.models.job import Job, JobStatus
from datetime import datetime, timezone
# Create a made-for-you job directly
job = Job(
job_id="test-mfy-model",
status=JobStatus.PENDING,
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
user_email="admin@nomadkaraoke.com",
made_for_you=True,
customer_email="customer@example.com",
customer_notes="Please make it perfect!",
)
assert job.made_for_you is True
assert job.customer_email == "customer@example.com"
assert job.customer_notes == "Please make it perfect!"
def test_job_model_defaults_made_for_you_false(self):
"""Regular jobs default made_for_you to False."""
from backend.models.job import Job, JobStatus
from datetime import datetime, timezone
job = Job(
job_id="test-regular-model",
status=JobStatus.PENDING,
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
user_email="user@example.com",
)
# Default should be False or None
assert job.made_for_you in [False, None]
assert job.customer_email is None
class TestOwnershipTransferLogic:
"""
Tests for ownership transfer logic (unit-style, runs in emulator context).
These tests verify the conditional logic for made-for-you ownership transfer
by importing the job_manager and testing its behavior directly.
"""
def test_ownership_transfer_condition_made_for_you_true(self):
"""
Verify the condition: if made_for_you=True and customer_email exists,
ownership should transfer.
"""
# Replicate the exact logic from _schedule_completion_email
class MockJob:
made_for_you = True
customer_email = "customer@example.com"
user_email = "admin@nomadkaraoke.com"
job_id = "test-123"
job = MockJob()
# The logic being tested
recipient_email = job.user_email
should_transfer = False
if job.made_for_you and job.customer_email:
recipient_email = job.customer_email
should_transfer = True
assert should_transfer is True
assert recipient_email == "customer@example.com"
def test_ownership_transfer_condition_made_for_you_false(self):
"""
Verify: if made_for_you=False, no ownership transfer.
"""
class MockJob:
made_for_you = False
customer_email = "customer@example.com"
user_email = "user@example.com"
job_id = "test-456"
job = MockJob()
recipient_email = job.user_email
should_transfer = False
if job.made_for_you and job.customer_email:
recipient_email = job.customer_email
should_transfer = True
assert should_transfer is False
assert recipient_email == "user@example.com"
def test_ownership_transfer_condition_missing_customer_email(self):
"""
Verify: if made_for_you=True but no customer_email, no transfer.
"""
class MockJob:
made_for_you = True
customer_email = None
user_email = "admin@nomadkaraoke.com"
job_id = "test-789"
job = MockJob()
recipient_email = job.user_email
should_transfer = False
if job.made_for_you and job.customer_email:
recipient_email = job.customer_email
should_transfer = True
assert should_transfer is False
assert recipient_email == "admin@nomadkaraoke.com"
@@ -1,443 +0,0 @@
"""
Integration tests for worker log subcollection storage.
Tests the subcollection approach using real Firestore emulator to verify:
- Logs are stored in subcollection (jobs/{job_id}/logs)
- TTL expiry field is set correctly
- Large log volumes work without 1MB limit issues
- Concurrent writes are handled correctly
- Logs can be queried efficiently
Run with: ./scripts/run-emulator-tests.sh
"""
import pytest
import time
import requests
import os
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone, timedelta
def emulators_running() -> bool:
"""Check if GCP emulators are running."""
try:
requests.get("http://127.0.0.1:8080", timeout=1)
return True
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
return False
# Skip all tests in this module if emulators aren't running
pytestmark = pytest.mark.skipif(
not emulators_running(),
reason="GCP emulators not running. Start with: scripts/start-emulators.sh"
)
# Set up environment for emulator
os.environ["FIRESTORE_EMULATOR_HOST"] = "127.0.0.1:8080"
os.environ["GOOGLE_CLOUD_PROJECT"] = "test-project"
class TestWorkerLogsSubcollectionDirect:
"""Direct Firestore tests for worker logs subcollection."""
@pytest.fixture(autouse=True)
def setup_firestore(self):
"""Set up Firestore client for each test."""
from google.cloud import firestore
self.db = firestore.Client(project="test-project")
self.collection = "test-subcollection-jobs"
yield
def _create_test_job(self):
"""Create a test job document."""
job_id = f"test-sub-{int(time.time() * 1000)}"
doc_ref = self.db.collection(self.collection).document(job_id)
doc_ref.set({
"job_id": job_id,
"status": "pending",
"created_at": datetime.now(timezone.utc),
"worker_logs": [] # Legacy field - kept empty for new jobs
})
return job_id
def _append_log_to_subcollection(self, job_id: str, worker: str, message: str, ttl_days: int = 30):
"""Add log to subcollection at jobs/{job_id}/logs."""
import uuid
log_id = str(uuid.uuid4())
now = datetime.now(timezone.utc)
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
logs_ref.document(log_id).set({
"id": log_id,
"job_id": job_id,
"timestamp": now,
"level": "INFO",
"worker": worker,
"message": message,
"ttl_expiry": now + timedelta(days=ttl_days)
})
return log_id
def _get_logs_from_subcollection(self, job_id: str, worker: str = None, limit: int = 500):
"""Get logs from subcollection."""
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
query = logs_ref.order_by("timestamp")
if worker:
from google.cloud.firestore_v1 import FieldFilter
query = query.where(filter=FieldFilter("worker", "==", worker))
query = query.limit(limit)
return [doc.to_dict() for doc in query.stream()]
def _count_logs_in_subcollection(self, job_id: str) -> int:
"""Count logs in subcollection."""
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
count_query = logs_ref.count()
result = count_query.get()
if result and len(result) > 0:
return result[0][0].value
return 0
def _delete_logs_subcollection(self, job_id: str, batch_size: int = 100) -> int:
"""Delete all logs in subcollection."""
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
deleted_count = 0
while True:
docs = list(logs_ref.limit(batch_size).stream())
if not docs:
break
batch = self.db.batch()
for doc in docs:
batch.delete(doc.reference)
deleted_count += 1
batch.commit()
return deleted_count
def test_subcollection_single_write(self):
"""Test writing single log to subcollection."""
job_id = self._create_test_job()
log_id = self._append_log_to_subcollection(job_id, "test", "Single log message")
time.sleep(0.1)
logs = self._get_logs_from_subcollection(job_id)
assert len(logs) == 1
assert logs[0]["message"] == "Single log message"
assert logs[0]["worker"] == "test"
assert logs[0]["id"] == log_id
print("Single write to subcollection works")
def test_subcollection_ttl_field_is_set(self):
"""Test TTL expiry field is set correctly."""
job_id = self._create_test_job()
self._append_log_to_subcollection(job_id, "test", "Log with TTL", ttl_days=7)
time.sleep(0.1)
logs = self._get_logs_from_subcollection(job_id)
assert len(logs) == 1
ttl_expiry = logs[0]["ttl_expiry"]
assert ttl_expiry is not None
# TTL should be ~7 days from now
expected_ttl = datetime.now(timezone.utc) + timedelta(days=7)
if hasattr(ttl_expiry, 'timestamp'):
# Firestore datetime object
diff = abs((ttl_expiry.replace(tzinfo=timezone.utc) - expected_ttl).total_seconds())
else:
diff = 0 # If already datetime
assert diff < 60, f"TTL expiry should be ~7 days from now, diff was {diff}s"
print("TTL field is set correctly")
def test_subcollection_sequential_writes(self):
"""Test multiple sequential writes to subcollection."""
job_id = self._create_test_job()
for i in range(20):
self._append_log_to_subcollection(job_id, "test", f"Log {i}")
time.sleep(0.2)
logs = self._get_logs_from_subcollection(job_id)
assert len(logs) == 20, f"Expected 20 logs, got {len(logs)}"
print(f"Sequential writes: {len(logs)} logs created")
def test_subcollection_concurrent_writes(self):
"""Test concurrent writes to subcollection - no race conditions."""
job_id = self._create_test_job()
num_writes = 50
def write_log(index):
self._append_log_to_subcollection(job_id, "worker", f"Concurrent Log {index}")
with ThreadPoolExecutor(max_workers=10) as executor:
futures = [executor.submit(write_log, i) for i in range(num_writes)]
for future in as_completed(futures):
future.result()
time.sleep(0.5)
logs = self._get_logs_from_subcollection(job_id)
assert len(logs) == num_writes, f"Expected {num_writes} logs, got {len(logs)}"
print(f"Concurrent writes: All {num_writes} logs preserved")
def test_subcollection_large_volume(self):
"""Test large volume of logs - demonstrates no 1MB limit."""
job_id = self._create_test_job()
num_logs = 500 # Would exceed 1MB in embedded array
# Write logs in batches
batch_size = 50
for batch_start in range(0, num_logs, batch_size):
with ThreadPoolExecutor(max_workers=10) as executor:
futures = []
for i in range(batch_start, min(batch_start + batch_size, num_logs)):
futures.append(executor.submit(
self._append_log_to_subcollection,
job_id,
"stress",
f"Large volume test log {i} - " + "x" * 500 # ~500 bytes per log
))
for future in as_completed(futures):
future.result()
time.sleep(1)
count = self._count_logs_in_subcollection(job_id)
assert count == num_logs, f"Expected {num_logs} logs, got {count}"
print(f"Large volume: {count} logs created (~{count * 500 / 1024:.1f}KB would exceed 1MB if embedded)")
def test_subcollection_filter_by_worker(self):
"""Test filtering logs by worker type."""
job_id = self._create_test_job()
# Add logs from different workers
for i in range(10):
self._append_log_to_subcollection(job_id, "audio", f"Audio log {i}")
self._append_log_to_subcollection(job_id, "lyrics", f"Lyrics log {i}")
self._append_log_to_subcollection(job_id, "video", f"Video log {i}")
time.sleep(0.3)
# Query by worker
audio_logs = self._get_logs_from_subcollection(job_id, worker="audio")
lyrics_logs = self._get_logs_from_subcollection(job_id, worker="lyrics")
all_logs = self._get_logs_from_subcollection(job_id)
assert len(audio_logs) == 10, f"Expected 10 audio logs, got {len(audio_logs)}"
assert len(lyrics_logs) == 10, f"Expected 10 lyrics logs, got {len(lyrics_logs)}"
assert len(all_logs) == 30, f"Expected 30 total logs, got {len(all_logs)}"
print("Worker filtering works correctly")
def test_subcollection_ordered_by_timestamp(self):
"""Test logs are returned in timestamp order."""
job_id = self._create_test_job()
# Add logs with slight delays to ensure different timestamps
for i in range(5):
self._append_log_to_subcollection(job_id, "test", f"Log {i}")
time.sleep(0.02)
logs = self._get_logs_from_subcollection(job_id)
# Verify order (timestamps should be ascending)
for i in range(len(logs) - 1):
assert logs[i]["timestamp"] <= logs[i + 1]["timestamp"], \
f"Logs not in order at index {i}"
print("Logs are ordered by timestamp")
def test_subcollection_delete_all_logs(self):
"""Test deleting all logs in subcollection."""
job_id = self._create_test_job()
# Add logs
for i in range(25):
self._append_log_to_subcollection(job_id, "test", f"Log {i}")
time.sleep(0.2)
count_before = self._count_logs_in_subcollection(job_id)
assert count_before == 25
# Delete all logs
deleted = self._delete_logs_subcollection(job_id)
time.sleep(0.2)
count_after = self._count_logs_in_subcollection(job_id)
assert deleted == 25, f"Expected to delete 25 logs, deleted {deleted}"
assert count_after == 0, f"Expected 0 logs after delete, got {count_after}"
print(f"Deleted {deleted} logs from subcollection")
def test_subcollection_concurrent_workers_interleaved(self):
"""Test simulating audio and lyrics workers writing concurrently."""
job_id = self._create_test_job()
logs_per_worker = 20
def audio_worker():
for i in range(logs_per_worker):
self._append_log_to_subcollection(job_id, "audio", f"Audio processing step {i}")
time.sleep(0.005)
def lyrics_worker():
for i in range(logs_per_worker):
self._append_log_to_subcollection(job_id, "lyrics", f"Lyrics processing step {i}")
time.sleep(0.005)
def video_worker():
for i in range(logs_per_worker):
self._append_log_to_subcollection(job_id, "video", f"Video encoding step {i}")
time.sleep(0.005)
# Start all workers
threads = [
threading.Thread(target=audio_worker),
threading.Thread(target=lyrics_worker),
threading.Thread(target=video_worker)
]
for t in threads:
t.start()
for t in threads:
t.join()
time.sleep(0.5)
audio_logs = self._get_logs_from_subcollection(job_id, worker="audio")
lyrics_logs = self._get_logs_from_subcollection(job_id, worker="lyrics")
video_logs = self._get_logs_from_subcollection(job_id, worker="video")
total = self._count_logs_in_subcollection(job_id)
assert len(audio_logs) == logs_per_worker
assert len(lyrics_logs) == logs_per_worker
assert len(video_logs) == logs_per_worker
assert total == logs_per_worker * 3
print(f"Concurrent workers: {len(audio_logs)} audio + {len(lyrics_logs)} lyrics + {len(video_logs)} video = {total}")
def test_subcollection_job_deletion_cleans_up_logs(self):
"""Test that deleting job document doesn't orphan logs."""
job_id = self._create_test_job()
# Add logs
for i in range(10):
self._append_log_to_subcollection(job_id, "test", f"Log {i}")
time.sleep(0.2)
assert self._count_logs_in_subcollection(job_id) == 10
# Delete logs first (must happen before parent doc deletion in real code)
deleted = self._delete_logs_subcollection(job_id)
assert deleted == 10
# Now delete job document
self.db.collection(self.collection).document(job_id).delete()
# Verify subcollection is empty (Firestore doesn't cascade delete subcollections)
time.sleep(0.2)
assert self._count_logs_in_subcollection(job_id) == 0
print("Job deletion with log cleanup works correctly")
class TestSubcollectionVsEmbeddedArray:
"""Compare subcollection approach with embedded array."""
@pytest.fixture(autouse=True)
def setup_firestore(self):
"""Set up Firestore client for each test."""
from google.cloud import firestore
self.db = firestore.Client(project="test-project")
self.collection = "test-comparison-jobs"
yield
def _create_test_job(self, use_array: bool):
"""Create test job."""
job_id = f"test-cmp-{int(time.time() * 1000)}-{'arr' if use_array else 'sub'}"
doc_ref = self.db.collection(self.collection).document(job_id)
doc_ref.set({
"job_id": job_id,
"status": "pending",
"created_at": datetime.now(timezone.utc),
"worker_logs": []
})
return job_id
def _append_log_array(self, job_id: str, message: str):
"""Add log to embedded array."""
from google.cloud import firestore
doc_ref = self.db.collection(self.collection).document(job_id)
doc_ref.update({
"worker_logs": firestore.ArrayUnion([{
"timestamp": datetime.now(timezone.utc).isoformat() + "Z",
"level": "INFO",
"worker": "test",
"message": message
}])
})
def _append_log_subcollection(self, job_id: str, message: str):
"""Add log to subcollection."""
import uuid
now = datetime.now(timezone.utc)
logs_ref = self.db.collection(self.collection).document(job_id).collection("logs")
logs_ref.document(str(uuid.uuid4())).set({
"timestamp": now,
"level": "INFO",
"worker": "test",
"message": message,
"ttl_expiry": now + timedelta(days=30)
})
def test_document_size_comparison(self):
"""Compare document sizes between approaches."""
array_job_id = self._create_test_job(use_array=True)
sub_job_id = self._create_test_job(use_array=False)
num_logs = 100
large_message = "x" * 5000 # 5KB per log
# Add logs to both
print("\nAdding logs to both approaches...")
for i in range(num_logs):
self._append_log_array(array_job_id, f"Array log {i}: {large_message}")
self._append_log_subcollection(sub_job_id, f"Sub log {i}: {large_message}")
time.sleep(0.5)
# Get document sizes (approximate via raw data)
array_doc = self.db.collection(self.collection).document(array_job_id).get()
sub_doc = self.db.collection(self.collection).document(sub_job_id).get()
array_data = array_doc.to_dict()
sub_data = sub_doc.to_dict()
array_logs = array_data.get("worker_logs", [])
sub_logs = sub_data.get("worker_logs", [])
print(f"\nEmbedded array: {len(array_logs)} logs in document")
print(f"Subcollection: {len(sub_logs)} logs in document (0 expected)")
# Embedded array should have all logs in document
assert len(array_logs) == num_logs
# Subcollection should have empty array in main document
assert len(sub_logs) == 0
# Subcollection logs are in subcollection
sub_count = self.db.collection(self.collection).document(sub_job_id).collection("logs").count().get()[0][0].value
assert sub_count == num_logs
print(f"Subcollection logs stored separately: {sub_count}")
print("Subcollection approach keeps main document small")
print("Worker logs subcollection integration tests ready")
-352
View File
@@ -1,352 +0,0 @@
"""
Unit tests for admin delete job outputs endpoint.
Tests the POST /api/admin/jobs/{job_id}/delete-outputs endpoint that allows admins
to delete distributed outputs (YouTube, Dropbox, Google Drive) while preserving
the job record.
"""
import pytest
from unittest.mock import Mock, patch, MagicMock
from fastapi.testclient import TestClient
from fastapi import FastAPI
from datetime import datetime
from backend.api.routes.admin import router
from backend.api.dependencies import require_admin
from backend.models.job import Job, JobStatus
# Create a test app with the admin router
app = FastAPI()
app.include_router(router, prefix="/api")
def get_mock_admin():
"""Override for require_admin dependency."""
from backend.api.dependencies import AuthResult, UserType
return AuthResult(
is_valid=True,
user_type=UserType.ADMIN,
remaining_uses=999,
message="Admin authenticated",
user_email="admin@example.com",
is_admin=True,
)
# Override the require_admin dependency
app.dependency_overrides[require_admin] = get_mock_admin
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
@pytest.fixture
def mock_complete_job():
"""Create a mock job in COMPLETE status with distribution data."""
job = Mock(spec=Job)
job.job_id = "test-job-123"
job.user_email = "user@example.com"
job.artist = "Test Artist"
job.title = "Test Title"
job.status = "complete"
job.dropbox_path = "/Karaoke/Organized"
job.outputs_deleted_at = None
job.outputs_deleted_by = None
job.state_data = {
"youtube_url": "https://youtu.be/abc123",
"brand_code": "NOMAD-1234",
"dropbox_link": "https://dropbox.com/...",
"gdrive_files": {"mp4": "file_id_1", "mp4_720p": "file_id_2"},
}
job.timeline = []
return job
@pytest.fixture
def mock_job_no_outputs():
"""Create a mock job in COMPLETE status without distribution data."""
job = Mock(spec=Job)
job.job_id = "test-job-456"
job.user_email = "user@example.com"
job.artist = "Test Artist"
job.title = "Test Title"
job.status = "complete"
job.dropbox_path = None
job.outputs_deleted_at = None
job.outputs_deleted_by = None
job.state_data = {}
job.timeline = []
return job
class TestDeleteOutputsSuccess:
"""Tests for successful output deletion."""
def test_delete_outputs_success(self, client, mock_complete_job):
"""Test successfully deleting outputs from a complete job."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
# Mock Firestore
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
assert data["job_id"] == "test-job-123"
assert "outputs_deleted_at" in data
def test_delete_outputs_clears_state_data_keys(self, client, mock_complete_job):
"""Test that output-related state_data keys are listed as cleared."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
# Mock Firestore
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 200
data = response.json()
# Should list cleared keys
assert "youtube_url" in data["cleared_state_data"]
assert "brand_code" in data["cleared_state_data"]
assert "dropbox_link" in data["cleared_state_data"]
assert "gdrive_files" in data["cleared_state_data"]
def test_delete_outputs_job_without_distribution(self, client, mock_job_no_outputs):
"""Test deleting outputs from job that has no distribution data."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_no_outputs
mock_jm_class.return_value = mock_jm
# Mock Firestore
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-456/delete-outputs")
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
# All services should be skipped
assert data["deleted_services"]["youtube"]["status"] == "skipped"
assert data["deleted_services"]["dropbox"]["status"] == "skipped"
assert data["deleted_services"]["gdrive"]["status"] == "skipped"
# No keys to clear
assert data["cleared_state_data"] == []
class TestDeleteOutputsValidation:
"""Tests for validation on delete outputs endpoint."""
def test_rejects_non_terminal_status(self, client, mock_complete_job):
"""Test that jobs not in terminal states are rejected."""
mock_complete_job.status = "encoding"
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 400
assert "terminal" in response.json()["detail"].lower()
def test_rejects_awaiting_review_status(self, client, mock_complete_job):
"""Test that AWAITING_REVIEW is rejected (not terminal)."""
mock_complete_job.status = "awaiting_review"
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 400
def test_accepts_prep_complete_status(self, client, mock_complete_job):
"""Test that prep_complete is accepted as terminal."""
mock_complete_job.status = "prep_complete"
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 200
def test_accepts_failed_status(self, client, mock_complete_job):
"""Test that failed is accepted as terminal."""
mock_complete_job.status = "failed"
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 200
def test_rejects_already_deleted(self, client, mock_complete_job):
"""Test that already-deleted outputs cannot be deleted again."""
mock_complete_job.outputs_deleted_at = datetime(2026, 1, 9, 12, 0, 0)
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 400
assert "already deleted" in response.json()["detail"].lower()
def test_returns_404_for_missing_job(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.post("/api/admin/jobs/nonexistent/delete-outputs")
assert response.status_code == 404
assert "not found" in response.json()["detail"].lower()
class TestDeleteOutputsServices:
"""Tests for service deletion behavior."""
def test_handles_missing_youtube(self, client, mock_complete_job):
"""Test handling when job has no YouTube URL."""
mock_complete_job.state_data = {"brand_code": "NOMAD-1234"}
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 200
data = response.json()
assert data["deleted_services"]["youtube"]["status"] == "skipped"
def test_handles_missing_dropbox_path(self, client, mock_complete_job):
"""Test handling when job has no dropbox_path."""
mock_complete_job.dropbox_path = None
mock_complete_job.state_data = {"youtube_url": "https://youtu.be/abc123"}
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 200
data = response.json()
assert data["deleted_services"]["dropbox"]["status"] == "skipped"
class TestDeleteOutputsLogging:
"""Tests for logging on delete outputs endpoint."""
def test_logs_admin_action(self, client, mock_complete_job):
"""Test that admin delete action is logged."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.get_user_service') as mock_user_service, \
patch('backend.api.routes.admin.logger') as mock_logger:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_complete_job
mock_jm_class.return_value = mock_jm
mock_db = Mock()
mock_job_ref = Mock()
mock_db.collection.return_value.document.return_value = mock_job_ref
mock_user_service.return_value.db = mock_db
response = client.post("/api/admin/jobs/test-job-123/delete-outputs")
assert response.status_code == 200
mock_logger.info.assert_called()
log_message = mock_logger.info.call_args[0][0]
assert "admin" in log_message.lower() or "deleted" in log_message.lower()
class TestDeleteOutputsAuthorization:
"""Tests for authorization on the delete outputs endpoint."""
def test_requires_admin_access(self, client, mock_complete_job):
"""Test that non-admin users cannot access the endpoint."""
original_override = app.dependency_overrides.get(require_admin)
def get_non_admin():
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Admin access required")
app.dependency_overrides[require_admin] = get_non_admin
try:
response = client.post(
"/api/admin/jobs/test-job-123/delete-outputs",
headers={"Authorization": "Bearer user-token"}
)
assert response.status_code == 403
finally:
if original_override:
app.dependency_overrides[require_admin] = original_override
else:
app.dependency_overrides[require_admin] = get_mock_admin
-411
View File
@@ -1,411 +0,0 @@
"""
Unit tests for admin email endpoints.
Tests the completion message and send email API endpoints.
"""
import pytest
from unittest.mock import Mock, patch, MagicMock, AsyncMock
from fastapi.testclient import TestClient
from fastapi import FastAPI
from backend.api.routes.admin import router
from backend.api.dependencies import require_admin
from backend.models.job import Job, JobStatus
# Create a test app with the admin router
# The router already has prefix="/admin", so we add /api prefix
app = FastAPI()
app.include_router(router, prefix="/api")
def get_mock_admin():
"""Override for require_admin dependency."""
return ("admin@example.com", "admin", 1)
# Override the require_admin dependency
app.dependency_overrides[require_admin] = get_mock_admin
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
@pytest.fixture
def mock_job():
"""Create a mock job."""
job = Mock(spec=Job)
job.job_id = "test-job-123"
job.user_email = "user@example.com"
job.artist = "Test Artist"
job.title = "Test Song"
job.status = JobStatus.COMPLETE
job.audio_hash = "hash123"
job.review_token = "review123"
job.instrumental_token = "inst123"
job.state_data = {
"youtube_url": "https://youtube.com/watch?v=test123",
"dropbox_link": "https://dropbox.com/folder/test",
"brand_code": "NOMAD-1234",
}
return job
class TestGetCompletionMessage:
"""Tests for GET /api/admin/jobs/{job_id}/completion-message endpoint."""
def test_returns_completion_message(self, client, mock_job):
"""Test successful completion message retrieval."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
# Setup mocks
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Your video is ready!"
mock_get_ns.return_value = mock_ns
response = client.get(
"/api/admin/jobs/test-job-123/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["job_id"] == "test-job-123"
assert data["message"] == "Your video is ready!"
# Subject format: "NOMAD-1234: Test Artist - Test Song (Your karaoke video is ready!)"
assert data["subject"] == "NOMAD-1234: Test Artist - Test Song (Your karaoke video is ready!)"
assert data["youtube_url"] == "https://youtube.com/watch?v=test123"
assert data["dropbox_url"] == "https://dropbox.com/folder/test"
def test_returns_404_when_job_not_found(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.get(
"/api/admin/jobs/nonexistent-job/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 404
def test_handles_none_state_data(self, client, mock_job):
"""Test handling of job with None state_data."""
mock_job.state_data = None
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Your video is ready!"
mock_get_ns.return_value = mock_ns
response = client.get(
"/api/admin/jobs/test-job-123/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["youtube_url"] is None
assert data["dropbox_url"] is None
def test_default_subject_without_song_info(self, client, mock_job):
"""Test default subject when no artist/title."""
mock_job.artist = None
mock_job.title = None
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
response = client.get(
"/api/admin/jobs/test-job-123/completion-message",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["subject"] == "Your karaoke video is ready!"
class TestSendCompletionEmail:
"""Tests for POST /api/admin/jobs/{job_id}/send-completion-email endpoint."""
def test_sends_email_successfully(self, client, mock_job):
"""Test successful email sending."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Your video is ready!"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com", "cc_admin": True},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["success"] is True
assert data["job_id"] == "test-job-123"
assert data["to_email"] == "customer@example.com"
# Verify email was sent with correct params
mock_es.send_job_completion.assert_called_once_with(
to_email="customer@example.com",
message_content="Your video is ready!",
artist="Test Artist",
title="Test Song",
brand_code="NOMAD-1234",
cc_admin=True,
)
def test_sends_email_without_cc(self, client, mock_job):
"""Test email sending without CC."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com", "cc_admin": False},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
# Verify CC was not included
call_kwargs = mock_es.send_job_completion.call_args.kwargs
assert call_kwargs["cc_admin"] is False
def test_returns_404_when_job_not_found(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/nonexistent-job/send-completion-email",
json={"to_email": "customer@example.com"},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 404
def test_returns_500_when_email_fails(self, client, mock_job):
"""Test 500 when email sending fails."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = False # Email failed
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com"},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 500
assert "Failed to send email" in response.json()["detail"]
def test_handles_none_state_data(self, client, mock_job):
"""Test handling of job with None state_data."""
mock_job.state_data = None
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "customer@example.com"},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
def test_accepts_any_email_string(self, client, mock_job):
"""Test that any string is currently accepted as email (no validation).
Documents current behavior: FastAPI/Pydantic doesn't validate email
format by default. If validation is added later, this test should
be updated to expect 422 for invalid emails.
"""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns, \
patch('backend.services.email_service.get_email_service') as mock_get_es:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.get_completion_message.return_value = "Message"
mock_get_ns.return_value = mock_ns
mock_es = Mock()
mock_es.send_job_completion.return_value = True
mock_get_es.return_value = mock_es
response = client.post(
"/api/admin/jobs/test-job-123/send-completion-email",
json={"to_email": "invalid-email"}, # Not a valid email format
headers={"Authorization": "Bearer admin-token"}
)
# Currently accepts any string - endpoint succeeds
assert response.status_code == 200
# Verify the invalid email was passed through
mock_es.send_job_completion.assert_called_once()
call_kwargs = mock_es.send_job_completion.call_args.kwargs
assert call_kwargs["to_email"] == "invalid-email"
class TestIdleReminderEndpoint:
"""Tests for the internal idle reminder endpoint."""
def test_sends_reminder_when_still_idle(self, mock_job):
"""Test that reminder is sent when user is still idle."""
from backend.api.routes.internal import router as internal_router
from backend.api.dependencies import require_admin
# Create test app for internal router
# The router already has prefix="/internal", so we add /api prefix
test_app = FastAPI()
test_app.include_router(internal_router, prefix="/api")
test_app.dependency_overrides[require_admin] = get_mock_admin
mock_job.status = JobStatus.AWAITING_REVIEW.value # Use .value for string comparison
mock_job.state_data = {
"blocking_state_entered_at": "2024-01-01T00:00:00",
"blocking_action_type": "lyrics",
"reminder_sent": False,
}
with patch('backend.api.routes.internal.JobManager') as mock_jm_class, \
patch('backend.services.job_notification_service.get_job_notification_service') as mock_get_ns:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.firestore = Mock() # Mock firestore for update_job call
mock_jm_class.return_value = mock_jm
mock_ns = Mock()
mock_ns.send_action_reminder_email = AsyncMock(return_value=True)
mock_get_ns.return_value = mock_ns
test_client = TestClient(test_app)
response = test_client.post(
"/api/internal/jobs/test-job-123/check-idle-reminder",
headers={"Authorization": "Bearer admin-token"}
)
# Should succeed and return "sent" status
assert response.status_code == 200
data = response.json()
assert data["status"] == "sent"
assert data["job_id"] == "test-job-123"
def test_skips_reminder_when_already_sent(self, mock_job):
"""Test that reminder is skipped if already sent."""
from backend.api.routes.internal import router as internal_router
from backend.api.dependencies import require_admin
# Create test app for internal router
# The router already has prefix="/internal", so we add /api prefix
test_app = FastAPI()
test_app.include_router(internal_router, prefix="/api")
test_app.dependency_overrides[require_admin] = get_mock_admin
mock_job.status = JobStatus.AWAITING_REVIEW.value # Use .value for string comparison
mock_job.state_data = {
"blocking_state_entered_at": "2024-01-01T00:00:00",
"blocking_action_type": "lyrics",
"reminder_sent": True, # Already sent
}
with patch('backend.api.routes.internal.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
test_client = TestClient(test_app)
response = test_client.post(
"/api/internal/jobs/test-job-123/check-idle-reminder",
headers={"Authorization": "Bearer admin-token"}
)
# Should succeed with "already_sent" status
assert response.status_code == 200
data = response.json()
assert data["status"] == "already_sent"
assert data["job_id"] == "test-job-123"
-337
View File
@@ -1,337 +0,0 @@
"""
Unit tests for admin job files endpoint.
Tests the GET /api/admin/jobs/{job_id}/files endpoint that returns
all files for a job with signed download URLs.
"""
import pytest
from unittest.mock import Mock, patch
from fastapi.testclient import TestClient
from fastapi import FastAPI
from backend.api.routes.admin import router
from backend.api.dependencies import require_admin
from backend.models.job import Job, JobStatus
# Create a test app with the admin router
app = FastAPI()
app.include_router(router, prefix="/api")
def get_mock_admin():
"""Override for require_admin dependency."""
from backend.api.dependencies import AuthResult, UserType
return AuthResult(
is_valid=True,
user_type=UserType.ADMIN,
remaining_uses=999,
message="Admin authenticated",
user_email="admin@example.com",
is_admin=True,
)
# Override the require_admin dependency
app.dependency_overrides[require_admin] = get_mock_admin
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
@pytest.fixture
def mock_job_with_files():
"""Create a mock job with comprehensive file_urls."""
job = Mock(spec=Job)
job.job_id = "test-job-123"
job.user_email = "user@example.com"
job.artist = "Test Artist"
job.title = "Test Song"
job.status = JobStatus.COMPLETE
job.file_urls = {
"input": "gs://bucket/jobs/test-job-123/input.flac",
"stems": {
"instrumental_clean": "gs://bucket/jobs/test-job-123/stems/instrumental_clean.flac",
"vocals": "gs://bucket/jobs/test-job-123/stems/vocals.flac",
},
"lyrics": {
"corrections": "gs://bucket/jobs/test-job-123/lyrics/corrections.json",
"lrc": "gs://bucket/jobs/test-job-123/lyrics/output.lrc",
},
"finals": {
"lossy_720p_mp4": "gs://bucket/jobs/test-job-123/finals/video_720p.mp4",
},
"youtube": {
"url": "https://youtube.com/watch?v=xyz789",
"video_id": "xyz789",
},
}
return job
@pytest.fixture
def mock_job_no_files():
"""Create a mock job with empty file_urls."""
job = Mock(spec=Job)
job.job_id = "test-job-empty"
job.user_email = "user@example.com"
job.artist = "Empty Artist"
job.title = "Empty Song"
job.status = JobStatus.PENDING
job.file_urls = {}
return job
@pytest.fixture
def mock_job_partial_files():
"""Create a mock job with only some files (partial processing)."""
job = Mock(spec=Job)
job.job_id = "test-job-partial"
job.user_email = "user@example.com"
job.artist = "Partial Artist"
job.title = "Partial Song"
job.status = JobStatus.SEPARATING_STAGE1
job.file_urls = {
"input": "gs://bucket/jobs/test-job-partial/input.flac",
"stems": {
"instrumental_clean": "gs://bucket/jobs/test-job-partial/stems/instrumental_clean.flac",
},
}
return job
class TestGetJobFiles:
"""Tests for GET /api/admin/jobs/{job_id}/files endpoint."""
def test_returns_files_with_signed_urls(self, client, mock_job_with_files):
"""Test that endpoint returns all files with signed URLs."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
# Setup JobManager mock
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_with_files
mock_jm_class.return_value = mock_jm
# Setup StorageService mock to return predictable signed URLs
mock_storage = Mock()
mock_storage.generate_signed_url.side_effect = lambda path, **kwargs: f"https://signed.url/{path}"
mock_storage_class.return_value = mock_storage
response = client.get(
"/api/admin/jobs/test-job-123/files",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["job_id"] == "test-job-123"
assert data["artist"] == "Test Artist"
assert data["title"] == "Test Song"
assert "files" in data
assert data["total_files"] > 0
def test_returns_correct_file_count(self, client, mock_job_with_files):
"""Test that file count matches actual GCS files (excluding non-GCS entries)."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_with_files
mock_jm_class.return_value = mock_jm
mock_storage = Mock()
mock_storage.generate_signed_url.side_effect = lambda path, **kwargs: f"https://signed.url/{path}"
mock_storage_class.return_value = mock_storage
response = client.get("/api/admin/jobs/test-job-123/files")
data = response.json()
# Should have: input, stems.instrumental_clean, stems.vocals,
# lyrics.corrections, lyrics.lrc, finals.lossy_720p_mp4
# NOT youtube.url or youtube.video_id (not GCS paths)
assert data["total_files"] == 6
def test_handles_nested_file_structure(self, client, mock_job_with_files):
"""Test that nested file_urls structure is properly traversed."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_with_files
mock_jm_class.return_value = mock_jm
mock_storage = Mock()
mock_storage.generate_signed_url.side_effect = lambda path, **kwargs: f"https://signed.url/{path}"
mock_storage_class.return_value = mock_storage
response = client.get("/api/admin/jobs/test-job-123/files")
data = response.json()
files = data["files"]
# Check that files from different categories are present
categories = {f["category"] for f in files}
assert "input" in categories or any(f["category"] == "" and f["file_key"] == "input" for f in files)
assert "stems" in categories
assert "lyrics" in categories
assert "finals" in categories
def test_file_info_structure(self, client, mock_job_with_files):
"""Test that each file has correct info structure."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_with_files
mock_jm_class.return_value = mock_jm
mock_storage = Mock()
mock_storage.generate_signed_url.side_effect = lambda path, **kwargs: f"https://signed.url/{path}"
mock_storage_class.return_value = mock_storage
response = client.get("/api/admin/jobs/test-job-123/files")
data = response.json()
files = data["files"]
# Every file should have these fields
for file_info in files:
assert "name" in file_info
assert "path" in file_info
assert "download_url" in file_info
assert "category" in file_info
assert "file_key" in file_info
# download_url should be a signed URL
assert file_info["download_url"].startswith("https://signed.url/")
def test_returns_404_when_job_not_found(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.get(
"/api/admin/jobs/nonexistent-job/files",
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 404
assert "not found" in response.json()["detail"].lower()
def test_returns_empty_files_for_new_job(self, client, mock_job_no_files):
"""Test that new jobs with no files return empty list."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_no_files
mock_jm_class.return_value = mock_jm
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
response = client.get("/api/admin/jobs/test-job-empty/files")
assert response.status_code == 200
data = response.json()
assert data["job_id"] == "test-job-empty"
assert data["files"] == []
assert data["total_files"] == 0
def test_handles_partial_processing(self, client, mock_job_partial_files):
"""Test job with only some files processed."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_partial_files
mock_jm_class.return_value = mock_jm
mock_storage = Mock()
mock_storage.generate_signed_url.side_effect = lambda path, **kwargs: f"https://signed.url/{path}"
mock_storage_class.return_value = mock_storage
response = client.get("/api/admin/jobs/test-job-partial/files")
assert response.status_code == 200
data = response.json()
# Should have input + stems.instrumental_clean = 2 files
assert data["total_files"] == 2
def test_skips_non_gcs_urls(self, client, mock_job_with_files):
"""Test that non-GCS URLs (like youtube URLs) are skipped."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_with_files
mock_jm_class.return_value = mock_jm
mock_storage = Mock()
mock_storage.generate_signed_url.side_effect = lambda path, **kwargs: f"https://signed.url/{path}"
mock_storage_class.return_value = mock_storage
response = client.get("/api/admin/jobs/test-job-123/files")
data = response.json()
files = data["files"]
# Ensure no youtube URLs or video IDs are in the files
for file_info in files:
assert "youtube.com" not in file_info["path"]
assert file_info["path"].startswith("gs://")
def test_signed_url_expiration(self, client, mock_job_with_files):
"""Test that signed URLs are generated with appropriate expiration."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.StorageService') as mock_storage_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job_with_files
mock_jm_class.return_value = mock_jm
mock_storage = Mock()
mock_storage.generate_signed_url.return_value = "https://signed.url/test"
mock_storage_class.return_value = mock_storage
client.get("/api/admin/jobs/test-job-123/files")
# Verify signed URLs were requested with expiration (default 120 minutes)
assert mock_storage.generate_signed_url.called
# Check that expiration_minutes was passed (could be any reasonable value)
call_kwargs = mock_storage.generate_signed_url.call_args_list[0][1]
if "expiration_minutes" in call_kwargs:
assert call_kwargs["expiration_minutes"] >= 60
class TestGetJobFilesAuthorization:
"""Tests for authorization on the files endpoint."""
def test_requires_admin_access(self, client, mock_job_with_files):
"""Test that non-admin users cannot access the endpoint."""
# Reset the dependency override to test auth
original_override = app.dependency_overrides.get(require_admin)
def get_non_admin():
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Admin access required")
app.dependency_overrides[require_admin] = get_non_admin
try:
response = client.get(
"/api/admin/jobs/test-job-123/files",
headers={"Authorization": "Bearer user-token"}
)
assert response.status_code == 403
finally:
# Restore the original override
if original_override:
app.dependency_overrides[require_admin] = original_override
else:
app.dependency_overrides[require_admin] = get_mock_admin
-384
View File
@@ -1,384 +0,0 @@
"""
Unit tests for admin job reset endpoint.
Tests the POST /api/admin/jobs/{job_id}/reset endpoint that allows admins
to reset a job to a specific state for re-processing.
"""
import pytest
from unittest.mock import Mock, patch
from fastapi.testclient import TestClient
from fastapi import FastAPI
from datetime import datetime
from backend.api.routes.admin import router
from backend.api.dependencies import require_admin
from backend.models.job import Job, JobStatus
# Create a test app with the admin router
app = FastAPI()
app.include_router(router, prefix="/api")
def get_mock_admin():
"""Override for require_admin dependency."""
from backend.api.dependencies import AuthResult, UserType
return AuthResult(
is_valid=True,
user_type=UserType.ADMIN,
remaining_uses=999,
message="Admin authenticated",
user_email="admin@example.com",
is_admin=True,
)
# Override the require_admin dependency
app.dependency_overrides[require_admin] = get_mock_admin
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
@pytest.fixture
def mock_job():
"""Create a mock job in COMPLETE status for testing."""
job = Mock(spec=Job)
job.job_id = "test-job-123"
job.user_email = "user@example.com"
job.artist = "Test Artist"
job.title = "Test Title"
job.status = JobStatus.COMPLETE
job.theme_id = "nomad"
job.file_urls = {
"input": "gs://bucket/input.flac",
"stems": {
"instrumental_clean": "gs://bucket/stems/clean.flac",
},
"lyrics": {
"corrections": "gs://bucket/lyrics/corrections.json",
},
}
job.state_data = {
"audio_search_results": [{"provider": "test"}],
"audio_selection": {"index": 0},
"review_complete": True,
"corrected_lyrics": {"lines": []},
"instrumental_selection": "clean",
"brand_code": "NOMAD-1234",
}
job.timeline = []
job.created_at = datetime(2026, 1, 9, 10, 0, 0)
job.updated_at = datetime(2026, 1, 9, 12, 0, 0)
return job
class TestResetJobToPending:
"""Tests for resetting job to PENDING state."""
def test_reset_to_pending_success(self, client, mock_job):
"""Test resetting a job to PENDING state."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "pending"},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
assert data["job_id"] == "test-job-123"
assert data["new_status"] == "pending"
assert "pending" in data["message"].lower()
def test_reset_to_pending_clears_state_data(self, client, mock_job):
"""Test that resetting to PENDING clears appropriate state_data keys."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "pending"},
)
assert response.status_code == 200
# Verify update_job was called with cleared state_data keys
call_args = mock_jm.update_job.call_args
update_data = call_args[0][1]
# Verify status was set
assert update_data["status"] == "pending"
class TestResetJobToAwaitingAudioSelection:
"""Tests for resetting job to AWAITING_AUDIO_SELECTION state."""
def test_reset_to_awaiting_audio_selection(self, client, mock_job):
"""Test resetting a job to AWAITING_AUDIO_SELECTION state."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "awaiting_audio_selection"},
)
assert response.status_code == 200
data = response.json()
assert data["new_status"] == "awaiting_audio_selection"
class TestResetJobToAwaitingReview:
"""Tests for resetting job to AWAITING_REVIEW state."""
def test_reset_to_awaiting_review(self, client, mock_job):
"""Test resetting a job to AWAITING_REVIEW state."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "awaiting_review"},
)
assert response.status_code == 200
data = response.json()
assert data["new_status"] == "awaiting_review"
class TestResetJobToAwaitingInstrumentalSelection:
"""Tests for resetting job to AWAITING_INSTRUMENTAL_SELECTION state."""
def test_reset_to_awaiting_instrumental_selection(self, client, mock_job):
"""Test resetting a job to AWAITING_INSTRUMENTAL_SELECTION state."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "awaiting_instrumental_selection"},
)
assert response.status_code == 200
data = response.json()
assert data["new_status"] == "awaiting_instrumental_selection"
class TestResetJobValidation:
"""Tests for reset endpoint validation."""
def test_rejects_invalid_target_state(self, client, mock_job):
"""Test that invalid target states are rejected."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "encoding"},
)
assert response.status_code == 400
assert "invalid" in response.json()["detail"].lower() or "not allowed" in response.json()["detail"].lower()
def test_rejects_complete_as_target(self, client, mock_job):
"""Test that COMPLETE is not a valid reset target."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "complete"},
)
assert response.status_code == 400
def test_rejects_failed_as_target(self, client, mock_job):
"""Test that FAILED is not a valid reset target."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "failed"},
)
assert response.status_code == 400
def test_returns_404_when_job_not_found(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/nonexistent-job/reset",
json={"target_state": "pending"},
)
assert response.status_code == 404
assert "not found" in response.json()["detail"].lower()
def test_rejects_missing_target_state(self, client, mock_job):
"""Test that target_state is required."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={},
)
assert response.status_code == 422 # Validation error
class TestResetJobLogging:
"""Tests for logging on the reset endpoint."""
def test_logs_admin_reset_action(self, client, mock_job):
"""Test that admin reset actions are logged."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.logger') as mock_logger:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "pending"},
)
assert response.status_code == 200
# Verify logging was called
mock_logger.info.assert_called()
log_message = mock_logger.info.call_args[0][0]
assert "admin" in log_message.lower() or "reset" in log_message.lower()
class TestResetJobTimeline:
"""Tests for timeline event on reset."""
def test_adds_timeline_event_on_reset(self, client, mock_job):
"""Test that a timeline event is added when job is reset."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "awaiting_review"},
)
assert response.status_code == 200
data = response.json()
# Timeline event should be included in response
assert "timeline_event" in data or data.get("message")
class TestResetJobAuthorization:
"""Tests for authorization on the reset endpoint."""
def test_requires_admin_access(self, client, mock_job):
"""Test that non-admin users cannot access the endpoint."""
original_override = app.dependency_overrides.get(require_admin)
def get_non_admin():
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Admin access required")
app.dependency_overrides[require_admin] = get_non_admin
try:
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "pending"},
headers={"Authorization": "Bearer user-token"}
)
assert response.status_code == 403
finally:
if original_override:
app.dependency_overrides[require_admin] = original_override
else:
app.dependency_overrides[require_admin] = get_mock_admin
class TestResetJobClearsStateData:
"""Tests for state_data clearing based on target state."""
def test_reset_to_pending_clears_all_processing_data(self, client, mock_job):
"""Test that resetting to PENDING clears audio search results."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "pending"},
)
assert response.status_code == 200
# The cleared_data field should list what was cleared
data = response.json()
assert "cleared_data" in data or "state_data" in str(data)
def test_reset_to_awaiting_review_preserves_audio_data(self, client, mock_job):
"""Test that resetting to AWAITING_REVIEW preserves audio stems."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "awaiting_review"},
)
assert response.status_code == 200
def test_reset_to_instrumental_preserves_review_data(self, client, mock_job):
"""Test that resetting to AWAITING_INSTRUMENTAL_SELECTION preserves review data."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.post(
"/api/admin/jobs/test-job-123/reset",
json={"target_state": "awaiting_instrumental_selection"},
)
assert response.status_code == 200
-326
View File
@@ -1,326 +0,0 @@
"""
Unit tests for admin job update endpoint.
Tests the PATCH /api/admin/jobs/{job_id} endpoint that allows admins
to update editable job fields.
"""
import pytest
from unittest.mock import Mock, patch
from fastapi.testclient import TestClient
from fastapi import FastAPI
from backend.api.routes.admin import router
from backend.api.dependencies import require_admin
from backend.models.job import Job, JobStatus
# Create a test app with the admin router
app = FastAPI()
app.include_router(router, prefix="/api")
def get_mock_admin():
"""Override for require_admin dependency."""
from backend.api.dependencies import AuthResult, UserType
return AuthResult(
is_valid=True,
user_type=UserType.ADMIN,
remaining_uses=999,
message="Admin authenticated",
user_email="admin@example.com",
is_admin=True,
)
# Override the require_admin dependency
app.dependency_overrides[require_admin] = get_mock_admin
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
@pytest.fixture
def mock_job():
"""Create a mock job for testing."""
job = Mock(spec=Job)
job.job_id = "test-job-123"
job.user_email = "user@example.com"
job.artist = "Original Artist"
job.title = "Original Title"
job.status = JobStatus.AWAITING_REVIEW
job.theme_id = "nomad"
job.enable_cdg = False
job.enable_txt = False
job.enable_youtube_upload = True
job.customer_email = None
job.customer_notes = None
job.brand_prefix = None
job.non_interactive = False
job.prep_only = False
job.discord_webhook_url = None
job.youtube_description = None
job.youtube_description_template = None
job.file_urls = {}
job.state_data = {}
job.created_at = "2026-01-09T10:00:00Z"
job.updated_at = "2026-01-09T10:30:00Z"
return job
class TestUpdateJob:
"""Tests for PATCH /api/admin/jobs/{job_id} endpoint."""
def test_update_artist_field(self, client, mock_job):
"""Test updating the artist field."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"artist": "New Artist"},
headers={"Authorization": "Bearer admin-token"}
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
assert data["job_id"] == "test-job-123"
assert "artist" in data["updated_fields"]
# Verify update_job was called with correct args
mock_jm.update_job.assert_called_once()
call_args = mock_jm.update_job.call_args
assert call_args[0][0] == "test-job-123"
assert call_args[0][1]["artist"] == "New Artist"
def test_update_multiple_fields(self, client, mock_job):
"""Test updating multiple fields at once."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={
"artist": "New Artist",
"title": "New Title",
"theme_id": "default",
"enable_cdg": True,
},
)
assert response.status_code == 200
data = response.json()
assert len(data["updated_fields"]) == 4
assert "artist" in data["updated_fields"]
assert "title" in data["updated_fields"]
assert "theme_id" in data["updated_fields"]
assert "enable_cdg" in data["updated_fields"]
def test_update_user_email(self, client, mock_job):
"""Test updating user_email field."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"user_email": "newuser@example.com"},
)
assert response.status_code == 200
call_args = mock_jm.update_job.call_args
assert call_args[0][1]["user_email"] == "newuser@example.com"
def test_update_boolean_fields(self, client, mock_job):
"""Test updating boolean fields."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={
"enable_cdg": True,
"enable_txt": True,
"enable_youtube_upload": False,
"non_interactive": True,
"prep_only": True,
},
)
assert response.status_code == 200
call_args = mock_jm.update_job.call_args
assert call_args[0][1]["enable_cdg"] is True
assert call_args[0][1]["enable_txt"] is True
assert call_args[0][1]["enable_youtube_upload"] is False
def test_rejects_non_editable_job_id(self, client, mock_job):
"""Test that job_id cannot be updated."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"job_id": "new-job-id"},
)
assert response.status_code == 400
assert "not editable" in response.json()["detail"].lower()
def test_rejects_non_editable_created_at(self, client, mock_job):
"""Test that created_at cannot be updated."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"created_at": "2025-01-01T00:00:00Z"},
)
assert response.status_code == 400
assert "not editable" in response.json()["detail"].lower()
def test_rejects_status_update(self, client, mock_job):
"""Test that status cannot be updated via PATCH (use reset endpoint)."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"status": "complete"},
)
assert response.status_code == 400
assert "not editable" in response.json()["detail"].lower()
def test_rejects_state_data_update(self, client, mock_job):
"""Test that state_data cannot be updated directly."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"state_data": {"foo": "bar"}},
)
assert response.status_code == 400
assert "not editable" in response.json()["detail"].lower()
def test_returns_404_when_job_not_found(self, client):
"""Test 404 when job doesn't exist."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = None
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/nonexistent-job",
json={"artist": "New Artist"},
)
assert response.status_code == 404
assert "not found" in response.json()["detail"].lower()
def test_returns_400_for_empty_update(self, client, mock_job):
"""Test 400 when no valid fields are provided."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={},
)
assert response.status_code == 400
assert "no valid" in response.json()["detail"].lower()
def test_logs_admin_changes(self, client, mock_job):
"""Test that admin changes are logged."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class, \
patch('backend.api.routes.admin.logger') as mock_logger:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"artist": "New Artist"},
)
assert response.status_code == 200
# Verify logging was called
mock_logger.info.assert_called()
log_message = mock_logger.info.call_args[0][0]
assert "admin@example.com" in log_message.lower() or "admin" in log_message.lower()
def test_update_customer_fields(self, client, mock_job):
"""Test updating made-for-you customer fields."""
with patch('backend.api.routes.admin.JobManager') as mock_jm_class:
mock_jm = Mock()
mock_jm.get_job.return_value = mock_job
mock_jm.update_job.return_value = True
mock_jm_class.return_value = mock_jm
response = client.patch(
"/api/admin/jobs/test-job-123",
json={
"customer_email": "customer@example.com",
"customer_notes": "Special request: add extra countdown",
},
)
assert response.status_code == 200
call_args = mock_jm.update_job.call_args
assert call_args[0][1]["customer_email"] == "customer@example.com"
assert call_args[0][1]["customer_notes"] == "Special request: add extra countdown"
class TestUpdateJobAuthorization:
"""Tests for authorization on the update endpoint."""
def test_requires_admin_access(self, client, mock_job):
"""Test that non-admin users cannot access the endpoint."""
original_override = app.dependency_overrides.get(require_admin)
def get_non_admin():
from fastapi import HTTPException
raise HTTPException(status_code=403, detail="Admin access required")
app.dependency_overrides[require_admin] = get_non_admin
try:
response = client.patch(
"/api/admin/jobs/test-job-123",
json={"artist": "New Artist"},
headers={"Authorization": "Bearer user-token"}
)
assert response.status_code == 403
finally:
if original_override:
app.dependency_overrides[require_admin] = original_override
else:
app.dependency_overrides[require_admin] = get_mock_admin
+6 -6
View File
@@ -28,20 +28,20 @@ class TestHealthRoutes:
from backend.main import app
return TestClient(app)
def test_health_endpoint_returns_200(self, client, auth_headers):
def test_health_endpoint_returns_200(self, client):
"""Test /api/health returns 200 OK."""
response = client.get("/api/health", )
response = client.get("/api/health")
assert response.status_code == 200
def test_health_endpoint_returns_healthy_status(self, client, auth_headers):
def test_health_endpoint_returns_healthy_status(self, client):
"""Test health endpoint returns healthy status."""
response = client.get("/api/health", )
response = client.get("/api/health")
data = response.json()
assert data["status"] == "healthy"
def test_root_endpoint_returns_200(self, client, auth_headers):
def test_root_endpoint_returns_200(self, client):
"""Test root endpoint returns 200."""
response = client.get("/", )
response = client.get("/")
assert response.status_code == 200
@@ -1,294 +0,0 @@
"""
Tests for audio_analysis_service.py - Audio analysis for backing vocals.
These tests mock the storage service and shared karaoke_gen modules to verify:
- GCS file download/upload operations
- Audio analysis delegation to shared AudioAnalyzer
- Waveform generation delegation to shared WaveformGenerator
"""
import pytest
from unittest.mock import Mock, MagicMock, patch, ANY
import tempfile
import os
class TestAudioAnalysisServiceInit:
"""Test AudioAnalysisService initialization."""
@patch("backend.services.audio_analysis_service.StorageService")
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_init_creates_dependencies(
self, mock_waveform_class, mock_analyzer_class, mock_storage_class
):
"""Test initialization creates storage service and analyzers."""
from backend.services.audio_analysis_service import AudioAnalysisService
service = AudioAnalysisService()
mock_storage_class.assert_called_once()
mock_analyzer_class.assert_called_once_with(
silence_threshold_db=-40.0,
min_segment_duration_ms=100,
)
mock_waveform_class.assert_called_once()
@patch("backend.services.audio_analysis_service.StorageService")
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_init_with_custom_params(
self, mock_waveform_class, mock_analyzer_class, mock_storage_class
):
"""Test initialization with custom threshold and duration."""
from backend.services.audio_analysis_service import AudioAnalysisService
service = AudioAnalysisService(
silence_threshold_db=-50.0,
min_segment_duration_ms=200,
)
mock_analyzer_class.assert_called_once_with(
silence_threshold_db=-50.0,
min_segment_duration_ms=200,
)
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_init_with_provided_storage_service(
self, mock_waveform_class, mock_analyzer_class
):
"""Test initialization with externally provided storage service."""
from backend.services.audio_analysis_service import AudioAnalysisService
mock_storage = Mock()
service = AudioAnalysisService(storage_service=mock_storage)
assert service.storage_service is mock_storage
class TestAnalyzeBackingVocals:
"""Test analyze_backing_vocals method."""
@patch("backend.services.audio_analysis_service.StorageService")
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_analyze_downloads_and_analyzes(
self, mock_waveform_class, mock_analyzer_class, mock_storage_class
):
"""Test analyze_backing_vocals downloads file and runs analysis."""
from backend.services.audio_analysis_service import AudioAnalysisService
# Setup mocks
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_result = Mock()
mock_result.has_audible_content = True
mock_result.segment_count = 5
mock_result.recommended_selection = Mock(value="with_backing")
mock_analyzer = Mock()
mock_analyzer.analyze.return_value = mock_result
mock_analyzer_class.return_value = mock_analyzer
service = AudioAnalysisService()
# Call method
result = service.analyze_backing_vocals(
gcs_audio_path="uploads/job123/backing_vocals.flac",
job_id="job123",
)
# Verify storage download was called
mock_storage.download_file.assert_called_once()
call_args = mock_storage.download_file.call_args
assert call_args[0][0] == "uploads/job123/backing_vocals.flac"
# Verify analyzer was called
mock_analyzer.analyze.assert_called_once()
# Verify result
assert result is mock_result
assert result.has_audible_content is True
@patch("backend.services.audio_analysis_service.StorageService")
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_analyze_cleans_up_temp_files(
self, mock_waveform_class, mock_analyzer_class, mock_storage_class
):
"""Test that temp files are cleaned up after analysis."""
from backend.services.audio_analysis_service import AudioAnalysisService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_result = Mock()
mock_result.has_audible_content = False
mock_result.segment_count = 0
mock_result.recommended_selection = Mock(value="clean")
mock_analyzer = Mock()
mock_analyzer.analyze.return_value = mock_result
mock_analyzer_class.return_value = mock_analyzer
service = AudioAnalysisService()
# Call method
service.analyze_backing_vocals(
gcs_audio_path="uploads/job123/backing.flac",
job_id="job123",
)
# Temp directory should be cleaned up (no assertion needed,
# TemporaryDirectory context manager handles this)
# Just verify the method completes without error
class TestAnalyzeAndGenerateWaveform:
"""Test analyze_and_generate_waveform method."""
@patch("backend.services.audio_analysis_service.StorageService")
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_analyze_and_generate_waveform(
self, mock_waveform_class, mock_analyzer_class, mock_storage_class
):
"""Test analyze_and_generate_waveform does both operations."""
from backend.services.audio_analysis_service import AudioAnalysisService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_result = Mock()
mock_result.has_audible_content = True
mock_result.audible_segments = [Mock(), Mock()]
mock_analyzer = Mock()
mock_analyzer.analyze.return_value = mock_result
mock_analyzer.silence_threshold_db = -40.0
mock_analyzer_class.return_value = mock_analyzer
mock_waveform = Mock()
mock_waveform_class.return_value = mock_waveform
service = AudioAnalysisService()
# Call method
result, waveform_path = service.analyze_and_generate_waveform(
gcs_audio_path="uploads/job123/backing.flac",
job_id="job123",
gcs_waveform_destination="uploads/job123/waveform.png",
)
# Verify analysis was performed
mock_analyzer.analyze.assert_called_once()
# Verify waveform was generated
mock_waveform.generate.assert_called_once()
call_kwargs = mock_waveform.generate.call_args.kwargs
assert call_kwargs["segments"] == mock_result.audible_segments
assert call_kwargs["show_time_axis"] is True
# Verify waveform was uploaded
assert mock_storage.upload_file.call_count == 1
# Verify return values
assert result is mock_result
assert waveform_path == "uploads/job123/waveform.png"
class TestGetWaveformData:
"""Test get_waveform_data method."""
@patch("backend.services.audio_analysis_service.StorageService")
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_get_waveform_data(
self, mock_waveform_class, mock_analyzer_class, mock_storage_class
):
"""Test get_waveform_data returns amplitude data."""
from backend.services.audio_analysis_service import AudioAnalysisService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_waveform = Mock()
mock_waveform.generate_data_only.return_value = (
[0.1, 0.3, 0.5, 0.3, 0.1], # amplitudes
180.5, # duration
)
mock_waveform_class.return_value = mock_waveform
service = AudioAnalysisService()
amplitudes, duration = service.get_waveform_data(
gcs_audio_path="uploads/job123/backing.flac",
job_id="job123",
num_points=500,
)
# Verify storage download
mock_storage.download_file.assert_called_once()
# Verify waveform generator called
mock_waveform.generate_data_only.assert_called_once()
call_kwargs = mock_waveform.generate_data_only.call_args.kwargs
assert call_kwargs["num_points"] == 500
# Verify return values
assert amplitudes == [0.1, 0.3, 0.5, 0.3, 0.1]
assert duration == 180.5
class TestGenerateWaveformWithMutes:
"""Test generate_waveform_with_mutes method."""
@patch("backend.services.audio_analysis_service.StorageService")
@patch("backend.services.audio_analysis_service.AudioAnalyzer")
@patch("backend.services.audio_analysis_service.WaveformGenerator")
def test_generate_waveform_with_mutes(
self, mock_waveform_class, mock_analyzer_class, mock_storage_class
):
"""Test generate_waveform_with_mutes highlights mute regions."""
from backend.services.audio_analysis_service import AudioAnalysisService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_result = Mock()
mock_result.audible_segments = [Mock(), Mock()]
mock_analyzer = Mock()
mock_analyzer.analyze.return_value = mock_result
mock_analyzer_class.return_value = mock_analyzer
mock_waveform = Mock()
mock_waveform_class.return_value = mock_waveform
# Create mock mute regions
mock_mute1 = Mock()
mock_mute2 = Mock()
mute_regions = [mock_mute1, mock_mute2]
service = AudioAnalysisService()
result_path = service.generate_waveform_with_mutes(
gcs_audio_path="uploads/job123/backing.flac",
job_id="job123",
gcs_waveform_destination="uploads/job123/waveform_muted.png",
mute_regions=mute_regions,
)
# Verify waveform generator was called with mute_regions
mock_waveform.generate.assert_called_once()
call_kwargs = mock_waveform.generate.call_args.kwargs
assert call_kwargs["mute_regions"] == mute_regions
assert call_kwargs["segments"] == mock_result.audible_segments
# Verify upload
mock_storage.upload_file.assert_called_once()
# Verify return
assert result_path == "uploads/job123/waveform_muted.png"
-386
View File
@@ -1,386 +0,0 @@
"""
Tests for audio_editing_service.py - Custom instrumental creation.
These tests mock the storage service and shared karaoke_gen modules to verify:
- GCS file download/upload operations
- Audio editing delegation to shared AudioEditor
- Mute region validation
"""
import pytest
from unittest.mock import Mock, MagicMock, patch, ANY
class TestAudioEditingServiceInit:
"""Test AudioEditingService initialization."""
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_init_creates_dependencies(self, mock_editor_class, mock_storage_class):
"""Test initialization creates storage service and editor."""
from backend.services.audio_editing_service import AudioEditingService
service = AudioEditingService()
mock_storage_class.assert_called_once()
mock_editor_class.assert_called_once_with(output_format="flac")
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_init_with_custom_format(self, mock_editor_class, mock_storage_class):
"""Test initialization with custom output format."""
from backend.services.audio_editing_service import AudioEditingService
service = AudioEditingService(output_format="wav")
mock_editor_class.assert_called_once_with(output_format="wav")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_init_with_provided_storage_service(self, mock_editor_class):
"""Test initialization with externally provided storage service."""
from backend.services.audio_editing_service import AudioEditingService
mock_storage = Mock()
service = AudioEditingService(storage_service=mock_storage)
assert service.storage_service is mock_storage
class TestCreateCustomInstrumental:
"""Test create_custom_instrumental method."""
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_create_custom_instrumental_success(
self, mock_editor_class, mock_storage_class
):
"""Test creating a custom instrumental successfully."""
from backend.services.audio_editing_service import AudioEditingService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_result = Mock()
mock_result.total_muted_duration_seconds = 15.5
mock_result.mute_regions_applied = [Mock(), Mock(), Mock()]
mock_result.output_path = None # Will be set by service
mock_editor = Mock()
mock_editor.create_custom_instrumental.return_value = mock_result
mock_editor_class.return_value = mock_editor
service = AudioEditingService()
# Create mock mute regions
mock_mute1 = Mock()
mock_mute2 = Mock()
mute_regions = [mock_mute1, mock_mute2]
result = service.create_custom_instrumental(
gcs_clean_instrumental_path="uploads/job123/clean.flac",
gcs_backing_vocals_path="uploads/job123/backing.flac",
mute_regions=mute_regions,
gcs_output_path="uploads/job123/custom.flac",
job_id="job123",
)
# Verify downloads
assert mock_storage.download_file.call_count == 2
# Verify editor was called
mock_editor.create_custom_instrumental.assert_called_once()
call_kwargs = mock_editor.create_custom_instrumental.call_args.kwargs
assert call_kwargs["mute_regions"] == mute_regions
# Verify upload
mock_storage.upload_file.assert_called_once()
# Verify result path was updated
assert result.output_path == "uploads/job123/custom.flac"
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_create_custom_instrumental_downloads_both_files(
self, mock_editor_class, mock_storage_class
):
"""Test that both input files are downloaded."""
from backend.services.audio_editing_service import AudioEditingService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_result = Mock()
mock_result.total_muted_duration_seconds = 0
mock_result.mute_regions_applied = []
mock_editor = Mock()
mock_editor.create_custom_instrumental.return_value = mock_result
mock_editor_class.return_value = mock_editor
service = AudioEditingService()
service.create_custom_instrumental(
gcs_clean_instrumental_path="uploads/job123/clean.flac",
gcs_backing_vocals_path="uploads/job123/backing.flac",
mute_regions=[],
gcs_output_path="uploads/job123/output.flac",
job_id="job123",
)
# Verify both files were downloaded
download_calls = mock_storage.download_file.call_args_list
assert len(download_calls) == 2
# First call should be clean instrumental
assert download_calls[0][0][0] == "uploads/job123/clean.flac"
# Second call should be backing vocals
assert download_calls[1][0][0] == "uploads/job123/backing.flac"
class TestCreatePreview:
"""Test create_preview method."""
@patch("pydub.AudioSegment")
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_create_preview_success(
self, mock_editor_class, mock_storage_class, mock_audio_segment
):
"""Test creating a preview successfully."""
from backend.services.audio_editing_service import AudioEditingService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_preview = Mock()
mock_preview.export = Mock()
mock_preview.__getitem__ = Mock(return_value=mock_preview)
mock_editor = Mock()
mock_editor.preview_with_mutes.return_value = mock_preview
mock_editor_class.return_value = mock_editor
service = AudioEditingService()
result = service.create_preview(
gcs_clean_instrumental_path="uploads/job123/clean.flac",
gcs_backing_vocals_path="uploads/job123/backing.flac",
mute_regions=[Mock(), Mock()],
gcs_preview_path="uploads/job123/preview.flac",
job_id="job123",
)
# Verify editor was called
mock_editor.preview_with_mutes.assert_called_once()
# Verify result
assert result == "uploads/job123/preview.flac"
@patch("pydub.AudioSegment")
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_create_preview_with_duration_limit(
self, mock_editor_class, mock_storage_class, mock_audio_segment
):
"""Test creating a preview with duration limit."""
from backend.services.audio_editing_service import AudioEditingService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_preview = Mock()
mock_truncated = Mock()
mock_preview.__getitem__ = Mock(return_value=mock_truncated)
mock_truncated.export = Mock()
mock_editor = Mock()
mock_editor.preview_with_mutes.return_value = mock_preview
mock_editor_class.return_value = mock_editor
service = AudioEditingService()
service.create_preview(
gcs_clean_instrumental_path="uploads/job123/clean.flac",
gcs_backing_vocals_path="uploads/job123/backing.flac",
mute_regions=[],
gcs_preview_path="uploads/job123/preview.flac",
job_id="job123",
preview_duration_seconds=30.0,
)
# Verify preview was truncated (30 seconds = 30000ms)
mock_preview.__getitem__.assert_called_once_with(slice(None, 30000))
class TestMuteBackingVocalsOnly:
"""Test mute_backing_vocals_only method."""
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_mute_backing_vocals_only(self, mock_editor_class, mock_storage_class):
"""Test muting backing vocals without combining."""
from backend.services.audio_editing_service import AudioEditingService
mock_storage = Mock()
mock_storage_class.return_value = mock_storage
mock_editor = Mock()
mock_editor_class.return_value = mock_editor
service = AudioEditingService()
mute_regions = [Mock(), Mock()]
result = service.mute_backing_vocals_only(
gcs_backing_vocals_path="uploads/job123/backing.flac",
mute_regions=mute_regions,
gcs_output_path="uploads/job123/muted_backing.flac",
job_id="job123",
)
# Verify only one file was downloaded
mock_storage.download_file.assert_called_once()
assert mock_storage.download_file.call_args[0][0] == "uploads/job123/backing.flac"
# Verify editor method was called
mock_editor.apply_mute_to_single_track.assert_called_once()
call_kwargs = mock_editor.apply_mute_to_single_track.call_args.kwargs
assert call_kwargs["mute_regions"] == mute_regions
# Verify upload
mock_storage.upload_file.assert_called_once()
# Verify result
assert result == "uploads/job123/muted_backing.flac"
class TestValidateMuteRegions:
"""Test validate_mute_regions method.
Note: MuteRegion uses pydantic validation that rejects invalid values at
creation time. These tests use Mock objects to test the validation logic
in the service, simulating edge cases that would normally be rejected.
"""
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_valid_mute_regions(self, mock_editor_class, mock_storage_class):
"""Test validation passes for valid mute regions."""
from backend.services.audio_editing_service import AudioEditingService
from karaoke_gen.instrumental_review import MuteRegion
service = AudioEditingService()
mute_regions = [
MuteRegion(start_seconds=10.0, end_seconds=15.0),
MuteRegion(start_seconds=30.0, end_seconds=45.0),
]
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=180.0)
assert errors == []
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_negative_start_seconds(self, mock_editor_class, mock_storage_class):
"""Test validation catches negative start time.
Uses Mock since MuteRegion pydantic model rejects negative values.
"""
from backend.services.audio_editing_service import AudioEditingService
service = AudioEditingService()
# Use mock to simulate invalid region (pydantic would reject this)
mock_region = Mock()
mock_region.start_seconds = -5.0
mock_region.end_seconds = 10.0
errors = service.validate_mute_regions([mock_region], total_duration_seconds=180.0)
assert len(errors) == 1
assert "cannot be negative" in errors[0]
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_end_before_start(self, mock_editor_class, mock_storage_class):
"""Test validation catches end time before start time."""
from backend.services.audio_editing_service import AudioEditingService
service = AudioEditingService()
# Use mock to simulate invalid region
mock_region = Mock()
mock_region.start_seconds = 30.0
mock_region.end_seconds = 20.0
errors = service.validate_mute_regions([mock_region], total_duration_seconds=180.0)
assert len(errors) == 1
assert "must be after" in errors[0]
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_start_exceeds_duration(self, mock_editor_class, mock_storage_class):
"""Test validation catches start time exceeding audio duration."""
from backend.services.audio_editing_service import AudioEditingService
from karaoke_gen.instrumental_review import MuteRegion
service = AudioEditingService()
mute_regions = [
MuteRegion(start_seconds=200.0, end_seconds=210.0),
]
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=180.0)
assert len(errors) == 1
assert "exceeds audio duration" in errors[0]
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_end_exceeds_duration_is_warning_not_error(
self, mock_editor_class, mock_storage_class
):
"""Test that end time exceeding duration is logged as warning, not error."""
from backend.services.audio_editing_service import AudioEditingService
from karaoke_gen.instrumental_review import MuteRegion
service = AudioEditingService()
mute_regions = [
MuteRegion(start_seconds=170.0, end_seconds=200.0), # End exceeds 180s
]
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=180.0)
# Should not be an error (will be clamped)
assert len(errors) == 0
@patch("backend.services.audio_editing_service.StorageService")
@patch("backend.services.audio_editing_service.AudioEditor")
def test_multiple_errors(self, mock_editor_class, mock_storage_class):
"""Test validation reports multiple errors."""
from backend.services.audio_editing_service import AudioEditingService
from karaoke_gen.instrumental_review import MuteRegion
service = AudioEditingService()
# Use mocks for invalid regions
mock_region1 = Mock()
mock_region1.start_seconds = -5.0
mock_region1.end_seconds = 10.0
mock_region2 = Mock()
mock_region2.start_seconds = 30.0
mock_region2.end_seconds = 20.0 # End before start
mock_region3 = Mock()
mock_region3.start_seconds = 200.0 # Exceeds duration
mock_region3.end_seconds = 210.0
mute_regions = [mock_region1, mock_region2, mock_region3]
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=180.0)
assert len(errors) == 3
File diff suppressed because it is too large. Load diff
-15
View File
@@ -3,28 +3,13 @@ Unit tests for audio analysis and editing services.
These tests verify the backend service wrappers that use the shared
karaoke_gen.instrumental_review module with GCS integration.
NOTE: These tests require ffmpeg to be installed. They will be skipped
if ffmpeg is not available (e.g., in CI environments without ffmpeg).
"""
import os
import shutil
import tempfile
import pytest
from unittest.mock import MagicMock, patch, ANY
# Check if ffmpeg is available
def _ffmpeg_available():
"""Check if ffmpeg is available on the system."""
return shutil.which('ffmpeg') is not None
# Skip all tests in this module if ffmpeg is not available
pytestmark = pytest.mark.skipif(
not _ffmpeg_available(),
reason="ffmpeg not available - required for audio processing tests"
)
from pydub import AudioSegment
from pydub.generators import Sine
+1 -89
View File
@@ -114,97 +114,9 @@ class TestWorkerServiceBasics:
assert 'http' in url
class TestSessionTokenValidation:
"""Tests for session token validation in AuthService.
PR #120: Fixed authentication to recognize session tokens from magic link auth.
Previously, validate_token() only checked admin tokens and auth_tokens collection.
Session tokens (created via magic link) are stored in the sessions collection,
so they were incorrectly rejected as "Invalid token".
"""
def test_validate_token_checks_session_on_fallback(self):
"""Test that validate_token falls back to checking session tokens."""
mock_firestore = MagicMock()
mock_firestore.get_token.return_value = None # Token not in auth_tokens
mock_user = MagicMock()
mock_user.email = "test@example.com"
mock_user.credits = 5
mock_user_service = MagicMock()
mock_user_service.validate_session.return_value = (True, mock_user, "Valid session")
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
# Clear the cached instance to get fresh state
import backend.services.auth_service
backend.services.auth_service._auth_service = None
from backend.services.auth_service import AuthService, UserType
service = AuthService()
is_valid, user_type, remaining, msg = service.validate_token("session-token-xyz")
assert is_valid is True
assert user_type == UserType.STRIPE
assert remaining == 5 # User's credits
mock_user_service.validate_session.assert_called_once_with("session-token-xyz")
def test_validate_token_returns_invalid_when_session_invalid(self):
"""Test that validate_token returns invalid when session validation fails."""
mock_firestore = MagicMock()
mock_firestore.get_token.return_value = None # Token not in auth_tokens
mock_user_service = MagicMock()
mock_user_service.validate_session.return_value = (False, None, "Invalid session")
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
import backend.services.auth_service
backend.services.auth_service._auth_service = None
from backend.services.auth_service import AuthService, UserType
service = AuthService()
is_valid, user_type, remaining, msg = service.validate_token("invalid-session")
assert is_valid is False
assert remaining == 0
assert "Invalid token" in msg
def test_validate_token_returns_zero_credits_when_user_has_none(self):
"""Test that validate_token handles users with 0 credits correctly."""
mock_firestore = MagicMock()
mock_firestore.get_token.return_value = None
mock_user = MagicMock()
mock_user.email = "test@example.com"
mock_user.credits = 0
mock_user_service = MagicMock()
mock_user_service.validate_session.return_value = (True, mock_user, "Valid session")
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
import backend.services.auth_service
backend.services.auth_service._auth_service = None
from backend.services.auth_service import AuthService, UserType
service = AuthService()
is_valid, user_type, remaining, msg = service.validate_token("session-token")
# User is authenticated but has no credits
assert is_valid is True
assert user_type == UserType.STRIPE
assert remaining == 0
assert "no credits" in msg.lower()
class TestJobManagerBasics:
"""Basic tests for JobManager."""
def test_job_manager_can_import(self):
"""Test JobManager can be imported."""
from backend.services.job_manager import JobManager
-244
View File
@@ -1,244 +0,0 @@
"""
Tests for DiscordNotificationService.
Tests cover:
- Service initialization
- Webhook URL validation
- Message posting
- Video notification posting
- Dry run mode
"""
import pytest
from unittest.mock import MagicMock, patch
from backend.services.discord_service import (
DiscordNotificationService,
get_discord_notification_service,
)
class TestDiscordNotificationServiceInit:
"""Test service initialization."""
def test_init_with_webhook_url(self):
"""Test initialization with valid webhook URL."""
url = "https://discord.com/api/webhooks/123456789/abcdef123456"
service = DiscordNotificationService(webhook_url=url)
assert service.webhook_url == url
assert service.is_enabled() is True
def test_init_without_webhook_url(self):
"""Test initialization without webhook URL."""
service = DiscordNotificationService()
assert service.webhook_url is None
assert service.is_enabled() is False
def test_init_with_dry_run(self):
"""Test initialization with dry run mode."""
service = DiscordNotificationService(dry_run=True)
assert service.dry_run is True
def test_init_with_invalid_webhook_url_raises(self):
"""Test that invalid webhook URL raises ValueError."""
with pytest.raises(ValueError) as exc_info:
DiscordNotificationService(webhook_url="https://example.com/webhook")
assert "Invalid Discord webhook URL" in str(exc_info.value)
def test_init_with_non_discord_url_raises(self):
"""Test that non-Discord URL raises ValueError."""
with pytest.raises(ValueError):
DiscordNotificationService(
webhook_url="https://slack.com/api/webhooks/123"
)
class TestDiscordNotificationServiceValidation:
"""Test webhook URL validation."""
def test_validate_valid_webhook_url(self):
"""Test that valid webhook URL passes validation."""
service = DiscordNotificationService()
# Should not raise
service._validate_webhook_url(
"https://discord.com/api/webhooks/123456789/abcdef"
)
def test_validate_invalid_webhook_url(self):
"""Test that invalid webhook URL raises ValueError."""
service = DiscordNotificationService()
with pytest.raises(ValueError) as exc_info:
service._validate_webhook_url("https://example.com/webhook")
assert "Invalid Discord webhook URL" in str(exc_info.value)
def test_validate_strips_whitespace(self):
"""Test that whitespace is stripped from URL."""
service = DiscordNotificationService()
# Should not raise - whitespace is stripped
service._validate_webhook_url(
" https://discord.com/api/webhooks/123456789/abcdef "
)
class TestDiscordNotificationServicePostMessage:
"""Test message posting."""
@patch("backend.services.discord_service.requests.post")
def test_post_message_success(self, mock_post):
"""Test successful message posting."""
mock_response = MagicMock()
mock_response.raise_for_status = MagicMock()
mock_post.return_value = mock_response
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = DiscordNotificationService(webhook_url=url)
result = service.post_message("Test message")
assert result is True
mock_post.assert_called_once_with(
url,
json={"content": "Test message"},
timeout=30
)
@patch("backend.services.discord_service.requests.post")
def test_post_message_with_custom_webhook(self, mock_post):
"""Test message posting with custom webhook URL."""
mock_response = MagicMock()
mock_response.raise_for_status = MagicMock()
mock_post.return_value = mock_response
service = DiscordNotificationService()
custom_url = "https://discord.com/api/webhooks/987654321/fedcba"
result = service.post_message("Test message", webhook_url=custom_url)
assert result is True
mock_post.assert_called_once_with(
custom_url,
json={"content": "Test message"},
timeout=30
)
def test_post_message_no_webhook_raises(self):
"""Test that posting without webhook URL raises ValueError."""
service = DiscordNotificationService()
with pytest.raises(ValueError) as exc_info:
service.post_message("Test message")
assert "No Discord webhook URL provided" in str(exc_info.value)
def test_post_message_dry_run(self):
"""Test message posting in dry run mode."""
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = DiscordNotificationService(webhook_url=url, dry_run=True)
result = service.post_message("Test message")
assert result is True
# In dry run mode, no actual request should be made
@patch("backend.services.discord_service.requests.post")
def test_post_message_http_error(self, mock_post):
"""Test handling of HTTP errors."""
import requests as req
mock_response = MagicMock()
mock_response.raise_for_status.side_effect = req.HTTPError("404 Not Found")
mock_post.return_value = mock_response
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = DiscordNotificationService(webhook_url=url)
with pytest.raises(req.HTTPError):
service.post_message("Test message")
class TestDiscordNotificationServiceVideoNotification:
"""Test video notification posting."""
@patch("backend.services.discord_service.requests.post")
def test_post_video_notification_success(self, mock_post):
"""Test successful video notification."""
mock_response = MagicMock()
mock_response.raise_for_status = MagicMock()
mock_post.return_value = mock_response
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = DiscordNotificationService(webhook_url=url)
result = service.post_video_notification(
"https://www.youtube.com/watch?v=abc123"
)
assert result is True
mock_post.assert_called_once()
call_args = mock_post.call_args
assert "New upload:" in call_args[1]["json"]["content"]
assert "abc123" in call_args[1]["json"]["content"]
def test_post_video_notification_no_youtube_url(self):
"""Test video notification with empty YouTube URL."""
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = DiscordNotificationService(webhook_url=url)
result = service.post_video_notification("")
assert result is False
def test_post_video_notification_none_youtube_url(self):
"""Test video notification with None YouTube URL."""
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = DiscordNotificationService(webhook_url=url)
result = service.post_video_notification(None)
assert result is False
def test_post_video_notification_dry_run(self):
"""Test video notification in dry run mode."""
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = DiscordNotificationService(webhook_url=url, dry_run=True)
result = service.post_video_notification(
"https://www.youtube.com/watch?v=abc123"
)
assert result is True
class TestGetDiscordNotificationService:
"""Test factory function."""
def test_get_service_creates_instance(self):
"""Test that factory function creates a new instance."""
import backend.services.discord_service as module
module._discord_notification_service = None
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = get_discord_notification_service(webhook_url=url)
assert service is not None
assert isinstance(service, DiscordNotificationService)
assert service.webhook_url == url
def test_get_service_without_webhook(self):
"""Test factory function without webhook URL."""
import backend.services.discord_service as module
module._discord_notification_service = None
service = get_discord_notification_service()
assert service is not None
assert service.webhook_url is None
assert service.is_enabled() is False
def test_get_service_with_dry_run(self):
"""Test factory function with dry run mode."""
import backend.services.discord_service as module
module._discord_notification_service = None
url = "https://discord.com/api/webhooks/123456789/abcdef"
service = get_discord_notification_service(webhook_url=url, dry_run=True)
assert service.dry_run is True
Loaded 100 of 613 files, more files were not shown because too many files have changed in this diff. Show more