mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 21:17:56 +09:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
275fc909f6 |
No files matched your search
@@ -1,109 +0,0 @@
|
||||
# Plan: Admin Feedback Visibility
|
||||
|
||||
**Created:** 2026-03-23
|
||||
**Branch:** feat/sess-20260323-1454-feedback-email-debug
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
User feedback submissions are stored in Firestore (`user_feedback` collection) but are completely invisible to the admin. The admin panel only shows a boolean "Feedback Submitted: Yes/No" badge per user. This plan adds:
|
||||
|
||||
1. **Admin UI page** to browse and review all submitted feedback
|
||||
2. **Email notification** to admin@nomadkaraoke.com on each new submission
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] Admin page at `/admin/feedback` listing all feedback with ratings, text, timestamps, and user info
|
||||
- [ ] Sortable by date (newest first by default) and filterable by search (email)
|
||||
- [ ] Star ratings rendered visually (not just numbers)
|
||||
- [ ] Email sent to admin@nomadkaraoke.com immediately when feedback is submitted
|
||||
- [ ] Email includes all feedback content (ratings, text fields, user email)
|
||||
- [ ] Test data filtering works (exclude_test toggle)
|
||||
- [ ] Sidebar navigation includes new "Feedback" item
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Admin API Endpoint
|
||||
Add `GET /api/admin/feedback` to `backend/api/routes/admin.py`. Follows the standard admin endpoint pattern: `require_admin` dependency, pagination with `limit`/`offset`/`search`/`exclude_test`, returns Pydantic response model. Queries `user_feedback` Firestore collection.
|
||||
|
||||
### Admin UI Page
|
||||
New page at `frontend/app/admin/feedback/page.tsx`. Follows existing admin page patterns (Card, Table, Badge, pagination, search input, loading states). Uses `adminApi.listFeedback()` to fetch data.
|
||||
|
||||
### Email Notification
|
||||
Add a new method to `EmailService` (or call `send_email` directly) from the `submit_user_feedback` endpoint in `users.py`. Uses the existing SendGrid infrastructure — no new template service templates needed since this is an internal admin notification (simple HTML table with the feedback data).
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Backend — Admin feedback list endpoint
|
||||
|
||||
1. [ ] Add `AdminFeedbackItem` and `AdminFeedbackListResponse` Pydantic models to `admin.py`
|
||||
2. [ ] Add `GET /api/admin/feedback` endpoint with:
|
||||
- Pagination: `limit=50`, `offset=0`
|
||||
- Search by user email
|
||||
- `exclude_test` filter (reuse `is_test_email`)
|
||||
- Sort by `created_at` descending
|
||||
- Return: items, total, offset, limit, has_more
|
||||
3. [ ] Add aggregate stats in response: average ratings, total count
|
||||
|
||||
### Step 2: Backend — Email notification on submission
|
||||
|
||||
4. [ ] Add `send_feedback_notification` method to `EmailService` that sends an HTML email to `admin@nomadkaraoke.com` containing:
|
||||
- User email
|
||||
- All 4 star ratings
|
||||
- All text fields
|
||||
- Would recommend / would use again
|
||||
- Submitted timestamp
|
||||
- Link to admin feedback page
|
||||
5. [ ] Call `send_feedback_notification` from `submit_user_feedback` in `users.py` (fire-and-forget, don't block the user response on email success)
|
||||
|
||||
### Step 3: Frontend — Admin API client
|
||||
|
||||
6. [ ] Add `listFeedback` method to `adminApi` in `frontend/lib/api.ts`
|
||||
7. [ ] Add TypeScript types for `AdminFeedbackItem` and `AdminFeedbackListResponse`
|
||||
|
||||
### Step 4: Frontend — Admin feedback page
|
||||
|
||||
8. [ ] Create `frontend/app/admin/feedback/page.tsx` with:
|
||||
- Table: user email, date, overall rating (stars), ease of use, lyrics accuracy, correction experience, would recommend, would use again
|
||||
- Expandable rows or click-to-detail for text fields (what went well, what could improve, additional comments)
|
||||
- Search by email
|
||||
- Pagination (50 per page)
|
||||
- Refresh button
|
||||
- Test data toggle (via `useAdminSettings`)
|
||||
9. [ ] Add "Feedback" item to admin sidebar in `frontend/components/admin/admin-sidebar.tsx` with `MessageSquare` icon
|
||||
|
||||
### Step 5: Testing
|
||||
|
||||
10. [ ] Backend unit test for `GET /api/admin/feedback` endpoint (pagination, search, exclude_test)
|
||||
11. [ ] Backend unit test for `send_feedback_notification` email method
|
||||
12. [ ] Frontend component test for feedback page (render, empty state)
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/api/routes/admin.py` | Modify | Add feedback list endpoint + response models |
|
||||
| `backend/api/routes/users.py` | Modify | Add email notification call in `submit_user_feedback` |
|
||||
| `backend/services/email_service.py` | Modify | Add `send_feedback_notification` method |
|
||||
| `frontend/lib/api.ts` | Modify | Add `adminApi.listFeedback()` method + types |
|
||||
| `frontend/app/admin/feedback/page.tsx` | Create | Admin feedback review page |
|
||||
| `frontend/components/admin/admin-sidebar.tsx` | Modify | Add "Feedback" nav item |
|
||||
| `backend/tests/test_admin_feedback.py` | Create | Tests for admin feedback endpoint |
|
||||
| `backend/tests/test_feedback_email.py` | Create | Tests for feedback email notification |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit tests**: Admin endpoint (pagination, filtering, sorting), email method (content formatting, send call)
|
||||
- **Integration**: Submit feedback → verify email service called
|
||||
- **Manual verification**: Check email arrives in admin@nomadkaraoke.com inbox after submission
|
||||
|
||||
## Open Questions
|
||||
|
||||
- [ ] Should the admin page show feedback from the old `beta_feedback` collection too, or just `user_feedback`? (Recommendation: just `user_feedback` — the beta collection is legacy)
|
||||
- [ ] Should there be any spam/quality detection flagging? (Some submissions appear to game the system for credits)
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
- Backend: Remove endpoint from admin.py, revert email call in users.py
|
||||
- Frontend: Delete feedback page, remove sidebar item
|
||||
- No data migration needed — Firestore collection unchanged
|
||||
@@ -1,120 +0,0 @@
|
||||
# Plan: Admin Lyrics Review Dashboard
|
||||
|
||||
**Created:** 2026-03-05
|
||||
**Branch:** feat/sess-20260305-0002-admin-lyrics-review
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
Add an "Edit Reviews" section to the admin dashboard for reviewing completed lyrics reviews. Three goals:
|
||||
|
||||
1. **View what users saw** - Jump into a read-only lyrics review UI showing original vs corrected lyrics
|
||||
2. **Replay user edits** - Step through edit log entries to see exactly what each user changed
|
||||
3. **Review raw data** - View human edits data to evaluate quality for AudioShake data sharing
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] New admin sidebar item "Edit Reviews" linking to `/admin/edit-reviews`
|
||||
- [ ] Backend endpoint to list recent jobs that have completed lyrics review (with edit logs)
|
||||
- [ ] Backend endpoint to serve immutable correction data for a job (read-only, cannot modify job)
|
||||
- [ ] Admin page showing list of recently reviewed jobs with key metrics
|
||||
- [ ] Ability to open the existing LyricsAnalyzer in read-only mode with snapshot data
|
||||
- [ ] Edit log replay/stepper UI to walk through user changes one by one
|
||||
- [ ] Raw data viewer for edit logs, annotations, and correction diffs
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Backend: 2 new admin endpoints
|
||||
|
||||
1. **`GET /api/admin/edit-reviews`** - List jobs with completed reviews
|
||||
- Query Firestore for jobs with `state_data.last_edit_log_path` set (indicating review was performed)
|
||||
- Filter by status (completed jobs, or optionally all post-review statuses)
|
||||
- Return summary: job_id, artist, title, user_email, review_completed_at, edit_count, feedback_count
|
||||
- Support pagination, search, exclude_test filtering
|
||||
|
||||
2. **`GET /api/admin/edit-reviews/{job_id}`** - Get full review snapshot
|
||||
- Returns immutable copy of:
|
||||
- Original corrections (`corrections.json` from GCS)
|
||||
- Updated corrections (`corrections_updated.json` from GCS)
|
||||
- Edit log (`edit_log_{session_id}.json` from GCS)
|
||||
- Annotations (`annotations.json` from GCS)
|
||||
- Job metadata (artist, title, user_email, timestamps)
|
||||
- Audio URL (signed, for playback in review UI)
|
||||
- This is a **read-only** endpoint - no job state mutations
|
||||
- Uses `require_admin` dependency (not `require_review_auth`)
|
||||
- Does NOT transition job status or modify any state
|
||||
|
||||
### Frontend: New admin page + Read-only review mode
|
||||
|
||||
1. **`/admin/edit-reviews/page.tsx`** - List page
|
||||
- Table of recently reviewed jobs
|
||||
- Columns: Artist/Title, User, Review Date, Edits Made, Feedback Given, Actions
|
||||
- "View Review" button → opens read-only lyrics review
|
||||
- "View Edit Log" button → opens edit replay view
|
||||
- "View Raw Data" button → shows raw JSON
|
||||
|
||||
2. **Read-only review viewer** - Reuse LyricsAnalyzer
|
||||
- Load data from the new admin endpoint (not the regular review endpoint)
|
||||
- Pass `isReadOnly={true}` and `apiClient={null}` to LyricsAnalyzer
|
||||
- Header shows "Admin Review - Read Only" indicator
|
||||
- No submit/save buttons, no state transitions
|
||||
- Audio playback still works (via signed URL)
|
||||
|
||||
3. **Edit log replay** - New component
|
||||
- Timeline/stepper showing each edit entry
|
||||
- For each entry: operation type, text before → text after, timestamp
|
||||
- Step forward/backward through edits
|
||||
- Show feedback reason if user provided one
|
||||
- Summary stats: total edits, edits with feedback, correction types breakdown
|
||||
|
||||
4. **Raw data viewer** - JSON viewer tabs
|
||||
- Tab 1: Edit Log (formatted JSON with syntax highlighting)
|
||||
- Tab 2: Annotations
|
||||
- Tab 3: Corrections diff (original vs updated)
|
||||
|
||||
### Key safety guarantee
|
||||
|
||||
The admin review endpoint serves a **snapshot** of the data. It reads from GCS files that were already written. The LyricsAnalyzer receives `apiClient={null}` so it physically cannot call any mutation endpoints. Even if code changes in the future, the admin endpoint itself never writes to Firestore or GCS.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. [ ] **Backend: Add `GET /api/admin/edit-reviews` endpoint** - List reviewed jobs with edit log metadata
|
||||
2. [ ] **Backend: Add `GET /api/admin/edit-reviews/{job_id}` endpoint** - Serve full immutable review snapshot
|
||||
3. [ ] **Frontend: Add admin API client methods** - `adminApi.listEditReviews()`, `adminApi.getEditReview(jobId)`
|
||||
4. [ ] **Frontend: Create `/admin/edit-reviews/page.tsx`** - List page with table
|
||||
5. [ ] **Frontend: Add sidebar nav item** - "Edit Reviews" with BookOpen icon
|
||||
6. [ ] **Frontend: Create read-only review viewer** - Load snapshot data into LyricsAnalyzer with isReadOnly
|
||||
7. [ ] **Frontend: Create edit log replay component** - Step-through UI for edit entries
|
||||
8. [ ] **Frontend: Create raw data viewer** - JSON viewer with tabs
|
||||
9. [ ] **Tests: Backend unit tests** for new admin endpoints
|
||||
10. [ ] **Tests: Frontend component tests** for new page
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/api/routes/admin.py` | Modify | Add edit-reviews endpoints |
|
||||
| `frontend/lib/api.ts` | Modify | Add adminApi methods for edit reviews |
|
||||
| `frontend/app/admin/edit-reviews/page.tsx` | Create | Main edit reviews list page |
|
||||
| `frontend/components/admin/edit-review-viewer.tsx` | Create | Read-only review wrapper using LyricsAnalyzer |
|
||||
| `frontend/components/admin/edit-log-replay.tsx` | Create | Edit log stepper/replay component |
|
||||
| `frontend/components/admin/raw-data-viewer.tsx` | Create | JSON viewer with tabs |
|
||||
| `frontend/components/admin/admin-sidebar.tsx` | Modify | Add Edit Reviews nav item |
|
||||
| `backend/tests/test_admin_edit_reviews.py` | Create | Backend tests |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- Unit tests for backend endpoints (mock Firestore + GCS)
|
||||
- Verify admin auth is required on all new endpoints
|
||||
- Verify no write operations occur on read-only endpoints
|
||||
- Frontend component tests for key interactions
|
||||
|
||||
## Open Questions
|
||||
|
||||
- [ ] Should we show ALL jobs that went through review, or only ones with edit logs? (Recommendation: only those with edit logs, since those are the interesting ones)
|
||||
- [ ] Do we need audio playback in the admin review viewer? (Recommendation: yes, it helps understand what the user heard)
|
||||
- [ ] Should the raw data be downloadable? (Nice-to-have, can add later)
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
All changes are additive (new endpoints, new pages). Removing the sidebar link effectively hides the feature. No existing behavior is modified.
|
||||
@@ -1,151 +0,0 @@
|
||||
# Plan: AI-Powered Credit Gating
|
||||
|
||||
**Created:** 2026-03-23
|
||||
**Branch:** feat/sess-20260323-1716-ai-credit-gating
|
||||
**Status:** Implemented
|
||||
|
||||
## Overview
|
||||
|
||||
Replace the automatic welcome/feedback credit grants with an AI-powered evaluation step. When a user triggers a credit grant (email verification or feedback submission), instead of immediately granting credits, we collect abuse signals and ask Gemini to make a grant/deny decision. Suspected abusers get a friendly rejection email (CC'd to Andrew) instead of free credits.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] Welcome credit grants go through AI evaluation before granting
|
||||
- [ ] Feedback credit grants go through AI evaluation before granting
|
||||
- [ ] AI evaluates: fingerprint correlations, IP correlations, IP geo info, user agent, signup recency patterns
|
||||
- [ ] Approved users get credits as before (seamless experience)
|
||||
- [ ] Rejected users get a friendly-but-firm email explaining the situation, CC'd to andrew@beveridge.uk
|
||||
- [ ] Rejection email invites them to reply if it's a mistake
|
||||
- [ ] Decision is logged to Firestore for audit trail
|
||||
- [ ] Fail-open: if AI evaluation fails, grant credits (don't punish users for our errors)
|
||||
- [ ] Admin can override decisions via existing admin credit grant endpoint
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
User verifies email / submits feedback
|
||||
↓
|
||||
Collect abuse signals (sync, fast):
|
||||
- User's fingerprint + IP
|
||||
- Other accounts with same fingerprint (from correlations)
|
||||
- Other accounts with same IP
|
||||
- IP geolocation info
|
||||
- Recent signup count from same IP/fingerprint (last 24h)
|
||||
- User agent string
|
||||
- For feedback: the feedback content itself
|
||||
↓
|
||||
Call Gemini (vertexai/gemini-2.5-pro-preview) with structured prompt:
|
||||
- Present all signals
|
||||
- Ask for JSON response: {decision: "grant" | "deny", reasoning: string, confidence: number}
|
||||
↓
|
||||
If "grant": grant credits normally, log decision
|
||||
If "deny": skip credit grant, send rejection email, log decision
|
||||
If AI call fails: grant credits (fail-open), log error
|
||||
```
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
1. **Two-phase verification**: The magic link verify endpoint returns immediately with session + a `credit_status: "evaluating"` field. The frontend shows a "preparing your account" loading screen, then polls/waits for the evaluation result via a new lightweight endpoint. This keeps the auth flow snappy while the AI evaluation runs in the background of the same request or via a short follow-up call. The frontend then shows either a celebratory "2 free credits!" interstitial or a "sorry, couldn't grant credits" interstitial before navigating to the dashboard.
|
||||
|
||||
2. **Fail-open**: If Gemini is down or returns an error, we grant credits. Better to let an occasional abuser through than block legitimate users.
|
||||
|
||||
3. **Use google-genai directly**: The project already has `langchain-google-genai` installed. We can use the simpler `google.generativeai` library directly for a focused prompt/response, or use the existing LangChain setup. We'll use `google.generativeai` for simplicity since this is a single-turn evaluation, not an agentic workflow.
|
||||
|
||||
4. **Structured JSON output**: Ask Gemini for structured output with decision, reasoning, and confidence to make decisions parseable and auditable.
|
||||
|
||||
5. **Log all decisions**: Store every evaluation in a `credit_evaluations` Firestore collection for audit trail and to tune the prompt over time.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Add google-generativeai dependency
|
||||
- Add `google-generativeai` to pyproject.toml (lightweight, direct Gemini access)
|
||||
- Verify it works with Vertex AI auth in the nomadkaraoke project
|
||||
|
||||
### Step 2: Create credit evaluation service
|
||||
- New file: `backend/services/credit_evaluation_service.py`
|
||||
- `CreditEvaluationService` class with:
|
||||
- `evaluate_credit_grant(email, grant_type, feedback_content=None) -> CreditEvaluation`
|
||||
- Collects all abuse signals by calling existing services (user_service, ip_geolocation_service)
|
||||
- Builds prompt with signals
|
||||
- Calls Gemini for decision
|
||||
- Logs decision to Firestore
|
||||
- Returns structured result
|
||||
|
||||
### Step 3: Build the Gemini prompt
|
||||
- System prompt explaining the role: "You are an anti-abuse evaluator for a karaoke video service..."
|
||||
- Present all signals in a structured format
|
||||
- Request JSON response: `{decision: "grant"|"deny", reasoning: "...", confidence: 0.0-1.0}`
|
||||
- Include clear guidelines: same fingerprint across accounts = very suspicious, same IP = somewhat suspicious, etc.
|
||||
|
||||
### Step 4: Create rejection email template
|
||||
- Add `send_credit_denied_email(email, reasoning)` to EmailService
|
||||
- Friendly-but-firm tone
|
||||
- CC andrew@beveridge.uk
|
||||
- Invite reply if it's a mistake
|
||||
- Use existing email template infrastructure (branding, styling)
|
||||
|
||||
### Step 5: Integrate into welcome credit flow
|
||||
- Modify `grant_welcome_credits_if_eligible()` in user_service.py
|
||||
- Before granting: call `CreditEvaluationService.evaluate_credit_grant(email, "welcome")`
|
||||
- If denied: send rejection email, return False
|
||||
- If error: log warning, grant anyway (fail-open)
|
||||
|
||||
### Step 6: Integrate into feedback credit flow
|
||||
- Modify feedback endpoint in users.py
|
||||
- Before granting credits: evaluate with feedback content included
|
||||
- If denied: return different response message, send rejection email
|
||||
- If error: grant anyway
|
||||
|
||||
### Step 7: Add Firestore collection for audit trail
|
||||
- Collection: `credit_evaluations`
|
||||
- Document fields: email, grant_type, decision, reasoning, confidence, signals_snapshot, model_used, evaluated_at
|
||||
- TTL: keep indefinitely (small volume, high audit value)
|
||||
|
||||
### Step 8: Tests
|
||||
- Unit tests for signal collection
|
||||
- Unit tests for prompt building
|
||||
- Unit tests for decision parsing (grant/deny/error cases)
|
||||
- Integration test for the evaluation service with mocked Gemini
|
||||
- Integration test for welcome credit flow with evaluation
|
||||
- Integration test for feedback credit flow with evaluation
|
||||
- Test fail-open behavior
|
||||
|
||||
### Step 9: Configuration
|
||||
- Add env vars: `CREDIT_EVAL_MODEL` (default: "gemini-2.5-pro-preview"), `CREDIT_EVAL_ENABLED` (default: true)
|
||||
- Add to config.py
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `pyproject.toml` | Modify | Add google-generativeai dependency |
|
||||
| `backend/config.py` | Modify | Add credit evaluation config vars |
|
||||
| `backend/services/credit_evaluation_service.py` | Create | New service: collect signals, call Gemini, log decision |
|
||||
| `backend/services/user_service.py` | Modify | Integrate evaluation into welcome credit grant |
|
||||
| `backend/api/routes/users.py` | Modify | Integrate evaluation into feedback credit grant |
|
||||
| `backend/services/email_service.py` | Modify | Add credit denied email template |
|
||||
| `backend/tests/test_credit_evaluation.py` | Create | Tests for evaluation service |
|
||||
| `backend/tests/test_anti_abuse.py` | Modify | Update welcome credit tests for evaluation |
|
||||
| `frontend/app/auth/verify/page.tsx` | Modify | Add "preparing account" loading + credit result interstitial |
|
||||
| `frontend/lib/api.ts` | Modify | Add credit evaluation status endpoint |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit tests**: Signal collection, prompt building, response parsing, fail-open behavior
|
||||
- **Integration tests**: Full evaluation flow with mocked Gemini responses
|
||||
- **E2E tests**: Manual verification in production with test accounts
|
||||
- **Edge cases**: Gemini timeout, malformed response, missing signals, first-ever user (no correlations)
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Gemini model**: Using `gemini-3.1-pro-preview` as specified — already available in the nomadkaraoke GCP project.
|
||||
|
||||
2. **Latency UX**: Frontend shows a friendly "preparing your account" loading screen while evaluation runs. Then transitions to either a celebratory "2 free credits!" message (with gentle reminder not to abuse) or a "sorry, couldn't grant free credits" message with a buy credits CTA. User clicks through to dashboard from either.
|
||||
|
||||
3. **Should we also gate the existing rate limiter?** Currently `is_signup_rate_limited()` silently blocks at 2 accounts per IP/fingerprint per 24h. The AI evaluation is complementary — rate limiter blocks obvious rapid abuse, AI evaluation catches more subtle patterns.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Set `CREDIT_EVAL_ENABLED=false` in Cloud Run env vars → immediate rollback to auto-grant behavior. No data model changes needed. The evaluation service is purely additive.
|
||||
@@ -1,150 +0,0 @@
|
||||
# Plan: Block Disposable Email Registration
|
||||
|
||||
**Created:** 2026-03-19
|
||||
**Branch:** feat/sess-20260319-1332-disposable-email-abuse
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
Users are signing up with disposable email addresses (yopmail.com, emailhook.site, etc.) to get unlimited free credits (2 per account). The `EmailValidationService` already exists with 160+ disposable domains but is **never called during signup**. We need to wire it into the magic link flow and show a friendly rejection message.
|
||||
|
||||
## Context
|
||||
|
||||
- The `EmailValidationService` (`backend/services/email_validation_service.py`) is fully implemented with disposable domain detection, email/IP blocklists, and admin management APIs
|
||||
- The admin blocklist management routes exist (`backend/api/routes/rate_limits.py`) and work
|
||||
- The magic link endpoint (`POST /api/users/auth/magic-link` in `backend/api/routes/users.py:133`) **never calls** the validation service
|
||||
- Currently 1 yopmail user and 1 emailhook.site user in `gen_users`
|
||||
- Each new account gets 2 free welcome credits automatically
|
||||
|
||||
## Requirements
|
||||
|
||||
- [x] Reject disposable email domains at the magic link endpoint (backend)
|
||||
- [x] Return a **specific, distinguishable error** so the frontend can show a friendly message (not the generic "email enumeration" response)
|
||||
- [x] Show a warm, friendly message in the AuthDialog explaining why and encouraging real email use
|
||||
- [x] Add `emailhook.site` to the default disposable domains list
|
||||
- [x] Add backend unit tests for the integration
|
||||
- [x] Add frontend test coverage for the new error state
|
||||
|
||||
## Design Decision: Explicit Rejection vs Silent Failure
|
||||
|
||||
The current magic link endpoint returns a generic success message to prevent email enumeration. For disposable emails, we will **explicitly reject** with a 422 status and a specific error message. This is the right trade-off because:
|
||||
|
||||
1. We *want* the user to know their disposable email won't work (so they can use a real one)
|
||||
2. Disposable domain detection is not sensitive information — the blocklist is public knowledge
|
||||
3. The friendly message serves our goal of converting them to real email users
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Backend — Add disposable email check to magic link endpoint
|
||||
|
||||
**File:** `backend/api/routes/users.py`
|
||||
|
||||
Add the `EmailValidationService` check **after** email lowercasing (line 158) and **before** tenant validation (line 164). Return a 422 with a friendly message.
|
||||
|
||||
```python
|
||||
# Check for disposable email domains
|
||||
email_validation = get_email_validation_service()
|
||||
if email_validation.is_disposable_domain(email):
|
||||
logger.warning(f"Blocked disposable email signup attempt: {_mask_email(email)}")
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="disposable_email_not_allowed"
|
||||
)
|
||||
```
|
||||
|
||||
Use a machine-readable error code (`disposable_email_not_allowed`) so the frontend can match on it and show the custom friendly message. This keeps the backend response lean and puts the UX copy in the frontend where it belongs.
|
||||
|
||||
### Step 2: Backend — Also check email and IP blocklists
|
||||
|
||||
In the same location, add checks for explicitly blocked emails and IPs:
|
||||
|
||||
```python
|
||||
if email_validation.is_email_blocked(email):
|
||||
logger.warning(f"Blocked email signup attempt: {_mask_email(email)}")
|
||||
# Silent reject — return generic success to prevent enumeration
|
||||
return SendMagicLinkResponse(
|
||||
status="success",
|
||||
message="If this email is registered, you will receive a sign-in link shortly."
|
||||
)
|
||||
|
||||
ip_address = http_request.client.host if http_request.client else None
|
||||
if ip_address and email_validation.is_ip_blocked(ip_address):
|
||||
logger.warning(f"Blocked IP signup attempt: {ip_address}")
|
||||
return SendMagicLinkResponse(
|
||||
status="success",
|
||||
message="If this email is registered, you will receive a sign-in link shortly."
|
||||
)
|
||||
```
|
||||
|
||||
Note: Blocked emails/IPs get silent rejection (anti-enumeration). Disposable domains get explicit rejection (we want the user to switch to a real email).
|
||||
|
||||
### Step 3: Backend — Add `emailhook.site` to default disposable domains
|
||||
|
||||
**File:** `backend/services/email_validation_service.py`
|
||||
|
||||
Add `emailhook.site` to `DEFAULT_DISPOSABLE_DOMAINS` set.
|
||||
|
||||
### Step 4: Frontend — Handle the disposable email error in AuthDialog
|
||||
|
||||
**File:** `frontend/components/auth/AuthDialog.tsx`
|
||||
|
||||
The `sendMagicLink` call in `auth.ts` already catches errors and sets `error` from the API response message. The `handleResponse` in `api.ts` throws an `ApiError` with the `detail` string for non-200 responses.
|
||||
|
||||
Update `handleSendMagicLink` to detect the `disposable_email_not_allowed` error code and show a friendly multi-line message instead of the terse error string:
|
||||
|
||||
```tsx
|
||||
const success = await sendMagicLink(email.trim().toLowerCase())
|
||||
if (success) {
|
||||
setStep("sent")
|
||||
}
|
||||
// If error contains our specific code, override with friendly message
|
||||
// (the error state is already set by the auth store)
|
||||
```
|
||||
|
||||
We need to detect the specific error. The cleanest approach: check the error string in the `displayError` rendering. If it matches `disposable_email_not_allowed`, render a custom friendly block instead of the red error text.
|
||||
|
||||
The friendly message:
|
||||
|
||||
> **We don't support disposable email addresses**
|
||||
>
|
||||
> We'd love to help you create karaoke videos! Please use your regular email address so we can send you a notification when your video is ready.
|
||||
>
|
||||
> We promise — no spam, no marketing emails, ever. We only email you about your karaoke videos.
|
||||
|
||||
### Step 5: Backend tests
|
||||
|
||||
**File:** `backend/tests/test_email_validation_integration.py` (new)
|
||||
|
||||
Test cases:
|
||||
1. Magic link with disposable email returns 422 with `disposable_email_not_allowed`
|
||||
2. Magic link with legitimate email proceeds normally
|
||||
3. Magic link with blocked email returns silent success (no actual email sent)
|
||||
4. Magic link with blocked IP returns silent success
|
||||
|
||||
### Step 6: Frontend tests
|
||||
|
||||
**File:** `frontend/components/__tests__/AuthDialog.test.tsx` (new or extend existing)
|
||||
|
||||
Test cases:
|
||||
1. Submitting a disposable email shows the friendly rejection message
|
||||
2. The friendly message contains the key copy (no spam promise, use real email)
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/api/routes/users.py` | Modify | Add email validation checks to magic link endpoint |
|
||||
| `backend/services/email_validation_service.py` | Modify | Add `emailhook.site` to default domains |
|
||||
| `frontend/components/auth/AuthDialog.tsx` | Modify | Handle disposable email error with friendly message |
|
||||
| `backend/tests/test_email_validation_integration.py` | Create | Integration tests for the validation in signup flow |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit tests:** Validate `is_disposable_domain` catches yopmail, emailhook.site (already covered in existing tests)
|
||||
- **Integration tests:** New test file verifying the magic link endpoint rejects disposable emails with 422
|
||||
- **Frontend tests:** Verify the AuthDialog shows friendly message on disposable email error
|
||||
- **Manual verification:** Try signing up with `test@yopmail.com` on dev server to see the UX
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
Remove the validation check from `send_magic_link()` in `users.py`. The `EmailValidationService` itself remains unchanged (it's already deployed and used by admin routes).
|
||||
@@ -1,111 +0,0 @@
|
||||
# Plan: Credit Refund on Job Deletion and Cancellation
|
||||
|
||||
**Created:** 2026-03-04
|
||||
**Branch:** feat/sess-20260304-1308-investigate-user-credits
|
||||
**Status:** Implemented
|
||||
|
||||
## Overview
|
||||
|
||||
When a user deletes or cancels a job, their credit is not refunded. This was discovered when user john2011@sky.com deleted a job 16 seconds after creation (accidental duplicate) and lost the credit. Job *failures* already auto-refund via `mark_job_failed()` → `refund_credit()`, but deletions and cancellations do not.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [x] Job failures already refund credits (existing behavior, no change needed)
|
||||
- [ ] Job cancellation refunds 1 credit
|
||||
- [ ] Job deletion refunds 1 credit (if not already completed)
|
||||
- [ ] No refund for completed jobs (user got their video)
|
||||
- [ ] No refund for admin-owned jobs (consistent with failure refund logic)
|
||||
- [ ] Idempotency: prevent double-refund if a job is cancelled then deleted, or fails then deleted
|
||||
- [ ] Credit transaction records the reason clearly ("job_cancelled", "job_deleted")
|
||||
- [ ] Refund failure must not block the cancel/delete operation
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Key Insight: Idempotency via Job Field
|
||||
|
||||
The current `refund_credit()` has no guard against double-refund for the same job. Since a job can transition through cancelled → deleted, or failed → deleted, we need a way to track that a refund was already issued.
|
||||
|
||||
**Approach**: Add a `credit_refunded` boolean field to the Job model. Check it before refunding. This is simpler and more reliable than scanning the user's transaction history.
|
||||
|
||||
### What Should and Shouldn't Be Refunded
|
||||
|
||||
| Job Status at Time of Action | Delete | Cancel | Notes |
|
||||
|------------------------------|--------|--------|-------|
|
||||
| PENDING through pre-COMPLETE | Refund | Refund | Job didn't deliver value |
|
||||
| COMPLETE / PREP_COMPLETE | No refund | N/A (can't cancel terminal) | User got their output |
|
||||
| FAILED | No refund (already refunded) | N/A (can't cancel terminal) | Already refunded by fail_job |
|
||||
| CANCELLED | No refund (already refunded) | N/A (already cancelled) | Already refunded by cancel |
|
||||
|
||||
For deletion: we need to read the job status *before* deleting it to decide whether to refund. The delete endpoint already fetches the job for ownership check, so this is straightforward.
|
||||
|
||||
### Refund Helper
|
||||
|
||||
Extract the refund logic (currently in `mark_job_failed`) into a reusable private method `_refund_credit_for_job()` that:
|
||||
1. Checks `job.credit_refunded` — skip if already refunded
|
||||
2. Checks `job.user_email` — skip if no user
|
||||
3. Checks if admin — skip if admin job
|
||||
4. Calls `user_service.refund_credit()` with appropriate reason
|
||||
5. Sets `credit_refunded = True` on the job (for cancel; for delete the job is about to be removed but we set it anyway for the brief window)
|
||||
6. Wraps everything in try/except so failures don't block the parent operation
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. [ ] **Add `credit_refunded` field to Job model** (`backend/models/job.py`)
|
||||
- Add `credit_refunded: bool = False` to the Job model
|
||||
- No migration needed — Firestore is schemaless, defaults to False for existing docs
|
||||
|
||||
2. [ ] **Extract refund helper in JobManager** (`backend/services/job_manager.py`)
|
||||
- Create `_refund_credit_for_job(self, job_id: str, job: Job, reason: str) -> bool`
|
||||
- Move the refund logic from `mark_job_failed` (lines 821-839) into this helper
|
||||
- Add `credit_refunded` check at the top
|
||||
- Update `mark_job_failed` to call the helper with `reason="job_failed"`
|
||||
|
||||
3. [ ] **Add refund to `cancel_job()`** (`backend/services/job_manager.py`)
|
||||
- After setting status to CANCELLED, call `_refund_credit_for_job(job_id, job, reason="job_cancelled")`
|
||||
- The job object is already fetched at line 1017
|
||||
|
||||
4. [ ] **Add refund to `delete_job()`** (`backend/services/job_manager.py`)
|
||||
- Need to fetch the job (already done when `delete_files=True`, but not always)
|
||||
- Before deletion, check if job status is NOT in `[COMPLETE, PREP_COMPLETE, FAILED, CANCELLED]`
|
||||
- If eligible, call `_refund_credit_for_job(job_id, job, reason="job_deleted")`
|
||||
- For FAILED/CANCELLED: skip because refund was already issued during that transition
|
||||
- For COMPLETE/PREP_COMPLETE: skip because user received their output
|
||||
|
||||
5. [ ] **Update `mark_job_failed` to set `credit_refunded`**
|
||||
- The helper handles this, but also update the Firestore doc
|
||||
|
||||
6. [ ] **Add unit tests** (`backend/tests/test_credit_enforcement.py`)
|
||||
- Test: cancel_job refunds credit for non-admin user
|
||||
- Test: cancel_job skips refund for admin jobs
|
||||
- Test: delete_job refunds credit for non-complete, non-terminal jobs
|
||||
- Test: delete_job does NOT refund for completed jobs
|
||||
- Test: delete_job does NOT double-refund for already-failed jobs
|
||||
- Test: delete_job does NOT double-refund for already-cancelled jobs
|
||||
- Test: refund failure doesn't block cancel/delete
|
||||
- Test: credit_refunded flag prevents double refund
|
||||
|
||||
7. [ ] **Backfill: Refund john2011@sky.com** (manual one-time)
|
||||
- Grant 1 credit for the deleted job 40dcca89 via admin API or direct Firestore update
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/models/job.py` | Modify | Add `credit_refunded: bool = False` field |
|
||||
| `backend/services/job_manager.py` | Modify | Extract `_refund_credit_for_job()` helper, add refund to `cancel_job()` and `delete_job()` |
|
||||
| `backend/tests/test_credit_enforcement.py` | Modify | Add tests for cancel/delete refund scenarios |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit tests**: Extend `test_credit_enforcement.py` with new test classes `TestCreditRefundOnJobCancellation` and `TestCreditRefundOnJobDeletion`, following the exact pattern of existing `TestCreditRefundOnJobFailure`
|
||||
- **Manual verification**: After deploy, cancel and delete test jobs and verify credit balance updates
|
||||
|
||||
## Open Questions
|
||||
|
||||
- [ ] Should we notify the user (email/in-app) when a credit is refunded? (Suggest: no, keep it simple for now — the credit just appears back in their balance)
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
- The `credit_refunded` field is additive (defaults to False for old jobs)
|
||||
- If issues arise, revert the PR — old behavior (no refund on cancel/delete) is restored
|
||||
- Any refunds already issued are just bonus credits to users, no harm done
|
||||
@@ -1,407 +0,0 @@
|
||||
# Plan: Add Retry Logic for Flacfetch Download Failures
|
||||
|
||||
**Created:** 2026-02-10
|
||||
**Branch:** feat/sess-20260210-1158-investigate-job-749141f8
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
Job 749141f8 failed with error: "Audio download failed: Remote download by ID failed: Download by ID request failed:"
|
||||
|
||||
Investigation revealed **multiple related issues** when flacfetch network calls fail:
|
||||
|
||||
1. **No retry logic** - A single network hiccup causes immediate job failure
|
||||
2. **Missing source info** - When download fails, source_id/source_name aren't saved to job
|
||||
3. **Broken retry button** - Can't retry because no source info was persisted
|
||||
4. **Silent UI failures** - Frontend doesn't show retry error messages to user
|
||||
|
||||
### Root Cause
|
||||
|
||||
Audio search flow saves selection info to `state_data`, but doesn't save download parameters (source_id, source_name, etc.) to the job document until AFTER download succeeds. If the flacfetch call fails (network error), the job fails with no way to retry.
|
||||
|
||||
### Solution
|
||||
|
||||
This plan fixes all four issues:
|
||||
- Add retry logic to flacfetch client (network resilience)
|
||||
- Persist download info before attempting download (enables retry)
|
||||
- Update retry endpoint to handle audio search jobs
|
||||
- Show error messages in frontend when retry fails
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional Requirements - Flacfetch Retry Logic
|
||||
- [x] Retry `download_by_id()` on `httpx.RequestError` (network failures)
|
||||
- [x] Use exponential backoff spread over ~5 minutes
|
||||
- [x] Configurable max retries via environment variable
|
||||
- [x] Log each retry attempt with context
|
||||
- [x] Do NOT retry on HTTP 4xx errors (client errors - immediate failure)
|
||||
- [x] Do retry on HTTP 5xx errors (server errors - transient)
|
||||
|
||||
### Functional Requirements - Persist Download Info
|
||||
- [x] Save source_id, source_name, target_file, download_url to job document
|
||||
- [x] Save BEFORE attempting download (not after)
|
||||
- [x] Store in top-level fields for easy retry access
|
||||
- [x] Preserve existing state_data fields (selected_audio_index, etc.)
|
||||
|
||||
### Functional Requirements - Retry Endpoint
|
||||
- [x] Add case for audio search jobs in `/jobs/{id}/retry` endpoint
|
||||
- [x] Check for saved download params (source_id, source_name)
|
||||
- [x] Re-download using saved params
|
||||
- [x] Clear error state before retrying
|
||||
- [x] Trigger appropriate workers after successful download
|
||||
|
||||
### Functional Requirements - Frontend Error Handling
|
||||
- [x] Show toast notification when retry fails
|
||||
- [x] Display actual error message from API
|
||||
- [x] Don't just console.error - make it visible to user
|
||||
|
||||
### Non-Functional Requirements
|
||||
- [x] Minimal code changes (use existing patterns)
|
||||
- [x] Clear error messages that include retry context
|
||||
- [x] Backwards compatible (no breaking changes)
|
||||
- [x] Should not significantly increase total timeout duration
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Architecture Decisions
|
||||
|
||||
**Option 1: Tenacity library (Recommended)**
|
||||
- ✅ Industry standard, well-tested
|
||||
- ✅ Declarative retry configuration
|
||||
- ✅ Built-in exponential backoff
|
||||
- ✅ Already used in project (check `poetry.lock`)
|
||||
- ❌ New dependency if not already present
|
||||
|
||||
**Option 2: Manual retry loop**
|
||||
- ✅ No new dependencies
|
||||
- ✅ Full control over logic
|
||||
- ❌ More code to maintain
|
||||
- ❌ Easy to get wrong (edge cases)
|
||||
|
||||
**Decision:** Use **tenacity** if already in dependencies, otherwise implement manual retry loop.
|
||||
|
||||
### Retry Strategy
|
||||
|
||||
```python
|
||||
# Retry on network errors only (RequestError)
|
||||
# Do NOT retry on:
|
||||
# - HTTP 4xx (client errors like 400, 401, 404)
|
||||
# - Already completed downloads
|
||||
# Do retry on:
|
||||
# - httpx.RequestError (connection timeout, refused, DNS failure)
|
||||
# - HTTP 5xx (server errors - service temporarily down)
|
||||
|
||||
Max retries: 8 (total 9 attempts)
|
||||
Backoff: Exponential with cap, multiplier=2, min=10s, max=60s
|
||||
- Attempt 1: immediate
|
||||
- Attempt 2: wait 10s
|
||||
- Attempt 3: wait 20s
|
||||
- Attempt 4: wait 40s
|
||||
- Attempt 5: wait 60s (capped)
|
||||
- Attempt 6: wait 60s (capped)
|
||||
- Attempt 7: wait 60s (capped)
|
||||
- Attempt 8: wait 60s (capped)
|
||||
- Attempt 9: wait 60s (capped)
|
||||
Total max time: ~370s (6+ minutes) of backoff + request timeouts
|
||||
|
||||
This gives the flacfetch service time to recover from restarts,
|
||||
network issues, or temporary VM problems.
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Add to `backend/config.py`:
|
||||
```python
|
||||
# Flacfetch client retry settings
|
||||
flacfetch_retry_max_attempts: int = int(os.getenv("FLACFETCH_RETRY_MAX_ATTEMPTS", "9")) # Total attempts (1 initial + 8 retries)
|
||||
flacfetch_retry_min_wait: float = float(os.getenv("FLACFETCH_RETRY_MIN_WAIT", "10.0")) # seconds
|
||||
flacfetch_retry_max_wait: float = float(os.getenv("FLACFETCH_RETRY_MAX_WAIT", "60.0")) # seconds
|
||||
```
|
||||
|
||||
### Error Message Improvements
|
||||
|
||||
Change from:
|
||||
```
|
||||
Download by ID request failed:
|
||||
```
|
||||
|
||||
To:
|
||||
```
|
||||
Download by ID request failed after 4 attempts: [ConnectTimeout] Connection to http://10.0.0.5:8080 timed out
|
||||
```
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Check Dependencies
|
||||
1. [ ] Check if `tenacity` is in `pyproject.toml` dependencies
|
||||
2. [ ] If not present, add `tenacity = "^8.2.0"` to dependencies
|
||||
3. [ ] Run `poetry lock` if dependencies changed
|
||||
|
||||
### Phase 2: Add Configuration
|
||||
4. [ ] Add retry configuration to `backend/config.py`
|
||||
- `flacfetch_retry_max_attempts = 9`
|
||||
- `flacfetch_retry_min_wait = 10.0`
|
||||
- `flacfetch_retry_max_wait = 60.0`
|
||||
|
||||
### Phase 3: Implement Flacfetch Retry Logic
|
||||
5. [ ] Import tenacity in `backend/services/flacfetch_client.py`
|
||||
6. [ ] Create retry decorator with:
|
||||
- Retry on `httpx.RequestError`
|
||||
- Retry on `httpx.HTTPStatusError` with 5xx status
|
||||
- Stop on `httpx.HTTPStatusError` with 4xx status
|
||||
- Exponential backoff with cap at 60s
|
||||
- Before-sleep logging callback
|
||||
7. [ ] Apply decorator to `download_by_id()` method
|
||||
8. [ ] Improve error messages to include:
|
||||
- Number of attempts made
|
||||
- Original exception type and message
|
||||
|
||||
### Phase 4: Persist Download Info Before Download
|
||||
9. [ ] Update `audio_search.py:_download_and_start_processing()`
|
||||
10. [ ] After getting source info from selected result (line 307-311)
|
||||
11. [ ] **Save to job document BEFORE download attempt:**
|
||||
```python
|
||||
job_manager.update_job(job_id, {
|
||||
'audio_source_type': 'audio_search',
|
||||
'source_name': source_name,
|
||||
'source_id': source_id,
|
||||
'target_file': target_file,
|
||||
'download_url': download_url,
|
||||
})
|
||||
```
|
||||
12. [ ] Then proceed with download (existing code)
|
||||
13. [ ] Add to Job model if needed (check if fields exist)
|
||||
|
||||
### Phase 5: Update Retry Endpoint for Audio Search
|
||||
14. [ ] Update `backend/api/routes/jobs.py:retry_job()`
|
||||
15. [ ] Add new case before "no input audio" error (line ~1088)
|
||||
16. [ ] Check if job has saved download params:
|
||||
```python
|
||||
elif job.audio_source_type == 'audio_search' and job.source_id:
|
||||
# Has audio search download info - retry download
|
||||
```
|
||||
17. [ ] Re-call audio search download logic with saved params
|
||||
18. [ ] Or refactor download into reusable function
|
||||
|
||||
### Phase 6: Frontend Error Handling
|
||||
19. [ ] Update `frontend/components/job/JobActions.tsx`
|
||||
20. [ ] Add toast notification library (check if exists)
|
||||
21. [ ] In `handleRetry()` catch block, show toast:
|
||||
```typescript
|
||||
catch (error) {
|
||||
const message = error.response?.data?.detail || "Failed to retry job"
|
||||
toast.error(message)
|
||||
console.error("Failed to retry job:", error)
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 7: Testing - Flacfetch Retry
|
||||
22. [ ] Add unit tests to `backend/tests/test_flacfetch_client.py`
|
||||
23. [ ] Test retry on RequestError (mock fails N times then succeeds)
|
||||
24. [ ] Test no retry on 4xx errors
|
||||
25. [ ] Test retry on 5xx errors
|
||||
26. [ ] Test max retries exhausted
|
||||
27. [ ] Verify logging output includes attempt numbers
|
||||
|
||||
### Phase 8: Testing - End-to-End
|
||||
28. [ ] Test audio search job with flacfetch down
|
||||
29. [ ] Verify source info is saved even when download fails
|
||||
30. [ ] Verify retry button works after source info saved
|
||||
31. [ ] Test retry succeeds when flacfetch comes back up
|
||||
32. [ ] Verify error toast appears when retry fails
|
||||
|
||||
### Phase 9: Documentation
|
||||
33. [ ] Update `backend/services/flacfetch_client.py` docstring
|
||||
34. [ ] Add to `docs/LESSONS-LEARNED.md`:
|
||||
- "Always add retries for external service calls"
|
||||
- "Save operation parameters before attempting operations"
|
||||
- "Persist enough state to enable retry from any failure point"
|
||||
35. [ ] Update `docs/ARCHITECTURE.md` if needed
|
||||
36. [ ] Document the new job fields in API.md
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/config.py` | Modify | Add retry configuration variables |
|
||||
| `backend/services/flacfetch_client.py` | Modify | Add retry logic to `download_by_id()` |
|
||||
| `backend/api/routes/audio_search.py` | Modify | Save download info before attempting download |
|
||||
| `backend/api/routes/jobs.py` | Modify | Add audio search case to retry endpoint |
|
||||
| `backend/models/job.py` | Modify (maybe) | Add fields: source_name, source_id, target_file, download_url |
|
||||
| `frontend/components/job/JobActions.tsx` | Modify | Show error toast when retry fails |
|
||||
| `backend/tests/test_flacfetch_client.py` | Modify | Add retry tests |
|
||||
| `backend/tests/test_audio_search.py` | Modify | Test download info persistence |
|
||||
| `backend/tests/test_jobs.py` | Modify | Test retry with audio search jobs |
|
||||
| `pyproject.toml` | Modify (maybe) | Add tenacity if not present |
|
||||
| `docs/LESSONS-LEARNED.md` | Modify | Document patterns |
|
||||
| `docs/API.md` | Modify | Document new job fields |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests (`backend/tests/test_flacfetch_client.py`)
|
||||
|
||||
1. **Test retry on RequestError**
|
||||
```python
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_by_id_retries_on_network_error():
|
||||
"""Should retry 3 times on connection errors."""
|
||||
client = FlacfetchClient(base_url="http://test", api_key="key")
|
||||
|
||||
with patch("httpx.AsyncClient.post") as mock_post:
|
||||
# Fail 2 times, then succeed
|
||||
mock_post.side_effect = [
|
||||
httpx.RequestError("Connection timeout"),
|
||||
httpx.RequestError("Connection refused"),
|
||||
Mock(json=lambda: {"download_id": "test123"}, status_code=200),
|
||||
]
|
||||
|
||||
result = await client.download_by_id("YouTube", "abc123")
|
||||
|
||||
assert result == "test123"
|
||||
assert mock_post.call_count == 3
|
||||
```
|
||||
|
||||
2. **Test no retry on 4xx errors**
|
||||
```python
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_by_id_no_retry_on_client_error():
|
||||
"""Should NOT retry on 400/401/404 errors."""
|
||||
client = FlacfetchClient(base_url="http://test", api_key="key")
|
||||
|
||||
with patch("httpx.AsyncClient.post") as mock_post:
|
||||
response = Mock(status_code=404, text="Not found")
|
||||
response.raise_for_status.side_effect = httpx.HTTPStatusError(
|
||||
"Not found", request=Mock(), response=response
|
||||
)
|
||||
mock_post.return_value = response
|
||||
|
||||
with pytest.raises(FlacfetchServiceError, match="404"):
|
||||
await client.download_by_id("YouTube", "abc123")
|
||||
|
||||
assert mock_post.call_count == 1 # No retries
|
||||
```
|
||||
|
||||
3. **Test retry on 5xx errors**
|
||||
```python
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_by_id_retries_on_server_error():
|
||||
"""Should retry on 500/503 errors."""
|
||||
# Similar to test 1 but with HTTP 503 errors
|
||||
```
|
||||
|
||||
4. **Test max retries exhausted**
|
||||
```python
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_by_id_fails_after_max_retries():
|
||||
"""Should fail with clear message after max retries."""
|
||||
# All attempts fail, verify error message includes attempt count
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
|
||||
5. **Test with actual flacfetch service** (if available in test environment)
|
||||
- Verify retry doesn't break normal operation
|
||||
- Test with deliberate network issues
|
||||
|
||||
### Manual Testing
|
||||
|
||||
6. **Simulate network failure**
|
||||
```bash
|
||||
# Temporarily stop flacfetch VM
|
||||
gcloud compute instances stop flacfetch-service --zone=us-central1-a
|
||||
|
||||
# Submit a test job
|
||||
# Observe logs showing retry attempts
|
||||
|
||||
# Restart VM during retries
|
||||
gcloud compute instances start flacfetch-service --zone=us-central1-a
|
||||
|
||||
# Job should succeed
|
||||
```
|
||||
|
||||
7. **Test with intentional delays**
|
||||
- Add sleep in flacfetch API handler
|
||||
- Verify retries don't cause issues
|
||||
|
||||
## Open Questions
|
||||
|
||||
- [x] ~~Is tenacity already in dependencies?~~ (Will check during implementation)
|
||||
- [x] ~~What's the appropriate total timeout budget?~~ - 9 attempts over ~6 minutes
|
||||
- [x] ~~Should we fix the Retry button UX in this PR?~~ - YES, it's the same root cause
|
||||
- [x] ~~Why did job 749141f8 fail before saving audio source info?~~ - Download info saved AFTER download, not before
|
||||
- [ ] Should we also add retries to `wait_for_download()`?
|
||||
- Probably NO - that already has its own polling/timeout logic
|
||||
- [ ] Should we add circuit breaker pattern?
|
||||
- Probably NO for now - keep it simple
|
||||
- Could add in future if we see cascading failures
|
||||
- [ ] Should retry logic be applied to other flacfetch methods?
|
||||
- `start_download()` - probably YES (same pattern)
|
||||
- `search()` - maybe YES (search can also have network issues)
|
||||
- `get_download_status()` - probably NO (polling already handles this)
|
||||
- **Decision**: Apply to `start_download()` too, skip `search()` for now
|
||||
- [ ] Which toast library should frontend use?
|
||||
- Check if already exists (react-hot-toast, sonner, etc.)
|
||||
- If not, add lightweight library
|
||||
- [ ] Should we save download params to state_data or top-level fields?
|
||||
- Top-level fields more visible and easier to query
|
||||
- **Decision**: Top-level fields (source_id, source_name, etc.)
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If this causes issues:
|
||||
|
||||
1. **Quick rollback**: Set env var `FLACFETCH_RETRY_MAX_ATTEMPTS=1` in Cloud Run
|
||||
- Disables retries without code change
|
||||
- Can deploy in ~2 minutes
|
||||
|
||||
2. **Full rollback**: Revert the PR
|
||||
- Git revert the merge commit
|
||||
- Deploy previous version
|
||||
|
||||
3. **Monitoring**: Watch for:
|
||||
- Increased request duration to flacfetch
|
||||
- Jobs timing out at higher layer (Cloud Tasks, Cloud Run)
|
||||
- Duplicate download attempts causing issues
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Jobs with transient flacfetch failures succeed instead of failing
|
||||
- [ ] Error messages are more informative (show attempts and root cause)
|
||||
- [ ] No increase in false positives (jobs that should fail, now hang)
|
||||
- [ ] All tests pass
|
||||
- [ ] No impact on successful jobs (retry overhead is minimal)
|
||||
|
||||
## Related Issues
|
||||
|
||||
### Job 749141f8 - The Incident
|
||||
|
||||
**What happened:**
|
||||
- User searched for audio via audio search
|
||||
- Selected "Elizabeth Moen - Songbird"
|
||||
- Backend tried to download via flacfetch
|
||||
- Flacfetch call failed with network error (connection timeout/refused)
|
||||
- Job immediately failed
|
||||
- No source info saved → Retry button broken
|
||||
- User sees "Retry" button but clicking does nothing (silent 400 error)
|
||||
|
||||
**How this fix helps:**
|
||||
1. ✅ Flacfetch retry logic → Download would have succeeded after retry
|
||||
2. ✅ Persist download info → Even if all retries fail, user can retry later
|
||||
3. ✅ Retry endpoint fix → Retry button will actually work
|
||||
4. ✅ Frontend toast → User sees helpful error if retry still fails
|
||||
|
||||
**Note:** Job 749141f8 cannot be fixed retroactively (no source info saved). But future jobs with same issue will be fixable via retry button.
|
||||
|
||||
### Future Work (Not in This PR)
|
||||
|
||||
Similar pattern should be applied to other external service calls:
|
||||
- AudioShake API (lyrics transcription)
|
||||
- Modal API (audio separation)
|
||||
- YouTube direct API calls (not via flacfetch)
|
||||
- Email sending (SendGrid)
|
||||
|
||||
## References
|
||||
|
||||
- [Tenacity documentation](https://tenacity.readthedocs.io/)
|
||||
- [LESSONS-LEARNED.md](docs/LESSONS-LEARNED.md) - External service response format mismatches
|
||||
- [flacfetch README](../../flacfetch/README.md) - Understanding the service
|
||||
@@ -1,149 +0,0 @@
|
||||
# Plan: Improve Job Observability
|
||||
|
||||
**Created:** 2026-03-07
|
||||
**Branch:** feat/sess-20260307-2216-improve-job-observability
|
||||
**Status:** Draft
|
||||
|
||||
## Overview
|
||||
|
||||
After shipping the "edit completed tracks" feature, we found it very difficult to trace what actually happened to edited jobs. Timeline events are too sparse (no output details), cleanup operations aren't logged to Cloud Logging or Firestore, state_data gets fully overwritten during re-processing (losing history of previous outputs), and API endpoints don't write to the job's Firestore log subcollection.
|
||||
|
||||
The goal is to make it trivially easy for future agents and humans to reconstruct the complete lifecycle of any job by reading its timeline and Firestore logs.
|
||||
|
||||
## Problem Analysis
|
||||
|
||||
### What we found investigating two edited jobs:
|
||||
|
||||
1. **Timeline events only have `status`, `timestamp`, `progress`, `message`** — no output details like YouTube video IDs, brand codes, Dropbox links, or GDrive file IDs
|
||||
2. **Edit endpoint cleanup results never logged** — the `cleanup_results` dict is built and returned in the HTTP response but never logged to Cloud Logging or the job's Firestore log subcollection
|
||||
3. **Distribution details incomplete in logs** — YouTube logs the URL but not video_id separately; Dropbox doesn't log folder path; GDrive doesn't log file IDs
|
||||
4. **`state_data` fully overwritten on re-processing** — `video_worker.py:342-354` does `{**job.state_data, ...}`, so any `DELETE_FIELD` operations from the edit endpoint get lost when old keys are re-merged from the in-memory copy
|
||||
5. **`outputs_deleted_at` cleared by video worker** — intentionally, but loses audit trail
|
||||
6. **No Firestore job logs from API endpoints** — edit/admin endpoints only log to Cloud Run stdout, not to `jobs/{job_id}/logs/`
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] Add `metadata` dict to `TimelineEvent` for structured output details
|
||||
- [ ] Log all cleanup operations (success AND failure) to both Cloud Logging and Firestore job logs
|
||||
- [ ] Log distribution output details (YouTube video ID, Dropbox path, brand code, GDrive file IDs) in timeline events and Firestore logs
|
||||
- [ ] Preserve previous output details in timeline when a job is edited (snapshot before cleanup)
|
||||
- [ ] Add helper to write Firestore job logs from API endpoints (not just workers)
|
||||
- [ ] Ensure edit/admin delete endpoints write to Firestore job logs
|
||||
- [ ] No breaking changes to existing timeline format (additive only)
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### 1. Add `metadata` field to `TimelineEvent`
|
||||
|
||||
The `TimelineEvent` model gains an optional `metadata: Optional[Dict[str, Any]]` field. This allows any timeline event to carry structured data without changing the schema. The field is additive — existing events without metadata continue to work.
|
||||
|
||||
Key events that gain metadata:
|
||||
- **`complete`**: `{youtube_url, youtube_video_id, dropbox_link, brand_code, gdrive_file_ids}`
|
||||
- **Edit initiated**: `{previous_outputs: {youtube_url, dropbox_link, brand_code, gdrive_files}, cleanup_results: {...}}`
|
||||
- **Outputs deleted**: `{deleted_outputs: {...}, cleanup_results: {...}}`
|
||||
|
||||
### 2. Write Firestore job logs from API endpoints
|
||||
|
||||
Create a simple helper function that API route handlers can call to append logs to the job's Firestore subcollection. Workers already have `JobLogAdapter` — we just need a lightweight equivalent for API routes:
|
||||
|
||||
```python
|
||||
def log_to_job(job_id: str, worker: str, level: str, message: str, metadata: dict = None):
|
||||
"""Write a log entry to the job's Firestore log subcollection."""
|
||||
entry = WorkerLogEntry.create(job_id, worker, level, message, metadata)
|
||||
FirestoreService().append_log_to_subcollection(job_id, entry)
|
||||
```
|
||||
|
||||
### 3. Enhance distribution logging in video_worker_orchestrator
|
||||
|
||||
After each distribution upload succeeds, log the details with structured metadata:
|
||||
- YouTube: video_id, video_url, channel
|
||||
- Dropbox: folder_path, sharing_link
|
||||
- GDrive: file_ids dict
|
||||
- Brand code: code, prefix, allocated vs recycled
|
||||
|
||||
### 4. Snapshot previous outputs on edit
|
||||
|
||||
Before the edit endpoint clears state_data distribution keys, capture the current values in the timeline event's metadata. This preserves the history permanently in the timeline array (which is append-only via `ArrayUnion`).
|
||||
|
||||
### 5. Log cleanup results from edit/admin endpoints
|
||||
|
||||
After cleanup runs, log each step's result to both Cloud Logging and the Firestore job log subcollection with structured metadata.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Step 1: Add `metadata` to `TimelineEvent` model
|
||||
- [ ] Add `metadata: Optional[Dict[str, Any]] = None` to `TimelineEvent` in `backend/models/job.py`
|
||||
- [ ] No migration needed — Firestore is schemaless, existing events just won't have the field
|
||||
|
||||
### Step 2: Create `log_to_job` helper
|
||||
- [ ] Add `log_to_job()` function in `backend/services/firestore_service.py` (or a small utility module)
|
||||
- [ ] Takes `job_id`, `worker`, `level`, `message`, optional `metadata`
|
||||
- [ ] Wraps `WorkerLogEntry.create()` + `append_log_to_subcollection()`
|
||||
- [ ] Catches and suppresses errors (logging should never break the caller)
|
||||
|
||||
### Step 3: Update `transition_to_state` to accept metadata
|
||||
- [ ] Add optional `timeline_metadata: Optional[Dict[str, Any]] = None` parameter to `job_manager.transition_to_state()`
|
||||
- [ ] Pass through to `update_job_status()` → `TimelineEvent` construction
|
||||
- [ ] Update `firestore_service.update_job_status()` to accept and pass `timeline_metadata`
|
||||
|
||||
### Step 4: Enhance edit endpoint logging
|
||||
- [ ] Before cleanup: snapshot current distribution outputs (`youtube_url`, `dropbox_link`, `brand_code`, `gdrive_files`) into `previous_outputs` dict
|
||||
- [ ] After each cleanup step: log result to Cloud Logging AND call `log_to_job()`
|
||||
- [ ] After all cleanup: include `previous_outputs` and `cleanup_results` in the timeline event metadata
|
||||
- [ ] The "Track edit initiated" timeline event gains metadata: `{initiated_by, previous_outputs, cleanup_results}`
|
||||
|
||||
### Step 5: Enhance admin delete-outputs endpoint logging
|
||||
- [ ] Same pattern as edit endpoint: snapshot outputs, log each cleanup step, include in timeline metadata
|
||||
- [ ] The "Outputs deleted by admin" timeline event gains metadata: `{deleted_by, deleted_outputs, cleanup_results}`
|
||||
|
||||
### Step 6: Enhance distribution logging in video_worker_orchestrator
|
||||
- [ ] YouTube upload success: log video_id and video_url with metadata `{video_id, video_url}`
|
||||
- [ ] Dropbox upload success: log folder_path and sharing_link with metadata `{folder_path, sharing_link}`
|
||||
- [ ] GDrive upload success: log file_ids with metadata `{file_ids: {...}}`
|
||||
- [ ] Brand code allocation: log with metadata `{brand_code, source: "allocated"|"recycled"}`
|
||||
|
||||
### Step 7: Add output details to `complete` timeline event
|
||||
- [ ] When `video_worker.py` calls `transition_to_state(COMPLETE)`, pass `timeline_metadata` with distribution output details
|
||||
- [ ] Include: `youtube_url`, `youtube_video_id`, `dropbox_link`, `brand_code`, `gdrive_file_ids`, `edit_count`
|
||||
|
||||
### Step 8: Tests
|
||||
- [ ] Unit test: `TimelineEvent` with metadata serializes correctly
|
||||
- [ ] Unit test: `log_to_job` writes to subcollection
|
||||
- [ ] Unit test: edit endpoint includes `previous_outputs` and `cleanup_results` in timeline event
|
||||
- [ ] Unit test: `complete` transition includes output metadata
|
||||
- [ ] Verify existing tests still pass
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/models/job.py` | Modify | Add `metadata` field to `TimelineEvent` |
|
||||
| `backend/services/firestore_service.py` | Modify | Add `log_to_job()` helper, update `update_job_status()` to accept timeline metadata |
|
||||
| `backend/services/job_manager.py` | Modify | Add `timeline_metadata` param to `transition_to_state()` |
|
||||
| `backend/api/routes/jobs.py` | Modify | Enhance edit endpoint with cleanup logging, output snapshots, timeline metadata |
|
||||
| `backend/api/routes/admin.py` | Modify | Enhance delete-outputs endpoint similarly |
|
||||
| `backend/workers/video_worker_orchestrator.py` | Modify | Add structured metadata to distribution log messages |
|
||||
| `backend/workers/video_worker.py` | Modify | Pass output metadata in `complete` transition (both orchestrator and legacy paths) |
|
||||
| `backend/tests/test_edit_completed_track.py` | Modify | Verify timeline metadata in edit tests |
|
||||
| `backend/tests/test_observability.py` | Create | Tests for TimelineEvent metadata and log_to_job |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit tests**: TimelineEvent metadata serialization, log_to_job helper, edit endpoint timeline metadata assertions
|
||||
- **Existing test verification**: `make test` must pass — changes are additive so existing tests should be unaffected
|
||||
- **No E2E tests needed**: This is purely backend logging/observability — no UI changes
|
||||
|
||||
## What This Does NOT Include
|
||||
|
||||
- **Dedicated admin UI for viewing job logs** — existing admin page already shows timeline
|
||||
- **Log aggregation/search UI** — Cloud Logging and Firestore viewer already serve this purpose
|
||||
- **Changing state_data overwrite behavior** — the `{**job.state_data, ...}` pattern in video_worker.py is intentional; instead we preserve history in timeline events
|
||||
- **Changing outputs_deleted_at clearing** — video worker clearing it after re-processing is correct behavior; the edit history lives in timeline metadata instead
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
All changes are additive (new optional field on TimelineEvent, additional log writes). Rolling back just means deploying the previous version — no data migration needed.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None — the approach is straightforward and low-risk.
|
||||
@@ -1,279 +0,0 @@
|
||||
# Plan: Private (Non-Published) Tracks
|
||||
|
||||
**Created:** 2026-02-24
|
||||
**Branch:** feat/sess-20260224-1149-private-tracks
|
||||
**Status:** Implemented
|
||||
|
||||
## Overview
|
||||
|
||||
Add a `is_private` flag to jobs that changes distribution behavior:
|
||||
- **Dropbox**: Uses `Tracks-NonPublished` folder (not `Tracks-Organized`) with brand code `NOMADNP` (not `NOMAD`)
|
||||
- **YouTube**: Skipped entirely
|
||||
- **Google Drive**: Skipped entirely
|
||||
- **Email**: Completion email includes Dropbox link but no YouTube section
|
||||
- **Discord**: Still notified (for internal tracking)
|
||||
|
||||
Available to all users at job creation and toggleable by admin on existing jobs.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] All users can mark a job as "Private (Non-Published)" when creating it
|
||||
- [ ] Admin can toggle `is_private` on any existing job via admin dashboard
|
||||
- [ ] Private jobs upload to Dropbox at `Tracks-NonPublished` with `NOMADNP-XXXX` brand codes
|
||||
- [ ] Private jobs skip YouTube and Google Drive uploads entirely
|
||||
- [ ] Completion emails omit YouTube section when no YouTube URL (private jobs + any other edge case)
|
||||
- [ ] When admin toggles a completed job to private, existing outputs are auto-deleted
|
||||
- [ ] Discord notifications still fire for private jobs
|
||||
- [ ] Existing (non-private) jobs are completely unaffected (backward compatible)
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Core Design: Single Boolean Flag with Runtime Overrides
|
||||
|
||||
Store `is_private: bool = False` on the job model. At distribution time, a helper function returns effective distribution parameters based on this flag. This means:
|
||||
|
||||
- **Toggling is_private automatically changes all distribution behavior** - no need to manually update dropbox_path, brand_prefix, enable_youtube_upload, gdrive_folder_id
|
||||
- **Job fields still store the "normal" defaults** - the override happens at the config-to-orchestrator boundary
|
||||
- **Admin can flip one field** and everything adapts
|
||||
|
||||
### Key Helper Function
|
||||
|
||||
```python
|
||||
# In job_defaults_service.py
|
||||
def get_effective_distribution_for_job(job) -> EffectiveDistributionSettings:
|
||||
"""Return distribution settings, applying private overrides if is_private=True."""
|
||||
if getattr(job, 'is_private', False):
|
||||
settings = get_settings()
|
||||
return EffectiveDistributionSettings(
|
||||
dropbox_path=settings.default_private_dropbox_path,
|
||||
brand_prefix=settings.default_private_brand_prefix,
|
||||
enable_youtube_upload=False,
|
||||
gdrive_folder_id=None,
|
||||
# Keep other settings from job (discord, youtube_description, etc.)
|
||||
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
|
||||
youtube_description=None,
|
||||
)
|
||||
# Non-private: return job's own settings
|
||||
return EffectiveDistributionSettings(
|
||||
dropbox_path=getattr(job, 'dropbox_path', None),
|
||||
brand_prefix=getattr(job, 'brand_prefix', None),
|
||||
enable_youtube_upload=getattr(job, 'enable_youtube_upload', False),
|
||||
gdrive_folder_id=getattr(job, 'gdrive_folder_id', None),
|
||||
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
|
||||
youtube_description=getattr(job, 'youtube_description_template', None),
|
||||
)
|
||||
```
|
||||
|
||||
This helper is used in both distribution paths (orchestrator and native fallback).
|
||||
|
||||
### Distribution Paths That Need Changes
|
||||
|
||||
Two code paths handle distribution (see LESSONS-LEARNED: "Fix Both Sides of Dual Code Paths"):
|
||||
|
||||
1. **Orchestrator path** (`video_worker_orchestrator.py`): Primary path. `create_orchestrator_config_from_job()` uses the helper to build config.
|
||||
2. **Native distribution path** (`video_worker.py` → `_handle_native_distribution()`): Fallback path. Also uses the helper instead of reading job fields directly.
|
||||
|
||||
### Email Template Change
|
||||
|
||||
Add conditional YouTube section removal in `render_template()`, matching the existing pattern for `feedback_url`:
|
||||
|
||||
```python
|
||||
# Remove YouTube section if no URL provided
|
||||
if not variables.get("youtube_url"):
|
||||
result = re.sub(
|
||||
r"\n*Here's the link for the karaoke video published to YouTube:\n\{youtube_url\}\n*",
|
||||
'\n',
|
||||
result,
|
||||
flags=re.DOTALL
|
||||
)
|
||||
variables["youtube_url"] = ""
|
||||
```
|
||||
|
||||
And change `render_job_completion()` to pass `None` instead of `"[YouTube URL not available]"` when youtube_url is empty.
|
||||
|
||||
### Admin Toggle Auto-Delete
|
||||
|
||||
When admin PATCH updates `is_private` from False to True on a job that has existing outputs, reuse the existing delete-outputs logic to remove YouTube + GDrive + Dropbox outputs. Admin can then "Reprocess" to re-distribute to the private Dropbox path.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Backend Model & Config (foundation)
|
||||
|
||||
1. [ ] **Add `is_private` to Job model** (`backend/models/job.py`)
|
||||
- Add `is_private: bool = False` field after `made_for_you` field
|
||||
- Add to `JobCreate` model as well
|
||||
|
||||
2. [ ] **Add private distribution config** (`backend/config.py`)
|
||||
- Add `default_private_dropbox_path: Optional[str] = os.getenv("DEFAULT_PRIVATE_DROPBOX_PATH")`
|
||||
- Add `default_private_brand_prefix: Optional[str] = os.getenv("DEFAULT_PRIVATE_BRAND_PREFIX", "NOMADNP")`
|
||||
|
||||
3. [ ] **Add helper function** (`backend/services/job_defaults_service.py`)
|
||||
- Add `get_effective_distribution_for_job(job)` function that returns distribution settings with private overrides applied
|
||||
|
||||
### Phase 2: Backend API (job creation + admin)
|
||||
|
||||
4. [ ] **Add `is_private` to request models** (`backend/models/requests.py`)
|
||||
- Add `is_private: Optional[bool] = None` to `URLSubmissionRequest`
|
||||
- Add `is_private: Optional[bool] = None` to `AudioSearchRequest`
|
||||
|
||||
5. [ ] **Update URL creation endpoint** (`backend/api/routes/jobs.py`)
|
||||
- Accept and pass through `is_private` field to `JobCreate`
|
||||
|
||||
6. [ ] **Update file upload endpoint** (`backend/api/routes/file_upload.py`)
|
||||
- Accept `is_private` Form parameter
|
||||
- Pass through to `JobCreate`
|
||||
|
||||
7. [ ] **Update audio search endpoint** (`backend/api/routes/audio_search.py`)
|
||||
- Accept `is_private` in `AudioSearchRequest`
|
||||
- Pass through to `JobCreate`
|
||||
|
||||
8. [ ] **Update admin endpoints** (`backend/api/routes/admin.py`)
|
||||
- Add `is_private: Optional[bool] = None` to `JobUpdateRequest`
|
||||
- Add `"is_private"` to `EDITABLE_JOB_FIELDS`
|
||||
- In the PATCH handler: detect when `is_private` changes from False→True on a completed job, auto-delete existing outputs (call existing delete-outputs service logic)
|
||||
|
||||
### Phase 3: Distribution Pipeline
|
||||
|
||||
9. [ ] **Update orchestrator config creation** (`backend/workers/video_worker_orchestrator.py`)
|
||||
- In `create_orchestrator_config_from_job()`: Use `get_effective_distribution_for_job()` helper to determine dropbox_path, brand_prefix, enable_youtube_upload, gdrive_folder_id instead of reading directly from job
|
||||
|
||||
10. [ ] **Update native distribution fallback** (`backend/workers/video_worker.py`)
|
||||
- In `_handle_native_distribution()`: Use `get_effective_distribution_for_job()` helper instead of reading directly from job fields
|
||||
|
||||
### Phase 4: Email Template
|
||||
|
||||
11. [ ] **Add YouTube section conditional removal** (`backend/services/template_service.py`)
|
||||
- In `render_template()`: Add regex to remove YouTube paragraph when `youtube_url` is empty (matching existing `feedback_url` pattern)
|
||||
- In `render_job_completion()`: Pass `None` for youtube_url when empty (instead of `"[YouTube URL not available]"`)
|
||||
|
||||
### Phase 5: Frontend
|
||||
|
||||
12. [ ] **Add private checkbox to job creation form** (`frontend/components/job/JobSubmission.tsx`)
|
||||
- Add "Private (Non-Published)" checkbox below the existing "Skip lyrics review" checkbox
|
||||
- Show for all users (not just admin)
|
||||
- Include brief description text: "Private tracks go to Dropbox only, not YouTube or Google Drive"
|
||||
|
||||
13. [ ] **Update API interfaces** (`frontend/lib/api.ts`)
|
||||
- Add `is_private?: boolean` to all job creation option interfaces
|
||||
- Add `is_private` to `JobUpdateRequest` interface
|
||||
- Pass through in `uploadJob()`, `createJobFromUrl()`, `searchAudio()` methods
|
||||
|
||||
14. [ ] **Add private toggle to admin job detail** (`frontend/app/admin/jobs/page.tsx`)
|
||||
- Add `is_private` as an inline-editable field in the job overview section (use Switch component)
|
||||
- Show current state with visual indicator (e.g., lock icon for private jobs)
|
||||
- When toggling to private on a completed job, show confirmation dialog about auto-deleting outputs
|
||||
|
||||
### Phase 6: Environment Configuration
|
||||
|
||||
15. [ ] **Update CI/deployment config** (`.github/workflows/ci.yml` or Cloud Run env)
|
||||
- Add `DEFAULT_PRIVATE_DROPBOX_PATH` environment variable (e.g., `/Karaoke/Tracks-NonPublished`)
|
||||
- Add `DEFAULT_PRIVATE_BRAND_PREFIX` environment variable (e.g., `NOMADNP`)
|
||||
- These should be configured in Cloud Run service and CI test environment
|
||||
|
||||
### Phase 7: Testing
|
||||
|
||||
16. [ ] **Unit tests for private distribution logic** (`backend/tests/test_private_tracks.py`)
|
||||
- Test `get_effective_distribution_for_job()` with private and non-private jobs
|
||||
- Test `create_orchestrator_config_from_job()` applies private overrides correctly
|
||||
- Test `_run_distribution()` skips YouTube and GDrive when private config applied
|
||||
- Test `_upload_to_dropbox()` uses private path and brand prefix
|
||||
- Test brand code generation with NOMADNP prefix in Tracks-NonPublished
|
||||
- Test `_handle_native_distribution()` respects private overrides
|
||||
- Test non-private jobs are completely unaffected (regression tests)
|
||||
|
||||
17. [ ] **Unit tests for email changes** (add to `backend/tests/test_email_service.py` or `test_template_service.py`)
|
||||
- Test YouTube section removed from email when youtube_url is None
|
||||
- Test YouTube section present when youtube_url is provided
|
||||
- Test Dropbox section still present for private jobs
|
||||
- Test full email rendering for private job scenario
|
||||
|
||||
18. [ ] **Unit tests for admin toggle** (add to `backend/tests/test_admin.py` or new file)
|
||||
- Test admin can update is_private field
|
||||
- Test toggling is_private to True on completed job triggers output deletion
|
||||
- Test toggling is_private to True on non-completed job does NOT delete outputs
|
||||
- Test toggling is_private to False does nothing special
|
||||
|
||||
19. [ ] **Integration test** (`backend/tests/emulator/` or `test_distribution_services.py`)
|
||||
- Test full distribution pipeline with is_private=True: verify Dropbox path, brand prefix, YouTube skip, GDrive skip
|
||||
|
||||
20. [ ] **Frontend E2E regression test** (`frontend/e2e/regression/`)
|
||||
- Test private checkbox appears in job creation form
|
||||
- Test private toggle in admin dashboard
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/models/job.py` | Modify | Add `is_private: bool = False` to Job and JobCreate |
|
||||
| `backend/config.py` | Modify | Add `default_private_dropbox_path` and `default_private_brand_prefix` |
|
||||
| `backend/services/job_defaults_service.py` | Modify | Add `get_effective_distribution_for_job()` helper |
|
||||
| `backend/models/requests.py` | Modify | Add `is_private` to URL and AudioSearch request models |
|
||||
| `backend/api/routes/jobs.py` | Modify | Accept and pass `is_private` |
|
||||
| `backend/api/routes/file_upload.py` | Modify | Accept and pass `is_private` |
|
||||
| `backend/api/routes/audio_search.py` | Modify | Accept and pass `is_private` |
|
||||
| `backend/api/routes/admin.py` | Modify | Add `is_private` to editable fields + auto-delete logic |
|
||||
| `backend/workers/video_worker_orchestrator.py` | Modify | Use helper in `create_orchestrator_config_from_job()` |
|
||||
| `backend/workers/video_worker.py` | Modify | Use helper in `_handle_native_distribution()` |
|
||||
| `backend/services/template_service.py` | Modify | Conditional YouTube section removal |
|
||||
| `frontend/components/job/JobSubmission.tsx` | Modify | Add private checkbox |
|
||||
| `frontend/lib/api.ts` | Modify | Add `is_private` to interfaces and API calls |
|
||||
| `frontend/app/admin/jobs/page.tsx` | Modify | Add private toggle to admin detail |
|
||||
| `backend/tests/test_private_tracks.py` | Create | Comprehensive unit tests for private track logic |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests (~25-30 tests in `test_private_tracks.py`)
|
||||
- `get_effective_distribution_for_job()` with private=True and False
|
||||
- `create_orchestrator_config_from_job()` private overrides (YouTube=False, GDrive=None, private Dropbox path, NOMADNP brand)
|
||||
- Distribution stage skips: YouTube not called, GDrive not called, Dropbox IS called with correct path
|
||||
- Brand code generation uses NOMADNP prefix and Tracks-NonPublished path
|
||||
- Native distribution fallback applies same overrides
|
||||
- **Regression tests**: Non-private jobs have unchanged behavior (same paths, same brand code, YouTube+GDrive+Dropbox all fire)
|
||||
|
||||
### Email Template Tests (~5 tests)
|
||||
- YouTube section stripped when youtube_url is None
|
||||
- YouTube section preserved when youtube_url has value
|
||||
- Dropbox section always present
|
||||
- Full private job email rendering (Dropbox + no YouTube)
|
||||
- Full non-private job email rendering (both present) - regression
|
||||
|
||||
### Admin Toggle Tests (~5 tests)
|
||||
- is_private editable via PATCH
|
||||
- Toggle to private on completed job deletes outputs
|
||||
- Toggle to private on pending/in-progress job does NOT delete
|
||||
- Toggle to non-private does nothing special
|
||||
|
||||
### Frontend Tests
|
||||
- E2E: Private checkbox visible and functional in job creation
|
||||
- E2E: Admin toggle visible and includes confirmation dialog
|
||||
|
||||
### Manual Verification
|
||||
- Create a private job end-to-end, verify Dropbox folder in Tracks-NonPublished with NOMADNP code
|
||||
- Verify completion email has Dropbox link but no YouTube section
|
||||
- Toggle existing completed job to private, verify outputs deleted
|
||||
|
||||
## Regression Risk Analysis
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|-----------|
|
||||
| Non-private jobs affected | `is_private` defaults to `False`; helper returns unchanged values for non-private. Explicit regression tests. |
|
||||
| Dual code paths (orchestrator vs native) | Both paths use same helper function. Tests cover both paths. (Lesson: "Fix Both Sides of Dual Code Paths") |
|
||||
| Email template regex breaks other content | Regex targets specific YouTube paragraph text. Tests verify non-YouTube content untouched. |
|
||||
| Brand code namespace collision | NOMADNP codes are in a completely separate Dropbox folder (Tracks-NonPublished), no overlap with NOMAD codes in Tracks-Organized |
|
||||
| Admin auto-delete on wrong job | Only triggers when `is_private` changes from False→True AND job has terminal status (complete). Confirmation dialog in frontend. |
|
||||
| Made-for-you orders | MFY webhook doesn't set is_private (defaults False). Admin can toggle after. No change to MFY flow. |
|
||||
| Existing Firestore documents | Missing `is_private` field defaults to `False` via model default. No migration needed. |
|
||||
|
||||
## Open Questions
|
||||
|
||||
- [x] Who sees private toggle? → **Everyone** (all users at creation, admin on existing)
|
||||
- [x] Auto-delete on toggle? → **Yes**, auto-delete all outputs when toggling to private on completed job
|
||||
- [x] Email format? → **Remove YouTube section** when no YouTube URL (improves all edge cases)
|
||||
- [x] Discord? → **Still notify** (useful for internal tracking)
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
1. **Feature flag**: The `is_private` field defaults to `False`, so setting `DEFAULT_PRIVATE_DROPBOX_PATH` to empty/None effectively disables private distribution
|
||||
2. **No schema migration needed**: Missing `is_private` field on existing Firestore documents defaults to `False`
|
||||
3. **Email change is backward-compatible**: The YouTube section removal only happens when youtube_url is None, which is already an edge case showing "[YouTube URL not available]"
|
||||
4. **Frontend**: Private checkbox is additive UI, removing it just means users can't create new private jobs
|
||||
@@ -1,239 +0,0 @@
|
||||
# Plan: Song Lookup & Autocomplete + Community Version Detection
|
||||
|
||||
**Created:** 2026-03-02
|
||||
**Branch:** feat/sess-20260302-2324-song-lookup-autocomplete
|
||||
**Status:** Implementation Complete
|
||||
|
||||
## Overview
|
||||
|
||||
Add two features to the karaoke-gen job creation flow:
|
||||
|
||||
1. **Artist/title autocomplete** using karaoke-decide's MusicBrainz + Spotify catalog data, so users get properly-styled artist/title text (e.g. "Fleetwood Mac" instead of "fleetwood mac")
|
||||
2. **Community karaoke version detection** that warns users when a high-quality community karaoke version already exists on YouTube (via karaokenerds data), saving them credits and time
|
||||
|
||||
Both features are **optional and non-blocking** — users can still type whatever they want and proceed even if a community version exists.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [ ] Artist field autocomplete across all 3 job creation tabs (Search, Upload, URL)
|
||||
- [ ] Title field autocomplete (optionally filtered by selected artist) across all 3 tabs
|
||||
- [ ] Autocomplete uses canonical MusicBrainz/Spotify text styling
|
||||
- [ ] Autocomplete is optional — users can ignore suggestions and type freely
|
||||
- [ ] Community version check triggers when both artist and title are filled
|
||||
- [ ] If community karaoke version exists, show a dismissible banner with YouTube link(s)
|
||||
- [ ] Community check does not block job submission — just informs the user
|
||||
- [ ] Graceful degradation if karaoke-decide API or karaokenerds.com is down
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
### How to access karaoke-decide catalog data
|
||||
|
||||
**Decision: Proxy through karaoke-gen backend**
|
||||
|
||||
Both services share the same GCP project (`nomadkaraoke`), but the frontend at `gen.nomadkaraoke.com` can't directly call `decide.nomadkaraoke.com/api/` due to CORS (different origins). Options considered:
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| **Proxy via karaoke-gen backend** | No CORS issues, no changes to karaoke-decide, frontend already calls api.nomadkaraoke.com | Added latency (extra hop), karaoke-decide dependency |
|
||||
| Direct frontend calls + CORS | Simpler, no proxy code | Requires karaoke-decide CORS changes, cross-project coupling |
|
||||
| Query BigQuery directly from karaoke-gen | No dependency on karaoke-decide service | Duplicates query logic, BigQuery setup needed, higher latency |
|
||||
|
||||
The proxy approach is cleanest: karaoke-gen frontend → karaoke-gen backend → karaoke-decide API. Response caching in the proxy minimizes latency impact.
|
||||
|
||||
### How to detect community karaoke versions
|
||||
|
||||
**Decision: Scrape karaokenerds.com from karaoke-gen backend** (same approach as kjbox)
|
||||
|
||||
The `karaokenerds_raw` BigQuery table has the catalog but NOT community status. Community status is determined by an `<img class="check">` badge in karaokenerds.com's HTML, only available via web scraping. The kjbox codebase has proven, well-tested scraping logic we can port directly.
|
||||
|
||||
Search URL: `https://karaokenerds.com/Search?query={artist}+{title}&webFilter=OnlyWeb`
|
||||
|
||||
Returns: songs with tracks, each track having `brand_name`, `brand_code`, `youtube_url`, `is_community`.
|
||||
|
||||
### Autocomplete UI component
|
||||
|
||||
**Decision: Use existing shadcn Command (cmdk) + Popover components**
|
||||
|
||||
The frontend already has `command.tsx` and `popover.tsx` from shadcn/ui. The standard combobox pattern (Popover + Command) provides:
|
||||
- Keyboard navigation (arrow keys, enter to select)
|
||||
- Debounced search-as-you-type
|
||||
- Clean dropdown with loading states
|
||||
- Works inline within forms (not modal)
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### Data flow
|
||||
|
||||
```
|
||||
User types in artist field
|
||||
→ debounced (300ms) API call to GET /api/catalog/artists?q=fle
|
||||
→ karaoke-gen backend proxies to decide.nomadkaraoke.com/api/catalog/artists?q=fle
|
||||
→ returns [{ name: "Fleetwood Mac", mbid: "...", popularity: 85 }, ...]
|
||||
→ user selects "Fleetwood Mac" → field populated with canonical name
|
||||
|
||||
User types in title field
|
||||
→ debounced (300ms) API call to GET /api/catalog/tracks?q=dreams&artist=Fleetwood+Mac
|
||||
→ returns [{ track_name: "Dreams", artist_name: "Fleetwood Mac", popularity: 82 }, ...]
|
||||
→ user selects "Dreams" → field populated with canonical title
|
||||
|
||||
Both fields filled → community check fires
|
||||
→ POST /api/catalog/community-check { artist: "Fleetwood Mac", title: "Dreams" }
|
||||
→ backend scrapes karaokenerds.com for "Fleetwood Mac Dreams" with webFilter=OnlyWeb
|
||||
→ if community tracks found with YouTube URLs, return them
|
||||
→ frontend shows banner: "A community karaoke version already exists! [Watch on YouTube →]"
|
||||
```
|
||||
|
||||
### Caching strategy
|
||||
|
||||
| Endpoint | Cache TTL | Scope |
|
||||
|----------|-----------|-------|
|
||||
| Artist search proxy | 5 min | In-memory (backend) |
|
||||
| Track search proxy | 5 min | In-memory (backend) |
|
||||
| Community check | 1 hour | In-memory (backend) |
|
||||
|
||||
### karaoke-decide API endpoints used
|
||||
|
||||
These are all **public, no auth required**:
|
||||
|
||||
- `GET /api/catalog/artists?q={query}&limit=10` — MBID-first artist search
|
||||
- Returns: `[{ mbid, name, disambiguation, artist_type, tags, spotify_id, popularity, genres }]`
|
||||
- `GET /api/catalog/tracks?q={query}&limit=10` — Track search (Spotify data)
|
||||
- Returns: `[{ track_id, track_name, artist_name, artist_id, popularity, duration_ms, explicit }]`
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Backend — Catalog proxy + community check endpoints
|
||||
|
||||
1. [ ] **Add catalog proxy service** — `backend/services/catalog_proxy_service.py`
|
||||
- HTTP client (httpx async) calling karaoke-decide API
|
||||
- In-memory TTL cache for responses (5 min)
|
||||
- Configurable karaoke-decide base URL
|
||||
- Graceful error handling (return empty results on failure)
|
||||
|
||||
2. [ ] **Port karaokenerds scraping logic** — `backend/services/karaokenerds_service.py`
|
||||
- Port `search()`, `parse_results()`, `_parse_tracks()`, `_parse_single_track()`, `_clean_youtube_url()` from `kjbox/kj-controller/karaoke_nerds.py`
|
||||
- Add async support (httpx instead of requests)
|
||||
- In-memory TTL cache (1 hour)
|
||||
- Return only community tracks with YouTube URLs
|
||||
|
||||
3. [ ] **Add catalog routes** — `backend/api/routes/catalog.py`
|
||||
- `GET /api/catalog/artists?q=&limit=10` — proxy to karaoke-decide
|
||||
- `GET /api/catalog/tracks?q=&artist=&limit=10` — proxy to karaoke-decide
|
||||
- `POST /api/catalog/community-check` — scrape karaokenerds.com
|
||||
- All endpoints require auth (same as job creation)
|
||||
- Pydantic request/response models
|
||||
|
||||
4. [ ] **Register routes in main.py** — add `catalog.router` to app
|
||||
|
||||
5. [ ] **Add config** — `backend/config.py`
|
||||
- `karaoke_decide_api_url` setting (default: `https://decide.nomadkaraoke.com`)
|
||||
|
||||
6. [ ] **Add `beautifulsoup4` + `httpx` dependencies** to `pyproject.toml` (if not already present)
|
||||
|
||||
### Phase 2: Frontend — Autocomplete component
|
||||
|
||||
7. [ ] **Create `AutocompleteInput` component** — `frontend/components/ui/autocomplete-input.tsx`
|
||||
- Wraps an Input field with a Popover + Command dropdown
|
||||
- Props: `value`, `onChange`, `onSelect`, `fetchSuggestions(query) => Promise<Suggestion[]>`, `placeholder`, `disabled`, `renderItem`
|
||||
- Debounced search (300ms) triggered on input change
|
||||
- Shows loading spinner while fetching
|
||||
- Shows "No results" when search returns empty
|
||||
- Keyboard navigation (up/down arrows, enter to select, escape to close)
|
||||
- Clicking outside closes dropdown
|
||||
- When user selects a suggestion, calls `onSelect(suggestion)` and populates input
|
||||
- When user types without selecting, uses their raw text (autocomplete is optional)
|
||||
|
||||
8. [ ] **Add catalog API functions** — `frontend/lib/api.ts`
|
||||
- `searchArtists(query: string, limit?: number)` → `GET /api/catalog/artists`
|
||||
- `searchTracks(query: string, artist?: string, limit?: number)` → `GET /api/catalog/tracks`
|
||||
- `checkCommunityVersions(artist: string, title: string)` → `POST /api/catalog/community-check`
|
||||
|
||||
### Phase 3: Frontend — Integrate autocomplete into job creation
|
||||
|
||||
9. [ ] **Replace artist Input fields with AutocompleteInput** in `JobSubmission.tsx`
|
||||
- All 3 tabs: `searchArtist`, `uploadArtist`, `youtubeArtist`
|
||||
- Fetch suggestions from `api.searchArtists(query)`
|
||||
- On select: populate artist field with canonical name
|
||||
- Show artist name + optional disambiguation in dropdown items
|
||||
|
||||
10. [ ] **Replace title Input fields with AutocompleteInput** in `JobSubmission.tsx`
|
||||
- All 3 tabs: `searchTitle`, `uploadTitle`, `youtubeTitle`
|
||||
- Fetch suggestions from `api.searchTracks(query, selectedArtist)`
|
||||
- On select: populate title field with canonical name
|
||||
- Show track name + artist name + duration in dropdown items
|
||||
|
||||
### Phase 4: Frontend — Community version detection
|
||||
|
||||
11. [ ] **Create `CommunityVersionBanner` component** — `frontend/components/job/CommunityVersionBanner.tsx`
|
||||
- Shows when community karaoke version(s) found
|
||||
- Displays: "A karaoke version of this song already exists!"
|
||||
- Lists community tracks with brand name and YouTube link
|
||||
- "Watch on YouTube" button for top community version
|
||||
- Dismissible (X button or "Continue anyway" link)
|
||||
- Styled with green border (matching kjbox community track styling)
|
||||
|
||||
12. [ ] **Add community check logic to `JobSubmission.tsx`** (Search tab only)
|
||||
- Debounced check (500ms) when both artist AND title fields have values
|
||||
- Only triggers when values change (not on every keystroke)
|
||||
- Shows `CommunityVersionBanner` between the form fields and the submit button
|
||||
- Does NOT block form submission
|
||||
- Clears when artist or title changes
|
||||
|
||||
### Phase 5: Testing
|
||||
|
||||
13. [ ] **Backend unit tests** — `backend/tests/unit/test_catalog_routes.py`
|
||||
- Test proxy endpoints with mocked karaoke-decide responses
|
||||
- Test community check with mocked karaokenerds HTML
|
||||
- Test error handling (karaoke-decide down, karaokenerds down)
|
||||
- Test caching behavior
|
||||
|
||||
14. [ ] **Backend unit tests** — `backend/tests/unit/test_karaokenerds_service.py`
|
||||
- Port relevant tests from `kjbox/kj-controller/tests/unit/test_karaoke_nerds.py`
|
||||
- Test HTML parsing, community detection, YouTube URL cleanup
|
||||
|
||||
15. [ ] **Frontend component tests** — `frontend/components/ui/__tests__/autocomplete-input.test.tsx`
|
||||
- Test suggestion fetching and display
|
||||
- Test keyboard navigation
|
||||
- Test selection behavior
|
||||
- Test free-text input (no selection)
|
||||
|
||||
16. [ ] **E2E tests** — `frontend/e2e/job-creation-autocomplete.spec.ts`
|
||||
- Test autocomplete appears on typing
|
||||
- Test selecting from autocomplete populates field
|
||||
- Test community banner appears for known songs
|
||||
- Test form submission works with and without autocomplete selection
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `backend/services/catalog_proxy_service.py` | Create | HTTP client for karaoke-decide API with caching |
|
||||
| `backend/services/karaokenerds_service.py` | Create | Karaokenerds.com scraper (ported from kjbox) |
|
||||
| `backend/api/routes/catalog.py` | Create | Proxy endpoints for artist/track search + community check |
|
||||
| `backend/main.py` | Modify | Register catalog router |
|
||||
| `backend/config.py` | Modify | Add karaoke_decide_api_url setting |
|
||||
| `pyproject.toml` | Modify | Add beautifulsoup4 dependency (if missing) |
|
||||
| `frontend/components/ui/autocomplete-input.tsx` | Create | Reusable autocomplete input component |
|
||||
| `frontend/components/job/CommunityVersionBanner.tsx` | Create | Community version warning banner |
|
||||
| `frontend/components/job/JobSubmission.tsx` | Modify | Replace Input with AutocompleteInput, add community check |
|
||||
| `frontend/lib/api.ts` | Modify | Add catalog API functions |
|
||||
| `backend/tests/unit/test_catalog_routes.py` | Create | Backend route tests |
|
||||
| `backend/tests/unit/test_karaokenerds_service.py` | Create | Scraper unit tests |
|
||||
| `frontend/components/ui/__tests__/autocomplete-input.test.tsx` | Create | Component tests |
|
||||
| `frontend/e2e/job-creation-autocomplete.spec.ts` | Create | E2E tests |
|
||||
|
||||
## Decisions (Resolved)
|
||||
|
||||
- **Track search scope**: All Spotify tracks (256M) — better coverage for new/obscure songs
|
||||
- **Community check scope**: Search tab only (Upload/URL users already have their audio)
|
||||
- **Rate limiting**: 20 requests/min per user on catalog proxy endpoints (prevents abuse of karaokenerds scraping). All endpoints require auth.
|
||||
- **"Display As" fields**: No autocomplete — they're for deliberate overrides like soundtrack names
|
||||
- **CORS**: Proxy approach (no changes needed to karaoke-decide)
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
- Feature is entirely additive — no existing functionality is modified
|
||||
- Autocomplete fields fall back to plain text Input if API calls fail
|
||||
- Community check is non-blocking — if it fails, form works normally
|
||||
- All new endpoints are behind the existing auth middleware
|
||||
- Can disable by reverting the frontend changes (backend proxy is harmless if unused)
|
||||
@@ -1,42 +0,0 @@
|
||||
# karaoke-gen backend environment — copy to .env and adjust.
|
||||
#
|
||||
# NOTE: scripts/run-backend-local.sh --with-emulators sets all the required
|
||||
# variables itself; you only need a .env for optional API keys or overrides.
|
||||
# None of the keys below are required to boot the backend locally.
|
||||
|
||||
# --- Core (defaults shown are what run-backend-local.sh uses in emulator mode) ---
|
||||
# ENVIRONMENT=development
|
||||
# GOOGLE_CLOUD_PROJECT=test-project
|
||||
# GCS_BUCKET_NAME=test-bucket
|
||||
# FIRESTORE_COLLECTION=test-jobs
|
||||
# ADMIN_TOKENS=local-dev-token
|
||||
# PORT=8000
|
||||
# FRONTEND_URL=http://localhost:3000
|
||||
|
||||
# --- Emulators (set automatically by run-backend-local.sh --with-emulators) ---
|
||||
# FIRESTORE_EMULATOR_HOST=127.0.0.1:8080
|
||||
# STORAGE_EMULATOR_HOST=http://127.0.0.1:4443
|
||||
|
||||
# --- Audio separation (pick one; without either, jobs stall at separation) ---
|
||||
# Local CPU/GPU separation via the audio-separator package. Models are
|
||||
# downloaded into this directory on first use (~2GB). CPU works but is slow.
|
||||
# MODEL_DIR=./models
|
||||
# ...or point at a remote audio-separator API service:
|
||||
# AUDIO_SEPARATOR_API_URL=
|
||||
|
||||
# --- Lyrics transcription ---
|
||||
# The lyrics worker reads AUDIOSHAKE_API_TOKEN (not AUDIOSHAKE_API_KEY).
|
||||
# Without it, transcription falls back to local Whisper if installed
|
||||
# (poetry install --extras local-whisper).
|
||||
# AUDIOSHAKE_API_TOKEN=
|
||||
|
||||
# --- Lyrics sources (optional; improves reference lyrics) ---
|
||||
# GENIUS_API_KEY=
|
||||
# SPOTIFY_COOKIE_SP_DC=
|
||||
|
||||
# --- Email (optional; without it, emails are logged to the console) ---
|
||||
# POSTMARK_SERVER_TOKEN=
|
||||
|
||||
# --- Payments (optional; only needed to exercise payment endpoints) ---
|
||||
# STRIPE_SECRET_KEY=
|
||||
# STRIPE_WEBHOOK_SECRET=
|
||||
@@ -1,58 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Nomad Karaoke pre-commit hook (karaoke-gen variant)
|
||||
# 1. Gitleaks secret scan (fail fast)
|
||||
# 2. i18n auto-translation when en.json changes
|
||||
# Enable: git config core.hooksPath .githooks
|
||||
set -eu
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Gitleaks secret scan
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
if command -v gitleaks >/dev/null 2>&1; then
|
||||
if ! gitleaks protect --staged --redact --no-banner; then
|
||||
printf '\n\033[31m✗ gitleaks detected a potential secret in staged changes.\033[0m\n' >&2
|
||||
printf ' • If real: remove it, rotate the credential, then re-commit.\n' >&2
|
||||
printf ' • If false positive: add an allowlist entry to .gitleaks.toml.\n' >&2
|
||||
printf ' • Emergency override (not recommended): git commit --no-verify\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
printf '\033[33mwarn:\033[0m gitleaks not installed — skipping secret scan\n' >&2
|
||||
printf ' install: brew install gitleaks\n' >&2
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. i18n auto-translation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
STAGED_EN=$(git diff --cached --name-only --diff-filter=ACM | grep -E '(messages|translations)/en\.json$' || true)
|
||||
|
||||
if [ -z "$STAGED_EN" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "i18n: English translations changed — running translation pipeline..."
|
||||
|
||||
for en_file in $STAGED_EN; do
|
||||
dir=$(dirname "$en_file")
|
||||
|
||||
if [[ "$dir" == frontend/messages ]]; then
|
||||
script="frontend/scripts/translate.py"
|
||||
elif [[ "$dir" == backend/translations ]]; then
|
||||
script="frontend/scripts/translate.py"
|
||||
else
|
||||
echo " Warning: Unknown messages dir $dir, skipping"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo " Translating $dir..."
|
||||
if python "$script" --messages-dir "$dir" --target all --skip-review 2>&1 | tail -5; then
|
||||
git add "$dir"/*.json
|
||||
echo " Done: $dir"
|
||||
else
|
||||
echo " Warning: Translation failed for $dir (GCP auth may be missing). CI will catch this."
|
||||
fi
|
||||
done
|
||||
|
||||
echo "i18n: Translation complete."
|
||||
@@ -1,327 +0,0 @@
|
||||
name: Build GHA Runner Images
|
||||
|
||||
# Produces the custom GCE images used by the ephemeral GitHub Actions runner
|
||||
# dispatcher: gha-runner-general, gha-runner-build, gha-runner-gpu,
|
||||
# gha-runner-gpu-windows.
|
||||
#
|
||||
# Flow per variant:
|
||||
# 1. Create a temporary build VM from the variant's base image with the
|
||||
# variant's provisioning script as its startup script:
|
||||
# - Linux variants: debian-12 + runner-image-provision.sh (startup-script)
|
||||
# - gpu-windows: windows-2022 + runner-image-provision.ps1
|
||||
# (windows-startup-script-ps1)
|
||||
# 2. Poll the serial console for the READY/FAILED marker (survives reboots).
|
||||
# 3. Stop the VM, snapshot its boot disk to a new image in the `gha-runner-<variant>`
|
||||
# image family with name gha-runner-<variant>-<timestamp>.
|
||||
# 4. Delete the build VM (boot disk auto-deletes).
|
||||
# 5. Delete older images in the family, keeping only the newest (GPU/windows images
|
||||
# are 15-21 GiB each; roll back by re-running this workflow at an older ref).
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
variants:
|
||||
description: 'Comma-separated variants to build (general,build,gpu,gpu-windows) or "all" (= all except build)'
|
||||
required: false
|
||||
default: 'all'
|
||||
skip_prune:
|
||||
description: 'Skip pruning (deletion) of old images (debugging)'
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
schedule:
|
||||
# Monthly on the 1st at 02:00 UTC
|
||||
- cron: '0 2 1 * *'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
|
||||
env:
|
||||
GCP_PROJECT: nomadkaraoke
|
||||
BUILD_REGION: us-central1
|
||||
BUILD_ZONE: us-central1-a
|
||||
BASE_IMAGE: projects/debian-cloud/global/images/family/debian-12
|
||||
KEEP_IMAGES_PER_FAMILY: 1
|
||||
|
||||
jobs:
|
||||
resolve-matrix:
|
||||
name: Resolve variant matrix
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
variants: ${{ steps.resolve.outputs.variants }}
|
||||
steps:
|
||||
- id: resolve
|
||||
run: |
|
||||
requested='${{ github.event.inputs.variants || 'all' }}'
|
||||
if [[ "$requested" == "all" || -z "$requested" ]]; then
|
||||
# "build" (docker-build label) is excluded from "all": no job uses it
|
||||
# since deploy-backend moved to ubuntu-latest + Cloud Build
|
||||
# (2026-10-01). Still buildable via an explicit variants=build.
|
||||
echo 'variants=["general","gpu","gpu-windows"]' >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
json="$(echo "$requested" | tr ',' '\n' | jq -R . | jq -sc .)"
|
||||
echo "variants=$json" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
build:
|
||||
name: Build ${{ matrix.variant }} image
|
||||
needs: resolve-matrix
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 90
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
variant: ${{ fromJSON(needs.resolve-matrix.outputs.variants) }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Authenticate to GCP
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
project_id: ${{ env.GCP_PROJECT }}
|
||||
|
||||
- name: Resolve build parameters
|
||||
id: params
|
||||
run: |
|
||||
variant='${{ matrix.variant }}'
|
||||
ts="$(date -u +%Y%m%d-%H%M%S)"
|
||||
name="gha-runner-${variant}-${ts}"
|
||||
family="gha-runner-${variant}"
|
||||
vm="gha-image-build-${variant}-${ts}"
|
||||
|
||||
# Per-variant base image + startup-script metadata key: Windows
|
||||
# VMs only execute scripts under windows-startup-script-ps1.
|
||||
image_flag="--image=${{ env.BASE_IMAGE }}"
|
||||
metadata_file="startup-script=infrastructure/scripts/runner-image-provision.sh"
|
||||
|
||||
case "$variant" in
|
||||
general|build)
|
||||
machine="n2-standard-8"
|
||||
disk_size="50"
|
||||
accel=""
|
||||
;;
|
||||
gpu)
|
||||
# Build with a T4 attached so DKMS module is built and validated
|
||||
# against the kernel that ships in the image.
|
||||
machine="n1-standard-4"
|
||||
disk_size="200"
|
||||
accel="--maintenance-policy=TERMINATE --accelerator=type=nvidia-tesla-t4,count=1"
|
||||
;;
|
||||
gpu-windows)
|
||||
# T4 attached so the GRID driver install can validate WDDM mode
|
||||
# (required for DirectML). Keep disk in sync with the
|
||||
# gpu-windows FamilySpec in runner_manager/ephemeral.py.
|
||||
machine="n1-standard-4"
|
||||
disk_size="100"
|
||||
accel="--maintenance-policy=TERMINATE --accelerator=type=nvidia-tesla-t4,count=1"
|
||||
image_flag="--image-family=windows-2022 --image-project=windows-cloud"
|
||||
metadata_file="windows-startup-script-ps1=infrastructure/scripts/runner-image-provision.ps1"
|
||||
;;
|
||||
*)
|
||||
echo "Unknown variant: $variant" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
{
|
||||
echo "name=$name"
|
||||
echo "family=$family"
|
||||
echo "vm=$vm"
|
||||
echo "machine=$machine"
|
||||
echo "disk_size=$disk_size"
|
||||
echo "accel=$accel"
|
||||
echo "image_flag=$image_flag"
|
||||
echo "metadata_file=$metadata_file"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Create build VM
|
||||
run: |
|
||||
set -euo pipefail
|
||||
gcloud compute instances create '${{ steps.params.outputs.vm }}' \
|
||||
--project='${{ env.GCP_PROJECT }}' \
|
||||
--zone='${{ env.BUILD_ZONE }}' \
|
||||
--machine-type='${{ steps.params.outputs.machine }}' \
|
||||
${{ steps.params.outputs.image_flag }} \
|
||||
--boot-disk-size='${{ steps.params.outputs.disk_size }}GB' \
|
||||
--boot-disk-type=pd-balanced \
|
||||
--boot-disk-auto-delete \
|
||||
--service-account="${{ secrets.GCP_SERVICE_ACCOUNT }}" \
|
||||
--scopes=https://www.googleapis.com/auth/cloud-platform \
|
||||
--metadata=enable-oslogin=TRUE,image-variant='${{ matrix.variant }}' \
|
||||
--metadata-from-file='${{ steps.params.outputs.metadata_file }}' \
|
||||
--labels=purpose=gha-runner-image-builder,variant='${{ matrix.variant }}' \
|
||||
${{ steps.params.outputs.accel }}
|
||||
|
||||
- name: Wait for provisioning to complete
|
||||
env:
|
||||
VM: ${{ steps.params.outputs.vm }}
|
||||
ZONE: ${{ env.BUILD_ZONE }}
|
||||
PROJECT: ${{ env.GCP_PROJECT }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
# Read the serial console — NOT SSH — to detect completion.
|
||||
#
|
||||
# The provisioner script tees its full output to /dev/ttyS0 (the GCE
|
||||
# serial console). `gcloud compute instances get-serial-port-output`
|
||||
# returns the full console buffer with zero SSH/IAP dependency. This
|
||||
# makes the wait robust against the IAP tunnel issues that previously
|
||||
# caused both false-positive readiness and false-negative timeouts.
|
||||
#
|
||||
# The script writes one of two distinctive marker lines:
|
||||
# "### runner-image: READY: ..." — successful completion
|
||||
# "### runner-image-provision FAILED ..." — fatal error (with line+rc)
|
||||
#
|
||||
# We also stream live progress to the GHA log so the operator can
|
||||
# see what's happening without leaving the run page.
|
||||
deadline=$(( $(date +%s) + 75 * 60 ))
|
||||
last_size=0
|
||||
attempt=0
|
||||
while (( $(date +%s) < deadline )); do
|
||||
attempt=$((attempt+1))
|
||||
console=$(gcloud compute instances get-serial-port-output "$VM" \
|
||||
--zone="$ZONE" --project="$PROJECT" --port=1 2>/dev/null || true)
|
||||
cur_size=${#console}
|
||||
# Stream new provisioner lines (filter syslog noise).
|
||||
if (( cur_size > last_size )); then
|
||||
echo "$console" | tail -c +$((last_size+1)) \
|
||||
| grep -E "runner-image|### |Attempt " || true
|
||||
last_size=$cur_size
|
||||
fi
|
||||
# Success marker
|
||||
if echo "$console" | grep -qE 'runner-image: READY:'; then
|
||||
echo
|
||||
echo "### Image ready (attempt $attempt)"
|
||||
echo "$console" | grep -E 'runner-image: READY:' | tail -1
|
||||
exit 0
|
||||
fi
|
||||
# Failure marker
|
||||
if echo "$console" | grep -qE 'runner-image-provision FAILED'; then
|
||||
echo
|
||||
echo "### Provisioner reported FAILURE — failure block + 40 lines of preceding context:" >&2
|
||||
# -B 40: include 40 lines BEFORE the marker (where the actual
|
||||
# error output of the failing command lives — python tracebacks,
|
||||
# apt errors, curl messages, etc).
|
||||
# -A 5: include the variant/line/cmd metadata lines after the marker.
|
||||
echo "$console" | grep -B 40 -A 5 'runner-image-provision FAILED' \
|
||||
| tail -80 >&2
|
||||
exit 1
|
||||
fi
|
||||
sleep 30
|
||||
done
|
||||
echo
|
||||
echo "### Timed out waiting for READY marker after 75 minutes" >&2
|
||||
echo "### Last serial console output (tail):" >&2
|
||||
gcloud compute instances get-serial-port-output "$VM" \
|
||||
--zone="$ZONE" --project="$PROJECT" --port=1 2>/dev/null \
|
||||
| grep -E "runner-image|### " | tail -50 >&2 || true
|
||||
exit 1
|
||||
|
||||
- name: Stop build VM
|
||||
env:
|
||||
VM: ${{ steps.params.outputs.vm }}
|
||||
ZONE: ${{ env.BUILD_ZONE }}
|
||||
PROJECT: ${{ env.GCP_PROJECT }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
gcloud compute instances stop "$VM" --zone="$ZONE" --project="$PROJECT" --quiet
|
||||
# Belt-and-suspenders wait for TERMINATED.
|
||||
for _ in $(seq 1 60); do
|
||||
status=$(gcloud compute instances describe "$VM" --zone="$ZONE" --project="$PROJECT" --format='value(status)')
|
||||
[[ "$status" == "TERMINATED" ]] && break
|
||||
sleep 5
|
||||
done
|
||||
|
||||
- name: Create image
|
||||
env:
|
||||
VM: ${{ steps.params.outputs.vm }}
|
||||
ZONE: ${{ env.BUILD_ZONE }}
|
||||
PROJECT: ${{ env.GCP_PROJECT }}
|
||||
IMAGE: ${{ steps.params.outputs.name }}
|
||||
FAMILY: ${{ steps.params.outputs.family }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# The "Create build VM" step doesn't pass --device-name or
|
||||
# --boot-disk-name, so GCE auto-names the boot disk after the
|
||||
# instance. We rely on that here instead of extracting the name
|
||||
# via a format expression — the previous
|
||||
# `disks[0].source.scope(disks).segment(1)` selector silently
|
||||
# returned empty and gcloud rejected the call with
|
||||
# "You cannot specify [--source-disk-zone] unless you are
|
||||
# specifying [--source-disk]".
|
||||
DISK="$VM"
|
||||
echo "Creating image $IMAGE in family $FAMILY from disk $DISK in $ZONE..."
|
||||
gcloud compute images create "$IMAGE" \
|
||||
--project="$PROJECT" \
|
||||
--source-disk="$DISK" \
|
||||
--source-disk-zone="$ZONE" \
|
||||
--family="$FAMILY" \
|
||||
--description="GHA ephemeral runner image (${{ matrix.variant }}) built ${{ github.run_id }}" \
|
||||
--labels=purpose=gha-ephemeral-runner,variant=${{ matrix.variant }},built-by=gha
|
||||
echo "Created $IMAGE in family $FAMILY"
|
||||
|
||||
- name: Delete build VM
|
||||
if: always()
|
||||
env:
|
||||
VM: ${{ steps.params.outputs.vm }}
|
||||
ZONE: ${{ env.BUILD_ZONE }}
|
||||
PROJECT: ${{ env.GCP_PROJECT }}
|
||||
run: |
|
||||
gcloud compute instances delete "$VM" --zone="$ZONE" --project="$PROJECT" --quiet || true
|
||||
|
||||
- name: Prune old images (keep newest ${{ env.KEEP_IMAGES_PER_FAMILY }})
|
||||
if: ${{ github.event.inputs.skip_prune != 'true' }}
|
||||
env:
|
||||
PROJECT: ${{ env.GCP_PROJECT }}
|
||||
FAMILY: ${{ steps.params.outputs.family }}
|
||||
KEEP: ${{ env.KEEP_IMAGES_PER_FAMILY }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# DELETE (not deprecate — deprecated images still bill for storage)
|
||||
# every image in this family beyond the newest $KEEP. The dispatcher
|
||||
# boots from the family URL, which always resolves to the newest image,
|
||||
# so older ones are only rollback insurance (~$0.05/GB-mo; gpu images
|
||||
# are ~21 GB each). Previously this step used an invalid
|
||||
# `-deprecated.state:*` filter: gcloud errored to stderr, stdout was
|
||||
# empty, and it silently reported "Nothing to deprecate" every month
|
||||
# while images piled up (31 runner images, ~$18/mo, by 2026-09).
|
||||
# Match the family EXACTLY client-side: a gcloud `family=gha-runner-gpu`
|
||||
# filter is a word-boundary regex that ALSO matches gha-runner-gpu-windows.
|
||||
# Rank by ACTIVE (non-deprecated) images: the family URL skips deprecated
|
||||
# images, so the newest $KEEP active ones are the real rollback path.
|
||||
# Deprecated images never count toward $KEEP.
|
||||
rows=$(gcloud compute images list \
|
||||
--project="$PROJECT" \
|
||||
--no-standard-images \
|
||||
--show-deprecated \
|
||||
--sort-by='~creationTimestamp' \
|
||||
--format='value(name,family,deprecated.state)' | awk -v f="$FAMILY" '$2==f')
|
||||
active=$(echo "$rows" | awk 'NF==2 {print $1}')
|
||||
if [[ -z "$active" ]]; then
|
||||
echo "::error::No active images found in family $FAMILY (list failed?) — refusing to prune."
|
||||
exit 1
|
||||
fi
|
||||
keep=$(echo "$active" | head -n "$KEEP")
|
||||
to_delete=$(echo "$rows" | awk '{print $1}' | grep -vxF "$keep" || true)
|
||||
if [[ -z "$to_delete" ]]; then
|
||||
echo "Nothing to prune (<= $KEEP images in $FAMILY)."
|
||||
exit 0
|
||||
fi
|
||||
# Never delete an image some existing disk was created from (e.g. a
|
||||
# long-lived VM still on an older image).
|
||||
in_use=$(gcloud compute disks list --project="$PROJECT" \
|
||||
--format='value(sourceImage.basename())' | sort -u)
|
||||
for img in $to_delete; do
|
||||
if grep -qx "$img" <<<"$in_use"; then
|
||||
echo "Keeping $img (source image of an existing disk)"
|
||||
continue
|
||||
fi
|
||||
echo "Deleting $img"
|
||||
gcloud compute images delete "$img" --project="$PROJECT" --quiet
|
||||
done
|
||||
+469
-1488
File diff suppressed because it is too large.
Load diff
@@ -1,180 +0,0 @@
|
||||
name: DR backup freshness monitor
|
||||
|
||||
# Off-platform monitor: runs in GitHub (not GCP) so it survives a full GCP
|
||||
# compromise. If the nightly backup pipeline dies silently, this catches it.
|
||||
#
|
||||
# Required secrets:
|
||||
# AWS_BACKUP_READONLY_ACCESS_KEY_ID
|
||||
# AWS_BACKUP_READONLY_SECRET_ACCESS_KEY
|
||||
# DR_MONITOR_DISCORD_WEBHOOK
|
||||
#
|
||||
# Setup: see docs/DISASTER-RECOVERY.md § "Backup freshness monitor".
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Daily at 14:00 UTC (~9h after the 1am ET backup window).
|
||||
- cron: "0 14 * * *"
|
||||
workflow_dispatch: {}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
check-freshness:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_BACKUP_READONLY_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_BACKUP_READONLY_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: us-east-1
|
||||
BUCKET: nomadkaraoke-backup
|
||||
DISCORD_WEBHOOK: ${{ secrets.DR_MONITOR_DISCORD_WEBHOOK }}
|
||||
|
||||
steps:
|
||||
- name: Verify required secrets are set
|
||||
run: |
|
||||
missing=0
|
||||
for var in AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY DISCORD_WEBHOOK; do
|
||||
if [ -z "${!var}" ]; then
|
||||
echo "::error::Required secret $var is empty"
|
||||
missing=1
|
||||
fi
|
||||
done
|
||||
[ "$missing" = "0" ] || exit 1
|
||||
|
||||
- name: Install boto3
|
||||
run: pip install --quiet boto3
|
||||
|
||||
- name: Check freshness of each tracked prefix
|
||||
id: check
|
||||
shell: bash
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import os, sys, datetime, boto3
|
||||
|
||||
BUCKET = os.environ["BUCKET"]
|
||||
# (label, prefix, max_age_hours, require_suffix)
|
||||
# require_suffix: only objects with this key suffix count toward
|
||||
# freshness (None = any object).
|
||||
# NOTE: Firestore and BigQuery export to GCS staging nightly but are
|
||||
# only uploaded to S3 *weekly* (Sundays) to cut cross-cloud egress —
|
||||
# see backup_to_aws/main.py (firestore_to_s3_today). This monitor only
|
||||
# sees the S3 (off-site) copy, so their limits must allow a full week
|
||||
# plus slack (192h ≈ 8 days), NOT the 36h nightly cadence. Secrets DO
|
||||
# upload to S3 nightly, so they stay at 36h. GCS job files are NOT
|
||||
# tracked: their off-site copy was stopped 2026-10-01 (egress cost
|
||||
# cut) — they rely on GCS versioning + soft-delete only.
|
||||
# Git repos are also weekly (Sundays) AND incremental since
|
||||
# 2026-09-26 — only repos pushed that week get a fresh bundle, so a
|
||||
# quiet week legitimately leaves every .bundle older than 8 days.
|
||||
# Freshness is judged on git-repos/manifest.json instead (rewritten
|
||||
# every weekly run), and the manifest's contents are checked below so
|
||||
# a run where every repo failed still reads as stale.
|
||||
TARGETS = [
|
||||
("Firestore", "firestore/", 192, None),
|
||||
("Encrypted secrets", "secrets/", 36, None),
|
||||
("BigQuery weekly", "bigquery/daily-refresh/", 192, None),
|
||||
("Git repos", "git-repos/manifest.json", 192, None),
|
||||
]
|
||||
|
||||
s3 = boto3.client("s3")
|
||||
now = datetime.datetime.now(datetime.timezone.utc)
|
||||
stale, ok = [], []
|
||||
|
||||
# Paginated scan — JMESPath --query on `aws s3api list-objects-v2`
|
||||
# operates per-page, which silently breaks "newest across all pages"
|
||||
# queries on prefixes with >1000 objects (e.g. firestore/ has ~22k).
|
||||
paginator = s3.get_paginator("list_objects_v2")
|
||||
for label, prefix, max_hours, require_suffix in TARGETS:
|
||||
latest = None
|
||||
for page in paginator.paginate(Bucket=BUCKET, Prefix=prefix):
|
||||
for obj in page.get("Contents", []):
|
||||
if require_suffix and not obj["Key"].endswith(require_suffix):
|
||||
continue
|
||||
if latest is None or obj["LastModified"] > latest:
|
||||
latest = obj["LastModified"]
|
||||
|
||||
if latest is None:
|
||||
stale.append(f"{label}: NO OBJECTS in s3://{BUCKET}/{prefix}")
|
||||
continue
|
||||
age = (now - latest).total_seconds()
|
||||
age_h = int(age // 3600)
|
||||
line = f"{label}: {age_h}h old (limit {max_hours}h) — last {latest.isoformat()}"
|
||||
(stale if age > max_hours * 3600 else ok).append(line)
|
||||
|
||||
# Git manifest contents: ANY bundle error alerts, even though the
|
||||
# manifest itself is fresh. The git step is weekly + incremental, so
|
||||
# a repo that failed keeps an ever-older S3 bundle until the next
|
||||
# Sunday — silence here would let it drift past the 7-day RPO.
|
||||
try:
|
||||
import json as _json
|
||||
m = _json.loads(s3.get_object(Bucket=BUCKET, Key="git-repos/manifest.json")["Body"].read())
|
||||
if int(m.get("errors", 0)) > 0:
|
||||
failed = [r.get("full_name") for r in m.get("repos", []) if r.get("status") == "error"]
|
||||
stale.append(f"Git repos: last run had {m.get('errors')} bundle errors: {', '.join(failed[:10])}")
|
||||
except Exception as e: # noqa: BLE001
|
||||
stale.append(f"Git repos: could not read manifest.json ({e})")
|
||||
|
||||
summary = ["## DR backup freshness", ""]
|
||||
if stale:
|
||||
summary.append("### Stale")
|
||||
summary.extend(f"- {s}" for s in stale)
|
||||
summary.append("")
|
||||
summary.append("### Fresh")
|
||||
summary.extend(f"- {o}" for o in ok)
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a") as f:
|
||||
f.write("\n".join(summary) + "\n")
|
||||
|
||||
with open(os.environ["GITHUB_OUTPUT"], "a") as f:
|
||||
f.write(f"ok_count={len(ok)}\n")
|
||||
f.write(f"stale_count={len(stale)}\n")
|
||||
if stale:
|
||||
f.write("STALE_LIST<<EOF\n")
|
||||
f.write("\n".join(stale) + "\n")
|
||||
f.write("EOF\n")
|
||||
|
||||
for o in ok:
|
||||
print("OK:", o)
|
||||
if stale:
|
||||
for s in stale:
|
||||
print("STALE:", s, file=sys.stderr)
|
||||
sys.exit(1)
|
||||
PY
|
||||
|
||||
- name: Alert Discord on stale backup
|
||||
if: failure() && steps.check.outputs.stale_count != ''
|
||||
env:
|
||||
STALE: ${{ steps.check.outputs.STALE_LIST }}
|
||||
run: |
|
||||
payload=$(jq -n \
|
||||
--arg title "🚨 DR backup is stale" \
|
||||
--arg desc "$STALE" \
|
||||
--arg url "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" \
|
||||
'{
|
||||
embeds: [{
|
||||
title: $title,
|
||||
description: $desc,
|
||||
color: 15158332,
|
||||
fields: [{name: "Workflow run", value: $url}]
|
||||
}]
|
||||
}')
|
||||
curl -fsS -X POST -H "Content-Type: application/json" \
|
||||
-d "$payload" "$DISCORD_WEBHOOK"
|
||||
|
||||
- name: Alert Discord on workflow failure (catch-all)
|
||||
if: failure() && steps.check.outputs.stale_count == ''
|
||||
run: |
|
||||
payload=$(jq -n \
|
||||
--arg title "⚠️ DR freshness monitor failed to run" \
|
||||
--arg desc "Could not list S3 backup bucket — credentials, network, or bucket policy may be broken." \
|
||||
--arg url "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" \
|
||||
'{
|
||||
embeds: [{
|
||||
title: $title,
|
||||
description: $desc,
|
||||
color: 15844367,
|
||||
fields: [{name: "Workflow run", value: $url}]
|
||||
}]
|
||||
}')
|
||||
curl -fsS -X POST -H "Content-Type: application/json" \
|
||||
-d "$payload" "$DISCORD_WEBHOOK"
|
||||
@@ -1,646 +0,0 @@
|
||||
name: E2E Daily Test (Payment + Happy Path)
|
||||
|
||||
# Full production E2E: buy credits via real Stripe, then use them for karaoke generation.
|
||||
# Replaces the previous e2e-happy-path.yml with an additional payment stage.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 6 * * *'
|
||||
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
debug_mode:
|
||||
description: 'Enable debug mode (slower, more screenshots)'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: boolean
|
||||
browser:
|
||||
description: 'Browser to test with'
|
||||
required: false
|
||||
default: 'chromium'
|
||||
type: choice
|
||||
options:
|
||||
- chromium
|
||||
- firefox
|
||||
- webkit
|
||||
skip_payment:
|
||||
description: 'Skip Stage 1 (payment) — use admin-granted credits for happy path only'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: boolean
|
||||
use_test_token:
|
||||
description: 'Skip signup entirely — use E2E_TEST_TOKEN (for quick iterations)'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: boolean
|
||||
self_heal_force_llm:
|
||||
description: 'Self-healing test: comma-separated Stripe step ids to force through the LLM fallback (e.g. selectCard). Learned recipes still open an auto-merge PR.'
|
||||
required: false
|
||||
default: ''
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
# =========================================================================
|
||||
# Stage 1: Purchase credits via real Stripe Checkout
|
||||
# =========================================================================
|
||||
e2e-credit-purchase:
|
||||
name: "Stage 1: Credit Purchase"
|
||||
runs-on: ubuntu-24.04 # Playwright 1.57 can't install chromium on ubuntu-latest (26.04)
|
||||
timeout-minutes: 15
|
||||
if: ${{ github.event.inputs.skip_payment != 'true' && github.event.inputs.use_test_token != 'true' }}
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # keyless GCP auth, only to read the Gemini key for self-healing
|
||||
|
||||
outputs:
|
||||
session_token: ${{ steps.extract_token.outputs.token }}
|
||||
result: ${{ steps.test_result.outputs.result }}
|
||||
failure_reason: ${{ steps.test_result.outputs.failure_reason }}
|
||||
healed: ${{ steps.self_heal.outputs.healed }}
|
||||
env:
|
||||
HAS_GEMINI_SECRET: ${{ secrets.GEMINI_API_KEY != '' }}
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps ${{ inputs.browser || 'chromium' }}
|
||||
|
||||
- name: Create test-results directory
|
||||
working-directory: frontend
|
||||
run: mkdir -p test-results
|
||||
|
||||
# ── Self-healing Stripe Checkout: LLM fallback key ───────────────────
|
||||
# The Stripe step runner (frontend/e2e/helpers/self-healing) falls back
|
||||
# to Gemini when every deterministic variant of a step fails. The key is
|
||||
# the AI Studio key (billed outside the nomadkaraoke project). Prefer a
|
||||
# GEMINI_API_KEY repo secret; otherwise read it from Secret Manager via
|
||||
# keyless auth. All of this is best-effort: without a key the test runs
|
||||
# deterministic-only, exactly as before.
|
||||
- name: Authenticate to Google Cloud (Gemini key)
|
||||
id: gcp_auth
|
||||
if: ${{ env.HAS_GEMINI_SECRET != 'true' }}
|
||||
continue-on-error: true
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
if: steps.gcp_auth.outcome == 'success'
|
||||
continue-on-error: true
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
|
||||
- name: Load Gemini key for self-healing fallback
|
||||
continue-on-error: true
|
||||
env:
|
||||
GEMINI_SECRET: ${{ secrets.GEMINI_API_KEY }}
|
||||
GCP_AUTHED: ${{ steps.gcp_auth.outcome == 'success' }}
|
||||
run: |
|
||||
KEY="$GEMINI_SECRET"
|
||||
if [ -z "$KEY" ] && [ "$GCP_AUTHED" = "true" ]; then
|
||||
KEY=$(gcloud secrets versions access latest --secret=gemini-api-key --project=nomadkaraoke 2>/dev/null || true)
|
||||
fi
|
||||
# Drop the deployer credentials before any test code runs.
|
||||
if [ -n "${GOOGLE_GHA_CREDS_PATH:-}" ]; then rm -f "$GOOGLE_GHA_CREDS_PATH"; fi
|
||||
if [ -n "${CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE:-}" ]; then rm -f "$CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE"; fi
|
||||
if [ -n "$KEY" ]; then
|
||||
echo "::add-mask::$KEY"
|
||||
echo "GEMINI_API_KEY=$KEY" >> "$GITHUB_ENV"
|
||||
echo "Self-healing LLM fallback: enabled"
|
||||
else
|
||||
echo "::warning::Self-healing LLM fallback disabled (no Gemini key) — Stripe steps run deterministic-only"
|
||||
fi
|
||||
|
||||
- name: Ensure testmail.app domain is allowlisted
|
||||
run: |
|
||||
curl -sf -X POST \
|
||||
"https://api.nomadkaraoke.com/api/admin/rate-limits/blocklists/allowlisted-domains" \
|
||||
-H "X-Admin-Token: ${{ secrets.E2E_ADMIN_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"domain": "inbox.testmail.app"}' || echo "Warning: allowlist may already exist"
|
||||
|
||||
- name: Run Credit Purchase Test
|
||||
id: run_test
|
||||
working-directory: frontend
|
||||
env:
|
||||
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
|
||||
TESTMAIL_NAMESPACE: ${{ secrets.TESTMAIL_NAMESPACE }}
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
E2E_STRIPE_CARD_NUMBER: ${{ secrets.E2E_STRIPE_CARD_NUMBER }}
|
||||
E2E_STRIPE_CARD_EXPIRY: ${{ secrets.E2E_STRIPE_CARD_EXPIRY }}
|
||||
E2E_STRIPE_CARD_CVC: ${{ secrets.E2E_STRIPE_CARD_CVC }}
|
||||
E2E_STRIPE_CARDHOLDER_NAME: ${{ secrets.E2E_STRIPE_CARDHOLDER_NAME }}
|
||||
E2E_STRIPE_ZIP: ${{ secrets.E2E_STRIPE_ZIP }}
|
||||
E2E_SELF_HEAL_FORCE_LLM: ${{ inputs.self_heal_force_llm || '' }}
|
||||
# No real card available → 'checkout-only' (no charge). Set the repo variable
|
||||
# E2E_REAL_PAYMENT=true to go back to buying a credit with E2E_STRIPE_CARD_*.
|
||||
E2E_PAYMENT_MODE: ${{ vars.E2E_REAL_PAYMENT == 'true' && 'real' || 'checkout-only' }}
|
||||
DEBUG_MODE: ${{ inputs.debug_mode || 'false' }}
|
||||
run: |
|
||||
echo "=== Stage 1: Credit Purchase via Real Stripe (mode: $E2E_PAYMENT_MODE) ==="
|
||||
echo "Time: $(date)"
|
||||
npx playwright test credit-purchase-real.spec.ts \
|
||||
--config=playwright.production.config.ts \
|
||||
--project=${{ inputs.browser || 'chromium' }} \
|
||||
--reporter=list \
|
||||
--timeout=600000 \
|
||||
2>&1 | tee test-output-payment.log
|
||||
# Capture Playwright's real exit code, not tee's (0) — the
|
||||
# authoritative pass/fail signal. See Stage 2 for rationale.
|
||||
echo "exit_code=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT"
|
||||
continue-on-error: true
|
||||
|
||||
- name: Check test result
|
||||
id: test_result
|
||||
run: |
|
||||
if [ "${{ steps.run_test.outputs.exit_code }}" = "0" ]; then
|
||||
echo "result=success" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "result=failure" >> "$GITHUB_OUTPUT"
|
||||
# A declined card is an issuer/card problem, not a broken checkout —
|
||||
# say so in the alerts (marker comes from e2e/helpers/stripe-checkout.ts).
|
||||
REASON=$(grep -m1 -o 'STRIPE_CARD_DECLINED:.*' frontend/test-output-payment.log 2>/dev/null | sed 's/ — .*//' | tr -d '"\\' || true)
|
||||
if [ -n "$REASON" ]; then
|
||||
echo "failure_reason=Test card declined, not a checkout bug (${REASON#STRIPE_CARD_DECLINED: }). Check the card/issuer for a fraud hold, then re-run." >> "$GITHUB_OUTPUT"
|
||||
echo "::error::$REASON"
|
||||
fi
|
||||
fi
|
||||
|
||||
- name: Detect self-healed Stripe steps
|
||||
id: self_heal
|
||||
if: always()
|
||||
working-directory: frontend
|
||||
run: |
|
||||
REPORT=test-results/self-heal/self-heal-report.json
|
||||
LEARNED=test-results/self-heal/learned-variants.json
|
||||
if [ -f "$REPORT" ]; then
|
||||
{
|
||||
echo "### Stripe Checkout self-healing"
|
||||
echo '```json'
|
||||
jq '{flowVerified, llmCalls, learnedWritten, added, outcomes: [.outcomes[] | {stepId, status, via, variant, model, reason}]}' "$REPORT"
|
||||
echo '```'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
fi
|
||||
# Only promote recipes from a run whose purchase was verified server-side.
|
||||
if [ "${{ steps.test_result.outputs.result }}" = "success" ] && [ -f "$LEARNED" ]; then
|
||||
echo "healed=true" >> "$GITHUB_OUTPUT"
|
||||
echo "::warning::Stripe Checkout changed — LLM fallback healed it; an auto-merge PR will record the new variant"
|
||||
else
|
||||
echo "healed=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Upload self-heal learned variants
|
||||
if: steps.self_heal.outputs.healed == 'true'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: self-heal-learned-${{ github.run_number }}
|
||||
path: frontend/test-results/self-heal/
|
||||
retention-days: 30
|
||||
|
||||
- name: Extract session token
|
||||
id: extract_token
|
||||
if: steps.test_result.outputs.result == 'success'
|
||||
working-directory: frontend
|
||||
run: |
|
||||
if [ -f test-results/e2e-session-token.txt ]; then
|
||||
TOKEN=$(cat test-results/e2e-session-token.txt)
|
||||
echo "::add-mask::$TOKEN"
|
||||
echo "token=$TOKEN" >> $GITHUB_OUTPUT
|
||||
echo "Session token extracted for Stage 2"
|
||||
else
|
||||
echo "No session token file found"
|
||||
echo "token=" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Scrub secrets from artifacts (public repo)
|
||||
id: scrub
|
||||
if: always()
|
||||
working-directory: frontend
|
||||
env:
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
|
||||
E2E_STRIPE_CARD_NUMBER: ${{ secrets.E2E_STRIPE_CARD_NUMBER }}
|
||||
E2E_STRIPE_CARD_EXPIRY: ${{ secrets.E2E_STRIPE_CARD_EXPIRY }}
|
||||
E2E_STRIPE_CARD_CVC: ${{ secrets.E2E_STRIPE_CARD_CVC }}
|
||||
E2E_SESSION_TOKEN: ${{ steps.extract_token.outputs.token }}
|
||||
run: node e2e/scripts/scrub-artifacts.mjs --drop-videos --blank-inputs test-results test-output-payment.log
|
||||
|
||||
- name: Upload artifacts
|
||||
if: always() && steps.scrub.outcome == 'success'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-payment-results-${{ github.run_number }}
|
||||
path: |
|
||||
frontend/test-results/
|
||||
frontend/test-output-payment.log
|
||||
retention-days: 30
|
||||
|
||||
- name: Discord notification (failure)
|
||||
if: steps.test_result.outputs.result == 'failure'
|
||||
env:
|
||||
# Via env + jq, never ${{ }} in the script: the reason contains Stripe's message text.
|
||||
REASON: ${{ steps.test_result.outputs.failure_reason || 'Payment flow is broken. Customers may not be receiving credits.' }}
|
||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
RUN_NUMBER: ${{ github.run_number }}
|
||||
WEBHOOK: ${{ secrets.DISCORD_ALERT_WEBHOOK_URL }}
|
||||
run: |
|
||||
jq -n --arg reason "$REASON" --arg run "$RUN_NUMBER" --arg url "$RUN_URL" \
|
||||
'{embeds: [{title: "🚨 E2E Daily Test FAILED — Stage 1: Credit Purchase", description: "\($reason)\n\n**Run:** #\($run)\n**Details:** \($url)", color: 15158332}]}' |
|
||||
curl -s -H "Content-Type: application/json" -d @- "$WEBHOOK"
|
||||
|
||||
- name: Fail if test failed
|
||||
if: steps.test_result.outputs.result == 'failure'
|
||||
run: exit 1
|
||||
|
||||
# =========================================================================
|
||||
# Self-heal: record LLM-discovered Stripe variants via an auto-merging PR
|
||||
# =========================================================================
|
||||
# One Stripe layout change → one LLM-healed run → one data-only PR (only
|
||||
# learned-variants.json) that auto-merges, so the next run replays the
|
||||
# recipe deterministically. No human in the loop.
|
||||
#
|
||||
# Merge gates for a bot PR (all verified live with #1117, 2026-10-03):
|
||||
# - its `pull_request` CI run is created "action_required" and, until run,
|
||||
# supersedes other CI Gate results → we approve it via the API;
|
||||
# - the `protect main` ruleset's require_extra_approval_for_unattributed_changes
|
||||
# (GitHub default true since ~Aug 2026) would demand a human review for
|
||||
# bot-authored changes → set to false on ruleset 11423976;
|
||||
# - review threads must be resolved → `@coderabbitai ignore` in the body.
|
||||
self-heal-pr:
|
||||
name: "Self-heal: auto-merge PR"
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
needs: e2e-credit-purchase
|
||||
# Only learn from main — branch runs (e.g. testing self_heal_force_llm on a
|
||||
# feature branch) must not open PRs against main.
|
||||
if: always() && needs.e2e-credit-purchase.outputs.healed == 'true' && github.ref == 'refs/heads/main'
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
actions: write
|
||||
outputs:
|
||||
pr_url: ${{ steps.pr.outputs.pr_url }}
|
||||
unchanged: ${{ steps.pr.outputs.unchanged }}
|
||||
steps:
|
||||
- name: Checkout main
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: main
|
||||
|
||||
- name: Download learned variants
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: self-heal-learned-${{ github.run_number }}
|
||||
path: /tmp/self-heal
|
||||
|
||||
- name: Open auto-merge PR
|
||||
id: pr
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TARGET=frontend/e2e/helpers/self-healing/learned-variants.json
|
||||
|
||||
# A bot-authored PR's `pull_request` CI run is created as
|
||||
# "action_required" (awaiting approval) and, until it runs, supersedes
|
||||
# any other CI Gate result for the ruleset. Approve it; succeed once a
|
||||
# pull_request run exists that is past approval.
|
||||
approve_ci() {
|
||||
local branch=$1 runs pending active
|
||||
for _ in $(seq 1 18); do
|
||||
runs=$(gh run list --workflow ci.yml --branch "$branch" --event pull_request --json databaseId,status,conclusion) || { sleep 10; continue; }
|
||||
pending=$(jq -r '.[] | select(.conclusion == "action_required" or .status == "waiting" or .status == "action_required") | .databaseId' <<<"$runs") || pending=""
|
||||
active=$(jq '[.[] | select(.conclusion != "action_required" and .status != "waiting" and .status != "action_required")] | length' <<<"$runs") || active=0
|
||||
for id in $pending; do
|
||||
if gh api -X POST "repos/${{ github.repository }}/actions/runs/$id/approve" >/dev/null; then
|
||||
echo "Approved CI run $id"
|
||||
else
|
||||
echo "::warning::Could not approve CI run $id with GITHUB_TOKEN"
|
||||
fi
|
||||
done
|
||||
if [ -z "$pending" ] && [ "$active" -gt 0 ]; then
|
||||
echo "CI is running for $branch"
|
||||
return 0
|
||||
fi
|
||||
sleep 10
|
||||
done
|
||||
return 1
|
||||
}
|
||||
REPORT=/tmp/self-heal/self-heal-report.json
|
||||
|
||||
OPEN=$(gh pr list --label e2e-self-heal --state open --json url --jq '.[0].url // ""')
|
||||
if [ -n "$OPEN" ]; then
|
||||
echo "A self-heal PR is already open ($OPEN) — not opening another; making sure its CI can run"
|
||||
echo "pr_url=$OPEN" >> "$GITHUB_OUTPUT"
|
||||
OPEN_BRANCH=$(gh pr view "$OPEN" --json headRefName --jq .headRefName || true)
|
||||
if [ -z "$OPEN_BRANCH" ] || ! approve_ci "$OPEN_BRANCH"; then
|
||||
echo "ci_needs_approval=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
cp /tmp/self-heal/learned-variants.json "$TARGET"
|
||||
if git diff --quiet -- "$TARGET"; then
|
||||
echo "learned-variants.json unchanged vs main — nothing to do"
|
||||
echo "unchanged=true" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
STEPS=$(jq -r '[.outcomes[] | select(.status == "healed") | .stepId] | unique | join(", ")' "$REPORT")
|
||||
BRANCH="self-heal/stripe-checkout-${{ github.run_id }}"
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git checkout -b "$BRANCH"
|
||||
git add "$TARGET"
|
||||
git commit -m "test(e2e): self-heal Stripe Checkout steps: $STEPS" -m "Learned by the LLM fallback in $RUN_URL (purchase verified server-side)."
|
||||
git push origin "$BRANCH"
|
||||
|
||||
gh label create e2e-self-heal --color FBCA04 --description "Auto-generated by the self-healing Stripe E2E" 2>/dev/null || true
|
||||
{
|
||||
echo "## 🩹 Stripe Checkout self-heal"
|
||||
echo
|
||||
echo "Stripe changed its hosted checkout page. In [run #${{ github.run_number }}]($RUN_URL) every deterministic variant of **$STEPS** failed; the Gemini fallback completed the step(s), and the purchase was then **verified server-side** (credit granted)."
|
||||
echo
|
||||
echo "This PR only adds the recipe(s) to \`learned-variants.json\`, so future runs replay them deterministically (no LLM). It auto-merges once CI passes."
|
||||
echo
|
||||
echo "### Learned recipes"
|
||||
echo '```json'
|
||||
jq '[.outcomes[] | select(.status == "healed") | {stepId, model, llmCalls, actions}]' "$REPORT"
|
||||
echo '```'
|
||||
echo
|
||||
echo "@coderabbitai ignore"
|
||||
echo
|
||||
echo "🤖 Generated by the E2E Daily self-healing workflow"
|
||||
} > /tmp/pr-body.md
|
||||
|
||||
PR_URL=$(gh pr create --base main --head "$BRANCH" --label e2e-self-heal \
|
||||
--title "test(e2e): self-heal Stripe Checkout ($STEPS)" --body-file /tmp/pr-body.md)
|
||||
echo "pr_url=$PR_URL" >> "$GITHUB_OUTPUT"
|
||||
gh pr merge "$PR_URL" --auto --squash
|
||||
|
||||
if approve_ci "$BRANCH"; then
|
||||
echo "Opened $PR_URL (auto-merge enabled, CI approved)"
|
||||
else
|
||||
# Fallback: dispatched CI (allowed from GITHUB_TOKEN) puts a CI Gate
|
||||
# check on the head SHA if no pull_request run ever appears.
|
||||
gh workflow run ci.yml --ref "$BRANCH"
|
||||
echo "ci_needs_approval=true" >> "$GITHUB_OUTPUT"
|
||||
echo "::warning::CI for $PR_URL may need a one-click approval (PR → Checks → Approve and run)"
|
||||
fi
|
||||
|
||||
- name: Discord notification (self-heal)
|
||||
# Recipe already on main (e.g. merged between checkout and now) → nothing to report.
|
||||
if: always() && steps.pr.outputs.unchanged != 'true'
|
||||
run: |
|
||||
PR="${{ steps.pr.outputs.pr_url }}"
|
||||
# #alerts = needs a human; #noise = informational (auto-merge handles it).
|
||||
# Noise falls back to the alert webhook if its secret is unset.
|
||||
ALERT_WEBHOOK="${{ secrets.DISCORD_ALERT_WEBHOOK_URL }}"
|
||||
NOISE_WEBHOOK="${{ secrets.DISCORD_NOISE_WEBHOOK_URL || secrets.DISCORD_ALERT_WEBHOOK_URL }}"
|
||||
if [ -n "$PR" ] && [ "${{ steps.pr.outputs.ci_needs_approval }}" = "true" ]; then
|
||||
TITLE="🩹 E2E self-healed Stripe — PR needs a CI approval click"
|
||||
DESC="LLM fallback healed a Stripe Checkout change. The auto-merge PR's CI run awaits approval (open the PR → Checks → Approve and run): $PR"
|
||||
COLOR=16776960
|
||||
WEBHOOK="$ALERT_WEBHOOK"
|
||||
elif [ -n "$PR" ]; then
|
||||
TITLE="🩹 E2E self-healed a Stripe Checkout change"
|
||||
DESC="Stripe changed its checkout page; the LLM fallback got through and the purchase was verified. Auto-merge PR: $PR"
|
||||
COLOR=16776960
|
||||
WEBHOOK="$NOISE_WEBHOOK"
|
||||
else
|
||||
TITLE="⚠️ E2E self-heal PR could not be opened"
|
||||
DESC="The LLM fallback healed a Stripe step but opening the PR failed — see the run."
|
||||
COLOR=15105570
|
||||
WEBHOOK="$ALERT_WEBHOOK"
|
||||
fi
|
||||
PAYLOAD="{\"embeds\": [{\"title\": \"$TITLE\", \"description\": \"$DESC\n\n**Run:** ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}\", \"color\": $COLOR}]}"
|
||||
# A failed #noise post retries on #alerts so it is never silently lost.
|
||||
curl -fsS -H "Content-Type: application/json" -d "$PAYLOAD" "$WEBHOOK" \
|
||||
|| { [ "$WEBHOOK" != "$ALERT_WEBHOOK" ] && curl -s -H "Content-Type: application/json" -d "$PAYLOAD" "$ALERT_WEBHOOK"; } \
|
||||
|| true
|
||||
|
||||
# =========================================================================
|
||||
# Stage 2: Happy Path (uses credit from Stage 1)
|
||||
# =========================================================================
|
||||
e2e-happy-path:
|
||||
name: "Stage 2: Happy Path"
|
||||
runs-on: ubuntu-24.04 # Playwright 1.57 can't install chromium on ubuntu-latest (26.04)
|
||||
timeout-minutes: 90
|
||||
needs: e2e-credit-purchase
|
||||
if: |
|
||||
always() && (
|
||||
needs.e2e-credit-purchase.result == 'success' ||
|
||||
github.event.inputs.skip_payment == 'true' ||
|
||||
github.event.inputs.use_test_token == 'true'
|
||||
)
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps ${{ inputs.browser || 'chromium' }}
|
||||
|
||||
- name: Create test-results directory
|
||||
working-directory: frontend
|
||||
run: mkdir -p test-results
|
||||
|
||||
- name: Ensure testmail.app domain is allowlisted
|
||||
run: |
|
||||
curl -sf -X POST \
|
||||
"https://api.nomadkaraoke.com/api/admin/rate-limits/blocklists/allowlisted-domains" \
|
||||
-H "X-Admin-Token: ${{ secrets.E2E_ADMIN_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"domain": "inbox.testmail.app"}' || echo "Warning: allowlist may already exist"
|
||||
|
||||
- name: Refresh E2E session token (when use_test_token enabled)
|
||||
id: refresh_token
|
||||
if: github.event.inputs.use_test_token == 'true'
|
||||
run: |
|
||||
RESPONSE=$(curl -sf -X POST \
|
||||
"https://api.nomadkaraoke.com/api/admin/users/e2e-test-runner@nomadkaraoke.com/impersonate" \
|
||||
-H "X-Admin-Token: ${{ secrets.E2E_ADMIN_TOKEN }}" \
|
||||
-H "Content-Type: application/json")
|
||||
TOKEN=$(echo "$RESPONSE" | python3 -c 'import json,sys; print(json.load(sys.stdin)["session_token"])')
|
||||
echo "::add-mask::$TOKEN"
|
||||
echo "token=$TOKEN" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Run Happy Path Test
|
||||
id: run_test
|
||||
working-directory: frontend
|
||||
env:
|
||||
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
|
||||
TESTMAIL_NAMESPACE: ${{ secrets.TESTMAIL_NAMESPACE }}
|
||||
E2E_TEST_TOKEN: ${{ steps.refresh_token.outputs.token || needs.e2e-credit-purchase.outputs.session_token || '' }}
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
E2E_BYPASS_KEY: ${{ secrets.E2E_BYPASS_KEY }}
|
||||
DEBUG_MODE: ${{ inputs.debug_mode || 'false' }}
|
||||
run: |
|
||||
echo "=== Stage 2: Happy Path ==="
|
||||
echo "Time: $(date)"
|
||||
TOKEN_SOURCE="none"
|
||||
if [ -n "${{ steps.refresh_token.outputs.token }}" ]; then
|
||||
TOKEN_SOURCE="use_test_token (impersonation)"
|
||||
elif [ -n "${{ needs.e2e-credit-purchase.outputs.session_token }}" ]; then
|
||||
TOKEN_SOURCE="Stage 1 (credit purchase)"
|
||||
fi
|
||||
echo "Token source: $TOKEN_SOURCE"
|
||||
|
||||
npx playwright test happy-path-real-user.spec.ts \
|
||||
--config=playwright.production.config.ts \
|
||||
--project=${{ inputs.browser || 'chromium' }} \
|
||||
--reporter=list \
|
||||
--timeout=3600000 \
|
||||
2>&1 | tee test-output-happy-path.log
|
||||
# Capture Playwright's real exit code, not tee's (0). This is the
|
||||
# authoritative pass/fail signal — grepping the log for "failed" is
|
||||
# fragile because benign retry warnings (e.g. "goto attempt 1/3
|
||||
# failed") contain that substring and produce false alarms.
|
||||
echo "exit_code=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT"
|
||||
continue-on-error: true
|
||||
|
||||
- name: Check test result
|
||||
id: test_result
|
||||
run: |
|
||||
if [ "${{ steps.run_test.outputs.exit_code }}" = "0" ]; then
|
||||
echo "result=success" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "result=failure" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Scrub secrets from artifacts (public repo)
|
||||
id: scrub
|
||||
if: always()
|
||||
working-directory: frontend
|
||||
env:
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
|
||||
E2E_BYPASS_KEY: ${{ secrets.E2E_BYPASS_KEY }}
|
||||
E2E_TEST_TOKEN: ${{ steps.refresh_token.outputs.token || needs.e2e-credit-purchase.outputs.session_token || '' }}
|
||||
run: node e2e/scripts/scrub-artifacts.mjs test-results playwright-report test-output-happy-path.log
|
||||
|
||||
- name: Upload test artifacts
|
||||
if: always() && steps.scrub.outcome == 'success'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-happy-path-results-${{ github.run_number }}
|
||||
path: |
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
frontend/test-output-happy-path.log
|
||||
retention-days: 30
|
||||
|
||||
- name: Upload video recordings
|
||||
if: always() && steps.scrub.outcome == 'success'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-happy-path-videos-${{ github.run_number }}
|
||||
path: frontend/test-results/**/*.webm
|
||||
retention-days: 30
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: Create summary
|
||||
if: always()
|
||||
run: |
|
||||
STAGE1="${{ needs.e2e-credit-purchase.result || 'skipped' }}"
|
||||
STAGE2="${{ steps.test_result.outputs.result || 'unknown' }}"
|
||||
echo "## E2E Daily Test Results" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Stage | Result |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "|-------|--------|" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Stage 1: Credit Purchase | ${STAGE1} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Stage 2: Happy Path | ${STAGE2} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Property | Value |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "|----------|-------|" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Run | #${{ github.run_number }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Trigger | ${{ github.event_name }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Browser | ${{ inputs.browser || 'chromium' }} |" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
- name: Discord notification (failure)
|
||||
if: steps.test_result.outputs.result == 'failure'
|
||||
run: |
|
||||
curl -s -H "Content-Type: application/json" \
|
||||
-d "{\"embeds\": [{\"title\": \"🚨 E2E Daily Test FAILED — Stage 2: Happy Path\", \"description\": \"Karaoke generation flow is broken.\n\n**Run:** #${{ github.run_number }}\n**Details:** ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}\", \"color\": 15158332}]}" \
|
||||
"${{ secrets.DISCORD_ALERT_WEBHOOK_URL }}"
|
||||
|
||||
- name: Fail if test failed
|
||||
if: steps.test_result.outputs.result == 'failure'
|
||||
run: exit 1
|
||||
|
||||
# =========================================================================
|
||||
# Final: Send notifications
|
||||
# =========================================================================
|
||||
notify:
|
||||
name: "Notify"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [e2e-credit-purchase, e2e-happy-path, self-heal-pr]
|
||||
if: always()
|
||||
|
||||
steps:
|
||||
- name: Discord notification (success)
|
||||
# Passes are not signal: they go to #noise (fallback: #alerts if the
|
||||
# noise secret is unset, so nothing is ever silently dropped).
|
||||
if: needs.e2e-happy-path.result == 'success'
|
||||
run: |
|
||||
STAGE1="${{ needs.e2e-credit-purchase.result || 'skipped' }}"
|
||||
ALERT_WEBHOOK="${{ secrets.DISCORD_ALERT_WEBHOOK_URL }}"
|
||||
NOISE_WEBHOOK="${{ secrets.DISCORD_NOISE_WEBHOOK_URL || secrets.DISCORD_ALERT_WEBHOOK_URL }}"
|
||||
PAYLOAD="{\"embeds\": [{\"title\": \"✅ E2E Daily Test PASSED\", \"description\": \"Payment: ${STAGE1} | Happy Path: success\n\n**Run:** #${{ github.run_number }}\", \"color\": 3066993}]}"
|
||||
curl -fsS -H "Content-Type: application/json" -d "$PAYLOAD" "$NOISE_WEBHOOK" \
|
||||
|| { [ "$NOISE_WEBHOOK" != "$ALERT_WEBHOOK" ] && curl -s -H "Content-Type: application/json" -d "$PAYLOAD" "$ALERT_WEBHOOK"; } \
|
||||
|| true
|
||||
|
||||
- name: Send email notification
|
||||
if: github.event_name == 'schedule'
|
||||
uses: dawidd6/action-send-mail@v3
|
||||
with:
|
||||
server_address: smtp.postmarkapp.com
|
||||
server_port: 587
|
||||
username: ${{ secrets.POSTMARK_SERVER_TOKEN }}
|
||||
password: ${{ secrets.POSTMARK_SERVER_TOKEN }}
|
||||
subject: "E2E Daily Test ${{ needs.e2e-happy-path.result == 'success' && 'PASSED' || 'FAILED' }} - Run #${{ github.run_number }}"
|
||||
to: andrew@nomadkaraoke.com
|
||||
from: "Karaoke Gen CI <noreply@nomadkaraoke.com>"
|
||||
body: |
|
||||
E2E Daily Test Results
|
||||
======================
|
||||
|
||||
Stage 1 (Credit Purchase): ${{ needs.e2e-credit-purchase.result || 'skipped' }}${{ vars.E2E_REAL_PAYMENT != 'true' && ' [checkout-only: no real payment]' || '' }}${{ needs.e2e-credit-purchase.outputs.failure_reason && format(' — {0}', needs.e2e-credit-purchase.outputs.failure_reason) || '' }}
|
||||
Stage 2 (Happy Path): ${{ needs.e2e-happy-path.result || 'skipped' }}
|
||||
Stripe self-heal: ${{ needs.e2e-credit-purchase.outputs.healed == 'true' && format('Stripe changed its checkout; LLM fallback healed it. Auto-merge PR: {0}', needs.self-heal-pr.outputs.pr_url || (needs.self-heal-pr.outputs.unchanged == 'true' && '(already on main)' || '(failed to open — see run)')) || 'not needed' }}
|
||||
|
||||
Run: #${{ github.run_number }}
|
||||
Trigger: ${{ github.event_name }}
|
||||
|
||||
View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
@@ -0,0 +1,174 @@
|
||||
name: E2E Happy Path Test
|
||||
|
||||
# Comprehensive end-to-end test that validates the full karaoke generation journey
|
||||
# using ONLY browser interactions (no API shortcuts)
|
||||
#
|
||||
# See docs/E2E-HAPPY-PATH-TEST-SPEC.md for full requirements
|
||||
|
||||
on:
|
||||
# Daily run at 6 AM UTC
|
||||
schedule:
|
||||
- cron: '0 6 * * *'
|
||||
|
||||
# Manual trigger with optional parameters
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
debug_mode:
|
||||
description: 'Enable debug mode (slower, more screenshots)'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: boolean
|
||||
browser:
|
||||
description: 'Browser to test with'
|
||||
required: false
|
||||
default: 'chromium'
|
||||
type: choice
|
||||
options:
|
||||
- chromium
|
||||
- firefox
|
||||
- webkit
|
||||
use_test_token:
|
||||
description: 'Skip signup and use E2E_TEST_TOKEN (for iterations)'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: boolean
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
e2e-happy-path:
|
||||
name: "Full Happy Path (Real User)"
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 90 # 1.5 hours max (test itself is 60 min)
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: |
|
||||
BROWSER=${{ inputs.browser || 'chromium' }}
|
||||
npx playwright install --with-deps $BROWSER
|
||||
|
||||
- name: Create test-results directory
|
||||
working-directory: frontend
|
||||
run: mkdir -p test-results
|
||||
|
||||
- name: Run E2E Happy Path Test
|
||||
working-directory: frontend
|
||||
env:
|
||||
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
|
||||
TESTMAIL_NAMESPACE: ${{ secrets.TESTMAIL_NAMESPACE }}
|
||||
E2E_TEST_TOKEN: ${{ inputs.use_test_token && secrets.E2E_TEST_TOKEN || '' }}
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
E2E_BYPASS_KEY: ${{ secrets.E2E_BYPASS_KEY }}
|
||||
DEBUG_MODE: ${{ inputs.debug_mode || 'false' }}
|
||||
run: |
|
||||
echo "Starting E2E Happy Path Test"
|
||||
echo "Browser: ${{ inputs.browser || 'chromium' }}"
|
||||
echo "Debug mode: ${{ inputs.debug_mode || 'false' }}"
|
||||
echo "Use test token: ${{ inputs.use_test_token || 'false' }}"
|
||||
echo "Time: $(date)"
|
||||
echo ""
|
||||
|
||||
# Run the real user happy path test
|
||||
npx playwright test happy-path-real-user.spec.ts \
|
||||
--config=playwright.production.config.ts \
|
||||
--project=${{ inputs.browser || 'chromium' }} \
|
||||
--reporter=list \
|
||||
--timeout=3600000 \
|
||||
2>&1 | tee test-output.log
|
||||
continue-on-error: true
|
||||
|
||||
- name: Check test result
|
||||
id: test_result
|
||||
working-directory: frontend
|
||||
run: |
|
||||
if grep -q "passed" test-output.log && ! grep -q "failed" test-output.log; then
|
||||
echo "result=success" >> $GITHUB_OUTPUT
|
||||
echo "Test PASSED"
|
||||
else
|
||||
echo "result=failure" >> $GITHUB_OUTPUT
|
||||
echo "Test FAILED - see artifacts for details"
|
||||
fi
|
||||
|
||||
- name: Upload test artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-happy-path-results-${{ github.run_number }}
|
||||
path: |
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
frontend/test-output.log
|
||||
retention-days: 30
|
||||
|
||||
- name: Upload video recordings
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-happy-path-videos-${{ github.run_number }}
|
||||
path: frontend/test-results/**/*.webm
|
||||
retention-days: 30
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: Create summary
|
||||
if: always()
|
||||
run: |
|
||||
echo "## E2E Happy Path Test Results" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Property | Value |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "|----------|-------|" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Run | #${{ github.run_number }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Trigger | ${{ github.event_name }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Browser | ${{ inputs.browser || 'chromium' }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Result | ${{ steps.test_result.outputs.result || 'unknown' }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
if [ -f frontend/test-output.log ]; then
|
||||
echo "### Test Output (last 100 lines)" >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
|
||||
tail -100 frontend/test-output.log >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
|
||||
- name: Send email notification
|
||||
if: always() && github.event_name == 'schedule'
|
||||
uses: dawidd6/action-send-mail@v3
|
||||
with:
|
||||
server_address: smtp.sendgrid.net
|
||||
server_port: 587
|
||||
username: apikey
|
||||
password: ${{ secrets.SENDGRID_API_KEY }}
|
||||
subject: "E2E Happy Path Test ${{ steps.test_result.outputs.result == 'success' && 'PASSED' || 'FAILED' }} - Run #${{ github.run_number }}"
|
||||
to: andrew@nomadkaraoke.com
|
||||
from: "Karaoke Gen CI <noreply@nomadkaraoke.com>"
|
||||
body: |
|
||||
E2E Happy Path Test Results
|
||||
===========================
|
||||
|
||||
Status: ${{ steps.test_result.outputs.result == 'success' && 'PASSED ✅' || 'FAILED ❌' }}
|
||||
Run: #${{ github.run_number }}
|
||||
Trigger: ${{ github.event_name }}
|
||||
Time: ${{ github.event.repository.updated_at }}
|
||||
|
||||
View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
|
||||
This is an automated notification from the daily E2E happy path test.
|
||||
|
||||
- name: Fail if test failed
|
||||
if: steps.test_result.outputs.result == 'failure'
|
||||
run: exit 1
|
||||
@@ -1,141 +0,0 @@
|
||||
name: E2E Daily Test (Tenant Portals)
|
||||
|
||||
# Full production E2E per white-label tenant: submit a job with the tenant's own
|
||||
# mixed + instrumental audio, approve the lyrics review, wait for the render, and assert
|
||||
# the tenant distribution (locked theme, no YouTube, no Google Drive, Dropbox folder +
|
||||
# downloadable outputs). Mirrors the consumer e2e-daily.yml. Each run cleans up its job.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Daily at 00:07 UTC = ~8pm US Eastern (8:07pm EDT / 7:07pm EST). Cron is fixed
|
||||
# UTC so the local time shifts 1h across DST. Off the :00 mark to avoid the
|
||||
# top-of-hour scheduler herd.
|
||||
- cron: '7 0 * * *'
|
||||
workflow_dispatch: {}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # for Workload Identity Federation
|
||||
|
||||
jobs:
|
||||
e2e-tenant:
|
||||
name: "Tenant E2E: ${{ matrix.tenant }}"
|
||||
runs-on: ubuntu-24.04 # Playwright 1.57 can't install chromium on ubuntu-latest (26.04)
|
||||
timeout-minutes: 60
|
||||
strategy:
|
||||
fail-fast: false
|
||||
# Run tenants sequentially: the GCE encoder is a single shared worker, and two
|
||||
# concurrent full renders overwhelm it ("lost contact with worker"). One at a time.
|
||||
max-parallel: 1
|
||||
matrix:
|
||||
tenant: [vocalstar, singa]
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Authenticate to GCP
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Fetch test audio (private GCS)
|
||||
run: bash scripts/e2e/fetch_test_audio.sh frontend/e2e/fixtures
|
||||
|
||||
- name: Ensure testmail.app domain is allowlisted (rate limit)
|
||||
run: |
|
||||
curl -sf -X POST \
|
||||
"https://api.nomadkaraoke.com/api/admin/rate-limits/blocklists/allowlisted-domains" \
|
||||
-H "X-Admin-Token: ${{ secrets.E2E_ADMIN_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"domain": "inbox.testmail.app"}' || echo "Warning: allowlist may already exist"
|
||||
|
||||
- name: Run tenant E2E (frontend journey)
|
||||
id: run_test
|
||||
working-directory: frontend
|
||||
env:
|
||||
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
|
||||
TESTMAIL_NAMESPACE: ${{ secrets.TESTMAIL_NAMESPACE }}
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
TENANT_ID: ${{ matrix.tenant }}
|
||||
TENANT_PORTAL_URL: https://${{ matrix.tenant }}.nomadkaraoke.com
|
||||
run: |
|
||||
echo "=== Tenant E2E (frontend): ${{ matrix.tenant }} ==="
|
||||
npx playwright test tenant-happy-path.spec.ts \
|
||||
--config=playwright.production.config.ts --project=chromium \
|
||||
--reporter=list --timeout=2400000 2>&1 | tee e2e-output.log
|
||||
echo "exit_code=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT"
|
||||
continue-on-error: true
|
||||
|
||||
- name: Scrub secrets from artifacts (public repo)
|
||||
id: scrub
|
||||
if: always()
|
||||
working-directory: frontend
|
||||
env:
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
TESTMAIL_API_KEY: ${{ secrets.TESTMAIL_API_KEY }}
|
||||
run: node e2e/scripts/scrub-artifacts.mjs test-results playwright-report e2e-output.log
|
||||
|
||||
- name: Upload logs
|
||||
if: always() && steps.scrub.outcome == 'success'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-tenant-${{ matrix.tenant }}-${{ github.run_number }}
|
||||
path: |
|
||||
frontend/e2e-output.log
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
retention-days: 14
|
||||
|
||||
- name: Discord notification (failure)
|
||||
if: ${{ steps.run_test.outputs.exit_code != '0' }}
|
||||
env:
|
||||
DISCORD_WEBHOOK: ${{ secrets.DISCORD_ALERT_WEBHOOK_URL }}
|
||||
run: |
|
||||
if [ -n "$DISCORD_WEBHOOK" ]; then
|
||||
curl -s -H "Content-Type: application/json" \
|
||||
-d "{\"embeds\": [{\"title\": \"❌ Tenant E2E FAILED — ${{ matrix.tenant }}\", \"description\": \"Run: #${{ github.run_number }}\", \"color\": 15158332}]}" \
|
||||
"$DISCORD_WEBHOOK" || true
|
||||
fi
|
||||
|
||||
- name: Fail the job if the test failed
|
||||
if: ${{ steps.run_test.outputs.exit_code != '0' }}
|
||||
run: exit 1
|
||||
|
||||
notify-email:
|
||||
name: "Email summary"
|
||||
runs-on: ubuntu-latest
|
||||
needs: e2e-tenant
|
||||
if: ${{ always() && github.event_name == 'schedule' }}
|
||||
steps:
|
||||
- name: Send email notification
|
||||
uses: dawidd6/action-send-mail@v3
|
||||
with:
|
||||
server_address: smtp.postmarkapp.com
|
||||
server_port: 587
|
||||
username: ${{ secrets.POSTMARK_SERVER_TOKEN }}
|
||||
password: ${{ secrets.POSTMARK_SERVER_TOKEN }}
|
||||
subject: "Tenant E2E Daily ${{ needs.e2e-tenant.result == 'success' && 'PASSED' || 'FAILED' }} - Run #${{ github.run_number }}"
|
||||
to: andrew@nomadkaraoke.com
|
||||
from: "Karaoke Gen CI <noreply@nomadkaraoke.com>"
|
||||
body: |
|
||||
Tenant E2E Daily Test (Vocal Star + Singa)
|
||||
Result: ${{ needs.e2e-tenant.result }}
|
||||
Run: #${{ github.run_number }}
|
||||
Details: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
@@ -1,88 +0,0 @@
|
||||
name: PyPI prune reminder
|
||||
|
||||
# PyPI caps total *project* size at 10 GB. karaoke-gen publishes a ~62 MiB wheel
|
||||
# whenever CLI code changes (scripts/pypi_publish_gate.py), so the project slowly
|
||||
# refills the cap and `poetry publish` then fails with "Project size too large".
|
||||
#
|
||||
# PyPI has NO deletion API (upload tokens are upload-only), so this can't
|
||||
# auto-delete. Instead it runs monthly, computes the retention plan
|
||||
# (scripts/prune_pypi_releases.py), and opens/updates a single tracking issue
|
||||
# containing a paste-into-browser-console snippet you run on your logged-in PyPI
|
||||
# releases page to actually prune. No secrets are stored.
|
||||
#
|
||||
# See docs/PYPI-STORAGE-PRUNE.md.
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 09:00 UTC on the 1st of each month.
|
||||
- cron: "0 9 1 * *"
|
||||
workflow_dispatch: {}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
|
||||
jobs:
|
||||
prune-reminder:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PROJECT: karaoke-gen
|
||||
KEEP_DAYS: "30"
|
||||
# Only nag once the project crosses this watermark — pruning holds it at
|
||||
# ~4-7 GB, and there are always old releases to consolidate, so without a
|
||||
# threshold this would open an issue every month for no real reason. 7 GB
|
||||
# leaves ~1 month of ~3 GB/month growth before the 10 GB cap.
|
||||
NAG_THRESHOLD_GB: "7"
|
||||
ISSUE_TITLE: "🧹 PyPI storage prune due: karaoke-gen"
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Compute prune plan
|
||||
id: plan
|
||||
run: |
|
||||
python scripts/prune_pypi_releases.py \
|
||||
--project "$PROJECT" --keep-days "$KEEP_DAYS" \
|
||||
--format markdown --output plan.md
|
||||
python scripts/prune_pypi_releases.py \
|
||||
--project "$PROJECT" --keep-days "$KEEP_DAYS" \
|
||||
--format json --output plan.json
|
||||
python - <<'PY' >> "$GITHUB_OUTPUT"
|
||||
import json, os
|
||||
d = json.load(open("plan.json"))
|
||||
threshold = float(os.environ["NAG_THRESHOLD_GB"])
|
||||
should_nag = d["delete_count"] > 0 and d["current_gb"] >= threshold
|
||||
print(f"delete_count={d['delete_count']}")
|
||||
print(f"reclaimable_gb={d['reclaimable_gb']:.2f}")
|
||||
print(f"current_gb={d['current_gb']:.2f}")
|
||||
print(f"should_nag={'true' if should_nag else 'false'}")
|
||||
PY
|
||||
|
||||
- name: Job summary
|
||||
run: cat plan.md >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Open or update tracking issue
|
||||
if: steps.plan.outputs.should_nag == 'true'
|
||||
run: |
|
||||
# Search by a stable ASCII substring (emoji in search queries is
|
||||
# flaky); the exact-title jq filter below is the real dedupe guard.
|
||||
existing=$(gh issue list --repo "$GITHUB_REPOSITORY" \
|
||||
--state open --search "PyPI storage prune due in:title" \
|
||||
--json number,title \
|
||||
--jq "map(select(.title == \"$ISSUE_TITLE\")) | .[0].number // empty")
|
||||
note="🔁 Refreshed $(date -u +%Y-%m-%d): ${{ steps.plan.outputs.delete_count }} releases prunable, ~${{ steps.plan.outputs.reclaimable_gb }} GB reclaimable (project at ${{ steps.plan.outputs.current_gb }} GB)."
|
||||
if [ -n "$existing" ]; then
|
||||
echo "Updating existing issue #$existing"
|
||||
gh issue edit "$existing" --repo "$GITHUB_REPOSITORY" --body-file plan.md
|
||||
gh issue comment "$existing" --repo "$GITHUB_REPOSITORY" --body "$note"
|
||||
else
|
||||
echo "Creating new tracking issue"
|
||||
gh issue create --repo "$GITHUB_REPOSITORY" \
|
||||
--title "$ISSUE_TITLE" --body-file plan.md
|
||||
fi
|
||||
@@ -1,10 +0,0 @@
|
||||
name: Security
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
gitleaks:
|
||||
uses: nomadkaraoke/.github/.github/workflows/gitleaks.yml@main
|
||||
@@ -13,12 +13,8 @@ tests/outputs
|
||||
/input
|
||||
/output/
|
||||
|
||||
# Local karaoke-gen CLI test output dirs (created during manual validation)
|
||||
/test-output-*/
|
||||
|
||||
# Cloud Infra
|
||||
.emulator-logs
|
||||
.local-stack
|
||||
|
||||
# Next.js / React Frontend (./frontend)
|
||||
node_modules/
|
||||
@@ -29,9 +25,6 @@ frontend/.swc/
|
||||
frontend/.turbo/
|
||||
frontend/build/
|
||||
frontend/dist/
|
||||
# Generated Next.js static export copied into the Python package at build time (#876).
|
||||
# Bundled into the wheel via pyproject `include`; must not be tracked in git.
|
||||
karaoke_gen/nextjs_frontend/out/
|
||||
|
||||
# Next.js specific
|
||||
frontend/*.tsbuildinfo
|
||||
@@ -239,4 +232,3 @@ flacfetch
|
||||
|
||||
# Test output directories
|
||||
Rick Astley*/
|
||||
.superpowers/
|
||||
@@ -1,63 +0,0 @@
|
||||
# Nomad Karaoke shared gitleaks config.
|
||||
# Extends the default ruleset with one custom rule for Discord webhooks
|
||||
# and an allowlist covering known false-positive paths.
|
||||
|
||||
title = "Nomad Karaoke"
|
||||
|
||||
[extend]
|
||||
useDefault = true
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Custom rules
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
[[rules]]
|
||||
id = "discord-webhook-url"
|
||||
description = "Discord webhook URL (token embedded in URL)"
|
||||
regex = '''https://(?:discord|discordapp)\.com/api/webhooks/\d{17,20}/[A-Za-z0-9_-]{60,}'''
|
||||
tags = ["discord", "webhook", "secret"]
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Global allowlist
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
[allowlist]
|
||||
description = "Paths and patterns that are known false positives"
|
||||
|
||||
paths = [
|
||||
'''(?i)\.(jpg|jpeg|png|gif|bmp|svg|webp|pdf|ico|mp3|mp4|wav|flac|m4a)$''',
|
||||
'''(^|/)(package-lock\.json|yarn\.lock|pnpm-lock\.yaml|poetry\.lock|Podfile\.lock|Cargo\.lock|go\.sum|composer\.lock|Gemfile\.lock|uv\.lock)$''',
|
||||
'''(^|/)node_modules/''',
|
||||
'''(^|/)vendor/''',
|
||||
'''(^|/)wp-content/''',
|
||||
'''(^|/)\.dart_tool/''',
|
||||
'''(^|/)\.venv/''',
|
||||
'''(^|/)\.next/''',
|
||||
'''(^|/)build/''',
|
||||
'''(^|/)dist/''',
|
||||
'''(^|/)\.flutter-plugins''',
|
||||
'''(^|/)test[-_]?fixtures?/''',
|
||||
'''(^|/)tests?/.*conftest\.py$''',
|
||||
'''(^|/)tests?/.*test_[^/]+\.py$''',
|
||||
'''(^|/)tests?/.*_test\.py$''',
|
||||
'''(^|/).*\.(test|spec)\.(js|mjs|cjs|ts|tsx|jsx)$''',
|
||||
'''(^|/).*\.test\.local\.(js|mjs|cjs|ts|tsx|jsx|py)$''',
|
||||
'''(^|/).*\.local\.(js|mjs|cjs|ts|tsx|jsx|py)$''',
|
||||
]
|
||||
|
||||
regexes = [
|
||||
# Explicit placeholder patterns
|
||||
'''YOUR_[A-Z_]{2,}''',
|
||||
'''EXAMPLE_[A-Z_]{2,}''',
|
||||
'''PLACEHOLDER_[A-Z_]{2,}''',
|
||||
'''CHANGE[_-]?ME''',
|
||||
'''<[A-Z_]{2,}>''',
|
||||
'''\$\{[A-Z_]{2,}\}''',
|
||||
|
||||
# Firebase / GCP client API keys — public by design (identify the project,
|
||||
# access is gated by Firebase Security Rules).
|
||||
'''AIzaSy[A-Za-z0-9_-]{33}''',
|
||||
|
||||
# Documented local-development placeholder token (CONTRIBUTING.md, dev scripts)
|
||||
'''local-dev-token''',
|
||||
]
|
||||
@@ -1 +0,0 @@
|
||||
[{"name":"generate-buildid","duration":272,"timestamp":244518298649,"id":4,"parentId":1,"tags":{},"startTime":1775262331361,"traceId":"b3ef0d179e8acedf"},{"name":"load-custom-routes","duration":314,"timestamp":244518299029,"id":5,"parentId":1,"tags":{},"startTime":1775262331361,"traceId":"b3ef0d179e8acedf"},{"name":"create-dist-dir","duration":333,"timestamp":244518299363,"id":6,"parentId":1,"tags":{},"startTime":1775262331362,"traceId":"b3ef0d179e8acedf"},{"name":"clean","duration":358,"timestamp":244518300558,"id":7,"parentId":1,"tags":{},"startTime":1775262331363,"traceId":"b3ef0d179e8acedf"},{"name":"next-build","duration":1670851,"timestamp":244516630180,"id":1,"tags":{"buildMode":"default","version":"16.2.2","bundler":"turbopack","failed":true},"startTime":1775262329693,"traceId":"b3ef0d179e8acedf"}]
|
||||
@@ -1 +0,0 @@
|
||||
[{"name":"next-build","duration":1670851,"timestamp":244516630180,"id":1,"tags":{"buildMode":"default","version":"16.2.2","bundler":"turbopack","failed":true},"startTime":1775262329693,"traceId":"b3ef0d179e8acedf"}]
|
||||
@@ -19,17 +19,14 @@
|
||||
| **Testing & code quality** | `docs/TESTING.md` |
|
||||
| API reference | `docs/API.md` |
|
||||
| Past learnings | `docs/LESSONS-LEARNED.md` |
|
||||
| **Operational runbooks** | `docs/TROUBLESHOOTING.md` |
|
||||
| GDrive validator & gaps | `docs/GDRIVE-VALIDATOR.md` |
|
||||
| **Product communication** | `docs/PRODUCT-COMMUNICATION-GUIDE.md` |
|
||||
| Brand style guide | `docs/BRAND-STYLE-GUIDE.md` |
|
||||
| **Making medleys** (multi-song karaoke) | `docs/MEDLEYS.md` + `scripts/medley/make_medley.py` |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Backend**: FastAPI on Cloud Run, Firestore, GCS, Secret Manager
|
||||
- **Frontend**: Next.js on Cloudflare Pages
|
||||
- **Processing**: Cloud Run GPU (L4 audio separation), AudioShake/Whisper (transcription)
|
||||
- **Processing**: Modal (GPU audio separation), AudioShake/Whisper (transcription)
|
||||
- **Infra**: Pulumi IaC, GitHub Actions CI/CD
|
||||
|
||||
## Essential Rules
|
||||
@@ -68,9 +65,8 @@ This single command:
|
||||
When asked to "test it yourself in prod" or verify a fix in production:
|
||||
|
||||
```bash
|
||||
# Get admin token. claude-readonly@ can read all secrets (Pulumi:
|
||||
# infrastructure/modules/iam/claude_readonly_sa.py), so no gcloud login is needed.
|
||||
export KARAOKE_ADMIN_TOKEN=$(gcloud secrets versions access latest --secret=admin-tokens --project=nomadkaraoke --account=claude-readonly@nomadkaraoke.iam.gserviceaccount.com | cut -d',' -f1)
|
||||
# Get admin token
|
||||
export KARAOKE_ADMIN_TOKEN=$(gcloud secrets versions access latest --secret=admin-tokens --project=nomadkaraoke | cut -d',' -f1)
|
||||
|
||||
# Option 1: Use the debug template directly
|
||||
cd frontend && node e2e/helpers/debug-prod-template.mjs
|
||||
@@ -93,32 +89,11 @@ See `docs/TESTING.md` § "Ad-Hoc Production Debugging" for full details.
|
||||
- Skip for docs-only changes
|
||||
|
||||
### Infrastructure
|
||||
- **All GCP changes via Pulumi PRs** - but CI does **not** run Pulumi (see the note in `.github/workflows/ci.yml`): a merged `infrastructure/` change is not live until someone runs `pulumi up` locally. Claude sessions have read-only ADC, so they need Andrew's `/gcp-admin` grant to apply. Ask for it early, and vet with a targeted `pulumi preview --diff` first
|
||||
- **`infrastructure/` is the shared platform IaC home**: the GCP project (`infrastructure/`) and the Nomad Karaoke Cloudflare account (`infrastructure/cloudflare/`, own Pulumi project + stack, see its README). Never hand-edit Cloudflare config that stack manages; app repos keep only app-specific stacks
|
||||
- **Run `pulumi up` locally BEFORE merging infra PRs** - CI only validates; it never applies. If you merge first, main and the live state drift until someone applies
|
||||
- **All GCP changes via Pulumi PRs** - changes in `infrastructure/` deploy automatically on merge to main
|
||||
- Never modify GCP resources directly via console or `gcloud` CLI
|
||||
- `gcloud` CLI for reading/debugging only (e.g., checking logs, SSH to VMs)
|
||||
- Local `pulumi preview` works but `pulumi up` requires broader permissions than CI
|
||||
- Stop and notify user on auth issues
|
||||
- **GCS ops on macOS: never use `gsutil -m`** — its fork-based multiprocessing segfaults
|
||||
on macOS (crash-report popups + the parent process hangs forever). Use `gcloud storage`
|
||||
instead (`gcloud storage cp/rm/rsync …`) — no fork, and faster. Plain `gsutil` (no `-m`)
|
||||
is safe but slower. `~/.boto` pins `parallel_process_count=1` as a safety net; don't remove it.
|
||||
|
||||
### Internationalization (i18n)
|
||||
|
||||
- All user-facing strings live in `frontend/messages/{locale}.json` (33 locales), NOT in components
|
||||
- Backend strings live in `backend/translations/{locale}.json` (currently 3 locales: en, es, de)
|
||||
- Components use `useTranslations('namespace')` from next-intl
|
||||
- Pages are under `frontend/app/[locale]/` — locale-aware routing (customer-facing: `/`, `/app/*`, `/order/*`, `/auth/*`, `/payment/*`, `/r/*`)
|
||||
- Non-locale routes at those same paths (e.g. `frontend/app/app/jobs/[[...slug]]/page.tsx`) are `<LocaleRedirect />` shims that redirect into `[locale]` — they are NOT English-only surfaces
|
||||
- The `/app/jobs/#/{id}/review`, `/instrumental`, and `/audio-edit` review UIs are fully multilingual (components under `components/lyrics-review/`, `components/instrumental-review/`, `components/audio-editor/` use `useTranslations` heavily)
|
||||
- Only `/admin/*` is English-only — it has no `[locale]/admin` counterpart and inherits `DefaultIntlProvider` from the root layout
|
||||
- Internal links use `Link` from `@/i18n/routing` (locale-aware)
|
||||
- After adding/changing English strings: `python frontend/scripts/translate.py --messages-dir frontend/messages --target all`
|
||||
- Translation uses Gemini 3.8 Flash via the **Gemini Developer API** (key: `GEMINI_API_KEY`, else Secret Manager `gemini-api-key` via your gcloud login — `frontend/scripts/gemini_client.py`) with GCS cache for efficiency. **Vertex AI is disabled in the GCP project — never use `vertexai=True`; backend code uses `backend/services/gemini_client.get_genai_client()`.**
|
||||
- CI validates all locale files have matching keys — **PR will fail if translations are missing**
|
||||
- Pre-commit hook auto-translates when en.json changes (enable: `git config core.hooksPath .githooks`)
|
||||
- Don't hardcode user-facing strings — add to `messages/en.json` and use `t('key')`
|
||||
|
||||
## API Authentication (for Agents)
|
||||
|
||||
@@ -129,7 +104,7 @@ Use the `X-Admin-Token` header with the token from Secret Manager:
|
||||
```bash
|
||||
# Trigger video worker
|
||||
curl -X POST "https://api.nomadkaraoke.com/api/internal/workers/video" \
|
||||
-H "X-Admin-Token: $(gcloud secrets versions access latest --secret=admin-tokens --project=nomadkaraoke --account=claude-readonly@nomadkaraoke.iam.gserviceaccount.com)" \
|
||||
-H "X-Admin-Token: $(gcloud secrets versions access latest --secret=admin-tokens --project=nomadkaraoke)" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"job_id": "YOUR_JOB_ID"}'
|
||||
|
||||
@@ -152,33 +127,16 @@ EOF
|
||||
```
|
||||
|
||||
### GCE Encoding Worker
|
||||
Since 2026-11 the only GCE encoding VM is the stopped on-demand fallback
|
||||
`encoding-worker-fallback-c2df` (c2d-highcpu-16, us-central1-f); the c4d Spot pair
|
||||
(`encoding-worker-a`/`-b`) was retired and deleted, and `primary_vm`/`secondary_vm` in
|
||||
`config/encoding-worker` are empty. The serving worker is `active_override_vm` (normally the
|
||||
AWS box below). Idle VMs stop after 5 min. CI's "blue-green" deploy now only runs when worker
|
||||
code changed (`infrastructure/encoding-worker/worker_code_paths.txt`, diffed against
|
||||
`validated_version`): it boots c2df, runs a test encode, records `validated_version` and stops
|
||||
it again, leaving the AWS override serving. Force it via Actions → CI → Run workflow →
|
||||
"Force a blue-green redeploy of the GCE encoding worker".
|
||||
```bash
|
||||
# Check health (includes wheel_version) on the GCE fallback.
|
||||
gcloud compute ssh encoding-worker-fallback-c2df --zone=us-central1-f --project=nomadkaraoke \
|
||||
--command="curl -s http://localhost:8080/health"
|
||||
|
||||
# Restart the service (pick up new worker-side code after a deploy)
|
||||
gcloud compute ssh encoding-worker-fallback-c2df --zone=us-central1-f --project=nomadkaraoke \
|
||||
# SSH to restart service (clears in-memory job queue)
|
||||
gcloud compute ssh encoding-worker --zone=us-central1-c --project=nomadkaraoke \
|
||||
--command="sudo systemctl restart encoding-worker"
|
||||
|
||||
# Check health
|
||||
gcloud compute ssh encoding-worker --zone=us-central1-c --project=nomadkaraoke \
|
||||
--command="curl -s http://localhost:8080/health"
|
||||
```
|
||||
|
||||
For job stuck at `encoding` status, see [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md).
|
||||
|
||||
**AWS encoding worker** (`aws-encoding-worker-1`, EC2 c7a.4xlarge on-demand, us-east-2) is a pool
|
||||
candidate too (`provider: aws` in `encoding-worker-fallback-vms`): last resort by default,
|
||||
preferred when Firestore `config/encoding-worker.speed_rank_overrides = {"c7a": 5}`. Not
|
||||
blue-green deployed, idle-stops itself, shell via SSM, logs are on the box (journald). See
|
||||
`infrastructure/aws/README.md`.
|
||||
|
||||
### What Doesn't Work
|
||||
- `gcloud auth print-identity-token` - Wrong token type for internal endpoints
|
||||
- Cloud Tasks without proper service account - Gets 401 from Cloud Run
|
||||
|
||||
-109
@@ -1,109 +0,0 @@
|
||||
# Contributing to karaoke-gen
|
||||
|
||||
Thanks for your interest in contributing! This guide focuses on getting a
|
||||
working **local development environment with no Google Cloud credentials or
|
||||
paid API keys** — everything runs against local emulators.
|
||||
|
||||
## What runs locally (no credentials needed)
|
||||
|
||||
| Component | Local story |
|
||||
|-----------|-------------|
|
||||
| FastAPI backend | Runs directly via uvicorn |
|
||||
| Firestore | Google's Firestore emulator |
|
||||
| Cloud Storage | [fake-gcs-server](https://github.com/fsouza/fake-gcs-server) (Docker) |
|
||||
| Background workers (audio, lyrics, screens, video…) | Run as local subprocesses / in-process tasks |
|
||||
| Audio separation | Local CPU/GPU via `audio-separator` (set `MODEL_DIR`; models download on first use) |
|
||||
| Lyrics transcription | Local Whisper (`poetry install --extras local-whisper`); AudioShake used only if you set a token |
|
||||
| Login (magic links) | Printed to the backend console instead of emailed |
|
||||
| Signed URLs (review audio, downloads, uploads) | Rewritten to plain emulator URLs |
|
||||
| Frontend (Next.js) | `npm run dev`, pointed at the local backend |
|
||||
|
||||
Things that do NOT run locally: payments (Stripe), production email
|
||||
(Postmark), YouTube/Drive/Dropbox distribution, and the `flacfetch`
|
||||
URL-download service — **jobs from URLs won't process locally; use file
|
||||
upload instead.**
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.12/3.13 + [Poetry](https://python-poetry.org/)
|
||||
- Node.js 20+ (for the frontend)
|
||||
- Docker (for the GCS emulator)
|
||||
- Java 21+ and the Google Cloud SDK with the Firestore emulator:
|
||||
`gcloud components install cloud-firestore-emulator`
|
||||
(no login/account needed — the emulator runs entirely offline)
|
||||
- ffmpeg (`brew install ffmpeg` / `apt install ffmpeg`)
|
||||
|
||||
## Backend quickstart
|
||||
|
||||
```bash
|
||||
poetry install # heavy: torch etc, ~10 min first time
|
||||
./scripts/run-backend-local.sh --with-emulators
|
||||
```
|
||||
|
||||
This starts the Firestore emulator (port 8080), fake-gcs-server (port 4443),
|
||||
creates the bucket, seeds a default theme (`scripts/seed-local-data.py`), and
|
||||
runs the backend at <http://localhost:8000> (API docs at `/docs`).
|
||||
|
||||
Optional API keys go in `.env` (see `.env.example`) — none are required to boot.
|
||||
|
||||
### Try it
|
||||
|
||||
```bash
|
||||
# Health
|
||||
curl http://localhost:8000/api/health
|
||||
|
||||
# Create a job from a local audio file (admin token is "local-dev-token")
|
||||
curl -X POST http://localhost:8000/api/jobs/upload \
|
||||
-H 'Authorization: Bearer local-dev-token' \
|
||||
-F "file=@/path/to/song.flac" -F "artist=Some Artist" -F "title=Some Song"
|
||||
|
||||
# Watch it process; when status hits awaiting_review, open the review UI
|
||||
curl http://localhost:8000/api/jobs/<job_id> -H 'Authorization: Bearer local-dev-token'
|
||||
```
|
||||
|
||||
For audio separation to run, set `MODEL_DIR` (e.g. `MODEL_DIR=./models`) —
|
||||
separation models (~2 GB) are downloaded on first use. Without it the job
|
||||
fails at the separation step with a clear error.
|
||||
|
||||
## Frontend quickstart
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
NEXT_PUBLIC_API_URL=http://localhost:8000 npx next dev
|
||||
```
|
||||
|
||||
Then open <http://localhost:3000>. To log in, enter any email address — the
|
||||
magic link is printed in the **backend** console (look for `EMAIL TO:`); open
|
||||
that link in your browser.
|
||||
|
||||
### Review UI without running the full pipeline
|
||||
|
||||
If you're working on the lyrics review UI specifically, there's a fixtures
|
||||
harness that serves real correction data through the review API shape without
|
||||
processing any audio:
|
||||
|
||||
```bash
|
||||
python scripts/review_test_fixtures.py # serves on :8765
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
make test # everything (backend + frontend)
|
||||
make test-backend # backend only (unit + emulator tests)
|
||||
```
|
||||
|
||||
Emulator-backed integration tests live in `backend/tests/emulator/` and use
|
||||
the same two emulators as the local dev setup.
|
||||
|
||||
## Pull requests
|
||||
|
||||
- Keep PRs focused; include tests for behavior changes.
|
||||
- Run `make test` before pushing — CI runs the same suite.
|
||||
- Bump `tool.poetry.version` in `pyproject.toml` for code changes.
|
||||
- CI must pass before merge; a maintainer will review from there.
|
||||
|
||||
See `docs/DEVELOPMENT.md` for deeper docs (architecture, deployment,
|
||||
observability) — most of that file concerns the production GCP deployment,
|
||||
which contributors don't need.
|
||||
@@ -1,25 +1,18 @@
|
||||
# Makefile for karaoke-gen project
|
||||
# Run `make help` to see available commands
|
||||
|
||||
.PHONY: help install install-backend install-frontend build-frontend dev-install test test-unit test-backend test-e2e test-frontend test-all lint clean emulators-start emulators-stop local-stack-up local-stack-down test-local-stack eval-custom-lyrics
|
||||
|
||||
# Per-stage timeout in seconds (default: 10 minutes)
|
||||
# Prevents hangs from stale dev servers, broken emulators, etc.
|
||||
# Override with: make test TEST_TIMEOUT=300
|
||||
TEST_TIMEOUT ?= 600
|
||||
.PHONY: help install install-backend install-frontend build-frontend test test-unit test-backend test-e2e test-frontend test-all lint clean emulators-start emulators-stop
|
||||
|
||||
# Default target
|
||||
help:
|
||||
@echo "Available commands:"
|
||||
@echo " make test - Run ALL tests (backend + frontend, installs deps automatically)"
|
||||
@echo " make test-backend - Run backend tests only (unit + emulator)"
|
||||
@echo " make test-frontend - Run frontend unit tests (Jest)"
|
||||
@echo " make test-frontend-e2e - Run frontend E2E tests (Playwright, requires dev server)"
|
||||
@echo " make test-frontend - Run frontend tests only (unit + E2E)"
|
||||
@echo " make test-unit - Run unit tests only (karaoke_gen package)"
|
||||
@echo " make test-e2e - Run emulator tests with auto-start/stop"
|
||||
@echo " make install - Install all dependencies (backend + frontend)"
|
||||
@echo " make build-frontend - Build frontend and copy to Python package"
|
||||
@echo " make dev-install - Build frontend + pip install -e . (for local CLI testing)"
|
||||
@echo " make build-frontend - Build frontend and copy to Python package (for local testing)"
|
||||
@echo " make lint - Run linter checks"
|
||||
@echo " make emulators-start - Start GCP emulators for local development"
|
||||
@echo " make emulators-stop - Stop GCP emulators"
|
||||
@@ -41,38 +34,23 @@ install-frontend:
|
||||
|
||||
install: install-backend install-frontend
|
||||
|
||||
# Build frontend and copy to Python package
|
||||
# Build frontend and copy to Python package (for local CLI testing)
|
||||
build-frontend: install-frontend
|
||||
@echo "=== Building frontend ==="
|
||||
cd frontend && npm run build
|
||||
@echo "=== Copying build to Python package ==="
|
||||
rm -rf karaoke_gen/nextjs_frontend/out
|
||||
mkdir -p karaoke_gen/nextjs_frontend/out
|
||||
cp -r frontend/out/* karaoke_gen/nextjs_frontend/out/
|
||||
@# Drop Next.js RSC payloads (*.txt, ~66 MB across 33 locales): the CLI review
|
||||
@# server never serves them (only HTML entrypoints, /_next and /static), and they
|
||||
@# pushed the wheel past PyPI's 100 MiB per-file limit (2026-10-27..29).
|
||||
find karaoke_gen/nextjs_frontend/out -name '*.txt' -type f -delete
|
||||
@echo "✅ Frontend built and copied to karaoke_gen/nextjs_frontend/out/"
|
||||
|
||||
# Build frontend + install package in editable mode for local CLI testing
|
||||
# After running this, `karaoke-gen` CLI will use the freshly built frontend.
|
||||
dev-install: build-frontend
|
||||
@echo "=== Installing package in editable mode ==="
|
||||
pip install -e .
|
||||
@echo ""
|
||||
@echo "✅ Ready! You can now run karaoke-gen with the updated frontend."
|
||||
@echo " Example: karaoke-gen input.flac \"Artist\" \"Title\""
|
||||
@echo "✅ Frontend built and ready for local testing"
|
||||
|
||||
# Run unit tests for karaoke_gen package
|
||||
test-unit: install-backend
|
||||
@echo "=== Running karaoke_gen unit tests ==="
|
||||
timeout $(TEST_TIMEOUT) poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=term-missing --cov-fail-under=69
|
||||
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=term-missing --cov-fail-under=69
|
||||
|
||||
# Run backend unit tests (excludes emulator tests)
|
||||
test-backend-unit: install-backend
|
||||
@echo "=== Running backend unit tests ==="
|
||||
timeout $(TEST_TIMEOUT) poetry run pytest backend/tests/ --ignore=backend/tests/emulator -v
|
||||
poetry run pytest backend/tests/ --ignore=backend/tests/emulator -v
|
||||
|
||||
# Run E2E integration tests with emulators
|
||||
test-e2e: install-backend
|
||||
@@ -84,7 +62,7 @@ test-e2e: install-backend
|
||||
export GCS_BUCKET_NAME=test-bucket && \
|
||||
export ADMIN_TOKENS=test-admin-token && \
|
||||
export ENVIRONMENT=test && \
|
||||
timeout $(TEST_TIMEOUT) poetry run pytest backend/tests/emulator/ -v; \
|
||||
poetry run pytest backend/tests/emulator/ -v; \
|
||||
TEST_RESULT=$$?; \
|
||||
./scripts/stop-emulators.sh; \
|
||||
exit $$TEST_RESULT
|
||||
@@ -94,15 +72,10 @@ test-backend: test-unit test-backend-unit test-e2e
|
||||
@echo ""
|
||||
@echo "✅ Backend tests passed!"
|
||||
|
||||
# Run frontend unit tests (Jest only - E2E requires manual setup)
|
||||
# Run frontend tests (unit + E2E)
|
||||
test-frontend: install-frontend
|
||||
@echo "=== Running frontend unit tests ==="
|
||||
cd frontend && timeout $(TEST_TIMEOUT) npm run test:unit
|
||||
|
||||
# Run frontend E2E tests (requires dev server - run manually, not in make test)
|
||||
test-frontend-e2e: install-frontend
|
||||
@echo "=== Running frontend E2E tests ==="
|
||||
cd frontend && timeout $(TEST_TIMEOUT) npm run test:e2e
|
||||
@echo "=== Running frontend tests ==="
|
||||
cd frontend && npm run test:all
|
||||
|
||||
# Run ALL tests (backend + frontend) - use this before committing!
|
||||
test: test-backend test-frontend
|
||||
@@ -128,17 +101,6 @@ emulators-stop:
|
||||
@echo "=== Stopping GCP emulators ==="
|
||||
./scripts/stop-emulators.sh
|
||||
|
||||
# Local E2E stack: real backend on emulators + fake workers (see docs/TESTING.md)
|
||||
local-stack-up:
|
||||
./scripts/local-stack.sh up --frontend
|
||||
|
||||
local-stack-down:
|
||||
./scripts/local-stack.sh down
|
||||
|
||||
# Full-journey Playwright suite against the local stack (starts it if needed)
|
||||
test-local-stack: install-frontend
|
||||
cd frontend && timeout $(TEST_TIMEOUT) npm run test:e2e:local-stack
|
||||
|
||||
# Clean up temporary files
|
||||
clean:
|
||||
find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
|
||||
@@ -148,7 +110,3 @@ clean:
|
||||
rm -rf htmlcov/ .coverage coverage.xml 2>/dev/null || true
|
||||
@echo "Cleaned up temporary files"
|
||||
|
||||
.PHONY: eval-custom-lyrics
|
||||
eval-custom-lyrics:
|
||||
cd backend && poetry run python -m backend.eval.custom_lyrics.run
|
||||
|
||||
@@ -675,6 +675,4 @@ MIT
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) — the whole
|
||||
backend runs locally against emulators with no Google Cloud credentials or
|
||||
paid API keys required.
|
||||
Contributions are welcome! Please see our contributing guidelines.
|
||||
@@ -29,8 +29,6 @@ tmp/
|
||||
# Google Cloud
|
||||
.gcloud/
|
||||
*.json
|
||||
# Allow i18n translation files
|
||||
!translations/*.json
|
||||
|
||||
# Testing
|
||||
.pytest_cache/
|
||||
|
||||
+1
-13
@@ -26,19 +26,7 @@ COPY backend /app/backend
|
||||
# Note: lyrics_transcriber is now a subpackage of karaoke_gen
|
||||
COPY karaoke_gen /app/karaoke_gen
|
||||
COPY pyproject.toml README.md LICENSE /app/
|
||||
# GPU images (MODEL_DIR set in base): use --no-deps to preserve pinned torch cu126
|
||||
# and audio-separator[gpu]. Without this, pip upgrades torch to cu130 (breaks CUDA
|
||||
# on Cloud Run L4 drivers) and reinstalls audio-separator[cpu] (breaks GPU ONNX).
|
||||
# CPU images: normal install resolves all deps from pyproject.toml.
|
||||
RUN if [ -n "$MODEL_DIR" ]; then \
|
||||
pip install --no-cache-dir --no-deps -e .; \
|
||||
else \
|
||||
pip install --no-cache-dir -e .; \
|
||||
fi
|
||||
|
||||
# Precompile bytecode so lazy imports never write .pyc at runtime: a cold
|
||||
# import that compiles + writes on the event loop has stalled requests 10-15s.
|
||||
RUN python -m compileall -q /app/backend /app/karaoke_gen
|
||||
RUN pip install --no-cache-dir -e .
|
||||
|
||||
# Set environment variables
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
+7
-38
@@ -1,15 +1,8 @@
|
||||
# Base image with all Python dependencies pre-installed
|
||||
# This image is rebuilt only when poetry.lock or this Dockerfile changes
|
||||
#
|
||||
# This image is rebuilt only when poetry.lock changes
|
||||
#
|
||||
# Build with: gcloud builds submit --config=cloudbuild-base.yaml
|
||||
# Or manually: docker build -f backend/Dockerfile.base -t karaoke-backend-base .
|
||||
#
|
||||
# Layer caching strategy:
|
||||
# 1. System deps + fonts + ffmpeg (rarely change, always cached)
|
||||
# 2. pyproject.toml + package stubs (changes with dep updates)
|
||||
# 3. pip install (the slow part - cached unless pyproject.toml changes)
|
||||
# 4. Source code copy (changes every rebuild, but AFTER deps are installed)
|
||||
# 5. spaCy model download (cached unless pip install layer changes)
|
||||
|
||||
FROM python:3.11-slim
|
||||
|
||||
@@ -23,7 +16,6 @@ RUN apt-get update && apt-get install -y \
|
||||
build-essential \
|
||||
curl \
|
||||
xz-utils \
|
||||
chromium \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install comprehensive font support for international lyrics and symbols
|
||||
@@ -58,39 +50,16 @@ RUN curl -L https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-stati
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Copy ONLY dependency specification files first (changes less often than source code)
|
||||
# Copy dependency files and source code needed for pip install
|
||||
COPY pyproject.toml README.md LICENSE /app/
|
||||
|
||||
# Create minimal package stubs so pip can resolve the editable install
|
||||
# pip install -e . needs package directories to exist but doesn't validate content
|
||||
RUN mkdir -p /app/karaoke_gen /app/lyrics_transcriber_temp /app/backend && \
|
||||
touch /app/karaoke_gen/__init__.py /app/lyrics_transcriber_temp/__init__.py /app/backend/__init__.py
|
||||
|
||||
# Install all Python dependencies (the slow part)
|
||||
# BuildKit cache mount preserves downloaded wheels across builds, so partial
|
||||
# dependency updates only download changed packages instead of everything
|
||||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||||
pip install --upgrade pip && \
|
||||
pip install -e .
|
||||
|
||||
# Pre-build matplotlib's font cache at BUILD time. Without it, the first
|
||||
# `import matplotlib.pyplot` on every new Cloud Run instance scanned all the
|
||||
# Noto fonts above (20-40s of GIL-holding CPU) and froze the API's event loop
|
||||
# — prod logged "Matplotlib is building the font cache" in 17 of 28
|
||||
# "servers unavailable" banner episodes (2026-10-03 analysis). The cache lives
|
||||
# in MPLCONFIGDIR so it ships in the image instead of the in-memory $HOME.
|
||||
ENV MPLCONFIGDIR=/opt/mplconfig
|
||||
RUN mkdir -p /opt/mplconfig && \
|
||||
python -c "import matplotlib; matplotlib.use('Agg'); import matplotlib.font_manager as fm; print(len(fm.fontManager.ttflist), 'fonts cached')" && \
|
||||
ls /opt/mplconfig/fontlist-*.json && \
|
||||
chmod -R a+rwX /opt/mplconfig
|
||||
|
||||
# Now copy actual source code (AFTER deps installed, so code changes don't
|
||||
# invalidate the expensive pip install layer in registry cache)
|
||||
COPY karaoke_gen /app/karaoke_gen
|
||||
COPY lyrics_transcriber_temp /app/lyrics_transcriber_temp
|
||||
COPY backend /app/backend
|
||||
|
||||
# Install all Python dependencies (the slow part - cached in base image)
|
||||
RUN pip install --no-cache-dir --upgrade pip && \
|
||||
pip install --no-cache-dir -e .
|
||||
|
||||
# Download spaCy language model for lyrics transcription
|
||||
# This must be done at build time, not runtime (Cloud Run network restrictions)
|
||||
RUN python -m spacy download en_core_web_sm
|
||||
|
||||
@@ -1,144 +0,0 @@
|
||||
# GPU base image with CUDA, PyTorch, audio-separator, and pre-downloaded models
|
||||
# This image is rebuilt only when deps or models change (infrequently).
|
||||
#
|
||||
# Build with: gcloud builds submit --config=backend/cloudbuild-gpu-base.yaml --project=nomadkaraoke --timeout=3600s
|
||||
# Or manually: docker build -f backend/Dockerfile.gpu-base -t karaoke-backend-gpu-base .
|
||||
#
|
||||
# Layer caching strategy:
|
||||
# 1. System deps + fonts + ffmpeg (rarely change, always cached)
|
||||
# 2. CUDA library path configuration
|
||||
# 3. pyproject.toml + package stubs (changes with dep updates)
|
||||
# 4. pip install karaoke-gen deps + audio-separator[gpu] (slow, cached unless pyproject.toml changes)
|
||||
# 5. Source code copy (changes every rebuild, but AFTER deps are installed)
|
||||
# 6. spaCy model download (cached unless pip install layer changes)
|
||||
# 7. Pre-downloaded ensemble preset models (~1-1.5 GB, cached unless pip layer changes)
|
||||
|
||||
FROM nvidia/cuda:12.6.3-runtime-ubuntu22.04
|
||||
|
||||
# Prevent interactive prompts during package installation
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# Install Python 3.11 from deadsnakes PPA and system dependencies
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
software-properties-common \
|
||||
&& add-apt-repository -y ppa:deadsnakes/ppa \
|
||||
&& apt-get update && apt-get install -y --no-install-recommends \
|
||||
# Python 3.11
|
||||
python3.11 \
|
||||
python3.11-dev \
|
||||
python3.11-venv \
|
||||
# Audio libraries
|
||||
libsndfile1 \
|
||||
libsox-dev \
|
||||
sox \
|
||||
# Build tools
|
||||
build-essential \
|
||||
# Utilities
|
||||
curl \
|
||||
xz-utils \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& python3.11 --version
|
||||
|
||||
# Install comprehensive font support for international lyrics and symbols
|
||||
# Noto fonts provide full Unicode coverage including:
|
||||
# - Latin, Greek, Cyrillic (core)
|
||||
# - CJK: Chinese, Japanese, Korean (~150MB)
|
||||
# - Arabic, Hebrew, Thai, Hindi, and other scripts (extra)
|
||||
# - Color emoji (limited support in libass, but available)
|
||||
# - Musical symbols (♪ ♫ etc.)
|
||||
# Note: This adds ~200-300MB to the image but ensures all languages render correctly
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
fonts-noto-core \
|
||||
fonts-noto-cjk \
|
||||
fonts-noto-extra \
|
||||
fonts-noto-color-emoji \
|
||||
fontconfig \
|
||||
&& fc-cache -f \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install FFmpeg from Ubuntu package
|
||||
# The audio worker only uses FFmpeg for format conversion and padding, not
|
||||
# encoding-intensive tasks, so the Debian package is sufficient. The static
|
||||
# build from johnvansickle.com is used for the encoding worker instead.
|
||||
RUN apt-get update && apt-get install -y ffmpeg && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Set Python 3.11 as default and install pip
|
||||
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1 \
|
||||
&& update-alternatives --install /usr/bin/python python /usr/bin/python3.11 1 \
|
||||
&& curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11 \
|
||||
&& python3 -m pip install --no-cache-dir --upgrade pip setuptools wheel
|
||||
|
||||
# Set up CUDA library paths so GPU-accelerated packages can find CUDA libs at runtime
|
||||
RUN echo '/usr/local/cuda/lib64' >> /etc/ld.so.conf.d/cuda.conf && ldconfig
|
||||
|
||||
# Environment configuration for CUDA and Python
|
||||
ENV LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH \
|
||||
PATH=/usr/local/cuda/bin:$PATH \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
MODEL_DIR=/models
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Copy ONLY dependency specification files first (changes less often than source code)
|
||||
COPY pyproject.toml README.md LICENSE /app/
|
||||
|
||||
# Create minimal package stubs so pip can resolve the editable install
|
||||
# pip install -e . needs package directories to exist but doesn't validate content
|
||||
RUN mkdir -p /app/karaoke_gen /app/lyrics_transcriber_temp /app/backend && \
|
||||
touch /app/karaoke_gen/__init__.py /app/lyrics_transcriber_temp/__init__.py /app/backend/__init__.py
|
||||
|
||||
# Install all karaoke-gen Python dependencies (the slow part)
|
||||
# BuildKit cache mount preserves downloaded wheels across builds
|
||||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||||
pip install --upgrade pip && \
|
||||
pip install -e .
|
||||
|
||||
# Install audio-separator with GPU support (PyTorch + CUDA + onnxruntime-gpu)
|
||||
# This is separate from pip install -e . to keep layer caching granular
|
||||
#
|
||||
# IMPORTANT: Install PyTorch cu126 FIRST, then audio-separator[gpu].
|
||||
# The default audio-separator[gpu] pulls PyTorch cu130, but Cloud Run L4 GPUs
|
||||
# have NVIDIA driver 570 which only supports up to CUDA 12.8. PyTorch cu130
|
||||
# fails with "NVIDIA driver is too old". PyTorch cu126 works with 12.8 drivers.
|
||||
# Installing torch first means audio-separator[gpu] sees it as already satisfied.
|
||||
#
|
||||
# PINNED to an exact version: an unpinned install silently picked up 0.47.0 on the
|
||||
# 2026-10-17 rebuild and separation went from ~5 to ~10-16 min. Cause: 0.47 (#299) made
|
||||
# RoFormer `overlap` a window count defaulting to each model YAML's num_overlap (4-8 for
|
||||
# the karaoke models = 4-6x the forward passes of 0.44's fixed 8 s step). karaoke_gen's
|
||||
# audio_processor now passes overlap=2 on >= 0.47 (SEPARATOR_MDXC_PARAMS). Time a real
|
||||
# job before bumping this, and read the audio-separator release notes for compute changes.
|
||||
ARG AUDIO_SEPARATOR_VERSION=0.47.1
|
||||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||||
pip install torch==2.6.0+cu126 torchvision==0.21.0+cu126 --index-url https://download.pytorch.org/whl/cu126 && \
|
||||
pip install "audio-separator[gpu]==${AUDIO_SEPARATOR_VERSION}"
|
||||
|
||||
# Now copy actual source code (AFTER deps installed, so code changes don't
|
||||
# invalidate the expensive pip install layer in registry cache)
|
||||
COPY karaoke_gen /app/karaoke_gen
|
||||
COPY lyrics_transcriber_temp /app/lyrics_transcriber_temp
|
||||
COPY backend /app/backend
|
||||
|
||||
# Download spaCy language model for lyrics transcription
|
||||
# Must be done at build time, not runtime (Cloud Run network restrictions)
|
||||
RUN python -m spacy download en_core_web_sm
|
||||
|
||||
# Create models directory and pre-download ensemble preset models
|
||||
# Baking models into the image eliminates cold-start model download latency.
|
||||
# Total model size: ~1-1.5 GB for instrumental_clean + karaoke presets.
|
||||
RUN mkdir -p /models
|
||||
COPY backend/scripts/download_models.py /tmp/download_models.py
|
||||
RUN python /tmp/download_models.py /models && \
|
||||
rm /tmp/download_models.py && \
|
||||
ls -lh /models/
|
||||
|
||||
# Label with build info
|
||||
ARG BUILD_DATE
|
||||
ARG BUILD_ID
|
||||
ARG BASE_CONTENT_HASH
|
||||
LABEL org.opencontainers.image.created="${BUILD_DATE}"
|
||||
LABEL org.opencontainers.image.revision="${BUILD_ID}"
|
||||
LABEL org.opencontainers.image.title="karaoke-backend-gpu-base"
|
||||
LABEL org.opencontainers.image.description="GPU base image with CUDA, PyTorch, audio-separator, and pre-downloaded models for karaoke-backend audio worker"
|
||||
LABEL base.content.hash="${BASE_CONTENT_HASH}"
|
||||
+23
-40
@@ -1,7 +1,6 @@
|
||||
"""
|
||||
FastAPI dependencies for authentication and authorization.
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, UTC
|
||||
@@ -90,8 +89,7 @@ async def require_auth(
|
||||
)
|
||||
|
||||
# Validate token using the full method
|
||||
# Sync Firestore reads: run off the event loop (every authed request passes here).
|
||||
auth_result = await asyncio.to_thread(auth_service.validate_token_full, token_str)
|
||||
auth_result = auth_service.validate_token_full(token_str)
|
||||
|
||||
# Get request_id from middleware for correlation
|
||||
request_id = getattr(request.state, "request_id", None)
|
||||
@@ -115,19 +113,6 @@ async def require_auth(
|
||||
detail=f"Authentication failed: {auth_result.message}"
|
||||
)
|
||||
|
||||
# If token has a tenant_id, override the hostname-based tenant detection.
|
||||
# This allows tenant-scoped tokens to enforce tenant context regardless of domain.
|
||||
# We must also load tenant_config so downstream handlers (e.g. job creation)
|
||||
# that rely on tenant_config for setting tenant_id on jobs work correctly.
|
||||
if auth_result.tenant_id:
|
||||
request.state.tenant_id = auth_result.tenant_id
|
||||
if not getattr(request.state, "tenant_config", None):
|
||||
from backend.services.tenant_service import get_tenant_service
|
||||
tenant_service = get_tenant_service()
|
||||
tenant_config = await asyncio.to_thread(tenant_service.get_tenant_config, auth_result.tenant_id)
|
||||
if tenant_config and tenant_config.is_active:
|
||||
request.state.tenant_config = tenant_config
|
||||
|
||||
# Log successful auth with request_id for correlation with request audit
|
||||
logger.info(
|
||||
"auth_success",
|
||||
@@ -137,7 +122,6 @@ async def require_auth(
|
||||
"user_type": auth_result.user_type.value if auth_result.user_type else None,
|
||||
"is_admin": auth_result.is_admin,
|
||||
"remaining_uses": auth_result.remaining_uses,
|
||||
"tenant_id": auth_result.tenant_id,
|
||||
"audit_type": "auth_event",
|
||||
}
|
||||
)
|
||||
@@ -167,7 +151,7 @@ async def require_auth_legacy(
|
||||
detail="Authentication required. Provide token via Authorization header or ?token= parameter"
|
||||
)
|
||||
|
||||
is_valid, user_type, remaining_uses, message = await asyncio.to_thread(auth_service.validate_token, token_str)
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(token_str)
|
||||
|
||||
if not is_valid:
|
||||
raise HTTPException(
|
||||
@@ -286,39 +270,38 @@ def require_review_auth_factory(job_id_param: str = "job_id"):
|
||||
full_token = token
|
||||
|
||||
if full_token:
|
||||
auth_result = await asyncio.to_thread(auth_service.validate_token_full, full_token)
|
||||
auth_result = auth_service.validate_token_full(full_token)
|
||||
if auth_result.is_valid:
|
||||
# For full auth, also verify job ownership
|
||||
job = await asyncio.to_thread(job_manager.get_job, job_id)
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Admin, owner, or a member of the job's tenant portal
|
||||
# (backend/services/job_access.py).
|
||||
from backend.services.job_access import can_access_job
|
||||
|
||||
if not await asyncio.to_thread(
|
||||
can_access_job, job, auth_result.user_email, bool(auth_result.is_admin)
|
||||
):
|
||||
logger.warning(
|
||||
f"Review access denied: user {auth_result.user_email} "
|
||||
f"tried to access job {job_id} owned by {job.user_email}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's review"
|
||||
)
|
||||
# Check ownership: admin can access all, users only their own jobs
|
||||
if not auth_result.is_admin:
|
||||
if auth_result.user_email and job.user_email:
|
||||
if auth_result.user_email.lower() != job.user_email.lower():
|
||||
logger.warning(
|
||||
f"Review access denied: user {auth_result.user_email} "
|
||||
f"tried to access job {job_id} owned by {job.user_email}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's review"
|
||||
)
|
||||
elif job.user_email:
|
||||
# Token auth without email trying to access a job with owner
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's review"
|
||||
)
|
||||
|
||||
logger.info(f"Review access granted via full auth ({auth_result.user_type}) for job {job_id}")
|
||||
# Who is reviewing — handlers record it on first open
|
||||
# (state_data.review_started_by, see JobManager.record_review_started).
|
||||
request.state.review_auth_email = auth_result.user_email
|
||||
request.state.review_auth_is_admin = bool(auth_result.is_admin)
|
||||
return job_id, "full"
|
||||
|
||||
# Try review token
|
||||
if review_token:
|
||||
job = await asyncio.to_thread(job_manager.get_job, job_id)
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
+148
-2870
File diff suppressed because it is too large.
Load diff
+169
-924
File diff suppressed because it is too large.
Load diff
+38
-44
@@ -9,7 +9,7 @@ Provides endpoints for:
|
||||
import logging
|
||||
from typing import Optional
|
||||
from datetime import datetime
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Request
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks
|
||||
from pydantic import BaseModel
|
||||
|
||||
from backend.services.credential_manager import (
|
||||
@@ -18,7 +18,6 @@ from backend.services.credential_manager import (
|
||||
CredentialCheckResult,
|
||||
)
|
||||
from backend.config import get_settings
|
||||
from backend.i18n import t, get_locale_from_request
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/auth", tags=["auth"])
|
||||
@@ -91,7 +90,7 @@ class CredentialValidationResponse(BaseModel):
|
||||
# ===========================================================================
|
||||
|
||||
@router.get("/status", response_model=AllCredentialsStatusResponse)
|
||||
def get_credentials_status():
|
||||
async def get_credentials_status():
|
||||
"""
|
||||
Get the status of all OAuth credentials.
|
||||
|
||||
@@ -126,16 +125,15 @@ def get_credentials_status():
|
||||
|
||||
|
||||
@router.get("/status/{service}", response_model=CredentialStatusResponse)
|
||||
def get_service_credential_status(service: str, request: Request):
|
||||
async def get_service_credential_status(service: str):
|
||||
"""
|
||||
Get the status of a specific service's OAuth credentials.
|
||||
|
||||
|
||||
Args:
|
||||
service: One of "youtube", "gdrive", "dropbox"
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
manager = get_credential_manager()
|
||||
|
||||
|
||||
if service == "youtube":
|
||||
result = manager.check_youtube_credentials()
|
||||
elif service == "gdrive":
|
||||
@@ -143,7 +141,7 @@ def get_service_credential_status(service: str, request: Request):
|
||||
elif service == "dropbox":
|
||||
result = manager.check_dropbox_credentials()
|
||||
else:
|
||||
raise HTTPException(status_code=400, detail=t(locale, "auth.unknownService", service=service))
|
||||
raise HTTPException(status_code=400, detail=f"Unknown service: {service}")
|
||||
|
||||
return CredentialStatusResponse(
|
||||
service=result.service,
|
||||
@@ -155,43 +153,42 @@ def get_service_credential_status(service: str, request: Request):
|
||||
|
||||
|
||||
@router.post("/validate", response_model=CredentialValidationResponse)
|
||||
def validate_credentials(request: CredentialValidationRequest, http_request: Request):
|
||||
async def validate_credentials(request: CredentialValidationRequest):
|
||||
"""
|
||||
Validate that credentials are available for requested services.
|
||||
|
||||
|
||||
Use this before submitting a job to ensure all required
|
||||
credentials are valid.
|
||||
"""
|
||||
locale = get_locale_from_request(http_request)
|
||||
manager = get_credential_manager()
|
||||
invalid_services = []
|
||||
|
||||
|
||||
if request.youtube:
|
||||
result = manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append("youtube")
|
||||
|
||||
|
||||
if request.gdrive:
|
||||
result = manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append("gdrive")
|
||||
|
||||
|
||||
if request.dropbox:
|
||||
result = manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append("dropbox")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
return CredentialValidationResponse(
|
||||
valid=False,
|
||||
invalid_services=invalid_services,
|
||||
message=t(locale, "auth.invalidCredentials", services=", ".join(invalid_services))
|
||||
message=f"The following services need re-authorization: {', '.join(invalid_services)}"
|
||||
)
|
||||
|
||||
|
||||
return CredentialValidationResponse(
|
||||
valid=True,
|
||||
invalid_services=[],
|
||||
message=t(locale, "auth.validCredentials")
|
||||
message="All requested credentials are valid"
|
||||
)
|
||||
|
||||
|
||||
@@ -200,45 +197,44 @@ def validate_credentials(request: CredentialValidationRequest, http_request: Req
|
||||
# ===========================================================================
|
||||
|
||||
@router.post("/youtube/device", response_model=DeviceAuthStartResponse)
|
||||
def start_youtube_device_auth(request: Optional[DeviceAuthStartRequest] = None, http_request: Request = None):
|
||||
async def start_youtube_device_auth(request: Optional[DeviceAuthStartRequest] = None):
|
||||
"""
|
||||
Start YouTube device authorization flow.
|
||||
|
||||
|
||||
This initiates a device auth flow that allows authorization
|
||||
from any device. The user must visit the verification URL
|
||||
and enter the user code.
|
||||
|
||||
|
||||
Client credentials are loaded from Secret Manager ('youtube-client-credentials')
|
||||
unless explicitly provided in the request body.
|
||||
|
||||
|
||||
After starting, poll /auth/youtube/device/{device_code} to
|
||||
check for completion.
|
||||
"""
|
||||
locale = get_locale_from_request(http_request) if http_request else "en"
|
||||
manager = get_credential_manager()
|
||||
|
||||
|
||||
try:
|
||||
device_info = manager.start_youtube_device_auth(
|
||||
client_id=request.client_id if request else None,
|
||||
client_secret=request.client_secret if request else None
|
||||
)
|
||||
|
||||
|
||||
return DeviceAuthStartResponse(
|
||||
device_code=device_info.device_code,
|
||||
user_code=device_info.user_code,
|
||||
verification_url=device_info.verification_url,
|
||||
expires_in=device_info.expires_in,
|
||||
interval=device_info.interval,
|
||||
instructions=t(locale, "auth.youtubeAuthInstructions", verification_url=device_info.verification_url, user_code=device_info.user_code)
|
||||
instructions=f"Visit {device_info.verification_url} and enter code: {device_info.user_code}"
|
||||
)
|
||||
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start YouTube device auth: {e}")
|
||||
raise HTTPException(status_code=500, detail=t(locale, "auth.youtubeAuthStartFailed", error=str(e)))
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
|
||||
@router.get("/youtube/device/{device_code}", response_model=DeviceAuthPollResponse)
|
||||
def poll_youtube_device_auth(device_code: str):
|
||||
async def poll_youtube_device_auth(device_code: str):
|
||||
"""
|
||||
Poll for YouTube device authorization completion.
|
||||
|
||||
@@ -263,40 +259,39 @@ def poll_youtube_device_auth(device_code: str):
|
||||
|
||||
|
||||
@router.post("/gdrive/device", response_model=DeviceAuthStartResponse)
|
||||
def start_gdrive_device_auth(request: Optional[DeviceAuthStartRequest] = None, http_request: Request = None):
|
||||
async def start_gdrive_device_auth(request: Optional[DeviceAuthStartRequest] = None):
|
||||
"""
|
||||
Start Google Drive device authorization flow.
|
||||
|
||||
|
||||
Client credentials are loaded from Secret Manager ('gdrive-client-credentials')
|
||||
unless explicitly provided in the request body.
|
||||
|
||||
|
||||
After starting, poll /auth/gdrive/device/{device_code} to check for completion.
|
||||
"""
|
||||
locale = get_locale_from_request(http_request) if http_request else "en"
|
||||
manager = get_credential_manager()
|
||||
|
||||
|
||||
try:
|
||||
device_info = manager.start_gdrive_device_auth(
|
||||
client_id=request.client_id if request else None,
|
||||
client_secret=request.client_secret if request else None
|
||||
)
|
||||
|
||||
|
||||
return DeviceAuthStartResponse(
|
||||
device_code=device_info.device_code,
|
||||
user_code=device_info.user_code,
|
||||
verification_url=device_info.verification_url,
|
||||
expires_in=device_info.expires_in,
|
||||
interval=device_info.interval,
|
||||
instructions=t(locale, "auth.gdriveLinkInstructions", verification_url=device_info.verification_url, user_code=device_info.user_code)
|
||||
instructions=f"Visit {device_info.verification_url} and enter code: {device_info.user_code}"
|
||||
)
|
||||
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start Google Drive device auth: {e}")
|
||||
raise HTTPException(status_code=500, detail=t(locale, "auth.gdriveAuthStartFailed", error=str(e)))
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
|
||||
@router.get("/gdrive/device/{device_code}", response_model=DeviceAuthPollResponse)
|
||||
def poll_gdrive_device_auth(device_code: str):
|
||||
async def poll_gdrive_device_auth(device_code: str):
|
||||
"""
|
||||
Poll for Google Drive device authorization completion.
|
||||
"""
|
||||
@@ -316,20 +311,19 @@ def poll_gdrive_device_auth(device_code: str):
|
||||
# ===========================================================================
|
||||
|
||||
@router.post("/test-alert")
|
||||
def test_credential_alert(background_tasks: BackgroundTasks, request: Request):
|
||||
async def test_credential_alert(background_tasks: BackgroundTasks):
|
||||
"""
|
||||
Test the credential alert mechanism.
|
||||
|
||||
|
||||
Sends a test alert to Discord if configured.
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
settings = get_settings()
|
||||
discord_url = settings.get_secret("discord-alert-webhook")
|
||||
|
||||
|
||||
if not discord_url:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=t(locale, "auth.discordAlertNotConfigured")
|
||||
detail="Discord alert webhook not configured"
|
||||
)
|
||||
|
||||
manager = get_credential_manager()
|
||||
|
||||
@@ -1,528 +0,0 @@
|
||||
"""Bulk Mode API routes.
|
||||
|
||||
Lets a user submit up to 100 karaoke jobs at once, by text (artist/title rows)
|
||||
or by album (MusicBrainz release lookup). Album tracklists and text rows are
|
||||
enriched with KaraokeNerds community-version availability so the user can skip
|
||||
tracks that already exist.
|
||||
|
||||
Submission (POST /bulk/submit) and progress (GET /bulk/{batch_id}) live here too
|
||||
(added in later phases).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Literal, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from backend.api.dependencies import require_auth
|
||||
from backend.i18n import get_full_locale_from_request
|
||||
from backend.services.auth_service import AuthResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Strong refs to shielded tasks so they aren't garbage-collected if the request is cancelled.
|
||||
_BACKGROUND_TASKS: set = set()
|
||||
|
||||
router = APIRouter(prefix="/bulk", tags=["bulk"])
|
||||
|
||||
MAX_BULK_SONGS = 100
|
||||
|
||||
|
||||
# --- Models -----------------------------------------------------------------
|
||||
|
||||
|
||||
class BulkArtist(BaseModel):
|
||||
mbid: Optional[str] = None
|
||||
name: str
|
||||
disambiguation: str = ""
|
||||
type: Optional[str] = None
|
||||
country: Optional[str] = None
|
||||
# Set when only a MusicBrainz alias matched the query (e.g. "Hikaru Utada").
|
||||
alias: Optional[str] = None
|
||||
|
||||
|
||||
class BulkAlbum(BaseModel):
|
||||
release_group_mbid: str
|
||||
title: str
|
||||
primary_type: Optional[str] = None
|
||||
secondary_types: list[str] = []
|
||||
first_release_date: str = ""
|
||||
is_studio: bool = True
|
||||
|
||||
|
||||
class BulkEdition(BaseModel):
|
||||
release_mbid: str
|
||||
title: Optional[str] = None
|
||||
status: Optional[str] = None
|
||||
date: str = ""
|
||||
country: Optional[str] = None
|
||||
track_count: int = 0
|
||||
|
||||
|
||||
class BulkEditionVariant(BaseModel):
|
||||
"""A distinct tracklist variant (country pressings collapsed by track count)."""
|
||||
|
||||
representative_release_mbid: str
|
||||
label: Literal["Original", "Reissue"] = "Original"
|
||||
track_count: int = 0
|
||||
year: str = ""
|
||||
pressing_count: int = 0
|
||||
delta_vs_original: int = 0
|
||||
|
||||
|
||||
class CommunityVersion(BaseModel):
|
||||
"""An existing community karaoke version the user can preview on YouTube."""
|
||||
|
||||
brand: str
|
||||
url: str
|
||||
|
||||
|
||||
class BulkTrack(BaseModel):
|
||||
position: Optional[int] = None
|
||||
title: str
|
||||
recording_mbid: Optional[str] = None
|
||||
length_ms: Optional[int] = None
|
||||
is_extra: bool = False
|
||||
extra_reason: str = ""
|
||||
# Availability (filled from KaraokeNerds); available=True => already exists
|
||||
available: bool = False
|
||||
brands: list[str] = []
|
||||
versions: list[CommunityVersion] = []
|
||||
|
||||
|
||||
class BulkTracklistResponse(BaseModel):
|
||||
release_mbid: Optional[str] = None
|
||||
canonical_release_mbid: Optional[str] = None
|
||||
selected_variant_mbid: Optional[str] = None
|
||||
title: Optional[str] = None
|
||||
date: str = ""
|
||||
tracks: list[BulkTrack] = []
|
||||
editions: list[BulkEdition] = []
|
||||
variants: list[BulkEditionVariant] = []
|
||||
|
||||
|
||||
class AvailabilityItem(BaseModel):
|
||||
artist: str = Field(..., min_length=1)
|
||||
title: str = Field(..., min_length=1)
|
||||
|
||||
|
||||
class AvailabilityRequest(BaseModel):
|
||||
tracks: list[AvailabilityItem]
|
||||
|
||||
|
||||
class AvailabilityResult(BaseModel):
|
||||
artist: str
|
||||
title: str
|
||||
available: bool
|
||||
brands: list[str] = []
|
||||
brand_count: int = 0
|
||||
versions: list[CommunityVersion] = []
|
||||
|
||||
|
||||
class AvailabilityResponse(BaseModel):
|
||||
results: list[AvailabilityResult]
|
||||
|
||||
|
||||
# --- Album lookup routes ----------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/album/artists", response_model=list[BulkArtist])
|
||||
async def album_artists(
|
||||
request: Request,
|
||||
q: str = Query(..., min_length=2, description="Artist name search"),
|
||||
limit: int = Query(8, ge=1, le=25),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Search MusicBrainz for artists (album-mode step 1)."""
|
||||
from backend.services.musicbrainz_service import get_musicbrainz_service
|
||||
|
||||
try:
|
||||
results = await get_musicbrainz_service().search_artists(q, limit=limit)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("MusicBrainz artist search failed: %r", e, exc_info=True)
|
||||
raise HTTPException(status_code=502, detail="Artist lookup is temporarily unavailable")
|
||||
return results
|
||||
|
||||
|
||||
@router.get("/album/albums", response_model=list[BulkAlbum])
|
||||
async def album_albums(
|
||||
request: Request,
|
||||
artist_mbid: str = Query(..., min_length=1),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""List an artist's albums (release-groups) for the album picker."""
|
||||
from backend.services.musicbrainz_service import get_musicbrainz_service
|
||||
|
||||
try:
|
||||
return await get_musicbrainz_service().get_albums(artist_mbid)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("MusicBrainz album list failed: %r", e, exc_info=True)
|
||||
raise HTTPException(status_code=502, detail="Album lookup is temporarily unavailable")
|
||||
|
||||
|
||||
@router.get("/album/tracklist", response_model=BulkTracklistResponse)
|
||||
async def album_tracklist(
|
||||
request: Request,
|
||||
artist: str = Query(..., min_length=1, description="Artist name for availability checks"),
|
||||
release_group_mbid: Optional[str] = Query(None),
|
||||
release_mbid: Optional[str] = Query(None, description="Specific edition to load"),
|
||||
locale: Optional[str] = Query(None, description="UI locale → biases representative-pressing choice"),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Resolve an album's canonical tracklist (or a specific edition) and enrich
|
||||
each track with KaraokeNerds community-version availability.
|
||||
|
||||
Response shape note: ``variants`` is only populated on the ``release_group_mbid``
|
||||
path (the initial album load). When loading a specific edition via
|
||||
``release_mbid`` it is intentionally ``[]`` — the client already holds the
|
||||
variant list from the initial load and only needs the chosen edition's tracks.
|
||||
"""
|
||||
from backend.services.musicbrainz_service import (
|
||||
get_musicbrainz_service,
|
||||
country_prefs_for_locale,
|
||||
)
|
||||
from backend.services.karaokenerds_service import check_community_versions_batch
|
||||
|
||||
if not release_group_mbid and not release_mbid:
|
||||
raise HTTPException(status_code=422, detail="release_group_mbid or release_mbid required")
|
||||
|
||||
mb = get_musicbrainz_service()
|
||||
try:
|
||||
if release_mbid:
|
||||
# Loading a specific variant — variants already known to the client.
|
||||
tracklist = await mb.get_release_tracklist(release_mbid, release_group_mbid)
|
||||
tracklist.setdefault("editions", [])
|
||||
tracklist.setdefault("canonical_release_mbid", release_mbid)
|
||||
tracklist.setdefault("variants", [])
|
||||
tracklist.setdefault("selected_variant_mbid", release_mbid)
|
||||
else:
|
||||
tracklist = await mb.get_album_tracklist(
|
||||
release_group_mbid, preferred_countries=country_prefs_for_locale(locale)
|
||||
)
|
||||
except LookupError:
|
||||
# Not in our mirror (or not part of that album) — retrying won't help.
|
||||
raise HTTPException(status_code=404, detail="Release not found")
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("MusicBrainz tracklist failed: %r", e, exc_info=True)
|
||||
raise HTTPException(status_code=502, detail="Tracklist lookup is temporarily unavailable")
|
||||
|
||||
tracks = tracklist.get("tracks", [])
|
||||
# Availability enrichment — never fatal.
|
||||
try:
|
||||
avail = await check_community_versions_batch(
|
||||
[{"artist": artist, "title": t["title"]} for t in tracks]
|
||||
)
|
||||
for t, a in zip(tracks, avail):
|
||||
t["available"] = a["available"]
|
||||
t["brands"] = a["brands"]
|
||||
t["versions"] = a.get("versions", [])
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("Availability enrichment failed (non-fatal): %s", e)
|
||||
|
||||
return BulkTracklistResponse(
|
||||
release_mbid=tracklist.get("release_mbid"),
|
||||
canonical_release_mbid=tracklist.get("canonical_release_mbid"),
|
||||
selected_variant_mbid=tracklist.get("selected_variant_mbid"),
|
||||
title=tracklist.get("title"),
|
||||
date=tracklist.get("date", ""),
|
||||
tracks=[BulkTrack(**t) for t in tracks],
|
||||
editions=[BulkEdition(**e) for e in tracklist.get("editions", [])],
|
||||
variants=[BulkEditionVariant(**v) for v in tracklist.get("variants", [])],
|
||||
)
|
||||
|
||||
|
||||
@router.post("/availability", response_model=AvailabilityResponse)
|
||||
async def bulk_availability(
|
||||
request: Request,
|
||||
body: AvailabilityRequest,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Batch KaraokeNerds availability check for text-mode rows."""
|
||||
from backend.services.karaokenerds_service import check_community_versions_batch
|
||||
|
||||
if len(body.tracks) > MAX_BULK_SONGS:
|
||||
raise HTTPException(status_code=422, detail=f"At most {MAX_BULK_SONGS} tracks per request")
|
||||
|
||||
results = await check_community_versions_batch(
|
||||
[{"artist": t.artist, "title": t.title} for t in body.tracks]
|
||||
)
|
||||
return AvailabilityResponse(results=[AvailabilityResult(**r) for r in results])
|
||||
|
||||
|
||||
# --- Submission -------------------------------------------------------------
|
||||
|
||||
|
||||
class BulkSong(BaseModel):
|
||||
artist: str = Field(..., min_length=1)
|
||||
title: str = Field(..., min_length=1)
|
||||
display_artist: Optional[str] = None
|
||||
display_title: Optional[str] = None
|
||||
# Album mode: the track's number on its disc (album-search tie-break).
|
||||
position: Optional[int] = None
|
||||
|
||||
|
||||
class BulkAlbumRef(BaseModel):
|
||||
"""The album an album-mode batch was picked from. Lets the search worker find
|
||||
the album's torrent once instead of searching every track separately."""
|
||||
artist: str = Field(..., min_length=1)
|
||||
title: str = Field(..., min_length=1)
|
||||
release_group_mbid: Optional[str] = None
|
||||
release_mbid: Optional[str] = None
|
||||
|
||||
|
||||
class BulkSettings(BaseModel):
|
||||
auto_select_if_lossless: bool = True
|
||||
is_private: bool = False
|
||||
skip_audio_edit: bool = True
|
||||
skip_customization: bool = True
|
||||
# Full-auto review (workstream C): batch default autonomy + backing preference.
|
||||
review_mode: str = "auto"
|
||||
backing_preference: str = "auto"
|
||||
translation_language: Optional[str] = None
|
||||
|
||||
|
||||
class BulkSubmitRequest(BaseModel):
|
||||
songs: list[BulkSong]
|
||||
settings: BulkSettings = Field(default_factory=BulkSettings)
|
||||
album: Optional[BulkAlbumRef] = None
|
||||
|
||||
|
||||
class BulkSubmitResponse(BaseModel):
|
||||
batch_id: str
|
||||
job_ids: list[str]
|
||||
total: int
|
||||
|
||||
|
||||
@router.post("/submit", response_model=BulkSubmitResponse)
|
||||
async def bulk_submit(
|
||||
request: Request,
|
||||
body: BulkSubmitRequest,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Create one job per song, gate on credits upfront, and dispatch the batch
|
||||
search worker. Jobs start parked (auto_download=False) and the worker either
|
||||
auto-selects a confident lossless match or leaves them awaiting selection."""
|
||||
from backend.config import get_settings
|
||||
from backend.exceptions import InsufficientCreditsError
|
||||
from backend.models.job import JobCreate
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.job_defaults_service import resolve_cdg_txt_defaults
|
||||
from backend.services.theme_service import get_theme_service
|
||||
from backend.services.worker_service import get_worker_service
|
||||
|
||||
n = len(body.songs)
|
||||
if n == 0:
|
||||
raise HTTPException(status_code=422, detail="At least one song is required")
|
||||
if n > MAX_BULK_SONGS:
|
||||
raise HTTPException(status_code=422, detail=f"At most {MAX_BULK_SONGS} songs per batch")
|
||||
|
||||
user_email = auth_result.user_email
|
||||
is_admin = bool(auth_result.is_admin)
|
||||
|
||||
locale = get_full_locale_from_request(request)
|
||||
|
||||
# Everything up to dispatch is sync Firestore/GCS work (credit check, default
|
||||
# theme, one create_job + update_job per song). Run it off the event loop: a
|
||||
# 2026-10-22 album batch stalled the loop 26s and 9 other requests with it.
|
||||
def _create_batch_jobs() -> tuple[str, list[str]]:
|
||||
# Upfront credit gate (1 credit/song minimum). Admins bypass.
|
||||
if not is_admin and user_email:
|
||||
from backend.services.user_service import get_user_service
|
||||
|
||||
available = get_user_service().check_credits(user_email)
|
||||
if available < n:
|
||||
raise HTTPException(
|
||||
status_code=402,
|
||||
detail={
|
||||
"error": "insufficient_credits",
|
||||
"message": "You don't have enough credits for this batch.",
|
||||
"credits_available": available,
|
||||
"credits_required": n,
|
||||
},
|
||||
)
|
||||
|
||||
settings = get_settings()
|
||||
theme_service = get_theme_service()
|
||||
theme_id = theme_service.get_default_theme_id()
|
||||
resolved_cdg, resolved_txt = resolve_cdg_txt_defaults(theme_id, None, None)
|
||||
|
||||
batch_id = uuid.uuid4().hex[:12]
|
||||
bulk_settings = body.settings.model_dump()
|
||||
bulk_album = body.album.model_dump() if body.album else None
|
||||
job_manager = JobManager()
|
||||
job_ids: list[str] = []
|
||||
|
||||
for song in body.songs:
|
||||
display_artist = (song.display_artist or song.artist).strip()
|
||||
display_title = (song.display_title or song.title).strip()
|
||||
request_metadata = {
|
||||
"created_from": "bulk",
|
||||
"batch_id": batch_id,
|
||||
"user_email": user_email,
|
||||
}
|
||||
job_create = JobCreate(
|
||||
artist=display_artist,
|
||||
title=display_title,
|
||||
theme_id=theme_id,
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
user_email=user_email,
|
||||
audio_search_artist=song.artist.strip(),
|
||||
audio_search_title=song.title.strip(),
|
||||
auto_download=False,
|
||||
is_private=body.settings.is_private,
|
||||
review_mode=body.settings.review_mode,
|
||||
backing_preference=body.settings.backing_preference,
|
||||
translation_language=body.settings.translation_language,
|
||||
brand_prefix=settings.default_brand_prefix,
|
||||
enable_youtube_upload=settings.default_enable_youtube_upload,
|
||||
youtube_description=settings.default_youtube_description,
|
||||
discord_webhook_url=settings.default_discord_webhook_url,
|
||||
dropbox_path=settings.default_dropbox_path,
|
||||
gdrive_folder_id=settings.default_gdrive_folder_id,
|
||||
request_metadata=request_metadata,
|
||||
locale=locale,
|
||||
)
|
||||
try:
|
||||
job = job_manager.create_job(job_create, is_admin=is_admin)
|
||||
except InsufficientCreditsError:
|
||||
# Ran out mid-batch (e.g. concurrent spend after the gate). Stop here;
|
||||
# already-created jobs are valid and will proceed.
|
||||
logger.warning("[batch:%s] ran out of credits after %d/%d jobs", batch_id, len(job_ids), n)
|
||||
break
|
||||
|
||||
bulk_state = {
|
||||
"state_data.batch_id": batch_id,
|
||||
"state_data.bulk_settings": bulk_settings,
|
||||
"state_data.bulk_pending_search": True,
|
||||
}
|
||||
if bulk_album is not None:
|
||||
bulk_state["state_data.bulk_album"] = bulk_album
|
||||
bulk_state["state_data.bulk_track_position"] = song.position
|
||||
job_manager.update_job(job.job_id, bulk_state)
|
||||
job_ids.append(job.job_id)
|
||||
return batch_id, job_ids
|
||||
|
||||
# Dispatch the batch search worker. Retry once on transient failure. If it
|
||||
# still fails the jobs exist (and are charged) but won't process, so surface
|
||||
# an explicit error with the batch_id — and tell the user NOT to resubmit
|
||||
# (that would create+charge a second batch); the batch can be re-dispatched.
|
||||
worker_service = get_worker_service()
|
||||
|
||||
async def _create_and_dispatch() -> tuple[str, list[str], bool]:
|
||||
batch_id, job_ids = await asyncio.to_thread(_create_batch_jobs)
|
||||
if not job_ids:
|
||||
return batch_id, job_ids, False
|
||||
triggered = await worker_service.trigger_bulk_search_worker(batch_id)
|
||||
if not triggered:
|
||||
triggered = await worker_service.trigger_bulk_search_worker(batch_id)
|
||||
return batch_id, job_ids, triggered
|
||||
|
||||
# Shielded: if the request is cancelled (client disconnect) mid-way, the jobs
|
||||
# being created and charged in the thread still get dispatched.
|
||||
task = asyncio.create_task(_create_and_dispatch())
|
||||
_BACKGROUND_TASKS.add(task)
|
||||
task.add_done_callback(_BACKGROUND_TASKS.discard)
|
||||
batch_id, job_ids, triggered = await asyncio.shield(task)
|
||||
|
||||
if not job_ids:
|
||||
raise HTTPException(status_code=402, detail="No jobs could be created")
|
||||
if not triggered:
|
||||
logger.critical(
|
||||
"[batch:%s] created %d jobs but FAILED to dispatch bulk-search worker",
|
||||
batch_id, len(job_ids),
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail={
|
||||
"error": "dispatch_failed",
|
||||
"message": "Your batch was created but processing couldn't start. "
|
||||
"Please don't resubmit — contact support with this batch ID and we'll start it.",
|
||||
"batch_id": batch_id,
|
||||
"job_ids": job_ids,
|
||||
},
|
||||
)
|
||||
logger.info("[batch:%s] created %d jobs, dispatched bulk-search worker", batch_id, len(job_ids))
|
||||
|
||||
return BulkSubmitResponse(batch_id=batch_id, job_ids=job_ids, total=len(job_ids))
|
||||
|
||||
|
||||
# --- Progress ---------------------------------------------------------------
|
||||
|
||||
|
||||
class BulkJobStatus(BaseModel):
|
||||
job_id: str
|
||||
artist: str
|
||||
title: str
|
||||
status: str
|
||||
auto_selected: bool = False
|
||||
|
||||
|
||||
class BulkBatchResponse(BaseModel):
|
||||
batch_id: str
|
||||
total: int
|
||||
counts: dict
|
||||
jobs: list[BulkJobStatus]
|
||||
|
||||
|
||||
# Status -> progress bucket for the batch summary.
|
||||
_AWAITING = "awaiting_audio_selection"
|
||||
_TERMINAL_OK = "complete"
|
||||
_TERMINAL_FAIL = "failed"
|
||||
_SEARCHING = {"pending", "searching_audio"}
|
||||
|
||||
|
||||
def _bucket(status: str, pending_search: bool) -> str:
|
||||
if pending_search or status in _SEARCHING:
|
||||
return "searching"
|
||||
if status == _AWAITING:
|
||||
return "awaiting_selection"
|
||||
if status == _TERMINAL_OK:
|
||||
return "complete"
|
||||
if status == _TERMINAL_FAIL:
|
||||
return "failed"
|
||||
return "processing"
|
||||
|
||||
|
||||
@router.get("/{batch_id}", response_model=BulkBatchResponse)
|
||||
def bulk_batch(
|
||||
request: Request,
|
||||
batch_id: str,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Progress summary for a batch: counts by bucket + per-job status."""
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
job_manager = JobManager()
|
||||
jobs = job_manager.list_jobs_by_batch_id(batch_id)
|
||||
|
||||
if not bool(auth_result.is_admin):
|
||||
email = (auth_result.user_email or "").lower()
|
||||
jobs = [j for j in jobs if (j.user_email or "").lower() == email]
|
||||
|
||||
if not jobs:
|
||||
raise HTTPException(status_code=404, detail="Batch not found")
|
||||
|
||||
counts = {"searching": 0, "awaiting_selection": 0, "processing": 0, "complete": 0, "failed": 0}
|
||||
job_statuses: list[BulkJobStatus] = []
|
||||
for j in jobs:
|
||||
status = j.status.value if hasattr(j.status, "value") else str(j.status)
|
||||
sd = j.state_data or {}
|
||||
bucket = _bucket(status, bool(sd.get("bulk_pending_search")))
|
||||
counts[bucket] += 1
|
||||
job_statuses.append(
|
||||
BulkJobStatus(
|
||||
job_id=j.job_id,
|
||||
artist=j.audio_search_artist or j.artist or "",
|
||||
title=j.audio_search_title or j.title or "",
|
||||
status=status,
|
||||
auto_selected=bool(sd.get("bulk_auto_selected")),
|
||||
)
|
||||
)
|
||||
|
||||
return BulkBatchResponse(
|
||||
batch_id=batch_id, total=len(jobs), counts=counts, jobs=job_statuses
|
||||
)
|
||||
@@ -1,229 +0,0 @@
|
||||
"""
|
||||
Catalog API routes for song/artist lookup and community version detection.
|
||||
|
||||
Provides autocomplete data from karaoke-decide's MusicBrainz + Spotify catalog,
|
||||
and checks karaokenerds.com for existing community karaoke versions.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
from typing import Literal, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from backend.api.dependencies import require_auth
|
||||
from backend.services.auth_service import AuthResult
|
||||
from backend.i18n import t, get_locale_from_request
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/catalog", tags=["catalog"])
|
||||
|
||||
# Simple per-user rate limiting: 20 requests/minute
|
||||
# { user_email: [timestamp, timestamp, ...] }
|
||||
_rate_limit_windows: dict[str, list[float]] = {}
|
||||
RATE_LIMIT_MAX_REQUESTS = 20
|
||||
RATE_LIMIT_WINDOW_SECONDS = 60
|
||||
|
||||
|
||||
def _check_rate_limit(user_email: str, locale: str = "en") -> None:
|
||||
"""Check per-user rate limit. Raises 429 if exceeded."""
|
||||
now = time.monotonic()
|
||||
window_start = now - RATE_LIMIT_WINDOW_SECONDS
|
||||
|
||||
# Get or create window for this user
|
||||
if user_email not in _rate_limit_windows:
|
||||
_rate_limit_windows[user_email] = []
|
||||
|
||||
# Remove expired entries
|
||||
timestamps = _rate_limit_windows[user_email]
|
||||
_rate_limit_windows[user_email] = [ts for ts in timestamps if ts > window_start]
|
||||
|
||||
if len(_rate_limit_windows[user_email]) >= RATE_LIMIT_MAX_REQUESTS:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail=t(locale, "catalog.rateLimitExceeded", max_requests=RATE_LIMIT_MAX_REQUESTS),
|
||||
)
|
||||
|
||||
_rate_limit_windows[user_email].append(now)
|
||||
|
||||
|
||||
# --- Response Models ---
|
||||
|
||||
|
||||
class ArtistResult(BaseModel):
|
||||
"""Artist search result from MusicBrainz/Spotify catalog."""
|
||||
name: str
|
||||
mbid: Optional[str] = None
|
||||
disambiguation: Optional[str] = None
|
||||
artist_type: Optional[str] = None
|
||||
spotify_id: Optional[str] = None
|
||||
popularity: Optional[int] = None
|
||||
genres: Optional[list[str]] = None
|
||||
tags: Optional[list[str]] = None
|
||||
|
||||
|
||||
class TrackResult(BaseModel):
|
||||
"""Track search result from Spotify catalog."""
|
||||
track_name: str
|
||||
artist_name: str
|
||||
track_id: Optional[str] = None
|
||||
artist_id: Optional[str] = None
|
||||
popularity: Optional[int] = None
|
||||
duration_ms: Optional[int] = None
|
||||
explicit: Optional[bool] = None
|
||||
|
||||
|
||||
class CommunityTrack(BaseModel):
|
||||
"""A community-approved karaoke track from karaokenerds."""
|
||||
brand_name: str
|
||||
brand_code: str
|
||||
youtube_url: str
|
||||
is_community: bool = True
|
||||
|
||||
|
||||
class CommunityCheckSong(BaseModel):
|
||||
"""A song with community karaoke tracks."""
|
||||
title: str
|
||||
artist: str
|
||||
community_tracks: list[CommunityTrack]
|
||||
|
||||
|
||||
class CommunityCheckRequest(BaseModel):
|
||||
"""Request body for community version check."""
|
||||
artist: str = Field(..., min_length=1)
|
||||
title: str = Field(..., min_length=1)
|
||||
|
||||
|
||||
class CommunityCheckResponse(BaseModel):
|
||||
"""Response for community version check."""
|
||||
has_community: bool
|
||||
songs: list[CommunityCheckSong]
|
||||
best_youtube_url: Optional[str] = None
|
||||
|
||||
|
||||
class MatchJudgeRequest(BaseModel):
|
||||
"""Request body for the artist/title match judge."""
|
||||
artist: str = Field(..., min_length=1)
|
||||
title: str = Field(..., min_length=1)
|
||||
# The audio search's confidence tier (1=strong..3=weak), so the judge can
|
||||
# tell whether weak results hint at a typo. Optional.
|
||||
audio_confidence_tier: Optional[int] = Field(None, ge=1, le=3)
|
||||
# "fast" = catalog-only pass (no AI, no tier needed) the client fires in
|
||||
# parallel with the audio search; "full" = complete pipeline once the tier is
|
||||
# known. Defaults to "full" so a single legacy call still works.
|
||||
stage: Literal["fast", "full"] = "full"
|
||||
|
||||
|
||||
class MatchJudgeAlternative(BaseModel):
|
||||
artist: str
|
||||
title: str
|
||||
|
||||
|
||||
class MatchJudgeResponse(BaseModel):
|
||||
"""Verdict on a typed artist/title — formatting + match acceptance."""
|
||||
kind: str # "cosmetic" | "content" | "ambiguous" | "none"
|
||||
confident: bool
|
||||
canonical_artist: str
|
||||
canonical_title: str
|
||||
alternatives: list[MatchJudgeAlternative] = []
|
||||
engine: str # "deterministic" | "catalog" | "ai"
|
||||
reason: str = ""
|
||||
# Set by a "fast" pass that couldn't decide — the client should follow up with
|
||||
# a "full" call once the audio tier is known.
|
||||
needs_ai: bool = False
|
||||
|
||||
|
||||
# --- Routes ---
|
||||
|
||||
|
||||
@router.get("/artists", response_model=list[ArtistResult])
|
||||
async def search_artists(
|
||||
request: Request,
|
||||
q: str = Query(..., min_length=2, description="Artist name search query"),
|
||||
limit: int = Query(10, ge=1, le=50, description="Maximum results"),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""
|
||||
Search for artists by name. Returns canonical MusicBrainz/Spotify artist data
|
||||
for autocomplete in job creation forms.
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
_check_rate_limit(auth_result.user_email or auth_result.token_id or "unknown", locale)
|
||||
|
||||
from backend.services.catalog_proxy_service import search_artists
|
||||
results = await search_artists(q, limit)
|
||||
return results
|
||||
|
||||
|
||||
@router.get("/tracks", response_model=list[TrackResult])
|
||||
async def search_tracks(
|
||||
request: Request,
|
||||
q: str = Query(..., min_length=2, description="Track name search query"),
|
||||
artist: Optional[str] = Query(None, description="Filter by artist name"),
|
||||
limit: int = Query(10, ge=1, le=50, description="Maximum results"),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""
|
||||
Search for tracks by name, optionally filtered by artist.
|
||||
Returns canonical Spotify track data for autocomplete in job creation forms.
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
_check_rate_limit(auth_result.user_email or auth_result.token_id or "unknown", locale)
|
||||
|
||||
from backend.services.catalog_proxy_service import search_tracks
|
||||
results = await search_tracks(q, artist, limit)
|
||||
return results
|
||||
|
||||
|
||||
@router.post("/community-check", response_model=CommunityCheckResponse)
|
||||
async def check_community_versions(
|
||||
request: Request,
|
||||
body: CommunityCheckRequest,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""
|
||||
Check if a song already has community-approved karaoke versions on YouTube
|
||||
via karaokenerds.com. Non-blocking — just informs the user.
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
_check_rate_limit(auth_result.user_email or auth_result.token_id or "unknown", locale)
|
||||
|
||||
from backend.services.karaokenerds_service import check_community_versions
|
||||
result = await check_community_versions(body.artist, body.title)
|
||||
return result
|
||||
|
||||
|
||||
@router.post("/match-judge", response_model=MatchJudgeResponse)
|
||||
async def match_judge(
|
||||
request: Request,
|
||||
body: MatchJudgeRequest,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""
|
||||
Judge a typed artist/title: decide official formatting + whether it matches a
|
||||
real song. Deterministic + catalog layers run first; a light AI model fires
|
||||
only when those aren't confident. Never blocks the submission flow — failures
|
||||
degrade to a "none" verdict.
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
_check_rate_limit(auth_result.user_email or auth_result.token_id or "unknown", locale)
|
||||
|
||||
from backend.services.fake_workers import fake_workers_enabled
|
||||
if fake_workers_enabled():
|
||||
# Local E2E stack: scripted verdict (per-user fake config), no catalog/Gemini.
|
||||
from backend.services.fake_workers.judge import fake_match_verdict
|
||||
return fake_match_verdict(body.artist, body.title, auth_result.user_email, stage=body.stage).to_dict()
|
||||
|
||||
from backend.services.match_judge.service import judge_match
|
||||
verdict = await judge_match(
|
||||
body.artist, body.title,
|
||||
audio_tier=body.audio_confidence_tier,
|
||||
stage=body.stage,
|
||||
)
|
||||
logger.info(
|
||||
"match-judge stage=%s kind=%s engine=%s confident=%s needs_ai=%s",
|
||||
body.stage, verdict.kind, verdict.engine, verdict.confident, verdict.needs_ai,
|
||||
)
|
||||
return verdict.to_dict()
|
||||
@@ -1,136 +0,0 @@
|
||||
"""POST /api/client-errors — ingest frontend crash reports.
|
||||
|
||||
Unauthenticated (logged-out users' crashes still matter). Deduped and alerted
|
||||
via the existing error-monitor pipeline — see
|
||||
``backend/services/error_monitor``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Request, HTTPException
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from backend.services.error_monitor.firestore_adapter import ErrorPatternsAdapter
|
||||
from backend.services.error_monitor.frontend_ingestion import (
|
||||
FrontendErrorReport,
|
||||
RateLimiter,
|
||||
build_pattern_data,
|
||||
is_bot_user_agent,
|
||||
is_host_app_injection_error,
|
||||
is_opaque_script_error,
|
||||
sanitize_url,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/client-errors", tags=["client-errors"])
|
||||
|
||||
_limiter = RateLimiter(max_per_minute=60)
|
||||
_adapter_singleton: ErrorPatternsAdapter | None = None
|
||||
|
||||
|
||||
def _get_adapter() -> ErrorPatternsAdapter:
|
||||
global _adapter_singleton
|
||||
if _adapter_singleton is None:
|
||||
_adapter_singleton = ErrorPatternsAdapter()
|
||||
return _adapter_singleton
|
||||
|
||||
|
||||
class ClientErrorPayload(BaseModel):
|
||||
message: str = Field(..., min_length=1, max_length=4000)
|
||||
stack: Optional[str] = Field(None, max_length=1_000_000)
|
||||
url: str = Field("", max_length=2048)
|
||||
user_agent: str = Field("", max_length=1024)
|
||||
release: str = Field("", max_length=64)
|
||||
user_email: Optional[str] = Field(None, max_length=320)
|
||||
viewport: Optional[dict] = None
|
||||
locale: str = Field("en", max_length=10)
|
||||
source: str = Field("unknown", max_length=64)
|
||||
# optional free-form extra for debugging. Capped server-side.
|
||||
extra: Optional[dict] = None
|
||||
|
||||
@field_validator("message", "url", "user_agent", "locale", "source")
|
||||
@classmethod
|
||||
def strip(cls, v: str) -> str:
|
||||
return (v or "").strip()
|
||||
|
||||
|
||||
class ClientErrorResponse(BaseModel):
|
||||
pattern_id: str
|
||||
is_new: bool
|
||||
|
||||
|
||||
@router.post("", response_model=ClientErrorResponse, status_code=202)
|
||||
def report_client_error(payload: ClientErrorPayload, request: Request) -> ClientErrorResponse:
|
||||
client_ip = request.client.host if request.client else "unknown"
|
||||
if not _limiter.allow(client_ip, time.monotonic()):
|
||||
raise HTTPException(status_code=429, detail="too many reports")
|
||||
|
||||
# Crawler/bot crashes are pure alert noise (bots run headless Chrome and trip
|
||||
# DOM errors no real user hits). Accept the report so the client doesn't retry,
|
||||
# but don't persist/alert on it.
|
||||
if is_bot_user_agent(payload.user_agent):
|
||||
# Log the bot UA (not sensitive) + sanitized URL (query/token stripped)
|
||||
# for visibility into what's being dropped, without persisting/alerting.
|
||||
logger.info(
|
||||
"frontend_crash_ignored_bot ua=%r url=%s source=%s",
|
||||
payload.user_agent[:120],
|
||||
sanitize_url(payload.url),
|
||||
payload.source,
|
||||
)
|
||||
return ClientErrorResponse(pattern_id="bot-ignored", is_new=False)
|
||||
|
||||
# Opaque cross-origin "Script error." — no file/line, not our code.
|
||||
if is_opaque_script_error(payload.message, payload.source, payload.extra):
|
||||
logger.info(
|
||||
"frontend_crash_ignored_opaque_script_error url=%s source=%s",
|
||||
sanitize_url(payload.url),
|
||||
payload.source,
|
||||
)
|
||||
return ClientErrorResponse(pattern_id="opaque-script-error-ignored", is_new=False)
|
||||
|
||||
# A script the in-app browser's host app injected (Zalo), not our code.
|
||||
if is_host_app_injection_error(payload.message, payload.user_agent):
|
||||
logger.info(
|
||||
"frontend_crash_ignored_host_app_injection url=%s source=%s",
|
||||
sanitize_url(payload.url),
|
||||
payload.source,
|
||||
)
|
||||
return ClientErrorResponse(pattern_id="host-app-injection-ignored", is_new=False)
|
||||
|
||||
report = FrontendErrorReport(
|
||||
message=payload.message,
|
||||
stack=payload.stack,
|
||||
url=payload.url,
|
||||
user_agent=payload.user_agent,
|
||||
release=payload.release,
|
||||
user_email=payload.user_email,
|
||||
viewport=payload.viewport,
|
||||
locale=payload.locale,
|
||||
extra=payload.extra,
|
||||
)
|
||||
pattern_data = build_pattern_data(report)
|
||||
|
||||
try:
|
||||
result = _get_adapter().upsert_pattern(pattern_data)
|
||||
except Exception: # pragma: no cover - Firestore transient errors
|
||||
logger.exception("Failed to upsert frontend error pattern")
|
||||
raise HTTPException(status_code=503, detail="storage unavailable")
|
||||
|
||||
logger.info(
|
||||
"frontend_crash_reported pattern_id=%s is_new=%s source=%s",
|
||||
result.pattern_id,
|
||||
result.is_new,
|
||||
payload.source,
|
||||
# Full client diagnostics + breadcrumbs, searchable in Cloud Logging
|
||||
# (the alert sample only carries a trimmed summary).
|
||||
extra={
|
||||
"error_pattern_id": result.pattern_id,
|
||||
"client_url": sanitize_url(payload.url),
|
||||
"client_release": payload.release,
|
||||
"client_extra": payload.extra,
|
||||
},
|
||||
)
|
||||
return ClientErrorResponse(pattern_id=result.pattern_id, is_new=result.is_new)
|
||||
@@ -1,212 +0,0 @@
|
||||
"""POST /api/client-events — persistent telemetry for user-visible degradation.
|
||||
|
||||
Records every time the frontend shows a user a degraded-service surface (the
|
||||
orange "trouble reaching our servers" banner, the stronger "temporarily
|
||||
unavailable" error, a failed lyrics load, a slow/failed waveform), with enough
|
||||
metadata to review frequency and context later. Two sinks per event:
|
||||
|
||||
1. A structured INFO log line (``client_event type=…``) — queryable in Cloud
|
||||
Logging / Log Analytics alongside backend symptoms from the same window.
|
||||
2. A Firestore doc in ``client_events`` — survives log retention, cheap to
|
||||
aggregate for "how often are users seeing this?" reviews.
|
||||
|
||||
Identity (2026-10-03): the client sends its session token (if any) and the
|
||||
server resolves ``user_email`` / ``is_admin`` / ``is_internal`` / ``is_test``
|
||||
from it — a client-supplied email is never trusted (and is ignored). Each
|
||||
event also carries the FingerprintJS ``device_fingerprint`` (same visitorId the
|
||||
magic-link abuse checks use), a per-tab ``tab_id`` and an ``episode_id`` that
|
||||
groups one degradation episode (first banner → ``banner_recovered``).
|
||||
|
||||
The watchdog in ``backend/services/loop_watchdog.py`` writes server-side
|
||||
``server_loop_stall`` docs into the same collection; clients cannot submit
|
||||
that type.
|
||||
|
||||
Unauthenticated (review-token users must be able to report) and rate-limited
|
||||
per IP, mirroring /api/client-errors. The limiter is process-local, so the
|
||||
effective ceiling is 120/min/IP × live instances — acceptable for this threat
|
||||
model (non-malicious browsers, 60s client-side per-type throttle); Cloudflare's
|
||||
zone-wide rate limit sits in front as a flood backstop. Events are dropped,
|
||||
never queued, when the limiter trips — telemetry, not a delivery guarantee.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Literal, Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
from backend.utils.test_data import is_internal_email, is_test_email
|
||||
from backend.services.error_monitor.frontend_ingestion import (
|
||||
RateLimiter,
|
||||
is_bot_user_agent,
|
||||
sanitize_url,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/client-events", tags=["client-events"])
|
||||
|
||||
# Sized for a legitimate worst case from ONE IP: a many-tab burst (e.g. 16 review
|
||||
# tabs at once) where every tab reports a banner + a waveform event — that must be
|
||||
# fully recorded, not self-suppressed. Each tab throttles per-type client-side.
|
||||
_limiter = RateLimiter(max_per_minute=120)
|
||||
_db_singleton = None
|
||||
|
||||
# The full closed set of reportable events. Reject anything else so the
|
||||
# collection stays reviewable (no free-form types accumulating).
|
||||
EventType = Literal[
|
||||
"banner_reconnecting",
|
||||
"banner_unavailable",
|
||||
"banner_waking",
|
||||
"banner_recovered",
|
||||
"lyrics_load_failed",
|
||||
"waveform_slow",
|
||||
"waveform_failed",
|
||||
]
|
||||
|
||||
|
||||
def _get_db():
|
||||
global _db_singleton
|
||||
if _db_singleton is None:
|
||||
from google.cloud import firestore # type: ignore[import]
|
||||
|
||||
_db_singleton = firestore.Client(project="nomadkaraoke")
|
||||
return _db_singleton
|
||||
|
||||
|
||||
class ClientEventPayload(BaseModel):
|
||||
type: EventType
|
||||
url: str = Field("", max_length=2048)
|
||||
job_id: Optional[str] = Field(None, max_length=64)
|
||||
# Legacy field from pre-2026-10-03 frontends — accepted but IGNORED (the
|
||||
# server resolves identity from the bearer token instead).
|
||||
user_email: Optional[str] = Field(None, max_length=320)
|
||||
device_fingerprint: Optional[str] = Field(None, max_length=128)
|
||||
tab_id: Optional[str] = Field(None, max_length=64)
|
||||
episode_id: Optional[str] = Field(None, max_length=64)
|
||||
locale: str = Field("en", max_length=10)
|
||||
release: str = Field("", max_length=64)
|
||||
# Small free-form context (stall age, http status, elapsed ms, probe state…).
|
||||
# Size-capped server-side; values are stored as-is.
|
||||
detail: Optional[dict] = None
|
||||
|
||||
@field_validator("url", "locale", "release")
|
||||
@classmethod
|
||||
def strip(cls, v: str) -> str:
|
||||
return (v or "").strip()
|
||||
|
||||
|
||||
def _capped_detail(detail: Optional[dict]) -> dict[str, Any]:
|
||||
"""Keep detail reviewable: at most 12 keys, scalar-ish values, short strings."""
|
||||
if not isinstance(detail, dict):
|
||||
return {}
|
||||
out: dict[str, Any] = {}
|
||||
for key, value in list(detail.items())[:12]:
|
||||
k = str(key)[:40]
|
||||
if isinstance(value, (int, float, bool)) or value is None:
|
||||
out[k] = value
|
||||
else:
|
||||
out[k] = str(value)[:200]
|
||||
return out
|
||||
|
||||
|
||||
_TENANT_HOST_EXCLUDE = {"gen", "www", "api", "app", "decide", "localhost"}
|
||||
|
||||
|
||||
def _tenant_from_url(url: str) -> Optional[str]:
|
||||
"""``randy-vild.nomadkaraoke.com/...`` → ``randy-vild``; consumer hosts → None."""
|
||||
try:
|
||||
host = (urlparse(url).hostname or "").lower()
|
||||
except ValueError:
|
||||
return None
|
||||
if not host.endswith(".nomadkaraoke.com"):
|
||||
return None
|
||||
sub = host[: -len(".nomadkaraoke.com")]
|
||||
if not sub or "." in sub or sub in _TENANT_HOST_EXCLUDE:
|
||||
return None
|
||||
return sub
|
||||
|
||||
|
||||
def _resolve_identity(request: Request) -> dict[str, Any]:
|
||||
"""Resolve the caller from its bearer token. Never raises; anonymous on failure."""
|
||||
identity: dict[str, Any] = {
|
||||
"user_email": None, "is_admin": False, "is_internal": False, "is_test": False,
|
||||
}
|
||||
auth = request.headers.get("authorization", "")
|
||||
if not auth.lower().startswith("bearer "):
|
||||
return identity
|
||||
token = auth[7:].strip()
|
||||
if not token:
|
||||
return identity
|
||||
try:
|
||||
from backend.services.auth_service import get_auth_service
|
||||
|
||||
result = get_auth_service().validate_token_full(token)
|
||||
except Exception:
|
||||
logger.debug("client_event token resolution failed", exc_info=True)
|
||||
return identity
|
||||
if not result.is_valid:
|
||||
return identity
|
||||
email = result.user_email
|
||||
identity["user_email"] = email
|
||||
identity["is_admin"] = bool(result.is_admin)
|
||||
identity["is_internal"] = bool(email and is_internal_email(email))
|
||||
identity["is_test"] = bool(email and is_test_email(email))
|
||||
return identity
|
||||
|
||||
|
||||
@router.post("", status_code=202)
|
||||
def report_client_event(payload: ClientEventPayload, request: Request) -> dict:
|
||||
client_ip = request.client.host if request.client else "unknown"
|
||||
if not _limiter.allow(client_ip, time.monotonic()):
|
||||
raise HTTPException(status_code=429, detail="too many events")
|
||||
|
||||
user_agent = request.headers.get("user-agent", "")[:1024]
|
||||
if is_bot_user_agent(user_agent):
|
||||
return {"status": "ignored"}
|
||||
|
||||
# Sync handler → runs in FastAPI's threadpool, so the token lookup
|
||||
# (Firestore) never touches the event loop.
|
||||
identity = _resolve_identity(request)
|
||||
|
||||
event = {
|
||||
"type": payload.type,
|
||||
"source": "client",
|
||||
"url": sanitize_url(payload.url),
|
||||
"tenant": _tenant_from_url(payload.url),
|
||||
"job_id": payload.job_id,
|
||||
**identity,
|
||||
"device_fingerprint": payload.device_fingerprint,
|
||||
"tab_id": payload.tab_id,
|
||||
"episode_id": payload.episode_id,
|
||||
"locale": payload.locale,
|
||||
"release": payload.release,
|
||||
"user_agent": user_agent,
|
||||
"detail": _capped_detail(payload.detail),
|
||||
"created_at": datetime.now(timezone.utc),
|
||||
}
|
||||
|
||||
# Sink 1: structured log line — correlate with backend symptoms in Logging.
|
||||
logger.info(
|
||||
"client_event type=%s job_id=%s user=%s admin=%s fp=%s episode=%s url=%s release=%s detail=%s",
|
||||
event["type"],
|
||||
event["job_id"],
|
||||
event["user_email"],
|
||||
event["is_admin"],
|
||||
event["device_fingerprint"],
|
||||
event["episode_id"],
|
||||
event["url"],
|
||||
event["release"],
|
||||
event["detail"],
|
||||
)
|
||||
|
||||
# Sink 2: Firestore — best-effort; telemetry must never error back to users.
|
||||
try:
|
||||
_get_db().collection("client_events").add(event)
|
||||
except Exception:
|
||||
logger.exception("Failed to persist client event (non-fatal)")
|
||||
|
||||
return {"status": "recorded"}
|
||||
@@ -1,104 +0,0 @@
|
||||
"""
|
||||
Encoding worker lifecycle endpoints.
|
||||
|
||||
Provides warmup and heartbeat endpoints for the blue-green
|
||||
encoding worker VMs. Called by the lyrics-review frontend to ensure the
|
||||
primary VM is running before the reviewer previews a render.
|
||||
|
||||
Auth: these are gated on ``require_review_auth`` (review access to a specific
|
||||
job), NOT ``require_admin``. The callers are ordinary customers on the
|
||||
lyrics-review page — gating on admin makes every non-admin caller 403, which
|
||||
silently defeats the JIT pre-warm. Scoping to the job being reviewed keeps the
|
||||
abuse posture identical to the rest of the review surface: you can only warm
|
||||
the shared encoding VM if you already have review access to a real job.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Tuple
|
||||
from fastapi import APIRouter, Depends
|
||||
from backend.api.dependencies import require_review_auth
|
||||
from backend.services.encoding_worker_manager import EncodingWorkerManager
|
||||
from backend.services.encoding_errors import EncodingWorkerStartError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/internal/encoding-worker", tags=["encoding-worker"])
|
||||
|
||||
# Lazy-loaded singleton
|
||||
_manager = None
|
||||
|
||||
|
||||
def get_worker_manager():
|
||||
global _manager
|
||||
if _manager is None:
|
||||
from google.cloud import compute_v1, firestore
|
||||
from backend.config import get_settings
|
||||
settings = get_settings()
|
||||
db = firestore.Client(project=settings.google_cloud_project)
|
||||
compute_client = compute_v1.InstancesClient()
|
||||
_manager = EncodingWorkerManager(
|
||||
db=db,
|
||||
compute_client=compute_client,
|
||||
project_id=settings.google_cloud_project,
|
||||
)
|
||||
return _manager
|
||||
|
||||
|
||||
@router.post("/warmup/{job_id}")
|
||||
async def warmup_encoding_worker(
|
||||
job_id: str,
|
||||
_auth: Tuple[str, str] = Depends(require_review_auth),
|
||||
manager=Depends(get_worker_manager),
|
||||
):
|
||||
"""Start the preferred encoding worker if it's stopped (normally the GCE primary).
|
||||
|
||||
Called on lyrics-review page load so the VM is warm by the time the
|
||||
reviewer previews a render. ``job_id`` scopes auth to the job under review.
|
||||
"""
|
||||
try:
|
||||
# ensure_primary_running does Firestore reads + GCE compute-API calls
|
||||
# (instances.get / instances.start) that can take 10s+. Run it off the
|
||||
# event loop: on the single-process server a sync call here froze the
|
||||
# WHOLE instance for ~11s on every review-page open with a cold VM,
|
||||
# stalling that same page's audio/lyrics requests behind it
|
||||
# (2026-09-19, follow-up to the concurrent-load investigation).
|
||||
result = await asyncio.to_thread(manager.ensure_preferred_running)
|
||||
if result["started"]:
|
||||
logger.info(f"Started encoding worker VM: {result['vm_name']}")
|
||||
return result
|
||||
except EncodingWorkerStartError as e:
|
||||
# The primary VM couldn't start (capacity exhaustion or a transient
|
||||
# 503 SERVICE_UNAVAILABLE from the GCE backend). This is NOT fatal:
|
||||
# the encoding flow's own warmup (ensure_any_running) falls back to
|
||||
# the alternate-zone workers, so the job still completes. Log at
|
||||
# WARNING so it doesn't trip the production error monitor and page us
|
||||
# for self-healing capacity events.
|
||||
logger.warning(
|
||||
f"Primary encoding worker warmup failed ({e}); "
|
||||
f"encoding will fall back to an alternate-zone worker"
|
||||
)
|
||||
return {"started": False, "error": str(e)}
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to warm up encoding worker: {e}")
|
||||
return {"started": False, "error": str(e)}
|
||||
|
||||
|
||||
@router.post("/heartbeat/{job_id}")
|
||||
async def heartbeat_encoding_worker(
|
||||
job_id: str,
|
||||
_auth: Tuple[str, str] = Depends(require_review_auth),
|
||||
manager=Depends(get_worker_manager),
|
||||
):
|
||||
"""Update activity timestamp to prevent idle shutdown.
|
||||
|
||||
Called periodically while the reviewer works. ``job_id`` scopes auth to the
|
||||
job under review.
|
||||
"""
|
||||
try:
|
||||
# Sync Firestore write — keep it off the event loop (see warmup above).
|
||||
await asyncio.to_thread(manager.update_activity)
|
||||
return {"status": "ok"}
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to update encoding worker heartbeat: {e}")
|
||||
return {"status": "error", "error": str(e)}
|
||||
+219
-1070
File diff suppressed because it is too large.
Load diff
@@ -1,28 +1,21 @@
|
||||
"""
|
||||
Health check routes.
|
||||
"""
|
||||
import asyncio
|
||||
import os
|
||||
import logging
|
||||
import subprocess
|
||||
import platform
|
||||
from fastapi import APIRouter, Depends
|
||||
from typing import Dict, Any, Optional, Tuple
|
||||
from fastapi import APIRouter
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
from backend.version import VERSION, COMMIT_SHA, PR_NUMBER, PR_TITLE, STARTUP_TIME
|
||||
from backend.api.dependencies import optional_auth
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.version import VERSION
|
||||
from backend.services.flacfetch_client import get_flacfetch_client
|
||||
from backend.services.email_service import get_email_service
|
||||
from backend.services.stripe_service import get_stripe_service
|
||||
from backend.services.encoding_service import get_encoding_service
|
||||
from backend.services.encoding_worker_preference import infer_machine_type, PRIMARY_MACHINE_TYPE
|
||||
from backend.services.spacy_preloader import get_preloaded_model, is_model_preloaded
|
||||
from backend.services.nltk_preloader import get_preloaded_cmudict, is_cmudict_preloaded
|
||||
from backend.services.langfuse_preloader import get_preloaded_langfuse_handler, is_langfuse_preloaded, is_langfuse_configured
|
||||
from backend.services.job_health_service import check_job_consistency_detailed
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.models.job import JobStatus
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -218,7 +211,7 @@ async def check_flacfetch_service_status() -> Dict[str, Any]:
|
||||
|
||||
async def check_encoding_worker_status() -> Dict[str, Any]:
|
||||
"""Check if GCE encoding worker is available and healthy."""
|
||||
encoding_service = await asyncio.to_thread(get_encoding_service)
|
||||
encoding_service = get_encoding_service()
|
||||
|
||||
if not encoding_service.is_configured:
|
||||
return {
|
||||
@@ -312,177 +305,6 @@ async def flacfetch_health() -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def check_audio_separator_status() -> Dict[str, Any]:
|
||||
"""Report audio separation availability.
|
||||
|
||||
Separation runs in the ``audio-separation-job`` GPU Cloud Run Job (local
|
||||
GPU mode, models baked into the image). The standalone ``audio-separator``
|
||||
Cloud Run GPU service was shut down on 2026-10-01 (cost cut), so there is
|
||||
no remote service to probe — this reports the installed package version
|
||||
only and never calls out to anything.
|
||||
"""
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
pkg_version = version("audio-separator")
|
||||
except Exception:
|
||||
pkg_version = None
|
||||
|
||||
return {
|
||||
"available": True,
|
||||
"status": "ok",
|
||||
"mode": "gpu_job",
|
||||
"version": pkg_version,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health/audio-separator")
|
||||
def audio_separator_health() -> Dict[str, Any]:
|
||||
"""
|
||||
Return audio separation status (installed audio-separator package version).
|
||||
|
||||
Does NOT call any remote service — separation runs in the
|
||||
audio-separation-job GPU Cloud Run Job. No authentication required.
|
||||
"""
|
||||
return check_audio_separator_status()
|
||||
|
||||
|
||||
def _get_encoding_worker_manager():
|
||||
"""Get the EncodingWorkerManager instance. Separated for testability."""
|
||||
try:
|
||||
encoding_service = get_encoding_service()
|
||||
if hasattr(encoding_service, '_worker_manager') and encoding_service._worker_manager:
|
||||
return encoding_service._worker_manager
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
@router.get("/health/system-status")
|
||||
async def system_status(
|
||||
auth: Optional[Tuple] = Depends(optional_auth),
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Aggregated system status for all services.
|
||||
|
||||
Returns basic info for everyone. Admin users get additional
|
||||
blue-green deployment details for the encoder.
|
||||
"""
|
||||
is_admin = auth is not None and len(auth) >= 2 and auth[1] == UserType.ADMIN
|
||||
|
||||
# Fetch all health statuses concurrently
|
||||
import asyncio
|
||||
encoding_task = check_encoding_worker_status()
|
||||
flacfetch_task = check_flacfetch_service_status()
|
||||
encoding_status, flacfetch_status = await asyncio.gather(
|
||||
encoding_task, flacfetch_task
|
||||
)
|
||||
separator_status = check_audio_separator_status()
|
||||
|
||||
# Build frontend service info (from env vars baked at build time — returned as-is)
|
||||
frontend_svc = {
|
||||
"status": "ok",
|
||||
"version": VERSION, # Frontend and backend share the same version from pyproject.toml
|
||||
}
|
||||
|
||||
# Build backend service info
|
||||
backend_svc = {
|
||||
"status": "ok",
|
||||
"version": VERSION,
|
||||
"deployed_at": STARTUP_TIME,
|
||||
}
|
||||
if COMMIT_SHA:
|
||||
backend_svc["commit_sha"] = COMMIT_SHA
|
||||
if PR_NUMBER:
|
||||
backend_svc["pr_number"] = PR_NUMBER
|
||||
if PR_TITLE:
|
||||
backend_svc["pr_title"] = PR_TITLE
|
||||
|
||||
# Build encoder service info
|
||||
encoder_svc = {
|
||||
"status": "ok" if encoding_status.get("available") else "offline",
|
||||
"version": encoding_status.get("wheel_version"),
|
||||
"active_jobs": encoding_status.get("active_jobs", 0),
|
||||
}
|
||||
|
||||
# Build flacfetch service info
|
||||
flacfetch_svc = {
|
||||
"status": "ok" if flacfetch_status.get("available") else "offline",
|
||||
"version": flacfetch_status.get("version"),
|
||||
}
|
||||
|
||||
# Build separator service info
|
||||
separator_svc = {
|
||||
"status": "ok" if separator_status.get("available") else "offline",
|
||||
"version": separator_status.get("version"),
|
||||
}
|
||||
|
||||
# Admin-only: blue-green encoder details from Firestore
|
||||
if is_admin:
|
||||
manager = _get_encoding_worker_manager()
|
||||
if manager:
|
||||
try:
|
||||
config = manager.get_config()
|
||||
|
||||
# Which VM is actually serving right now? When a capacity
|
||||
# fallback is active (e.g. a c4d stockout routed us to an
|
||||
# alternate family) the override points at the live VM;
|
||||
# otherwise it's the blue-green primary. The primary pair is
|
||||
# always c4d — fallbacks carry their family in the VM name
|
||||
# (…-fallback-c2df), which infer_machine_type
|
||||
# decodes when no explicit machine_type is stored on the config.
|
||||
on_fallback = bool(config.active_override_vm)
|
||||
active_vm = config.active_override_vm or config.primary_vm
|
||||
default_zone = getattr(manager, "_zone", "us-central1-c")
|
||||
active_zone = config.active_override_zone or default_zone
|
||||
if on_fallback:
|
||||
active_machine_type = infer_machine_type({"vm": active_vm})
|
||||
else:
|
||||
active_machine_type = (
|
||||
getattr(config, "primary_machine_type", None)
|
||||
or PRIMARY_MACHINE_TYPE
|
||||
)
|
||||
|
||||
encoder_svc["admin_details"] = {
|
||||
"primary_vm": config.primary_vm,
|
||||
"primary_ip": config.primary_ip,
|
||||
"primary_version": config.primary_version,
|
||||
"primary_deployed_at": config.primary_deployed_at,
|
||||
"secondary_vm": config.secondary_vm,
|
||||
"secondary_ip": config.secondary_ip,
|
||||
"secondary_version": config.secondary_version,
|
||||
"secondary_deployed_at": config.secondary_deployed_at,
|
||||
"last_swap_at": config.last_swap_at,
|
||||
"deploy_in_progress": config.deploy_in_progress,
|
||||
"active_jobs": encoding_status.get("active_jobs", 0),
|
||||
"queue_length": encoding_status.get("queue_length", 0),
|
||||
# Currently-serving worker (fallback-aware)
|
||||
"active_vm": active_vm,
|
||||
"active_zone": active_zone,
|
||||
"active_machine_type": active_machine_type,
|
||||
"on_fallback": on_fallback,
|
||||
}
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get blue-green config: {e}")
|
||||
|
||||
# Add error details for offline services
|
||||
if not flacfetch_status.get("available"):
|
||||
flacfetch_svc["admin_details"] = {"error": flacfetch_status.get("error")}
|
||||
if not separator_status.get("available"):
|
||||
separator_svc["admin_details"] = {"error": separator_status.get("error")}
|
||||
if not encoding_status.get("available") and "admin_details" not in encoder_svc:
|
||||
encoder_svc["admin_details"] = {"error": encoding_status.get("error")}
|
||||
|
||||
return {
|
||||
"services": {
|
||||
"frontend": frontend_svc,
|
||||
"backend": backend_svc,
|
||||
"encoder": encoder_svc,
|
||||
"flacfetch": flacfetch_svc,
|
||||
"separator": separator_svc,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health/detailed")
|
||||
async def detailed_health_check() -> Dict[str, Any]:
|
||||
"""
|
||||
@@ -547,7 +369,7 @@ async def detailed_health_check() -> Dict[str, Any]:
|
||||
|
||||
|
||||
@router.get("/health/preload-status")
|
||||
def preload_status() -> Dict[str, Any]:
|
||||
async def preload_status() -> Dict[str, Any]:
|
||||
"""
|
||||
Check status of preloaded resources for performance optimization.
|
||||
|
||||
@@ -608,69 +430,6 @@ def preload_status() -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health/job-consistency")
|
||||
def job_consistency_check(
|
||||
status: Optional[str] = None,
|
||||
limit: int = 50
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Check job consistency across the system.
|
||||
|
||||
This endpoint identifies jobs with state inconsistencies that may indicate
|
||||
bugs in the state machine or trigger logic. Use this to find stuck jobs
|
||||
or jobs in invalid states.
|
||||
|
||||
Args:
|
||||
status: Optional filter by job status (e.g., "pending", "downloading")
|
||||
limit: Maximum number of jobs to check (default 50)
|
||||
|
||||
Returns:
|
||||
Dictionary with:
|
||||
- total_checked: Number of jobs checked
|
||||
- inconsistent_count: Number of jobs with issues
|
||||
- inconsistent_jobs: List of job details with their issues
|
||||
- summary: Summary of issue types found
|
||||
"""
|
||||
job_manager = JobManager()
|
||||
|
||||
# Query jobs to check
|
||||
query_params = {}
|
||||
if status:
|
||||
try:
|
||||
job_status = JobStatus(status)
|
||||
query_params['status'] = job_status
|
||||
except ValueError:
|
||||
return {
|
||||
"error": f"Invalid status: {status}",
|
||||
"valid_statuses": [s.value for s in JobStatus]
|
||||
}
|
||||
|
||||
# Get jobs from Firestore
|
||||
jobs = job_manager.list_jobs(limit=limit, **query_params)
|
||||
|
||||
# Check each job for consistency
|
||||
inconsistent_jobs = []
|
||||
issue_summary = {}
|
||||
|
||||
for job in jobs:
|
||||
result = check_job_consistency_detailed(job)
|
||||
if not result['is_healthy']:
|
||||
inconsistent_jobs.append(result)
|
||||
# Count issue types
|
||||
for issue in result['issues']:
|
||||
issue_type = issue.split(':')[0] if ':' in issue else issue
|
||||
issue_summary[issue_type] = issue_summary.get(issue_type, 0) + 1
|
||||
|
||||
return {
|
||||
"total_checked": len(jobs),
|
||||
"inconsistent_count": len(inconsistent_jobs),
|
||||
"inconsistent_jobs": inconsistent_jobs,
|
||||
"summary": issue_summary,
|
||||
"status_filter": status,
|
||||
"limit": limit,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/readiness")
|
||||
async def readiness_check() -> Dict[str, str]:
|
||||
"""Readiness check endpoint for Cloud Run."""
|
||||
|
||||
+27
-1294
File diff suppressed because it is too large.
Load diff
+183
-2412
File diff suppressed because it is too large.
Load diff
@@ -1,94 +0,0 @@
|
||||
"""POST /api/karaokehunt/request — intake for retired-KaraokeHunt-app requests.
|
||||
|
||||
A Cloudflare Worker on create.karaokehunt.com (the URL baked into the
|
||||
unchangeable app binary) answers the app instantly and forwards each payload
|
||||
here with a shared secret. This route validates + logs the payload, then runs
|
||||
the full conversion synchronously (see ``workers/karaokehunt_conversion``) —
|
||||
the Worker doesn't wait for us, so latency is a non-issue and uvicorn finishes
|
||||
the handler even if the forwarding subrequest is reaped.
|
||||
|
||||
Unauthenticated by necessity (the app can't auth), gated three ways: the
|
||||
shared secret (503 until configured — deploys dark), a per-IP rate limit, and
|
||||
the conversion worker's own daily job cap + per-song dedup.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hmac
|
||||
import logging
|
||||
import time
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.config import get_settings
|
||||
from backend.services.error_monitor.frontend_ingestion import RateLimiter
|
||||
from backend.workers import karaokehunt_conversion
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/karaokehunt", tags=["karaokehunt"])
|
||||
|
||||
# ~1 legit request every 3 days historically; 10/min/IP is already generous.
|
||||
_limiter = RateLimiter(max_per_minute=10)
|
||||
|
||||
|
||||
def _client_ip(request: Request) -> str:
|
||||
# The Worker forwards the app's real IP; direct callers fall back to peer IP.
|
||||
return (request.headers.get("x-kh-client-ip")
|
||||
or (request.client.host if request.client else "unknown"))
|
||||
|
||||
|
||||
@router.post("/request", status_code=200)
|
||||
async def intake_karaokehunt_request(request: Request) -> dict:
|
||||
settings = get_settings()
|
||||
if not settings.karaokehunt_forwarder_secret:
|
||||
raise HTTPException(status_code=503, detail="not configured")
|
||||
|
||||
provided = request.headers.get("x-kh-forwarder-secret", "")
|
||||
if not hmac.compare_digest(provided, settings.karaokehunt_forwarder_secret):
|
||||
raise HTTPException(status_code=403, detail="forbidden")
|
||||
|
||||
if not _limiter.allow(_client_ip(request), time.monotonic()):
|
||||
raise HTTPException(status_code=429, detail="too many requests")
|
||||
|
||||
# The live app binary's exact schema is unverifiable (built from a
|
||||
# FlutterFlow export newer than any repo branch), so parse leniently:
|
||||
# any JSON object is accepted and logged; unusable ones become
|
||||
# outcome="invalid" audit rows rather than errors.
|
||||
try:
|
||||
payload = await request.json()
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("payload must be a JSON object")
|
||||
except Exception: # noqa: BLE001
|
||||
raise HTTPException(status_code=400, detail="invalid JSON")
|
||||
|
||||
doc = karaokehunt_conversion.create_intake(
|
||||
payload,
|
||||
client_ip=_client_ip(request),
|
||||
user_agent=request.headers.get("user-agent", "")[:512],
|
||||
)
|
||||
if doc["outcome"] == "invalid":
|
||||
return {"status": "success", "id": doc["id"]}
|
||||
|
||||
try:
|
||||
result = await karaokehunt_conversion.process_intake(doc["id"])
|
||||
logger.info("karaokehunt: processed %s -> %s", doc["id"], result.get("status"))
|
||||
except Exception: # noqa: BLE001 — the intake doc + reprocess endpoint carry retries
|
||||
logger.exception("karaokehunt: conversion failed for %s", doc["id"])
|
||||
try:
|
||||
karaokehunt_conversion._get_db().collection(
|
||||
karaokehunt_conversion.COLLECTION
|
||||
).document(doc["id"]).update({"outcome": "error", "error": "unhandled exception"})
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception("karaokehunt: failed to mark %s as error", doc["id"])
|
||||
|
||||
# The app ignores the body; always succeed so nothing user-visible breaks.
|
||||
return {"status": "success", "id": doc["id"]}
|
||||
|
||||
|
||||
@router.post("/internal/reprocess/{doc_id}")
|
||||
async def reprocess_karaokehunt_request(
|
||||
doc_id: str, auth_result=Depends(require_admin)
|
||||
) -> dict:
|
||||
"""Admin retry for a failed/errored intake (idempotent markers prevent
|
||||
double credits/jobs). Also accepts force=true via query for terminal docs."""
|
||||
return await karaokehunt_conversion.process_intake(doc_id, force=True)
|
||||
@@ -1,625 +0,0 @@
|
||||
"""kjbox partner API — singer email-code sign-in + "free at the show" credits.
|
||||
|
||||
kjbox (the KJ device app at live karaoke nights) lets a patron verify their
|
||||
email inside the kjbox singer web page, then creates gen jobs AS that real gen
|
||||
user (with the returned session token) so they get normal gen delivery emails.
|
||||
|
||||
Every endpoint requires ``X-Kjbox-Secret`` (503 until ``KJBOX_PARTNER_SECRET``
|
||||
is configured — deploys dark; 403 if wrong). Contract:
|
||||
|
||||
- POST /api/kjbox/auth/send-code {email, locale?, venue?} → {"status": "sent"}
|
||||
- POST /api/kjbox/auth/verify-code {email, code} → {session_token, user,
|
||||
credits_granted, credit_status}
|
||||
- POST /api/kjbox/auth/google | apple {id_token, locale?, venue?} → same as
|
||||
verify-code (one-tap sign-in in the singer page)
|
||||
- POST /api/kjbox/credits/show-credit (+ Authorization: Bearer <session>)
|
||||
{idempotency_key, venue?, only_if_empty?}
|
||||
→ {granted, credits}
|
||||
|
||||
A venue is many singers behind one IP, so gen's 2-signups-per-IP cap does NOT
|
||||
apply here; instead a partner-wide rolling-24h cap on new VERIFIED accounts
|
||||
(accounts are only created at verify-code), a per-email code throttle, and a
|
||||
per-user daily show-credit cap bound abuse.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hmac
|
||||
import logging
|
||||
import threading
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, Response
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from backend.api.routes.users import (
|
||||
_build_user_public,
|
||||
_mask_email,
|
||||
_precompute_credit_eval,
|
||||
complete_verified_login,
|
||||
)
|
||||
from backend.config import get_settings, is_production
|
||||
from backend.i18n import SUPPORTED_LOCALES, t
|
||||
from backend.services.account_deletion_service import (
|
||||
AccountDeletionService,
|
||||
get_account_deletion_service,
|
||||
)
|
||||
from backend.services.email_service import EmailService, get_email_service
|
||||
from backend.services.email_validation_service import get_email_validation_service
|
||||
from backend.services.kjbox_partner_service import (
|
||||
CODE_EXPIRY_MINUTES,
|
||||
LOGIN_CODES_COLLECTION,
|
||||
KjboxPartnerService,
|
||||
CodeThrottled,
|
||||
ShowCreditCapReached,
|
||||
TransactionContention,
|
||||
VerifyOutcome,
|
||||
)
|
||||
from backend.services.user_service import UserService, get_user_service
|
||||
from backend.utils.request_helpers import get_client_ip
|
||||
from backend.utils.test_data import is_test_email
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/kjbox", tags=["kjbox"])
|
||||
|
||||
SIGNUP_SOURCE = "kjbox"
|
||||
_MAX_EMAIL_LEN = 254
|
||||
_MAX_VENUE_LEN = 200
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ auth gate
|
||||
|
||||
def require_kjbox_partner(x_kjbox_secret: Optional[str] = Header(default=None)) -> str:
|
||||
"""Shared-secret gate. Returns the secret (also used as the code-hash pepper)."""
|
||||
secret = get_settings().kjbox_partner_secret
|
||||
if not secret:
|
||||
raise HTTPException(status_code=503, detail="not configured")
|
||||
if not hmac.compare_digest((x_kjbox_secret or "").encode(), secret.encode()):
|
||||
raise HTTPException(status_code=403, detail="forbidden")
|
||||
return secret
|
||||
|
||||
|
||||
def require_resolve_partner(
|
||||
x_kjbox_secret: Optional[str] = Header(default=None),
|
||||
x_partner_secret: Optional[str] = Header(default=None),
|
||||
) -> str:
|
||||
"""Gate for ``catalog/resolve``: kjbox (``X-Kjbox-Secret``) or karaoke-decide
|
||||
(``X-Partner-Secret``). Returns the partner name ("kjbox" / "decide"), which
|
||||
picks the rate-budget bucket. 503 when neither secret is configured."""
|
||||
settings = get_settings()
|
||||
kjbox_secret = settings.kjbox_partner_secret
|
||||
decide_secret = getattr(settings, "decide_partner_secret", "")
|
||||
if not kjbox_secret and not decide_secret:
|
||||
raise HTTPException(status_code=503, detail="not configured")
|
||||
if kjbox_secret and x_kjbox_secret and hmac.compare_digest(
|
||||
x_kjbox_secret.encode(), kjbox_secret.encode()):
|
||||
return "kjbox"
|
||||
if decide_secret and x_partner_secret and hmac.compare_digest(
|
||||
x_partner_secret.encode(), decide_secret.encode()):
|
||||
return "decide"
|
||||
raise HTTPException(status_code=403, detail="forbidden")
|
||||
|
||||
|
||||
def get_kjbox_service(
|
||||
secret: str = Depends(require_kjbox_partner),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
) -> KjboxPartnerService:
|
||||
return KjboxPartnerService(user_service.db, pepper=secret)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- models
|
||||
|
||||
class SendCodeRequest(BaseModel):
|
||||
email: str
|
||||
locale: Optional[str] = None
|
||||
venue: Optional[str] = Field(default=None, max_length=_MAX_VENUE_LEN)
|
||||
|
||||
|
||||
class SendCodeResponse(BaseModel):
|
||||
status: str
|
||||
|
||||
|
||||
class VerifyCodeRequest(BaseModel):
|
||||
email: str
|
||||
code: str = Field(max_length=32)
|
||||
|
||||
|
||||
class SocialSignInRequest(BaseModel):
|
||||
# Google Identity Services / Sign in with Apple JS ID token from the singer page
|
||||
# (or ``fake:<email>`` on the local stack).
|
||||
id_token: str = Field(min_length=1, max_length=8192)
|
||||
locale: Optional[str] = None
|
||||
venue: Optional[str] = Field(default=None, max_length=_MAX_VENUE_LEN)
|
||||
|
||||
|
||||
class VerifyCodeResponse(BaseModel):
|
||||
session_token: str
|
||||
user: dict
|
||||
credits_granted: int
|
||||
credit_status: str
|
||||
|
||||
|
||||
class ShowCreditRequest(BaseModel):
|
||||
idempotency_key: str = Field(min_length=1, max_length=128)
|
||||
venue: Optional[str] = Field(default=None, max_length=_MAX_VENUE_LEN)
|
||||
# Grant only if the singer currently has no credits (< 1). The key is claimed
|
||||
# only when a credit is actually granted, so a later call with the same key
|
||||
# and only_if_empty=false grants iff this one didn't.
|
||||
only_if_empty: bool = False
|
||||
|
||||
|
||||
class ShowCreditResponse(BaseModel):
|
||||
granted: bool
|
||||
credits: int
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- helpers
|
||||
|
||||
def _normalize_email(raw: str) -> str:
|
||||
email = (raw or "").strip().lower()
|
||||
local, _, domain = email.partition("@")
|
||||
if (not local or "." not in domain or len(email) > _MAX_EMAIL_LEN
|
||||
or any(c.isspace() for c in email) or email.count("@") != 1):
|
||||
raise HTTPException(status_code=422, detail="invalid_email")
|
||||
return email
|
||||
|
||||
|
||||
def _split_locale(raw: Optional[str]) -> tuple[Optional[str], Optional[str]]:
|
||||
"""Return (email_locale in en/es/de or None, ui_locale primary subtag or None)."""
|
||||
if not raw:
|
||||
return None, None
|
||||
primary = raw.strip().replace("_", "-").split("-")[0].lower()
|
||||
if not primary.isalpha() or len(primary) < 2:
|
||||
return None, None
|
||||
return (primary if primary in SUPPORTED_LOCALES else None), primary
|
||||
|
||||
|
||||
def _pretend_sent() -> SendCodeResponse:
|
||||
return SendCodeResponse(status="sent")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ endpoints
|
||||
|
||||
@router.post("/auth/send-code", response_model=SendCodeResponse)
|
||||
def send_code(
|
||||
body: SendCodeRequest,
|
||||
http_request: Request,
|
||||
svc: KjboxPartnerService = Depends(get_kjbox_service),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
settings = get_settings()
|
||||
email = _normalize_email(body.email)
|
||||
email_locale, ui_locale = _split_locale(body.locale)
|
||||
locale = email_locale or "en"
|
||||
venue = (body.venue or "").strip() or None
|
||||
|
||||
email_validation = get_email_validation_service()
|
||||
if email_validation.is_disposable_domain(email):
|
||||
logger.warning(f"kjbox: blocked disposable email {_mask_email(email)}")
|
||||
raise HTTPException(status_code=422, detail=t(locale, "users.disposableEmailBlocked"))
|
||||
if email_validation.is_email_blocked(email):
|
||||
logger.warning(f"kjbox: blocked email {_mask_email(email)} — pretending success")
|
||||
return _pretend_sent()
|
||||
ip_address = get_client_ip(http_request)
|
||||
if ip_address and email_validation.is_ip_blocked(ip_address):
|
||||
logger.warning(f"kjbox: blocked IP {ip_address} — pretending success")
|
||||
return _pretend_sent()
|
||||
|
||||
if not email_service.is_configured() and is_production():
|
||||
logger.error("kjbox: email service not configured - cannot send codes")
|
||||
raise HTTPException(status_code=503, detail="email not configured")
|
||||
|
||||
# No account is created here — only once the email is verified (verify-code),
|
||||
# so unverified addresses can neither burn the signup cap nor create accounts
|
||||
# for other people's emails. Venue/locale ride on the code record until then.
|
||||
user = user_service.get_user(email)
|
||||
try:
|
||||
issued = svc.issue_code(
|
||||
email,
|
||||
max_per_hour=settings.kjbox_codes_per_email_per_hour,
|
||||
needs_credit_eval=(
|
||||
not (user and getattr(user, "welcome_credits_granted", False))
|
||||
and not is_test_email(email)
|
||||
),
|
||||
locale=email_locale,
|
||||
ui_locale=ui_locale,
|
||||
venue=venue,
|
||||
ip_address=ip_address,
|
||||
user_agent=http_request.headers.get("user-agent"),
|
||||
)
|
||||
except (CodeThrottled, TransactionContention):
|
||||
# Contention on one email's code doc = a burst of parallel sends to the
|
||||
# same address, which is exactly what the throttle exists to stop.
|
||||
raise HTTPException(status_code=429, detail="too_many_codes")
|
||||
|
||||
sent = email_service.send_kjbox_login_code(
|
||||
email, issued.code, expiry_minutes=CODE_EXPIRY_MINUTES, locale=locale,
|
||||
)
|
||||
if not sent:
|
||||
if getattr(email_service, "last_send_suppressed", False):
|
||||
logger.info(f"kjbox: code recipient {_mask_email(email)} is Postmark-suppressed")
|
||||
else:
|
||||
logger.error(f"kjbox: failed to send code email to {_mask_email(email)}")
|
||||
|
||||
# Warm the welcome-credit AI evaluation while the singer reads their email,
|
||||
# so verify-code is instant (same optimisation as magic links).
|
||||
if issued.needs_credit_eval:
|
||||
threading.Thread(
|
||||
target=_precompute_credit_eval,
|
||||
args=(issued.doc_id, email, LOGIN_CODES_COLLECTION),
|
||||
daemon=True,
|
||||
).start()
|
||||
|
||||
return SendCodeResponse(status="sent")
|
||||
|
||||
|
||||
@router.post("/auth/verify-code", response_model=VerifyCodeResponse)
|
||||
def verify_code(
|
||||
body: VerifyCodeRequest,
|
||||
http_request: Request,
|
||||
svc: KjboxPartnerService = Depends(get_kjbox_service),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
settings = get_settings()
|
||||
email = _normalize_email(body.email)
|
||||
|
||||
# New accounts are created (and counted against the partner-wide cap) only
|
||||
# here, after the email is proven. Check the cap BEFORE consuming the code so
|
||||
# a capped singer can retry the same code later.
|
||||
existing = user_service.get_user(email)
|
||||
if existing is None and svc.count_recent_signups() >= settings.kjbox_signup_cap_per_24h:
|
||||
logger.warning("kjbox: partner-wide verified-signup cap reached")
|
||||
raise HTTPException(status_code=429, detail="signup_cap")
|
||||
|
||||
outcome, record = svc.verify_code(email, body.code)
|
||||
if outcome == VerifyOutcome.TOO_MANY_ATTEMPTS:
|
||||
raise HTTPException(status_code=429, detail="too_many_attempts")
|
||||
if outcome == VerifyOutcome.INVALID:
|
||||
raise HTTPException(status_code=401, detail="invalid_code")
|
||||
if outcome != VerifyOutcome.OK or record is None:
|
||||
raise HTTPException(status_code=401, detail="expired")
|
||||
|
||||
precomputed_eval = {
|
||||
k: record.get(k)
|
||||
for k in ("credit_eval_decision", "credit_eval_reasoning", "credit_eval_error")
|
||||
if record.get(k) is not None
|
||||
} or None
|
||||
return _finish_sign_in(
|
||||
email, http_request, svc=svc, user_service=user_service, email_service=email_service,
|
||||
venue=record.get("venue"), email_locale=record.get("locale"),
|
||||
ui_locale=record.get("ui_locale"), precomputed_eval=precomputed_eval, how="verified",
|
||||
)
|
||||
|
||||
|
||||
def _finish_sign_in(email, http_request, *, svc, user_service, email_service, venue,
|
||||
email_locale, ui_locale, precomputed_eval, how):
|
||||
"""A proven email (code, or a Google/Apple ID token) → the singer's gen session."""
|
||||
user = user_service.get_user(email)
|
||||
if user is None:
|
||||
# No signup_ip: the caller is the kjbox box, whose IP is the whole
|
||||
# venue's — recording it would trip gen's per-IP cap for everyone there.
|
||||
user = user_service.get_or_create_user(
|
||||
email, signup_source=SIGNUP_SOURCE, signup_venue=venue,
|
||||
)
|
||||
svc.record_signup(email, venue)
|
||||
logger.info(f"kjbox: created verified gen user {_mask_email(email)} (venue={venue!r}, via {how})")
|
||||
# Same first-login definition as magic-link verify (checked before last_login_at is set).
|
||||
is_first_login = user.total_jobs_created == 0 and not user.last_login_at
|
||||
user = user_service.update_user(
|
||||
email, email_verified=True, last_login_at=datetime.utcnow(),
|
||||
) or user
|
||||
|
||||
result = complete_verified_login(
|
||||
user,
|
||||
user_service=user_service,
|
||||
email_service=email_service,
|
||||
is_first_login=is_first_login,
|
||||
precomputed_eval=precomputed_eval,
|
||||
grant_welcome_credit=True,
|
||||
referral_code=None,
|
||||
locale=email_locale,
|
||||
ui_locale=ui_locale,
|
||||
ip_address=get_client_ip(http_request),
|
||||
user_agent=http_request.headers.get("user-agent"),
|
||||
tenant_id=None,
|
||||
device_fingerprint=None,
|
||||
email_locale=email_locale or user.locale or "en",
|
||||
)
|
||||
logger.info(f"kjbox: {how} {_mask_email(email)} (credit_status={result.credit_status})")
|
||||
|
||||
return VerifyCodeResponse(
|
||||
session_token=result.session.token,
|
||||
user=_build_user_public(result.user).model_dump(mode="json"),
|
||||
credits_granted=result.credits_granted,
|
||||
credit_status=result.credit_status,
|
||||
)
|
||||
|
||||
|
||||
def _social_sign_in(provider, body, http_request, svc, user_service, email_service):
|
||||
"""Google / Apple one-tap: the provider has proven the email, so no code.
|
||||
Same venue rules as verify-code: no per-IP signup cap, the partner-wide 24h
|
||||
cap on new accounts instead. Apple "Hide My Email" relays are their own account."""
|
||||
from backend.api.routes.login_code import APPLE_RELAY_DOMAIN, _verify_identity
|
||||
|
||||
identity = _verify_identity(provider, body.id_token) # 401 invalid_token / 503
|
||||
email = _normalize_email(identity.email)
|
||||
email_locale, ui_locale = _split_locale(body.locale)
|
||||
|
||||
# The same eligibility checks as send-code. There's no email to hide a
|
||||
# refusal behind, so (like the app's social sign-in) refusals are explicit.
|
||||
email_validation = get_email_validation_service()
|
||||
is_relay = provider == "apple" and email.endswith("@" + APPLE_RELAY_DOMAIN)
|
||||
if not is_relay and email_validation.is_disposable_domain(email):
|
||||
logger.warning(f"kjbox: {provider} sign-in refused, disposable email {_mask_email(email)}")
|
||||
raise HTTPException(status_code=422, detail=t(email_locale or "en", "users.disposableEmailBlocked"))
|
||||
ip_address = get_client_ip(http_request)
|
||||
if email_validation.is_email_blocked(email) or (ip_address and email_validation.is_ip_blocked(ip_address)):
|
||||
logger.warning(f"kjbox: {provider} sign-in refused, blocked email/IP {_mask_email(email)}")
|
||||
raise HTTPException(status_code=422, detail="sign_in_not_allowed")
|
||||
if user_service.get_user(email) is None and \
|
||||
svc.count_recent_signups() >= get_settings().kjbox_signup_cap_per_24h:
|
||||
logger.warning("kjbox: partner-wide verified-signup cap reached")
|
||||
raise HTTPException(status_code=429, detail="signup_cap")
|
||||
return _finish_sign_in(
|
||||
email, http_request, svc=svc, user_service=user_service, email_service=email_service,
|
||||
venue=(body.venue or None), email_locale=email_locale, ui_locale=ui_locale,
|
||||
precomputed_eval=None, how=f"{provider} sign-in",
|
||||
)
|
||||
|
||||
|
||||
@router.post("/auth/google", response_model=VerifyCodeResponse)
|
||||
def google_sign_in(
|
||||
body: SocialSignInRequest,
|
||||
http_request: Request,
|
||||
svc: KjboxPartnerService = Depends(get_kjbox_service),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""Google ID token (audience ``GOOGLE_OAUTH_CLIENT_IDS``) → singer session."""
|
||||
return _social_sign_in("google", body, http_request, svc, user_service, email_service)
|
||||
|
||||
|
||||
@router.post("/auth/apple", response_model=VerifyCodeResponse)
|
||||
def apple_sign_in(
|
||||
body: SocialSignInRequest,
|
||||
http_request: Request,
|
||||
svc: KjboxPartnerService = Depends(get_kjbox_service),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""Apple ID token (audience ``APPLE_SIGNIN_AUDIENCES``, incl. the web Services ID) → singer session."""
|
||||
return _social_sign_in("apple", body, http_request, svc, user_service, email_service)
|
||||
|
||||
|
||||
@router.post("/credits/show-credit", response_model=ShowCreditResponse)
|
||||
def show_credit(
|
||||
body: ShowCreditRequest,
|
||||
svc: KjboxPartnerService = Depends(get_kjbox_service),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
authorization: Optional[str] = Header(default=None),
|
||||
):
|
||||
"""Quietly grant +1 credit so a singer's make-it job is free at the show.
|
||||
|
||||
No "credits added" email. Idempotent per ``idempotency_key`` (claimed only
|
||||
when a credit is granted); ``only_if_empty`` grants only at a 0 balance;
|
||||
capped per user per rolling 24h. All decisions + the increment are one
|
||||
Firestore transaction.
|
||||
"""
|
||||
settings = get_settings()
|
||||
scheme, _, token = (authorization or "").partition(" ")
|
||||
token = token.strip()
|
||||
if scheme.lower() != "bearer" or not token:
|
||||
raise HTTPException(status_code=401, detail="invalid_session")
|
||||
valid, user, _msg = user_service.validate_session(token)
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail="invalid_session")
|
||||
|
||||
email = user.email.lower()
|
||||
venue = (body.venue or "").strip() or None
|
||||
user_ref = user_service.get_user_doc_ref(email)
|
||||
if user_ref is None:
|
||||
raise HTTPException(status_code=401, detail="invalid_session")
|
||||
|
||||
try:
|
||||
result = svc.grant_show_credit(
|
||||
user_ref,
|
||||
email,
|
||||
body.idempotency_key,
|
||||
venue=venue,
|
||||
only_if_empty=body.only_if_empty,
|
||||
max_per_24h=settings.kjbox_show_credits_per_user_per_24h,
|
||||
)
|
||||
except ShowCreditCapReached:
|
||||
raise HTTPException(status_code=429, detail="show_credit_cap")
|
||||
except LookupError:
|
||||
raise HTTPException(status_code=401, detail="invalid_session")
|
||||
except TransactionContention:
|
||||
# Nothing was written; kjbox retries with the same idempotency key.
|
||||
raise HTTPException(status_code=409, detail="busy_retry")
|
||||
|
||||
if result.granted:
|
||||
logger.info(f"kjbox: show credit granted to {_mask_email(email)} (venue={venue!r}) → {result.credits}")
|
||||
return ShowCreditResponse(granted=result.granted, credits=result.credits)
|
||||
|
||||
|
||||
# ------------------------------------------------------------- review link
|
||||
|
||||
class ReviewLinkRequest(BaseModel):
|
||||
locale: Optional[str] = None
|
||||
|
||||
|
||||
class ReviewLinkResponse(BaseModel):
|
||||
url: str
|
||||
status: str
|
||||
# "admin" = the KJ / support already opened the review; "owner" = the
|
||||
# singer did; None = nobody has opened it yet.
|
||||
review_started_by: Optional[str] = None
|
||||
|
||||
|
||||
_REVIEW_LINK_STATUSES = ("awaiting_review", "in_review")
|
||||
|
||||
|
||||
# ------------------------------------------------- account deletion queue
|
||||
|
||||
@router.get("/account-deletions")
|
||||
def list_account_deletions(
|
||||
since: Optional[datetime] = Query(default=None),
|
||||
_secret: str = Depends(require_kjbox_partner),
|
||||
deletion: AccountDeletionService = Depends(get_account_deletion_service),
|
||||
):
|
||||
"""Deleted gen accounts the box still has to erase (it pulls: it may be offline).
|
||||
|
||||
Only the sha256 of the email is sent; the box hashes the ``gen_email`` values
|
||||
it stores to match. Lists entries still pending for kjbox, created after ``since``.
|
||||
"""
|
||||
return {"deletions": deletion.list_kjbox_pending(since)}
|
||||
|
||||
|
||||
@router.post("/account-deletions/{entry_id}/ack", status_code=204)
|
||||
def ack_account_deletion(
|
||||
entry_id: str,
|
||||
_secret: str = Depends(require_kjbox_partner),
|
||||
deletion: AccountDeletionService = Depends(get_account_deletion_service),
|
||||
):
|
||||
"""The box erased this account's data; stop listing it. ``entry_id`` = sha256(email)."""
|
||||
if not deletion.ack_kjbox(entry_id):
|
||||
raise HTTPException(status_code=404, detail="not_found")
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
@router.post("/jobs/{job_id}/review-link", response_model=ReviewLinkResponse)
|
||||
async def review_link(
|
||||
job_id: str,
|
||||
body: ReviewLinkRequest,
|
||||
_secret: str = Depends(require_kjbox_partner),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
authorization: Optional[str] = Header(default=None),
|
||||
):
|
||||
"""One-click sign-in link straight to the singer's own job's lyrics review.
|
||||
|
||||
Backs the "Tap here to review it yourself" link on the kjbox singer page:
|
||||
the same single-use ``job_review:<id>`` magic link the kjbox review email
|
||||
carries, minted on demand. Only the job's owner (the Bearer session) may
|
||||
get one, and only while the job is in review (409 ``not_in_review``).
|
||||
"""
|
||||
scheme, _, token = (authorization or "").partition(" ")
|
||||
token = token.strip()
|
||||
if scheme.lower() != "bearer" or not token:
|
||||
raise HTTPException(status_code=401, detail="invalid_session")
|
||||
locale = body.locale if body.locale in SUPPORTED_LOCALES else "en"
|
||||
# Session check, job read and token mint are blocking Firestore calls —
|
||||
# keep them off the event loop (it serves every in-flight request).
|
||||
url, status, started_by = await asyncio.to_thread(
|
||||
_mint_review_link, user_service, token, job_id, locale)
|
||||
return ReviewLinkResponse(url=url, status=status, review_started_by=started_by)
|
||||
|
||||
|
||||
def get_owned_review_job(user_service: UserService, session_token: str, job_id: str):
|
||||
"""Blocking: the session owner's job, checked to be in review → (job, status).
|
||||
|
||||
Shared by kjbox's review link and the mobile app's web handoff
|
||||
(``POST /api/users/auth/handoff-token``). Raises 401 ``invalid_session``,
|
||||
404 ``not_found`` (missing or not yours — same answer, so other users' job
|
||||
ids aren't confirmed) or 409 ``not_in_review``.
|
||||
"""
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
valid, user, _msg = user_service.validate_session(session_token)
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail="invalid_session")
|
||||
job = JobManager().get_job(job_id)
|
||||
if not job or (job.user_email or "").lower() != user.email.lower():
|
||||
raise HTTPException(status_code=404, detail="not_found")
|
||||
status = str(getattr(job.status, "value", job.status))
|
||||
if status not in _REVIEW_LINK_STATUSES:
|
||||
raise HTTPException(status_code=409, detail="not_in_review")
|
||||
return job, status
|
||||
|
||||
|
||||
def _mint_review_link(user_service: UserService, token: str, job_id: str, locale: str):
|
||||
"""Blocking half of ``review_link`` → (url, status, review_started_by)."""
|
||||
from backend.services.job_notification_service import build_review_login_url
|
||||
|
||||
job, status = get_owned_review_job(user_service, token, job_id)
|
||||
url = build_review_login_url(job.job_id, job.user_email, locale=locale)
|
||||
if not url:
|
||||
raise HTTPException(status_code=502, detail="link_unavailable")
|
||||
started_by = (job.state_data or {}).get("review_started_by") if status == "in_review" else None
|
||||
return url, status, started_by
|
||||
|
||||
|
||||
# ---------------------------------------------------------- search resolve
|
||||
|
||||
class ResolveRequest(BaseModel):
|
||||
query: str = Field(min_length=1, max_length=200)
|
||||
|
||||
|
||||
_RESOLVE_CACHE_MAX = 2000
|
||||
_RESOLVE_PER_MINUTE = 120
|
||||
_resolve_cache: "dict[str, dict]" = {}
|
||||
_RESOLVE_DECIDE_PER_MINUTE = 60 # karaoke-decide's own bucket, so kjbox nights are never starved
|
||||
_resolve_calls: "list[float]" = []
|
||||
_resolve_calls_decide: "list[float]" = []
|
||||
_resolve_lock = threading.Lock()
|
||||
|
||||
|
||||
def _resolve_budget_ok(partner: str = "kjbox") -> bool:
|
||||
"""Per-partner in-process cap on AI calls (cache hits are free)."""
|
||||
import time
|
||||
now = time.monotonic()
|
||||
if partner == "decide":
|
||||
calls, limit = _resolve_calls_decide, _RESOLVE_DECIDE_PER_MINUTE
|
||||
else:
|
||||
calls, limit = _resolve_calls, _RESOLVE_PER_MINUTE
|
||||
with _resolve_lock:
|
||||
while calls and calls[0] < now - 60:
|
||||
calls.pop(0)
|
||||
if len(calls) >= limit:
|
||||
return False
|
||||
calls.append(now)
|
||||
return True
|
||||
|
||||
|
||||
@router.post("/catalog/resolve")
|
||||
async def catalog_resolve(body: ResolveRequest, partner: str = Depends(require_resolve_partner)):
|
||||
"""Split + auto-correct a singer's free-text song search.
|
||||
|
||||
kjbox calls this only when its own catalogue search found nothing, then
|
||||
shows gen's AudioSourceStep "Tidied to … / keep what I typed" (re-searching
|
||||
when the song changed, and pre-filling its generate form). The AI splits the
|
||||
one-line query; the job flow's judge_match catalog pass then tidies it, so
|
||||
the canonical artist/title match what gen's own job form would use.
|
||||
Response = the match judge verdict shape + typed_artist / typed_title.
|
||||
Never 5xx on model trouble: kind "none".
|
||||
"""
|
||||
from backend.services.match_judge.free_text import _none, resolve_and_tidy
|
||||
|
||||
query = " ".join(body.query.split())
|
||||
if not query:
|
||||
raise HTTPException(status_code=422, detail="empty_query")
|
||||
from backend.services.fake_workers import fake_workers_enabled
|
||||
if fake_workers_enabled(get_settings()):
|
||||
# Local E2E stack: scripted verdict, no Gemini, no cache/budget (tests change the knobs).
|
||||
from backend.services.fake_workers.judge import fake_resolve
|
||||
return fake_resolve(query)
|
||||
key = query.casefold() # shared by both partners: same query -> same verdict
|
||||
with _resolve_lock:
|
||||
hit = _resolve_cache.get(key)
|
||||
if hit is not None:
|
||||
# Same query modulo case/spacing: the verdict applies, but echo THIS
|
||||
# request's text where the verdict fell back to the typed query.
|
||||
if not hit.get("typed_artist"):
|
||||
hit = {**hit, "typed_title": query}
|
||||
return hit
|
||||
if not _resolve_budget_ok(partner):
|
||||
raise HTTPException(status_code=429, detail="rate_limited")
|
||||
verdict = await resolve_and_tidy(query)
|
||||
if verdict.get("reason") != "unavailable": # don't cache transient failures
|
||||
with _resolve_lock:
|
||||
if len(_resolve_cache) >= _RESOLVE_CACHE_MAX:
|
||||
_resolve_cache.pop(next(iter(_resolve_cache)))
|
||||
_resolve_cache[key] = verdict
|
||||
return verdict or _none(query)
|
||||
@@ -1,628 +0,0 @@
|
||||
"""gen app sign-in: email code (``send-code`` + ``verify-code``), Google / Apple
|
||||
ID tokens, and the app → web handoff link.
|
||||
|
||||
An alternative to the magic link for clients that can't follow an emailed link
|
||||
back into the same session (the mobile app, a second device, local E2E). The
|
||||
email carries BOTH a 6-digit code and a regular magic link; either signs in.
|
||||
|
||||
- POST /api/users/auth/send-code {email, locale?, device_fingerprint?,
|
||||
referral_code?, attribution?} → 202 {"status": "sent"}
|
||||
- POST /api/users/auth/verify-code {email, code} → VerifyMagicLinkResponse
|
||||
- POST /api/users/auth/google | apple {id_token, locale?, device_fingerprint?,
|
||||
referral_code?, attribution?} → VerifyMagicLinkResponse
|
||||
- POST /api/users/auth/handoff-token {target, job_id, locale?} (Bearer)
|
||||
→ {url, expires_in_seconds}: the mobile app's
|
||||
2-minute, single-use signed-in web link
|
||||
|
||||
Code mechanics come from :class:`LoginCodeStore` (shared with kjbox): HMAC-hashed
|
||||
at rest with a server pepper, 10-minute TTL, 5 verify attempts then 429,
|
||||
single-use marker, and per-email + per-IP rolling-hour send throttles checked in
|
||||
the same Firestore transaction as the code write.
|
||||
|
||||
Abuse checks mirror ``POST /auth/magic-link``: disposable domain → 422 (unless a
|
||||
tenant explicitly allowlisted the address), blocked email / IP → pretend
|
||||
success, new-user per-IP/fingerprint signup cap → pretend success, tenant
|
||||
email-domain restriction → pretend success, no email provider in production →
|
||||
503. Pretend-success responses are indistinguishable from a real send.
|
||||
|
||||
Account creation: the email's magic link is minted with the normal
|
||||
``UserService.create_magic_link`` machinery, which (exactly like the magic-link
|
||||
endpoint) creates the account at send time, recording signup IP, device
|
||||
fingerprint and first-touch attribution. That keeps magic-link parity (same
|
||||
signup caps, same audit fields) instead of inventing a lazily-minted link.
|
||||
verify-code still re-creates the account from the code record if it's gone.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import threading
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Annotated, Dict, Literal, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Request
|
||||
from pydantic import BaseModel, Field, StringConstraints
|
||||
|
||||
from backend.api.routes.kjbox import _normalize_email, _split_locale, get_owned_review_job
|
||||
from backend.api.routes.users import (
|
||||
_build_user_public,
|
||||
_mask_email,
|
||||
_precompute_credit_eval,
|
||||
complete_verified_login,
|
||||
)
|
||||
from backend.config import get_settings, is_production
|
||||
from backend.i18n import WEB_LOCALES, get_full_locale_from_request, get_locale_from_request, t
|
||||
from backend.models.user import VerifyMagicLinkResponse
|
||||
from backend.services.email_service import EmailService, get_email_service
|
||||
from backend.services.email_validation_service import get_email_validation_service
|
||||
from backend.services.fake_workers import fake_workers_enabled
|
||||
from backend.services import social_identity
|
||||
from backend.services.login_code_store import (
|
||||
CODE_EXPIRY_MINUTES,
|
||||
CodeThrottled,
|
||||
LoginCodeStore,
|
||||
TransactionContention,
|
||||
VerifyOutcome,
|
||||
)
|
||||
from backend.services.user_service import MAGIC_LINKS_COLLECTION, UserService, get_user_service
|
||||
from backend.utils.request_helpers import get_client_ip
|
||||
from backend.utils.test_data import is_test_email
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/users", tags=["users"])
|
||||
|
||||
LOGIN_CODES_COLLECTION = "login_codes"
|
||||
LOGIN_CODE_USES_COLLECTION = "login_code_uses"
|
||||
LOGIN_CODE_IP_COLLECTION = "login_code_ip_sends"
|
||||
|
||||
# Used only outside production when neither LOGIN_CODE_PEPPER nor an admin token
|
||||
# is configured (local dev / unit tests). Never used in production.
|
||||
_DEV_PEPPER = "gen-login-code-dev-pepper"
|
||||
|
||||
|
||||
def _resolve_pepper() -> Optional[str]:
|
||||
"""HMAC pepper for stored codes.
|
||||
|
||||
Order: ``LOGIN_CODE_PEPPER`` → sha256("gen-login-code-pepper:" + first admin
|
||||
token) → (non-production only) a fixed dev pepper. Returns None in production
|
||||
with nothing configured (endpoints then 503). Rotating the first admin token
|
||||
without setting LOGIN_CODE_PEPPER invalidates outstanding codes (≤10 min).
|
||||
"""
|
||||
settings = get_settings()
|
||||
if settings.login_code_pepper:
|
||||
return settings.login_code_pepper
|
||||
tokens = [tok.strip() for tok in (settings.admin_tokens or "").split(",") if tok.strip()]
|
||||
if tokens:
|
||||
return hashlib.sha256(f"gen-login-code-pepper:{tokens[0]}".encode("utf-8")).hexdigest()
|
||||
if not is_production():
|
||||
return _DEV_PEPPER
|
||||
return None
|
||||
|
||||
|
||||
def get_login_code_store(user_service: UserService = Depends(get_user_service)) -> LoginCodeStore:
|
||||
pepper = _resolve_pepper()
|
||||
if not pepper:
|
||||
logger.error("login-code: no LOGIN_CODE_PEPPER / admin token configured in production")
|
||||
raise HTTPException(status_code=503, detail="not configured")
|
||||
return LoginCodeStore(
|
||||
user_service.db,
|
||||
pepper,
|
||||
LOGIN_CODES_COLLECTION,
|
||||
LOGIN_CODE_USES_COLLECTION,
|
||||
ip_collection=LOGIN_CODE_IP_COLLECTION,
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- models
|
||||
|
||||
class SendLoginCodeRequest(BaseModel):
|
||||
email: str
|
||||
locale: Optional[str] = None
|
||||
device_fingerprint: Optional[str] = Field(default=None, max_length=256)
|
||||
referral_code: Optional[str] = Field(default=None, max_length=64)
|
||||
# First-touch attribution, same shape/limits as SendMagicLinkRequest.attribution.
|
||||
attribution: Optional[
|
||||
Dict[Annotated[str, StringConstraints(max_length=32)], Optional[Annotated[str, StringConstraints(max_length=500)]]]
|
||||
] = Field(default=None, max_length=16)
|
||||
|
||||
|
||||
class SendLoginCodeResponse(BaseModel):
|
||||
status: str
|
||||
|
||||
|
||||
class VerifyLoginCodeRequest(BaseModel):
|
||||
email: str
|
||||
code: str = Field(max_length=32)
|
||||
|
||||
|
||||
def _sent() -> SendLoginCodeResponse:
|
||||
return SendLoginCodeResponse(status="sent")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ endpoints
|
||||
|
||||
@router.post("/auth/send-code", response_model=SendLoginCodeResponse, status_code=202)
|
||||
def send_login_code(
|
||||
body: SendLoginCodeRequest,
|
||||
http_request: Request,
|
||||
store: LoginCodeStore = Depends(get_login_code_store),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""Email a 6-digit sign-in code (plus a magic link) to ``email``."""
|
||||
from backend.middleware.tenant import get_tenant_config_from_request, get_tenant_from_request
|
||||
|
||||
settings = get_settings()
|
||||
email = _normalize_email(body.email)
|
||||
email_locale, ui_locale = _split_locale(body.locale)
|
||||
has_accept_language = bool(http_request.headers.get("accept-language"))
|
||||
locale = email_locale or get_locale_from_request(http_request)
|
||||
ui_locale = ui_locale or get_full_locale_from_request(http_request)
|
||||
# Locale persisted on the user at verify: only an explicit choice (body or
|
||||
# Accept-Language), never the "en" fallback — that would clobber a stored one.
|
||||
persist_locale = email_locale or (locale if has_accept_language else None)
|
||||
|
||||
tenant_id = get_tenant_from_request(http_request)
|
||||
tenant_config = get_tenant_config_from_request(http_request)
|
||||
ip_address = get_client_ip(http_request)
|
||||
device_fingerprint = body.device_fingerprint
|
||||
|
||||
check = _sign_in_checks(
|
||||
email, http_request, user_service,
|
||||
device_fingerprint=device_fingerprint, tenant_config=tenant_config, log_prefix="login-code",
|
||||
)
|
||||
if check.refusal == "disposable":
|
||||
raise HTTPException(status_code=422, detail=t(locale, "users.disposableEmailBlocked"))
|
||||
if check.refusal == "tenant_disallowed":
|
||||
logger.warning(f"login-code: email not allowed for tenant {tenant_id}: {_mask_email(email)}")
|
||||
return _sent()
|
||||
if check.refusal: # blocked email / IP, new-user signup cap: indistinguishable from a send
|
||||
return _sent()
|
||||
existing_user = check.existing_user
|
||||
|
||||
if not email_service.is_configured() and is_production():
|
||||
logger.error("login-code: email service not configured - cannot send codes")
|
||||
raise HTTPException(status_code=503, detail=t(locale, "users.emailServiceNotConfigured"))
|
||||
|
||||
user_agent = http_request.headers.get("user-agent")
|
||||
try:
|
||||
issued = store.issue_code(
|
||||
email,
|
||||
max_per_hour=settings.login_codes_per_email_per_hour,
|
||||
needs_credit_eval=(
|
||||
not (existing_user and getattr(existing_user, "welcome_credits_granted", False))
|
||||
and not is_test_email(email)
|
||||
and not tenant_id
|
||||
),
|
||||
fields={
|
||||
"locale": persist_locale,
|
||||
"email_locale": locale,
|
||||
"ui_locale": ui_locale,
|
||||
"ip_address": ip_address,
|
||||
"user_agent": user_agent,
|
||||
"device_fingerprint": device_fingerprint,
|
||||
"tenant_id": tenant_id,
|
||||
"referral_code": body.referral_code,
|
||||
},
|
||||
throttle_ip=ip_address,
|
||||
max_per_ip_per_hour=settings.login_codes_per_ip_per_hour,
|
||||
)
|
||||
except (CodeThrottled, TransactionContention):
|
||||
# Contention on one email's code doc = a burst of parallel sends to the
|
||||
# same address, which is exactly what the throttle exists to stop.
|
||||
raise HTTPException(status_code=429, detail="too_many_codes")
|
||||
|
||||
# Same machinery as POST /auth/magic-link (creates the account at send time,
|
||||
# recording signup IP / fingerprint / attribution) — see module docstring.
|
||||
magic_link = user_service.create_magic_link(
|
||||
email,
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent,
|
||||
tenant_id=tenant_id,
|
||||
device_fingerprint=device_fingerprint,
|
||||
referral_code=body.referral_code,
|
||||
attribution=body.attribution,
|
||||
)
|
||||
magic_link_url = email_service.build_magic_link_url(
|
||||
magic_link.token,
|
||||
locale=locale,
|
||||
tenant_frontend_url=tenant_config.get_frontend_url() if tenant_config else None,
|
||||
referral_code=body.referral_code,
|
||||
)
|
||||
sent = email_service.send_login_code(
|
||||
email,
|
||||
issued.code,
|
||||
magic_link_url=magic_link_url,
|
||||
expiry_minutes=CODE_EXPIRY_MINUTES,
|
||||
locale=locale,
|
||||
sender_email=tenant_config.get_sender_email() if tenant_config else None,
|
||||
tenant_name=tenant_config.name if tenant_config else None,
|
||||
)
|
||||
if not sent:
|
||||
if getattr(email_service, "last_send_suppressed", False):
|
||||
logger.info(f"login-code: recipient {_mask_email(email)} is Postmark-suppressed")
|
||||
else:
|
||||
logger.error(f"login-code: failed to send code email to {_mask_email(email)}")
|
||||
# Don't reveal failure (anti-enumeration).
|
||||
|
||||
# Warm the welcome-credit AI evaluation while the user reads their email, on
|
||||
# both sign-in docs (code record + magic link) so either path is instant.
|
||||
if issued.needs_credit_eval:
|
||||
threading.Thread(
|
||||
target=_precompute_credit_eval,
|
||||
args=(issued.doc_id, email, LOGIN_CODES_COLLECTION, ((MAGIC_LINKS_COLLECTION, magic_link.token),)),
|
||||
daemon=True,
|
||||
).start()
|
||||
|
||||
return _sent()
|
||||
|
||||
|
||||
@router.post("/auth/verify-code", response_model=VerifyMagicLinkResponse)
|
||||
def verify_login_code(
|
||||
body: VerifyLoginCodeRequest,
|
||||
http_request: Request,
|
||||
store: LoginCodeStore = Depends(get_login_code_store),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""Exchange an emailed code for a session (same response as GET /auth/verify)."""
|
||||
email = _normalize_email(body.email)
|
||||
|
||||
settings = get_settings()
|
||||
demo_email = (settings.app_review_demo_email or "").strip().lower()
|
||||
demo_code = settings.app_review_demo_code or ""
|
||||
if demo_email and demo_code and email == demo_email:
|
||||
# App Review demo account: reviewers can't receive email, so this one
|
||||
# address signs in with the fixed code. Attempt limits still apply.
|
||||
outcome, record = store.verify_fixed_code(email, body.code, demo_code)
|
||||
if outcome == VerifyOutcome.OK:
|
||||
logger.info("app review demo login")
|
||||
else:
|
||||
outcome, record = store.verify_code(email, body.code)
|
||||
if outcome == VerifyOutcome.TOO_MANY_ATTEMPTS:
|
||||
raise HTTPException(status_code=429, detail="too_many_attempts")
|
||||
if outcome == VerifyOutcome.INVALID:
|
||||
raise HTTPException(status_code=401, detail="invalid_code")
|
||||
if outcome != VerifyOutcome.OK or record is None:
|
||||
raise HTTPException(status_code=401, detail="expired")
|
||||
|
||||
tenant_id = record.get("tenant_id")
|
||||
user = user_service.get_user(email)
|
||||
if user is None:
|
||||
# Normally created at send time by create_magic_link; recreate from the
|
||||
# code record (same signup audit fields) if it was removed meanwhile.
|
||||
user = user_service.get_or_create_user(
|
||||
email,
|
||||
tenant_id=tenant_id,
|
||||
signup_ip=record.get("ip_address"),
|
||||
device_fingerprint=record.get("device_fingerprint"),
|
||||
)
|
||||
precomputed_eval = {
|
||||
k: record.get(k)
|
||||
for k in ("credit_eval_decision", "credit_eval_reasoning", "credit_eval_error")
|
||||
if record.get(k) is not None
|
||||
} or None
|
||||
persist_locale = record.get("locale")
|
||||
return finish_verified_sign_in(
|
||||
user,
|
||||
http_request,
|
||||
user_service=user_service,
|
||||
email_service=email_service,
|
||||
tenant_id=tenant_id,
|
||||
precomputed_eval=precomputed_eval,
|
||||
referral_code=http_request.headers.get("x-referral-code") or record.get("referral_code"),
|
||||
persist_locale=persist_locale,
|
||||
ui_locale=record.get("ui_locale"),
|
||||
email_locale=persist_locale or record.get("email_locale"),
|
||||
device_fingerprint=record.get("device_fingerprint"),
|
||||
log_prefix="login-code",
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------- shared sign-in steps
|
||||
|
||||
@dataclass
|
||||
class SignInCheck:
|
||||
"""Outcome of :func:`_sign_in_checks`; ``refusal`` is None when sign-in may go ahead."""
|
||||
refusal: Optional[str] # disposable | blocked_email | blocked_ip | signup_limited | tenant_disallowed
|
||||
existing_user: object = None
|
||||
|
||||
|
||||
APPLE_RELAY_DOMAIN = "privaterelay.appleid.com"
|
||||
|
||||
|
||||
def _sign_in_checks(
|
||||
email: str,
|
||||
http_request: Request,
|
||||
user_service: UserService,
|
||||
*,
|
||||
device_fingerprint: Optional[str],
|
||||
tenant_config,
|
||||
log_prefix: str,
|
||||
allow_apple_relay: bool = False,
|
||||
) -> SignInCheck:
|
||||
"""Abuse / eligibility checks shared by every sign-in that can create an account
|
||||
(email code, Google, Apple), mirroring ``POST /auth/magic-link``: disposable
|
||||
domain (unless a tenant explicitly allowlisted the address, or it's an Apple
|
||||
"Hide My Email" relay on the Apple route), blocked email, blocked IP, new-user
|
||||
per-IP/fingerprint signup cap, tenant email-domain restriction. Blocking
|
||||
Firestore / DeBounce calls: run from sync routes only.
|
||||
"""
|
||||
email_validation = get_email_validation_service()
|
||||
vouched = bool(tenant_config and tenant_config.is_explicitly_allowlisted(email))
|
||||
if allow_apple_relay and email.rsplit("@", 1)[-1] == APPLE_RELAY_DOMAIN:
|
||||
vouched = True # Apple-issued forwarding address tied to a real Apple ID
|
||||
if not vouched and email_validation.is_disposable_domain(email):
|
||||
logger.warning(f"{log_prefix}: blocked disposable email {_mask_email(email)}")
|
||||
return SignInCheck("disposable")
|
||||
if email_validation.is_email_blocked(email):
|
||||
logger.warning(f"{log_prefix}: blocked email {_mask_email(email)}")
|
||||
return SignInCheck("blocked_email")
|
||||
ip_address = get_client_ip(http_request)
|
||||
if ip_address and email_validation.is_ip_blocked(ip_address):
|
||||
logger.warning(f"{log_prefix}: blocked IP {ip_address}")
|
||||
return SignInCheck("blocked_ip")
|
||||
existing_user = user_service.get_user(email)
|
||||
if existing_user is None and user_service.is_signup_rate_limited(
|
||||
ip_address=ip_address, device_fingerprint=device_fingerprint,
|
||||
):
|
||||
logger.warning(f"{log_prefix}: signup rate limit hit: IP={ip_address} for {_mask_email(email)}")
|
||||
return SignInCheck("signup_limited")
|
||||
if tenant_config and not tenant_config.is_email_allowed(email):
|
||||
return SignInCheck("tenant_disallowed")
|
||||
return SignInCheck(None, existing_user)
|
||||
|
||||
|
||||
def finish_verified_sign_in(
|
||||
user,
|
||||
http_request: Request,
|
||||
*,
|
||||
user_service: UserService,
|
||||
email_service: EmailService,
|
||||
tenant_id: Optional[str],
|
||||
precomputed_eval: Optional[dict],
|
||||
referral_code: Optional[str],
|
||||
persist_locale: Optional[str],
|
||||
ui_locale: Optional[str],
|
||||
email_locale: Optional[str],
|
||||
device_fingerprint: Optional[str],
|
||||
log_prefix: str,
|
||||
) -> VerifyMagicLinkResponse:
|
||||
"""After an identity proof (email code, Google or Apple ID token): mark the
|
||||
user verified, then the shared ``complete_verified_login`` (welcome credit,
|
||||
referral, locale, session, welcome email) and the ``VerifyMagicLinkResponse``.
|
||||
|
||||
``user`` must already exist. Tenant portal sign-ins never touch consumer
|
||||
credits (as in magic-link verify).
|
||||
"""
|
||||
email = user.email
|
||||
# Same first-login definition as magic-link verify (checked before last_login_at is set).
|
||||
is_first_login = user.total_jobs_created == 0 and not user.last_login_at
|
||||
user = user_service.update_user(
|
||||
email, email_verified=True, last_login_at=datetime.utcnow(),
|
||||
) or user
|
||||
email_locale = email_locale or user.locale or "en"
|
||||
|
||||
result = complete_verified_login(
|
||||
user,
|
||||
user_service=user_service,
|
||||
email_service=email_service,
|
||||
is_first_login=is_first_login,
|
||||
precomputed_eval=precomputed_eval,
|
||||
grant_welcome_credit=not tenant_id,
|
||||
referral_code=referral_code if not tenant_id else None,
|
||||
locale=persist_locale,
|
||||
ui_locale=ui_locale,
|
||||
ip_address=get_client_ip(http_request),
|
||||
user_agent=http_request.headers.get("user-agent"),
|
||||
tenant_id=tenant_id,
|
||||
device_fingerprint=device_fingerprint,
|
||||
email_locale=email_locale,
|
||||
)
|
||||
logger.info(f"{log_prefix}: verified {_mask_email(email)} (credit_status={result.credit_status})")
|
||||
|
||||
tenant_subdomain = None
|
||||
if tenant_id:
|
||||
from backend.services.tenant_service import get_tenant_service
|
||||
tenant_cfg = get_tenant_service().get_tenant_config(tenant_id)
|
||||
if tenant_cfg:
|
||||
tenant_subdomain = tenant_cfg.subdomain
|
||||
|
||||
return VerifyMagicLinkResponse(
|
||||
status="success",
|
||||
session_token=result.session.token,
|
||||
user=_build_user_public(result.user),
|
||||
message=t(email_locale, "users.magicLinkSuccess"),
|
||||
tenant_subdomain=tenant_subdomain,
|
||||
credits_granted=result.credits_granted,
|
||||
credit_status=result.credit_status,
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------- web handoff
|
||||
|
||||
# App → web handoff links carry a real sign-in token in the URL, so they live
|
||||
# only long enough for the app to open the browser, and are single-use (the
|
||||
# magic-link verify transaction marks them used).
|
||||
HANDOFF_TOKEN_TTL_SECONDS = 120
|
||||
|
||||
|
||||
class HandoffTokenRequest(BaseModel):
|
||||
# Allowlist of web destinations; Phase 1 needs only the job's lyrics review.
|
||||
target: Literal["job_review"]
|
||||
job_id: str = Field(min_length=1, max_length=64, pattern=r"^[A-Za-z0-9_-]+$")
|
||||
locale: Optional[str] = None
|
||||
|
||||
|
||||
class HandoffTokenResponse(BaseModel):
|
||||
url: str
|
||||
expires_in_seconds: int
|
||||
|
||||
|
||||
def _bearer_token(authorization: Optional[str]) -> str:
|
||||
scheme, _, token = (authorization or "").partition(" ")
|
||||
token = token.strip()
|
||||
if scheme.lower() != "bearer" or not token:
|
||||
raise HTTPException(status_code=401, detail="invalid_session")
|
||||
return token
|
||||
|
||||
|
||||
@router.post("/auth/handoff-token", response_model=HandoffTokenResponse)
|
||||
def create_handoff_token(
|
||||
body: HandoffTokenRequest,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
authorization: Optional[str] = Header(default=None),
|
||||
):
|
||||
"""Signed-in web link for the app's own job review (the mobile "Review on web").
|
||||
|
||||
The URL is ``{FRONTEND_URL}/{locale}/auth/verify?token=…&handoff=1``: a
|
||||
``job_review:<id>`` magic-link token valid for ``HANDOFF_TOKEN_TTL_SECONDS``
|
||||
and usable once. ``handoff=1`` makes the verify page sign in without the
|
||||
email-scanner "Complete Sign-In" tap; a used or expired link shows the
|
||||
normal "link expired" state. 401 ``invalid_session``, 404 ``not_found``
|
||||
(missing or not the caller's job), 409 ``not_in_review``.
|
||||
"""
|
||||
from backend.services.job_notification_service import build_review_login_url
|
||||
|
||||
token = _bearer_token(authorization)
|
||||
# Any of the web app's 33 locales (a French app user lands on /fr/…/review).
|
||||
locale = body.locale if body.locale in WEB_LOCALES else "en"
|
||||
# Sync route (threadpool): session check, job read and mint are blocking Firestore calls.
|
||||
job, _status = get_owned_review_job(user_service, token, body.job_id)
|
||||
url = build_review_login_url(
|
||||
job.job_id, job.user_email, locale=locale, handoff_seconds=HANDOFF_TOKEN_TTL_SECONDS,
|
||||
)
|
||||
if not url:
|
||||
raise HTTPException(status_code=502, detail="link_unavailable")
|
||||
return HandoffTokenResponse(url=url, expires_in_seconds=HANDOFF_TOKEN_TTL_SECONDS)
|
||||
|
||||
|
||||
# ------------------------------------------------------- Google / Apple sign-in
|
||||
|
||||
class SocialSignInRequest(BaseModel):
|
||||
# The ID token from the native Google / Apple SDK (or ``fake:<email>`` on the local stack).
|
||||
id_token: str = Field(min_length=1, max_length=8192)
|
||||
locale: Optional[str] = None
|
||||
device_fingerprint: Optional[str] = Field(default=None, max_length=256)
|
||||
referral_code: Optional[str] = Field(default=None, max_length=64)
|
||||
attribution: Optional[
|
||||
Dict[Annotated[str, StringConstraints(max_length=32)], Optional[Annotated[str, StringConstraints(max_length=500)]]]
|
||||
] = Field(default=None, max_length=16)
|
||||
|
||||
|
||||
def _verify_identity(provider: str, id_token: str) -> "social_identity.VerifiedIdentity":
|
||||
"""ID token → verified identity, or 401 ``invalid_token`` / 503 ``google_not_configured``."""
|
||||
settings = get_settings()
|
||||
try:
|
||||
if id_token.startswith(social_identity.FAKE_TOKEN_PREFIX):
|
||||
# Local stack only (FAKE_WORKERS, refused in production); never a network call.
|
||||
if not fake_workers_enabled():
|
||||
raise social_identity.InvalidIdentityToken("fake token outside fake mode")
|
||||
return social_identity.fake_identity(id_token, provider)
|
||||
if provider == "google":
|
||||
audiences = social_identity.parse_audiences(getattr(settings, "google_oauth_client_ids", ""))
|
||||
if not audiences:
|
||||
raise HTTPException(status_code=503, detail="google_not_configured")
|
||||
return social_identity.verify_google_id_token(id_token, audiences)
|
||||
audiences = social_identity.parse_audiences(getattr(settings, "apple_signin_audiences", ""))
|
||||
return social_identity.verify_apple_id_token(id_token, audiences)
|
||||
except social_identity.InvalidIdentityToken as e:
|
||||
logger.warning(f"{provider}-signin: {e}")
|
||||
raise HTTPException(status_code=401, detail="invalid_token")
|
||||
except social_identity.IdentityProviderUnavailable as e:
|
||||
logger.error(f"{provider}-signin: {e}")
|
||||
raise HTTPException(status_code=503, detail="identity_provider_unavailable")
|
||||
|
||||
|
||||
def _social_sign_in(
|
||||
provider: str,
|
||||
body: SocialSignInRequest,
|
||||
http_request: Request,
|
||||
user_service: UserService,
|
||||
email_service: EmailService,
|
||||
) -> VerifyMagicLinkResponse:
|
||||
"""Verify the ID token, apply the same abuse checks as send-code, find or
|
||||
create the user BY EMAIL, then the same session path as verify-code."""
|
||||
from backend.middleware.tenant import get_tenant_config_from_request, get_tenant_from_request
|
||||
|
||||
log_prefix = f"{provider}-signin"
|
||||
identity = _verify_identity(provider, body.id_token)
|
||||
email = identity.email
|
||||
|
||||
email_locale, ui_locale = _split_locale(body.locale)
|
||||
request_locale = get_locale_from_request(http_request)
|
||||
has_accept_language = bool(http_request.headers.get("accept-language"))
|
||||
persist_locale = email_locale or (request_locale if has_accept_language else None)
|
||||
ui_locale = ui_locale or get_full_locale_from_request(http_request)
|
||||
message_locale = email_locale or request_locale
|
||||
|
||||
tenant_id = get_tenant_from_request(http_request)
|
||||
tenant_config = get_tenant_config_from_request(http_request)
|
||||
check = _sign_in_checks(
|
||||
email, http_request, user_service,
|
||||
device_fingerprint=body.device_fingerprint, tenant_config=tenant_config,
|
||||
log_prefix=log_prefix, allow_apple_relay=provider == "apple",
|
||||
)
|
||||
# Unlike send-code there's no email to hide behind, so refusals are explicit.
|
||||
if check.refusal == "disposable":
|
||||
raise HTTPException(status_code=422, detail=t(message_locale, "users.disposableEmailBlocked"))
|
||||
if check.refusal == "signup_limited":
|
||||
raise HTTPException(status_code=429, detail="signup_rate_limited")
|
||||
if check.refusal: # blocked email / IP, tenant email restriction
|
||||
raise HTTPException(status_code=422, detail="sign_in_not_allowed")
|
||||
|
||||
user = check.existing_user or user_service.get_or_create_user_with_attribution(
|
||||
email,
|
||||
tenant_id=tenant_id,
|
||||
signup_ip=get_client_ip(http_request),
|
||||
device_fingerprint=body.device_fingerprint,
|
||||
attribution=body.attribution,
|
||||
)
|
||||
return finish_verified_sign_in(
|
||||
user,
|
||||
http_request,
|
||||
user_service=user_service,
|
||||
email_service=email_service,
|
||||
tenant_id=tenant_id,
|
||||
precomputed_eval=None,
|
||||
referral_code=http_request.headers.get("x-referral-code") or body.referral_code,
|
||||
persist_locale=persist_locale,
|
||||
ui_locale=ui_locale,
|
||||
email_locale=persist_locale,
|
||||
device_fingerprint=body.device_fingerprint,
|
||||
log_prefix=log_prefix,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/auth/google", response_model=VerifyMagicLinkResponse)
|
||||
def google_sign_in(
|
||||
body: SocialSignInRequest,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""Exchange a Google ID token for a gen session (same response as verify-code).
|
||||
|
||||
Audience = ``GOOGLE_OAUTH_CLIENT_IDS``. 401 ``invalid_token`` (bad signature,
|
||||
issuer, audience, expiry, or unverified email), 503 ``google_not_configured``,
|
||||
503 ``identity_provider_unavailable`` (couldn't fetch the provider's keys;
|
||||
retryable), 422 disposable / blocked, 429 ``signup_rate_limited`` (new accounts only).
|
||||
"""
|
||||
return _social_sign_in("google", body, http_request, user_service, email_service)
|
||||
|
||||
|
||||
@router.post("/auth/apple", response_model=VerifyMagicLinkResponse)
|
||||
def apple_sign_in(
|
||||
body: SocialSignInRequest,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""Exchange an Apple ID token for a gen session (same response as verify-code).
|
||||
|
||||
Audience = ``APPLE_SIGNIN_AUDIENCES``. "Hide My Email" relay addresses
|
||||
(``@privaterelay.appleid.com``) are accepted as their own account. Errors as
|
||||
for ``/auth/google`` (no 503: the Apple audience has a default).
|
||||
"""
|
||||
return _social_sign_in("apple", body, http_request, user_service, email_service)
|
||||
@@ -1,52 +0,0 @@
|
||||
"""POST /api/parse-karaoke-titles — batch karaoke-filename → artist/title.
|
||||
|
||||
Internal admin endpoint used by kjbox to canonicalise downloaded-file names.
|
||||
Reuses the Vertex Gemini parser; never blocks the caller (kjbox degrades to a
|
||||
deterministic guess when this is unavailable).
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.services.auth_service import AuthResult
|
||||
from backend.services.parse_titles import parse_titles
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(tags=["parse-titles"])
|
||||
|
||||
|
||||
class ParseItem(BaseModel):
|
||||
id: str
|
||||
filename: str
|
||||
channel: Optional[str] = None
|
||||
source: Optional[str] = None
|
||||
|
||||
|
||||
class ParseRequest(BaseModel):
|
||||
items: list[ParseItem]
|
||||
|
||||
|
||||
class ParseResult(BaseModel):
|
||||
id: str
|
||||
artist: str
|
||||
title: str
|
||||
confidence: float
|
||||
|
||||
|
||||
class ParseResponse(BaseModel):
|
||||
results: list[ParseResult]
|
||||
|
||||
|
||||
@router.post("/parse-karaoke-titles", response_model=ParseResponse)
|
||||
async def parse_karaoke_titles(
|
||||
body: ParseRequest,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
items = [i.model_dump() for i in body.items]
|
||||
results = await parse_titles(items)
|
||||
logger.info("parse-karaoke-titles: %d items", len(items))
|
||||
return {"results": results}
|
||||
+21
-87
@@ -5,23 +5,19 @@ Provides endpoints for managing Web Push notification subscriptions:
|
||||
- GET /api/push/vapid-public-key: Get VAPID public key for client-side subscription
|
||||
- POST /api/push/subscribe: Register a push subscription
|
||||
- POST /api/push/unsubscribe: Remove a push subscription
|
||||
- POST/DELETE /api/push/expo-token: Register/remove an Expo push token (mobile app)
|
||||
- GET /api/push/subscriptions: List user's subscriptions
|
||||
- POST /api/push/test: Send a test notification (admin only)
|
||||
"""
|
||||
import logging
|
||||
import asyncio
|
||||
from typing import Literal, Optional, Dict, List
|
||||
from typing import Optional, Dict, List
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
from pydantic import BaseModel, Field
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from backend.config import get_settings
|
||||
from backend.api.dependencies import require_auth, require_admin
|
||||
from backend.services.auth_service import AuthResult
|
||||
from backend.services.push_notification_service import get_push_notification_service
|
||||
from backend.services.expo_push_service import get_expo_push_service
|
||||
from backend.i18n import t, get_locale_from_request
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -41,7 +37,6 @@ class SubscribeRequest(BaseModel):
|
||||
endpoint: str
|
||||
keys: Dict[str, str] # p256dh and auth
|
||||
device_name: Optional[str] = None
|
||||
tenant_id: Optional[str] = None # Tenant scope for notification isolation
|
||||
|
||||
|
||||
class SubscribeResponse(BaseModel):
|
||||
@@ -88,21 +83,10 @@ class TestNotificationResponse(BaseModel):
|
||||
message: str
|
||||
|
||||
|
||||
class ExpoTokenRequest(BaseModel):
|
||||
"""Register an Expo push token for one device of the mobile app."""
|
||||
token: str = Field(pattern=r"^Expo(nent)?PushToken\[[^\]\s]+\]$")
|
||||
platform: Literal["ios", "android"]
|
||||
device_id: str = Field(min_length=1, max_length=200)
|
||||
|
||||
|
||||
class ExpoTokenDeleteRequest(BaseModel):
|
||||
token: str
|
||||
|
||||
|
||||
# Routes
|
||||
|
||||
@router.get("/vapid-public-key", response_model=VapidPublicKeyResponse)
|
||||
def get_vapid_public_key():
|
||||
async def get_vapid_public_key():
|
||||
"""
|
||||
Get the VAPID public key for push subscription.
|
||||
|
||||
@@ -127,8 +111,7 @@ def get_vapid_public_key():
|
||||
|
||||
@router.post("/subscribe", response_model=SubscribeResponse)
|
||||
async def subscribe_push(
|
||||
subscribe_request: SubscribeRequest,
|
||||
http_request: Request,
|
||||
request: SubscribeRequest,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
@@ -136,53 +119,39 @@ async def subscribe_push(
|
||||
|
||||
Requires authentication. Users can have up to 5 subscriptions
|
||||
(configurable via MAX_PUSH_SUBSCRIPTIONS_PER_USER).
|
||||
|
||||
Subscriptions are scoped by tenant_id to prevent cross-tenant
|
||||
notification leakage. The tenant is resolved from:
|
||||
1. Request body tenant_id field (explicit from frontend)
|
||||
2. Tenant middleware (X-Tenant-ID header or subdomain detection)
|
||||
"""
|
||||
locale = get_locale_from_request(http_request)
|
||||
|
||||
settings = get_settings()
|
||||
if not settings.enable_push_notifications:
|
||||
raise HTTPException(status_code=503, detail=t(locale, "push.disabled"))
|
||||
raise HTTPException(status_code=503, detail="Push notifications are not enabled")
|
||||
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=401, detail=t(locale, "push.userEmailNotAvailable"))
|
||||
raise HTTPException(status_code=401, detail="User email not available")
|
||||
|
||||
push_service = get_push_notification_service()
|
||||
|
||||
# Validate keys
|
||||
if "p256dh" not in subscribe_request.keys or "auth" not in subscribe_request.keys:
|
||||
raise HTTPException(status_code=400, detail=t(locale, "push.missingEncryptionKeys"))
|
||||
|
||||
# Resolve tenant_id: prefer explicit from body, fall back to middleware
|
||||
tenant_id = subscribe_request.tenant_id
|
||||
if tenant_id is None:
|
||||
tenant_id = getattr(http_request.state, "tenant_id", None)
|
||||
if "p256dh" not in request.keys or "auth" not in request.keys:
|
||||
raise HTTPException(status_code=400, detail="Missing required keys (p256dh, auth)")
|
||||
|
||||
success = await push_service.add_subscription(
|
||||
user_email=auth_result.user_email,
|
||||
endpoint=subscribe_request.endpoint,
|
||||
keys=subscribe_request.keys,
|
||||
device_name=subscribe_request.device_name,
|
||||
tenant_id=tenant_id or None,
|
||||
endpoint=request.endpoint,
|
||||
keys=request.keys,
|
||||
device_name=request.device_name
|
||||
)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=500, detail=t(locale, "push.subscriptionSaveFailed"))
|
||||
raise HTTPException(status_code=500, detail="Failed to save subscription")
|
||||
|
||||
return SubscribeResponse(
|
||||
status="success",
|
||||
message=t(locale, "push.subscriptionRegistered")
|
||||
message="Push subscription registered successfully"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/unsubscribe", response_model=UnsubscribeResponse)
|
||||
async def unsubscribe_push(
|
||||
request: UnsubscribeRequest,
|
||||
http_request: Request,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
@@ -190,10 +159,8 @@ async def unsubscribe_push(
|
||||
|
||||
Requires authentication. Users can only remove their own subscriptions.
|
||||
"""
|
||||
locale = get_locale_from_request(http_request)
|
||||
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=401, detail=t(locale, "push.userEmailNotAvailable"))
|
||||
raise HTTPException(status_code=401, detail="User email not available")
|
||||
|
||||
push_service = get_push_notification_service()
|
||||
|
||||
@@ -206,45 +173,17 @@ async def unsubscribe_push(
|
||||
# Don't error if subscription wasn't found - might already be removed
|
||||
return UnsubscribeResponse(
|
||||
status="success",
|
||||
message=t(locale, "push.subscriptionNotFound")
|
||||
message="Subscription removed (or was not found)"
|
||||
)
|
||||
|
||||
return UnsubscribeResponse(
|
||||
status="success",
|
||||
message=t(locale, "push.subscriptionRemoved")
|
||||
message="Push subscription removed successfully"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/expo-token", status_code=204)
|
||||
async def register_expo_token(
|
||||
body: ExpoTokenRequest,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Register this device's Expo token. Moves it off any other account that held it."""
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=401, detail="User email not available")
|
||||
await asyncio.to_thread(
|
||||
get_expo_push_service().register_token,
|
||||
auth_result.user_email, body.token, body.platform, body.device_id,
|
||||
)
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
@router.delete("/expo-token", status_code=204)
|
||||
async def delete_expo_token(
|
||||
body: ExpoTokenDeleteRequest,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Remove one of the caller's Expo tokens (the app calls this before sign-out)."""
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=401, detail="User email not available")
|
||||
await asyncio.to_thread(get_expo_push_service().remove_tokens, auth_result.user_email, [body.token])
|
||||
return Response(status_code=204)
|
||||
|
||||
|
||||
@router.get("/subscriptions", response_model=SubscriptionsListResponse)
|
||||
async def list_subscriptions(
|
||||
request: Request,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
@@ -252,10 +191,8 @@ async def list_subscriptions(
|
||||
|
||||
Requires authentication.
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=401, detail=t(locale, "push.userEmailNotAvailable"))
|
||||
raise HTTPException(status_code=401, detail="User email not available")
|
||||
|
||||
push_service = get_push_notification_service()
|
||||
|
||||
@@ -270,7 +207,6 @@ async def list_subscriptions(
|
||||
@router.post("/test", response_model=TestNotificationResponse)
|
||||
async def send_test_notification(
|
||||
request: TestNotificationRequest,
|
||||
http_request: Request,
|
||||
auth_result: AuthResult = Depends(require_admin)
|
||||
):
|
||||
"""
|
||||
@@ -278,14 +214,12 @@ async def send_test_notification(
|
||||
|
||||
Admin only. Useful for testing push notification setup.
|
||||
"""
|
||||
locale = get_locale_from_request(http_request)
|
||||
|
||||
settings = get_settings()
|
||||
if not settings.enable_push_notifications:
|
||||
raise HTTPException(status_code=503, detail=t(locale, "push.disabled"))
|
||||
raise HTTPException(status_code=503, detail="Push notifications are not enabled")
|
||||
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=401, detail=t(locale, "push.userEmailNotAvailable"))
|
||||
raise HTTPException(status_code=401, detail="User email not available")
|
||||
|
||||
push_service = get_push_notification_service()
|
||||
|
||||
@@ -300,5 +234,5 @@ async def send_test_notification(
|
||||
return TestNotificationResponse(
|
||||
status="success",
|
||||
sent_count=sent_count,
|
||||
message=t(locale, "push.testNotificationSent", sent_count=sent_count)
|
||||
message=f"Test notification sent to {sent_count} device(s)"
|
||||
)
|
||||
+249
-167
@@ -1,22 +1,24 @@
|
||||
"""
|
||||
Admin API routes for blocklist and YouTube queue management.
|
||||
Admin API routes for rate limit management.
|
||||
|
||||
Handles:
|
||||
- Rate limit statistics and monitoring
|
||||
- Blocklist management (disposable domains, blocked emails, blocked IPs)
|
||||
- YouTube upload queue management
|
||||
- User override management (whitelist/bypass permissions)
|
||||
"""
|
||||
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from typing import List, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel, EmailStr
|
||||
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.services.auth_service import AuthResult
|
||||
from backend.services.rate_limit_service import get_rate_limit_service, RateLimitService
|
||||
from backend.services.email_validation_service import get_email_validation_service, EmailValidationService
|
||||
from backend.i18n import t, get_locale_from_request
|
||||
from backend.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/admin/rate-limits", tags=["admin", "rate-limits"])
|
||||
@@ -26,40 +28,45 @@ router = APIRouter(prefix="/admin/rate-limits", tags=["admin", "rate-limits"])
|
||||
# Response Models
|
||||
# =============================================================================
|
||||
|
||||
class YouTubeQueueEntry(BaseModel):
|
||||
"""A queued YouTube upload entry."""
|
||||
job_id: str
|
||||
status: str
|
||||
reason: Optional[str] = None
|
||||
user_email: Optional[str] = None
|
||||
artist: Optional[str] = None
|
||||
title: Optional[str] = None
|
||||
brand_code: Optional[str] = None
|
||||
queued_at: Optional[datetime] = None
|
||||
attempts: int = 0
|
||||
max_attempts: int = 5
|
||||
last_error: Optional[str] = None
|
||||
youtube_url: Optional[str] = None
|
||||
notification_sent: bool = False
|
||||
class RateLimitStatsResponse(BaseModel):
|
||||
"""Current rate limit statistics."""
|
||||
# Configuration
|
||||
jobs_per_day_limit: int
|
||||
youtube_uploads_per_day_limit: int
|
||||
beta_ip_per_day_limit: int
|
||||
rate_limiting_enabled: bool
|
||||
|
||||
# Current usage
|
||||
youtube_uploads_today: int
|
||||
youtube_uploads_remaining: int
|
||||
|
||||
# Blocklist stats
|
||||
disposable_domains_count: int
|
||||
blocked_emails_count: int
|
||||
blocked_ips_count: int
|
||||
|
||||
# User override stats
|
||||
total_overrides: int
|
||||
|
||||
|
||||
class YouTubeQueueListResponse(BaseModel):
|
||||
"""List of YouTube upload queue entries."""
|
||||
entries: List[YouTubeQueueEntry]
|
||||
stats: dict
|
||||
class UserRateLimitStatusResponse(BaseModel):
|
||||
"""Rate limit status for a specific user."""
|
||||
email: str
|
||||
jobs_today: int
|
||||
jobs_limit: int
|
||||
jobs_remaining: int
|
||||
has_bypass: bool
|
||||
custom_limit: Optional[int]
|
||||
bypass_reason: Optional[str]
|
||||
|
||||
|
||||
class BlocklistsResponse(BaseModel):
|
||||
"""All blocklist data with domain source separation."""
|
||||
external_domains: List[str]
|
||||
manual_domains: List[str]
|
||||
allowlisted_domains: List[str]
|
||||
"""All blocklist data."""
|
||||
disposable_domains: List[str]
|
||||
blocked_emails: List[str]
|
||||
blocked_ips: List[str]
|
||||
last_sync_at: Optional[datetime] = None
|
||||
last_sync_count: Optional[int] = None
|
||||
updated_at: Optional[datetime] = None
|
||||
updated_by: Optional[str] = None
|
||||
updated_at: Optional[datetime]
|
||||
updated_by: Optional[str]
|
||||
|
||||
|
||||
class DomainRequest(BaseModel):
|
||||
@@ -77,28 +84,159 @@ class IPRequest(BaseModel):
|
||||
ip_address: str
|
||||
|
||||
|
||||
class UserOverride(BaseModel):
|
||||
"""User override configuration."""
|
||||
email: str
|
||||
bypass_job_limit: bool
|
||||
custom_daily_job_limit: Optional[int]
|
||||
reason: str
|
||||
created_by: str
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class UserOverrideRequest(BaseModel):
|
||||
"""Request to set/update a user override."""
|
||||
bypass_job_limit: bool = False
|
||||
custom_daily_job_limit: Optional[int] = None
|
||||
reason: str
|
||||
|
||||
|
||||
class UserOverridesListResponse(BaseModel):
|
||||
"""List of all user overrides."""
|
||||
overrides: List[UserOverride]
|
||||
total: int
|
||||
|
||||
|
||||
class SuccessResponse(BaseModel):
|
||||
"""Generic success response."""
|
||||
success: bool
|
||||
message: str
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Statistics Endpoints
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/stats", response_model=RateLimitStatsResponse)
|
||||
async def get_rate_limit_stats(
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""
|
||||
Get current rate limit statistics.
|
||||
|
||||
Returns configuration values, current usage counts, and blocklist stats.
|
||||
"""
|
||||
rate_limit_service = get_rate_limit_service()
|
||||
email_validation = get_email_validation_service()
|
||||
|
||||
# Get YouTube uploads today
|
||||
youtube_today = rate_limit_service.get_youtube_uploads_today()
|
||||
youtube_limit = settings.rate_limit_youtube_uploads_per_day
|
||||
youtube_remaining = max(0, youtube_limit - youtube_today) if youtube_limit > 0 else -1
|
||||
|
||||
# Get blocklist stats
|
||||
blocklist_stats = email_validation.get_blocklist_stats()
|
||||
|
||||
# Get override count
|
||||
overrides = rate_limit_service.get_all_overrides()
|
||||
|
||||
return RateLimitStatsResponse(
|
||||
jobs_per_day_limit=settings.rate_limit_jobs_per_day,
|
||||
youtube_uploads_per_day_limit=settings.rate_limit_youtube_uploads_per_day,
|
||||
beta_ip_per_day_limit=settings.rate_limit_beta_ip_per_day,
|
||||
rate_limiting_enabled=settings.enable_rate_limiting,
|
||||
youtube_uploads_today=youtube_today,
|
||||
youtube_uploads_remaining=youtube_remaining,
|
||||
disposable_domains_count=blocklist_stats["disposable_domains_count"],
|
||||
blocked_emails_count=blocklist_stats["blocked_emails_count"],
|
||||
blocked_ips_count=blocklist_stats["blocked_ips_count"],
|
||||
total_overrides=len(overrides),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/users/{email}", response_model=UserRateLimitStatusResponse)
|
||||
async def get_user_rate_limit_status(
|
||||
email: str,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""
|
||||
Get rate limit status for a specific user.
|
||||
|
||||
Returns their current usage and any override settings.
|
||||
"""
|
||||
rate_limit_service = get_rate_limit_service()
|
||||
|
||||
jobs_today = rate_limit_service.get_user_job_count_today(email)
|
||||
override = rate_limit_service.get_user_override(email)
|
||||
|
||||
if override:
|
||||
has_bypass = override.get("bypass_job_limit", False)
|
||||
custom_limit = override.get("custom_daily_job_limit")
|
||||
bypass_reason = override.get("reason")
|
||||
jobs_limit = custom_limit if custom_limit is not None else settings.rate_limit_jobs_per_day
|
||||
else:
|
||||
has_bypass = False
|
||||
custom_limit = None
|
||||
bypass_reason = None
|
||||
jobs_limit = settings.rate_limit_jobs_per_day
|
||||
|
||||
jobs_remaining = max(0, jobs_limit - jobs_today) if not has_bypass else -1
|
||||
|
||||
return UserRateLimitStatusResponse(
|
||||
email=email,
|
||||
jobs_today=jobs_today,
|
||||
jobs_limit=jobs_limit,
|
||||
jobs_remaining=jobs_remaining,
|
||||
has_bypass=has_bypass,
|
||||
custom_limit=custom_limit,
|
||||
bypass_reason=bypass_reason,
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Blocklist Management Endpoints
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/blocklists", response_model=BlocklistsResponse)
|
||||
def get_blocklists(
|
||||
async def get_blocklists(
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""Get all blocklist data with domain source separation."""
|
||||
"""
|
||||
Get all blocklist data.
|
||||
|
||||
Returns disposable domains, blocked emails, and blocked IPs.
|
||||
"""
|
||||
email_validation = get_email_validation_service()
|
||||
raw_data = email_validation.get_blocklist_raw_data()
|
||||
return BlocklistsResponse(**raw_data)
|
||||
|
||||
# Force refresh to get latest data
|
||||
config = email_validation.get_blocklist_config(force_refresh=True)
|
||||
|
||||
# Get the raw document for metadata
|
||||
from google.cloud import firestore
|
||||
from backend.services.firestore_service import get_firestore_client
|
||||
from backend.services.email_validation_service import BLOCKLISTS_COLLECTION, BLOCKLIST_CONFIG_DOC
|
||||
|
||||
db = get_firestore_client()
|
||||
doc = db.collection(BLOCKLISTS_COLLECTION).document(BLOCKLIST_CONFIG_DOC).get()
|
||||
|
||||
updated_at = None
|
||||
updated_by = None
|
||||
if doc.exists:
|
||||
data = doc.to_dict()
|
||||
updated_at = data.get("updated_at")
|
||||
updated_by = data.get("updated_by")
|
||||
|
||||
return BlocklistsResponse(
|
||||
disposable_domains=sorted(list(config["disposable_domains"])),
|
||||
blocked_emails=sorted(list(config["blocked_emails"])),
|
||||
blocked_ips=sorted(list(config["blocked_ips"])),
|
||||
updated_at=updated_at,
|
||||
updated_by=updated_by,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/blocklists/disposable-domains", response_model=SuccessResponse)
|
||||
def add_disposable_domain(
|
||||
async def add_disposable_domain(
|
||||
request: DomainRequest,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
@@ -107,18 +245,18 @@ def add_disposable_domain(
|
||||
|
||||
domain = request.domain.lower().strip()
|
||||
if not domain or "." not in domain:
|
||||
raise HTTPException(status_code=400, detail=t("en", "admin.invalidDomainFormat"))
|
||||
raise HTTPException(status_code=400, detail="Invalid domain format")
|
||||
|
||||
email_validation.add_disposable_domain(domain, auth_result.user_email)
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.domainAddedToBl", domain=domain)
|
||||
message=f"Domain '{domain}' added to disposable domains blocklist"
|
||||
)
|
||||
|
||||
|
||||
@router.delete("/blocklists/disposable-domains/{domain}", response_model=SuccessResponse)
|
||||
def remove_disposable_domain(
|
||||
async def remove_disposable_domain(
|
||||
domain: str,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
@@ -127,67 +265,16 @@ def remove_disposable_domain(
|
||||
|
||||
domain = domain.lower().strip()
|
||||
if not email_validation.remove_disposable_domain(domain, auth_result.user_email):
|
||||
raise HTTPException(status_code=404, detail=t("en", "admin.domainNotFoundInBl"))
|
||||
raise HTTPException(status_code=404, detail="Domain not found in blocklist")
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.domainRemovedFromBl", domain=domain)
|
||||
)
|
||||
|
||||
|
||||
@router.post("/blocklists/allowlisted-domains", response_model=SuccessResponse)
|
||||
def add_allowlisted_domain(
|
||||
request: DomainRequest,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""Add a domain to the allowlist (overrides external blocklist)."""
|
||||
email_validation = get_email_validation_service()
|
||||
domain = request.domain.lower().strip()
|
||||
if not domain or "." not in domain:
|
||||
raise HTTPException(status_code=400, detail=t("en", "admin.invalidDomainFormat"))
|
||||
email_validation.add_allowlisted_domain(domain, auth_result.user_email)
|
||||
return SuccessResponse(success=True, message=t("en", "admin.domainAddedToAllowlist", domain=domain))
|
||||
|
||||
|
||||
@router.delete("/blocklists/allowlisted-domains/{domain}", response_model=SuccessResponse)
|
||||
def remove_allowlisted_domain(
|
||||
domain: str,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""Remove a domain from the allowlist."""
|
||||
email_validation = get_email_validation_service()
|
||||
domain = domain.lower().strip()
|
||||
if not email_validation.remove_allowlisted_domain(domain, auth_result.user_email):
|
||||
raise HTTPException(status_code=404, detail=t("en", "admin.domainNotFoundInAllowlist"))
|
||||
return SuccessResponse(success=True, message=t("en", "admin.domainRemovedFromAllowlist", domain=domain))
|
||||
|
||||
|
||||
@router.post("/blocklists/sync", response_model=SuccessResponse)
|
||||
async def trigger_sync(
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""Manually trigger a sync of the external disposable domain blocklist."""
|
||||
import asyncio
|
||||
from backend.services.disposable_domain_sync_service import (
|
||||
fetch_external_blocklist, sync_disposable_domains
|
||||
)
|
||||
from backend.services.firestore_service import get_firestore_client
|
||||
|
||||
domains = await fetch_external_blocklist()
|
||||
db = get_firestore_client()
|
||||
result = await asyncio.to_thread(sync_disposable_domains, db, domains)
|
||||
|
||||
# Invalidate cache
|
||||
EmailValidationService._blocklist_cache = None
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.syncComplete", external_count=result['external_count'], added=result['added'], removed=result['removed'])
|
||||
message=f"Domain '{domain}' removed from disposable domains blocklist"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/blocklists/blocked-emails", response_model=SuccessResponse)
|
||||
def add_blocked_email(
|
||||
async def add_blocked_email(
|
||||
request: EmailRequest,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
@@ -196,18 +283,18 @@ def add_blocked_email(
|
||||
|
||||
email = request.email.lower().strip()
|
||||
if not email or "@" not in email:
|
||||
raise HTTPException(status_code=400, detail=t("en", "admin.invalidEmailFormat"))
|
||||
raise HTTPException(status_code=400, detail="Invalid email format")
|
||||
|
||||
email_validation.add_blocked_email(email, auth_result.user_email)
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.emailAddedToBl", email=email)
|
||||
message=f"Email '{email}' added to blocked emails list"
|
||||
)
|
||||
|
||||
|
||||
@router.delete("/blocklists/blocked-emails/{email}", response_model=SuccessResponse)
|
||||
def remove_blocked_email(
|
||||
async def remove_blocked_email(
|
||||
email: str,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
@@ -216,16 +303,16 @@ def remove_blocked_email(
|
||||
|
||||
email = email.lower().strip()
|
||||
if not email_validation.remove_blocked_email(email, auth_result.user_email):
|
||||
raise HTTPException(status_code=404, detail=t("en", "admin.emailNotFoundInBl"))
|
||||
raise HTTPException(status_code=404, detail="Email not found in blocklist")
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.emailRemovedFromBl", email=email)
|
||||
message=f"Email '{email}' removed from blocked emails list"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/blocklists/blocked-ips", response_model=SuccessResponse)
|
||||
def add_blocked_ip(
|
||||
async def add_blocked_ip(
|
||||
request: IPRequest,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
@@ -234,18 +321,18 @@ def add_blocked_ip(
|
||||
|
||||
ip_address = request.ip_address.strip()
|
||||
if not ip_address:
|
||||
raise HTTPException(status_code=400, detail=t("en", "admin.invalidIPAddress"))
|
||||
raise HTTPException(status_code=400, detail="Invalid IP address")
|
||||
|
||||
email_validation.add_blocked_ip(ip_address, auth_result.user_email)
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.ipAddedToBl", ip_address=ip_address)
|
||||
message=f"IP '{ip_address}' added to blocked IPs list"
|
||||
)
|
||||
|
||||
|
||||
@router.delete("/blocklists/blocked-ips/{ip_address}", response_model=SuccessResponse)
|
||||
def remove_blocked_ip(
|
||||
async def remove_blocked_ip(
|
||||
ip_address: str,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
@@ -254,93 +341,88 @@ def remove_blocked_ip(
|
||||
|
||||
ip_address = ip_address.strip()
|
||||
if not email_validation.remove_blocked_ip(ip_address, auth_result.user_email):
|
||||
raise HTTPException(status_code=404, detail=t("en", "admin.ipNotFoundInBl"))
|
||||
raise HTTPException(status_code=404, detail="IP not found in blocklist")
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.ipRemovedFromBl", ip_address=ip_address)
|
||||
message=f"IP '{ip_address}' removed from blocked IPs list"
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# YouTube Upload Queue Management Endpoints
|
||||
# User Override Management Endpoints
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/youtube-queue", response_model=YouTubeQueueListResponse)
|
||||
def get_youtube_queue(
|
||||
@router.get("/overrides", response_model=UserOverridesListResponse)
|
||||
async def get_all_overrides(
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""
|
||||
Get YouTube upload queue entries and stats.
|
||||
"""Get all user overrides."""
|
||||
rate_limit_service = get_rate_limit_service()
|
||||
|
||||
Returns all queue entries (any status) and aggregate statistics.
|
||||
"""
|
||||
from backend.services.youtube_upload_queue_service import get_youtube_upload_queue_service
|
||||
overrides_data = rate_limit_service.get_all_overrides()
|
||||
|
||||
queue_service = get_youtube_upload_queue_service()
|
||||
entries = queue_service.get_all_queue_entries(limit=50)
|
||||
stats = queue_service.get_queue_stats()
|
||||
|
||||
return YouTubeQueueListResponse(
|
||||
entries=[YouTubeQueueEntry(**entry) for entry in entries],
|
||||
stats=stats,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/youtube-queue/{job_id}/retry", response_model=SuccessResponse)
|
||||
def retry_youtube_upload(
|
||||
job_id: str,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""
|
||||
Manually retry a failed YouTube upload.
|
||||
|
||||
Resets the entry back to 'queued' status with 0 attempts.
|
||||
"""
|
||||
from backend.services.youtube_upload_queue_service import get_youtube_upload_queue_service
|
||||
|
||||
queue_service = get_youtube_upload_queue_service()
|
||||
|
||||
if not queue_service.retry_upload(job_id):
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=t("en", "admin.youtubeUploadNotFound", job_id=job_id)
|
||||
overrides = [
|
||||
UserOverride(
|
||||
email=email,
|
||||
bypass_job_limit=data.get("bypass_job_limit", False),
|
||||
custom_daily_job_limit=data.get("custom_daily_job_limit"),
|
||||
reason=data.get("reason", ""),
|
||||
created_by=data.get("created_by", "unknown"),
|
||||
created_at=data.get("created_at", datetime.now(timezone.utc)),
|
||||
)
|
||||
for email, data in overrides_data.items()
|
||||
]
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.youtubeUploadQueuedForRetry", job_id=job_id)
|
||||
return UserOverridesListResponse(
|
||||
overrides=overrides,
|
||||
total=len(overrides),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/youtube-queue/process", response_model=SuccessResponse)
|
||||
async def trigger_youtube_queue_processing(
|
||||
@router.put("/overrides/{email}", response_model=SuccessResponse)
|
||||
async def set_user_override(
|
||||
email: str,
|
||||
request: UserOverrideRequest,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""
|
||||
Manually trigger YouTube upload queue processing.
|
||||
"""Set or update a user override."""
|
||||
rate_limit_service = get_rate_limit_service()
|
||||
|
||||
Same as the scheduled processor but triggered from the admin dashboard.
|
||||
Runs in the background and returns immediately.
|
||||
"""
|
||||
from backend.workers.youtube_queue_processor import process_youtube_upload_queue
|
||||
import asyncio
|
||||
email = email.lower().strip()
|
||||
if not email or "@" not in email:
|
||||
raise HTTPException(status_code=400, detail="Invalid email format")
|
||||
|
||||
async def _process():
|
||||
try:
|
||||
result = await process_youtube_upload_queue()
|
||||
logger.info(f"Manual YouTube queue processing complete: {result}")
|
||||
except Exception as e:
|
||||
logger.exception(f"Manual YouTube queue processing failed: {e}")
|
||||
if not request.reason or len(request.reason.strip()) < 3:
|
||||
raise HTTPException(status_code=400, detail="Reason is required (min 3 characters)")
|
||||
|
||||
# Fire and forget
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
loop.create_task(_process())
|
||||
except RuntimeError:
|
||||
pass
|
||||
rate_limit_service.set_user_override(
|
||||
user_email=email,
|
||||
bypass_job_limit=request.bypass_job_limit,
|
||||
custom_daily_job_limit=request.custom_daily_job_limit,
|
||||
reason=request.reason,
|
||||
admin_email=auth_result.user_email,
|
||||
)
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=t("en", "admin.youtubeQueueProcessingStarted")
|
||||
message=f"Override set for user '{email}'"
|
||||
)
|
||||
|
||||
|
||||
@router.delete("/overrides/{email}", response_model=SuccessResponse)
|
||||
async def remove_user_override(
|
||||
email: str,
|
||||
auth_result: AuthResult = Depends(require_admin),
|
||||
):
|
||||
"""Remove a user override."""
|
||||
rate_limit_service = get_rate_limit_service()
|
||||
|
||||
email = email.lower().strip()
|
||||
if not rate_limit_service.remove_user_override(email, auth_result.user_email):
|
||||
raise HTTPException(status_code=404, detail="Override not found for user")
|
||||
|
||||
return SuccessResponse(
|
||||
success=True,
|
||||
message=f"Override removed for user '{email}'"
|
||||
)
|
||||
@@ -1,461 +0,0 @@
|
||||
"""
|
||||
Referral API routes.
|
||||
|
||||
Handles:
|
||||
- Public referral interstitial (click tracking)
|
||||
- Authenticated referral dashboard
|
||||
- Stripe Connect onboarding
|
||||
- Admin vanity link creation and link management
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request
|
||||
from fastapi.responses import Response
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from backend.api.dependencies import require_auth, require_admin
|
||||
from backend.models.referral import (
|
||||
CreateReferralLinkRequest,
|
||||
UpdateReferralLinkRequest,
|
||||
ReferralInterstitialResponse,
|
||||
VanityRequestResponse,
|
||||
)
|
||||
from backend.services.flyer_service import get_flyer_service, FlyerError
|
||||
from backend.services.referral_service import get_referral_service
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/referrals", tags=["referrals"])
|
||||
|
||||
|
||||
class GenerateFlyerRequest(BaseModel):
|
||||
theme: str = Field(..., pattern="^(light|dark)$")
|
||||
qr_data_url: str = Field(..., max_length=500_000)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Public endpoints
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@router.get("/r/{code}", response_model=ReferralInterstitialResponse)
|
||||
def get_referral_interstitial(code: str, no_track: bool = False):
|
||||
"""
|
||||
Public endpoint: get interstitial data for a referral link.
|
||||
|
||||
Increments click count unless no_track=true. Returns valid=False if code not found.
|
||||
"""
|
||||
service = get_referral_service()
|
||||
link = service.get_link_by_code(code)
|
||||
|
||||
if not link:
|
||||
return ReferralInterstitialResponse(valid=False)
|
||||
|
||||
# Track the click (skip when fetching for display purposes)
|
||||
if not no_track:
|
||||
try:
|
||||
service.increment_clicks(code)
|
||||
except Exception:
|
||||
logger.warning("Failed to increment clicks for code %s", code, exc_info=True)
|
||||
|
||||
return ReferralInterstitialResponse(
|
||||
valid=True,
|
||||
referral_code=link.code,
|
||||
referrer_display_name=link.display_name,
|
||||
discount_percent=link.discount_percent,
|
||||
discount_duration_days=link.discount_duration_days,
|
||||
custom_message=link.custom_message,
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Authenticated user endpoints
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@router.get("/me")
|
||||
def get_my_dashboard(auth=Depends(require_auth)):
|
||||
"""Get referral dashboard data for the authenticated user."""
|
||||
service = get_referral_service()
|
||||
return service.get_dashboard_data(auth.user_email)
|
||||
|
||||
|
||||
@router.put("/me")
|
||||
def update_my_link(
|
||||
updates: UpdateReferralLinkRequest,
|
||||
auth=Depends(require_auth),
|
||||
):
|
||||
"""Update display_name and custom_message on the user's own referral link."""
|
||||
service = get_referral_service()
|
||||
link = service.get_or_create_link(auth.user_email)
|
||||
|
||||
update_fields = {}
|
||||
if updates.display_name is not None:
|
||||
update_fields["display_name"] = updates.display_name
|
||||
if updates.custom_message is not None:
|
||||
update_fields["custom_message"] = updates.custom_message
|
||||
|
||||
if not update_fields:
|
||||
return {"ok": True, "message": "No changes"}
|
||||
|
||||
success, message = service.update_link(link.code, **update_fields)
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
return {"ok": True, "message": message}
|
||||
|
||||
|
||||
@router.post("/me/connect")
|
||||
def start_stripe_connect(auth=Depends(require_auth)):
|
||||
"""Start Stripe Connect onboarding for the authenticated user."""
|
||||
# Check if user already has a Connect account
|
||||
from backend.services.user_service import get_user_service
|
||||
user_service = get_user_service()
|
||||
user = user_service.get_user(auth.user_email)
|
||||
if user and user.stripe_connect_account_id:
|
||||
raise HTTPException(status_code=400, detail="Stripe Connect account already configured")
|
||||
|
||||
service = get_referral_service()
|
||||
account_id, onboarding_url = service.create_connect_account(auth.user_email)
|
||||
|
||||
if not account_id:
|
||||
raise HTTPException(status_code=500, detail="Failed to create Stripe Connect account")
|
||||
|
||||
user_service.update_user(auth.user_email, stripe_connect_account_id=account_id)
|
||||
|
||||
return {"account_id": account_id, "onboarding_url": onboarding_url}
|
||||
|
||||
|
||||
@router.post("/me/connect/dashboard-link")
|
||||
def get_connect_dashboard_link(auth=Depends(require_auth)):
|
||||
"""Get a login link to the Stripe Express dashboard for managing payout settings."""
|
||||
from backend.services.user_service import get_user_service
|
||||
user = get_user_service().get_user(auth.user_email)
|
||||
if not user or not user.stripe_connect_account_id:
|
||||
raise HTTPException(status_code=400, detail="No Stripe Connect account configured")
|
||||
|
||||
service = get_referral_service()
|
||||
url = service.create_connect_login_link(user.stripe_connect_account_id)
|
||||
if not url:
|
||||
raise HTTPException(status_code=500, detail="Failed to create dashboard link")
|
||||
|
||||
return {"url": url}
|
||||
|
||||
|
||||
@router.post("/me/connect/update-link")
|
||||
def get_connect_update_link(auth=Depends(require_auth)):
|
||||
"""Get a link to update Stripe Connect account info (re-onboarding)."""
|
||||
from backend.services.user_service import get_user_service
|
||||
user = get_user_service().get_user(auth.user_email)
|
||||
if not user or not user.stripe_connect_account_id:
|
||||
raise HTTPException(status_code=400, detail="No Stripe Connect account configured")
|
||||
|
||||
service = get_referral_service()
|
||||
url = service.create_connect_update_link(user.stripe_connect_account_id)
|
||||
if not url:
|
||||
raise HTTPException(status_code=500, detail="Failed to create update link")
|
||||
|
||||
return {"url": url}
|
||||
|
||||
|
||||
class VanityRequest(BaseModel):
|
||||
desired_code: str = Field(..., min_length=2, max_length=30, pattern="^[a-z0-9-]+$")
|
||||
|
||||
|
||||
@router.post("/me/vanity-request")
|
||||
def request_vanity_url(
|
||||
request: VanityRequest,
|
||||
auth=Depends(require_auth),
|
||||
):
|
||||
"""Request a custom vanity referral URL. Sends email to admin for review."""
|
||||
from backend.services.email_service import get_email_service
|
||||
|
||||
service = get_referral_service()
|
||||
|
||||
# Validate up-front against the same rules approval enforces (reserved words,
|
||||
# vanity pattern) so a request that would later be auto-rejected never lands
|
||||
# in the admin queue. Do this before get_or_create_link so a rejected request
|
||||
# from a brand-new user doesn't leave an orphaned auto-generated link behind.
|
||||
valid, msg = service._validate_vanity_code(request.desired_code)
|
||||
if not valid:
|
||||
raise HTTPException(status_code=400, detail=msg)
|
||||
|
||||
# Check if code is already taken
|
||||
existing = service.get_link_by_code(request.desired_code)
|
||||
if existing:
|
||||
raise HTTPException(status_code=409, detail="That code is already taken. Please try another.")
|
||||
|
||||
link = service.get_or_create_link(auth.user_email)
|
||||
|
||||
# Persist the request so it shows up as a pending item in the admin dashboard.
|
||||
service.create_vanity_request(
|
||||
owner_email=auth.user_email,
|
||||
current_code=link.code,
|
||||
desired_code=request.desired_code,
|
||||
)
|
||||
|
||||
email_service = get_email_service()
|
||||
email_service.provider.send_email(
|
||||
to_email="andrew@nomadkaraoke.com",
|
||||
subject=f"Vanity referral URL request: {request.desired_code}",
|
||||
html_content=f"""
|
||||
<h2>Vanity Referral URL Request</h2>
|
||||
<p><strong>User:</strong> {auth.user_email}</p>
|
||||
<p><strong>Current code:</strong> {link.code}</p>
|
||||
<p><strong>Desired vanity code:</strong> {request.desired_code}</p>
|
||||
<p>Approve or deny it under "Pending Vanity Requests" in the <a href="https://gen.nomadkaraoke.com/admin/referrals">admin dashboard</a>.</p>
|
||||
""",
|
||||
text_content=f"Vanity URL request from {auth.user_email}: {request.desired_code} (current: {link.code})",
|
||||
)
|
||||
|
||||
logger.info("Vanity URL request from %s: %s", auth.user_email, request.desired_code)
|
||||
return {"ok": True, "message": "Request sent"}
|
||||
|
||||
|
||||
@router.post("/me/flyer")
|
||||
def generate_flyer(
|
||||
request: GenerateFlyerRequest,
|
||||
auth=Depends(require_auth),
|
||||
):
|
||||
"""Generate a personalized referral flyer PDF."""
|
||||
service = get_referral_service()
|
||||
link = service.get_or_create_link(auth.user_email)
|
||||
|
||||
try:
|
||||
flyer_service = get_flyer_service()
|
||||
pdf_bytes = flyer_service.generate_pdf(
|
||||
theme=request.theme,
|
||||
referral_code=link.code,
|
||||
discount_percent=link.discount_percent,
|
||||
qr_data_url=request.qr_data_url,
|
||||
)
|
||||
except FlyerError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
return Response(
|
||||
content=pdf_bytes,
|
||||
media_type="application/pdf",
|
||||
headers={
|
||||
"Content-Disposition": 'attachment; filename="nomad-karaoke-referral-flyer.pdf"',
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Admin endpoints
|
||||
# ============================================================================
|
||||
|
||||
|
||||
@router.post("/admin/vanity")
|
||||
def create_vanity_link(
|
||||
request: CreateReferralLinkRequest,
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: create a vanity referral link."""
|
||||
if not request.vanity_code:
|
||||
raise HTTPException(status_code=400, detail="vanity_code is required")
|
||||
|
||||
service = get_referral_service()
|
||||
success, link, message = service.create_vanity_link(
|
||||
code=request.vanity_code,
|
||||
owner_email=request.owner_email,
|
||||
display_name=request.display_name,
|
||||
custom_message=request.custom_message,
|
||||
discount_percent=request.discount_percent,
|
||||
kickback_percent=request.kickback_percent,
|
||||
discount_duration_days=request.discount_duration_days,
|
||||
earning_duration_days=request.earning_duration_days,
|
||||
)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
return {"ok": True, "code": link.code, "message": message}
|
||||
|
||||
|
||||
@router.get("/admin/vanity-requests")
|
||||
def list_vanity_requests(
|
||||
status: str = Query("pending", pattern="^(pending|approved|denied)$"),
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: list vanity code requests (pending by default)."""
|
||||
service = get_referral_service()
|
||||
requests = service.list_vanity_requests(status=status)
|
||||
return {
|
||||
"requests": [
|
||||
VanityRequestResponse(**r.model_dump()).model_dump(mode="json")
|
||||
for r in requests
|
||||
],
|
||||
"count": len(requests),
|
||||
}
|
||||
|
||||
|
||||
@router.post("/admin/vanity-requests/{request_id}/approve")
|
||||
def approve_vanity_request(
|
||||
request_id: str,
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: approve a vanity request by renaming the owner's link, then notify them."""
|
||||
from backend.services.email_service import get_email_service
|
||||
|
||||
service = get_referral_service()
|
||||
req = service.get_vanity_request(request_id)
|
||||
if not req:
|
||||
raise HTTPException(status_code=404, detail="Request not found")
|
||||
|
||||
success, link, message = service.approve_vanity_request(
|
||||
request_id, resolved_by=auth.user_email
|
||||
)
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
# Notify the requester that their vanity URL is live.
|
||||
try:
|
||||
referral_url = f"https://nomadkaraoke.com/r/{link.code}"
|
||||
get_email_service().provider.send_email(
|
||||
to_email=req.owner_email,
|
||||
subject="Your custom referral link is live 🎉",
|
||||
html_content=f"""
|
||||
<h2>Your custom referral link is ready</h2>
|
||||
<p>Good news — your requested referral code <strong>{link.code}</strong> is now active.</p>
|
||||
<p>Share it anywhere: <a href="{referral_url}">{referral_url}</a></p>
|
||||
<p>All your existing clicks, signups, and earnings carried over automatically.</p>
|
||||
""",
|
||||
text_content=f"Your custom referral link is live: {referral_url}",
|
||||
)
|
||||
except Exception:
|
||||
logger.warning("Failed to send vanity-approved email to %s", req.owner_email, exc_info=True)
|
||||
|
||||
return {"ok": True, "code": link.code, "message": message}
|
||||
|
||||
|
||||
@router.post("/admin/vanity-requests/{request_id}/deny")
|
||||
def deny_vanity_request(
|
||||
request_id: str,
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: deny a vanity request."""
|
||||
service = get_referral_service()
|
||||
success, message = service.deny_vanity_request(request_id, resolved_by=auth.user_email)
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
return {"ok": True, "message": message}
|
||||
|
||||
|
||||
@router.get("/admin/links")
|
||||
def list_links(
|
||||
limit: int = Query(50, ge=1, le=200),
|
||||
offset: int = Query(0, ge=0),
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: list all referral links."""
|
||||
service = get_referral_service()
|
||||
links = service.list_links(limit=limit, offset=offset)
|
||||
return {
|
||||
"links": [link.model_dump(mode="json") for link in links],
|
||||
"count": len(links),
|
||||
}
|
||||
|
||||
|
||||
@router.post("/admin/apply-discount")
|
||||
async def admin_apply_discount(
|
||||
request: Request,
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: apply a referral discount to any user, even without a referral link."""
|
||||
from backend.services.user_service import get_user_service
|
||||
|
||||
body = await request.json()
|
||||
user_email = body.get("email", "").lower()
|
||||
discount_percent = body.get("discount_percent", 10)
|
||||
duration_days = body.get("duration_days", 30)
|
||||
referral_code = body.get("referral_code", "admin-grant")
|
||||
|
||||
if not user_email:
|
||||
raise HTTPException(status_code=400, detail="Email is required")
|
||||
|
||||
user_service = get_user_service()
|
||||
user = user_service.get_user(user_email)
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
now = datetime.utcnow()
|
||||
|
||||
user_service.update_user(
|
||||
user_email,
|
||||
referred_by_code=referral_code,
|
||||
referred_at=now,
|
||||
referral_discount_expires_at=now + timedelta(days=duration_days),
|
||||
)
|
||||
|
||||
# Create/ensure the Stripe coupon exists for this discount percent
|
||||
service = get_referral_service()
|
||||
service.get_or_create_stripe_coupon(discount_percent)
|
||||
|
||||
logger.info(f"Admin applied {discount_percent}% discount for {duration_days} days to {user_email}")
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"message": f"Applied {discount_percent}% discount for {duration_days} days",
|
||||
"discount_expires_at": (now + timedelta(days=duration_days)).isoformat(),
|
||||
}
|
||||
|
||||
|
||||
@router.put("/admin/links/{code}")
|
||||
def update_link(
|
||||
code: str,
|
||||
updates: UpdateReferralLinkRequest,
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: update any referral link's settings."""
|
||||
service = get_referral_service()
|
||||
|
||||
update_fields = {
|
||||
k: v for k, v in updates.model_dump().items() if v is not None
|
||||
}
|
||||
|
||||
if not update_fields:
|
||||
return {"ok": True, "message": "No changes"}
|
||||
|
||||
success, message = service.update_link(code, **update_fields)
|
||||
if not success:
|
||||
raise HTTPException(status_code=404, detail=message)
|
||||
|
||||
return {"ok": True, "message": message}
|
||||
|
||||
|
||||
@router.post("/admin/links/{code}/flyer")
|
||||
def admin_generate_flyer(
|
||||
code: str,
|
||||
request: GenerateFlyerRequest,
|
||||
auth=Depends(require_admin),
|
||||
):
|
||||
"""Admin: generate a personalized referral flyer PDF for any code."""
|
||||
service = get_referral_service()
|
||||
link = service.get_link_by_code(code)
|
||||
|
||||
if not link:
|
||||
raise HTTPException(status_code=404, detail=f"Referral link '{code}' not found")
|
||||
|
||||
try:
|
||||
flyer_service = get_flyer_service()
|
||||
pdf_bytes = flyer_service.generate_pdf(
|
||||
theme=request.theme,
|
||||
referral_code=link.code,
|
||||
discount_percent=link.discount_percent,
|
||||
qr_data_url=request.qr_data_url,
|
||||
)
|
||||
except FlyerError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
return Response(
|
||||
content=pdf_bytes,
|
||||
media_type="application/pdf",
|
||||
headers={
|
||||
"Content-Disposition": f'attachment; filename="nomad-karaoke-flyer-{link.code}.pdf"',
|
||||
},
|
||||
)
|
||||
@@ -1,208 +0,0 @@
|
||||
"""
|
||||
Public song-request voting board API (requests.nomadkaraoke.com).
|
||||
|
||||
Anyone signed in with an email magic link can submit a song request (auto-corrected
|
||||
artist/title) and cast one vote per calendar day. The board itself (GET /requests) is
|
||||
readable without auth so it can be linked from YouTube descriptions and browsed freely.
|
||||
|
||||
Phase 2 (daily auto-picker, free-credit grant, ownership handoff, voter emails) is not
|
||||
here — see docs/archive/2026-09-02-requests-voting-board-plan.md.
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.security import HTTPAuthorizationCredentials
|
||||
|
||||
from backend.api.dependencies import require_auth, security, get_token_from_request
|
||||
from backend.services.auth_service import AuthResult, get_auth_service
|
||||
from backend.services.match_judge.classifier import normalize_for_match
|
||||
from backend.models.song_request import (
|
||||
BoardResponse,
|
||||
DailyVoteStatus,
|
||||
SongRequest,
|
||||
SongRequestPublic,
|
||||
SubmitRequestBody,
|
||||
SubmitResponse,
|
||||
VoteBody,
|
||||
)
|
||||
from backend.services.song_request_service import (
|
||||
InvalidSubmitter,
|
||||
RequestNotFound,
|
||||
SubmissionRateLimited,
|
||||
VoteAlreadyUsed,
|
||||
WouldDeleteRequest,
|
||||
get_song_request_service,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/requests-board", tags=["requests-board"])
|
||||
|
||||
|
||||
async def optional_user_email(
|
||||
request: Request,
|
||||
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
|
||||
) -> Optional[str]:
|
||||
"""Resolve the caller's email if a valid token is present, else None.
|
||||
|
||||
Unlike require_auth this never raises — the board is publicly viewable, we just
|
||||
annotate the viewer's own vote when they happen to be signed in.
|
||||
"""
|
||||
token = await get_token_from_request(request, credentials, None)
|
||||
if not token:
|
||||
return None
|
||||
try:
|
||||
result = get_auth_service().validate_token_full(token)
|
||||
return result.user_email if result.is_valid else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _was_corrected(req: SongRequest) -> bool:
|
||||
return (
|
||||
normalize_for_match(req.artist_raw) != normalize_for_match(req.artist)
|
||||
or normalize_for_match(req.title_raw) != normalize_for_match(req.title)
|
||||
)
|
||||
|
||||
|
||||
def _to_public(req: SongRequest, your_vote: Optional[int] = None) -> SongRequestPublic:
|
||||
return SongRequestPublic(
|
||||
id=req.id,
|
||||
artist=req.artist,
|
||||
title=req.title,
|
||||
status=req.status,
|
||||
source=req.source,
|
||||
vote_count=req.vote_count,
|
||||
created_at=req.created_at.isoformat(),
|
||||
youtube_url=req.youtube_url,
|
||||
was_corrected=_was_corrected(req),
|
||||
your_vote=your_vote,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/requests", response_model=BoardResponse)
|
||||
def list_requests(
|
||||
viewer_email: Optional[str] = Depends(optional_user_email),
|
||||
):
|
||||
"""The ranked board: active requests + recently published, with the caller's vote."""
|
||||
service = get_song_request_service()
|
||||
active = service.list_active()
|
||||
published = service.list_published()
|
||||
|
||||
daily_vote = service.get_daily_vote(viewer_email) if viewer_email else None
|
||||
voted_request_id = daily_vote.request_id if daily_vote else None
|
||||
voted_value = daily_vote.value if daily_vote else None
|
||||
|
||||
return BoardResponse(
|
||||
requests=[
|
||||
_to_public(r, your_vote=voted_value if r.id == voted_request_id else None)
|
||||
for r in active
|
||||
],
|
||||
published=[_to_public(r) for r in published],
|
||||
voted_today=bool(daily_vote) if viewer_email is not None else None,
|
||||
your_vote_request_id=voted_request_id,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/requests", response_model=SubmitResponse)
|
||||
async def submit_request(
|
||||
body: SubmitRequestBody,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Submit a song request (requires sign-in). Auto-corrects artist/title and dedupes."""
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=403, detail="A signed-in email is required to submit.")
|
||||
|
||||
service = get_song_request_service()
|
||||
try:
|
||||
request, already_existed, canonical_artist, canonical_title = await service.submit_request(
|
||||
auth_result.user_email, body.artist, body.title
|
||||
)
|
||||
except SubmissionRateLimited:
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail={
|
||||
"code": "submission_limit",
|
||||
"message": "You can request one song per day — please come back tomorrow.",
|
||||
},
|
||||
)
|
||||
except VoteAlreadyUsed:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail={
|
||||
"code": "vote_already_used",
|
||||
"message": "Requesting a song uses your one vote for today, and you've already "
|
||||
"voted on another request. Remove that vote first, or come back tomorrow.",
|
||||
},
|
||||
)
|
||||
except InvalidSubmitter:
|
||||
raise HTTPException(status_code=403, detail="A signed-in email is required to submit.")
|
||||
|
||||
daily = service.get_daily_vote(auth_result.user_email)
|
||||
your_vote = daily.value if daily and daily.request_id == request.id else None
|
||||
return SubmitResponse(
|
||||
status="already_exists" if already_existed else "created",
|
||||
request=_to_public(request, your_vote=your_vote),
|
||||
canonical_artist=canonical_artist,
|
||||
canonical_title=canonical_title,
|
||||
was_corrected=_was_corrected(request),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/requests/{request_id}/vote", response_model=SongRequestPublic)
|
||||
def vote(
|
||||
request_id: str,
|
||||
body: VoteBody,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Cast/move/undo your single daily vote on a request."""
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(status_code=403, detail="A signed-in email is required to vote.")
|
||||
|
||||
service = get_song_request_service()
|
||||
try:
|
||||
service.cast_vote(
|
||||
auth_result.user_email, request_id, body.direction, confirm_delete=body.confirm_delete
|
||||
)
|
||||
except RequestNotFound:
|
||||
raise HTTPException(status_code=404, detail="That request no longer exists.")
|
||||
except WouldDeleteRequest as exc:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail={
|
||||
"code": "would_delete_request",
|
||||
# The request that would be deleted (not necessarily the one being voted on:
|
||||
# moving your only up-vote away deletes the OLD request).
|
||||
"request_id": exc.request_id,
|
||||
"message": "Removing your vote will delete this request from the board "
|
||||
"(nobody else is backing it). Continue?",
|
||||
},
|
||||
)
|
||||
except InvalidSubmitter:
|
||||
raise HTTPException(status_code=403, detail="A signed-in email is required to vote.")
|
||||
|
||||
updated = service.get_request(request_id)
|
||||
if not updated:
|
||||
# Its last up-vote was withdrawn (confirmed), so the request was deleted.
|
||||
return SongRequestPublic(
|
||||
id=request_id, artist="", title="", status="open", source="human",
|
||||
vote_count=0, created_at="", request_deleted=True,
|
||||
)
|
||||
daily = service.get_daily_vote(auth_result.user_email)
|
||||
your_vote = daily.value if daily and daily.request_id == request_id else None
|
||||
return _to_public(updated, your_vote=your_vote)
|
||||
|
||||
|
||||
@router.get("/me", response_model=DailyVoteStatus)
|
||||
def my_daily_status(
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""The caller's daily-vote status (has today's vote been used, and on what)."""
|
||||
service = get_song_request_service()
|
||||
daily = service.get_daily_vote(auth_result.user_email) if auth_result.user_email else None
|
||||
return DailyVoteStatus(
|
||||
voted_today=bool(daily),
|
||||
request_id=daily.request_id if daily else None,
|
||||
value=daily.value if daily else None,
|
||||
)
|
||||
+324
-3018
File diff suppressed because it is too large.
Load diff
@@ -12,7 +12,6 @@ from fastapi import APIRouter, Query, Request
|
||||
|
||||
from backend.models.tenant import TenantConfigResponse, TenantPublicConfig
|
||||
from backend.services.tenant_service import get_tenant_service
|
||||
from backend.i18n import t, get_locale_from_request
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -20,7 +19,7 @@ router = APIRouter(prefix="/api/tenant", tags=["tenant"])
|
||||
|
||||
|
||||
@router.get("/config", response_model=TenantConfigResponse)
|
||||
def get_tenant_config(
|
||||
async def get_tenant_config(
|
||||
request: Request,
|
||||
tenant: Optional[str] = Query(
|
||||
None,
|
||||
@@ -81,7 +80,7 @@ def get_tenant_config(
|
||||
|
||||
|
||||
@router.get("/config/{tenant_id}", response_model=TenantConfigResponse)
|
||||
def get_tenant_config_by_id(tenant_id: str):
|
||||
async def get_tenant_config_by_id(tenant_id: str):
|
||||
"""
|
||||
Get tenant configuration by explicit tenant ID.
|
||||
|
||||
@@ -102,7 +101,7 @@ def get_tenant_config_by_id(tenant_id: str):
|
||||
|
||||
|
||||
@router.get("/asset/{tenant_id}/{asset_name}")
|
||||
def get_tenant_asset(tenant_id: str, asset_name: str):
|
||||
async def get_tenant_asset(tenant_id: str, asset_name: str):
|
||||
"""
|
||||
Get a signed URL for a tenant asset (logo, favicon, etc.).
|
||||
|
||||
@@ -116,6 +115,6 @@ def get_tenant_asset(tenant_id: str, asset_name: str):
|
||||
if not url:
|
||||
from fastapi import HTTPException
|
||||
|
||||
raise HTTPException(status_code=404, detail=t("en", "tenant.assetNotFound"))
|
||||
raise HTTPException(status_code=404, detail="Asset not found")
|
||||
|
||||
return RedirectResponse(url=url)
|
||||
@@ -1,375 +0,0 @@
|
||||
"""
|
||||
Admin API routes for provisioning white-label tenants.
|
||||
|
||||
These endpoints let an admin mint a tenant (branded portal + locked theme) from
|
||||
the UI, replacing the hand-run ``scripts/setup-*-tenant.py`` recipe. Once created,
|
||||
the tenant can be driven immediately via the admin preview path
|
||||
``?preview_tenant=<id>`` — including the tenant bulk-upload flow (folder of tracks,
|
||||
auto-paired mixed/instrumental, resumable uploads, locked theme on every job).
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
|
||||
from fastapi.concurrency import run_in_threadpool
|
||||
from pydantic import BaseModel, ValidationError
|
||||
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.models.tenant import TenantPublicConfig
|
||||
from backend.models.theme import ColorOverrides
|
||||
from backend.services.auth_service import AuthResult
|
||||
from backend.services.tenant_admin_service import (
|
||||
IMAGE_CONTENT_TYPES,
|
||||
TenantConflictError,
|
||||
TenantNotFoundError,
|
||||
TenantProvisioningError,
|
||||
TenantValidationError,
|
||||
create_tenant,
|
||||
delete_tenant,
|
||||
get_default_style_params,
|
||||
get_tenant_detail,
|
||||
get_tenant_domain_status,
|
||||
list_tenants,
|
||||
provision_tenant_domain,
|
||||
update_tenant,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/admin/tenants", tags=["admin", "tenant"])
|
||||
|
||||
# Public app origin used to build the admin-preview link.
|
||||
APP_ORIGIN = "https://gen.nomadkaraoke.com"
|
||||
|
||||
# Reject oversized image uploads (backgrounds/logo).
|
||||
MAX_IMAGE_BYTES = 15 * 1024 * 1024
|
||||
|
||||
|
||||
class TenantSummary(BaseModel):
|
||||
id: Optional[str] = None
|
||||
name: Optional[str] = None
|
||||
subdomain: Optional[str] = None
|
||||
is_active: bool = True
|
||||
locked_theme: Optional[str] = None
|
||||
theme_id: Optional[str] = None
|
||||
dropbox_path: Optional[str] = None
|
||||
brand_prefix: Optional[str] = None
|
||||
created_at: Optional[str] = None
|
||||
|
||||
|
||||
class TenantListResponse(BaseModel):
|
||||
tenants: List[TenantSummary]
|
||||
|
||||
|
||||
class TenantCreateResponse(BaseModel):
|
||||
tenant: TenantPublicConfig
|
||||
preview_url: str
|
||||
subdomain_url: str
|
||||
# Cloudflare state of the portal subdomain: {hostname, state, dns_ok, pages_status}.
|
||||
# state is "active" once the certificate is issued (usually within ~1 min).
|
||||
domain: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
class TenantDetailResponse(BaseModel):
|
||||
tenant: Dict[str, Any]
|
||||
theme_id: str
|
||||
style_params: Dict[str, Any]
|
||||
assets: List[str]
|
||||
preview_url: str
|
||||
domain: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
class TenantDomainResponse(BaseModel):
|
||||
domain: Dict[str, Any]
|
||||
|
||||
|
||||
class TenantTemplateResponse(BaseModel):
|
||||
style_params: Dict[str, Any]
|
||||
|
||||
|
||||
class TenantUpdateResponse(BaseModel):
|
||||
tenant: TenantPublicConfig
|
||||
|
||||
|
||||
def _clean(value: Optional[str]) -> Optional[str]:
|
||||
"""Normalize empty/whitespace form values to None."""
|
||||
if value is None:
|
||||
return None
|
||||
value = value.strip()
|
||||
return value or None
|
||||
|
||||
|
||||
def _preview_url(tenant_id: str) -> str:
|
||||
return f"{APP_ORIGIN}/en/app?preview_tenant={tenant_id}"
|
||||
|
||||
|
||||
def _parse_json_field(raw: Optional[str], label: str) -> Optional[Any]:
|
||||
"""Parse an optional JSON string form field, mapping errors to 400."""
|
||||
if raw is None or not raw.strip():
|
||||
return None
|
||||
try:
|
||||
return json.loads(raw)
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise HTTPException(status_code=400, detail=f"{label}: invalid JSON ({exc}).")
|
||||
|
||||
|
||||
def _parse_domains(raw: Optional[str]) -> List[str]:
|
||||
if not raw:
|
||||
return []
|
||||
parts = raw.replace("\n", ",").replace(" ", ",").split(",")
|
||||
return [p.strip().lower() for p in parts if p.strip()]
|
||||
|
||||
|
||||
async def _read_image(upload: Optional[UploadFile], label: str) -> Optional[Tuple[bytes, str]]:
|
||||
"""Read+validate an uploaded image, returning (bytes, ext) or None."""
|
||||
if upload is None or not upload.filename:
|
||||
return None
|
||||
ext = upload.filename.rsplit(".", 1)[-1].lower() if "." in upload.filename else ""
|
||||
if ext not in IMAGE_CONTENT_TYPES:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"{label}: unsupported image type '.{ext}'. Use PNG, JPG, GIF, or WEBP.",
|
||||
)
|
||||
data = await upload.read()
|
||||
if not data:
|
||||
return None
|
||||
if len(data) > MAX_IMAGE_BYTES:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"{label}: image is too large ({len(data) // (1024 * 1024)}MB). Max 15MB.",
|
||||
)
|
||||
return data, ext
|
||||
|
||||
|
||||
# Assets replaceable in the manage flow: images plus fonts.
|
||||
ASSET_EXTENSIONS = set(IMAGE_CONTENT_TYPES) | {"ttf", "otf", "woff", "woff2"}
|
||||
|
||||
|
||||
async def _read_asset(upload: Optional[UploadFile], label: str) -> Optional[Tuple[bytes, str]]:
|
||||
"""Read+validate an uploaded theme asset (image or font)."""
|
||||
if upload is None or not upload.filename:
|
||||
return None
|
||||
ext = upload.filename.rsplit(".", 1)[-1].lower() if "." in upload.filename else ""
|
||||
if ext not in ASSET_EXTENSIONS:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"{label}: unsupported asset type '.{ext}'. Use an image or font file.",
|
||||
)
|
||||
data = await upload.read()
|
||||
if not data:
|
||||
return None
|
||||
if len(data) > MAX_IMAGE_BYTES:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"{label}: file is too large ({len(data) // (1024 * 1024)}MB). Max 15MB.",
|
||||
)
|
||||
return data, ext
|
||||
|
||||
|
||||
@router.get("", response_model=TenantListResponse)
|
||||
def admin_list_tenants(auth_data: AuthResult = Depends(require_admin)):
|
||||
"""List all white-label tenants."""
|
||||
return TenantListResponse(tenants=[TenantSummary(**t) for t in list_tenants()])
|
||||
|
||||
|
||||
@router.get("/_template", response_model=TenantTemplateResponse)
|
||||
def admin_tenant_theme_template(auth_data: AuthResult = Depends(require_admin)):
|
||||
"""Return the default Nomad theme's full style_params as an editing starting point."""
|
||||
try:
|
||||
return TenantTemplateResponse(style_params=get_default_style_params())
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=500, detail=str(exc))
|
||||
|
||||
|
||||
# Sync handlers: FastAPI runs them in its threadpool, so the blocking GCS and
|
||||
# Cloudflare calls don't stall the event loop (the admin UI polls this).
|
||||
@router.get("/{tenant_id}", response_model=TenantDetailResponse)
|
||||
def admin_get_tenant(tenant_id: str, auth_data: AuthResult = Depends(require_admin)):
|
||||
"""Return a tenant's full config, theme style_params, and asset list for editing."""
|
||||
try:
|
||||
detail = get_tenant_detail(tenant_id)
|
||||
except TenantNotFoundError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc))
|
||||
return TenantDetailResponse(
|
||||
preview_url=_preview_url(tenant_id),
|
||||
domain=get_tenant_domain_status(tenant_id),
|
||||
**detail,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{tenant_id}/domain", response_model=TenantDomainResponse)
|
||||
def admin_provision_tenant_domain(tenant_id: str, auth_data: AuthResult = Depends(require_admin)):
|
||||
"""(Re)provision the tenant's {id}.nomadkaraoke.com subdomain (idempotent)."""
|
||||
try:
|
||||
domain = provision_tenant_domain(tenant_id)
|
||||
except TenantNotFoundError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc))
|
||||
except TenantConflictError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc))
|
||||
except TenantProvisioningError as exc:
|
||||
raise HTTPException(status_code=502, detail=str(exc))
|
||||
logger.info(f"Admin {auth_data.user_email or 'admin:unknown'} provisioned domain for tenant '{tenant_id}'")
|
||||
return TenantDomainResponse(domain=domain)
|
||||
|
||||
|
||||
@router.delete("/{tenant_id}", status_code=204)
|
||||
def admin_delete_tenant(tenant_id: str, auth_data: AuthResult = Depends(require_admin)):
|
||||
"""Delete a tenant: subdomain (Cloudflare), theme, and config. Jobs are kept."""
|
||||
try:
|
||||
delete_tenant(tenant_id)
|
||||
except TenantNotFoundError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc))
|
||||
except TenantProvisioningError as exc:
|
||||
raise HTTPException(status_code=502, detail=str(exc))
|
||||
logger.info(f"Admin {auth_data.user_email or 'admin:unknown'} deleted tenant '{tenant_id}'")
|
||||
|
||||
|
||||
@router.put("/{tenant_id}", response_model=TenantUpdateResponse)
|
||||
async def admin_update_tenant(
|
||||
tenant_id: str,
|
||||
auth_data: AuthResult = Depends(require_admin),
|
||||
config: Optional[str] = Form(None),
|
||||
style_params: Optional[str] = Form(None),
|
||||
logo: Optional[UploadFile] = File(None),
|
||||
assets: Optional[List[UploadFile]] = File(None),
|
||||
):
|
||||
"""
|
||||
Update a tenant: merge config fields (JSON), replace the full theme
|
||||
style_params (JSON), and/or add/replace theme assets (uploaded files keyed
|
||||
by their filename). Re-render jobs to apply an updated theme.
|
||||
"""
|
||||
config_updates = _parse_json_field(config, "config")
|
||||
if config_updates is not None and not isinstance(config_updates, dict):
|
||||
raise HTTPException(status_code=400, detail="config must be a JSON object.")
|
||||
style = _parse_json_field(style_params, "style_params")
|
||||
|
||||
asset_map: Dict[str, Tuple[bytes, str]] = {}
|
||||
for upload in assets or []:
|
||||
asset = await _read_asset(upload, f"asset '{upload.filename}'")
|
||||
if asset and upload.filename:
|
||||
asset_map[upload.filename] = asset
|
||||
logo_image = await _read_image(logo, "logo")
|
||||
|
||||
try:
|
||||
updated = await run_in_threadpool(
|
||||
update_tenant,
|
||||
tenant_id,
|
||||
config_updates=config_updates,
|
||||
style_params=style,
|
||||
assets=asset_map or None,
|
||||
logo=logo_image,
|
||||
)
|
||||
except TenantNotFoundError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc))
|
||||
except TenantConflictError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc))
|
||||
except TenantValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except TenantProvisioningError as exc:
|
||||
raise HTTPException(status_code=502, detail=str(exc))
|
||||
except ValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid config: {exc}")
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except Exception as exc: # pragma: no cover - defensive
|
||||
logger.exception("Failed to update tenant")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to update tenant: {exc}")
|
||||
|
||||
admin_id = auth_data.user_email or "admin:unknown"
|
||||
logger.info(f"Admin {admin_id} updated tenant '{tenant_id}'")
|
||||
return TenantUpdateResponse(tenant=TenantPublicConfig.from_config(updated))
|
||||
|
||||
|
||||
@router.post("", response_model=TenantCreateResponse, status_code=201)
|
||||
async def admin_create_tenant(
|
||||
auth_data: AuthResult = Depends(require_admin),
|
||||
name: str = Form(...),
|
||||
tenant_id: Optional[str] = Form(None),
|
||||
subdomain: Optional[str] = Form(None),
|
||||
allowed_email_domains: Optional[str] = Form(None),
|
||||
allowed_emails: Optional[str] = Form(None),
|
||||
artist_color: Optional[str] = Form(None),
|
||||
title_color: Optional[str] = Form(None),
|
||||
sung_lyrics_color: Optional[str] = Form(None),
|
||||
unsung_lyrics_color: Optional[str] = Form(None),
|
||||
tagline: Optional[str] = Form(None),
|
||||
distribution_mode: str = Form("download_only"),
|
||||
dropbox_path: Optional[str] = Form(None),
|
||||
brand_prefix: Optional[str] = Form(None),
|
||||
style_params: Optional[str] = Form(None),
|
||||
karaoke_background: Optional[UploadFile] = File(None),
|
||||
intro_background: Optional[UploadFile] = File(None),
|
||||
end_background: Optional[UploadFile] = File(None),
|
||||
logo: Optional[UploadFile] = File(None),
|
||||
):
|
||||
"""Provision a new white-label tenant (theme + config) from admin form input."""
|
||||
try:
|
||||
colors = ColorOverrides(
|
||||
artist_color=_clean(artist_color),
|
||||
title_color=_clean(title_color),
|
||||
sung_lyrics_color=_clean(sung_lyrics_color),
|
||||
unsung_lyrics_color=_clean(unsung_lyrics_color),
|
||||
)
|
||||
except ValidationError:
|
||||
raise HTTPException(
|
||||
status_code=400, detail="Colors must be hex values like #ff5bb8."
|
||||
)
|
||||
|
||||
backgrounds = {}
|
||||
for field, upload in (
|
||||
("karaoke_background", karaoke_background),
|
||||
("intro_background", intro_background),
|
||||
("end_background", end_background),
|
||||
):
|
||||
image = await _read_image(upload, field)
|
||||
if image:
|
||||
backgrounds[field] = image
|
||||
logo_image = await _read_image(logo, "logo")
|
||||
|
||||
if distribution_mode not in ("all", "download_only", "cloud_only"):
|
||||
raise HTTPException(status_code=400, detail="Invalid distribution_mode.")
|
||||
|
||||
style_override = _parse_json_field(style_params, "style_params")
|
||||
if style_override is not None and not isinstance(style_override, dict):
|
||||
raise HTTPException(status_code=400, detail="style_params must be a JSON object.")
|
||||
|
||||
try:
|
||||
config = await run_in_threadpool(
|
||||
create_tenant,
|
||||
name=name,
|
||||
tenant_id=_clean(tenant_id),
|
||||
subdomain=_clean(subdomain),
|
||||
allowed_email_domains=_parse_domains(allowed_email_domains),
|
||||
allowed_emails=_parse_domains(allowed_emails),
|
||||
colors=colors,
|
||||
style_params_override=style_override,
|
||||
tagline=_clean(tagline),
|
||||
distribution_mode=distribution_mode,
|
||||
dropbox_path=_clean(dropbox_path),
|
||||
brand_prefix=_clean(brand_prefix),
|
||||
backgrounds=backgrounds,
|
||||
logo=logo_image,
|
||||
)
|
||||
except TenantConflictError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc))
|
||||
except TenantValidationError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except TenantProvisioningError as exc:
|
||||
raise HTTPException(status_code=502, detail=str(exc))
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except Exception as exc: # pragma: no cover - defensive
|
||||
logger.exception("Failed to create tenant")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to create tenant: {exc}")
|
||||
|
||||
admin_id = auth_data.user_email or "admin:unknown"
|
||||
logger.info(f"Admin {admin_id} created tenant '{config.id}'")
|
||||
|
||||
return TenantCreateResponse(
|
||||
tenant=TenantPublicConfig.from_config(config),
|
||||
preview_url=_preview_url(config.id),
|
||||
subdomain_url=config.get_frontend_url(),
|
||||
domain=await run_in_threadpool(get_tenant_domain_status, config.id),
|
||||
)
|
||||
@@ -1,89 +0,0 @@
|
||||
"""Authenticated tenant bulk-upload endpoints.
|
||||
|
||||
Separate from ``tenant.py`` (which is public config only): these require a
|
||||
tenant session and gate on the ``bulk_upload`` feature flag.
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import List
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from backend.api.dependencies import require_auth
|
||||
from backend.middleware.tenant import get_tenant_config_from_request
|
||||
from backend.services.auth_service import AuthResult
|
||||
from pathlib import Path
|
||||
|
||||
from backend.services.tenant_bulk import analyze_filenames
|
||||
from backend.services.tenant_bulk.analyze import AUDIO_EXTENSIONS, default_generate
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/tenant/bulk", tags=["tenant-bulk"])
|
||||
|
||||
# Mirror consumer Bulk Mode's cap to bound LLM cost and upload load. Counts
|
||||
# audio files (candidate tracks), not incidental non-audio files a folder pick
|
||||
# may include (cover art, etc.), so an image-heavy folder isn't falsely blocked.
|
||||
MAX_AUDIO_FILES = 100
|
||||
# Absolute payload guard so a single request can't send an unbounded list.
|
||||
MAX_FILENAMES = 2000
|
||||
|
||||
|
||||
class BulkAnalyzeRequest(BaseModel):
|
||||
filenames: List[str] = Field(
|
||||
..., description="Filenames from the operator's folder pick (no paths needed)"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/analyze")
|
||||
async def analyze_bulk_filenames(
|
||||
request: Request,
|
||||
body: BulkAnalyzeRequest,
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Analyse a list of filenames into proposed Mixed/Instrumental job rows.
|
||||
|
||||
Pure analysis: filenames only, no uploads, no audio, no state written.
|
||||
Returns ``{rows, unpaired, ignored}`` for an editable review table.
|
||||
"""
|
||||
tenant_config = get_tenant_config_from_request(request)
|
||||
if not tenant_config:
|
||||
raise HTTPException(
|
||||
status_code=403, detail="Bulk upload is only available on tenant portals"
|
||||
)
|
||||
if not getattr(tenant_config.features, "bulk_upload", False):
|
||||
raise HTTPException(
|
||||
status_code=403, detail="Bulk upload is not enabled for this portal"
|
||||
)
|
||||
|
||||
filenames = [f for f in (body.filenames or []) if f and f.strip()]
|
||||
if not filenames:
|
||||
raise HTTPException(status_code=400, detail="No filenames provided")
|
||||
if len(filenames) > MAX_FILENAMES:
|
||||
raise HTTPException(status_code=400, detail="Too many files in request")
|
||||
audio_count = sum(
|
||||
1 for f in filenames if Path(f).suffix.lower() in AUDIO_EXTENSIONS
|
||||
)
|
||||
if audio_count > MAX_AUDIO_FILES:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Too many tracks ({audio_count}). Please submit at most {MAX_AUDIO_FILES} at a time.",
|
||||
)
|
||||
|
||||
# The LLM pass is a blocking Vertex call; run it off the event loop. It only
|
||||
# fires for files the regex could not pair, and analyze_filenames falls back
|
||||
# to the regex result if the model errors — so this never hard-fails.
|
||||
def _run() -> dict:
|
||||
return analyze_filenames(filenames, generate=default_generate).to_dict()
|
||||
|
||||
analysis = await asyncio.to_thread(_run)
|
||||
logger.info(
|
||||
"tenant-bulk analyze: tenant=%s files=%d rows=%d unpaired=%d ignored=%d",
|
||||
tenant_config.id,
|
||||
len(filenames),
|
||||
len(analysis["rows"]),
|
||||
len(analysis["unpaired"]),
|
||||
len(analysis["ignored"]),
|
||||
)
|
||||
return analysis
|
||||
@@ -1,298 +0,0 @@
|
||||
"""
|
||||
Tenant self-service theme editor API (``/api/tenant/theme``).
|
||||
|
||||
Any user allowed on a tenant's portal can view, preview and save that tenant's
|
||||
theme. Tenant context comes from the request (subdomain or the client-controlled
|
||||
``X-Tenant-ID`` header), so every endpoint re-checks membership against the
|
||||
tenant's allowlist (admins always pass) — a header alone grants nothing.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from typing import Any, Dict, List, Literal, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, Request, UploadFile
|
||||
from fastapi.concurrency import run_in_threadpool
|
||||
from pydantic import BaseModel, Field, constr
|
||||
|
||||
from backend.api.dependencies import require_auth
|
||||
from backend.middleware.tenant import get_tenant_config_from_request
|
||||
from backend.models.tenant import TenantConfig
|
||||
from backend.services.auth_service import AuthResult
|
||||
from backend.services.tenant_admin_service import TenantValidationError
|
||||
from backend.services.tenant_theme_service import (
|
||||
ThemeNotEditableError,
|
||||
ThemeNotFoundError,
|
||||
get_theme_for_editor,
|
||||
prepare_preview_styles,
|
||||
save_tenant_theme,
|
||||
store_uploaded_asset,
|
||||
)
|
||||
from backend.services.theme_change_service import outdated_jobs, refresh_inflight_jobs
|
||||
from backend.services.theme_preview_service import ThemePreviewError, render_theme_preview
|
||||
from backend.services.tenant_admin_service import _theme_id_for
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/tenant/theme", tags=["tenant-theme"])
|
||||
|
||||
MAX_UPLOAD_BYTES = 15 * 1024 * 1024
|
||||
|
||||
# Previews run ffmpeg/PIL on the API instance — bound concurrency like the review
|
||||
# preview path so a burst of edits can't starve request handling.
|
||||
_PREVIEW_SEMAPHORE = asyncio.Semaphore(2)
|
||||
_PREVIEW_EXECUTOR = ThreadPoolExecutor(max_workers=2, thread_name_prefix="theme-preview")
|
||||
PREVIEW_TIMEOUT_S = 45
|
||||
|
||||
|
||||
def require_tenant_member(
|
||||
request: Request, auth_result: AuthResult = Depends(require_auth)
|
||||
) -> TenantConfig:
|
||||
"""The request's tenant, iff the authenticated user is allowed on it."""
|
||||
config = get_tenant_config_from_request(request)
|
||||
if config is None or not config.is_active:
|
||||
raise HTTPException(status_code=404, detail="Theme editing is only available on a tenant portal.")
|
||||
if auth_result.tenant_id and auth_result.tenant_id != config.id:
|
||||
raise HTTPException(status_code=403, detail="Your sign-in belongs to a different portal.")
|
||||
if not auth_result.user_email or not config.is_email_allowed(auth_result.user_email):
|
||||
raise HTTPException(status_code=403, detail="You don't have access to this portal's theme.")
|
||||
return config
|
||||
|
||||
|
||||
def _raise_http(exc: TenantValidationError) -> None:
|
||||
if isinstance(exc, ThemeNotFoundError):
|
||||
raise HTTPException(status_code=404, detail=str(exc))
|
||||
if isinstance(exc, ThemeNotEditableError):
|
||||
raise HTTPException(status_code=403, detail=str(exc))
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
|
||||
|
||||
class ThemeResponse(BaseModel):
|
||||
theme_id: str
|
||||
style_params: Dict[str, Any]
|
||||
images: List[str]
|
||||
fonts: List[str]
|
||||
# The images that fit the 300x216 CD+G screen (offered on the CD+G tab).
|
||||
cdg_images: List[str] = Field(default_factory=list)
|
||||
# Built-in CD+G reveal transitions (greyscale draw-order masks).
|
||||
cdg_transitions: List[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class PreviewSample(BaseModel):
|
||||
artist: str = Field("Artist Name", max_length=80)
|
||||
title: str = Field("Song Title", max_length=80)
|
||||
lyrics: Optional[List[constr(max_length=120)]] = Field(None, max_length=4)
|
||||
|
||||
|
||||
class PreviewRequest(BaseModel):
|
||||
style_params: Dict[str, Any]
|
||||
sample: PreviewSample = Field(default_factory=PreviewSample)
|
||||
# Also render the CD+G up card + a lyrics page (slower; the editor asks on its CD+G tab).
|
||||
include_cdg: bool = False
|
||||
# MP4 screens to render (default all): the editor asks only for the tab's screen.
|
||||
kinds: Optional[List[Literal["title_card", "karaoke_frame", "end_screen"]]] = None
|
||||
|
||||
|
||||
class PreviewResponse(BaseModel):
|
||||
# data:image/jpeg;base64,... — null for a screen the theme leaves out of videos.
|
||||
title_card: Optional[str] = None
|
||||
karaoke_frame: Optional[str] = None
|
||||
end_screen: Optional[str] = None
|
||||
# data:image/png;base64,... — only when include_cdg was requested.
|
||||
cdg_title_card: Optional[str] = None
|
||||
cdg_lyrics: Optional[str] = None
|
||||
cdg_instrumental: Optional[str] = None
|
||||
cdg_outro: Optional[str] = None
|
||||
|
||||
|
||||
class SaveRequest(BaseModel):
|
||||
style_params: Dict[str, Any]
|
||||
|
||||
|
||||
class AssetResponse(BaseModel):
|
||||
name: str
|
||||
|
||||
|
||||
@router.get("", response_model=ThemeResponse)
|
||||
def get_theme(config: TenantConfig = Depends(require_tenant_member)):
|
||||
try:
|
||||
return ThemeResponse(**get_theme_for_editor(config))
|
||||
except TenantValidationError as exc:
|
||||
_raise_http(exc)
|
||||
|
||||
|
||||
@router.post("/assets", response_model=AssetResponse)
|
||||
async def upload_asset(
|
||||
file: UploadFile = File(...),
|
||||
purpose: Optional[str] = Form(None),
|
||||
config: TenantConfig = Depends(require_tenant_member),
|
||||
):
|
||||
if purpose not in (None, "", "cdg"):
|
||||
raise HTTPException(status_code=400, detail="Unknown upload purpose.")
|
||||
# Read in chunks and stop as soon as the limit is exceeded, so an oversized
|
||||
# upload never gets materialised in memory in full.
|
||||
chunks, size = [], 0
|
||||
while True:
|
||||
chunk = await file.read(1024 * 1024)
|
||||
if not chunk:
|
||||
break
|
||||
size += len(chunk)
|
||||
if size > MAX_UPLOAD_BYTES:
|
||||
raise HTTPException(status_code=400, detail="The file is too large (max 15 MB).")
|
||||
chunks.append(chunk)
|
||||
data = b"".join(chunks)
|
||||
if not data:
|
||||
raise HTTPException(status_code=400, detail="The file is empty.")
|
||||
try:
|
||||
name = await run_in_threadpool(
|
||||
lambda: store_uploaded_asset(config, file.filename or "asset", data, purpose=purpose or None)
|
||||
)
|
||||
except TenantValidationError as exc:
|
||||
_raise_http(exc)
|
||||
return AssetResponse(name=name)
|
||||
|
||||
|
||||
@router.post("/preview", response_model=PreviewResponse)
|
||||
async def preview_theme(body: PreviewRequest, config: TenantConfig = Depends(require_tenant_member)):
|
||||
try:
|
||||
styles = await run_in_threadpool(prepare_preview_styles, config, body.style_params)
|
||||
except TenantValidationError as exc:
|
||||
_raise_http(exc)
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
await _PREVIEW_SEMAPHORE.acquire()
|
||||
future = loop.run_in_executor(
|
||||
_PREVIEW_EXECUTOR,
|
||||
lambda: render_theme_preview(
|
||||
_theme_id_for(config),
|
||||
styles,
|
||||
artist=body.sample.artist,
|
||||
title=body.sample.title,
|
||||
lyrics=body.sample.lyrics,
|
||||
include_cdg=body.include_cdg,
|
||||
kinds=body.kinds,
|
||||
),
|
||||
)
|
||||
# The slot is released only when the render really finishes — a timed-out
|
||||
# render keeps running in its thread, so releasing early would let a burst
|
||||
# queue unbounded work behind it.
|
||||
future.add_done_callback(lambda _f: _PREVIEW_SEMAPHORE.release())
|
||||
try:
|
||||
images = await asyncio.wait_for(asyncio.shield(future), timeout=PREVIEW_TIMEOUT_S)
|
||||
except ThemePreviewError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc))
|
||||
except asyncio.TimeoutError:
|
||||
raise HTTPException(status_code=504, detail="The preview took too long to render. Try again.")
|
||||
return PreviewResponse(**images.as_data_urls())
|
||||
|
||||
|
||||
class SaveResponse(ThemeResponse):
|
||||
# In-progress tracks switched to the new theme by this save.
|
||||
refreshed_jobs: int = 0
|
||||
# The caller's finished tracks made with an older theme (re-render to update).
|
||||
outdated_job_ids: List[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class OutdatedJobsResponse(BaseModel):
|
||||
theme_updated_at: Optional[str] = None
|
||||
job_ids: List[str]
|
||||
|
||||
|
||||
class RerenderOutdatedResponse(BaseModel):
|
||||
started: List[str]
|
||||
failed: Dict[str, str]
|
||||
# Outdated tracks beyond this call's cap (ask again to start them).
|
||||
remaining: List[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
# Tracks started per bulk call: bounds the request time and the burst of
|
||||
# render/encode work it queues. The UI offers the remainder again.
|
||||
MAX_BULK_RERENDER = 20
|
||||
|
||||
|
||||
def _owner_scope(auth_result: AuthResult) -> Optional[str]:
|
||||
"""Every caller here is a portal member (require_tenant_member), and tenant
|
||||
portals are shared workspaces (backend/services/job_access.py), so theme
|
||||
refresh/rerender covers all of the portal's tracks, whoever submitted them."""
|
||||
return None
|
||||
|
||||
|
||||
@router.put("", response_model=SaveResponse)
|
||||
def save_theme(
|
||||
body: SaveRequest,
|
||||
config: TenantConfig = Depends(require_tenant_member),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
try:
|
||||
save_tenant_theme(config, body.style_params)
|
||||
theme = get_theme_for_editor(config)
|
||||
except TenantValidationError as exc:
|
||||
_raise_http(exc)
|
||||
theme_id = _theme_id_for(config)
|
||||
# Never fail the save over the follow-up work: the theme is already stored.
|
||||
refreshed = 0
|
||||
try:
|
||||
refreshed = refresh_inflight_jobs(config.id, theme_id)["updated"]
|
||||
except Exception:
|
||||
logger.exception(f"Tenant '{config.id}': refreshing in-progress tracks after theme save failed")
|
||||
outdated: List[str] = []
|
||||
try:
|
||||
outdated = outdated_jobs(config.id, theme_id, _owner_scope(auth_result))["job_ids"]
|
||||
except Exception:
|
||||
logger.exception(f"Tenant '{config.id}': listing outdated tracks after theme save failed")
|
||||
return SaveResponse(**theme, refreshed_jobs=refreshed, outdated_job_ids=outdated)
|
||||
|
||||
|
||||
@router.get("/outdated-jobs", response_model=OutdatedJobsResponse)
|
||||
def get_outdated_jobs(
|
||||
config: TenantConfig = Depends(require_tenant_member),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Finished tracks made with an older version of the theme (re-render to update)."""
|
||||
return OutdatedJobsResponse(**outdated_jobs(config.id, _theme_id_for(config), _owner_scope(auth_result)))
|
||||
|
||||
|
||||
@router.post("/rerender-outdated", response_model=RerenderOutdatedResponse)
|
||||
async def rerender_outdated(
|
||||
config: TenantConfig = Depends(require_tenant_member),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
):
|
||||
"""Re-render every outdated finished track with the current theme.
|
||||
|
||||
Quiet: no per-track "your video is ready" email/push (one per track would
|
||||
flood the inbox); the portal shows progress on each track.
|
||||
"""
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.theme_rerender_service import RerenderError, ThemeRerenderService
|
||||
|
||||
theme_id = _theme_id_for(config)
|
||||
job_ids = (await run_in_threadpool(
|
||||
outdated_jobs, config.id, theme_id, _owner_scope(auth_result)
|
||||
))["job_ids"]
|
||||
batch, remaining = job_ids[:MAX_BULK_RERENDER], job_ids[MAX_BULK_RERENDER:]
|
||||
job_manager = JobManager()
|
||||
service = ThemeRerenderService(job_manager)
|
||||
started: List[str] = []
|
||||
failed: Dict[str, str] = {}
|
||||
for job_id in batch:
|
||||
job = await run_in_threadpool(job_manager.get_job, job_id)
|
||||
if job is None:
|
||||
continue
|
||||
try:
|
||||
await service.start(
|
||||
job, theme_id=theme_id,
|
||||
requested_by=auth_result.user_email or "unknown",
|
||||
notify_customer=False,
|
||||
)
|
||||
started.append(job_id)
|
||||
except RerenderError as e:
|
||||
failed[job_id] = str(e)
|
||||
except Exception as e:
|
||||
logger.exception(f"[job:{job_id}] Bulk theme re-render failed to start")
|
||||
failed[job_id] = "Couldn't start the re-render."
|
||||
logger.info(
|
||||
f"Tenant '{config.id}': bulk re-render started {len(started)}, failed {len(failed)}, "
|
||||
f"remaining {len(remaining)}"
|
||||
)
|
||||
return RerenderOutdatedResponse(started=started, failed=failed, remaining=remaining)
|
||||
@@ -1,315 +0,0 @@
|
||||
"""Test-only API for the local E2E stack (``/api/test/*``).
|
||||
|
||||
Mounted ONLY when FAKE_WORKERS is on (``backend/main.py``), which startup
|
||||
refuses in production, and unauthenticated by design: the stack runs on
|
||||
emulators on a developer machine or CI runner. Web Playwright and mobile Maestro
|
||||
tests use it to reset state, seed scenarios, tune the fake workers, read
|
||||
captured email and complete fake Stripe checkouts.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import html
|
||||
import logging
|
||||
import os
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse
|
||||
from pydantic import BaseModel
|
||||
|
||||
from backend.config import get_settings
|
||||
from backend.services import fake_workers
|
||||
from backend.services.fake_workers import checkout as fake_checkout
|
||||
from backend.services.fake_workers.config import clear_fake_config, get_fake_config, set_fake_config
|
||||
from backend.services.fake_workers.outbox import clear_pushes, list_emails, list_pushes
|
||||
from backend.services.fake_workers.scenarios import SCENARIOS, ScenarioError, run_scenario
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/test", tags=["test-support"])
|
||||
|
||||
# Seeded by scripts/seed-local-data.py; kept across resets.
|
||||
PRESERVED_BLOB_PREFIXES = ("themes/",)
|
||||
|
||||
|
||||
class ScenarioRequest(BaseModel):
|
||||
name: str
|
||||
email: Optional[str] = None
|
||||
credits: Optional[int] = None
|
||||
artist: Optional[str] = None
|
||||
title: Optional[str] = None
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
def status() -> Dict[str, Any]:
|
||||
return {
|
||||
"fake_workers": True,
|
||||
"active_workers": fake_workers.active_worker_count(),
|
||||
"config": get_fake_config().to_dict(),
|
||||
"scenarios": SCENARIOS,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/reset")
|
||||
def reset() -> Dict[str, Any]:
|
||||
"""Wipe every Firestore document and every bucket object except the seed (themes)."""
|
||||
emulator = os.getenv("FIRESTORE_EMULATOR_HOST")
|
||||
if not emulator:
|
||||
raise HTTPException(status_code=409, detail="reset only runs against the Firestore emulator")
|
||||
project = get_settings().google_cloud_project
|
||||
resp = httpx.delete(
|
||||
f"http://{emulator}/emulator/v1/projects/{project}/databases/(default)/documents", timeout=30,
|
||||
)
|
||||
if resp.status_code >= 300:
|
||||
raise HTTPException(status_code=502, detail=f"emulator reset failed: {resp.status_code} {resp.text}")
|
||||
|
||||
from backend.services.storage_service import StorageService
|
||||
|
||||
bucket = StorageService().bucket
|
||||
deleted = 0
|
||||
for blob in list(bucket.list_blobs()):
|
||||
if not blob.name.startswith(PRESERVED_BLOB_PREFIXES):
|
||||
blob.delete()
|
||||
deleted += 1
|
||||
clear_fake_config()
|
||||
clear_pushes()
|
||||
return {"status": "reset", "blobs_deleted": deleted}
|
||||
|
||||
|
||||
@router.get("/fake-config")
|
||||
def read_config(email: Optional[str] = None) -> Dict[str, Any]:
|
||||
return get_fake_config(email=email).to_dict()
|
||||
|
||||
|
||||
@router.put("/fake-config")
|
||||
def update_config(updates: Dict[str, Any], email: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""Merge ``updates`` into the global knobs, or into ``?email=``'s per-user overrides."""
|
||||
try:
|
||||
return set_fake_config(updates, email=email).to_dict()
|
||||
except (ValueError, TypeError) as e:
|
||||
raise HTTPException(status_code=422, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/scenario")
|
||||
def scenario(body: ScenarioRequest) -> Dict[str, Any]:
|
||||
try:
|
||||
return run_scenario(body.name, email=body.email, credits=body.credits, artist=body.artist, title=body.title)
|
||||
except ScenarioError as e:
|
||||
raise HTTPException(status_code=422, detail=str(e))
|
||||
|
||||
|
||||
@router.get("/jobs/{job_id}/review-result")
|
||||
def review_result(job_id: str) -> Dict[str, Any]:
|
||||
"""What the review submitted: the saved corrected lines plus the instrumental / duet choice.
|
||||
|
||||
Lets Maestro assert on a review without going through the review auth.
|
||||
``lines`` is empty until a review was completed (corrections_updated.json exists).
|
||||
"""
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
|
||||
job = JobManager().get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="unknown job")
|
||||
state = job.state_data or {}
|
||||
path = ((job.file_urls or {}).get("lyrics") or {}).get("corrections_updated") \
|
||||
or f"jobs/{job_id}/lyrics/corrections_updated.json"
|
||||
storage = StorageService()
|
||||
lines = []
|
||||
if storage.file_exists(path):
|
||||
for seg in storage.download_json(path).get("corrected_segments") or []:
|
||||
lines.append({
|
||||
"text": seg.get("text"),
|
||||
"singer": seg.get("singer"),
|
||||
"words": [
|
||||
{"text": w.get("text"), "start_time": w.get("start_time"), "end_time": w.get("end_time")}
|
||||
for w in seg.get("words") or []
|
||||
],
|
||||
})
|
||||
return {
|
||||
"status": getattr(job.status, "value", job.status),
|
||||
"instrumental_selection": state.get("instrumental_selection"),
|
||||
"is_duet": bool(state.get("is_duet", False)),
|
||||
"lines": lines,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/emails")
|
||||
def emails(to: Optional[str] = None, limit: int = 20) -> Dict[str, Any]:
|
||||
return {"emails": list_emails(to, limit=limit)}
|
||||
|
||||
|
||||
@router.get("/push-outbox")
|
||||
def push_outbox(to: Optional[str] = None, limit: int = 50) -> Dict[str, Any]:
|
||||
"""Expo pushes captured instead of sent (newest first)."""
|
||||
return {"pushes": list_pushes(to, limit=limit)}
|
||||
|
||||
|
||||
@router.delete("/push-outbox")
|
||||
def clear_push_outbox() -> Dict[str, Any]:
|
||||
clear_pushes()
|
||||
return {"status": "cleared"}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ fake Stripe
|
||||
|
||||
_CHECKOUT_PAGE = """<!doctype html>
|
||||
<html><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>Fake Checkout</title>
|
||||
<style>body{{font-family:system-ui,sans-serif;max-width:420px;margin:48px auto;padding:0 16px}}
|
||||
button{{font-size:18px;padding:12px 20px;margin:8px 8px 0 0;border-radius:8px;border:0}}
|
||||
#pay{{background:#635bff;color:#fff}}#cancel{{background:#eee}}</style></head>
|
||||
<body><h1>Fake Checkout</h1>
|
||||
<p data-testid="fake-checkout-product">{product}</p>
|
||||
<p data-testid="fake-checkout-amount">${amount}</p>
|
||||
<p>{email}</p>
|
||||
<form method="post" action="{base}/api/test/fake-checkout/{sid}/pay" style="display:inline">
|
||||
<button id="pay" data-testid="fake-checkout-pay" type="submit">Pay</button></form>
|
||||
<form method="post" action="{base}/api/test/fake-checkout/{sid}/cancel" style="display:inline">
|
||||
<button id="cancel" data-testid="fake-checkout-cancel" type="submit">Cancel</button></form>
|
||||
</body></html>"""
|
||||
|
||||
|
||||
def _checkout_or_404(session_id: str) -> Dict[str, Any]:
|
||||
record = fake_checkout.get_checkout(session_id)
|
||||
if not record:
|
||||
raise HTTPException(status_code=404, detail="unknown fake checkout session")
|
||||
return record
|
||||
|
||||
|
||||
@router.get("/fake-checkout/{session_id}", response_class=HTMLResponse)
|
||||
def checkout_page(session_id: str) -> HTMLResponse:
|
||||
record = _checkout_or_404(session_id)
|
||||
return HTMLResponse(_CHECKOUT_PAGE.format(
|
||||
product=html.escape(record.get("product_name") or ""),
|
||||
amount=f"{(record.get('amount_cents') or 0) / 100:.2f}",
|
||||
email=html.escape(record.get("customer_email") or ""),
|
||||
base=fake_checkout.public_api_url(),
|
||||
sid=html.escape(session_id),
|
||||
))
|
||||
|
||||
|
||||
@router.post("/fake-checkout/{session_id}/pay")
|
||||
async def checkout_pay(session_id: str):
|
||||
"""Complete the session through the same handler as the real Stripe webhook."""
|
||||
from backend.api.routes.internal import TestWebhookRequest, test_webhook
|
||||
|
||||
record = _checkout_or_404(session_id)
|
||||
result = await test_webhook(
|
||||
TestWebhookRequest(
|
||||
event_type="checkout.session.completed",
|
||||
session_id=session_id,
|
||||
customer_email=record.get("customer_email") or "",
|
||||
metadata=record.get("metadata") or {},
|
||||
),
|
||||
auth_data=None,
|
||||
)
|
||||
if result.status not in ("processed", "already_processed"):
|
||||
raise HTTPException(status_code=500, detail=f"fake checkout completion failed: {result.message}")
|
||||
fake_checkout.mark_checkout(session_id, "complete")
|
||||
return RedirectResponse(fake_checkout.resolve_url(record.get("success_url"), session_id), status_code=303)
|
||||
|
||||
|
||||
@router.post("/fake-checkout/{session_id}/cancel")
|
||||
def checkout_cancel(session_id: str):
|
||||
record = _checkout_or_404(session_id)
|
||||
fake_checkout.mark_checkout(session_id, "cancelled")
|
||||
return RedirectResponse(fake_checkout.resolve_url(record.get("cancel_url"), session_id), status_code=303)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ fake in-app purchases
|
||||
|
||||
|
||||
class FakeIapPurchaseRequest(BaseModel):
|
||||
"""A store purchase, as RevenueCat would report it. Give ``email`` or ``app_user_id``."""
|
||||
product_id: str
|
||||
email: Optional[str] = None
|
||||
app_user_id: Optional[str] = None
|
||||
transaction_id: Optional[str] = None
|
||||
store: str = "APP_STORE"
|
||||
price: Optional[float] = None
|
||||
takehome_percentage: Optional[float] = None
|
||||
country_code: str = "US"
|
||||
|
||||
|
||||
class FakeIapRefundRequest(BaseModel):
|
||||
"""Refunds debit the account the purchase granted, so no user is needed."""
|
||||
transaction_id: str
|
||||
product_id: str
|
||||
store: str = "APP_STORE"
|
||||
|
||||
|
||||
def _fake_app_user_id(email: Optional[str], app_user_id: Optional[str]) -> str:
|
||||
from backend.services.user_service import get_user_service
|
||||
|
||||
if app_user_id:
|
||||
return app_user_id
|
||||
if not email:
|
||||
raise HTTPException(status_code=422, detail="email or app_user_id is required")
|
||||
user_service = get_user_service()
|
||||
user_service.get_or_create_user(email)
|
||||
return user_service.ensure_app_user_id(email)
|
||||
|
||||
|
||||
def _fake_iap_result(event: Dict[str, Any]) -> Dict[str, Any]:
|
||||
from backend.services.iap_service import get_iap_service
|
||||
|
||||
result = get_iap_service().handle_event(event)
|
||||
return {
|
||||
"action": result.action,
|
||||
"credits": result.credits,
|
||||
"store": event["store"],
|
||||
"transaction_id": event["transaction_id"],
|
||||
}
|
||||
|
||||
|
||||
# US store prices of the credit packs (spec: $9.99 / $23.99 / $34.99 / $59.99).
|
||||
_FAKE_IAP_PRICES = {
|
||||
"nomad_credits_1": 9.99,
|
||||
"nomad_credits_3": 23.99,
|
||||
"nomad_credits_5": 34.99,
|
||||
"nomad_credits_10": 59.99,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/fake-iap/purchase")
|
||||
def fake_iap_purchase(body: FakeIapPurchaseRequest) -> Dict[str, Any]:
|
||||
"""Grant a store credit pack through the same IapService as the RevenueCat webhook."""
|
||||
import uuid
|
||||
|
||||
price = body.price if body.price is not None else _FAKE_IAP_PRICES.get(body.product_id, 0.0)
|
||||
event: Dict[str, Any] = {
|
||||
"type": "NON_RENEWING_PURCHASE",
|
||||
"app_user_id": _fake_app_user_id(body.email, body.app_user_id),
|
||||
"product_id": body.product_id,
|
||||
"transaction_id": body.transaction_id or f"fake-{uuid.uuid4().hex}",
|
||||
"store": body.store,
|
||||
"price": price,
|
||||
"price_in_purchased_currency": price,
|
||||
"currency": "USD",
|
||||
"country_code": body.country_code,
|
||||
"environment": "SANDBOX",
|
||||
}
|
||||
if body.takehome_percentage is not None:
|
||||
event["takehome_percentage"] = body.takehome_percentage
|
||||
return _fake_iap_result(event)
|
||||
|
||||
|
||||
@router.post("/fake-iap/refund")
|
||||
def fake_iap_refund(body: FakeIapRefundRequest) -> Dict[str, Any]:
|
||||
"""Refund a fake store purchase (RevenueCat CANCELLATION / CUSTOMER_SUPPORT)."""
|
||||
price = -_FAKE_IAP_PRICES.get(body.product_id, 0.0)
|
||||
event: Dict[str, Any] = {
|
||||
"type": "CANCELLATION",
|
||||
"cancel_reason": "CUSTOMER_SUPPORT",
|
||||
"app_user_id": "",
|
||||
"product_id": body.product_id,
|
||||
"transaction_id": body.transaction_id,
|
||||
"store": body.store,
|
||||
"price": price,
|
||||
"price_in_purchased_currency": price,
|
||||
"currency": "USD",
|
||||
"environment": "SANDBOX",
|
||||
}
|
||||
return _fake_iap_result(event)
|
||||
@@ -6,15 +6,13 @@ These endpoints are public (no authentication required) to allow theme selection
|
||||
before job creation.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from fastapi import APIRouter, HTTPException
|
||||
|
||||
from backend.models.theme import ThemeDetailResponse, ThemesListResponse
|
||||
from backend.services.theme_service import get_theme_service
|
||||
from backend.i18n import t, get_locale_from_request
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -22,7 +20,7 @@ router = APIRouter(prefix="/themes", tags=["themes"])
|
||||
|
||||
|
||||
@router.get("", response_model=ThemesListResponse)
|
||||
def list_themes(request: Request) -> ThemesListResponse:
|
||||
async def list_themes() -> ThemesListResponse:
|
||||
"""
|
||||
List all available themes.
|
||||
|
||||
@@ -31,7 +29,6 @@ def list_themes(request: Request) -> ThemesListResponse:
|
||||
|
||||
This endpoint is public and does not require authentication.
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
try:
|
||||
theme_service = get_theme_service()
|
||||
themes = theme_service.list_themes()
|
||||
@@ -40,41 +37,12 @@ def list_themes(request: Request) -> ThemesListResponse:
|
||||
logger.error(f"Error listing themes: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=t(locale, "themes.themesLoadError")
|
||||
detail="Failed to load themes"
|
||||
)
|
||||
|
||||
|
||||
@router.get("/translation-preview")
|
||||
async def get_translation_preview(language: str, request: Request) -> dict:
|
||||
"""
|
||||
Preview frame of a translated-lyrics karaoke video in ``language``.
|
||||
|
||||
Renders the default theme's karaoke frame with sample lyrics and their
|
||||
(pre-generated) translation beneath each line, so users can see the option
|
||||
before enabling it at job creation. Public, like the other theme endpoints.
|
||||
|
||||
Returns:
|
||||
Object with ``image`` (JPEG data URL)
|
||||
|
||||
Raises:
|
||||
400: Unsupported language
|
||||
"""
|
||||
from backend.services.lyrics_translation import normalize_language
|
||||
from backend.services.translation_preview import render_translation_preview
|
||||
|
||||
code = normalize_language(language)
|
||||
if code is None:
|
||||
raise HTTPException(status_code=400, detail=f"Unsupported language: {language}")
|
||||
try:
|
||||
image = await asyncio.to_thread(render_translation_preview, code)
|
||||
except Exception as e:
|
||||
logger.error(f"Error rendering translation preview for {code}: {e}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail="Could not render the translation preview")
|
||||
return {"image": image}
|
||||
|
||||
|
||||
@router.get("/{theme_id}", response_model=ThemeDetailResponse)
|
||||
def get_theme(theme_id: str, request: Request) -> ThemeDetailResponse:
|
||||
async def get_theme(theme_id: str) -> ThemeDetailResponse:
|
||||
"""
|
||||
Get detailed information about a specific theme.
|
||||
|
||||
@@ -90,7 +58,6 @@ def get_theme(theme_id: str, request: Request) -> ThemeDetailResponse:
|
||||
Raises:
|
||||
404: Theme not found
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
try:
|
||||
theme_service = get_theme_service()
|
||||
theme = theme_service.get_theme(theme_id)
|
||||
@@ -98,7 +65,7 @@ def get_theme(theme_id: str, request: Request) -> ThemeDetailResponse:
|
||||
if theme is None:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=t(locale, "themes.themeNotFound", theme_id=theme_id)
|
||||
detail=f"Theme not found: {theme_id}"
|
||||
)
|
||||
|
||||
return ThemeDetailResponse(theme=theme)
|
||||
@@ -108,12 +75,12 @@ def get_theme(theme_id: str, request: Request) -> ThemeDetailResponse:
|
||||
logger.error(f"Error getting theme {theme_id}: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=t(locale, "themes.themeLoadError")
|
||||
detail="Failed to load theme"
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{theme_id}/preview")
|
||||
def get_theme_preview(theme_id: str, request: Request) -> dict:
|
||||
async def get_theme_preview(theme_id: str) -> dict:
|
||||
"""
|
||||
Get a signed URL for the theme's preview image.
|
||||
|
||||
@@ -129,21 +96,20 @@ def get_theme_preview(theme_id: str, request: Request) -> dict:
|
||||
Raises:
|
||||
404: Theme not found or no preview available
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
try:
|
||||
theme_service = get_theme_service()
|
||||
|
||||
if not theme_service.theme_exists(theme_id):
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=t(locale, "themes.themeNotFound", theme_id=theme_id)
|
||||
detail=f"Theme not found: {theme_id}"
|
||||
)
|
||||
|
||||
theme = theme_service.get_theme(theme_id)
|
||||
if theme is None or theme.preview_url is None:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=t(locale, "themes.previewNotAvailable", theme_id=theme_id)
|
||||
detail=f"No preview available for theme: {theme_id}"
|
||||
)
|
||||
|
||||
return {"preview_url": theme.preview_url}
|
||||
@@ -153,12 +119,12 @@ def get_theme_preview(theme_id: str, request: Request) -> dict:
|
||||
logger.error(f"Error getting preview for theme {theme_id}: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=t(locale, "themes.previewLoadError")
|
||||
detail="Failed to load theme preview"
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{theme_id}/youtube-description")
|
||||
def get_theme_youtube_description(theme_id: str, request: Request) -> dict:
|
||||
async def get_theme_youtube_description(theme_id: str) -> dict:
|
||||
"""
|
||||
Get the YouTube description template for a theme.
|
||||
|
||||
@@ -175,14 +141,13 @@ def get_theme_youtube_description(theme_id: str, request: Request) -> dict:
|
||||
Raises:
|
||||
404: Theme not found
|
||||
"""
|
||||
locale = get_locale_from_request(request)
|
||||
try:
|
||||
theme_service = get_theme_service()
|
||||
|
||||
if not theme_service.theme_exists(theme_id):
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=t(locale, "themes.youtubeDescriptionNotFound", theme_id=theme_id)
|
||||
detail=f"Theme not found: {theme_id}"
|
||||
)
|
||||
|
||||
description = theme_service.get_youtube_description(theme_id)
|
||||
@@ -193,5 +158,5 @@ def get_theme_youtube_description(theme_id: str, request: Request) -> dict:
|
||||
logger.error(f"Error getting YouTube description for theme {theme_id}: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=t(locale, "themes.youtubeDescriptionLoadError")
|
||||
detail="Failed to load YouTube description"
|
||||
)
|
||||
+575
-1782
File diff suppressed because it is too large.
Load diff
@@ -1,35 +0,0 @@
|
||||
# Cloud Build config for GPU base image
|
||||
#
|
||||
# Builds the CUDA + PyTorch + models base image used by the audio worker.
|
||||
# This image is large (~10GB) so builds take 15-30 minutes.
|
||||
#
|
||||
# Trigger manually:
|
||||
# gcloud builds submit --config=backend/cloudbuild-gpu-base.yaml --project=nomadkaraoke --timeout=3600s
|
||||
#
|
||||
# The app-layer Dockerfile uses this as its BASE_IMAGE for audio worker deploys.
|
||||
|
||||
steps:
|
||||
- name: 'gcr.io/cloud-builders/docker'
|
||||
args:
|
||||
- 'build'
|
||||
- '-f'
|
||||
- 'backend/Dockerfile.gpu-base'
|
||||
- '-t'
|
||||
- 'us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-gpu-base:latest'
|
||||
- '-t'
|
||||
- 'us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-gpu-base:$BUILD_ID'
|
||||
- '--build-arg'
|
||||
- 'BUILD_DATE=$_BUILD_DATE'
|
||||
- '--build-arg'
|
||||
- 'BUILD_ID=$BUILD_ID'
|
||||
- '.'
|
||||
|
||||
images:
|
||||
- 'us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-gpu-base:latest'
|
||||
- 'us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-gpu-base:$BUILD_ID'
|
||||
|
||||
timeout: '3600s'
|
||||
|
||||
options:
|
||||
machineType: 'E2_HIGHCPU_32'
|
||||
diskSizeGb: 100
|
||||
+18
-493
@@ -4,17 +4,9 @@ Configuration management for the karaoke generation backend.
|
||||
import os
|
||||
import logging
|
||||
from typing import Optional, Dict
|
||||
from dotenv import load_dotenv
|
||||
from pydantic_settings import BaseSettings
|
||||
from google.cloud import secretmanager
|
||||
|
||||
# google.cloud.secretmanager is imported lazily in get_secret() — importing it
|
||||
# (and the google.api_core/grpc graph behind it) costs ~1.7s and config is
|
||||
# imported by every backend entrypoint (cold-start work, 2026-09-22).
|
||||
|
||||
# Load .env for local development (no-op when absent, e.g. in Cloud Run).
|
||||
# Must run before the Settings class body below, which reads os.getenv at
|
||||
# class-definition time. Real environment variables take precedence.
|
||||
load_dotenv()
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -28,27 +20,7 @@ class Settings(BaseSettings):
|
||||
gcs_temp_bucket: str = os.getenv("GCS_TEMP_BUCKET", "karaoke-gen-temp")
|
||||
gcs_output_bucket: str = os.getenv("GCS_OUTPUT_BUCKET", "karaoke-gen-outputs")
|
||||
firestore_collection: str = os.getenv("FIRESTORE_COLLECTION", "jobs")
|
||||
|
||||
# Nomad master fast-sync: push freshly-published Nomad 720p masters straight to the
|
||||
# Divebar GCS mirror so kjbox (which mirrors that folder every 5 min) picks them up
|
||||
# within minutes instead of waiting for the nightly Drive->GCS VM. Nomad-brand only.
|
||||
divebar_files_bucket: str = os.getenv("DIVEBAR_FILES_BUCKET", "nomadkaraoke-divebar-files")
|
||||
nomad_master_gcs_prefix: str = os.getenv("NOMAD_MASTER_GCS_PREFIX", "files/Nomad Karaoke/MP4-720p")
|
||||
nomad_master_fast_sync_enabled: bool = os.getenv("NOMAD_MASTER_FAST_SYNC_ENABLED", "true").lower() == "true"
|
||||
# Original-vocals guide: a padded isolated-vocals file (silence[intro] + vocals) pushed to
|
||||
# a sibling prefix in the same Divebar bucket, which kjbox mirrors into NOMAD-vocals-padded/
|
||||
# to power the "Original Vocals" sing-along slider. Same brand gating as the master push.
|
||||
nomad_vocals_guide_gcs_prefix: str = os.getenv("NOMAD_VOCALS_GUIDE_GCS_PREFIX", "files/Nomad Karaoke/vocals-padded")
|
||||
|
||||
# KaraokeNerds community catalog: our own daily-refreshed copy of KN's community
|
||||
# (web-only, directly-playable) tracks, written by the authorized `kn-data-sync`
|
||||
# export job. The community-version check reads THIS, never scrapes karaokenerds.com.
|
||||
# Source of truth is BigQuery `karaoke_decide.karaokenerds_community`; the export job
|
||||
# also mirrors it to this gzipped-JSON GCS object, which the backend loads in-process.
|
||||
kn_community_bucket: str = os.getenv("KN_COMMUNITY_BUCKET", "nomadkaraoke-kn-data")
|
||||
kn_community_blob: str = os.getenv("KN_COMMUNITY_BLOB", "community/community-data-latest.json.gz")
|
||||
kn_community_ttl_seconds: int = int(os.getenv("KN_COMMUNITY_TTL_SECONDS", "21600")) # 6h
|
||||
|
||||
|
||||
# Audio Separator API (for GPU processing)
|
||||
audio_separator_api_url: Optional[str] = os.getenv("AUDIO_SEPARATOR_API_URL")
|
||||
|
||||
@@ -60,17 +32,9 @@ class Settings(BaseSettings):
|
||||
|
||||
# Authentication
|
||||
admin_tokens: Optional[str] = os.getenv("ADMIN_TOKENS") # Comma-separated list
|
||||
scheduler_service_account: str = os.getenv(
|
||||
"SCHEDULER_SERVICE_ACCOUNT",
|
||||
"karaoke-backend@nomadkaraoke.iam.gserviceaccount.com"
|
||||
)
|
||||
|
||||
# Application
|
||||
environment: str = os.getenv("ENVIRONMENT", "development")
|
||||
# Event-loop stall watchdog (backend/services/loop_watchdog.py): logs the
|
||||
# blocking stack when the single uvicorn loop stops ticking for >1s, and
|
||||
# records stalls >=5s to Firestore client_events (production only).
|
||||
loop_watchdog_enabled: bool = os.getenv("LOOP_WATCHDOG_ENABLED", "true").lower() in ("true", "1", "yes")
|
||||
log_level: str = os.getenv("LOG_LEVEL", "INFO")
|
||||
|
||||
# Processing
|
||||
@@ -86,176 +50,24 @@ class Settings(BaseSettings):
|
||||
|
||||
# Agentic AI Correction (for lyrics correction via LLM)
|
||||
# Only used when skip_correction=false
|
||||
# When enabled, uses Gemini (Developer API, key GEMINI_API_KEY) for intelligent lyrics correction
|
||||
# When enabled, uses Gemini via Vertex AI for intelligent lyrics correction
|
||||
use_agentic_ai: bool = os.getenv("USE_AGENTIC_AI", "true").lower() in ("true", "1", "yes")
|
||||
agentic_ai_model: str = os.getenv("AGENTIC_AI_MODEL", "gemini/gemini-3.8-flash")
|
||||
agentic_ai_model: str = os.getenv("AGENTIC_AI_MODEL", "vertexai/gemini-3-flash-preview")
|
||||
# Timeout for agentic correction in seconds. If correction takes longer, abort and
|
||||
# use uncorrected transcription - human review will fix any issues.
|
||||
agentic_correction_timeout_seconds: int = int(os.getenv("AGENTIC_CORRECTION_TIMEOUT_SECONDS", "180"))
|
||||
|
||||
# Lyrics transcription outer (safety-net) timeout, scaled by audio duration.
|
||||
# Long inputs (e.g. 30+ minute mashups) legitimately take longer to transcribe,
|
||||
# so the timeout = clamp(floor, duration * per_audio_second, cap). The cap must
|
||||
# stay under the Cloud Run Job task timeout so we surface a clean RuntimeError
|
||||
# before the platform hard-kills the task. The lyrics-transcription-job sets the
|
||||
# task timeout to 3000s and overrides this cap to 2700s via env; the 1700s
|
||||
# default here is the safe value if that env is ever unset (under an 1800s task).
|
||||
transcription_timeout_floor_seconds: int = int(os.getenv("TRANSCRIPTION_TIMEOUT_FLOOR_SECONDS", "1200"))
|
||||
transcription_timeout_cap_seconds: int = int(os.getenv("TRANSCRIPTION_TIMEOUT_CAP_SECONDS", "1700"))
|
||||
transcription_timeout_per_audio_second: float = float(os.getenv("TRANSCRIPTION_TIMEOUT_PER_AUDIO_SECOND", "1.0"))
|
||||
|
||||
# AI Credit Evaluation (anti-abuse gating for free credits)
|
||||
# When enabled, welcome and feedback credit grants are evaluated by Gemini
|
||||
# before granting. Fail-CLOSED: if evaluation fails, the decision becomes
|
||||
# "pending_review" (no auto-grant; an admin is notified to decide).
|
||||
credit_eval_enabled: bool = os.getenv("CREDIT_EVAL_ENABLED", "true").lower() in ("true", "1", "yes")
|
||||
credit_eval_model: str = os.getenv("CREDIT_EVAL_MODEL", "gemini-3.8-flash")
|
||||
|
||||
# Custom Lyrics (LLM-powered) — used by the Custom Lyrics mode in the
|
||||
# Edit All Lyrics modal. Stateless service: takes operator-provided
|
||||
# custom-lyrics text/file and returns transformed lyric lines.
|
||||
custom_lyrics_model: str = os.getenv("CUSTOM_LYRICS_MODEL", "gemini-3.8-flash")
|
||||
custom_lyrics_max_file_mb: int = int(os.getenv("CUSTOM_LYRICS_MAX_FILE_MB", "5"))
|
||||
custom_lyrics_max_input_lines: int = int(os.getenv("CUSTOM_LYRICS_MAX_INPUT_LINES", "500"))
|
||||
custom_lyrics_max_iterations: int = int(os.getenv("CUSTOM_LYRICS_MAX_ITERATIONS", "4"))
|
||||
custom_lyrics_default_strictness: str = os.getenv("CUSTOM_LYRICS_DEFAULT_STRICTNESS", "balanced")
|
||||
custom_lyrics_max_output_lines_multiplier: float = float(os.getenv("CUSTOM_LYRICS_MAX_OUTPUT_LINES_MULTIPLIER", "2.0"))
|
||||
|
||||
# Translated lyrics — per-job opt-in (Job.translation_language): the reviewed
|
||||
# lyrics are translated line-by-line just before the karaoke video renders.
|
||||
lyrics_translation_model: str = os.getenv("LYRICS_TRANSLATION_MODEL", "gemini-3.8-flash")
|
||||
|
||||
# AI auto-correct suggestions — opt-in, user-triggered from the lyrics
|
||||
# review UI. Stateless: one whole-song LLM call returning word-level
|
||||
# suggestions the reviewer accepts/rejects individually.
|
||||
auto_correct_model: str = os.getenv("AUTO_CORRECT_MODEL", "gemini-3.8-flash")
|
||||
# Models used by the multi-model "compare" mode (semicolon-separated —
|
||||
# Cloud Run --set-env-vars is comma-delimited). Empty = single-model only.
|
||||
auto_correct_compare_models: str = os.getenv("AUTO_CORRECT_COMPARE_MODELS", "")
|
||||
# Proactively generate (and cache) auto-correct suggestions once
|
||||
# transcription + references are ready, so the review UI has them instantly.
|
||||
# This flag gates whether the lyrics worker fires the trigger; the actual
|
||||
# multi-model LLM call + caching runs on the API service (auto_correct_worker),
|
||||
# which holds the model credentials — the lyrics-transcription-job itself does
|
||||
# NOT need ANTHROPIC_API_KEY / AUTO_CORRECT_COMPARE_MODELS. Best-effort:
|
||||
# failures never block or fail the job. Default off.
|
||||
auto_correct_proactive_enabled: bool = os.getenv(
|
||||
"AUTO_CORRECT_PROACTIVE_ENABLED", "false"
|
||||
).lower() in ("true", "1", "yes")
|
||||
|
||||
# Auto-approval ENFORCEMENT: when the auto-approval scorer is fully confident
|
||||
# (see backend/services/auto_approval/), skip the combined review screens and
|
||||
# go straight to render. Scoring + verdict recording happen on every job
|
||||
# regardless of this flag; it only gates actually skipping the human.
|
||||
# Per-job opt-out via Job.review_mode="always_review". Kill switch: false.
|
||||
auto_approval_enforce_enabled: bool = os.getenv(
|
||||
"AUTO_APPROVAL_ENFORCE_ENABLED", "true"
|
||||
).lower() in ("true", "1", "yes")
|
||||
|
||||
# Backing decider ENFORCEMENT (Phase 2B): allow a confident WITH_BACKING
|
||||
# verdict from the 3-stem decider to auto-select the with-backing
|
||||
# instrumental. Shadow-validated 2026-08-28/29 (20-job corpus + live
|
||||
# batteries + human review pass: 10/11 pick agreement, the one
|
||||
# auto-eligible job shipped identically to the human) -> default ON.
|
||||
# Default lives HERE, not per-environment env vars, because the executor
|
||||
# runs in BOTH the backend service and the audio-separation Cloud Run Job
|
||||
# (independent env; a service-only env flip left the audio_worker trigger
|
||||
# reading false). Kill switch: set false in ci.yml backend deploy env
|
||||
# AND infrastructure audio-separation-job envs (or revert this default).
|
||||
auto_approval_backing_keep_enabled: bool = os.getenv(
|
||||
"AUTO_APPROVAL_BACKING_KEEP_ENABLED", "true"
|
||||
).lower() in ("true", "1", "yes")
|
||||
|
||||
# Timing-plausibility gate: when the text signals say AUTO, check the word
|
||||
# timing against the lead-vocal stem (backend/services/auto_approval/
|
||||
# timing_check.py) and demote to review when it fires. Only ever ADDS
|
||||
# review (never auto-ships more), so it defaults on. Code-level default for
|
||||
# the same reason as backing_keep above (executor runs in both the backend
|
||||
# service and the audio-separation job). Kill switch: set false in ci.yml
|
||||
# backend deploy env AND the infrastructure audio-separation-job envs.
|
||||
auto_approval_timing_gate_enabled: bool = os.getenv(
|
||||
"AUTO_APPROVAL_TIMING_GATE_ENABLED", "true"
|
||||
).lower() in ("true", "1", "yes")
|
||||
|
||||
# Review hold: when stems are still separating after lyrics finish, keep
|
||||
# the job out of review until the audio-worker re-score has run and the
|
||||
# review is fully prepared (backend/services/auto_approval/review_gate.py). Code-level default for
|
||||
# the same two-environment reason as above. Kill switch: false.
|
||||
auto_approval_audio_hold_enabled: bool = os.getenv(
|
||||
"AUTO_APPROVAL_AUDIO_HOLD_ENABLED", "true"
|
||||
).lower() in ("true", "1", "yes")
|
||||
|
||||
# Match judge — artist/title formatting + match-acceptance for the job
|
||||
# submission flow. Deterministic + catalog layers always run; the light AI
|
||||
# layer (Vertex Gemini Flash) fires only when those aren't confident.
|
||||
match_judge_enabled: bool = os.getenv("MATCH_JUDGE_ENABLED", "true").lower() in (
|
||||
"true", "1", "yes",
|
||||
)
|
||||
match_judge_model: str = os.getenv("MATCH_JUDGE_MODEL", "gemini-3.8-flash")
|
||||
# Bounds the AI call so a hung model never strands the submission UI. Kept
|
||||
# generous because Vertex returns 504 DEADLINE_EXCEEDED if generation can't
|
||||
# finish within this deadline (3s was too tight for gemini-3.5-flash); the
|
||||
# call still runs off the critical path and degrades to "none" on timeout.
|
||||
match_judge_timeout_ms: int = int(os.getenv("MATCH_JUDGE_TIMEOUT_MS", "12000"))
|
||||
|
||||
# Batch karaoke-filename parser (kjbox download-naming). Reuses the Vertex
|
||||
# Gemini stack. Larger timeout than match_judge because it batches ~200 items.
|
||||
parse_titles_enabled: bool = os.getenv("PARSE_TITLES_ENABLED", "true").lower() in (
|
||||
"1", "true", "yes",
|
||||
)
|
||||
parse_titles_model: str = os.getenv("PARSE_TITLES_MODEL", "gemini-3.8-flash")
|
||||
parse_titles_timeout_ms: int = int(os.getenv("PARSE_TITLES_TIMEOUT_MS", "20000"))
|
||||
parse_titles_max_items: int = int(os.getenv("PARSE_TITLES_MAX_ITEMS", "200"))
|
||||
# Items per Gemini call: large client batches are split into concurrent
|
||||
# chunks of this size so one call never outgrows parse_titles_timeout_ms
|
||||
# (a 100-item single call did, blanking the whole batch).
|
||||
parse_titles_chunk_size: int = int(os.getenv("PARSE_TITLES_CHUNK_SIZE", "10"))
|
||||
|
||||
# Cloud Tasks (for scalable worker coordination)
|
||||
# When enabled, workers are triggered via Cloud Tasks for guaranteed delivery
|
||||
# When disabled (default), workers are triggered via direct HTTP (for development)
|
||||
enable_cloud_tasks: bool = os.getenv("ENABLE_CLOUD_TASKS", "false").lower() in ("true", "1", "yes")
|
||||
gcp_region: str = os.getenv("GCP_REGION", "us-central1")
|
||||
# ALL Cloud Tasks queues live in us-central1, but workers running in other
|
||||
# regions (the GPU audio-separation-job sets GCP_REGION to its GPU region)
|
||||
# must still enqueue to the real queue location — building queue paths from
|
||||
# gcp_region 404s from those workers ("Queue does not exist").
|
||||
cloud_tasks_region: str = os.getenv("CLOUD_TASKS_REGION", "us-central1")
|
||||
# Region of the latency-critical CPU Cloud Run Jobs (audio-download-job,
|
||||
# lyrics-transcription-job, bulk-search-job, video-encoding-job). Cloud Run
|
||||
# Jobs in us-central1 queue 2-5 min before the container starts (measured
|
||||
# 2026-09-28 with a tiny sample image: us-central1 4-5 min vs us-east4 8-19s),
|
||||
# independent of image size or CPU. video-encoding-job reads ~0.3 GB/run from
|
||||
# the us-central1 bucket cross-region (~$1.60/mo, 2026-09-29 model). Rollback:
|
||||
# recreate the jobs in us-central1 via Pulumi, then set CPU_JOBS_REGION on the
|
||||
# service AND the jobs (see docs/TROUBLESHOOTING.md). Deliberately separate from gcp_region,
|
||||
# which these workers use for other resources (queues, encoding VM zones).
|
||||
cpu_jobs_region: str = os.getenv("CPU_JOBS_REGION", "us-east4")
|
||||
|
||||
# Cloud Run Jobs (for long-running video encoding)
|
||||
# When enabled AND enable_cloud_tasks is true, video worker uses Cloud Run Jobs
|
||||
# instead of Cloud Tasks. This supports encoding times >30 minutes (up to 24 hours).
|
||||
# Default is false - Cloud Tasks is sufficient for most videos (15-20 min).
|
||||
# Inside a Cloud Run Job (Cloud Run sets CLOUD_RUN_JOB) both flags default to
|
||||
# true: a worker job that triggers the next stage (audio worker resuming a
|
||||
# parked encode, auto-approval starting a render) must never fall back to
|
||||
# Cloud Tasks -> a BackgroundTask on the API service. That runs the whole
|
||||
# video worker on the API's event loop (2026-10-19: 52s + 81s freezes,
|
||||
# ~30 requests each). Explicit env values still win.
|
||||
use_cloud_run_jobs_for_video: bool = os.getenv(
|
||||
"USE_CLOUD_RUN_JOBS_FOR_VIDEO", "true" if os.getenv("CLOUD_RUN_JOB") else "false"
|
||||
).lower() in ("true", "1", "yes")
|
||||
|
||||
# When enabled AND enable_cloud_tasks is true, the post-review render worker
|
||||
# also runs as a Cloud Run Job (reusing video-encoding-job with an args
|
||||
# override). As a BackgroundTask on the service it was killed by every
|
||||
# deploy rollout that landed mid-render — Cloud Run gives instances only
|
||||
# 10s after SIGTERM, far less than a render takes, so the job froze at
|
||||
# rendering_video until the 45-min stuck-job sweep (incident 2026-09-13,
|
||||
# job 41e06b90; same class as incident 2026-03-08 for the video worker).
|
||||
use_cloud_run_jobs_for_render: bool = os.getenv(
|
||||
"USE_CLOUD_RUN_JOBS_FOR_RENDER", "true" if os.getenv("CLOUD_RUN_JOB") else "false"
|
||||
).lower() in ("true", "1", "yes")
|
||||
use_cloud_run_jobs_for_video: bool = os.getenv("USE_CLOUD_RUN_JOBS_FOR_VIDEO", "false").lower() in ("true", "1", "yes")
|
||||
|
||||
# GCE Encoding Worker (for high-performance video encoding)
|
||||
# When enabled, video encoding is offloaded to a dedicated C4 GCE instance
|
||||
@@ -265,23 +77,12 @@ class Settings(BaseSettings):
|
||||
encoding_worker_url: Optional[str] = os.getenv("ENCODING_WORKER_URL") # e.g., http://136.119.50.148:8080
|
||||
encoding_worker_api_key: Optional[str] = os.getenv("ENCODING_WORKER_API_KEY")
|
||||
|
||||
# Optional capacity-fallback VMs in alternate zones. JSON list of objects:
|
||||
# '[{"vm":"encoding-worker-fallback-a","zone":"us-central1-a","ip":"34.x.x.x"}]'
|
||||
# Empty by default. After `pulumi up` provisions the fallback VMs (Phase 3
|
||||
# of the capacity-resilience plan), populate this var with their public
|
||||
# IPs to enable multi-zone failover when us-central1-c is exhausted.
|
||||
encoding_worker_fallback_vms: Optional[str] = os.getenv("ENCODING_WORKER_FALLBACK_VMS")
|
||||
|
||||
# GCE Preview Encoding (for faster preview video generation)
|
||||
# When enabled, preview video encoding during lyrics review is offloaded to the GCE worker.
|
||||
# This reduces preview generation time from 60+ seconds to ~15-20 seconds.
|
||||
# Requires use_gce_encoding to be enabled and the GCE worker to support /encode-preview endpoint.
|
||||
use_gce_preview_encoding: bool = os.getenv("USE_GCE_PREVIEW_ENCODING", "false").lower() in ("true", "1", "yes")
|
||||
|
||||
# Karaoke-Decide main API: catalog autocomplete, and the partner call that
|
||||
# erases a deleted account's decide data (DELETE /api/partner/gen/users)
|
||||
karaoke_decide_api_url: str = os.getenv("KARAOKE_DECIDE_API_URL", "https://decide.nomadkaraoke.com")
|
||||
|
||||
# Storage paths
|
||||
temp_dir: str = os.getenv("TEMP_DIR", "/tmp/karaoke-gen")
|
||||
|
||||
@@ -296,28 +97,7 @@ class Settings(BaseSettings):
|
||||
# This is required for torrent downloads since Cloud Run doesn't support BitTorrent.
|
||||
flacfetch_api_url: Optional[str] = os.getenv("FLACFETCH_API_URL") # e.g., http://10.0.0.5:8080
|
||||
flacfetch_api_key: Optional[str] = os.getenv("FLACFETCH_API_KEY")
|
||||
|
||||
# TornadoAPI: tried first for YouTube downloads when the key is set, with flacfetch as fallback.
|
||||
# Set YOUTUBE_DOWNLOAD_PROVIDER=flacfetch to switch it off without a code change.
|
||||
tornado_api_key: Optional[str] = os.getenv("TORNADO_API_KEY")
|
||||
youtube_download_provider: str = os.getenv("YOUTUBE_DOWNLOAD_PROVIDER", "tornado")
|
||||
|
||||
# Flacfetch client retry settings
|
||||
# Retries help handle transient network failures when calling the flacfetch service
|
||||
flacfetch_retry_max_attempts: int = int(os.getenv("FLACFETCH_RETRY_MAX_ATTEMPTS", "9")) # Total attempts (1 initial + 8 retries)
|
||||
flacfetch_retry_min_wait: float = float(os.getenv("FLACFETCH_RETRY_MIN_WAIT", "10.0")) # seconds
|
||||
flacfetch_retry_max_wait: float = float(os.getenv("FLACFETCH_RETRY_MAX_WAIT", "60.0")) # seconds
|
||||
|
||||
# Search retry settings are much lighter than the download defaults above:
|
||||
# search is called synchronously from user-facing requests that sit behind
|
||||
# Cloudflare's ~100s edge timeout, so there's only room for one quick retry
|
||||
# (worst case: 60s timeout + 1s wait + fast cache-hit retry).
|
||||
flacfetch_search_retry_max_attempts: int = int(os.getenv("FLACFETCH_SEARCH_RETRY_MAX_ATTEMPTS", "2"))
|
||||
flacfetch_search_retry_wait: float = float(os.getenv("FLACFETCH_SEARCH_RETRY_WAIT", "1.0")) # seconds
|
||||
# Total budget across all search attempts, kept under Cloudflare's ~100s edge
|
||||
# deadline so callers get a clean FlacfetchServiceError instead of an edge 524.
|
||||
flacfetch_search_total_timeout: float = float(os.getenv("FLACFETCH_SEARCH_TOTAL_TIMEOUT", "95.0")) # seconds
|
||||
|
||||
|
||||
# Default distribution settings (can be overridden per-request)
|
||||
default_dropbox_path: Optional[str] = os.getenv("DEFAULT_DROPBOX_PATH")
|
||||
default_gdrive_folder_id: Optional[str] = os.getenv("DEFAULT_GDRIVE_FOLDER_ID")
|
||||
@@ -330,221 +110,25 @@ class Settings(BaseSettings):
|
||||
default_enable_youtube_upload: bool = os.getenv("DEFAULT_ENABLE_YOUTUBE_UPLOAD", "false").lower() in ("true", "1", "yes")
|
||||
default_brand_prefix: Optional[str] = os.getenv("DEFAULT_BRAND_PREFIX")
|
||||
|
||||
# Requests voting board — Phase 2 daily free-community-track automation.
|
||||
# Master kill-switch: when False the daily picker claims the day and records a
|
||||
# "skipped" lock but never grants credits or creates a job (safe to deploy dark).
|
||||
community_daily_pick_enabled: bool = os.getenv(
|
||||
"COMMUNITY_DAILY_PICK_ENABLED", "false"
|
||||
).lower() in ("true", "1", "yes")
|
||||
# Max distinct voters the 24h ownership handoff will try before parking a track.
|
||||
community_handoff_max_attempts: int = int(os.getenv("COMMUNITY_HANDOFF_MAX_ATTEMPTS", "5"))
|
||||
# Hours an owner has to complete their review before the job is handed off.
|
||||
community_handoff_hours: int = int(os.getenv("COMMUNITY_HANDOFF_HOURS", "24"))
|
||||
# When a flagged pick is "kept on board", how long before it can be re-flagged.
|
||||
community_review_snooze_days: int = int(os.getenv("COMMUNITY_REVIEW_SNOOZE_DAYS", "30"))
|
||||
# Rate Limiting Configuration
|
||||
# Enable/disable rate limiting system-wide (useful for development)
|
||||
enable_rate_limiting: bool = os.getenv("ENABLE_RATE_LIMITING", "true").lower() in ("true", "1", "yes")
|
||||
# Maximum jobs a user can create per day (0 = unlimited)
|
||||
rate_limit_jobs_per_day: int = int(os.getenv("RATE_LIMIT_JOBS_PER_DAY", "5"))
|
||||
# Maximum YouTube uploads system-wide per day (0 = unlimited)
|
||||
rate_limit_youtube_uploads_per_day: int = int(os.getenv("RATE_LIMIT_YOUTUBE_UPLOADS_PER_DAY", "10"))
|
||||
# Maximum beta enrollments from same IP per day (0 = unlimited)
|
||||
rate_limit_beta_ip_per_day: int = int(os.getenv("RATE_LIMIT_BETA_IP_PER_DAY", "1"))
|
||||
|
||||
# KaraokeHunt retired-app request interceptor (POST /api/karaokehunt/request).
|
||||
# Shared secret set by the Cloudflare Worker on create.karaokehunt.com; the
|
||||
# endpoint 503s while this is empty, so deploys are dark until the secret is
|
||||
# wired (Secret Manager: karaokehunt-forwarder-secret).
|
||||
karaokehunt_forwarder_secret: str = os.getenv("KARAOKEHUNT_FORWARDER_SECRET", "")
|
||||
# Max conversion jobs per UTC day (bounds unattended generation spend);
|
||||
# over-cap requests fall through to the free community requests board.
|
||||
karaokehunt_daily_job_cap: int = int(os.getenv("KARAOKEHUNT_DAILY_JOB_CAP", "3"))
|
||||
|
||||
# kjbox partner API (/api/kjbox/*): singer email-code sign-in + show credits
|
||||
# for the karaoke-night "make it" flow. Every endpoint 503s while the secret is
|
||||
# empty, so deploys are dark until it's wired (Secret Manager: kjbox-partner-secret).
|
||||
kjbox_partner_secret: str = os.getenv("KJBOX_PARTNER_SECRET", "")
|
||||
# karaoke-decide's shared secret (header X-Partner-Secret, Secret Manager:
|
||||
# decide-gen-partner-secret): decide sends it to POST /api/kjbox/catalog/resolve, and
|
||||
# gen sends it to decide's DELETE /api/partner/gen/users on account deletion.
|
||||
# Empty = decide is not accepted, and account deletion 503s at the decide step.
|
||||
decide_partner_secret: str = os.getenv("DECIDE_GEN_PARTNER_SECRET", "")
|
||||
# Partner-wide cap on NEW gen accounts created via kjbox per rolling 24h (a venue
|
||||
# shares one IP, so gen's per-IP signup cap is bypassed for kjbox; this bounds it).
|
||||
kjbox_signup_cap_per_24h: int = int(os.getenv("KJBOX_SIGNUP_CAP_PER_24H", "100"))
|
||||
# Max emailed sign-in codes per address per rolling hour.
|
||||
kjbox_codes_per_email_per_hour: int = int(os.getenv("KJBOX_CODES_PER_EMAIL_PER_HOUR", "5"))
|
||||
# Max "free at the show" credits kjbox may grant one user per rolling 24h.
|
||||
kjbox_show_credits_per_user_per_24h: int = int(os.getenv("KJBOX_SHOW_CREDITS_PER_USER_PER_24H", "5"))
|
||||
|
||||
# gen email-code sign-in (/api/users/auth/send-code + verify-code).
|
||||
# HMAC pepper for stored codes; when empty it's derived from the first admin
|
||||
# token (see backend/api/routes/login_code.py:_resolve_pepper).
|
||||
login_code_pepper: str = os.getenv("LOGIN_CODE_PEPPER", "")
|
||||
# Max emailed sign-in codes per address / per client IP per rolling hour.
|
||||
login_codes_per_email_per_hour: int = int(os.getenv("LOGIN_CODES_PER_EMAIL_PER_HOUR", "5"))
|
||||
login_codes_per_ip_per_hour: int = int(os.getenv("LOGIN_CODES_PER_IP_PER_HOUR", "20"))
|
||||
|
||||
# Private (non-published) track distribution settings
|
||||
default_private_dropbox_path: Optional[str] = os.getenv("DEFAULT_PRIVATE_DROPBOX_PATH")
|
||||
default_private_brand_prefix: Optional[str] = os.getenv("DEFAULT_PRIVATE_BRAND_PREFIX", "NOMADNP")
|
||||
|
||||
# Output files NOT uploaded to Nomad's own Dropbox folders (DEFAULT_DROPBOX_PATH /
|
||||
# DEFAULT_PRIVATE_DROPBOX_PATH), as "|"-separated filename suffixes. They stay in
|
||||
# GCS (job-page downloads, YouTube upload, AWS backup) — this only saves the
|
||||
# multi-GB internet egress + upload time. Tenant Dropbox folders are exempt
|
||||
# (B2B tenants expect the full set). Default skips only the lossless 4K MP4,
|
||||
# which no customer-facing text promises in Dropbox. The lossless MKV IS promised
|
||||
# there (completion email + the Fiverr bot), so add
|
||||
# " (Final Karaoke Lossless 4k).mkv" only after updating those.
|
||||
# See docs/archive/2026-09-26-encoding-cost-cuts.md.
|
||||
dropbox_skip_output_suffixes: str = os.getenv(
|
||||
"DROPBOX_SKIP_OUTPUT_SUFFIXES", " (Final Karaoke Lossless 4k).mp4"
|
||||
)
|
||||
|
||||
# YouTube Data API v3 Quota Configuration
|
||||
# Daily quota limit (YouTube grants 10,000 units/day by default)
|
||||
youtube_quota_daily_limit: int = int(os.getenv("YOUTUBE_QUOTA_DAILY_LIMIT", "10000"))
|
||||
# Estimated units consumed per upload (videos.insert=100 + search.list=100 + thumbnails.set=50 + buffer)
|
||||
youtube_quota_upload_cost: int = int(os.getenv("YOUTUBE_QUOTA_UPLOAD_COST", "300"))
|
||||
# Safety margin to reserve (avoid hitting exact limit)
|
||||
youtube_quota_safety_margin: int = int(os.getenv("YOUTUBE_QUOTA_SAFETY_MARGIN", "500"))
|
||||
# E2E test bypass key for rate limiting (set via secret in production)
|
||||
e2e_bypass_key: str = os.getenv("E2E_BYPASS_KEY", "")
|
||||
# Local-stack E2E mode (scripts/local-stack.sh): workers are replaced by in-process
|
||||
# fakes that walk jobs through their real states with canned media, the
|
||||
# /api/test/* router is mounted, and external lookups (Secret Manager, DeBounce,
|
||||
# ip-api, encoding VM) are skipped. Startup refuses it in production.
|
||||
# See backend/services/fake_workers/ and docs/TESTING.md § Local stack.
|
||||
fake_workers: bool = os.getenv("FAKE_WORKERS", "").lower() in ("1", "true", "yes")
|
||||
# Mobile app account deletion (DELETE /api/users/me). Off in prod until
|
||||
# Andrew's manual pass; on in the local stack.
|
||||
account_deletion_enabled: bool = os.getenv("ACCOUNT_DELETION_ENABLED", "").lower() in ("1", "true", "yes")
|
||||
# App Review demo login: inert unless this AND the app-review-demo-code secret are set.
|
||||
app_review_demo_email: str = os.getenv("APP_REVIEW_DEMO_EMAIL", "")
|
||||
# Secret Manager: app-review-demo-code, mounted as APP_REVIEW_DEMO_CODE.
|
||||
app_review_demo_code: str = os.getenv("APP_REVIEW_DEMO_CODE", "")
|
||||
# RevenueCat (mobile in-app credit packs). The webhook (POST
|
||||
# /api/users/webhooks/revenuecat) compares the Authorization header to this
|
||||
# value and 503s while it is empty (Secret Manager: revenuecat-webhook-auth).
|
||||
revenuecat_webhook_auth: str = os.getenv("REVENUECAT_WEBHOOK_AUTH", "")
|
||||
# RevenueCat secret REST API key (Secret Manager: revenuecat-secret-api-key).
|
||||
revenuecat_secret_api_key: str = os.getenv("REVENUECAT_SECRET_API_KEY", "")
|
||||
# RevenueCat project the v2 key belongs to (account deletion's customer delete).
|
||||
revenuecat_project_id: str = os.getenv("REVENUECAT_PROJECT_ID", "9ea947ce")
|
||||
# URL scheme of the Nomad Karaoke mobile app. App checkouts return through
|
||||
# GET /api/users/credits/app-return, which 302s to {scheme}://checkout?status=…
|
||||
mobile_app_scheme: str = os.getenv("MOBILE_APP_SCHEME", "nomadkaraoke")
|
||||
# Accepted `aud` values for Google ID tokens (POST /api/users/auth/google),
|
||||
# comma-separated. OAuth client IDs are public, not secrets. Defaults: the
|
||||
# "Nomad Karaoke Web" client (the audience of tokens from Android Credential
|
||||
# Manager / iOS with serverClientId) and the iOS client (iOS SDK tokens can
|
||||
# carry it). The Android client is device-only and never an audience. See
|
||||
# docs/ARCHITECTURE.md § Mobile app sign-in. Empty → the endpoint returns 503.
|
||||
google_oauth_client_ids: str = os.getenv(
|
||||
"GOOGLE_OAUTH_CLIENT_IDS",
|
||||
"718638054799-fush0smd1e4icq9pt0immp8n59jjlebh.apps.googleusercontent.com,"
|
||||
"718638054799-hre535n83p8i85tsqf6ai759nlubhd2o.apps.googleusercontent.com",
|
||||
)
|
||||
# Accepted `aud` values for Apple ID tokens (POST /api/users/auth/apple):
|
||||
# the app's bundle ID (native iOS), plus any Services ID used for web/Android:
|
||||
# com.nomadkaraoke.sing = Sign in with Apple JS on the kjbox singer page
|
||||
# (sing.nomadkaraoke.com → POST /api/kjbox/auth/apple).
|
||||
apple_signin_audiences: str = os.getenv("APPLE_SIGNIN_AUDIENCES", "com.nomadkaraoke.mobile,com.nomadkaraoke.sing")
|
||||
|
||||
# Sign in with Apple token revocation (account deletion). The .p8 PEM text
|
||||
# and key id come from Secret Manager; unset => revocation is skipped.
|
||||
apple_siwa_key_p8: Optional[str] = os.getenv("APPLE_SIWA_KEY_P8")
|
||||
apple_siwa_key_id: Optional[str] = os.getenv("APPLE_SIWA_KEY_ID")
|
||||
apple_team_id: str = os.getenv("APPLE_TEAM_ID", "55B76HN73D")
|
||||
apple_client_id: str = os.getenv("APPLE_CLIENT_ID", "com.nomadkaraoke.mobile")
|
||||
# Canonical YouTube description template. Placeholders ({title}, {artist},
|
||||
# {artist_hashtag}, {brand_code}) are filled by
|
||||
# backend.services.youtube_description.render_youtube_description, which is the
|
||||
# single source of truth shared with the bulk-rewrite tool
|
||||
# (scripts/youtube-descriptions). Keep those two in sync when editing this.
|
||||
default_youtube_description: str = os.getenv(
|
||||
"DEFAULT_YOUTUBE_DESCRIPTION",
|
||||
'🎤 Karaoke version of "{title}" by {artist}: original audio, lead vocals '
|
||||
"removed, hand-synced lyrics.\n"
|
||||
"▶️ Make a karaoke video of ANY song: "
|
||||
"https://nomadkaraoke.com/r/youtube?utm_source=youtube&utm_medium=description&utm_campaign=make_cta_top\n\n"
|
||||
"⬆️ REQUEST THE NEXT NOMAD RELEASE\n"
|
||||
"Vote for the songs you want us to make next — the most-upvoted requests get "
|
||||
"made into karaoke videos and published free, right here on this channel. "
|
||||
"Nothing to buy, no password needed:\n"
|
||||
"👉 https://requests.nomadkaraoke.com/?utm_source=youtube&utm_medium=description&utm_campaign=requests\n\n"
|
||||
"▶️ MAKE A VIDEO LIKE THIS FOR ANY SONG\n"
|
||||
"Create studio-quality karaoke videos of ANY song, ready in under an hour:\n"
|
||||
"👉 https://nomadkaraoke.com/r/youtube?utm_source=youtube&utm_medium=description&utm_campaign=make_cta\n"
|
||||
"Follow that link for 50% off all credit purchases for your first 30 days.\n\n"
|
||||
"🎶 KARAOKE LOVER LINKS\n"
|
||||
"• Nomad Karaoke Community Discord: https://discord.nomadkaraoke.com\n"
|
||||
"• Decide what to sing at karaoke: https://decide.nomadkaraoke.com\n"
|
||||
"• Search all community karaoke tracks: https://karaokenerds.com\n\n"
|
||||
"🔔 Subscribe for new karaoke tracks, and comment to show some love!\n\n"
|
||||
"#karaoke #{artist_hashtag} #instrumental #lyrics #singalong\n"
|
||||
"—\n"
|
||||
"COPYRIGHT: I don't own the rights to the original music; all rights belong "
|
||||
"to the respective copyright holders. If you enjoy the song, please buy the "
|
||||
"original and support the artist. Under §107 of the U.S. Copyright Act 1976, "
|
||||
'allowance is made for "fair use" (criticism, comment, teaching, and '
|
||||
"research).\n\n"
|
||||
"Brand Code: {brand_code}"
|
||||
"Karaoke video created with Nomad Karaoke (https://nomadkaraoke.com)\n\n"
|
||||
"AI-powered vocal separation and synchronized lyrics.\n\n"
|
||||
"#karaoke #music #singing #instrumental #lyrics"
|
||||
)
|
||||
|
||||
# Offer an opt-in "tip the creator" optional item on gen Stripe Checkout pages
|
||||
# (credits + Made For You). Kill switch: CHECKOUT_TIP_ENABLED=false.
|
||||
checkout_tip_enabled: bool = os.getenv("CHECKOUT_TIP_ENABLED", "true").lower() in ("true", "1", "yes")
|
||||
# Free-to-paid conversion emails (backend/services/conversion_emails.py).
|
||||
# OFF until Andrew approves the copy: CONVERSION_EMAILS_ENABLED=true sends E1
|
||||
# ("your next song" + time-limited discount) and E2 (offer ending) on Postmark's
|
||||
# broadcast stream; REVIEW_LAST_CHANCE_ENABLED=true adds the 42h review nudge.
|
||||
conversion_emails_enabled: bool = os.getenv("CONVERSION_EMAILS_ENABLED", "false").lower() in ("true", "1", "yes")
|
||||
review_last_chance_enabled: bool = os.getenv("REVIEW_LAST_CHANCE_ENABLED", "false").lower() in ("true", "1", "yes")
|
||||
conversion_offer_percent: int = int(os.getenv("CONVERSION_OFFER_PERCENT", "25"))
|
||||
conversion_offer_hours: int = int(os.getenv("CONVERSION_OFFER_HOURS", "48"))
|
||||
conversion_e1_delay_hours: float = float(os.getenv("CONVERSION_E1_DELAY_HOURS", "2"))
|
||||
conversion_e2_hours_before_expiry: float = float(os.getenv("CONVERSION_E2_HOURS_BEFORE_EXPIRY", "8"))
|
||||
# CAN-SPAM: commercial email must carry a physical postal address.
|
||||
# Default = the business address already published on nomadkaraoke.com.
|
||||
commercial_email_postal_address: str = os.getenv(
|
||||
"COMMERCIAL_EMAIL_POSTAL_ADDRESS", "Nomad Karaoke, 612 Joan St, Columbia, SC 29203, USA"
|
||||
)
|
||||
|
||||
# YouTube description back-catalogue rewrite (scheduled daily drain worker).
|
||||
# Rewrites existing channel videos to match `default_youtube_description`.
|
||||
# Reusable: whenever the template changes, the worker re-drains the channel.
|
||||
youtube_backfill_enabled: bool = os.getenv("YOUTUBE_BACKFILL_ENABLED", "true").lower() in ("true", "1", "yes")
|
||||
# Hard ceiling on updates per daily run (each costs ~50 quota units).
|
||||
youtube_backfill_daily_max_updates: int = int(os.getenv("YOUTUBE_BACKFILL_DAILY_MAX_UPDATES", "150"))
|
||||
# Quota units to leave untouched for the live upload pipeline each day.
|
||||
youtube_backfill_quota_reserve: int = int(os.getenv("YOUTUBE_BACKFILL_QUOTA_RESERVE", "3000"))
|
||||
# Refresh tags too (not just description) when rewriting.
|
||||
youtube_backfill_enrich_tags: bool = os.getenv("YOUTUBE_BACKFILL_ENRICH_TAGS", "true").lower() in ("true", "1", "yes")
|
||||
# Where the daily progress + completion emails are sent.
|
||||
youtube_backfill_report_email: str = os.getenv("YOUTUBE_BACKFILL_REPORT_EMAIL", "andrew@nomadkaraoke.com")
|
||||
|
||||
# YouTube comment auto-reply (backend/workers/youtube_comment_bot_worker.py):
|
||||
# replies to song requests in channel comments with a requests-board link.
|
||||
# OFF by default; go-live only after Andrew signs off a local replay
|
||||
# (scripts/youtube-comments/replay.py). Dry runs work while off.
|
||||
youtube_comment_autoreply_enabled: bool = os.getenv("YOUTUBE_COMMENT_AUTOREPLY_ENABLED", "false").lower() in ("true", "1", "yes")
|
||||
youtube_comment_autoreply_max_replies: int = int(os.getenv("YOUTUBE_COMMENT_AUTOREPLY_MAX_REPLIES", "20"))
|
||||
youtube_comment_autoreply_min_confidence: float = float(os.getenv("YOUTUBE_COMMENT_AUTOREPLY_MIN_CONFIDENCE", "0.8"))
|
||||
# ISO date/time: on the FIRST run, comments since this are eligible (a one-time
|
||||
# backfill of requests nobody answered). Empty = only comments after go-live.
|
||||
youtube_comment_autoreply_start: str = os.getenv("YOUTUBE_COMMENT_AUTOREPLY_START", "")
|
||||
|
||||
# Storage retention (backend/services/storage_retention.py): a daily job purges
|
||||
# regenerable files (big finals, rendered videos, stems, ...) from jobs
|
||||
# completed more than `storage_retention_min_age_days` ago. DRY-RUN BY DEFAULT:
|
||||
# it only writes a report until STORAGE_RETENTION_DRY_RUN=false is set after a
|
||||
# human has reviewed one.
|
||||
storage_retention_enabled: bool = os.getenv("STORAGE_RETENTION_ENABLED", "true").lower() in ("true", "1", "yes")
|
||||
storage_retention_dry_run: bool = os.getenv("STORAGE_RETENTION_DRY_RUN", "true").lower() not in ("false", "0", "no")
|
||||
storage_retention_min_age_days: int = int(os.getenv("STORAGE_RETENTION_MIN_AGE_DAYS", "30"))
|
||||
# Real (non-dry) runs purge at most this many jobs per run, resuming from a cursor.
|
||||
storage_retention_max_jobs_per_run: int = int(os.getenv("STORAGE_RETENTION_MAX_JOBS_PER_RUN", "50"))
|
||||
# Comma-separated tenant ids whose jobs are never purged.
|
||||
storage_retention_excluded_tenants: str = os.getenv("STORAGE_RETENTION_EXCLUDED_TENANTS", "")
|
||||
|
||||
# Customer "Regenerate video" limits (per job / per user, rolling 24h).
|
||||
regenerate_max_per_job_per_day: int = int(os.getenv("REGENERATE_MAX_PER_JOB_PER_DAY", "3"))
|
||||
regenerate_max_per_user_per_day: int = int(os.getenv("REGENERATE_MAX_PER_USER_PER_DAY", "10"))
|
||||
|
||||
# Default CDG/TXT generation settings
|
||||
# When True, CDG and TXT packages are generated by default (when a theme is set)
|
||||
# These can be overridden per-request via explicit enable_cdg/enable_txt parameters
|
||||
@@ -591,18 +175,12 @@ class Settings(BaseSettings):
|
||||
self._secret_cache[secret_id] = env_value
|
||||
return env_value
|
||||
|
||||
# Local stack runs offline on emulators: never reach for Secret Manager.
|
||||
if self.fake_workers is True and not is_production():
|
||||
return None
|
||||
|
||||
# Try Secret Manager (production mode)
|
||||
if not self.google_cloud_project:
|
||||
logger.warning(f"Cannot fetch secret {secret_id}: No GCP project configured")
|
||||
return None
|
||||
|
||||
try:
|
||||
from google.cloud import secretmanager
|
||||
|
||||
client = secretmanager.SecretManagerServiceClient()
|
||||
name = f"projects/{self.google_cloud_project}/secrets/{secret_id}/versions/latest"
|
||||
response = client.access_secret_version(request={"name": name})
|
||||
@@ -627,56 +205,3 @@ def get_settings() -> Settings:
|
||||
"""Get the global settings instance."""
|
||||
return settings
|
||||
|
||||
|
||||
def is_production() -> bool:
|
||||
"""True when running in a deployed/production environment.
|
||||
|
||||
Cloud Run automatically sets ``K_SERVICE``, so we treat its presence as
|
||||
production even if ``ENVIRONMENT`` were somehow unset — a production-safe
|
||||
default. Local dev/test have no ``K_SERVICE`` and a non-"production"
|
||||
``ENVIRONMENT``. (Fallback audit 2026-06-09, Theme 7.)
|
||||
"""
|
||||
env = (os.environ.get("ENVIRONMENT") or os.environ.get("ENV") or "").lower()
|
||||
if env == "production":
|
||||
return True
|
||||
return bool(os.environ.get("K_SERVICE"))
|
||||
|
||||
|
||||
# Env vars that MUST be explicitly set in production. A missing one used to
|
||||
# silently fall back to a dev default and produce wrong behaviour with only a
|
||||
# warning (wrong GCP project for secret lookups, wrong GCS bucket, localhost
|
||||
# worker URL). We only require vars that infra confirms are set on every service
|
||||
# (GCS temp/output buckets are intentionally excluded — their defaults are
|
||||
# load-bearing and not set via env). (Fallback audit 2026-06-09, Theme 7.)
|
||||
_REQUIRED_PRODUCTION_ENV_VARS = (
|
||||
"GOOGLE_CLOUD_PROJECT",
|
||||
"GCS_BUCKET_NAME",
|
||||
"CLOUD_RUN_SERVICE_URL",
|
||||
)
|
||||
|
||||
|
||||
def validate_production_config() -> None:
|
||||
"""Fail fast at startup if required production env vars are missing.
|
||||
|
||||
In production each var in ``_REQUIRED_PRODUCTION_ENV_VARS`` must be explicitly
|
||||
set; otherwise the service would silently run with a dev default (wrong GCP
|
||||
project / wrong GCS bucket / localhost worker URL) and appear to work while
|
||||
doing the wrong thing. Raises RuntimeError listing every missing var. No-op
|
||||
outside production. (Fallback audit 2026-06-09, Theme 7.)
|
||||
"""
|
||||
if not is_production():
|
||||
return
|
||||
if settings.fake_workers is True:
|
||||
raise RuntimeError(
|
||||
"FAKE_WORKERS is set in production. It replaces every worker with a fake "
|
||||
"and mounts the unauthenticated /api/test/* router. Refusing to start."
|
||||
)
|
||||
missing = [v for v in _REQUIRED_PRODUCTION_ENV_VARS if not (os.environ.get(v) or "").strip()]
|
||||
if missing:
|
||||
raise RuntimeError(
|
||||
"Production misconfiguration: required environment variable(s) not set: "
|
||||
+ ", ".join(missing)
|
||||
+ ". Refusing to start with silent dev defaults (wrong GCP project / "
|
||||
"GCS bucket / localhost worker URL)."
|
||||
)
|
||||
|
||||
@@ -1,17 +0,0 @@
|
||||
# Video IDs to FORCE-rewrite even though the title isn't high-confidence
|
||||
# "Artist - Title (Karaoke)" — e.g. genuine tracks whose "(Karaoke)" suffix was
|
||||
# truncated by YouTube's 100-char title limit.
|
||||
#
|
||||
# Two forms per line ('#' starts a comment):
|
||||
# <id> use the tool's best-effort parsed artist/title
|
||||
# <id> | <Artist> | <Title> override artist/title (for truncated titles)
|
||||
#
|
||||
# --- Genuine tracks rescued from the "review candidates" bucket (2026-09-03) ---
|
||||
|
||||
# Truncated titles restored to full song titles:
|
||||
5zSFKuCEIhs | Los Campesinos! | This Is How You Spell "HAHAHA, We Destroyed the Hopes and Dreams of a Generation of Faux-Romantics"
|
||||
kkT-bZhi9jI | Arctic Monkeys | You Probably Couldn't See for the Lights but You Were Looking Straight at Me
|
||||
|
||||
# Long medley titles parsed fine — no override needed:
|
||||
le347q8nHT4 # Daniel O'Donnell - Medley: Galway Bay, The Mountains Of Mourne, ...
|
||||
3ihDlPse--k # Daniel O'Donnell - Medley: It's a Long Way to Tipperary / ...
|
||||
@@ -1,29 +0,0 @@
|
||||
# Video IDs that should NEVER have their description rewritten by the bulk tool.
|
||||
# One YouTube video ID per line. Everything after a '#' is ignored.
|
||||
#
|
||||
# The "unparseable" videos are already auto-skipped by the classifier; these are
|
||||
# listed explicitly for the record + belt-and-suspenders (so they stay skipped
|
||||
# even if a title changes later).
|
||||
|
||||
# --- Lyrics videos WITH vocals (NOT instrumental karaoke) ---
|
||||
SUkl49w9GPA # buildspace, MAYV - the end (Lyrics Video)
|
||||
jAAhbfmh8Bc # 100 gecs - mememe (Lyrics Video With Vocals)
|
||||
mcZtK8kkx3E # Fleet Foxes - Montezuma (Lyrics / With Vocals)
|
||||
|
||||
# --- Demos / tutorials / screencasts / pitch / channel-explainer (not karaoke) ---
|
||||
VTR3oaqgn0A # Nomad karaoke-gen hosted demo screencast 2026-05-16
|
||||
-dI3r7qXo3A # Nomad Karaoke Generator - Full Tutorial
|
||||
C0Zyy_9V5F8 # Purim Shpiel custom lyrics karaoke tracks production screencast
|
||||
josVDsYwsFM # Nomad Karaoke Lyrics Review Demo (for AudioShake) Screencast
|
||||
NZ8j0uxq6tA # YouTube "Reused content" claim appeal - channel explanation
|
||||
asnwjLyqY4k # Nomad Karaoke production workflow (real time) 2025-02-26
|
||||
dPWyYfjfHvw # Pitch Night - BIC Energy
|
||||
63-Fk3mfZ7Q # Nomad Karaoke Creation Workflow (Normal Speed) 2023-12-19
|
||||
ZsROHgqAVHs # Nomad Karaoke Creation Workflow (MidiCo Tutorial) 2023-12-19
|
||||
|
||||
# --- Unparseable-title non-karaoke (already auto-skipped; explicit for record) ---
|
||||
RLkIEHGwFcU # Nomad Karaoke Generator Quick Demo
|
||||
GJY60IfIHOQ # Nomad Karaoke Generator Demo 2026
|
||||
v1Hox6XMYEw # "YouTube Karaoke just suffered a major loss" (short/commentary)
|
||||
R3eOl1aeq5c # Nomad Karaoke stands with Lemmy Caution (commentary)
|
||||
wHiMti3xLJE # Nomad Karaoke Creation Tutorial / Demo 1 (MidiCo, July 2023)
|
||||
Whitespace-only changes.
Whitespace-only changes.
@@ -1,81 +0,0 @@
|
||||
"""One-shot script to populate the year-5-stars fixture from a real production job.
|
||||
|
||||
Run once:
|
||||
poetry run python -m backend.eval.custom_lyrics._bootstrap_year5
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from google.cloud import firestore
|
||||
|
||||
|
||||
JOB_ID = "2cb49a45"
|
||||
FIXTURE_DIR = Path(__file__).parent / "fixtures" / "year-5-stars-shake-it-off"
|
||||
CLIENT_INPUT_SRC = Path("/Users/andrew/Projects/nomadkaraoke/year-5-stars-client-custom-lyrics.txt")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
FIXTURE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
fs = firestore.Client(project="nomadkaraoke")
|
||||
job_doc = fs.collection("jobs").document(JOB_ID).get()
|
||||
if not job_doc.exists:
|
||||
raise SystemExit(f"Job {JOB_ID} not found")
|
||||
job = job_doc.to_dict()
|
||||
|
||||
# Corrected lyrics are stored inline in state_data (not in GCS for this job)
|
||||
corrected = (
|
||||
job.get("state_data", {}).get("corrected_lyrics")
|
||||
or job.get("corrected_lyrics")
|
||||
or {}
|
||||
)
|
||||
if not corrected:
|
||||
# Fallback: try GCS path if present
|
||||
corrected_path = job.get("corrected_json_gcs_path") or job.get("lyrics", {}).get("corrected_path")
|
||||
if corrected_path:
|
||||
from google.cloud import storage # noqa: PLC0415
|
||||
storage_client = storage.Client(project="nomadkaraoke")
|
||||
bucket_name, *blob_parts = corrected_path.replace("gs://", "").split("/", 1)
|
||||
blob = storage_client.bucket(bucket_name).blob(blob_parts[0])
|
||||
corrected = json.loads(blob.download_as_bytes())
|
||||
else:
|
||||
raise SystemExit(f"Job {JOB_ID} has no corrected lyrics (checked state_data and GCS path fields)")
|
||||
|
||||
segments = corrected.get("corrected_segments") or corrected.get("segments") or []
|
||||
if not segments:
|
||||
raise SystemExit("No segments found in corrected.json")
|
||||
|
||||
(FIXTURE_DIR / "original_segments.json").write_text(json.dumps(segments, indent=2))
|
||||
(FIXTURE_DIR / "original_lyrics.txt").write_text(
|
||||
"\n".join(seg.get("text", "").strip() for seg in segments) + "\n"
|
||||
)
|
||||
|
||||
if CLIENT_INPUT_SRC.exists():
|
||||
shutil.copy(CLIENT_INPUT_SRC, FIXTURE_DIR / "client_input.txt")
|
||||
else:
|
||||
print(f"WARNING: {CLIENT_INPUT_SRC} not found; populate client_input.txt manually")
|
||||
|
||||
metadata = {
|
||||
"id": "year-5-stars-shake-it-off",
|
||||
"artist": job.get("artist") or "Taylor Swift",
|
||||
"title": job.get("title") or "Shake It Off",
|
||||
"source_job_id": JOB_ID,
|
||||
"difficulty": "hard",
|
||||
"input_style": "long substantive lines, name-heavy, mismatched syllable budget",
|
||||
"settings_to_test": [
|
||||
{"name": "default", "allow_reword": True, "allow_omit": True, "fixed_line_count": True, "strictness": "balanced"},
|
||||
{"name": "verbatim", "allow_reword": True, "allow_omit": True, "fixed_line_count": True, "strictness": "verbatim"},
|
||||
{"name": "strict", "allow_reword": True, "allow_omit": True, "fixed_line_count": True, "strictness": "strict"},
|
||||
],
|
||||
}
|
||||
(FIXTURE_DIR / "metadata.json").write_text(json.dumps(metadata, indent=2))
|
||||
|
||||
print(f"Fixture populated at {FIXTURE_DIR}")
|
||||
print(f" - {len(segments)} segments")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Disk-backed cache for LLM responses, keyed by (system, user, model, settings)."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
class LlmCallCache:
|
||||
def __init__(self, cache_dir: Path, replay_only: bool = False) -> None:
|
||||
self.cache_dir = cache_dir
|
||||
self.cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.replay_only = replay_only
|
||||
|
||||
def _key(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
*,
|
||||
model: str,
|
||||
settings_dict: dict[str, Any],
|
||||
) -> str:
|
||||
h = hashlib.sha256()
|
||||
h.update(system_prompt.encode())
|
||||
h.update(b"\x00")
|
||||
h.update(user_prompt.encode())
|
||||
h.update(b"\x00")
|
||||
h.update(model.encode())
|
||||
h.update(b"\x00")
|
||||
h.update(json.dumps(settings_dict, sort_keys=True).encode())
|
||||
return h.hexdigest()
|
||||
|
||||
def _path(self, key: str) -> Path:
|
||||
return self.cache_dir / f"{key}.json"
|
||||
|
||||
def get(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
*,
|
||||
model: str,
|
||||
settings_dict: dict[str, Any],
|
||||
) -> Optional[list[str]]:
|
||||
path = self._path(self._key(system_prompt, user_prompt, model=model, settings_dict=settings_dict))
|
||||
if not path.exists():
|
||||
if self.replay_only:
|
||||
raise RuntimeError(f"cache miss in replay-only mode: {path.name}")
|
||||
return None
|
||||
return json.loads(path.read_text())["lines"]
|
||||
|
||||
def set(
|
||||
self,
|
||||
system_prompt: str,
|
||||
user_prompt: str,
|
||||
*,
|
||||
model: str,
|
||||
settings_dict: dict[str, Any],
|
||||
response_lines: list[str],
|
||||
) -> None:
|
||||
if self.replay_only:
|
||||
return
|
||||
path = self._path(self._key(system_prompt, user_prompt, model=model, settings_dict=settings_dict))
|
||||
path.write_text(json.dumps({"lines": response_lines}, indent=2))
|
||||
@@ -1,2 +0,0 @@
|
||||
*
|
||||
!.gitignore
|
||||
@@ -1 +0,0 @@
|
||||
We're singing this for Alex's birthday! Can you change the lyrics so it says Alex's name instead of "he"? Like "For she's a jolly good fellow" or however it makes sense with Alex. Just make it clear it's for Alex. It's a birthday celebration for our friend Alex.
|
||||
@@ -1,11 +0,0 @@
|
||||
{
|
||||
"id": "jolly-good-fellow-name-replace",
|
||||
"artist": "Traditional",
|
||||
"title": "For He's a Jolly Good Fellow",
|
||||
"difficulty": "easy",
|
||||
"input_style": "explicit name replacement instruction",
|
||||
"settings_to_test": [
|
||||
{"name": "default", "allow_reword": true, "allow_omit": true, "fixed_line_count": true, "strictness": "balanced"},
|
||||
{"name": "strict", "allow_reword": true, "allow_omit": true, "fixed_line_count": true, "strictness": "strict"}
|
||||
]
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
For he's a jolly good fellow
|
||||
For he's a jolly good fellow
|
||||
For he's a jolly good fellow
|
||||
Which nobody can deny
|
||||
-30
@@ -1,30 +0,0 @@
|
||||
[
|
||||
{
|
||||
"id": "seg-00",
|
||||
"start_time": 0.0,
|
||||
"end_time": 2.5,
|
||||
"text": "For he's a jolly good fellow",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-01",
|
||||
"start_time": 2.5,
|
||||
"end_time": 5.0,
|
||||
"text": "For he's a jolly good fellow",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-02",
|
||||
"start_time": 5.0,
|
||||
"end_time": 7.5,
|
||||
"text": "For he's a jolly good fellow",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-03",
|
||||
"start_time": 7.5,
|
||||
"end_time": 10.0,
|
||||
"text": "Which nobody can deny",
|
||||
"words": []
|
||||
}
|
||||
]
|
||||
@@ -1 +0,0 @@
|
||||
We need this song to be about Mike and Sarah's wedding. Mike is a software engineer who loves climbing mountains, hiking on Saturdays with his border collie Max, and brewing his own beer in the basement. Sarah is a pediatric nurse with three sisters — Jenny, Tara, and Rachel — and she runs marathons every fall in the city park near her apartment. They met five years ago at a friend's birthday party in Brooklyn and bought their first house together last summer in Queens. Make the song celebrate their journey together. Highlight Max the dog if you can, and mention the marathons and the brewing.
|
||||
@@ -1,11 +0,0 @@
|
||||
{
|
||||
"id": "row-row-line-surplus",
|
||||
"artist": "Traditional",
|
||||
"title": "Row, Row, Row Your Boat",
|
||||
"difficulty": "medium",
|
||||
"input_style": "client input far exceeds song's structural capacity (line surplus)",
|
||||
"settings_to_test": [
|
||||
{"name": "default", "allow_reword": true, "allow_omit": true, "fixed_line_count": true, "strictness": "balanced"},
|
||||
{"name": "strict", "allow_reword": true, "allow_omit": true, "fixed_line_count": true, "strictness": "strict"}
|
||||
]
|
||||
}
|
||||
@@ -1,4 +0,0 @@
|
||||
Row, row, row your boat,
|
||||
Gently down the stream,
|
||||
Merrily, merrily, merrily, merrily,
|
||||
Life is but a dream.
|
||||
@@ -1,30 +0,0 @@
|
||||
[
|
||||
{
|
||||
"text": "Row, row, row your boat,",
|
||||
"start_time": 0.0,
|
||||
"end_time": 1.5,
|
||||
"id": "rr-0",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"text": "Gently down the stream,",
|
||||
"start_time": 1.5,
|
||||
"end_time": 3.0,
|
||||
"id": "rr-1",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"text": "Merrily, merrily, merrily, merrily,",
|
||||
"start_time": 3.0,
|
||||
"end_time": 4.5,
|
||||
"id": "rr-2",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"text": "Life is but a dream.",
|
||||
"start_time": 4.5,
|
||||
"end_time": 6.0,
|
||||
"id": "rr-3",
|
||||
"words": []
|
||||
}
|
||||
]
|
||||
@@ -1 +0,0 @@
|
||||
Hi! Can you change the word "star" to "Sam" throughout the song? Every time "star" appears, I want it to say "Sam" instead. It's for my son Sam's birthday party. Thanks!
|
||||
@@ -1,11 +0,0 @@
|
||||
{
|
||||
"id": "twinkle-name-swap",
|
||||
"artist": "Traditional",
|
||||
"title": "Twinkle Twinkle Little Star",
|
||||
"difficulty": "easy",
|
||||
"input_style": "simple word substitution, well-defined target",
|
||||
"settings_to_test": [
|
||||
{"name": "default", "allow_reword": true, "allow_omit": true, "fixed_line_count": true, "strictness": "balanced"},
|
||||
{"name": "strict", "allow_reword": true, "allow_omit": true, "fixed_line_count": true, "strictness": "strict"}
|
||||
]
|
||||
}
|
||||
@@ -1,6 +0,0 @@
|
||||
Twinkle, twinkle, little star
|
||||
How I wonder what you are
|
||||
Up above the world so high
|
||||
Like a diamond in the sky
|
||||
Twinkle, twinkle, little star
|
||||
How I wonder what you are
|
||||
@@ -1,44 +0,0 @@
|
||||
[
|
||||
{
|
||||
"id": "seg-00",
|
||||
"start_time": 0.0,
|
||||
"end_time": 2.0,
|
||||
"text": "Twinkle, twinkle, little star",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-01",
|
||||
"start_time": 2.0,
|
||||
"end_time": 4.0,
|
||||
"text": "How I wonder what you are",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-02",
|
||||
"start_time": 4.0,
|
||||
"end_time": 6.0,
|
||||
"text": "Up above the world so high",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-03",
|
||||
"start_time": 6.0,
|
||||
"end_time": 8.0,
|
||||
"text": "Like a diamond in the sky",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-04",
|
||||
"start_time": 8.0,
|
||||
"end_time": 10.0,
|
||||
"text": "Twinkle, twinkle, little star",
|
||||
"words": []
|
||||
},
|
||||
{
|
||||
"id": "seg-05",
|
||||
"start_time": 10.0,
|
||||
"end_time": 12.0,
|
||||
"text": "How I wonder what you are",
|
||||
"words": []
|
||||
}
|
||||
]
|
||||
@@ -1,68 +0,0 @@
|
||||
We walk in the room, yeah we own this place
|
||||
Anna hits the floor, watch her lead the way
|
||||
Grace brings the rhythm, dancing every day
|
||||
Oona on guitar, getting in the groove
|
||||
Savannah steps forward, watch her make her move
|
||||
Eirini singing, Frankie right there
|
||||
Voices together, filling the air
|
||||
Step aside now, nobody stands a chance
|
||||
|
||||
|
||||
Mr Moulton jokes, we try not to laugh
|
||||
But we’re ready now, we’ve done the graft
|
||||
|
||||
|
||||
We’re gonna win, win, win, hear the crowd go loud
|
||||
Singing all together, yeah we’re making you proud
|
||||
Clap your hands, feel the beat, come and take a glance
|
||||
We’re Year 5 strong, nobody stands a chance
|
||||
Oh oh, here we go
|
||||
Oh oh, steal the show
|
||||
Watch us now, take our stance
|
||||
Nobody stands a chance
|
||||
|
||||
|
||||
Mr Bunney listening, nodding to the beat
|
||||
Mr Woodward watching, tapping his feet
|
||||
We’ll impress him with every note we play
|
||||
Best performance of the whole school day
|
||||
Mrs Woodbridge smiling, watching from the side
|
||||
Thinking “Year 6 will be full of pride”
|
||||
We’re stepping up, this is our dance
|
||||
Better step back, nobody stands a chance
|
||||
|
||||
|
||||
If we feel nervous, we just smile
|
||||
We’ve been ready for a while
|
||||
|
||||
|
||||
We’re gonna win, win, win, hear the crowd go loud
|
||||
Singing all together, yeah we’re making you proud
|
||||
Clap your hands, feel the beat, come and take a glance
|
||||
We’re Year 5 strong, nobody stands a chance
|
||||
Oh oh, here we go
|
||||
Oh oh, steal the show
|
||||
Watch us now, take our stance
|
||||
Nobody stands a chance
|
||||
|
||||
|
||||
Savannah solo
|
||||
I step to the front, now it’s my time
|
||||
Hear my voice, watch me shine
|
||||
All eyes on me, feel that vibe
|
||||
This is my moment, I come alive
|
||||
|
||||
Strum it strong, feel the sound
|
||||
Let the music spin around
|
||||
Keep the rhythm, don’t slow down
|
||||
Now we bring it back around
|
||||
|
||||
|
||||
We’re gonna win, win, win, loud and clear
|
||||
Best group performance of the year
|
||||
Sing it out, give it all, take a chance
|
||||
We’re Year 5 stars, nobody stands a chance
|
||||
Oh oh, this is it
|
||||
Oh oh, we won’t quit
|
||||
Watch us now, take our stance
|
||||
Nobody stands a chance
|
||||
@@ -1,31 +0,0 @@
|
||||
{
|
||||
"id": "year-5-stars-shake-it-off",
|
||||
"artist": "Taylor Swift",
|
||||
"title": "Year 5 Stars",
|
||||
"source_job_id": "2cb49a45",
|
||||
"difficulty": "hard",
|
||||
"input_style": "long substantive lines, name-heavy, mismatched syllable budget",
|
||||
"settings_to_test": [
|
||||
{
|
||||
"name": "default",
|
||||
"allow_reword": true,
|
||||
"allow_omit": true,
|
||||
"fixed_line_count": true,
|
||||
"strictness": "balanced"
|
||||
},
|
||||
{
|
||||
"name": "verbatim",
|
||||
"allow_reword": true,
|
||||
"allow_omit": true,
|
||||
"fixed_line_count": true,
|
||||
"strictness": "verbatim"
|
||||
},
|
||||
{
|
||||
"name": "strict",
|
||||
"allow_reword": true,
|
||||
"allow_omit": true,
|
||||
"fixed_line_count": true,
|
||||
"strictness": "strict"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,70 +0,0 @@
|
||||
We walk in the room
|
||||
Yeah we own this place
|
||||
Anna hits the floor, mm- mm
|
||||
Watch her lead the way, mm- mm
|
||||
Grace brings the rhythm
|
||||
Dancing every day
|
||||
Oona on guitar, mm- mm
|
||||
Getting in the groove, mm- mm
|
||||
Eirini singing, Frankie right there
|
||||
Voices together, filling the air
|
||||
Nobody stands a chance
|
||||
'Cause the players gonna play, play, play, play, play
|
||||
And the haters gonna hate, hate, hate, hate, hate
|
||||
Baby, I'm just gonna shake, shake, shake, shake, shake
|
||||
I shake it off, I shake it off
|
||||
Heartbreakers gonna break, break, break, break, break
|
||||
And the fakers gonna fake, fake, fake, fake, fake
|
||||
Baby, I'm just gonna shake, shake, shake, shake, shake
|
||||
I shake it off, I shake it off
|
||||
Mr Moulton jokes
|
||||
We try not to laugh
|
||||
But we’re ready now, mm- mm
|
||||
We’ve done the graft, mm- mm
|
||||
We’re gonna win (hear the crowd go loud)
|
||||
Singing all together (yeah we’re making you proud)
|
||||
Clap your hands, feel the beat, mm- mm
|
||||
Come and take a glance, mm- mm
|
||||
We’re Year 5 strong
|
||||
Here we go, steal the show
|
||||
Watch us now, take our stance
|
||||
Nobody stands a chance
|
||||
'Cause the players gonna play, play, play, play, play
|
||||
And the haters gonna hate, hate, hate, hate, hate
|
||||
Baby, I'm just gonna shake, shake, shake, shake, shake
|
||||
I shake it off, I shake it off
|
||||
Heartbreakers gonna break, break, break, break, break
|
||||
And the fakers gonna fake, fake, fake, fake, fake
|
||||
Baby, I'm just gonna shake, shake, shake, shake, shake
|
||||
I shake it off, I shake it off
|
||||
Shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
Hey, hey, hey
|
||||
Just think, while you've been gettin' down and out about the liars
|
||||
And the dirty, dirty cheats of the world
|
||||
You could've been gettin' down to this sick beat
|
||||
My ex- man brought his new girlfriend
|
||||
She's like, "Oh my God"
|
||||
But I'm just gonna shake
|
||||
And to the fella over there with the hella good hair
|
||||
Won't you come on over, baby? We can shake, shake, shake
|
||||
Yeah, oh
|
||||
'Cause the players gonna play, play, play, play, play
|
||||
And the haters gonna hate, hate, hate, hate, hate
|
||||
I'm just gonna shake, shake, shake, shake, shake
|
||||
I shake it off, I shake it off
|
||||
Heartbreakers gonna break, break, break, break, break
|
||||
And the fakers gonna fake, fake, fake, fake
|
||||
I'm just gonna shake, shake, shake, shake
|
||||
I shake it off, I shake it off
|
||||
Shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
Shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
(You got to)
|
||||
I- I- I shake it off, I shake it off
|
||||
I- I- I shake it off, I shake it off
|
||||
-4153
File diff suppressed because it is too large.
Load diff
@@ -1,109 +0,0 @@
|
||||
"""Markdown report generation for eval runs."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from backend.eval.custom_lyrics.runner import FixtureRunResult
|
||||
from backend.eval.custom_lyrics.scorer import aggregate_corpus
|
||||
|
||||
|
||||
def write_run_reports(
|
||||
results: list[FixtureRunResult],
|
||||
out_dir: Path,
|
||||
baseline: Optional[dict] = None,
|
||||
) -> None:
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
(out_dir / "per_fixture").mkdir(exist_ok=True)
|
||||
|
||||
summary_lines = ["# Custom Lyrics Eval Run", ""]
|
||||
summary_lines.append(f"_Generated: {datetime.now(timezone.utc).isoformat()}_")
|
||||
summary_lines.append("")
|
||||
summary_lines.append("## Per-fixture summary")
|
||||
summary_lines.append("")
|
||||
summary_lines.append("| Fixture | Settings | pass@±2 | Δ | mean_Δ | iters | stop |")
|
||||
summary_lines.append("|---------|----------|---------|---|--------|-------|------|")
|
||||
for r in results:
|
||||
s = r.score
|
||||
delta_str = ""
|
||||
if baseline:
|
||||
prev = baseline.get(r.fixture_id, {}).get(r.settings_name, {})
|
||||
if "pct_pass_at_2" in prev:
|
||||
d = s.pct_pass_at_2 - prev["pct_pass_at_2"]
|
||||
delta_str = f"{d:+.2f}"
|
||||
summary_lines.append(
|
||||
f"| {r.fixture_id} | {r.settings_name} | {s.pct_pass_at_2:.2f} | "
|
||||
f"{delta_str} | {s.mean_delta:.2f} | {s.iterations_used} | {s.stop_reason} |"
|
||||
)
|
||||
|
||||
summary_lines.append("")
|
||||
corpus = aggregate_corpus([r.score for r in results])
|
||||
summary_lines.append("## Corpus aggregate")
|
||||
summary_lines.append("")
|
||||
summary_lines.append(f"- Macro pass@±2: {corpus.get('macro_pct_pass_at_2', 0):.2f}")
|
||||
summary_lines.append(f"- Macro pass@±1: {corpus.get('macro_pct_pass_at_1', 0):.2f}")
|
||||
summary_lines.append(f"- Macro pass@±0: {corpus.get('macro_pct_pass_at_0', 0):.2f}")
|
||||
summary_lines.append(f"- Total Gemini calls: {corpus.get('total_gemini_calls', 0)}")
|
||||
|
||||
(out_dir / "summary.md").write_text("\n".join(summary_lines))
|
||||
|
||||
for r in results:
|
||||
per_dir = out_dir / "per_fixture" / f"{r.fixture_id}__{r.settings_name}"
|
||||
per_dir.mkdir(parents=True, exist_ok=True)
|
||||
(per_dir / "output.json").write_text(json.dumps({
|
||||
"fixture_id": r.fixture_id,
|
||||
"settings_name": r.settings_name,
|
||||
"score": r.score.__dict__,
|
||||
"lines": r.output_lines,
|
||||
"line_metadata": r.line_metadata,
|
||||
}, indent=2))
|
||||
_write_per_fixture_md(per_dir, r)
|
||||
|
||||
|
||||
def _write_per_fixture_md(out_dir: Path, r: FixtureRunResult) -> None:
|
||||
lines = [
|
||||
f"# {r.fixture_id} — {r.settings_name}",
|
||||
"",
|
||||
f"Stop reason: **{r.score.stop_reason}** after {r.score.iterations_used} iterations.",
|
||||
f"Pass@±2: **{r.score.pct_pass_at_2:.0%}** ({sum(1 for v in r.line_metadata if v['min_delta']<=2)}/{r.score.line_count})",
|
||||
"",
|
||||
"| # | Original target | Candidate | Δ | Severity |",
|
||||
"|---|------------------|-----------|---|----------|",
|
||||
]
|
||||
for v in r.line_metadata:
|
||||
lines.append(
|
||||
f"| {v['line_index']+1} | {_escape(v['target_text'])} | "
|
||||
f"{_escape(v['candidate_text'])} | {v['min_delta']} | {v['severity']} |"
|
||||
)
|
||||
lines += [
|
||||
"",
|
||||
"## Your rating",
|
||||
"",
|
||||
"Add your qualitative notes here:",
|
||||
"- [ ] Singable end-to-end",
|
||||
"- [ ] Names placed naturally",
|
||||
"- [ ] Stress patterns reasonable",
|
||||
"- Notes:",
|
||||
"",
|
||||
]
|
||||
(out_dir / "output.md").write_text("\n".join(lines))
|
||||
|
||||
|
||||
def _escape(s: str) -> str:
|
||||
return s.replace("|", "\\|").replace("\n", " ")
|
||||
|
||||
|
||||
def write_baseline(results: list[FixtureRunResult], path: Path) -> None:
|
||||
out: dict = {}
|
||||
for r in results:
|
||||
bucket = out.setdefault(r.fixture_id, {})
|
||||
bucket[r.settings_name] = {
|
||||
"pct_pass_at_2": r.score.pct_pass_at_2,
|
||||
"pct_pass_at_0": r.score.pct_pass_at_0,
|
||||
"mean_delta": r.score.mean_delta,
|
||||
"iterations_used": r.score.iterations_used,
|
||||
"captured_at": datetime.now(timezone.utc).date().isoformat(),
|
||||
}
|
||||
path.write_text(json.dumps(out, indent=2))
|
||||
@@ -1,2 +0,0 @@
|
||||
*
|
||||
!.gitignore
|
||||
@@ -1,86 +0,0 @@
|
||||
"""CLI: python -m backend.eval.custom_lyrics.run"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from backend.eval.custom_lyrics.cache import LlmCallCache
|
||||
from backend.eval.custom_lyrics.report import write_baseline, write_run_reports
|
||||
from backend.eval.custom_lyrics.runner import run_fixture
|
||||
from backend.services.custom_lyrics.service import CustomLyricsService
|
||||
|
||||
EVAL_ROOT = Path(__file__).parent
|
||||
FIXTURES_DIR = EVAL_ROOT / "fixtures"
|
||||
RESULTS_DIR = EVAL_ROOT / "results"
|
||||
CACHE_DIR = EVAL_ROOT / "cache"
|
||||
BASELINE_PATH = EVAL_ROOT / "baseline.json"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--fixtures", default="all", help="fixture id, or comma-separated, or 'all'")
|
||||
parser.add_argument("--settings", default=None, help="settings name from metadata.settings_to_test")
|
||||
parser.add_argument("--no-cache", action="store_true")
|
||||
parser.add_argument("--replay-only", action="store_true")
|
||||
parser.add_argument("--save-as-baseline", default=None,
|
||||
help="path to write baseline JSON (e.g., baseline-post.json)")
|
||||
args = parser.parse_args()
|
||||
|
||||
cache = LlmCallCache(
|
||||
CACHE_DIR,
|
||||
replay_only=args.replay_only,
|
||||
) if not args.no_cache else _NullCache()
|
||||
|
||||
fixture_dirs = _select_fixtures(args.fixtures)
|
||||
service = CustomLyricsService()
|
||||
|
||||
results = []
|
||||
for fdir in fixture_dirs:
|
||||
meta = json.loads((fdir / "metadata.json").read_text())
|
||||
candidate_settings = meta.get("settings_to_test", [{
|
||||
"name": "default",
|
||||
"allow_reword": True,
|
||||
"allow_omit": True,
|
||||
"fixed_line_count": True,
|
||||
"strictness": "balanced",
|
||||
}])
|
||||
for s in candidate_settings:
|
||||
if args.settings and s["name"] != args.settings:
|
||||
continue
|
||||
settings_dict = {k: v for k, v in s.items() if k != "name"}
|
||||
print(f"Running {fdir.name} / {s['name']}...", file=sys.stderr)
|
||||
results.append(run_fixture(
|
||||
fdir, s["name"], settings_dict,
|
||||
cache=cache, service=service,
|
||||
))
|
||||
|
||||
ts = datetime.now(timezone.utc).strftime("%Y-%m-%d-%H%M")
|
||||
out_dir = RESULTS_DIR / ts
|
||||
baseline = json.loads(BASELINE_PATH.read_text()) if BASELINE_PATH.exists() else None
|
||||
write_run_reports(results, out_dir, baseline=baseline)
|
||||
|
||||
if args.save_as_baseline:
|
||||
write_baseline(results, EVAL_ROOT / args.save_as_baseline)
|
||||
print(f"Baseline saved to {args.save_as_baseline}", file=sys.stderr)
|
||||
|
||||
print(f"\nReport: {out_dir / 'summary.md'}", file=sys.stderr)
|
||||
return 0
|
||||
|
||||
|
||||
def _select_fixtures(spec: str) -> list[Path]:
|
||||
if spec == "all":
|
||||
return sorted(p for p in FIXTURES_DIR.iterdir() if p.is_dir() and (p / "metadata.json").exists())
|
||||
ids = [s.strip() for s in spec.split(",")]
|
||||
return [FIXTURES_DIR / i for i in ids]
|
||||
|
||||
|
||||
class _NullCache:
|
||||
def get(self, *args, **kwargs): return None
|
||||
def set(self, *args, **kwargs): pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,112 +0,0 @@
|
||||
"""Eval runner: load fixture → call service (cached) → score → return result."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
from backend.eval.custom_lyrics.cache import LlmCallCache
|
||||
from backend.eval.custom_lyrics.scorer import FixtureScore, aggregate_fixture
|
||||
from backend.services.custom_lyrics.service import CustomLyricsService
|
||||
from backend.services.custom_lyrics.settings import (
|
||||
GenerationSettings,
|
||||
settings_from_dict,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class FixtureRunResult:
|
||||
fixture_id: str
|
||||
settings_name: str
|
||||
score: FixtureScore
|
||||
output_lines: list[str]
|
||||
line_metadata: list[dict[str, Any]]
|
||||
fixture_dir: Path
|
||||
|
||||
|
||||
def load_fixture(fixture_dir: Path) -> dict:
|
||||
metadata = json.loads((fixture_dir / "metadata.json").read_text())
|
||||
metadata["original_lyrics"] = (fixture_dir / "original_lyrics.txt").read_text().strip().splitlines()
|
||||
metadata["original_segments"] = json.loads((fixture_dir / "original_segments.json").read_text())
|
||||
metadata["client_input"] = (fixture_dir / "client_input.txt").read_text()
|
||||
notes_path = fixture_dir / "notes.txt"
|
||||
metadata["notes"] = notes_path.read_text() if notes_path.exists() else None
|
||||
return metadata
|
||||
|
||||
|
||||
def run_fixture(
|
||||
fixture_dir: Path,
|
||||
settings_name: str,
|
||||
settings_dict: dict[str, Any],
|
||||
*,
|
||||
cache: LlmCallCache,
|
||||
service: CustomLyricsService,
|
||||
) -> FixtureRunResult:
|
||||
fixture = load_fixture(fixture_dir)
|
||||
settings = settings_from_dict(settings_dict)
|
||||
gemini_call_count = 0
|
||||
|
||||
real_call = service._call_gemini
|
||||
|
||||
def cached_call(*, system_prompt: str, user_prompt: str, pdf_bytes, settings: GenerationSettings):
|
||||
nonlocal gemini_call_count
|
||||
cached = cache.get(
|
||||
system_prompt, user_prompt,
|
||||
model=service.settings.custom_lyrics_model,
|
||||
settings_dict=settings.to_dict(),
|
||||
)
|
||||
if cached is not None:
|
||||
return cached
|
||||
gemini_call_count += 1
|
||||
result = real_call(
|
||||
system_prompt=system_prompt, user_prompt=user_prompt,
|
||||
pdf_bytes=pdf_bytes, settings=settings,
|
||||
)
|
||||
cache.set(
|
||||
system_prompt, user_prompt,
|
||||
model=service.settings.custom_lyrics_model,
|
||||
settings_dict=settings.to_dict(),
|
||||
response_lines=result,
|
||||
)
|
||||
return result
|
||||
|
||||
with patch.object(service, "_call_gemini", side_effect=cached_call):
|
||||
result = service.generate(
|
||||
job_id=fixture["id"],
|
||||
target_lines=fixture["original_lyrics"],
|
||||
target_segments=fixture["original_segments"],
|
||||
artist=fixture.get("artist"),
|
||||
title=fixture.get("title"),
|
||||
custom_text=fixture["client_input"],
|
||||
file_bytes=None, file_mime=None, file_name=None,
|
||||
notes=fixture.get("notes"),
|
||||
settings=settings,
|
||||
)
|
||||
|
||||
score = aggregate_fixture(
|
||||
fixture_id=fixture["id"],
|
||||
settings_name=settings_name,
|
||||
metadata=result.line_metadata,
|
||||
iterations_used=result.iterations_used,
|
||||
stop_reason=result.stop_reason.value,
|
||||
duration_ms=result.duration_ms,
|
||||
gemini_calls=gemini_call_count,
|
||||
line_count_match=not result.line_count_mismatch,
|
||||
)
|
||||
|
||||
return FixtureRunResult(
|
||||
fixture_id=fixture["id"],
|
||||
settings_name=settings_name,
|
||||
score=score,
|
||||
output_lines=result.lines,
|
||||
line_metadata=[{
|
||||
"line_index": v.line_index,
|
||||
"target_text": v.target_text,
|
||||
"candidate_text": v.candidate_text,
|
||||
"min_delta": v.min_delta,
|
||||
"severity": v.severity.value,
|
||||
} for v in result.line_metadata],
|
||||
fixture_dir=fixture_dir,
|
||||
)
|
||||
@@ -1,113 +0,0 @@
|
||||
"""Pure scoring functions for custom-lyrics eval."""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Iterable
|
||||
|
||||
from backend.services.custom_lyrics.validator import LineValidation
|
||||
|
||||
|
||||
@dataclass
|
||||
class PerLineScore:
|
||||
line_index: int
|
||||
min_delta: int
|
||||
pass_at_0: bool
|
||||
pass_at_1: bool
|
||||
pass_at_2: bool
|
||||
pass_at_4: bool
|
||||
severity: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class FixtureScore:
|
||||
fixture_id: str
|
||||
settings_name: str
|
||||
line_count: int
|
||||
pct_pass_at_2: float
|
||||
pct_pass_at_1: float
|
||||
pct_pass_at_0: float
|
||||
pct_pass_at_4: float
|
||||
mean_delta: float
|
||||
median_delta: float
|
||||
max_delta: int
|
||||
iterations_used: int
|
||||
stop_reason: str
|
||||
duration_ms: int
|
||||
gemini_calls: int
|
||||
line_count_match: bool
|
||||
severity_breakdown: dict[str, int]
|
||||
|
||||
|
||||
def score_per_line(val: LineValidation) -> PerLineScore:
|
||||
return PerLineScore(
|
||||
line_index=val.line_index,
|
||||
min_delta=val.min_delta,
|
||||
pass_at_0=val.min_delta <= 0,
|
||||
pass_at_1=val.min_delta <= 1,
|
||||
pass_at_2=val.min_delta <= 2,
|
||||
pass_at_4=val.min_delta <= 4,
|
||||
severity=val.severity.value,
|
||||
)
|
||||
|
||||
|
||||
def aggregate_fixture(
|
||||
*,
|
||||
fixture_id: str,
|
||||
settings_name: str,
|
||||
metadata: list[LineValidation],
|
||||
iterations_used: int,
|
||||
stop_reason: str,
|
||||
duration_ms: int,
|
||||
gemini_calls: int,
|
||||
line_count_match: bool,
|
||||
) -> FixtureScore:
|
||||
n = len(metadata)
|
||||
if n == 0:
|
||||
return FixtureScore(
|
||||
fixture_id=fixture_id, settings_name=settings_name,
|
||||
line_count=0,
|
||||
pct_pass_at_2=0, pct_pass_at_1=0, pct_pass_at_0=0, pct_pass_at_4=0,
|
||||
mean_delta=0, median_delta=0, max_delta=0,
|
||||
iterations_used=iterations_used, stop_reason=stop_reason,
|
||||
duration_ms=duration_ms, gemini_calls=gemini_calls,
|
||||
line_count_match=line_count_match,
|
||||
severity_breakdown={"ok": 0, "minor": 0, "major": 0},
|
||||
)
|
||||
deltas = sorted(v.min_delta for v in metadata)
|
||||
severity_breakdown = {"ok": 0, "minor": 0, "major": 0}
|
||||
for v in metadata:
|
||||
severity_breakdown[v.severity.value] += 1
|
||||
return FixtureScore(
|
||||
fixture_id=fixture_id,
|
||||
settings_name=settings_name,
|
||||
line_count=n,
|
||||
pct_pass_at_2=sum(1 for d in deltas if d <= 2) / n,
|
||||
pct_pass_at_1=sum(1 for d in deltas if d <= 1) / n,
|
||||
pct_pass_at_0=sum(1 for d in deltas if d <= 0) / n,
|
||||
pct_pass_at_4=sum(1 for d in deltas if d <= 4) / n,
|
||||
mean_delta=sum(deltas) / n,
|
||||
median_delta=deltas[n // 2],
|
||||
max_delta=deltas[-1],
|
||||
iterations_used=iterations_used,
|
||||
stop_reason=stop_reason,
|
||||
duration_ms=duration_ms,
|
||||
gemini_calls=gemini_calls,
|
||||
line_count_match=line_count_match,
|
||||
severity_breakdown=severity_breakdown,
|
||||
)
|
||||
|
||||
|
||||
def aggregate_corpus(per_fixture: Iterable[FixtureScore]) -> dict:
|
||||
"""Macro-averaged corpus aggregate."""
|
||||
fixtures = list(per_fixture)
|
||||
if not fixtures:
|
||||
return {}
|
||||
return {
|
||||
"fixture_count": len(fixtures),
|
||||
"macro_pct_pass_at_2": sum(f.pct_pass_at_2 for f in fixtures) / len(fixtures),
|
||||
"macro_pct_pass_at_1": sum(f.pct_pass_at_1 for f in fixtures) / len(fixtures),
|
||||
"macro_pct_pass_at_0": sum(f.pct_pass_at_0 for f in fixtures) / len(fixtures),
|
||||
"macro_pct_pass_at_4": sum(f.pct_pass_at_4 for f in fixtures) / len(fixtures),
|
||||
"macro_mean_delta": sum(f.mean_delta for f in fixtures) / len(fixtures),
|
||||
"total_gemini_calls": sum(f.gemini_calls for f in fixtures),
|
||||
}
|
||||
Whitespace-only changes.
@@ -1,38 +0,0 @@
|
||||
"""Tests for the LLM-call disk cache."""
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from backend.eval.custom_lyrics.cache import LlmCallCache
|
||||
|
||||
|
||||
def test_miss_returns_none() -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
c = LlmCallCache(Path(tmp))
|
||||
assert c.get("system", "user", model="m1", settings_dict={"x": 1}) is None
|
||||
|
||||
|
||||
def test_set_then_get_hits() -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
c = LlmCallCache(Path(tmp))
|
||||
c.set("system", "user", model="m1", settings_dict={"x": 1}, response_lines=["a", "b"])
|
||||
assert c.get("system", "user", model="m1", settings_dict={"x": 1}) == ["a", "b"]
|
||||
|
||||
|
||||
def test_different_settings_different_keys() -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
c = LlmCallCache(Path(tmp))
|
||||
c.set("system", "user", model="m1", settings_dict={"x": 1}, response_lines=["a"])
|
||||
assert c.get("system", "user", model="m1", settings_dict={"x": 2}) is None
|
||||
|
||||
|
||||
def test_replay_only_raises_on_miss() -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
c = LlmCallCache(Path(tmp), replay_only=True)
|
||||
try:
|
||||
c.get("system", "user", model="m1", settings_dict={"x": 1})
|
||||
except RuntimeError as e:
|
||||
assert "cache miss" in str(e).lower()
|
||||
else:
|
||||
raise AssertionError("expected RuntimeError")
|
||||
@@ -1,66 +0,0 @@
|
||||
"""Tests for eval scorer."""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from backend.eval.custom_lyrics.scorer import (
|
||||
PerLineScore,
|
||||
FixtureScore,
|
||||
score_per_line,
|
||||
aggregate_fixture,
|
||||
)
|
||||
from backend.services.custom_lyrics.validator import LineValidation, Severity
|
||||
|
||||
|
||||
def _line_val(idx: int, target: list[int], cand: list[int], passes: bool, severity: Severity, delta: int) -> LineValidation:
|
||||
return LineValidation(
|
||||
line_index=idx,
|
||||
target_text=f"t{idx}",
|
||||
candidate_text=f"c{idx}",
|
||||
target_syllables=target,
|
||||
candidate_syllables=cand,
|
||||
min_delta=delta,
|
||||
passes=passes,
|
||||
severity=severity,
|
||||
time_budget_seconds=1.0,
|
||||
)
|
||||
|
||||
|
||||
def test_score_per_line_pass_fail_thresholds() -> None:
|
||||
val = _line_val(0, [5, 5, 5, 5], [6, 6, 6, 6], passes=False, severity=Severity.MINOR, delta=1)
|
||||
s = score_per_line(val)
|
||||
assert s.pass_at_0 is False
|
||||
assert s.pass_at_1 is True
|
||||
assert s.pass_at_2 is True
|
||||
assert s.pass_at_4 is True
|
||||
assert s.min_delta == 1
|
||||
|
||||
|
||||
def test_score_per_line_major_violation() -> None:
|
||||
val = _line_val(0, [5, 5, 5, 5], [12, 12, 12, 12], passes=False, severity=Severity.MAJOR, delta=7)
|
||||
s = score_per_line(val)
|
||||
assert s.pass_at_0 is False
|
||||
assert s.pass_at_4 is False
|
||||
assert s.severity == "major"
|
||||
|
||||
|
||||
def test_aggregate_fixture_basic() -> None:
|
||||
metadata = [
|
||||
_line_val(0, [5]*4, [5]*4, passes=True, severity=Severity.OK, delta=0),
|
||||
_line_val(1, [5]*4, [6]*4, passes=False, severity=Severity.MINOR, delta=1),
|
||||
_line_val(2, [5]*4, [9]*4, passes=False, severity=Severity.MAJOR, delta=4),
|
||||
]
|
||||
fixture = aggregate_fixture(
|
||||
fixture_id="x",
|
||||
settings_name="default",
|
||||
metadata=metadata,
|
||||
iterations_used=2,
|
||||
stop_reason="max_iters_reached",
|
||||
duration_ms=1500,
|
||||
gemini_calls=3,
|
||||
line_count_match=True,
|
||||
)
|
||||
assert fixture.pct_pass_at_2 == pytest.approx(2 / 3)
|
||||
assert fixture.pct_pass_at_0 == pytest.approx(1 / 3)
|
||||
assert fixture.mean_delta == pytest.approx((0 + 1 + 4) / 3)
|
||||
assert fixture.max_delta == 4
|
||||
+31
-59
@@ -2,10 +2,40 @@
|
||||
Custom exceptions for the karaoke generation backend.
|
||||
|
||||
These exceptions are used for structured error handling across the application,
|
||||
particularly for validation errors and credit checks.
|
||||
particularly for rate limiting and validation errors.
|
||||
"""
|
||||
|
||||
|
||||
class RateLimitExceededError(Exception):
|
||||
"""
|
||||
Raised when a rate limit is exceeded.
|
||||
|
||||
Includes information about when the limit will reset for client retry logic.
|
||||
|
||||
Attributes:
|
||||
message: Human-readable error message
|
||||
limit_type: Type of limit exceeded (e.g., "jobs_per_day", "youtube_uploads")
|
||||
remaining_seconds: Seconds until the limit resets (for Retry-After header)
|
||||
current_count: Current usage count
|
||||
limit_value: The limit that was exceeded
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
limit_type: str = "unknown",
|
||||
remaining_seconds: int = 0,
|
||||
current_count: int = 0,
|
||||
limit_value: int = 0
|
||||
):
|
||||
self.message = message
|
||||
self.limit_type = limit_type
|
||||
self.remaining_seconds = remaining_seconds
|
||||
self.current_count = current_count
|
||||
self.limit_value = limit_value
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class EmailValidationError(Exception):
|
||||
"""
|
||||
Raised when email validation fails.
|
||||
@@ -34,61 +64,3 @@ class IPBlockedError(Exception):
|
||||
self.message = message
|
||||
self.ip_address = ip_address
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class InsufficientCreditsError(Exception):
|
||||
"""
|
||||
Raised when a user attempts to create a job without sufficient credits.
|
||||
|
||||
Attributes:
|
||||
message: Human-readable error message
|
||||
credits_available: Number of credits the user currently has
|
||||
credits_required: Number of credits required (always 1 for now)
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
credits_available: int = 0,
|
||||
credits_required: int = 1
|
||||
):
|
||||
self.message = message
|
||||
self.credits_available = credits_available
|
||||
self.credits_required = credits_required
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class InvalidStateTransitionError(Exception):
|
||||
"""
|
||||
Raised when an invalid job state transition is attempted.
|
||||
|
||||
This exception is raised by JobManager.transition_to_state() when the
|
||||
requested transition is not allowed by the state machine defined in
|
||||
STATE_TRANSITIONS.
|
||||
|
||||
Example: Trying to transition from PENDING to GENERATING_SCREENS is invalid
|
||||
because PENDING can only transition to DOWNLOADING, SEARCHING_AUDIO, FAILED,
|
||||
or CANCELLED.
|
||||
|
||||
Attributes:
|
||||
message: Human-readable error message
|
||||
job_id: The job ID that failed to transition
|
||||
from_status: Current status of the job
|
||||
to_status: Attempted target status
|
||||
valid_transitions: List of valid transitions from current status
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
job_id: str = "",
|
||||
from_status: str = "",
|
||||
to_status: str = "",
|
||||
valid_transitions: list = None
|
||||
):
|
||||
self.message = message
|
||||
self.job_id = job_id
|
||||
self.from_status = from_status
|
||||
self.to_status = to_status
|
||||
self.valid_transitions = valid_transitions or []
|
||||
super().__init__(message)
|
||||
-144
@@ -1,144 +0,0 @@
|
||||
"""
|
||||
Lightweight i18n utility for backend translations.
|
||||
|
||||
Usage:
|
||||
from backend.i18n import t, get_locale_from_request, SUPPORTED_LOCALES
|
||||
|
||||
# In a route handler:
|
||||
locale = get_locale_from_request(request)
|
||||
message = t(locale, "auth.userNotFound")
|
||||
message = t(locale, "jobs.notFound", job_id="abc123")
|
||||
"""
|
||||
|
||||
import json
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from starlette.requests import Request
|
||||
|
||||
SUPPORTED_LOCALES = ("en", "es", "de")
|
||||
# Every locale the web app serves (frontend/i18n/routing.ts, frontend/messages/*.json;
|
||||
# a test keeps them in sync). Use for web URL prefixes; SUPPORTED_LOCALES is the
|
||||
# backend's own (email / API message) translations.
|
||||
WEB_LOCALES = (
|
||||
"en", "es", "de", "pt", "fr", "ja", "ko", "zh", "it", "nl",
|
||||
"pl", "tr", "ru", "th", "id", "vi", "tl", "hi", "ar", "sv",
|
||||
"nb", "da", "fi", "cs", "ro", "hu", "el", "he", "ms", "uk",
|
||||
"hr", "sk", "ca",
|
||||
)
|
||||
DEFAULT_LOCALE = "en"
|
||||
TRANSLATIONS_DIR = Path(__file__).parent / "translations"
|
||||
|
||||
|
||||
@lru_cache(maxsize=len(SUPPORTED_LOCALES))
|
||||
def _load_translations(locale: str) -> dict[str, Any]:
|
||||
"""Load and cache translation file for a locale."""
|
||||
path = TRANSLATIONS_DIR / f"{locale}.json"
|
||||
if not path.exists():
|
||||
if locale != DEFAULT_LOCALE:
|
||||
return _load_translations(DEFAULT_LOCALE)
|
||||
return {}
|
||||
with open(path, encoding="utf-8") as f:
|
||||
result: dict[str, Any] = json.load(f)
|
||||
return result
|
||||
|
||||
|
||||
def t(locale: str, key: str, **kwargs: Any) -> str:
|
||||
"""Get a translated string by dotted key path.
|
||||
|
||||
Args:
|
||||
locale: Language code (en, es, de)
|
||||
key: Dotted path like "auth.userNotFound" or "emails.magicLink.subject"
|
||||
**kwargs: Interpolation variables (e.g., job_id="abc123")
|
||||
|
||||
Returns:
|
||||
Translated string with variables interpolated, or the key itself if not found.
|
||||
"""
|
||||
if locale not in SUPPORTED_LOCALES:
|
||||
locale = DEFAULT_LOCALE
|
||||
|
||||
translations = _load_translations(locale)
|
||||
|
||||
# Walk the dotted path
|
||||
value: Any = translations
|
||||
for part in key.split("."):
|
||||
if isinstance(value, dict) and part in value:
|
||||
value = value[part]
|
||||
else:
|
||||
# Key not found — fall back to English
|
||||
if locale != DEFAULT_LOCALE:
|
||||
return t(DEFAULT_LOCALE, key, **kwargs)
|
||||
return key # Return the key itself as last resort
|
||||
|
||||
if not isinstance(value, str):
|
||||
return key
|
||||
|
||||
result: str = value
|
||||
# Interpolate variables
|
||||
if kwargs:
|
||||
try:
|
||||
result = result.format(**kwargs)
|
||||
except KeyError:
|
||||
pass # Missing variable — return template as-is
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def get_locale_from_request(request: Request) -> str:
|
||||
"""Extract locale from Accept-Language header.
|
||||
|
||||
Returns the first supported locale found, or DEFAULT_LOCALE.
|
||||
"""
|
||||
accept_lang = request.headers.get("accept-language", "")
|
||||
|
||||
# Simple parsing — Accept-Language can be "de", "de-DE", "en-US,en;q=0.9,de;q=0.8"
|
||||
for part in accept_lang.split(","):
|
||||
lang = part.split(";")[0].strip().split("-")[0].lower()
|
||||
if lang in SUPPORTED_LOCALES:
|
||||
return lang
|
||||
|
||||
return DEFAULT_LOCALE
|
||||
|
||||
|
||||
def get_full_locale_from_request(request: Request) -> str | None:
|
||||
"""Extract the user's actual UI language tag from Accept-Language.
|
||||
|
||||
Unlike :func:`get_locale_from_request`, this does NOT narrow to the 3
|
||||
email-supported locales (en/es/de). It returns the primary language subtag
|
||||
of the first entry in the header (e.g. ``pt``, ``ja``, ``zh``) so the admin
|
||||
UI can show the real language a customer was using the product in.
|
||||
|
||||
Returns the lowercased language subtag, or ``None`` if the header is absent
|
||||
or unparseable.
|
||||
"""
|
||||
accept_lang = request.headers.get("accept-language", "")
|
||||
|
||||
for part in accept_lang.split(","):
|
||||
# Each part looks like "en-US;q=0.9" — take the language-region token
|
||||
# before the quality factor, then the primary subtag before any region.
|
||||
segments = part.split(";")
|
||||
token = segments[0].strip()
|
||||
if not token or token == "*":
|
||||
continue
|
||||
# Skip ranges the client explicitly rejects with q=0.
|
||||
if any(seg.strip().lower().replace(" ", "") in ("q=0", "q=0.0", "q=0.00", "q=0.000")
|
||||
for seg in segments[1:]):
|
||||
continue
|
||||
lang = token.split("-")[0].strip().lower()
|
||||
if lang.isalpha() and len(lang) >= 2:
|
||||
return lang
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def get_web_locale_prefix(locale: str) -> str:
|
||||
"""Web URL locale prefix for any of the web app's locales (fallback '/en')."""
|
||||
return f"/{locale if locale in WEB_LOCALES else DEFAULT_LOCALE}"
|
||||
|
||||
|
||||
def get_locale_prefix(locale: str) -> str:
|
||||
"""Get URL locale prefix for frontend URLs (e.g., '/en', '/es')."""
|
||||
if locale not in SUPPORTED_LOCALES:
|
||||
locale = DEFAULT_LOCALE
|
||||
return f"/{locale}"
|
||||
+35
-315
@@ -1,21 +1,13 @@
|
||||
"""
|
||||
FastAPI application entry point for karaoke generation backend.
|
||||
|
||||
(Trivial comment touch to trigger backend CI on ephemeral runners after
|
||||
the 2026-05-17 dispatcher e2 fix; safe to remove on next backend edit.)
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from backend.config import settings, validate_production_config
|
||||
from backend.services.fake_workers import fake_workers_enabled
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users, admin, tenant, tenant_admin, tenant_bulk, tenant_theme, rate_limits, push, catalog, encoding_worker, client_errors, client_events, bulk, parse_titles
|
||||
from backend.config import settings
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users, admin, tenant, rate_limits, push
|
||||
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
|
||||
from backend.services.structured_logging import setup_structured_logging
|
||||
from backend.services.spacy_preloader import preload_spacy_model
|
||||
@@ -23,7 +15,6 @@ from backend.services.nltk_preloader import preload_all_nltk_resources
|
||||
from backend.services.langfuse_preloader import preload_langfuse_handler
|
||||
from backend.middleware.audit_logging import AuditLoggingMiddleware
|
||||
from backend.middleware.tenant import TenantMiddleware
|
||||
from backend.middleware.edge_auth import EdgeAuthMiddleware
|
||||
from backend.workers.registry import worker_registry
|
||||
|
||||
|
||||
@@ -43,7 +34,7 @@ tracing_enabled = setup_tracing(
|
||||
)
|
||||
|
||||
|
||||
def validate_credentials_on_startup():
|
||||
async def validate_credentials_on_startup():
|
||||
"""Validate OAuth credentials on startup and send alerts if needed."""
|
||||
try:
|
||||
from backend.services.credential_manager import get_credential_manager, CredentialStatus
|
||||
@@ -73,122 +64,17 @@ def validate_credentials_on_startup():
|
||||
logger.error(f"Failed to validate credentials on startup: {e}")
|
||||
|
||||
|
||||
# Modules that route handlers import lazily. On a fresh instance the first such
|
||||
# import ran ON the event loop: httpx/httpcore (via flacfetch_client /
|
||||
# worker_service) froze it 3-15s on cold disk reads (EVENT_LOOP_STALL in
|
||||
# importlib, 2026-10-08). Lifespan startup imports them in a worker thread before
|
||||
# readiness — a background thread alone would still race the first request.
|
||||
_LAZY_ROUTE_IMPORTS = (
|
||||
"httpx",
|
||||
# Imported lazily by httpx / worker_service on first use; still froze the loop
|
||||
# after the list above existed (httpcore ~1s; google.cloud.run_v2 14.9s with
|
||||
# 7 requests in flight, 2026-10-16).
|
||||
"httpcore",
|
||||
"google.cloud.run_v2",
|
||||
"backend.services.worker_service",
|
||||
"backend.services.flacfetch_client",
|
||||
"backend.services.catalog_proxy_service",
|
||||
"backend.services.ip_geolocation_service",
|
||||
"backend.services.email_validation_service",
|
||||
"backend.services.musicbrainz_service",
|
||||
# audio_search's auto_download path: select_best -> flacfetch.core.manager ->
|
||||
# yt_dlp import froze the loop ~11-12s on fresh instances (2026-10-23, 10-26).
|
||||
"flacfetch.core.manager",
|
||||
"flacfetch.downloaders.youtube",
|
||||
"yt_dlp",
|
||||
# starlette/middleware/base.py picks the anyio backend on the first request:
|
||||
# 17.3s on the loop with 18 requests in flight (2026-10-30).
|
||||
"anyio._backends._asyncio",
|
||||
)
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Lifespan event handler for startup and shutdown."""
|
||||
# Startup
|
||||
logger.info("Starting karaoke generation backend")
|
||||
logger.info(f"Environment: {settings.environment}")
|
||||
logger.info(f"GCS Bucket: {settings.gcs_bucket_name}")
|
||||
logger.info(f"Tracing enabled: {tracing_enabled}")
|
||||
|
||||
|
||||
LAZY_ROUTE_IMPORT_TIMEOUT_S = 30
|
||||
|
||||
|
||||
def _import_lazy_route_modules() -> None:
|
||||
import importlib
|
||||
|
||||
for module in _LAZY_ROUTE_IMPORTS:
|
||||
try:
|
||||
importlib.import_module(module)
|
||||
except Exception as e:
|
||||
logger.warning(f"Startup import of {module} failed (will import lazily): {e}")
|
||||
|
||||
|
||||
def _warm_route_contexts(application: FastAPI) -> int:
|
||||
"""Build FastAPI's per-route contexts (path regexes, pydantic schemas) up front.
|
||||
|
||||
Newer FastAPI builds them lazily on the first request, and the OpenTelemetry
|
||||
FastAPI instrumentation walks every route (``_get_route_details`` ->
|
||||
``iter_route_contexts``) for each request's span name. On a fresh instance that
|
||||
first walk compiled hundreds of route regexes ON the event loop (2026-10-23: 11s,
|
||||
3 requests; 5.1s on 10-22). The results are cached (by routes version, under a
|
||||
lock), so one walk here, before readiness, covers every later request.
|
||||
Best-effort: a FastAPI version without these internals just skips it.
|
||||
Returns the number of route contexts built (0 if unsupported).
|
||||
"""
|
||||
try:
|
||||
from fastapi.routing import iter_route_contexts
|
||||
except ImportError: # older FastAPI compiles routes eagerly at include time
|
||||
return 0
|
||||
count = 0
|
||||
for _ in iter_route_contexts(application.routes):
|
||||
count += 1
|
||||
for route in application.routes:
|
||||
low_priority = getattr(route, "effective_low_priority_routes", None)
|
||||
if callable(low_priority):
|
||||
count += len(low_priority())
|
||||
return count
|
||||
|
||||
|
||||
def _warm_middleware_stack(application: Any) -> None:
|
||||
"""Send one request through the app, so the first user's request isn't what builds
|
||||
the middleware stack and runs its first-use imports on the loop.
|
||||
|
||||
Blocking: runs its own event loop, so call it from a worker thread. /api/health is
|
||||
exempt from the edge lock, skips the audit log and touches no storage. Best-effort.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
async def _request() -> None:
|
||||
transport = httpx.ASGITransport(app=application)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://localhost") as client:
|
||||
await client.get("/api/health")
|
||||
|
||||
start = time.time()
|
||||
try:
|
||||
asyncio.run(_request())
|
||||
except Exception as e: # noqa: BLE001 - warmup must never block startup
|
||||
logger.warning(f"Middleware warm request failed (stack builds on first request): {e}")
|
||||
return
|
||||
logger.info(f"Warmed middleware stack in {time.time() - start:.2f}s")
|
||||
|
||||
|
||||
def _warm_route_contexts_safely(application: FastAPI) -> None:
|
||||
start = time.time()
|
||||
try:
|
||||
built = _warm_route_contexts(application)
|
||||
except Exception as e: # noqa: BLE001 - warmup must never block startup
|
||||
logger.warning(f"Route-context warmup failed (routes build lazily): {e}")
|
||||
return
|
||||
if built:
|
||||
logger.info(f"Warmed {built} route contexts in {time.time() - start:.2f}s")
|
||||
|
||||
|
||||
def _run_background_warmup():
|
||||
"""Warm caches that used to block startup (runs in a daemon thread).
|
||||
|
||||
Historically these preloads ran inline in lifespan startup, which meant
|
||||
Cloud Run held every routed request for the full ~7.5s they take (spaCy
|
||||
~1s, NLTK ~2.5s, Langfuse ~3s, credential checks ~1.5s) on top of import
|
||||
time. None of them are needed to serve HTTP: they exist to make the FIRST
|
||||
lyrics-processing job fast (see docs/archive/2026-01-08-performance-
|
||||
investigation.md). Running them in a background thread right after boot
|
||||
keeps that warm-cache property (they finish within seconds, long before
|
||||
any real job runs) without gating readiness. Workers that race the warmup
|
||||
simply fall back to the preloaders' lazy paths.
|
||||
"""
|
||||
warmup_start = time.time()
|
||||
# Preload NLP models and resources to avoid cold start delays
|
||||
# See docs/archive/2026-01-08-performance-investigation.md for background
|
||||
|
||||
# 1. SpaCy model (60+ second delay without preload)
|
||||
try:
|
||||
@@ -208,159 +94,27 @@ def _run_background_warmup():
|
||||
except Exception as e:
|
||||
logger.warning(f"Langfuse preload failed (will initialize lazily): {e}")
|
||||
|
||||
if fake_workers_enabled(settings):
|
||||
# Local stack: no encoding VM and no OAuth credentials to validate.
|
||||
logger.info(f"Background warmup complete in {time.time() - warmup_start:.2f}s (fake workers)")
|
||||
return
|
||||
|
||||
# 4. Encoding service singleton — its first construction imports
|
||||
# google.cloud.compute_v1 (~10s on a fresh instance). Build it here, off the
|
||||
# event loop, before /api/health/detailed or a preview encode needs it.
|
||||
# Validate OAuth credentials (non-blocking)
|
||||
try:
|
||||
from backend.services.encoding_service import get_encoding_service
|
||||
|
||||
get_encoding_service()
|
||||
except Exception as e:
|
||||
logger.warning(f"Encoding service warmup failed (will build lazily): {e}")
|
||||
|
||||
# Validate OAuth credentials (alerting only, nothing depends on it)
|
||||
try:
|
||||
validate_credentials_on_startup()
|
||||
await validate_credentials_on_startup()
|
||||
except Exception as e:
|
||||
logger.error(f"Credential validation failed: {e}")
|
||||
|
||||
logger.info(f"Background warmup complete in {time.time() - warmup_start:.2f}s")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Lifespan event handler for startup and shutdown."""
|
||||
# Startup
|
||||
logger.info("Starting karaoke generation backend")
|
||||
logger.info(f"Environment: {settings.environment}")
|
||||
logger.info(f"GCS Bucket: {settings.gcs_bucket_name}")
|
||||
logger.info(f"Tracing enabled: {tracing_enabled}")
|
||||
|
||||
# Fail fast if required production config is missing, rather than silently
|
||||
# running on dev defaults (wrong GCP project / GCS bucket / localhost worker
|
||||
# URL). No-op outside production. (Fallback audit 2026-06-09, Theme 7.)
|
||||
validate_production_config()
|
||||
|
||||
# Off the loop, but before readiness (Cloud Run holds requests meanwhile).
|
||||
# Bounded so a hung cold-disk read can't keep the instance from ever going
|
||||
# ready; the import keeps running in its thread and routes fall back to it.
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
asyncio.to_thread(_import_lazy_route_modules), timeout=LAZY_ROUTE_IMPORT_TIMEOUT_S
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
logger.error(
|
||||
f"Startup import of lazy route modules exceeded {LAZY_ROUTE_IMPORT_TIMEOUT_S}s; "
|
||||
"continuing startup (first requests may import them on the event loop)"
|
||||
)
|
||||
|
||||
# Same reason: build the route contexts in a thread, before the first request
|
||||
# (and the OTel route walk) would build them on the loop.
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
asyncio.to_thread(_warm_route_contexts_safely, app), timeout=LAZY_ROUTE_IMPORT_TIMEOUT_S
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning(
|
||||
f"Route-context warmup exceeded {LAZY_ROUTE_IMPORT_TIMEOUT_S}s; continuing startup"
|
||||
)
|
||||
|
||||
# Same reason, for what only the first request through the middleware touches.
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
asyncio.to_thread(_warm_middleware_stack, app), timeout=LAZY_ROUTE_IMPORT_TIMEOUT_S
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning(
|
||||
f"Middleware warmup exceeded {LAZY_ROUTE_IMPORT_TIMEOUT_S}s; continuing startup"
|
||||
)
|
||||
|
||||
# NLP model / credential warmup runs in the background so it doesn't gate
|
||||
# readiness — Cloud Run holds all routed requests until lifespan startup
|
||||
# returns, and these preloads were ~7.5s of the ~16s cold start.
|
||||
threading.Thread(
|
||||
target=_run_background_warmup, name="startup-warmup", daemon=True
|
||||
).start()
|
||||
|
||||
# asyncio.to_thread's default executor is min(32, cpu+4) ≈ 6 threads on our
|
||||
# 2 vCPUs. The screens worker, stale-review cron, auto-correct and many
|
||||
# review endpoints all offload into it (mostly I/O-bound), so size it up —
|
||||
# otherwise review-page requests queue behind a few long-running jobs.
|
||||
import concurrent.futures
|
||||
|
||||
asyncio.get_running_loop().set_default_executor(
|
||||
concurrent.futures.ThreadPoolExecutor(max_workers=48, thread_name_prefix="to-thread")
|
||||
)
|
||||
|
||||
# Event-loop stall watchdog: makes "every request froze for 30s" episodes
|
||||
# self-diagnosing (logs the blocking stack; records >=5s stalls in prod).
|
||||
watchdog = None
|
||||
if settings.loop_watchdog_enabled:
|
||||
from backend.services.loop_watchdog import (
|
||||
LoopWatchdog,
|
||||
firestore_stall_recorder,
|
||||
request_tracker,
|
||||
)
|
||||
|
||||
watchdog = LoopWatchdog(
|
||||
recorder=firestore_stall_recorder if settings.environment == "production" else None,
|
||||
# Stalls nobody waited on (idle, CPU-throttled autoscaled instances)
|
||||
# stay WARNING so they don't trip the >=10s stall alert.
|
||||
request_tracker=request_tracker,
|
||||
)
|
||||
watchdog.start()
|
||||
|
||||
|
||||
yield
|
||||
|
||||
if watchdog is not None:
|
||||
await watchdog.stop()
|
||||
|
||||
# Shutdown - best-effort parking of any still-registered workers.
|
||||
#
|
||||
# Reality check (incident 2026-09-13, job 41e06b90): Cloud Run services
|
||||
# SIGKILL instances ~10 seconds after SIGTERM — NOT the 600s this comment
|
||||
# previously claimed — and uvicorn only runs this lifespan-shutdown code
|
||||
# AFTER all BackgroundTasks have completed ("Waiting for background tasks
|
||||
# to complete"). A long-running worker task therefore keeps uvicorn stuck
|
||||
# in that wait until SIGKILL, and this hook never executes at all. It can
|
||||
# NOT be relied on to protect in-flight renders; that protection comes
|
||||
# from running them as Cloud Run Jobs instead (USE_CLOUD_RUN_JOBS_FOR_RENDER,
|
||||
# mirroring USE_CLOUD_RUN_JOBS_FOR_VIDEO from incident 2026-03-08).
|
||||
#
|
||||
# This hook is kept as a cheap safety net for the legacy flag-off path and
|
||||
# for the narrow case where workers finish right as SIGTERM lands. The wait
|
||||
# is capped well inside the 10s kill window so the parking pass below still
|
||||
# gets a chance to write Firestore state.
|
||||
# Shutdown - wait for any active workers to complete before terminating
|
||||
# This prevents Cloud Run from killing workers mid-processing
|
||||
logger.info("Shutdown requested, checking for active workers...")
|
||||
if worker_registry.has_active_workers():
|
||||
active = worker_registry.get_active_workers()
|
||||
logger.info(f"Active workers found: {active}")
|
||||
logger.info("Waiting for workers to complete (timeout: 5s)...")
|
||||
completed = await worker_registry.wait_for_completion(timeout=5)
|
||||
logger.info("Waiting for workers to complete (timeout: 600s)...")
|
||||
completed = await worker_registry.wait_for_completion(timeout=600) # 10 min max
|
||||
if not completed:
|
||||
logger.error(
|
||||
"Shutdown timeout - some workers may not have completed cleanly. "
|
||||
f"Remaining workers: {worker_registry.get_active_workers()}"
|
||||
)
|
||||
|
||||
# Park any still-active render jobs so the auto-retry scheduler can
|
||||
# recover them. Safe to call unconditionally — it's a no-op if nothing
|
||||
# render-related is in flight.
|
||||
try:
|
||||
from backend.workers.render_video_worker import park_active_render_jobs_for_shutdown
|
||||
parked = park_active_render_jobs_for_shutdown()
|
||||
if parked:
|
||||
logger.warning(
|
||||
f"Parked {parked} in-flight render job(s) for auto-retry "
|
||||
f"due to shutdown"
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(f"Render-job shutdown park failed: {exc!r}")
|
||||
else:
|
||||
logger.info("No active workers, proceeding with shutdown")
|
||||
|
||||
@@ -382,8 +136,8 @@ if tracing_enabled:
|
||||
# Configure CORS
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=False,
|
||||
allow_origins=["*"], # Configure this properly for production
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
@@ -394,76 +148,42 @@ app.add_middleware(AuditLoggingMiddleware)
|
||||
# Add tenant detection middleware (extracts tenant from subdomain/headers)
|
||||
app.add_middleware(TenantMiddleware)
|
||||
|
||||
# Edge origin lock (added last = outermost, runs first). Rejects direct-to-origin
|
||||
# requests that bypassed the Cloudflare edge, before tenant/audit middleware do
|
||||
# any work. No-op unless EDGE_AUTH_MODE is warn/enforce. See
|
||||
# backend/middleware/edge_auth.py.
|
||||
app.add_middleware(EdgeAuthMiddleware)
|
||||
|
||||
# Counts in-flight requests for the loop watchdog (outermost, so every request that
|
||||
# waited on a stalled loop counts — even ones the edge lock rejects).
|
||||
from backend.services.loop_watchdog import RequestTrackingMiddleware # noqa: E402
|
||||
|
||||
app.add_middleware(RequestTrackingMiddleware)
|
||||
|
||||
# Include routers
|
||||
app.include_router(health.router, prefix="/api")
|
||||
app.include_router(jobs.router, prefix="/api")
|
||||
app.include_router(file_upload.router, prefix="/api") # File upload endpoint
|
||||
app.include_router(internal.router, prefix="/api") # Internal worker endpoints
|
||||
app.include_router(encoding_worker.router, prefix="/api") # Encoding worker lifecycle
|
||||
app.include_router(review.router, prefix="/api") # Review UI compatibility endpoints
|
||||
app.include_router(auth.router, prefix="/api") # OAuth credential management
|
||||
app.include_router(audio_search.router, prefix="/api") # Audio search (artist+title mode)
|
||||
app.include_router(themes.router, prefix="/api") # Theme selection for styles
|
||||
app.include_router(users.router, prefix="/api") # User auth, credits, and Stripe webhooks
|
||||
from backend.api.routes import referrals
|
||||
app.include_router(referrals.router, prefix="/api") # Referral links and payouts
|
||||
app.include_router(admin.router, prefix="/api") # Admin dashboard and management
|
||||
app.include_router(rate_limits.router, prefix="/api") # Rate limits admin management
|
||||
app.include_router(push.router, prefix="/api") # Push notification subscription management
|
||||
app.include_router(catalog.router, prefix="/api") # Catalog proxy for song/artist autocomplete
|
||||
app.include_router(parse_titles.router, prefix="/api") # kjbox karaoke-filename parser
|
||||
app.include_router(bulk.router, prefix="/api") # Bulk Mode: multi-job submission
|
||||
from backend.api.routes import requests_board
|
||||
app.include_router(requests_board.router, prefix="/api") # Public song-request voting board
|
||||
app.include_router(client_errors.router, prefix="/api") # Frontend crash reports
|
||||
app.include_router(client_events.router, prefix="/api") # Frontend degradation telemetry (banner/unavailable/waveform events)
|
||||
from backend.api.routes import karaokehunt
|
||||
app.include_router(karaokehunt.router, prefix="/api") # Retired KaraokeHunt app request interceptor
|
||||
from backend.api.routes import kjbox
|
||||
app.include_router(kjbox.router, prefix="/api") # kjbox partner API: singer email-code sign-in + show credits
|
||||
from backend.api.routes import login_code
|
||||
app.include_router(login_code.router, prefix="/api") # gen email-code sign-in (/users/auth/send-code, verify-code)
|
||||
app.include_router(tenant.router) # Tenant/white-label configuration (no /api prefix, router has it)
|
||||
app.include_router(tenant_admin.router) # Admin tenant provisioning (router has /api prefix)
|
||||
app.include_router(tenant_bulk.router) # Tenant bulk-upload filename analysis (router has /api prefix)
|
||||
app.include_router(tenant_theme.router) # Tenant self-service theme editor (router has /api prefix)
|
||||
if fake_workers_enabled(settings):
|
||||
# Local E2E stack only (startup refuses FAKE_WORKERS in production).
|
||||
from backend.api.routes import test_support
|
||||
app.include_router(test_support.router, prefix="/api")
|
||||
|
||||
|
||||
# Exception handlers
|
||||
# Exception handler for rate limiting
|
||||
from fastapi import Request
|
||||
from fastapi.responses import JSONResponse
|
||||
from backend.exceptions import InsufficientCreditsError
|
||||
from backend.i18n import t, get_locale_from_request
|
||||
from backend.exceptions import RateLimitExceededError
|
||||
|
||||
|
||||
@app.exception_handler(InsufficientCreditsError)
|
||||
async def insufficient_credits_exception_handler(request: Request, exc: InsufficientCreditsError):
|
||||
"""Handle insufficient credits errors with 402 Payment Required status."""
|
||||
locale = get_locale_from_request(request)
|
||||
@app.exception_handler(RateLimitExceededError)
|
||||
async def rate_limit_exception_handler(request: Request, exc: RateLimitExceededError):
|
||||
"""Handle rate limit exceeded errors with 429 status."""
|
||||
return JSONResponse(
|
||||
status_code=402,
|
||||
status_code=429,
|
||||
content={
|
||||
"detail": t(locale, "errors.insufficientCredits"),
|
||||
"credits_available": exc.credits_available,
|
||||
"credits_required": exc.credits_required,
|
||||
"buy_url": t(locale, "errors.insufficientCreditsBuyUrl"),
|
||||
"detail": exc.message,
|
||||
"limit_type": exc.limit_type,
|
||||
"current_count": exc.current_count,
|
||||
"limit_value": exc.limit_value,
|
||||
},
|
||||
headers={
|
||||
"Retry-After": str(exc.remaining_seconds),
|
||||
} if exc.remaining_seconds > 0 else None,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -1,115 +0,0 @@
|
||||
"""
|
||||
Edge origin-lock middleware.
|
||||
|
||||
Cloudflare fronts ``api.nomadkaraoke.com`` (WAF + rate limiting). To stop
|
||||
attackers from bypassing the edge by hitting the Cloud Run origin directly
|
||||
(``*.run.app`` / ``ghs.googlehosted.com``), Cloudflare injects a shared-secret
|
||||
header (``X-Edge-Auth``) on every proxied request via a Transform Rule. This
|
||||
middleware rejects public requests that lack the correct header.
|
||||
|
||||
A Cloud Run *domain mapping* can't use ``ingress=internal-and-cloud-load-balancing``
|
||||
(that needs a load balancer), so this app-layer check is how we lock the origin.
|
||||
|
||||
Modes (env ``EDGE_AUTH_MODE``, default ``off``):
|
||||
off - passthrough, no checks (deploy-dormant; behaviour unchanged).
|
||||
warn - log requests missing/failing the header, but ALLOW them. Use to
|
||||
confirm nothing legit is missing the header before enforcing.
|
||||
enforce - reject requests missing/failing the header with 403.
|
||||
|
||||
Secret: env ``EDGE_ORIGIN_SECRET`` (same value Cloudflare injects). If enforce
|
||||
mode is set but the secret is unconfigured, the middleware FAILS CLOSED (503) —
|
||||
a security control must not silently allow direct-origin traffic. (Low risk:
|
||||
Cloud Run won't start a revision whose secret ref is missing, and enforce is
|
||||
only enabled after staging confirms the secret.) In warn mode it still allows.
|
||||
|
||||
Exemptions (always allowed, even in enforce mode):
|
||||
* Health / root endpoints and Cloud Run probes, which hit the origin
|
||||
directly (not via Cloudflare) and legitimately lack the header.
|
||||
"""
|
||||
|
||||
import hmac
|
||||
import logging
|
||||
import os
|
||||
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import JSONResponse, Response
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Header Cloudflare injects (must match CloudflareConfig.ORIGIN_AUTH_HEADER in
|
||||
# infrastructure/modules/edge_security.py).
|
||||
EDGE_AUTH_HEADER = "X-Edge-Auth"
|
||||
|
||||
# Paths always allowed without the edge header. These are non-sensitive and are
|
||||
# probed directly on the origin by Cloud Run startup/liveness checks and GCP
|
||||
# uptime monitoring, which don't traverse Cloudflare.
|
||||
_EXEMPT_EXACT = {"/", "/api/health", "/api/health/"}
|
||||
_EXEMPT_PREFIXES = ("/api/health/",)
|
||||
|
||||
_VALID_MODES = {"off", "warn", "enforce"}
|
||||
|
||||
|
||||
def _mode() -> str:
|
||||
mode = os.environ.get("EDGE_AUTH_MODE", "off").lower().strip()
|
||||
return mode if mode in _VALID_MODES else "off"
|
||||
|
||||
|
||||
def _is_exempt(path: str) -> bool:
|
||||
if path in _EXEMPT_EXACT:
|
||||
return True
|
||||
return any(path.startswith(p) for p in _EXEMPT_PREFIXES)
|
||||
|
||||
|
||||
class EdgeAuthMiddleware(BaseHTTPMiddleware):
|
||||
"""Reject origin requests that didn't come through the Cloudflare edge."""
|
||||
|
||||
async def dispatch(self, request: Request, call_next) -> Response:
|
||||
mode = _mode()
|
||||
if mode == "off":
|
||||
return await call_next(request)
|
||||
|
||||
if _is_exempt(request.url.path):
|
||||
return await call_next(request)
|
||||
|
||||
secret = os.environ.get("EDGE_ORIGIN_SECRET", "")
|
||||
if not secret:
|
||||
# Misconfiguration: enforce is requested but no secret is wired up.
|
||||
# In enforce mode, fail CLOSED (503) — a security control must not
|
||||
# silently allow direct-origin traffic when it can't do its job.
|
||||
# This is low-risk operationally: Cloud Run won't even start a
|
||||
# revision whose --set-secrets ref is missing, and enforce is only
|
||||
# turned on after staging confirms the secret. In warn mode we still
|
||||
# allow (observational only).
|
||||
logger.error(
|
||||
"EDGE_AUTH_MODE=%s but EDGE_ORIGIN_SECRET is unset.", mode,
|
||||
)
|
||||
if mode == "enforce":
|
||||
return JSONResponse(
|
||||
status_code=503,
|
||||
content={"detail": "Origin authentication unavailable"},
|
||||
)
|
||||
return await call_next(request)
|
||||
|
||||
provided = request.headers.get(EDGE_AUTH_HEADER, "")
|
||||
# Constant-time comparison to avoid leaking the secret via timing.
|
||||
ok = bool(provided) and hmac.compare_digest(provided, secret)
|
||||
|
||||
if ok:
|
||||
return await call_next(request)
|
||||
|
||||
client = request.client.host if request.client else "?"
|
||||
if mode == "warn":
|
||||
logger.warning(
|
||||
"edge-auth: request missing/invalid %s header (warn mode, "
|
||||
"allowing). path=%s ip=%s",
|
||||
EDGE_AUTH_HEADER, request.url.path, client,
|
||||
)
|
||||
return await call_next(request)
|
||||
|
||||
# enforce
|
||||
logger.warning(
|
||||
"edge-auth: blocked direct-to-origin request. path=%s ip=%s",
|
||||
request.url.path, client,
|
||||
)
|
||||
return JSONResponse(status_code=403, content={"detail": "Forbidden"})
|
||||
@@ -13,7 +13,6 @@ The middleware is non-blocking - if no tenant is detected, the request
|
||||
proceeds as a default Nomad Karaoke request (tenant_id = None).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from typing import Optional
|
||||
@@ -27,19 +26,13 @@ from backend.models.tenant import TenantConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Only allow the `?tenant=` query-param override in non-production environments —
|
||||
# in production it is a tenant-spoofing vector. Production-safe by default: Cloud
|
||||
# Run always sets K_SERVICE, so we treat its presence as production even if
|
||||
# ENVIRONMENT were somehow unset (which would otherwise re-enable the override).
|
||||
# (Fallback audit 2026-06-09, Theme 7.)
|
||||
# Only allow query param tenant override in non-production environments
|
||||
IS_PRODUCTION = os.environ.get("ENV", "").lower() == "production" or \
|
||||
os.environ.get("ENVIRONMENT", "").lower() == "production" or \
|
||||
bool(os.environ.get("K_SERVICE"))
|
||||
os.environ.get("ENVIRONMENT", "").lower() == "production"
|
||||
|
||||
|
||||
# Known non-tenant subdomains that should be treated as default Nomad Karaoke.
|
||||
# Mirror of frontend/lib/edge/nonTenantSubdomains.ts ("requests" = voting-board host).
|
||||
NON_TENANT_SUBDOMAINS = {"gen", "api", "www", "buy", "admin", "app", "beta", "requests"}
|
||||
# Known non-tenant subdomains that should be treated as default Nomad Karaoke
|
||||
NON_TENANT_SUBDOMAINS = {"gen", "api", "www", "buy", "admin", "app", "beta"}
|
||||
|
||||
|
||||
class TenantMiddleware(BaseHTTPMiddleware):
|
||||
@@ -67,8 +60,7 @@ class TenantMiddleware(BaseHTTPMiddleware):
|
||||
tenant_config: Optional[TenantConfig] = None
|
||||
if tenant_id:
|
||||
tenant_service = get_tenant_service()
|
||||
# get_tenant_config can hit GCS on a cache miss: keep it off the event loop.
|
||||
tenant_config = await asyncio.to_thread(tenant_service.get_tenant_config, tenant_id)
|
||||
tenant_config = tenant_service.get_tenant_config(tenant_id)
|
||||
|
||||
if tenant_config and not tenant_config.is_active:
|
||||
# Tenant exists but is inactive - treat as default
|
||||
|
||||
@@ -1,110 +0,0 @@
|
||||
"""
|
||||
Audio edit session model for persisting input audio editing progress.
|
||||
|
||||
Stored as a Firestore subcollection: jobs/{job_id}/audio_edit_sessions/{session_id}
|
||||
Full edit data is stored in GCS to avoid Firestore's 1MB document limit.
|
||||
"""
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional, Dict, Any, List
|
||||
import uuid
|
||||
|
||||
|
||||
@dataclass
|
||||
class AudioEditSessionSummary:
|
||||
"""Compact summary of operations in an audio edit session, for list views."""
|
||||
total_operations: int = 0
|
||||
operations_breakdown: Dict[str, int] = field(default_factory=dict)
|
||||
duration_change_seconds: float = 0.0
|
||||
net_duration_seconds: float = 0.0
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"total_operations": self.total_operations,
|
||||
"operations_breakdown": self.operations_breakdown,
|
||||
"duration_change_seconds": self.duration_change_seconds,
|
||||
"net_duration_seconds": self.net_duration_seconds,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> "AudioEditSessionSummary":
|
||||
return cls(
|
||||
total_operations=data.get("total_operations", 0),
|
||||
operations_breakdown=data.get("operations_breakdown", {}),
|
||||
duration_change_seconds=data.get("duration_change_seconds", 0.0),
|
||||
net_duration_seconds=data.get("net_duration_seconds", 0.0),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class AudioEditSession:
|
||||
"""
|
||||
A saved snapshot of an audio editing session.
|
||||
|
||||
Stored at: jobs/{job_id}/audio_edit_sessions/{session_id}
|
||||
Edit data stored at: jobs/{job_id}/audio_edit_sessions/{session_id}.json (GCS)
|
||||
"""
|
||||
session_id: str = field(default_factory=lambda: str(uuid.uuid4()))
|
||||
job_id: str = ""
|
||||
user_email: str = ""
|
||||
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
|
||||
updated_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
|
||||
edit_count: int = 0
|
||||
trigger: str = "auto" # "auto" | "manual" | "submit"
|
||||
audio_duration_seconds: Optional[float] = None
|
||||
original_duration_seconds: Optional[float] = None
|
||||
artist: Optional[str] = None
|
||||
title: Optional[str] = None
|
||||
summary: AudioEditSessionSummary = field(default_factory=AudioEditSessionSummary)
|
||||
edit_data_gcs_path: str = ""
|
||||
|
||||
# Hash of the edit data JSON, used to skip duplicate saves
|
||||
data_hash: str = ""
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"session_id": self.session_id,
|
||||
"job_id": self.job_id,
|
||||
"user_email": self.user_email,
|
||||
"created_at": self.created_at,
|
||||
"updated_at": self.updated_at,
|
||||
"edit_count": self.edit_count,
|
||||
"trigger": self.trigger,
|
||||
"audio_duration_seconds": self.audio_duration_seconds,
|
||||
"original_duration_seconds": self.original_duration_seconds,
|
||||
"artist": self.artist,
|
||||
"title": self.title,
|
||||
"summary": self.summary.to_dict(),
|
||||
"edit_data_gcs_path": self.edit_data_gcs_path,
|
||||
"data_hash": self.data_hash,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> "AudioEditSession":
|
||||
created_at = data.get("created_at")
|
||||
if isinstance(created_at, str):
|
||||
created_at = datetime.fromisoformat(created_at.rstrip("Z")).replace(tzinfo=timezone.utc)
|
||||
|
||||
updated_at = data.get("updated_at")
|
||||
if isinstance(updated_at, str):
|
||||
updated_at = datetime.fromisoformat(updated_at.rstrip("Z")).replace(tzinfo=timezone.utc)
|
||||
|
||||
summary_data = data.get("summary", {})
|
||||
summary = AudioEditSessionSummary.from_dict(summary_data) if isinstance(summary_data, dict) else AudioEditSessionSummary()
|
||||
|
||||
return cls(
|
||||
session_id=data.get("session_id", str(uuid.uuid4())),
|
||||
job_id=data.get("job_id", ""),
|
||||
user_email=data.get("user_email", ""),
|
||||
created_at=created_at or datetime.now(timezone.utc),
|
||||
updated_at=updated_at or datetime.now(timezone.utc),
|
||||
edit_count=data.get("edit_count", 0),
|
||||
trigger=data.get("trigger", "auto"),
|
||||
audio_duration_seconds=data.get("audio_duration_seconds"),
|
||||
original_duration_seconds=data.get("original_duration_seconds"),
|
||||
artist=data.get("artist"),
|
||||
title=data.get("title"),
|
||||
summary=summary,
|
||||
edit_data_gcs_path=data.get("edit_data_gcs_path", ""),
|
||||
data_hash=data.get("data_hash", ""),
|
||||
)
|
||||
+17
-217
@@ -16,9 +16,8 @@ class JobStatus(str, Enum):
|
||||
"""
|
||||
Job status enumeration - Complete state machine.
|
||||
|
||||
The workflow has 8 main stages with up to 3 human interaction points:
|
||||
The workflow has 7 main stages with 2 human interaction points:
|
||||
1. Input & Setup (may include audio source selection)
|
||||
1b. Optional Audio Editing (BLOCKING) - trim/cut/mute/join input audio
|
||||
2. Parallel Processing (audio + lyrics)
|
||||
3. Title/End Screen Generation + Backing Vocals Analysis
|
||||
4. Countdown Padding Synchronization
|
||||
@@ -37,18 +36,9 @@ class JobStatus(str, Enum):
|
||||
SEARCHING_AUDIO = "searching_audio" # Searching for audio sources via flacfetch
|
||||
AWAITING_AUDIO_SELECTION = "awaiting_audio_selection" # ⚠️ WAITING FOR USER - select audio source
|
||||
DOWNLOADING_AUDIO = "downloading_audio" # Downloading selected audio from source
|
||||
DOWNLOAD_PENDING_RETRY = "download_pending_retry" # Download hit a transient issue (few/no seeders, tracker blip); auto-retrying for up to 24h
|
||||
|
||||
DOWNLOADING = "downloading" # Downloading from URL or processing upload
|
||||
|
||||
# Optional: Audio editing (before processing begins)
|
||||
AWAITING_AUDIO_EDIT = "awaiting_audio_edit" # ⚠️ WAITING FOR USER - edit input audio
|
||||
IN_AUDIO_EDIT = "in_audio_edit" # User is actively editing audio
|
||||
AUDIO_EDIT_COMPLETE = "audio_edit_complete" # User submitted edited audio
|
||||
|
||||
# Optional: duration cost confirmation (BLOCKING) - confirm credits before heavy processing
|
||||
AWAITING_DURATION_CONFIRM = "awaiting_duration_confirm" # ⚠️ WAITING FOR USER - confirm duration-based cost
|
||||
|
||||
# Stage 2a: Audio separation (parallel track 1)
|
||||
SEPARATING_STAGE1 = "separating_stage1" # Clean instrumental separation (Modal API)
|
||||
SEPARATING_STAGE2 = "separating_stage2" # Backing vocals separation (Modal API)
|
||||
@@ -72,7 +62,6 @@ class JobStatus(str, Enum):
|
||||
|
||||
# Stage 5.5: Render video with corrected lyrics (post-review)
|
||||
RENDERING_VIDEO = "rendering_video" # Using OutputGenerator to create with_vocals.mkv
|
||||
RENDER_PENDING_CAPACITY = "render_pending_capacity" # Waiting for GCE encoding capacity (auto-retry)
|
||||
|
||||
# Stage 6: Instrumental already selected (during combined review)
|
||||
# Note: AWAITING_INSTRUMENTAL_SELECTION kept for DB compatibility with historical jobs only
|
||||
@@ -104,36 +93,16 @@ class JobStatus(str, Enum):
|
||||
|
||||
# Valid state transitions
|
||||
STATE_TRANSITIONS = {
|
||||
# PENDING can go to DOWNLOADING (file upload), SEARCHING_AUDIO (artist+title search),
|
||||
# or DOWNLOADING_AUDIO (create-from-search where audio is pre-selected)
|
||||
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
# PENDING can go to DOWNLOADING (file upload) or SEARCHING_AUDIO (artist+title search)
|
||||
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
|
||||
# Audio search flow (for artist+title search mode)
|
||||
JobStatus.SEARCHING_AUDIO: [JobStatus.AWAITING_AUDIO_SELECTION, JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED],
|
||||
JobStatus.AWAITING_AUDIO_SELECTION: [JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
JobStatus.DOWNLOADING_AUDIO: [JobStatus.DOWNLOADING, JobStatus.AWAITING_DURATION_CONFIRM, JobStatus.DOWNLOAD_PENDING_RETRY, JobStatus.FAILED],
|
||||
# DOWNLOAD_PENDING_RETRY = parked when a torrent download hits a transient
|
||||
# issue (few/no seeders, tracker outage); auto-retried by the recover-stuck
|
||||
# cron for up to 24h before failing permanently.
|
||||
JobStatus.DOWNLOAD_PENDING_RETRY: [JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
JobStatus.DOWNLOADING_AUDIO: [JobStatus.DOWNLOADING, JobStatus.FAILED],
|
||||
|
||||
# DOWNLOADING allows parallel processing (audio + lyrics) and then screens when both complete
|
||||
# Can also enter optional audio edit phase if requires_audio_edit flag is set
|
||||
JobStatus.DOWNLOADING: [JobStatus.SEPARATING_STAGE1, JobStatus.TRANSCRIBING, JobStatus.GENERATING_SCREENS, JobStatus.AWAITING_AUDIO_EDIT, JobStatus.AWAITING_DURATION_CONFIRM, JobStatus.FAILED],
|
||||
|
||||
# Optional audio edit phase (trim, cut, mute, join input audio before processing)
|
||||
JobStatus.AWAITING_AUDIO_EDIT: [JobStatus.IN_AUDIO_EDIT, JobStatus.AUDIO_EDIT_COMPLETE, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
JobStatus.IN_AUDIO_EDIT: [JobStatus.AWAITING_AUDIO_EDIT, JobStatus.AUDIO_EDIT_COMPLETE, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
JobStatus.AUDIO_EDIT_COMPLETE: [JobStatus.SEPARATING_STAGE1, JobStatus.TRANSCRIBING, JobStatus.GENERATING_SCREENS, JobStatus.AWAITING_DURATION_CONFIRM, JobStatus.FAILED],
|
||||
|
||||
# Duration cost confirmation (BLOCKING) - user confirms before heavy processing begins
|
||||
JobStatus.AWAITING_DURATION_CONFIRM: [
|
||||
# DOWNLOADING = confirmed, back to the prep phase (see jobs.confirm_duration).
|
||||
JobStatus.DOWNLOADING,
|
||||
JobStatus.SEPARATING_STAGE1, JobStatus.TRANSCRIBING,
|
||||
JobStatus.GENERATING_SCREENS, JobStatus.AWAITING_AUDIO_EDIT,
|
||||
JobStatus.FAILED, JobStatus.CANCELLED,
|
||||
],
|
||||
JobStatus.DOWNLOADING: [JobStatus.SEPARATING_STAGE1, JobStatus.TRANSCRIBING, JobStatus.GENERATING_SCREENS, JobStatus.FAILED],
|
||||
|
||||
# Audio separation flow
|
||||
JobStatus.SEPARATING_STAGE1: [JobStatus.SEPARATING_STAGE2, JobStatus.FAILED],
|
||||
@@ -146,37 +115,17 @@ STATE_TRANSITIONS = {
|
||||
JobStatus.LYRICS_COMPLETE: [JobStatus.GENERATING_SCREENS, JobStatus.FAILED],
|
||||
|
||||
# Post-parallel processing (screens + backing vocals analysis)
|
||||
# GENERATING_SCREENS -> REVIEW_COMPLETE is the auto-approval path: when the
|
||||
# scorer is fully confident the review screens are skipped entirely (no
|
||||
# review token / "ready for review" notification is ever issued).
|
||||
JobStatus.GENERATING_SCREENS: [JobStatus.APPLYING_PADDING, JobStatus.AWAITING_REVIEW, JobStatus.REVIEW_COMPLETE, JobStatus.FAILED],
|
||||
JobStatus.GENERATING_SCREENS: [JobStatus.APPLYING_PADDING, JobStatus.AWAITING_REVIEW, JobStatus.FAILED],
|
||||
JobStatus.APPLYING_PADDING: [JobStatus.AWAITING_REVIEW, JobStatus.FAILED],
|
||||
|
||||
# Combined human review flow (lyrics + instrumental selection)
|
||||
# AWAITING_REVIEW can go directly to REVIEW_COMPLETE (quick review) or to IN_REVIEW (editing)
|
||||
JobStatus.AWAITING_REVIEW: [JobStatus.IN_REVIEW, JobStatus.REVIEW_COMPLETE, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
JobStatus.IN_REVIEW: [JobStatus.REVIEW_COMPLETE, JobStatus.AWAITING_REVIEW, JobStatus.FAILED],
|
||||
JobStatus.REVIEW_COMPLETE: [
|
||||
JobStatus.RENDERING_VIDEO,
|
||||
JobStatus.PREP_COMPLETE, # for prep-only jobs
|
||||
JobStatus.RENDER_PENDING_CAPACITY, # GCE capacity exhausted before render started
|
||||
JobStatus.FAILED,
|
||||
],
|
||||
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.PREP_COMPLETE, JobStatus.FAILED], # PREP_COMPLETE for prep-only jobs
|
||||
|
||||
# Video rendering (post-review) - instrumental was already selected during combined review
|
||||
# RENDER_PENDING_CAPACITY = parked when GCE encoding capacity is exhausted; auto-retried by scheduler.
|
||||
JobStatus.RENDERING_VIDEO: [
|
||||
JobStatus.INSTRUMENTAL_SELECTED,
|
||||
JobStatus.PREP_COMPLETE,
|
||||
JobStatus.RENDER_PENDING_CAPACITY,
|
||||
JobStatus.FAILED,
|
||||
],
|
||||
JobStatus.RENDER_PENDING_CAPACITY: [
|
||||
JobStatus.RENDERING_VIDEO, # auto-retry succeeded, picked up again
|
||||
JobStatus.REVIEW_COMPLETE, # admin / scheduler resets to re-trigger render
|
||||
JobStatus.FAILED, # exceeded max wait or operator gave up
|
||||
JobStatus.CANCELLED, # user cancelled while waiting
|
||||
],
|
||||
JobStatus.RENDERING_VIDEO: [JobStatus.INSTRUMENTAL_SELECTED, JobStatus.PREP_COMPLETE, JobStatus.FAILED],
|
||||
|
||||
# AWAITING_INSTRUMENTAL_SELECTION is LEGACY - kept for historical jobs in DB
|
||||
# New jobs never enter this state; they go directly to INSTRUMENTAL_SELECTED after render
|
||||
@@ -192,24 +141,20 @@ STATE_TRANSITIONS = {
|
||||
JobStatus.UPLOADING: [JobStatus.NOTIFYING, JobStatus.COMPLETE, JobStatus.FAILED],
|
||||
JobStatus.NOTIFYING: [JobStatus.COMPLETE, JobStatus.FAILED],
|
||||
|
||||
# Terminal states - COMPLETE allows edit (reopen review) and visibility change
|
||||
# Terminal states - COMPLETE, PREP_COMPLETE have no transitions
|
||||
# FAILED and CANCELLED allow retry transitions to resume from checkpoints
|
||||
# PREP_COMPLETE allows continuation from combined review
|
||||
JobStatus.COMPLETE: [JobStatus.AWAITING_REVIEW, JobStatus.LYRICS_COMPLETE], # Edit track or visibility change
|
||||
JobStatus.COMPLETE: [],
|
||||
JobStatus.PREP_COMPLETE: [JobStatus.AWAITING_REVIEW, JobStatus.FAILED], # Continue from combined review
|
||||
JobStatus.FAILED: [
|
||||
JobStatus.DOWNLOADING, # Retry from beginning (if input audio exists)
|
||||
JobStatus.DOWNLOADING_AUDIO, # Retry audio download (audio search flow)
|
||||
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
|
||||
JobStatus.REVIEW_COMPLETE, # Retry from render stage
|
||||
JobStatus.LYRICS_COMPLETE, # Retry from screens generation
|
||||
JobStatus.AWAITING_REVIEW, # Retry from combined review
|
||||
JobStatus.RENDER_PENDING_CAPACITY, # Operator parks a stale-failed job for auto-retry
|
||||
JobStatus.AWAITING_AUDIO_SELECTION, # Choose different audio after a failed download
|
||||
],
|
||||
JobStatus.CANCELLED: [
|
||||
JobStatus.DOWNLOADING, # Retry from beginning (if input audio exists)
|
||||
JobStatus.DOWNLOADING_AUDIO, # Retry audio download (audio search flow)
|
||||
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
|
||||
JobStatus.REVIEW_COMPLETE, # Retry from render stage
|
||||
JobStatus.LYRICS_COMPLETE, # Retry from screens generation
|
||||
@@ -231,7 +176,6 @@ class TimelineEvent(BaseModel):
|
||||
timestamp: str
|
||||
progress: Optional[int] = None
|
||||
message: Optional[str] = None
|
||||
metadata: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
class LogEntry(BaseModel):
|
||||
@@ -261,14 +205,7 @@ class Job(BaseModel):
|
||||
title: Optional[str] = None
|
||||
filename: Optional[str] = None # Original uploaded filename
|
||||
input_media_gcs_path: Optional[str] = None # GCS path to uploaded file
|
||||
|
||||
# Audio source tracking (for retry capability)
|
||||
audio_source_type: Optional[str] = None # Source type: 'audio_search', 'youtube_url', 'file_upload'
|
||||
source_name: Optional[str] = None # Provider name (RED, OPS, YouTube, Spotify) - for audio_search
|
||||
source_id: Optional[str] = None # Source-specific ID (torrent ID, video ID, track ID) - for audio_search
|
||||
target_file: Optional[str] = None # For torrents, specific file to extract - for audio_search
|
||||
download_url: Optional[str] = None # Direct download URL if available - for audio_search
|
||||
|
||||
|
||||
# User preferences
|
||||
enable_cdg: bool = False # Generate CDG+MP3 package (requires style config)
|
||||
enable_txt: bool = False # Generate TXT+MP3 package (requires style config)
|
||||
@@ -278,39 +215,11 @@ class Job(BaseModel):
|
||||
user_email: Optional[str] = None # Email for notifications
|
||||
non_interactive: bool = False # Skip interactive steps (lyrics review, instrumental selection)
|
||||
|
||||
# Review autonomy level. "auto" (default): skip the combined review screens
|
||||
# whenever the auto-approval scorer is fully confident, falling back to human
|
||||
# review otherwise. "always_review": every job stops at the review screens.
|
||||
# (Distinct from legacy non_interactive, which is CLI packaging/upload
|
||||
# semantics and never gated the cloud review flow.)
|
||||
review_mode: str = "auto"
|
||||
|
||||
# Backing-vocals preference chosen up-front at submission. "auto" (default):
|
||||
# retain backing vocals wherever the 3-stem decider is confident it's safe,
|
||||
# else fall through to human review. "clean": always use the clean
|
||||
# instrumental (never retain backing). "review": always let a human pick the
|
||||
# instrumental. Consumed by the auto-approval instrumental decision.
|
||||
backing_preference: str = "auto"
|
||||
|
||||
# Translated lyrics: language code (one of the UI locales, e.g. "en", "es") whose
|
||||
# LLM translation of each reviewed line is drawn in smaller text beneath it in the
|
||||
# karaoke videos. None = no translation row. See backend/services/lyrics_translation.py.
|
||||
translation_language: Optional[str] = None
|
||||
|
||||
# Multi-tenant support ("" = consumer portal, "vocalstar"/etc = tenant portal)
|
||||
tenant_id: str = "" # Tenant ID for white-label portal scoping
|
||||
|
||||
# UI language the job was submitted in (primary Accept-Language subtag, e.g.
|
||||
# "pt", "ja", "zh"). Any of the 33 supported UI locales — NOT narrowed to the
|
||||
# 3 email locales. Admin-only signal for support/intervention. None for jobs
|
||||
# created before this field existed or where the header was absent.
|
||||
locale: Optional[str] = None
|
||||
|
||||
# Multi-tenant support (None = default Nomad Karaoke)
|
||||
tenant_id: Optional[str] = None # Tenant ID for white-label portal scoping
|
||||
|
||||
# Theme configuration (pre-made themes from GCS)
|
||||
theme_id: Optional[str] = None # Theme identifier (e.g., "nomad", "default")
|
||||
# When the theme was last re-snapshotted onto the job after creation (theme
|
||||
# edit refresh, re-render, Edit). None = the creation-time snapshot (created_at).
|
||||
theme_applied_at: Optional[datetime] = None
|
||||
color_overrides: Dict[str, str] = Field(default_factory=dict)
|
||||
"""
|
||||
User color overrides applied on top of theme. Keys:
|
||||
@@ -383,38 +292,10 @@ class Job(BaseModel):
|
||||
customer_email: Optional[str] = None # Customer email for final delivery (job owned by admin during processing)
|
||||
customer_notes: Optional[str] = None # Notes provided by customer with their order
|
||||
|
||||
# Private (non-published) track mode
|
||||
is_private: bool = False # Private tracks: Dropbox only (Tracks-NonPublished/NOMADNP), no YouTube/GDrive
|
||||
|
||||
# Creation-time decision params (for observability — what did the user actually choose?)
|
||||
creation_params: Dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
# Anti-abuse: IP address at job creation time (for cross-account correlation)
|
||||
creation_ip: Optional[str] = None
|
||||
|
||||
# Output deletion tracking (for admin cleanup without deleting job)
|
||||
outputs_deleted_at: Optional[datetime] = None # Timestamp when outputs were deleted by admin
|
||||
outputs_deleted_by: Optional[str] = None # Admin email who deleted outputs
|
||||
|
||||
# Storage retention (backend/services/storage_retention.py): set when the
|
||||
# daily purge deleted this job's regenerable renders / separated stems.
|
||||
# Cleared when they're rebuilt (regenerate / re-render / stems restore).
|
||||
renders_purged_at: Optional[datetime] = None
|
||||
stems_purged_at: Optional[datetime] = None
|
||||
renders_regenerated_at: Optional[datetime] = None # last GCS-only regenerate
|
||||
storage_purge: Optional[Dict[str, Any]] = None # manifest of the last purge
|
||||
|
||||
# Credit refund tracking (prevents double-refund on cancel→delete or fail→delete)
|
||||
credit_refunded: bool = False
|
||||
|
||||
# Edit tracking (how many times a completed track has been re-edited)
|
||||
edit_count: int = 0
|
||||
|
||||
# Whole-track tempo change applied in the audio editor (1.0 / None = original
|
||||
# speed). When set, job.title carries a "(90% Tempo)" label so every published
|
||||
# output says so; see backend/services/tempo_label.py.
|
||||
tempo_factor: Optional[float] = None
|
||||
|
||||
# Processing state
|
||||
track_output_dir: Optional[str] = None # Local output directory (temp)
|
||||
audio_hash: Optional[str] = None # Hash for deduplication
|
||||
@@ -505,43 +386,6 @@ class Job(BaseModel):
|
||||
}
|
||||
"""
|
||||
|
||||
# Processing metadata (immutable provenance data written by workers during processing)
|
||||
processing_metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
"""
|
||||
Immutable processing provenance data. Written once by each worker during processing.
|
||||
Never modified after initial write. Used for analytics, debugging, and data export.
|
||||
|
||||
Structure:
|
||||
{
|
||||
"transcription": {
|
||||
"provider": "audioshake",
|
||||
"audioshake_task_id": "...",
|
||||
"audioshake_asset_id": "...",
|
||||
"language_detected": "en",
|
||||
"word_count": 358,
|
||||
"segment_count": 42,
|
||||
},
|
||||
"correction": {
|
||||
"handlers_applied": [...],
|
||||
"corrections_made": 12,
|
||||
"correction_ratio": 0.034,
|
||||
"reference_sources_found": ["genius", "spotify"],
|
||||
"agentic_routing": "rule-based",
|
||||
"duration_seconds": 33.2,
|
||||
},
|
||||
"separation": {
|
||||
"provider": "modal",
|
||||
"clean_model": "...",
|
||||
"duration_seconds": 420.5,
|
||||
},
|
||||
"timing": {
|
||||
"lyrics_worker_seconds": 125.3,
|
||||
"audio_worker_seconds": 420.5,
|
||||
...
|
||||
},
|
||||
}
|
||||
"""
|
||||
|
||||
# Request metadata (captured at job creation for tracking and filtering)
|
||||
request_metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
"""
|
||||
@@ -598,15 +442,7 @@ class JobCreate(BaseModel):
|
||||
webhook_url: Optional[str] = None
|
||||
user_email: Optional[str] = None
|
||||
non_interactive: bool = False # Skip interactive steps (lyrics review, instrumental selection)
|
||||
# "auto" (default) = skip review screens when the auto-approval scorer is
|
||||
# fully confident; "always_review" = always stop at the review screens.
|
||||
review_mode: str = "auto"
|
||||
# "auto" (default) = retain backing vocals where confidently safe; "clean" =
|
||||
# always strip backing vocals; "review" = always let a human pick the instrumental.
|
||||
backing_preference: str = "auto"
|
||||
# Translated lyrics language (see Job.translation_language); None = off
|
||||
translation_language: Optional[str] = None
|
||||
|
||||
|
||||
# Theme configuration (pre-made themes from GCS)
|
||||
theme_id: Optional[str] = None # Theme identifier (e.g., "nomad", "default")
|
||||
color_overrides: Dict[str, str] = Field(default_factory=dict)
|
||||
@@ -663,14 +499,6 @@ class JobCreate(BaseModel):
|
||||
customer_email: Optional[str] = None # Customer email for final delivery (job owned by admin during processing)
|
||||
customer_notes: Optional[str] = None # Notes provided by customer with their order
|
||||
|
||||
# Private (non-published) track mode
|
||||
is_private: bool = False # Private tracks: Dropbox only (Tracks-NonPublished/NOMADNP), no YouTube/GDrive
|
||||
|
||||
# Credit pricing: how many credits to charge for this job at creation time.
|
||||
# Search and upload jobs pass the default (1) and may top up later.
|
||||
# URL jobs with a known duration pass ceil(duration/600) credits.
|
||||
credits: int = 1
|
||||
|
||||
# Request metadata (set by API endpoint from request headers)
|
||||
request_metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
"""
|
||||
@@ -683,12 +511,8 @@ class JobCreate(BaseModel):
|
||||
- custom_headers: All X-* headers
|
||||
"""
|
||||
|
||||
# Tenant scoping for white-label portals ("" = consumer, "vocalstar"/etc = tenant)
|
||||
tenant_id: str = "" # Tenant ID for job scoping
|
||||
|
||||
# UI language the job was submitted in (primary Accept-Language subtag, e.g.
|
||||
# "pt", "ja"). Set by the API endpoint from the request header; admin-only.
|
||||
locale: Optional[str] = None
|
||||
# Tenant scoping for white-label portals
|
||||
tenant_id: Optional[str] = None # Tenant ID for job scoping
|
||||
|
||||
@validator('url')
|
||||
def validate_url(cls, v):
|
||||
@@ -697,30 +521,6 @@ class JobCreate(BaseModel):
|
||||
raise ValueError("Field cannot be empty string")
|
||||
return v.strip() if isinstance(v, str) else v
|
||||
|
||||
@validator('review_mode')
|
||||
def _normalize_review_mode(cls, v):
|
||||
"""Coerce an out-of-set value to the fail-safe default (always_review =
|
||||
more human oversight), so a typo can never silently skip review. The
|
||||
executor already treats any non-"auto" value as an enforcement blocker."""
|
||||
return v if v in ("auto", "always_review") else "always_review"
|
||||
|
||||
@validator('translation_language')
|
||||
def _normalize_translation_language(cls, v):
|
||||
"""Supported language code; empty or unsupported values turn the feature off
|
||||
(every creation route passes the client value straight through)."""
|
||||
if v is None or (isinstance(v, str) and not v.strip()):
|
||||
return None
|
||||
from backend.services.lyrics_translation import normalize_language
|
||||
|
||||
return normalize_language(v)
|
||||
|
||||
@validator('backing_preference')
|
||||
def _normalize_backing_preference(cls, v):
|
||||
"""Coerce an out-of-set value to the default. Every creation route passes
|
||||
the client value straight through, so this is the single choke point that
|
||||
keeps a typo from reaching the instrumental decision as an unknown value."""
|
||||
return v if v in ("auto", "clean", "review") else "auto"
|
||||
|
||||
@validator('artist', 'title')
|
||||
def normalize_artist_title(cls, v):
|
||||
"""Normalize artist/title text to standardize Unicode characters.
|
||||
|
||||
Loaded 100 of 2186 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user