mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 21:17:56 +09:00
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2614a2c14c | ||
|
|
80115b7e34 |
No files matched your search
@@ -1,118 +0,0 @@
|
||||
---
|
||||
description: Review and address CodeRabbit comments on current PR
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write, Bash, mcp__github__pull_request_read, mcp__github__get_me, mcp__github__list_pull_requests, mcp__github__add_issue_comment
|
||||
---
|
||||
|
||||
# CodeRabbit Comment Review
|
||||
|
||||
You are reviewing the current PR to address all CodeRabbit automated review comments.
|
||||
|
||||
## Your Task
|
||||
|
||||
### 1. Identify the Current PR
|
||||
|
||||
First, determine the current branch and find the associated PR:
|
||||
```bash
|
||||
git branch --show-current
|
||||
```
|
||||
|
||||
Then use GitHub tools to find the PR for this branch in the `nomadkaraoke/karaoke-gen` repository.
|
||||
|
||||
### 2. Fetch CodeRabbit Comments
|
||||
|
||||
Get all reviews, review comments, and regular comments on the PR:
|
||||
|
||||
**PR Reviews (MOST IMPORTANT - Contains Nitpicks and Issues)**
|
||||
- Use `pull_request_read` with `method: "get_reviews"` to get all PR reviews
|
||||
- Look for reviews from the `coderabbitai[bot]` user
|
||||
- The review `body` field contains the detailed analysis with:
|
||||
- "Nitpick comments" section with specific code suggestions
|
||||
- "Potential issues" flagged in the diff
|
||||
- Code improvement suggestions with file paths and line numbers
|
||||
- **This is the primary source of CodeRabbit feedback** - parse this carefully
|
||||
|
||||
**Review Comments (Inline Code Comments)**
|
||||
- Use `pull_request_read` with `method: "get_review_comments"` to get inline code comments
|
||||
- These are comments attached to specific lines in the diff
|
||||
- Also check for `coderabbitai[bot]` comments here
|
||||
|
||||
**PR Comments (Summary)**
|
||||
- Use `pull_request_read` with `method: "get_comments"` to get regular PR comments
|
||||
- CodeRabbit posts a walkthrough/summary comment here
|
||||
- Less actionable than the review body, but may contain additional context
|
||||
|
||||
### 3. Analyze Each Comment
|
||||
|
||||
For each CodeRabbit comment:
|
||||
1. Read the comment carefully to understand the concern
|
||||
2. Look at the referenced code location
|
||||
3. Decide if a code change is warranted:
|
||||
- **Make changes** for: valid bugs, security issues, clear improvements, style violations
|
||||
- **Skip changes** for: subjective preferences, false positives, already addressed, not applicable
|
||||
4. If skipping, note the reason
|
||||
|
||||
### 4. Make Code Changes
|
||||
|
||||
For each comment requiring action:
|
||||
1. Navigate to the file and line mentioned
|
||||
2. Understand the surrounding context
|
||||
3. Make the minimal change needed to address the concern
|
||||
4. Verify the change doesn't break anything obvious
|
||||
|
||||
### 5. Commit and Push
|
||||
|
||||
After making all changes:
|
||||
```bash
|
||||
# Stage all changes
|
||||
git add -A
|
||||
|
||||
# Commit with descriptive message
|
||||
git commit -m "fix: address CodeRabbit review comments
|
||||
|
||||
- [list each fix briefly]
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
|
||||
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
|
||||
|
||||
# Push to the PR branch
|
||||
git push
|
||||
```
|
||||
|
||||
### 6. Resolve Threads
|
||||
|
||||
After pushing:
|
||||
- Reply to each CodeRabbit review thread indicating what was done
|
||||
- If a comment was addressed, reply with what was changed
|
||||
- If a comment was intentionally skipped, explain why
|
||||
- This helps mark threads as resolved so the PR can be merged
|
||||
|
||||
## Output Format
|
||||
|
||||
Provide a summary report:
|
||||
|
||||
```
|
||||
## CodeRabbit Review Summary
|
||||
|
||||
### Comments Addressed
|
||||
- [ ] File:line - Brief description of issue → What was changed
|
||||
|
||||
### Comments Skipped (with reason)
|
||||
- [ ] File:line - Issue → Reason for skipping
|
||||
|
||||
### Actions Taken
|
||||
- Committed: [commit hash]
|
||||
- Pushed to: [branch name]
|
||||
- Threads replied to: [count]
|
||||
|
||||
### Status
|
||||
Ready for merge: Yes/No (if No, explain why)
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
|
||||
- Focus on code quality improvements, not just making CodeRabbit happy
|
||||
- Don't blindly accept every suggestion - use judgment
|
||||
- Group related fixes into logical commits if needed
|
||||
- Keep changes minimal and focused on the specific issues raised
|
||||
- If a suggestion would require significant refactoring, note it for future work instead
|
||||
@@ -1,74 +0,0 @@
|
||||
---
|
||||
description: Periodic documentation maintenance and organization check
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write, Bash
|
||||
---
|
||||
|
||||
# Documentation Maintenance Check
|
||||
|
||||
You are performing periodic maintenance on the project documentation.
|
||||
|
||||
## Checks to Perform
|
||||
|
||||
### 1. Structure Compliance
|
||||
Verify docs/ follows the correct structure:
|
||||
```
|
||||
docs/
|
||||
├── README.md # Current status + navigation
|
||||
├── ARCHITECTURE.md # System design
|
||||
├── DEVELOPMENT.md # Dev setup, testing, deployment
|
||||
├── API.md # Backend API reference
|
||||
├── LESSONS-LEARNED.md # Accumulated wisdom
|
||||
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
|
||||
```
|
||||
|
||||
Report any files that don't belong or are misplaced.
|
||||
|
||||
### 2. Freshness Check
|
||||
- Read `docs/README.md` - is the status section current?
|
||||
- Check recent git commits - do they suggest docs should be updated?
|
||||
- Look for stale information that contradicts current code
|
||||
|
||||
### 3. Archive Organization
|
||||
- Verify all archive files have YYYY-MM-DD prefix
|
||||
- Check for any docs that should be archived (completed work, old plans)
|
||||
- Remove truly obsolete content (empty files, duplicate info)
|
||||
|
||||
### 4. Cross-Reference Check
|
||||
- Verify CLAUDE.md points to current doc locations
|
||||
- Check that docs reference each other correctly
|
||||
- Ensure no broken internal links
|
||||
|
||||
### 5. Content Quality
|
||||
- Flag any docs over 500 lines (may need splitting)
|
||||
- Identify duplicate information across docs
|
||||
- Note sections that seem outdated
|
||||
|
||||
## Actions to Take
|
||||
|
||||
1. **Fix structural issues** - Move/rename misplaced files
|
||||
2. **Update stale content** - Refresh outdated information
|
||||
3. **Archive completed work** - Move old docs to archive/
|
||||
4. **Report findings** - Summarize what was found and fixed
|
||||
|
||||
## Output Format
|
||||
|
||||
Provide a maintenance report:
|
||||
```
|
||||
## Documentation Maintenance Report - YYYY-MM-DD
|
||||
|
||||
### Issues Found
|
||||
- [list of issues]
|
||||
|
||||
### Actions Taken
|
||||
- [list of fixes made]
|
||||
|
||||
### Recommendations
|
||||
- [suggestions for future attention]
|
||||
|
||||
### Status
|
||||
- [ ] Structure: OK/Fixed
|
||||
- [ ] Freshness: OK/Updated
|
||||
- [ ] Archive: OK/Organized
|
||||
- [ ] Cross-refs: OK/Fixed
|
||||
- [ ] Quality: OK/Flagged
|
||||
```
|
||||
@@ -1,44 +0,0 @@
|
||||
---
|
||||
description: Review current session for documentation updates before merging PR
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write
|
||||
---
|
||||
|
||||
# Pre-Merge Documentation Review
|
||||
|
||||
You are reviewing this chat session to identify documentation updates needed before merging the current PR.
|
||||
|
||||
## Your Task
|
||||
|
||||
1. **Analyze the current session** - What work was done? What was learned? What changed?
|
||||
|
||||
2. **Check if docs need updates:**
|
||||
- Does `docs/README.md` status section need updating?
|
||||
- Should anything be added to `docs/LESSONS-LEARNED.md`?
|
||||
- Does `docs/ARCHITECTURE.md` need changes (if architecture changed)?
|
||||
- Does `docs/DEVELOPMENT.md` need changes (if dev workflow changed)?
|
||||
- Does `docs/API.md` need changes (if API changed)?
|
||||
|
||||
3. **For significant completed work:**
|
||||
- Create `docs/archive/YYYY-MM-DD-topic.md` summarizing the work
|
||||
- Use today's date and a descriptive topic name
|
||||
|
||||
4. **Update docs/README.md** with:
|
||||
- Current status if it changed
|
||||
- Any new known issues discovered
|
||||
- Link to new archive doc if created
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **IMPORTANT**: Make all documentation changes on the current feature branch (git worktree), NOT on main. Check `git branch` or the worktree path to ensure you're editing files in the correct location.
|
||||
- Keep updates concise and factual
|
||||
- Focus on information useful for future AI agents
|
||||
- Don't duplicate information that's in the code
|
||||
- Archive docs should capture "what was done and why" not implementation details
|
||||
|
||||
## Output
|
||||
|
||||
After reviewing, either:
|
||||
- Make the necessary documentation updates
|
||||
- Report "No documentation updates needed" with brief explanation why
|
||||
|
||||
Remember: Good documentation helps the next agent continue where you left off.
|
||||
@@ -1,86 +0,0 @@
|
||||
---
|
||||
description: Maintain git worktrees - identify merged PRs for cleanup and rename generic branches
|
||||
allowed-tools: Read, Glob, Grep, Bash, mcp__github__list_pull_requests, mcp__github__pull_request_read, mcp__github__get_me, mcp__github__search_pull_requests, AskUserQuestion
|
||||
---
|
||||
|
||||
# Git Worktree Maintenance
|
||||
|
||||
You are performing maintenance on the project's git worktrees.
|
||||
|
||||
## Process
|
||||
|
||||
### 1. List All Worktrees
|
||||
Run `git worktree list` to see all current worktrees.
|
||||
|
||||
### 2. Get PR Status for Each Branch
|
||||
For each worktree (excluding main), check the PR status:
|
||||
- Use `gh pr list --state all --limit 50 --json number,title,headRefName,state,mergedAt` to get PR info
|
||||
- Match each worktree branch to its PR
|
||||
- Identify: MERGED (can cleanup), OPEN (keep), NO PR (investigate)
|
||||
|
||||
### 3. Categorize Worktrees
|
||||
|
||||
**Can be cleaned up (PRs merged):**
|
||||
- List worktrees where the associated PR has been merged
|
||||
- These are safe to remove with `git worktree remove <path>`
|
||||
|
||||
**Outstanding/WIP (keep):**
|
||||
- List worktrees with open PRs
|
||||
- Check if the worktree name is descriptive or generic (e.g., "work-20251230-151202", "session-YYYY-MM-DD-HHMM")
|
||||
- For generic names, read the PR title/description to suggest a better name
|
||||
|
||||
**No PR found:**
|
||||
- Flag these for user attention - may be abandoned work or local-only branches
|
||||
|
||||
### 4. Present Findings
|
||||
|
||||
Show a summary table:
|
||||
```
|
||||
## Can be cleaned up (merged):
|
||||
| Worktree | Branch | PR # |
|
||||
|----------|--------|------|
|
||||
|
||||
## Outstanding PRs (keep):
|
||||
| Worktree | Branch | PR # | Suggested Rename |
|
||||
|----------|--------|------|------------------|
|
||||
|
||||
## No PR found:
|
||||
| Worktree | Branch | Notes |
|
||||
|----------|--------|-------|
|
||||
```
|
||||
|
||||
### 5. Ask User for Action
|
||||
|
||||
Use AskUserQuestion to ask what to do:
|
||||
- Clean up all merged worktrees?
|
||||
- Rename generic worktree names to descriptive ones?
|
||||
- Investigate worktrees with no PR?
|
||||
|
||||
### 6. Execute Cleanup
|
||||
|
||||
If user approves:
|
||||
|
||||
**Removing merged worktrees:**
|
||||
```bash
|
||||
git worktree remove <path>
|
||||
```
|
||||
If there are uncommitted changes, check if they're already in main before force-removing.
|
||||
|
||||
**Renaming worktrees:**
|
||||
```bash
|
||||
mv <old-path> <new-path>
|
||||
git worktree repair <new-path>
|
||||
git worktree prune
|
||||
```
|
||||
|
||||
### 7. Final Verification
|
||||
|
||||
Run `git worktree list` again to confirm the final state.
|
||||
|
||||
## Important Notes
|
||||
|
||||
- Never remove the main worktree
|
||||
- Always check for uncommitted changes before removing
|
||||
- If a worktree has changes not in main, offer to create a PR first
|
||||
- Generic names to watch for: `work-YYYYMMDD-*`, `session-YYYY-MM-DD-*`, `feature/work-*`
|
||||
- Good descriptive names should reflect the PR content (e.g., `log-timing-analyzer`, `e2e-completion`)
|
||||
@@ -1,83 +0,0 @@
|
||||
# .dockerignore - Reduce Docker build context size
|
||||
# This is critical for fast builds as it reduces upload time and cache invalidation
|
||||
|
||||
# Git
|
||||
.git/
|
||||
.gitignore
|
||||
.gitmodules
|
||||
|
||||
# Node.js / Frontend (not needed for backend image)
|
||||
node_modules/
|
||||
frontend/
|
||||
frontend-react/
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
|
||||
# Python virtual environments (use Docker's Python instead)
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
.venv/
|
||||
venv/
|
||||
backend/venv/
|
||||
ENV/
|
||||
env/
|
||||
*.egg-info/
|
||||
.eggs/
|
||||
dist/
|
||||
build/
|
||||
*.egg
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
|
||||
# Testing (not needed in production image)
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
coverage.xml
|
||||
*.cover
|
||||
tests/
|
||||
backend/tests/
|
||||
|
||||
# Documentation (not needed in Docker image)
|
||||
docs/
|
||||
*.md
|
||||
!README.md
|
||||
|
||||
# Infrastructure (not needed for Docker build)
|
||||
infrastructure/
|
||||
|
||||
# CI/CD configs
|
||||
.github/
|
||||
|
||||
# Cloud Build configs (keep for reference but not in image)
|
||||
cloudbuild*.yaml
|
||||
|
||||
# Local development files
|
||||
lyrics_transcriber_local/
|
||||
*.local
|
||||
.env
|
||||
.env.*
|
||||
*.bak
|
||||
*.tmp
|
||||
Thumbs.db
|
||||
|
||||
# Large test/sample files
|
||||
tests/data/
|
||||
*.mp3
|
||||
*.wav
|
||||
*.mp4
|
||||
*.mkv
|
||||
*.flac
|
||||
+171
-446
@@ -17,195 +17,12 @@ concurrency:
|
||||
|
||||
jobs:
|
||||
# ============================================
|
||||
# PATH FILTERING - Determine what changed
|
||||
# Only needed for PRs (main branch skips tests)
|
||||
# ============================================
|
||||
|
||||
changes:
|
||||
name: "Detect Changes"
|
||||
runs-on: ubuntu-latest
|
||||
if: github.ref != 'refs/heads/main'
|
||||
outputs:
|
||||
frontend: ${{ steps.filter.outputs.frontend }}
|
||||
backend: ${{ steps.filter.outputs.backend }}
|
||||
package: ${{ steps.filter.outputs.package }}
|
||||
infra: ${{ steps.filter.outputs.infra }}
|
||||
deps: ${{ steps.filter.outputs.deps }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: dorny/paths-filter@v3
|
||||
id: filter
|
||||
with:
|
||||
filters: |
|
||||
frontend:
|
||||
- 'frontend/**'
|
||||
backend:
|
||||
- 'backend/**'
|
||||
package:
|
||||
- 'karaoke_gen/**'
|
||||
- 'lyrics_transcriber_temp/**'
|
||||
- 'tests/**'
|
||||
infra:
|
||||
- 'infrastructure/**'
|
||||
deps:
|
||||
- 'poetry.lock'
|
||||
|
||||
- name: Summary
|
||||
run: |
|
||||
echo "### Change Detection Results" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Area | Changed |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "|------|---------|" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Frontend | ${{ steps.filter.outputs.frontend }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Backend | ${{ steps.filter.outputs.backend }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Package | ${{ steps.filter.outputs.package }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Infrastructure | ${{ steps.filter.outputs.infra }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Dependencies | ${{ steps.filter.outputs.deps }} |" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
# ============================================
|
||||
# FRONTEND TESTS (PRs only - main skips tests)
|
||||
# ============================================
|
||||
|
||||
frontend-unit-tests:
|
||||
name: "Frontend - Unit Tests"
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Run unit tests with coverage
|
||||
working-directory: frontend
|
||||
run: npm run test:ci
|
||||
|
||||
- name: Upload coverage report
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: frontend-coverage
|
||||
path: frontend/coverage/
|
||||
retention-days: 7
|
||||
|
||||
frontend-e2e-smoke:
|
||||
name: "Frontend - E2E Smoke (~2 min)"
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
# Quick smoke tests run on every PR with frontend changes
|
||||
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Run smoke tests only
|
||||
working-directory: frontend
|
||||
run: npx playwright test smoke.spec.ts --project=chromium --reporter=line
|
||||
|
||||
- name: Upload Playwright report
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: playwright-smoke-report
|
||||
path: frontend/playwright-report/
|
||||
retention-days: 7
|
||||
|
||||
frontend-e2e-full:
|
||||
name: "Frontend - E2E Full"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
# Full E2E suite only runs when PR has 'e2e' label (on-demand)
|
||||
if: >-
|
||||
github.ref != 'refs/heads/main' &&
|
||||
needs.changes.outputs.frontend == 'true' &&
|
||||
contains(github.event.pull_request.labels.*.name, 'e2e')
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Run full E2E tests
|
||||
working-directory: frontend
|
||||
run: npx playwright test --reporter=list -x
|
||||
|
||||
- name: Upload Playwright report
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: playwright-report
|
||||
path: frontend/playwright-report/
|
||||
retention-days: 7
|
||||
|
||||
frontend-build:
|
||||
name: "Frontend - Build"
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Build frontend
|
||||
working-directory: frontend
|
||||
env:
|
||||
NEXT_PUBLIC_API_URL: https://api.nomadkaraoke.com
|
||||
run: npm run build
|
||||
|
||||
# ============================================
|
||||
# PACKAGE TESTS (PRs only - main skips tests)
|
||||
# PACKAGE TESTS (karaoke-gen Python CLI)
|
||||
# ============================================
|
||||
|
||||
package-unit-tests:
|
||||
name: "Package - Unit Tests (70% coverage)"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -222,14 +39,6 @@ jobs:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-package-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install --with dev
|
||||
|
||||
@@ -238,12 +47,6 @@ jobs:
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y ffmpeg
|
||||
|
||||
- name: Cache spaCy model
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/spacy
|
||||
key: spacy-en_core_web_sm-${{ runner.os }}
|
||||
|
||||
- name: Install spaCy language models
|
||||
run: poetry run python -m spacy download en_core_web_sm
|
||||
|
||||
@@ -261,9 +64,7 @@ jobs:
|
||||
|
||||
package-integration-tests:
|
||||
name: "Package - Integration Tests"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -280,14 +81,6 @@ jobs:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-package-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install --with dev
|
||||
|
||||
@@ -296,27 +89,16 @@ jobs:
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y ffmpeg
|
||||
|
||||
- name: Cache spaCy model
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/spacy
|
||||
key: spacy-en_core_web_sm-${{ runner.os }}
|
||||
|
||||
- name: Install spaCy language models
|
||||
run: poetry run python -m spacy download en_core_web_sm
|
||||
|
||||
- name: Run integration tests
|
||||
run: poetry run pytest tests/integration/ -v
|
||||
|
||||
package-build-test:
|
||||
name: "Package - Build & Install (Test)"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: [changes, package-unit-tests, package-integration-tests]
|
||||
if: >-
|
||||
github.ref != 'refs/heads/main' && always() &&
|
||||
(needs.package-unit-tests.result == 'success' || needs.package-unit-tests.result == 'skipped') &&
|
||||
(needs.package-integration-tests.result == 'success' || needs.package-integration-tests.result == 'skipped') &&
|
||||
(needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
package-build:
|
||||
name: "Package - Build & Install"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [package-unit-tests, package-integration-tests]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -336,20 +118,36 @@ jobs:
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
|
||||
- name: Free disk space
|
||||
uses: jlumbroso/free-disk-space@main
|
||||
with:
|
||||
tool-cache: false
|
||||
android: true
|
||||
haskell: true
|
||||
dotnet: false
|
||||
large-packages: false
|
||||
docker-images: false
|
||||
swap-storage: false
|
||||
|
||||
- name: Test package installation
|
||||
run: |
|
||||
pip install dist/*.whl
|
||||
python -m karaoke_gen.utils.gen_cli --help
|
||||
|
||||
- name: Upload build artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: package-dist
|
||||
path: dist/
|
||||
retention-days: 1
|
||||
|
||||
# ============================================
|
||||
# BACKEND TESTS (PRs only - main skips tests)
|
||||
# BACKEND TESTS (Cloud Run API service)
|
||||
# ============================================
|
||||
|
||||
backend-unit-tests:
|
||||
name: "Backend - Unit Tests"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
@@ -384,10 +182,15 @@ jobs:
|
||||
|
||||
backend-emulator-tests:
|
||||
name: "Backend - Emulator Integration Tests"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /opt/ghc
|
||||
sudo rm -rf /usr/local/share/boost
|
||||
df -h
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -410,6 +213,14 @@ jobs:
|
||||
distribution: "temurin"
|
||||
java-version: "21"
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Install Firestore emulator
|
||||
run: gcloud components install beta cloud-firestore-emulator --quiet
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
@@ -431,9 +242,7 @@ jobs:
|
||||
|
||||
backend-lint:
|
||||
name: "Backend - Code Quality"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
@@ -451,14 +260,6 @@ jobs:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-lint-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-lint-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
@@ -467,82 +268,10 @@ jobs:
|
||||
poetry run python -m py_compile backend/main.py backend/config.py
|
||||
echo "✅ Basic syntax validation passed"
|
||||
|
||||
# ============================================
|
||||
# CI GATE - Required status check for PRs
|
||||
# Only runs on PRs, not on main
|
||||
# ============================================
|
||||
|
||||
ci-gate:
|
||||
name: "CI Gate"
|
||||
runs-on: ubuntu-latest
|
||||
# Note: frontend-e2e-full is optional (requires 'e2e' label) so not included here
|
||||
# frontend-e2e-smoke IS required - fast smoke tests catch critical regressions
|
||||
needs: [frontend-unit-tests, frontend-e2e-smoke, frontend-build, package-unit-tests, package-integration-tests, backend-unit-tests, backend-emulator-tests, backend-lint]
|
||||
if: github.ref != 'refs/heads/main' && always()
|
||||
steps:
|
||||
- name: Check results
|
||||
run: |
|
||||
results=(
|
||||
"${{ needs.frontend-unit-tests.result }}"
|
||||
"${{ needs.frontend-e2e-smoke.result }}"
|
||||
"${{ needs.frontend-build.result }}"
|
||||
"${{ needs.package-unit-tests.result }}"
|
||||
"${{ needs.package-integration-tests.result }}"
|
||||
"${{ needs.backend-unit-tests.result }}"
|
||||
"${{ needs.backend-emulator-tests.result }}"
|
||||
"${{ needs.backend-lint.result }}"
|
||||
)
|
||||
for result in "${results[@]}"; do
|
||||
if [[ "$result" == "failure" || "$result" == "cancelled" ]]; then
|
||||
echo "❌ CI failed"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
echo "✅ CI passed"
|
||||
|
||||
# ============================================
|
||||
# DEPLOYMENTS (main branch only)
|
||||
# Tests already passed on PR, so we skip them here
|
||||
# ============================================
|
||||
|
||||
package-build:
|
||||
name: "Package - Build"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-package-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
|
||||
- name: Upload build artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: package-dist
|
||||
path: dist/
|
||||
retention-days: 1
|
||||
|
||||
package-publish:
|
||||
name: "Deploy - Publish to PyPI"
|
||||
runs-on: ubuntu-latest
|
||||
@@ -634,16 +363,17 @@ jobs:
|
||||
name: "Deploy - Frontend (GitHub Pages)"
|
||||
runs-on: ubuntu-latest
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Enable Yarn (for lyrics frontend)
|
||||
run: corepack enable
|
||||
@@ -660,14 +390,6 @@ jobs:
|
||||
run: npm run build
|
||||
|
||||
# ===== Build Lyrics Review Frontend (Vite) =====
|
||||
- name: Cache Yarn dependencies (lyrics frontend)
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: lyrics_transcriber_temp/lyrics_transcriber/frontend/node_modules
|
||||
key: yarn-lyrics-${{ runner.os }}-${{ hashFiles('lyrics_transcriber_temp/lyrics_transcriber/frontend/yarn.lock') }}
|
||||
restore-keys: |
|
||||
yarn-lyrics-${{ runner.os }}-
|
||||
|
||||
- name: Install lyrics frontend dependencies
|
||||
working-directory: lyrics_transcriber_temp/lyrics_transcriber/frontend
|
||||
run: yarn install --frozen-lockfile
|
||||
@@ -702,30 +424,22 @@ jobs:
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
|
||||
# NOTE: buy-site was consolidated into the main frontend (PR #122, Dec 2024)
|
||||
# Landing page and payment flow are now at gen.nomadkaraoke.com (root)
|
||||
# buy.nomadkaraoke.com subdomain has been deleted
|
||||
|
||||
deploy-backend:
|
||||
name: "Deploy - Backend (Cloud Run)"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
runs-on: ubuntu-latest
|
||||
needs: [backend-unit-tests, backend-emulator-tests, backend-lint]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
env:
|
||||
REGISTRY: us-central1-docker.pkg.dev
|
||||
PROJECT_ID: nomadkaraoke
|
||||
REPOSITORY: karaoke-repo
|
||||
BASE_IMAGE_NAME: karaoke-backend-base
|
||||
APP_IMAGE_NAME: karaoke-backend
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
fetch-depth: 0 # Need full history for tag check
|
||||
|
||||
- name: Set up Python 3.13
|
||||
- name: Set up Python (for version extraction)
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
@@ -733,15 +447,24 @@ jobs:
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
|
||||
- name: Extract version
|
||||
- name: Extract version and check if deployment needed
|
||||
id: version_check
|
||||
run: |
|
||||
VERSION=$(poetry version --short)
|
||||
echo "version=$VERSION" >> $GITHUB_OUTPUT
|
||||
echo "📋 Current version: $VERSION"
|
||||
echo "🚀 Deploying on every push to main"
|
||||
|
||||
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
|
||||
echo "needs_deploy=false" >> $GITHUB_OUTPUT
|
||||
echo "⏭️ Tag v$VERSION already exists - skipping backend deployment"
|
||||
echo " (Bump version in pyproject.toml to trigger a new deployment)"
|
||||
else
|
||||
echo "needs_deploy=true" >> $GITHUB_OUTPUT
|
||||
echo "✨ Tag v$VERSION does not exist - proceeding with deployment"
|
||||
fi
|
||||
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
id: auth
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
@@ -749,49 +472,27 @@ jobs:
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Configure Docker for Artifact Registry
|
||||
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
|
||||
|
||||
# ============================================
|
||||
# Docker Build with BuildKit + Registry Cache
|
||||
# This replaces expensive Cloud Build with free GitHub Actions
|
||||
# Registry cache persists between builds for fast rebuilds
|
||||
# ============================================
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Calculate base image hash
|
||||
id: base-hash
|
||||
run: |
|
||||
# Hash both poetry.lock AND Dockerfile.base - rebuild if either changes
|
||||
CURRENT_HASH=$(cat poetry.lock backend/Dockerfile.base | sha256sum | cut -d' ' -f1)
|
||||
echo "hash=$CURRENT_HASH" >> $GITHUB_OUTPUT
|
||||
echo "📋 Base content hash: $CURRENT_HASH"
|
||||
|
||||
- name: Check if base image exists and is current
|
||||
- name: Check if base image needs rebuild
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
id: check-base
|
||||
run: |
|
||||
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
|
||||
echo "🔍 Checking if base image needs rebuild..."
|
||||
|
||||
# Try to get the existing base image's hash label
|
||||
# First check if image exists locally (fast, no network call)
|
||||
# Only pull from registry if not cached locally
|
||||
# Hash both poetry.lock AND Dockerfile.base - rebuild if either changes
|
||||
CURRENT_HASH=$(cat poetry.lock backend/Dockerfile.base | sha256sum | cut -d' ' -f1)
|
||||
echo "Current combined hash (poetry.lock + Dockerfile.base): $CURRENT_HASH"
|
||||
|
||||
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
|
||||
|
||||
BASE_IMAGE="us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest"
|
||||
EXISTING_HASH=""
|
||||
if docker image inspect $BASE_IMAGE >/dev/null 2>&1; then
|
||||
echo "✅ Base image found in local cache (skipping slow registry pull)"
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
if [ -z "$EXISTING_HASH" ]; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
|
||||
fi
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
elif docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
echo "📥 Base image pulled from registry"
|
||||
|
||||
if docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
if [ -z "$EXISTING_HASH" ]; then
|
||||
@@ -799,10 +500,10 @@ jobs:
|
||||
fi
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
else
|
||||
echo "⚠️ No existing base image found - will build"
|
||||
echo "⚠️ No existing base image found"
|
||||
fi
|
||||
|
||||
if [ "${{ steps.base-hash.outputs.hash }}" != "$EXISTING_HASH" ]; then
|
||||
if [ "$CURRENT_HASH" != "$EXISTING_HASH" ]; then
|
||||
echo "🔄 Base image needs rebuild (poetry.lock or Dockerfile.base changed)"
|
||||
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
@@ -810,73 +511,91 @@ jobs:
|
||||
echo "needs_rebuild=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Build and push base image (if needed)
|
||||
if: steps.check-base.outputs.needs_rebuild == 'true'
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: backend/Dockerfile.base
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:${{ github.sha }}
|
||||
# Registry cache for ALL layers (mode=max) - persists between builds
|
||||
# ignore-error=true allows build to continue if cache doesn't exist yet
|
||||
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:cache,ignore-error=true
|
||||
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:cache,mode=max,ignore-error=true
|
||||
build-args: |
|
||||
BUILD_DATE=${{ github.run_id }}
|
||||
BUILD_ID=${{ github.sha }}
|
||||
BASE_CONTENT_HASH=${{ steps.base-hash.outputs.hash }}
|
||||
# Provenance disabled for smaller image and faster push
|
||||
provenance: false
|
||||
|
||||
- name: Build and push app image
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: backend/Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:latest
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:${{ github.sha }}
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:v${{ steps.version_check.outputs.version }}
|
||||
# Registry cache for app layers
|
||||
# ignore-error=true allows build to continue if cache doesn't exist yet
|
||||
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:cache,ignore-error=true
|
||||
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:cache,mode=max,ignore-error=true
|
||||
provenance: false
|
||||
|
||||
# ============================================
|
||||
# Deploy to Cloud Run
|
||||
# ============================================
|
||||
|
||||
- name: Deploy to Cloud Run
|
||||
- name: Rebuild base image (if needed)
|
||||
if: steps.version_check.outputs.needs_deploy == 'true' && steps.check-base.outputs.needs_rebuild == 'true'
|
||||
run: |
|
||||
VERSION="${{ steps.version_check.outputs.version }}"
|
||||
IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:${{ github.sha }}"
|
||||
echo "🏗️ Rebuilding base image with updated dependencies..."
|
||||
|
||||
echo "🚀 Deploying to Cloud Run..."
|
||||
echo "Image: $IMAGE"
|
||||
echo "Version: $VERSION"
|
||||
BASE_BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild-base.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
|
||||
gcloud run deploy karaoke-backend \
|
||||
--image $IMAGE \
|
||||
--region us-central1 \
|
||||
--platform managed \
|
||||
--allow-unauthenticated \
|
||||
--memory 16Gi \
|
||||
--cpu 8 \
|
||||
--cpu-boost \
|
||||
--execution-environment gen2 \
|
||||
--timeout 1800 \
|
||||
--concurrency 1 \
|
||||
--max-instances 50 \
|
||||
--min-instances 0 \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest,SENDGRID_API_KEY=sendgrid-api-key:latest,STRIPE_SECRET_KEY=stripe-secret-key:latest,STRIPE_WEBHOOK_SECRET=stripe-webhook-secret:latest,LANGFUSE_PUBLIC_KEY=langfuse-public-key:latest,LANGFUSE_SECRET_KEY=langfuse-secret-key:latest,LANGFUSE_HOST=langfuse-host:latest"
|
||||
echo "Base Build ID: $BASE_BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BASE_BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
echo "Waiting for base image build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BASE_BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Base build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Base image build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Base image build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BASE_BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
gcloud builds log $BASE_BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Trigger Cloud Build
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
run: |
|
||||
echo "🚀 Triggering Cloud Build for deployment..."
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
echo "Version: ${{ steps.version_check.outputs.version }}"
|
||||
|
||||
BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
|
||||
echo "Build ID: $BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
echo "Waiting for build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
echo ""
|
||||
gcloud builds log $BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Verify deployment and version
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
run: |
|
||||
EXPECTED_VERSION="${{ steps.version_check.outputs.version }}"
|
||||
SERVICE_URL=$(gcloud run services describe karaoke-backend \
|
||||
@@ -885,22 +604,22 @@ jobs:
|
||||
|
||||
echo "Service URL: $SERVICE_URL"
|
||||
echo "Expected version: $EXPECTED_VERSION"
|
||||
|
||||
|
||||
# Wait for new revision to be ready
|
||||
echo "Waiting for new revision to be ready..."
|
||||
sleep 15
|
||||
|
||||
|
||||
# Basic health check
|
||||
curl -f "$SERVICE_URL/api/health" || exit 1
|
||||
echo "✅ Service is healthy"
|
||||
|
||||
|
||||
# Version verification with retries (Cloud Run may take time to route to new revision)
|
||||
MAX_RETRIES=6
|
||||
RETRY_DELAY=10
|
||||
for i in $(seq 1 $MAX_RETRIES); do
|
||||
echo "Checking deployed version (attempt $i/$MAX_RETRIES)..."
|
||||
DEPLOYED_VERSION=$(curl -sf "$SERVICE_URL/api/health/detailed" | jq -r '.version // empty')
|
||||
|
||||
|
||||
if [ "$DEPLOYED_VERSION" = "$EXPECTED_VERSION" ]; then
|
||||
echo "✅ Version verified: $DEPLOYED_VERSION"
|
||||
break
|
||||
@@ -917,13 +636,19 @@ jobs:
|
||||
done
|
||||
|
||||
- name: Summary
|
||||
if: success()
|
||||
if: steps.version_check.outputs.needs_deploy == 'true' && success()
|
||||
run: |
|
||||
echo "🎉 Deployment successful!"
|
||||
echo "Version: ${{ steps.version_check.outputs.version }}"
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
echo "Branch: ${{ github.ref_name }}"
|
||||
echo "Run: ${{ github.run_number }}"
|
||||
|
||||
- name: Skipped deployment summary
|
||||
if: steps.version_check.outputs.needs_deploy == 'false'
|
||||
run: |
|
||||
echo "⏭️ Backend deployment skipped - version ${{ steps.version_check.outputs.version }} already deployed"
|
||||
echo ""
|
||||
echo "📊 Built on self-hosted GCP runner (200GB SSD, no disk space issues)"
|
||||
echo " Runner: gcp-runner with 4 vCPU, 16GB RAM"
|
||||
echo "To deploy a new version:"
|
||||
echo " 1. Update version in pyproject.toml"
|
||||
echo " 2. Commit and push to main"
|
||||
+1
-9
@@ -12,9 +12,6 @@ tests/outputs
|
||||
/input
|
||||
/output/
|
||||
|
||||
# Cloud Infra
|
||||
.emulator-logs
|
||||
|
||||
# Next.js / React Frontend (./frontend)
|
||||
node_modules/
|
||||
frontend/node_modules/
|
||||
@@ -47,7 +44,6 @@ frontend/coverage/
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
frontend/playwright/.cache/
|
||||
frontend/e2e/screenshots/
|
||||
|
||||
# TypeScript
|
||||
frontend/*.tsbuildinfo
|
||||
@@ -83,9 +79,8 @@ eggs/
|
||||
# Python lib directories (not frontend)
|
||||
lib/
|
||||
lib64/
|
||||
# But allow frontend lib directories
|
||||
# But allow frontend lib
|
||||
!frontend/lib/
|
||||
!buy-site/lib/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
@@ -229,6 +224,3 @@ cython_debug/
|
||||
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
||||
#.idea/
|
||||
flacfetch
|
||||
|
||||
# Test output directories
|
||||
Rick Astley*/
|
||||
File renamed without changes.
@@ -1,98 +1,131 @@
|
||||
# Karaoke-Gen AI Assistant Guidelines
|
||||
# Project Guidelines for AI Assistants
|
||||
|
||||
## Project Overview
|
||||
This document contains guidelines and rules for AI assistants working on this codebase.
|
||||
|
||||
**Karaoke video generation platform** - CLI tool and web service that creates professional karaoke videos with synchronized lyrics.
|
||||
## Core Principles
|
||||
|
||||
- **Production**: <https://gen.nomadkaraoke.com> (frontend), <https://api.nomadkaraoke.com> (backend)
|
||||
- **CLI**: `karaoke-gen` (local), `karaoke-gen-remote` (cloud)
|
||||
- **Repo**: <https://github.com/nomadkaraoke/karaoke-gen>
|
||||
**Always be testing.** Proactively evaluate, read, run, fix, and improve tests whenever you touch any part of the system—frontend, backend, infrastructure. Every code change should be thoroughly tested. Leave everything better tested than you found it.
|
||||
|
||||
## Quick Reference
|
||||
**Always be documenting.** Continuously read, write, improve, update, and organize markdown documentation. Reference existing docs and keep them accurate as the codebase evolves.
|
||||
|
||||
| What | Where |
|
||||
|------|-------|
|
||||
| Product vision & goals | `docs/PRODUCT-VISION.md` |
|
||||
| Current status | `docs/README.md` |
|
||||
| Architecture | `docs/ARCHITECTURE.md` |
|
||||
| Dev setup & testing | `docs/DEVELOPMENT.md` |
|
||||
| API reference | `docs/API.md` |
|
||||
| Past learnings | `docs/LESSONS-LEARNED.md` |
|
||||
**Build for maintainability.** Keep code SOLID, modular, and clean. Favor simplicity and clarity. Future developers (and AI agents) will thank you.
|
||||
|
||||
## Tech Stack
|
||||
## Branching Workflow
|
||||
|
||||
- **Backend**: FastAPI on Cloud Run, Firestore, GCS, Secret Manager
|
||||
- **Frontend**: Next.js on Cloudflare Pages
|
||||
- **Processing**: Modal (GPU audio separation), AudioShake/Whisper (transcription)
|
||||
- **Infra**: Pulumi IaC, GitHub Actions CI/CD
|
||||
**Never commit directly to `main`.** Always work on a feature branch.
|
||||
|
||||
## Essential Rules
|
||||
Create descriptive branch names like `feature/add-discord-notifications`, `fix/audio-sync-issue`, or `refactor/cleanup-api-routes`.
|
||||
|
||||
### Git Workflow
|
||||
- **Never commit directly to `main`** - always use feature branches
|
||||
- **Use git worktrees** for all changes:
|
||||
### Using Git Worktrees
|
||||
|
||||
**For non-trivial changes** (new features, multi-file edits, refactors):
|
||||
- Create a git worktree in the parent folder before making any file changes:
|
||||
```bash
|
||||
git worktree add -b feature/xyz ../karaoke-gen-feat-xyz main
|
||||
git worktree add ../karaoke-gen-feat-xyz feature/xyz
|
||||
```
|
||||
- Create PR with summary, changes, testing info
|
||||
- Merge only after CI passes
|
||||
- This keeps the main directory clean on `main` and enables parallel work on multiple features
|
||||
- Name the worktree folder descriptively, e.g., `../karaoke-gen-fix-audio-sync`
|
||||
|
||||
### Testing (Required Before Commit)
|
||||
**For trivial changes** (typos, single-line fixes):
|
||||
- Creating a branch in the current directory is acceptable
|
||||
- Use `git checkout -b fix/typo-xyz` before making changes
|
||||
|
||||
**Cleanup:** When done with a worktree, remove it with:
|
||||
```bash
|
||||
# Backend (10 min timeout, limit output)
|
||||
make test 2>&1 | tail -n 500
|
||||
git worktree remove ../karaoke-gen-feat-xyz
|
||||
```
|
||||
|
||||
# Frontend
|
||||
### Creating a Pull Request
|
||||
|
||||
Once all changes are complete, tests pass, and commits are pushed, create a PR with:
|
||||
|
||||
1. **Summary** - Brief overview of the changes
|
||||
2. **Changes Made** - List of specific modifications
|
||||
3. **Testing** - How changes were tested (unit tests, manual testing)
|
||||
4. **Related Issues** - Reference any GitHub issues (e.g., "Fixes #123")
|
||||
|
||||
Only merge after review and CI checks pass.
|
||||
|
||||
## Infrastructure Changes
|
||||
|
||||
All Google Cloud infrastructure (IAM, Cloud Storage, Cloud Run, Secret Manager, Artifact Registry, etc.) must be managed via **Pulumi IaC** in the `infrastructure/` folder.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Reading/querying** with `gcloud` CLI is fine for investigation and debugging
|
||||
- **Temporary changes** via `gcloud` CLI or Console are acceptable for urgent fixes or experimentation, but **must be cleaned up** and properly codified in the IaC afterward
|
||||
- **Permanent changes** must go through Pulumi — never leave `gcloud` or Console changes as the final state
|
||||
|
||||
### Workflow
|
||||
|
||||
1. Make changes in `infrastructure/`
|
||||
2. Run `pulumi preview` and verify the output
|
||||
3. Run `pulumi up` to apply
|
||||
|
||||
### Authentication Issues
|
||||
|
||||
If you encounter any authentication issues with `gcloud` or `pulumi` (e.g., expired tokens, permission errors), **stop and notify the user** so they can run the appropriate auth commands in their external terminal:
|
||||
|
||||
```bash
|
||||
gcloud auth login
|
||||
gcloud auth application-default login
|
||||
```
|
||||
|
||||
Do not attempt to work around auth issues or make assumptions about credentials.
|
||||
|
||||
## Version Management
|
||||
|
||||
When committing and pushing changes, always bump the patch version in `pyproject.toml`.
|
||||
|
||||
The version is located at `tool.poetry.version` in `pyproject.toml`. For example, if the current version is `0.70.3`, bump it to `0.70.4`.
|
||||
|
||||
This ensures every commit pushed to the repository has a unique version number for tracking and deployment purposes.
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
**Before committing and pushing ANY changes, you MUST run the full test suite.**
|
||||
|
||||
### Backend Test Command
|
||||
|
||||
```bash
|
||||
make test 2>&1 | tail -n 500
|
||||
```
|
||||
|
||||
**Important for AI agents:**
|
||||
- Set a **10 minute timeout** when running tests (they can take a while with emulators)
|
||||
- Always pipe output through `tail -n 500` to limit output and preserve context window
|
||||
- The tail output will show test results and any failures at the end
|
||||
|
||||
This runs all backend tests in order and fails fast if any test fails:
|
||||
1. Unit tests (karaoke_gen package) with coverage check
|
||||
2. Backend unit tests
|
||||
3. E2E integration tests with emulators (auto-starts/stops emulators)
|
||||
|
||||
### Frontend Test Command
|
||||
|
||||
When working in the `frontend/` directory, run:
|
||||
|
||||
```bash
|
||||
cd frontend && npm run test:all 2>&1 | tail -n 200
|
||||
```
|
||||
|
||||
### Version Bumping
|
||||
- Bump `tool.poetry.version` in `pyproject.toml` for code changes
|
||||
- Skip for docs-only changes
|
||||
This runs:
|
||||
1. **Unit tests** (Jest) - Component and utility tests
|
||||
2. **E2E tests** (Playwright) - Browser-based integration tests
|
||||
|
||||
### Infrastructure
|
||||
- All GCP changes via Pulumi in `infrastructure/`
|
||||
- `gcloud` CLI for reading/debugging only
|
||||
- Stop and notify user on auth issues
|
||||
**Note:** E2E tests require `KARAOKE_ACCESS_TOKEN` environment variable to be set in `frontend/.env.local`.
|
||||
|
||||
## Documentation Maintenance
|
||||
Individual test commands:
|
||||
- `npm run test:unit` - Run unit tests only
|
||||
- `npm run test:e2e` - Run E2E tests only
|
||||
- `npm run test:ci` - Run unit tests with coverage
|
||||
|
||||
### Structure
|
||||
```text
|
||||
docs/
|
||||
├── README.md # Current status + navigation (UPDATE THIS)
|
||||
├── ARCHITECTURE.md # System design
|
||||
├── DEVELOPMENT.md # Dev setup, testing, deployment
|
||||
├── API.md # Backend API reference
|
||||
├── LESSONS-LEARNED.md # Key insights for future agents
|
||||
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
|
||||
```
|
||||
### Pre-commit Checklist
|
||||
|
||||
### Rules
|
||||
1. **Before merging PRs**: Run `/docs-review` to check if docs need updating
|
||||
2. **Periodically**: Run `/docs-maintain` to verify docs are organized
|
||||
3. **For significant work**: Create `docs/archive/YYYY-MM-DD-topic.md`
|
||||
4. **Add learnings**: Update `docs/LESSONS-LEARNED.md` with insights
|
||||
5. **Keep status current**: Update `docs/README.md` when project state changes
|
||||
- [ ] `make test` passes (for backend changes)
|
||||
- [ ] `cd frontend && npm run test:all` passes (for frontend changes)
|
||||
- [ ] Version in `pyproject.toml` has been bumped
|
||||
- [ ] `coderabbit review --plain` has been run to get comprehensive analysis and suggestions for cleaner, more maintainable code. Apply the feedback to improve accessibility, structure, and best practices.
|
||||
|
||||
### What Goes Where
|
||||
- **README.md**: Current status, known issues, quick links
|
||||
- **ARCHITECTURE.md**: System design, data flow, tech decisions
|
||||
- **DEVELOPMENT.md**: Setup, testing, deployment, CI/CD
|
||||
- **API.md**: Endpoints, request/response formats
|
||||
- **LESSONS-LEARNED.md**: What worked, what didn't, gotchas
|
||||
- **archive/**: Completed work, session summaries, old plans
|
||||
**Do not commit or push code that fails tests. Fix any test failures before proceeding.**
|
||||
|
||||
## Related Projects
|
||||
|
||||
### flacfetch
|
||||
Location: `/Users/andrew/Projects/flacfetch`
|
||||
|
||||
Workflow: Make changes on `main`, bump version, push, wait for CI, then `poetry update flacfetch` in karaoke-gen.
|
||||
|
||||
## Pre-Commit Checklist
|
||||
|
||||
- [ ] Tests pass (`make test` / `npm run test:all`)
|
||||
- [ ] Version bumped (if code changed)
|
||||
- [ ] `/docs-review` run (check for doc updates)
|
||||
- [ ] PR created with clear description
|
||||
@@ -71,40 +71,8 @@ export AUDIOSHAKE_API_TOKEN="your_audioshake_token"
|
||||
|
||||
Get an API key at [https://www.audioshake.ai/](https://www.audioshake.ai/) - business only, at time of writing this.
|
||||
|
||||
#### Option 2: Local Whisper (No Cloud Required)
|
||||
Run Whisper directly on your local machine using whisper-timestamped. Works on CPU, NVIDIA GPU (CUDA), or Apple Silicon.
|
||||
|
||||
```bash
|
||||
# Install with local Whisper support
|
||||
pip install "karaoke-gen[local-whisper]"
|
||||
|
||||
# Optional: Configure model size (tiny, base, small, medium, large)
|
||||
export WHISPER_MODEL_SIZE="medium"
|
||||
|
||||
# Optional: Force specific device (cpu, cuda, mps)
|
||||
export WHISPER_DEVICE="cpu"
|
||||
```
|
||||
|
||||
**Model Size Guide:**
|
||||
| Model | VRAM | Speed | Quality |
|
||||
|-------|------|-------|---------|
|
||||
| tiny | ~1GB | Fast | Lower |
|
||||
| base | ~1GB | Fast | Basic |
|
||||
| small | ~2GB | Medium | Good |
|
||||
| medium | ~5GB | Slower | Better |
|
||||
| large | ~10GB | Slowest | Best |
|
||||
|
||||
**CPU-Only Installation** (no GPU required):
|
||||
```bash
|
||||
# Pre-install CPU-only PyTorch first
|
||||
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
|
||||
pip install "karaoke-gen[local-whisper]"
|
||||
```
|
||||
|
||||
Local Whisper runs automatically as a fallback when no cloud transcription services are configured.
|
||||
|
||||
#### Option 3: Whisper via RunPod
|
||||
Cloud-based alternative using OpenAI's Whisper model on RunPod infrastructure.
|
||||
#### Option 2: Whisper via RunPod
|
||||
Open-source alternative using OpenAI's Whisper model on RunPod infrastructure.
|
||||
|
||||
```bash
|
||||
export RUNPOD_API_KEY="your_runpod_key"
|
||||
@@ -606,44 +574,6 @@ If the output video has quality problems:
|
||||
- Check available codecs: `ffmpeg -codecs`
|
||||
- For 4K output, ensure sufficient disk space (10GB+ per track)
|
||||
|
||||
### Local Whisper Issues
|
||||
|
||||
#### GPU Out of Memory
|
||||
If you get CUDA out of memory errors:
|
||||
```bash
|
||||
# Use a smaller model
|
||||
export WHISPER_MODEL_SIZE="small" # or "tiny"
|
||||
|
||||
# Or force CPU mode
|
||||
export WHISPER_DEVICE="cpu"
|
||||
```
|
||||
|
||||
#### Slow Transcription on CPU
|
||||
CPU transcription is significantly slower than GPU. For faster processing:
|
||||
- Use a smaller model (`tiny` or `base`)
|
||||
- Consider using cloud transcription (AudioShake or RunPod)
|
||||
- On Apple Silicon, the `small` model offers good speed/quality balance
|
||||
|
||||
#### Model Download Issues
|
||||
Whisper models are downloaded on first use (~1-3GB depending on size). If downloads fail:
|
||||
- Check your internet connection
|
||||
- Set a custom cache directory: `export WHISPER_CACHE_DIR="/path/with/space"`
|
||||
- Models are cached in `~/.cache/whisper/` by default
|
||||
|
||||
#### whisper-timestamped Not Found
|
||||
If you get "whisper-timestamped is not installed":
|
||||
```bash
|
||||
pip install "karaoke-gen[local-whisper]"
|
||||
# Or install directly:
|
||||
pip install whisper-timestamped
|
||||
```
|
||||
|
||||
#### Disabling Local Whisper
|
||||
If you want to disable local Whisper (e.g., to force cloud transcription):
|
||||
```bash
|
||||
export ENABLE_LOCAL_WHISPER="false"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Development
|
||||
|
||||
@@ -1,75 +0,0 @@
|
||||
[
|
||||
{
|
||||
"test_name": "Sound-alike error (wurld -> world)",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "ReplaceWord",
|
||||
"word_id": null,
|
||||
"word_ids": [
|
||||
"w1"
|
||||
],
|
||||
"replacement_text": "world",
|
||||
"confidence": 0.75,
|
||||
"reason": "Sound-alike error. Reference suggests: 'world'. The transcription 'wurld' is a phonetic misspelling or sound-alike of the word 'world' which appears in both reference sources (Genius and LRCLIB).",
|
||||
"gap_category": "GapCategory.SOUND_ALIKE",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"test_name": "Punctuation difference",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "NoAction",
|
||||
"word_id": null,
|
||||
"word_ids": [
|
||||
"w2"
|
||||
],
|
||||
"replacement_text": null,
|
||||
"confidence": 0.99,
|
||||
"reason": "Transcription matches at least one reference source. The gap text 'Hello' followed by the context 'is it me you're looking for' matches the Genius reference lyrics exactly, excluding punctuation. According to the guidelines, if any reference source matches the transcription text (ignoring punctuation), it should be classified as NO_ERROR.",
|
||||
"gap_category": "GapCategory.NO_ERROR",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"test_name": "Extra filler word",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "DeleteWord",
|
||||
"word_id": "w3",
|
||||
"word_ids": null,
|
||||
"replacement_text": null,
|
||||
"confidence": 0.8,
|
||||
"reason": "Extra filler word 'And' at sentence start not in reference. The transcription includes the filler word 'And' at the beginning of the phrase, which is not present in the reference lyrics. This matches the definition of EXTRA_WORDS.",
|
||||
"gap_category": "GapCategory.EXTRA_WORDS",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"test_name": "Complex multi-word error",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "NoAction",
|
||||
"word_id": null,
|
||||
"word_ids": [
|
||||
"w7",
|
||||
"w8",
|
||||
"w9",
|
||||
"w10"
|
||||
],
|
||||
"replacement_text": null,
|
||||
"confidence": 0.99,
|
||||
"reason": "Transcription matches at least one reference source. The transcription text 'gonna give you up' matches the Genius reference lyrics exactly when ignoring punctuation and capitalization. In context, the full phrase 'never gonna give you up' is correctly transcribed and aligns perfectly with the known lyrics and title of the song.",
|
||||
"gap_category": "GapCategory.NO_ERROR",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
+70
-182
@@ -4,11 +4,11 @@ FastAPI dependencies for authentication and authorization.
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, UTC
|
||||
from typing import Optional, Tuple, Callable, Union
|
||||
from typing import Optional, Tuple, Callable
|
||||
from fastapi import Depends, HTTPException, Header, Query, Request, Path
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
|
||||
from backend.services.auth_service import get_auth_service, UserType, AuthService, AuthResult
|
||||
from backend.services.auth_service import get_auth_service, UserType, AuthService
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -66,130 +66,70 @@ async def require_auth(
|
||||
auth_service: AuthService = Depends(get_auth_service),
|
||||
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
|
||||
token: Optional[str] = Query(None)
|
||||
) -> AuthResult:
|
||||
) -> Tuple[str, UserType, int]:
|
||||
"""
|
||||
Require authentication for an endpoint.
|
||||
|
||||
|
||||
Returns:
|
||||
AuthResult with full authentication context including:
|
||||
- is_valid, user_type, remaining_uses, message (backward compatible via tuple unpacking)
|
||||
- user_email: Email of authenticated user (if session/API key auth)
|
||||
- is_admin: Whether user has admin privileges
|
||||
|
||||
(token, user_type, remaining_uses)
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if authentication fails
|
||||
"""
|
||||
# Get token from request
|
||||
token_str = await get_token_from_request(request, credentials, token)
|
||||
|
||||
|
||||
if not token_str:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide token via Authorization header or ?token= parameter"
|
||||
)
|
||||
|
||||
# Validate token using the full method
|
||||
auth_result = auth_service.validate_token_full(token_str)
|
||||
|
||||
# Get request_id from middleware for correlation
|
||||
request_id = getattr(request.state, "request_id", None)
|
||||
|
||||
if not auth_result.is_valid:
|
||||
# Log auth failure with request_id for correlation
|
||||
|
||||
# Validate token
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(token_str)
|
||||
|
||||
if not is_valid:
|
||||
# Log more details for debugging token issues
|
||||
auth_header = request.headers.get("Authorization", "")
|
||||
logger.warning(
|
||||
"auth_failed",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"audit_type": "auth_event",
|
||||
"auth_message": auth_result.message,
|
||||
"token_provided": bool(token_str),
|
||||
"token_length": len(token_str) if token_str else 0,
|
||||
"auth_header_present": bool(auth_header),
|
||||
}
|
||||
f"Authentication failed: {message}. "
|
||||
f"Token provided: {bool(token_str)}, "
|
||||
f"Token length: {len(token_str) if token_str else 0}, "
|
||||
f"Auth header present: {bool(auth_header)}, "
|
||||
f"Header prefix: {auth_header[:20] if auth_header else 'none'}..."
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail=f"Authentication failed: {auth_result.message}"
|
||||
)
|
||||
|
||||
# Log successful auth with request_id for correlation with request audit
|
||||
logger.info(
|
||||
"auth_success",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"user_email": auth_result.user_email,
|
||||
"user_type": auth_result.user_type.value if auth_result.user_type else None,
|
||||
"is_admin": auth_result.is_admin,
|
||||
"remaining_uses": auth_result.remaining_uses,
|
||||
"audit_type": "auth_event",
|
||||
}
|
||||
)
|
||||
|
||||
return auth_result
|
||||
|
||||
|
||||
async def require_auth_legacy(
|
||||
request: Request,
|
||||
auth_service: AuthService = Depends(get_auth_service),
|
||||
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
|
||||
token: Optional[str] = Query(None)
|
||||
) -> Tuple[str, UserType, int]:
|
||||
"""
|
||||
Legacy authentication dependency that returns tuple.
|
||||
|
||||
DEPRECATED: Use require_auth which returns AuthResult instead.
|
||||
|
||||
Returns:
|
||||
(token, user_type, remaining_uses)
|
||||
"""
|
||||
token_str = await get_token_from_request(request, credentials, token)
|
||||
|
||||
if not token_str:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide token via Authorization header or ?token= parameter"
|
||||
)
|
||||
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(token_str)
|
||||
|
||||
if not is_valid:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail=f"Authentication failed: {message}"
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Authenticated as {user_type} (remaining: {remaining_uses})")
|
||||
|
||||
return token_str, user_type, remaining_uses
|
||||
|
||||
|
||||
async def require_admin(
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> AuthResult:
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> Tuple[str, UserType, int]:
|
||||
"""
|
||||
Require admin access for an endpoint.
|
||||
|
||||
Admin access is granted if:
|
||||
- Using an admin token from ADMIN_TOKENS env var
|
||||
- User email is from admin domain (e.g., @nomadkaraoke.com)
|
||||
- User role is set to ADMIN in database
|
||||
|
||||
|
||||
Returns:
|
||||
AuthResult with admin privileges
|
||||
|
||||
(token, user_type, remaining_uses)
|
||||
|
||||
Raises:
|
||||
HTTPException: 403 if user is not admin
|
||||
"""
|
||||
if not auth_result.is_admin:
|
||||
logger.warning(
|
||||
f"Admin access denied for {auth_result.user_type} user "
|
||||
f"(email: {auth_result.user_email or 'unknown'})"
|
||||
)
|
||||
token, user_type, remaining_uses = auth_data
|
||||
|
||||
if user_type != UserType.ADMIN:
|
||||
logger.warning(f"Admin access denied for {user_type} user")
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="Admin access required"
|
||||
)
|
||||
|
||||
return auth_result
|
||||
|
||||
return token, user_type, remaining_uses
|
||||
|
||||
|
||||
def optional_auth(
|
||||
@@ -228,14 +168,14 @@ def optional_auth(
|
||||
def require_review_auth_factory(job_id_param: str = "job_id"):
|
||||
"""
|
||||
Factory to create a review authentication dependency.
|
||||
|
||||
|
||||
Accepts either:
|
||||
1. Full user authentication (admin/user token) - also validates job ownership
|
||||
1. Full user authentication (admin/user token)
|
||||
2. Job-specific review token (only valid for the specific job)
|
||||
|
||||
|
||||
Args:
|
||||
job_id_param: Name of the path parameter containing the job ID
|
||||
|
||||
|
||||
Returns:
|
||||
Dependency function that validates review access
|
||||
"""
|
||||
@@ -249,63 +189,37 @@ def require_review_auth_factory(job_id_param: str = "job_id"):
|
||||
) -> Tuple[str, str]:
|
||||
"""
|
||||
Validate review access for a job.
|
||||
|
||||
|
||||
Returns:
|
||||
(job_id, auth_type) where auth_type is "full" or "review_token"
|
||||
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if authentication fails
|
||||
HTTPException: 403 if user doesn't own the job (for full auth)
|
||||
"""
|
||||
# Import here to avoid circular dependency
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
job_manager = JobManager()
|
||||
|
||||
|
||||
# Try full authentication first
|
||||
full_token = None
|
||||
if credentials:
|
||||
full_token = credentials.credentials
|
||||
elif token:
|
||||
full_token = token
|
||||
|
||||
|
||||
if full_token:
|
||||
auth_result = auth_service.validate_token_full(full_token)
|
||||
if auth_result.is_valid:
|
||||
# For full auth, also verify job ownership
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Check ownership: admin can access all, users only their own jobs
|
||||
if not auth_result.is_admin:
|
||||
if auth_result.user_email and job.user_email:
|
||||
if auth_result.user_email.lower() != job.user_email.lower():
|
||||
logger.warning(
|
||||
f"Review access denied: user {auth_result.user_email} "
|
||||
f"tried to access job {job_id} owned by {job.user_email}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's review"
|
||||
)
|
||||
elif job.user_email:
|
||||
# Token auth without email trying to access a job with owner
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's review"
|
||||
)
|
||||
|
||||
logger.info(f"Review access granted via full auth ({auth_result.user_type}) for job {job_id}")
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(full_token)
|
||||
if is_valid:
|
||||
logger.info(f"Review access granted via full auth ({user_type}) for job {job_id}")
|
||||
return job_id, "full"
|
||||
|
||||
|
||||
# Try review token
|
||||
if review_token:
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
|
||||
# Validate review token matches
|
||||
if job.review_token and secrets.compare_digest(job.review_token, review_token):
|
||||
# Check expiry if set
|
||||
@@ -315,25 +229,25 @@ def require_review_auth_factory(job_id_param: str = "job_id"):
|
||||
expiry = job.review_token_expires_at
|
||||
if expiry.tzinfo is None:
|
||||
expiry = expiry.replace(tzinfo=UTC)
|
||||
|
||||
|
||||
if now > expiry:
|
||||
logger.warning(f"Review token expired for job {job_id}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Review token has expired. Please request a new review link."
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Review access granted via review_token for job {job_id}")
|
||||
return job_id, "review_token"
|
||||
else:
|
||||
logger.warning(f"Invalid review token for job {job_id}")
|
||||
|
||||
|
||||
# No valid authentication
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide either a full access token or a valid review_token for this job."
|
||||
)
|
||||
|
||||
|
||||
return require_review_auth
|
||||
|
||||
|
||||
@@ -344,14 +258,14 @@ require_review_auth = require_review_auth_factory()
|
||||
def require_instrumental_auth_factory(job_id_param: str = "job_id"):
|
||||
"""
|
||||
Factory to create an instrumental review authentication dependency.
|
||||
|
||||
|
||||
Accepts either:
|
||||
1. Full user authentication (admin/user token) - also validates job ownership
|
||||
1. Full user authentication (admin/user token)
|
||||
2. Job-specific instrumental token (only valid for the specific job)
|
||||
|
||||
|
||||
Args:
|
||||
job_id_param: Name of the path parameter containing the job ID
|
||||
|
||||
|
||||
Returns:
|
||||
Dependency function that validates instrumental review access
|
||||
"""
|
||||
@@ -365,63 +279,37 @@ def require_instrumental_auth_factory(job_id_param: str = "job_id"):
|
||||
) -> Tuple[str, str]:
|
||||
"""
|
||||
Validate instrumental review access for a job.
|
||||
|
||||
|
||||
Returns:
|
||||
(job_id, auth_type) where auth_type is "full" or "instrumental_token"
|
||||
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if authentication fails
|
||||
HTTPException: 403 if user doesn't own the job (for full auth)
|
||||
"""
|
||||
# Import here to avoid circular dependency
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
job_manager = JobManager()
|
||||
|
||||
|
||||
# Try full authentication first
|
||||
full_token = None
|
||||
if credentials:
|
||||
full_token = credentials.credentials
|
||||
elif token:
|
||||
full_token = token
|
||||
|
||||
|
||||
if full_token:
|
||||
auth_result = auth_service.validate_token_full(full_token)
|
||||
if auth_result.is_valid:
|
||||
# For full auth, also verify job ownership
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Check ownership: admin can access all, users only their own jobs
|
||||
if not auth_result.is_admin:
|
||||
if auth_result.user_email and job.user_email:
|
||||
if auth_result.user_email.lower() != job.user_email.lower():
|
||||
logger.warning(
|
||||
f"Instrumental access denied: user {auth_result.user_email} "
|
||||
f"tried to access job {job_id} owned by {job.user_email}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's instrumental review"
|
||||
)
|
||||
elif job.user_email:
|
||||
# Token auth without email trying to access a job with owner
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's instrumental review"
|
||||
)
|
||||
|
||||
logger.info(f"Instrumental access granted via full auth ({auth_result.user_type}) for job {job_id}")
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(full_token)
|
||||
if is_valid:
|
||||
logger.info(f"Instrumental access granted via full auth ({user_type}) for job {job_id}")
|
||||
return job_id, "full"
|
||||
|
||||
|
||||
# Try instrumental token
|
||||
if instrumental_token:
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
|
||||
# Validate instrumental token matches
|
||||
if job.instrumental_token and secrets.compare_digest(job.instrumental_token, instrumental_token):
|
||||
# Check expiry if set
|
||||
@@ -431,25 +319,25 @@ def require_instrumental_auth_factory(job_id_param: str = "job_id"):
|
||||
expiry = job.instrumental_token_expires_at
|
||||
if expiry.tzinfo is None:
|
||||
expiry = expiry.replace(tzinfo=UTC)
|
||||
|
||||
|
||||
if now > expiry:
|
||||
logger.warning(f"Instrumental token expired for job {job_id}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Instrumental review token has expired. Please request a new review link."
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Instrumental access granted via instrumental_token for job {job_id}")
|
||||
return job_id, "instrumental_token"
|
||||
else:
|
||||
logger.warning(f"Invalid instrumental token for job {job_id}")
|
||||
|
||||
|
||||
# No valid authentication
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide either a full access token or a valid instrumental_token for this job."
|
||||
)
|
||||
|
||||
|
||||
return require_instrumental_auth
|
||||
|
||||
|
||||
|
||||
@@ -85,13 +85,9 @@ class AudioSearchRequest(BaseModel):
|
||||
# Auto-download mode
|
||||
auto_download: bool = Field(False, description="Automatically select best result and download")
|
||||
|
||||
# Theme configuration
|
||||
theme_id: Optional[str] = Field(None, description="Theme ID to use (e.g., 'nomad', 'default'). If set, CDG/TXT are enabled by default.")
|
||||
color_overrides: Optional[Dict[str, str]] = Field(None, description="Color overrides: artist_color, title_color, sung_lyrics_color, unsung_lyrics_color (hex #RRGGBB)")
|
||||
|
||||
# Processing options (CDG/TXT require style config or theme, disabled by default unless theme is set)
|
||||
enable_cdg: Optional[bool] = Field(None, description="Generate CDG+MP3 package. Default: True if theme_id set, False otherwise")
|
||||
enable_txt: Optional[bool] = Field(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise")
|
||||
# Processing options (CDG/TXT require style config, disabled by default)
|
||||
enable_cdg: bool = Field(False, description="Generate CDG+MP3 package (requires style config)")
|
||||
enable_txt: bool = Field(False, description="Generate TXT+MP3 package (requires style config)")
|
||||
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
|
||||
@@ -184,35 +180,6 @@ class AudioSelectResponse(BaseModel):
|
||||
selected_provider: str
|
||||
|
||||
|
||||
def _resolve_cdg_txt_defaults(
|
||||
theme_id: Optional[str],
|
||||
enable_cdg: Optional[bool],
|
||||
enable_txt: Optional[bool]
|
||||
) -> Tuple[bool, bool]:
|
||||
"""
|
||||
Resolve CDG/TXT settings based on theme and explicit settings.
|
||||
|
||||
When a theme is selected, CDG and TXT are enabled by default.
|
||||
Explicit True/False values always override the default.
|
||||
|
||||
Args:
|
||||
theme_id: Theme identifier (if any)
|
||||
enable_cdg: Explicit CDG setting (None means use default)
|
||||
enable_txt: Explicit TXT setting (None means use default)
|
||||
|
||||
Returns:
|
||||
Tuple of (resolved_enable_cdg, resolved_enable_txt)
|
||||
"""
|
||||
# Default based on whether theme is set
|
||||
default_enabled = theme_id is not None
|
||||
|
||||
# Explicit values override defaults, None uses default
|
||||
resolved_cdg = enable_cdg if enable_cdg is not None else default_enabled
|
||||
resolved_txt = enable_txt if enable_txt is not None else default_enabled
|
||||
|
||||
return resolved_cdg, resolved_txt
|
||||
|
||||
|
||||
def extract_request_metadata(request: Request, created_from: str = "audio_search") -> Dict[str, Any]:
|
||||
"""Extract metadata from request for job tracking."""
|
||||
headers = dict(request.headers)
|
||||
@@ -494,20 +461,13 @@ async def search_audio(
|
||||
|
||||
# Extract request metadata
|
||||
request_metadata = extract_request_metadata(request, created_from="audio_search")
|
||||
|
||||
# Resolve CDG/TXT defaults based on theme
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
body.theme_id, body.enable_cdg, body.enable_txt
|
||||
)
|
||||
|
||||
|
||||
# Create job
|
||||
job_create = JobCreate(
|
||||
artist=body.artist,
|
||||
title=body.title,
|
||||
theme_id=body.theme_id,
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
enable_cdg=body.enable_cdg,
|
||||
enable_txt=body.enable_txt,
|
||||
brand_prefix=effective_brand_prefix,
|
||||
enable_youtube_upload=effective_enable_youtube_upload,
|
||||
youtube_description=effective_youtube_description,
|
||||
@@ -537,28 +497,7 @@ async def search_audio(
|
||||
'audio_search_title': body.title,
|
||||
'auto_download': body.auto_download,
|
||||
})
|
||||
|
||||
# If theme is set and no custom style files are being uploaded, prepare theme style now
|
||||
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
|
||||
# can access the style configuration for preview videos
|
||||
if body.theme_id and not body.style_files:
|
||||
from backend.api.routes.file_upload import _prepare_theme_for_job
|
||||
try:
|
||||
style_params_path, theme_style_assets, youtube_desc = _prepare_theme_for_job(
|
||||
job_id, body.theme_id, body.color_overrides
|
||||
)
|
||||
theme_update = {
|
||||
'style_params_gcs_path': style_params_path,
|
||||
'style_assets': theme_style_assets,
|
||||
}
|
||||
if youtube_desc and not effective_youtube_description:
|
||||
theme_update['youtube_description_template'] = youtube_desc
|
||||
job_manager.update_job(job_id, theme_update)
|
||||
logger.info(f"Applied theme '{body.theme_id}' to job {job_id}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to prepare theme '{body.theme_id}' for job {job_id}: {e}")
|
||||
# Continue without theme - job can still be processed with defaults
|
||||
|
||||
|
||||
# Handle style file uploads if provided
|
||||
style_upload_urls: List[StyleUploadUrl] = []
|
||||
style_assets = {}
|
||||
|
||||
+101
-213
@@ -15,7 +15,6 @@ import json
|
||||
import logging
|
||||
import tempfile
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from fastapi import APIRouter, UploadFile, File, Form, HTTPException, BackgroundTasks, Request, Body, Depends
|
||||
from pathlib import Path
|
||||
from typing import Optional, List, Dict, Any, Tuple
|
||||
@@ -276,46 +275,6 @@ def _resolve_cdg_txt_defaults(
|
||||
return resolved_cdg, resolved_txt
|
||||
|
||||
|
||||
@dataclass
|
||||
class EffectiveDistributionSettings:
|
||||
"""Settings with defaults applied from environment variables."""
|
||||
dropbox_path: Optional[str]
|
||||
gdrive_folder_id: Optional[str]
|
||||
discord_webhook_url: Optional[str]
|
||||
brand_prefix: Optional[str]
|
||||
|
||||
|
||||
def _get_effective_distribution_settings(
|
||||
dropbox_path: Optional[str] = None,
|
||||
gdrive_folder_id: Optional[str] = None,
|
||||
discord_webhook_url: Optional[str] = None,
|
||||
brand_prefix: Optional[str] = None,
|
||||
) -> EffectiveDistributionSettings:
|
||||
"""
|
||||
Get distribution settings with defaults applied from environment variables.
|
||||
|
||||
This ensures consistent handling of defaults across all job creation endpoints.
|
||||
Each parameter, if not provided (None), falls back to the corresponding
|
||||
environment variable configured in settings.
|
||||
|
||||
Args:
|
||||
dropbox_path: Explicit Dropbox path or None for default
|
||||
gdrive_folder_id: Explicit Google Drive folder ID or None for default
|
||||
discord_webhook_url: Explicit Discord webhook URL or None for default
|
||||
brand_prefix: Explicit brand prefix or None for default
|
||||
|
||||
Returns:
|
||||
EffectiveDistributionSettings with defaults applied
|
||||
"""
|
||||
settings = get_settings()
|
||||
return EffectiveDistributionSettings(
|
||||
dropbox_path=dropbox_path or settings.default_dropbox_path,
|
||||
gdrive_folder_id=gdrive_folder_id or settings.default_gdrive_folder_id,
|
||||
discord_webhook_url=discord_webhook_url or settings.default_discord_webhook_url,
|
||||
brand_prefix=brand_prefix or settings.default_brand_prefix,
|
||||
)
|
||||
|
||||
|
||||
def _prepare_theme_for_job(
|
||||
job_id: str,
|
||||
theme_id: str,
|
||||
@@ -382,12 +341,9 @@ async def upload_and_create_job(
|
||||
style_cdg_instrumental_background: Optional[UploadFile] = File(None, description="CDG instrumental background"),
|
||||
style_cdg_title_background: Optional[UploadFile] = File(None, description="CDG title screen background"),
|
||||
style_cdg_outro_background: Optional[UploadFile] = File(None, description="CDG outro screen background"),
|
||||
# Theme configuration (use pre-made theme from GCS)
|
||||
theme_id: Optional[str] = Form(None, description="Theme ID to use (e.g., 'nomad', 'default'). If set, CDG/TXT are enabled by default."),
|
||||
color_overrides: Optional[str] = Form(None, description="JSON-encoded color overrides: artist_color, title_color, sung_lyrics_color, unsung_lyrics_color (hex #RRGGBB)"),
|
||||
# Processing options (CDG/TXT require style config or theme, disabled by default unless theme is set)
|
||||
enable_cdg: Optional[bool] = Form(None, description="Generate CDG+MP3 package. Default: True if theme_id set, False otherwise"),
|
||||
enable_txt: Optional[bool] = Form(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise"),
|
||||
# Processing options (CDG/TXT require style config, disabled by default)
|
||||
enable_cdg: bool = Form(False, description="Generate CDG+MP3 package (requires style config)"),
|
||||
enable_txt: bool = Form(False, description="Generate TXT+MP3 package (requires style config)"),
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Form(None, description="Brand code prefix (e.g., NOMAD)"),
|
||||
enable_youtube_upload: bool = Form(False, description="Upload to YouTube"),
|
||||
@@ -476,42 +432,38 @@ async def upload_and_create_job(
|
||||
)
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=dropbox_path,
|
||||
gdrive_folder_id=gdrive_folder_id,
|
||||
discord_webhook_url=discord_webhook_url,
|
||||
brand_prefix=brand_prefix,
|
||||
)
|
||||
|
||||
if dist.dropbox_path and not dropbox_path:
|
||||
logger.info(f"Using default dropbox_path: {dist.dropbox_path}")
|
||||
if dist.gdrive_folder_id and not gdrive_folder_id:
|
||||
logger.info(f"Using default gdrive_folder_id: {dist.gdrive_folder_id}")
|
||||
if dist.discord_webhook_url and not discord_webhook_url:
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
if effective_dropbox_path and not dropbox_path:
|
||||
logger.info(f"Using default dropbox_path: {effective_dropbox_path}")
|
||||
if effective_gdrive_folder_id and not gdrive_folder_id:
|
||||
logger.info(f"Using default gdrive_folder_id: {effective_gdrive_folder_id}")
|
||||
if effective_discord_webhook_url and not discord_webhook_url:
|
||||
logger.info("Using default discord_webhook_url (from env)")
|
||||
if dist.brand_prefix and not brand_prefix:
|
||||
logger.info(f"Using default brand_prefix: {dist.brand_prefix}")
|
||||
|
||||
|
||||
# Validate credentials for requested distribution services (including defaults)
|
||||
# This prevents accepting jobs that will fail later due to missing credentials
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if dist.dropbox_path:
|
||||
|
||||
if effective_dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -525,35 +477,7 @@ async def upload_and_create_job(
|
||||
|
||||
# Extract request metadata for tracking and filtering
|
||||
request_metadata = extract_request_metadata(request, created_from="upload")
|
||||
|
||||
# Parse color_overrides from JSON if provided
|
||||
parsed_color_overrides: Dict[str, str] = {}
|
||||
if color_overrides:
|
||||
try:
|
||||
parsed_color_overrides = json.loads(color_overrides)
|
||||
except json.JSONDecodeError as e:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid color_overrides JSON: {e}"
|
||||
)
|
||||
|
||||
# Resolve CDG/TXT defaults based on theme
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
theme_id, enable_cdg, enable_txt
|
||||
)
|
||||
|
||||
# Check if any custom style files are being uploaded (overrides theme)
|
||||
has_custom_style_files = any([
|
||||
style_params,
|
||||
style_intro_background,
|
||||
style_karaoke_background,
|
||||
style_end_background,
|
||||
style_font,
|
||||
style_cdg_instrumental_background,
|
||||
style_cdg_title_background,
|
||||
style_cdg_outro_background,
|
||||
])
|
||||
|
||||
|
||||
# Parse comma-separated model lists into arrays
|
||||
parsed_backing_vocals_models = None
|
||||
if backing_vocals_models:
|
||||
@@ -568,19 +492,17 @@ async def upload_and_create_job(
|
||||
artist=artist,
|
||||
title=title,
|
||||
filename=file.filename,
|
||||
theme_id=theme_id,
|
||||
color_overrides=parsed_color_overrides,
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_cdg=enable_cdg,
|
||||
enable_txt=enable_txt,
|
||||
brand_prefix=brand_prefix,
|
||||
enable_youtube_upload=enable_youtube_upload,
|
||||
youtube_description=youtube_description,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
webhook_url=webhook_url,
|
||||
user_email=user_email,
|
||||
# Native API distribution (preferred for remote CLI)
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
# Legacy rclone distribution (deprecated)
|
||||
organised_dir_rclone_root=organised_dir_rclone_root,
|
||||
# Lyrics configuration (overrides for search/transcription)
|
||||
@@ -599,27 +521,9 @@ async def upload_and_create_job(
|
||||
|
||||
# Record job creation metric
|
||||
metrics.record_job_created(job_id, source="upload")
|
||||
|
||||
|
||||
logger.info(f"Created job {job_id} for {artist} - {title}")
|
||||
|
||||
# If theme is set and no custom style files are being uploaded, prepare theme style now
|
||||
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
|
||||
# can access the style configuration for preview videos
|
||||
theme_style_params_path = None
|
||||
theme_style_assets = {}
|
||||
theme_youtube_desc = None
|
||||
if theme_id and not has_custom_style_files:
|
||||
try:
|
||||
theme_style_params_path, theme_style_assets, theme_youtube_desc = _prepare_theme_for_job(
|
||||
job_id, theme_id, parsed_color_overrides or None
|
||||
)
|
||||
logger.info(f"Applied theme '{theme_id}' to job {job_id}")
|
||||
except HTTPException:
|
||||
raise # Re-raise validation errors (e.g., theme not found)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to prepare theme '{theme_id}' for job {job_id}: {e}")
|
||||
# Continue without theme - job can still be processed with defaults
|
||||
|
||||
|
||||
# Upload main audio file to GCS
|
||||
audio_gcs_path = f"uploads/{job_id}/audio/{file.filename}"
|
||||
logger.info(f"Uploading audio to GCS: {audio_gcs_path}")
|
||||
@@ -688,36 +592,27 @@ async def upload_and_create_job(
|
||||
update_data = {
|
||||
'input_media_gcs_path': audio_gcs_path,
|
||||
'filename': file.filename,
|
||||
'enable_cdg': resolved_cdg,
|
||||
'enable_txt': resolved_txt,
|
||||
'enable_cdg': enable_cdg,
|
||||
'enable_txt': enable_txt,
|
||||
}
|
||||
|
||||
# Handle style assets - either from custom uploads or from theme
|
||||
|
||||
if style_assets:
|
||||
# Custom style files uploaded
|
||||
update_data['style_assets'] = style_assets
|
||||
if 'style_params' in style_assets:
|
||||
update_data['style_params_gcs_path'] = style_assets['style_params']
|
||||
elif theme_style_assets:
|
||||
# Theme style assets (no custom uploads)
|
||||
update_data['style_assets'] = theme_style_assets
|
||||
if theme_style_params_path:
|
||||
update_data['style_params_gcs_path'] = theme_style_params_path
|
||||
|
||||
if dist.brand_prefix:
|
||||
update_data['brand_prefix'] = dist.brand_prefix
|
||||
if dist.discord_webhook_url:
|
||||
update_data['discord_webhook_url'] = dist.discord_webhook_url
|
||||
# Use theme YouTube description if no custom one provided
|
||||
effective_youtube_description = youtube_description or theme_youtube_desc
|
||||
if effective_youtube_description:
|
||||
update_data['youtube_description_template'] = effective_youtube_description
|
||||
|
||||
|
||||
if brand_prefix:
|
||||
update_data['brand_prefix'] = brand_prefix
|
||||
if effective_discord_webhook_url:
|
||||
update_data['discord_webhook_url'] = effective_discord_webhook_url
|
||||
if youtube_description:
|
||||
update_data['youtube_description_template'] = youtube_description
|
||||
|
||||
# Native API distribution (use effective values which include defaults)
|
||||
if dist.dropbox_path:
|
||||
update_data['dropbox_path'] = dist.dropbox_path
|
||||
if dist.gdrive_folder_id:
|
||||
update_data['gdrive_folder_id'] = dist.gdrive_folder_id
|
||||
if effective_dropbox_path:
|
||||
update_data['dropbox_path'] = effective_dropbox_path
|
||||
if effective_gdrive_folder_id:
|
||||
update_data['gdrive_folder_id'] = effective_gdrive_folder_id
|
||||
|
||||
# Legacy rclone distribution (deprecated)
|
||||
if organised_dir_rclone_root:
|
||||
@@ -773,21 +668,21 @@ async def upload_and_create_job(
|
||||
|
||||
# Build distribution services info for response
|
||||
distribution_services: Dict[str, Any] = {}
|
||||
|
||||
if dist.dropbox_path:
|
||||
|
||||
if effective_dropbox_path:
|
||||
dropbox_result = credential_manager.check_dropbox_credentials()
|
||||
distribution_services["dropbox"] = {
|
||||
"enabled": True,
|
||||
"path": dist.dropbox_path,
|
||||
"path": effective_dropbox_path,
|
||||
"credentials_valid": dropbox_result.status == CredentialStatus.VALID,
|
||||
"using_default": dropbox_path is None,
|
||||
}
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
gdrive_result = credential_manager.check_gdrive_credentials()
|
||||
distribution_services["gdrive"] = {
|
||||
"enabled": True,
|
||||
"folder_id": dist.gdrive_folder_id,
|
||||
"folder_id": effective_gdrive_folder_id,
|
||||
"credentials_valid": gdrive_result.status == CredentialStatus.VALID,
|
||||
"using_default": gdrive_folder_id is None,
|
||||
}
|
||||
@@ -799,7 +694,7 @@ async def upload_and_create_job(
|
||||
"credentials_valid": youtube_result.status == CredentialStatus.VALID,
|
||||
}
|
||||
|
||||
if dist.discord_webhook_url:
|
||||
if effective_discord_webhook_url:
|
||||
distribution_services["discord"] = {
|
||||
"enabled": True,
|
||||
"using_default": discord_webhook_url is None,
|
||||
@@ -1016,34 +911,32 @@ async def create_job_with_upload_urls(
|
||||
status_code=400,
|
||||
detail=f"Invalid file extension '{ext}' for file_type '{file_info.file_type}'. Allowed: {allowed_extensions}"
|
||||
)
|
||||
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if dist.dropbox_path:
|
||||
|
||||
if effective_dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1054,7 +947,7 @@ async def create_job_with_upload_urls(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata for tracking
|
||||
request_metadata = extract_request_metadata(request, created_from="signed_url_upload")
|
||||
|
||||
@@ -1078,15 +971,14 @@ async def create_job_with_upload_urls(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
brand_prefix=body.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
youtube_description_template=body.youtube_description, # video_worker reads this field
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
webhook_url=body.webhook_url,
|
||||
user_email=body.user_email,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
organised_dir_rclone_root=body.organised_dir_rclone_root,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
@@ -1430,34 +1322,32 @@ async def create_job_from_url(
|
||||
status_code=400,
|
||||
detail="Invalid URL. Please provide a valid YouTube, Vimeo, SoundCloud, or other supported video URL."
|
||||
)
|
||||
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if dist.dropbox_path:
|
||||
|
||||
if effective_dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1468,7 +1358,7 @@ async def create_job_from_url(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata for tracking
|
||||
request_metadata = extract_request_metadata(request, created_from="url")
|
||||
|
||||
@@ -1491,14 +1381,14 @@ async def create_job_from_url(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
brand_prefix=body.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
webhook_url=body.webhook_url,
|
||||
user_email=body.user_email,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
organised_dir_rclone_root=body.organised_dir_rclone_root,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
@@ -1673,32 +1563,30 @@ async def create_finalise_only_job(
|
||||
)
|
||||
|
||||
# Apply default distribution settings
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
# Validate distribution credentials if services are requested
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if dist.dropbox_path:
|
||||
|
||||
if effective_dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1709,7 +1597,7 @@ async def create_finalise_only_job(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata
|
||||
request_metadata = extract_request_metadata(request, created_from="finalise_only_upload")
|
||||
|
||||
@@ -1730,12 +1618,12 @@ async def create_finalise_only_job(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
brand_prefix=body.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
finalise_only=True,
|
||||
keep_brand_code=body.keep_brand_code,
|
||||
request_metadata=request_metadata,
|
||||
|
||||
@@ -8,8 +8,6 @@ from typing import Dict, Any, Optional
|
||||
|
||||
from backend.version import VERSION
|
||||
from backend.services.flacfetch_client import get_flacfetch_client
|
||||
from backend.services.email_service import get_email_service
|
||||
from backend.services.stripe_service import get_stripe_service
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -105,42 +103,23 @@ async def check_flacfetch_service_status() -> Dict[str, Any]:
|
||||
async def detailed_health_check() -> Dict[str, Any]:
|
||||
"""
|
||||
Detailed health check including dependencies.
|
||||
|
||||
|
||||
Use this to debug issues with Transmission, flacfetch service, etc.
|
||||
"""
|
||||
transmission_status = check_transmission_status()
|
||||
flacfetch_status = await check_flacfetch_service_status()
|
||||
|
||||
# Check email service
|
||||
email_service = get_email_service()
|
||||
email_status = {
|
||||
"configured": email_service.is_configured(),
|
||||
"provider": type(email_service.provider).__name__,
|
||||
}
|
||||
|
||||
# Check Stripe service
|
||||
stripe_service = get_stripe_service()
|
||||
stripe_status = {
|
||||
"configured": stripe_service.is_configured(),
|
||||
}
|
||||
|
||||
|
||||
# Log for debugging
|
||||
if not transmission_status.get("available"):
|
||||
logger.warning(f"Local Transmission not available: {transmission_status.get('error')}")
|
||||
else:
|
||||
logger.info(f"Local Transmission available at {transmission_status.get('host')}:{transmission_status.get('port')}")
|
||||
|
||||
|
||||
if flacfetch_status.get("configured") and not flacfetch_status.get("available"):
|
||||
logger.warning(f"Remote flacfetch service not available: {flacfetch_status.get('error')}")
|
||||
elif flacfetch_status.get("available"):
|
||||
logger.info(f"Remote flacfetch service healthy: {flacfetch_status.get('status')}")
|
||||
|
||||
if not email_status["configured"]:
|
||||
logger.warning("Email service not configured - magic links will not work")
|
||||
|
||||
if not stripe_status["configured"]:
|
||||
logger.warning("Stripe service not configured - payments will not work")
|
||||
|
||||
|
||||
return {
|
||||
"status": "healthy",
|
||||
"service": "karaoke-gen-backend",
|
||||
@@ -148,10 +127,6 @@ async def detailed_health_check() -> Dict[str, Any]:
|
||||
"dependencies": {
|
||||
"transmission_local": transmission_status,
|
||||
"flacfetch_remote": flacfetch_status,
|
||||
},
|
||||
"services": {
|
||||
"email": email_status,
|
||||
"stripe": stripe_status,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+49
-161
@@ -27,7 +27,7 @@ from backend.services.worker_service import get_worker_service
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.config import get_settings
|
||||
from backend.api.dependencies import require_admin, require_auth, require_instrumental_auth
|
||||
from backend.services.auth_service import UserType, AuthResult
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.services.metrics import metrics
|
||||
|
||||
|
||||
@@ -57,7 +57,7 @@ async def _trigger_workers_parallel(job_id: str) -> None:
|
||||
async def create_job(
|
||||
request: URLSubmissionRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> JobResponse:
|
||||
"""
|
||||
Create a new karaoke generation job from a URL.
|
||||
@@ -69,24 +69,6 @@ async def create_job(
|
||||
4. When both complete, job transitions to AWAITING_REVIEW
|
||||
"""
|
||||
try:
|
||||
# Determine job owner email:
|
||||
# All authentication methods must provide a user_email for job ownership
|
||||
if auth_result.user_email:
|
||||
# Use authenticated user's email (standard case)
|
||||
user_email = auth_result.user_email
|
||||
else:
|
||||
# This should never happen - all auth methods now require user_email
|
||||
logger.error("Authentication succeeded but no user_email provided")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="Authentication error: no user identity available"
|
||||
)
|
||||
|
||||
# Admins can optionally create jobs on behalf of other users
|
||||
if request.user_email and auth_result.is_admin and request.user_email != auth_result.user_email:
|
||||
user_email = request.user_email
|
||||
logger.info(f"Admin {auth_result.user_email} creating job on behalf of {user_email}")
|
||||
|
||||
# Create job with all preferences
|
||||
job_create = JobCreate(
|
||||
url=str(request.url),
|
||||
@@ -97,7 +79,7 @@ async def create_job(
|
||||
enable_youtube_upload=request.enable_youtube_upload,
|
||||
youtube_description=request.youtube_description,
|
||||
webhook_url=request.webhook_url,
|
||||
user_email=user_email
|
||||
user_email=request.user_email
|
||||
)
|
||||
job = job_manager.create_job(job_create)
|
||||
|
||||
@@ -127,49 +109,20 @@ async def create_job(
|
||||
@router.get("/{job_id}", response_model=Job)
|
||||
async def get_job(
|
||||
job_id: str,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> Job:
|
||||
"""Get job status and details."""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership - users can only see their own jobs, admins can see all
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
|
||||
# If job is complete, include download URLs
|
||||
if job.status == JobStatus.COMPLETE:
|
||||
job.download_urls = job_manager.get_output_urls(job_id)
|
||||
|
||||
|
||||
return job
|
||||
|
||||
|
||||
def _check_job_ownership(job: Job, auth_result: AuthResult) -> bool:
|
||||
"""
|
||||
Check if the authenticated user owns the job or has admin access.
|
||||
|
||||
Returns:
|
||||
True if user can access the job, False otherwise
|
||||
"""
|
||||
# Admins can access all jobs
|
||||
if auth_result.is_admin:
|
||||
return True
|
||||
|
||||
# Check if user owns the job
|
||||
if auth_result.user_email and job.user_email:
|
||||
return auth_result.user_email.lower() == job.user_email.lower()
|
||||
|
||||
# If no user_email on auth (token auth without email), deny access to jobs with user_email
|
||||
# This prevents token-based auth from accessing user jobs
|
||||
if job.user_email:
|
||||
return False
|
||||
|
||||
# Legacy jobs without user_email - allow access for backward compatibility
|
||||
# TODO: Consider restricting this in the future
|
||||
return True
|
||||
|
||||
|
||||
@router.get("", response_model=List[Job])
|
||||
async def list_jobs(
|
||||
status: Optional[JobStatus] = None,
|
||||
@@ -178,13 +131,11 @@ async def list_jobs(
|
||||
created_after: Optional[str] = None,
|
||||
created_before: Optional[str] = None,
|
||||
limit: int = 100,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> List[Job]:
|
||||
"""
|
||||
List jobs with optional filters.
|
||||
|
||||
Regular users only see their own jobs. Admins see all jobs.
|
||||
|
||||
|
||||
Args:
|
||||
status: Filter by job status (pending, complete, failed, etc.)
|
||||
environment: Filter by request_metadata.environment (test/production/development)
|
||||
@@ -192,52 +143,37 @@ async def list_jobs(
|
||||
created_after: Filter jobs created after this ISO datetime (e.g., 2024-01-01T00:00:00Z)
|
||||
created_before: Filter jobs created before this ISO datetime
|
||||
limit: Maximum number of jobs to return (default 100)
|
||||
|
||||
|
||||
Returns:
|
||||
List of jobs matching filters, ordered by created_at descending
|
||||
"""
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
try:
|
||||
# Parse datetime strings if provided
|
||||
created_after_dt = None
|
||||
created_before_dt = None
|
||||
|
||||
|
||||
if created_after:
|
||||
try:
|
||||
created_after_dt = datetime.fromisoformat(created_after.replace('Z', '+00:00'))
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_after format: {created_after}") from e
|
||||
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_after format: {created_after}")
|
||||
|
||||
if created_before:
|
||||
try:
|
||||
created_before_dt = datetime.fromisoformat(created_before.replace('Z', '+00:00'))
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_before format: {created_before}") from e
|
||||
|
||||
# Determine user_email filter based on admin status
|
||||
# Admins see all jobs, regular users only see their own
|
||||
user_email_filter = None
|
||||
if not auth_result.is_admin:
|
||||
if auth_result.user_email:
|
||||
user_email_filter = auth_result.user_email
|
||||
logger.debug(f"Filtering jobs for user: {user_email_filter}")
|
||||
else:
|
||||
# Token-based auth without user email - show no jobs for security
|
||||
logger.warning("Non-admin auth without user_email, returning empty job list")
|
||||
return []
|
||||
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_before format: {created_before}")
|
||||
|
||||
jobs = job_manager.list_jobs(
|
||||
status=status,
|
||||
environment=environment,
|
||||
client_id=client_id,
|
||||
created_after=created_after_dt,
|
||||
created_before=created_before_dt,
|
||||
user_email=user_email_filter,
|
||||
limit=limit
|
||||
)
|
||||
|
||||
logger.debug(f"Listed {len(jobs)} jobs for user={auth_result.user_email}, admin={auth_result.is_admin}")
|
||||
return jobs
|
||||
except HTTPException:
|
||||
raise
|
||||
@@ -250,20 +186,16 @@ async def list_jobs(
|
||||
async def delete_job(
|
||||
job_id: str,
|
||||
delete_files: bool = True,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""Delete a job and optionally its output files."""
|
||||
try:
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership - users can only delete their own jobs
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to delete this job")
|
||||
|
||||
|
||||
job_manager.delete_job(job_id, delete_files=delete_files)
|
||||
|
||||
|
||||
return {"status": "success", "message": f"Job {job_id} deleted"}
|
||||
except HTTPException:
|
||||
raise
|
||||
@@ -280,7 +212,7 @@ async def bulk_delete_jobs(
|
||||
created_before: Optional[str] = None,
|
||||
delete_files: bool = True,
|
||||
confirm: bool = False,
|
||||
_auth_result: AuthResult = Depends(require_admin)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
|
||||
) -> dict:
|
||||
"""
|
||||
Delete multiple jobs matching filter criteria.
|
||||
@@ -384,22 +316,18 @@ async def bulk_delete_jobs(
|
||||
@router.get("/{job_id}/review-data")
|
||||
async def get_review_data(
|
||||
job_id: str,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Get data needed for lyrics review interface.
|
||||
|
||||
|
||||
Returns corrections JSON URL and audio URL.
|
||||
Frontend loads these to render the review UI.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
|
||||
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -448,21 +376,13 @@ async def get_review_data(
|
||||
async def start_review(
|
||||
job_id: str,
|
||||
request: StartReviewRequest,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Mark job as IN_REVIEW (user opened review interface).
|
||||
|
||||
|
||||
This helps track that the user is actively working on the review.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
success = job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.IN_REVIEW,
|
||||
@@ -480,24 +400,20 @@ async def submit_corrections(
|
||||
job_id: str,
|
||||
submission: CorrectionsSubmission,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Save corrected lyrics during human review.
|
||||
|
||||
|
||||
This endpoint saves review progress but does NOT complete the review.
|
||||
Call POST /{job_id}/complete-review to finish and trigger video rendering.
|
||||
|
||||
|
||||
Can be called multiple times to save progress.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to modify this job")
|
||||
|
||||
|
||||
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -543,11 +459,11 @@ async def submit_corrections(
|
||||
async def complete_review(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Complete the human review and trigger video rendering.
|
||||
|
||||
|
||||
This is the FIRST critical human-in-the-loop completion point.
|
||||
After this:
|
||||
1. Job transitions to REVIEW_COMPLETE
|
||||
@@ -558,11 +474,7 @@ async def complete_review(
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to modify this job")
|
||||
|
||||
|
||||
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1036,22 +948,18 @@ async def select_instrumental(
|
||||
@router.get("/{job_id}/download-urls")
|
||||
async def get_download_urls(
|
||||
job_id: str,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Get download URLs for all job output files.
|
||||
|
||||
|
||||
Returns a dictionary mapping file types to download URLs.
|
||||
Uses the streaming download endpoint which proxies through the backend.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
|
||||
if job.status != JobStatus.COMPLETE:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1087,25 +995,21 @@ async def download_file(
|
||||
job_id: str,
|
||||
category: str,
|
||||
file_key: str,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Stream download a specific file from a completed job.
|
||||
|
||||
|
||||
This endpoint proxies the file from GCS through the backend,
|
||||
so no client-side authentication is required.
|
||||
"""
|
||||
from fastapi.responses import StreamingResponse
|
||||
import tempfile
|
||||
|
||||
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
|
||||
if job.status != JobStatus.COMPLETE:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1182,21 +1086,13 @@ async def download_file(
|
||||
async def cancel_job(
|
||||
job_id: str,
|
||||
request: CancelJobRequest,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Cancel a job.
|
||||
|
||||
|
||||
Jobs can be cancelled at any stage before completion.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to cancel this job")
|
||||
|
||||
success = job_manager.cancel_job(job_id, reason=request.reason)
|
||||
|
||||
if not success:
|
||||
@@ -1213,7 +1109,7 @@ async def cancel_job(
|
||||
async def retry_job(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Retry a failed or cancelled job from the last successful checkpoint.
|
||||
@@ -1231,10 +1127,6 @@ async def retry_job(
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to retry this job")
|
||||
|
||||
if job.status not in [JobStatus.FAILED, JobStatus.CANCELLED]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1409,19 +1301,19 @@ async def get_worker_logs(
|
||||
job_id: str,
|
||||
since_index: int = 0,
|
||||
worker: Optional[str] = None,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Get worker logs for debugging.
|
||||
|
||||
|
||||
This endpoint returns worker logs stored in Firestore.
|
||||
Use `since_index` for efficient polling (returns only new logs).
|
||||
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
since_index: Return only logs after this index (for pagination/polling)
|
||||
worker: Filter by worker name (audio, lyrics, screens, video, render)
|
||||
|
||||
|
||||
Returns:
|
||||
{
|
||||
"logs": [{"timestamp": "...", "level": "INFO", "worker": "audio", "message": "..."}],
|
||||
@@ -1432,14 +1324,10 @@ async def get_worker_logs(
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership - users can only see logs for their own jobs
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access logs for this job")
|
||||
|
||||
|
||||
logs = job_manager.get_worker_logs(job_id, since_index=since_index, worker=worker)
|
||||
total = len(job.worker_logs) if job.worker_logs else 0
|
||||
|
||||
|
||||
return {
|
||||
"logs": logs,
|
||||
"next_index": total,
|
||||
|
||||
@@ -1,835 +0,0 @@
|
||||
"""
|
||||
User and authentication API routes.
|
||||
|
||||
Handles:
|
||||
- Magic link authentication (send, verify)
|
||||
- Session management (logout)
|
||||
- User profile and credits
|
||||
- Stripe checkout and webhooks
|
||||
- Admin user management
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional, Tuple
|
||||
from fastapi import APIRouter, HTTPException, Depends, Request, Header
|
||||
from pydantic import BaseModel, EmailStr
|
||||
|
||||
from backend.models.user import (
|
||||
UserRole,
|
||||
UserPublic,
|
||||
SendMagicLinkRequest,
|
||||
SendMagicLinkResponse,
|
||||
VerifyMagicLinkResponse,
|
||||
AddCreditsRequest,
|
||||
AddCreditsResponse,
|
||||
UserListResponse,
|
||||
BetaTesterStatus,
|
||||
BetaTesterEnrollRequest,
|
||||
BetaTesterEnrollResponse,
|
||||
BetaFeedbackRequest,
|
||||
BetaFeedbackResponse,
|
||||
BetaTesterFeedback,
|
||||
)
|
||||
from backend.services.user_service import get_user_service, UserService
|
||||
from backend.services.email_service import get_email_service, EmailService
|
||||
from backend.services.stripe_service import get_stripe_service, StripeService, CREDIT_PACKAGES
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.services.auth_service import UserType
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/users", tags=["users"])
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Request/Response Models
|
||||
# =============================================================================
|
||||
|
||||
class CreateCheckoutRequest(BaseModel):
|
||||
"""Request to create a Stripe checkout session."""
|
||||
package_id: str
|
||||
email: EmailStr
|
||||
|
||||
|
||||
class CreateCheckoutResponse(BaseModel):
|
||||
"""Response with checkout URL."""
|
||||
status: str
|
||||
checkout_url: str
|
||||
message: str
|
||||
|
||||
|
||||
class CreditPackage(BaseModel):
|
||||
"""Credit package information."""
|
||||
id: str
|
||||
credits: int
|
||||
price_cents: int
|
||||
name: str
|
||||
description: str
|
||||
|
||||
|
||||
class CreditPackagesResponse(BaseModel):
|
||||
"""Response listing available credit packages."""
|
||||
packages: list[CreditPackage]
|
||||
|
||||
|
||||
class UserProfileResponse(BaseModel):
|
||||
"""Response with user profile."""
|
||||
user: UserPublic
|
||||
has_session: bool
|
||||
|
||||
|
||||
class LogoutResponse(BaseModel):
|
||||
"""Response after logout."""
|
||||
status: str
|
||||
message: str
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Magic Link Authentication
|
||||
# =============================================================================
|
||||
|
||||
@router.post("/auth/magic-link", response_model=SendMagicLinkResponse)
|
||||
async def send_magic_link(
|
||||
request: SendMagicLinkRequest,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Send a magic link email for passwordless authentication.
|
||||
|
||||
The user will receive an email with a link that logs them in.
|
||||
Links expire after 15 minutes and can only be used once.
|
||||
"""
|
||||
# Check if email service is configured
|
||||
if not email_service.is_configured():
|
||||
logger.error("Email service not configured - cannot send magic links")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="Email service is not available. Please contact support."
|
||||
)
|
||||
|
||||
email = request.email.lower()
|
||||
|
||||
# Get client info for security logging
|
||||
ip_address = http_request.client.host if http_request.client else None
|
||||
user_agent = http_request.headers.get("user-agent")
|
||||
|
||||
# Create magic link token
|
||||
magic_link = user_service.create_magic_link(
|
||||
email,
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent
|
||||
)
|
||||
|
||||
# Send email
|
||||
sent = email_service.send_magic_link(email, magic_link.token)
|
||||
|
||||
if not sent:
|
||||
logger.error(f"Failed to send magic link email to {email}")
|
||||
# Don't reveal failure to prevent email enumeration
|
||||
# Still return success
|
||||
|
||||
return SendMagicLinkResponse(
|
||||
status="success",
|
||||
message="If this email is registered, you will receive a sign-in link shortly."
|
||||
)
|
||||
|
||||
|
||||
@router.get("/auth/verify", response_model=VerifyMagicLinkResponse)
|
||||
async def verify_magic_link(
|
||||
token: str,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Verify a magic link token and create a session.
|
||||
|
||||
Returns a session token that should be stored and used for subsequent requests.
|
||||
"""
|
||||
# Get client info
|
||||
ip_address = http_request.client.host if http_request.client else None
|
||||
user_agent = http_request.headers.get("user-agent")
|
||||
|
||||
# Check if this is a first login BEFORE verification (which sets last_login_at)
|
||||
# We need to get the user's state before verify_magic_link updates it
|
||||
from backend.services.user_service import MAGIC_LINKS_COLLECTION
|
||||
magic_link_doc = user_service.db.collection(MAGIC_LINKS_COLLECTION).document(token).get()
|
||||
is_first_login = False
|
||||
if magic_link_doc.exists:
|
||||
magic_link_data = magic_link_doc.to_dict()
|
||||
pre_verify_user = user_service.get_user(magic_link_data.get('email', ''))
|
||||
if pre_verify_user:
|
||||
is_first_login = pre_verify_user.total_jobs_created == 0 and not pre_verify_user.last_login_at
|
||||
|
||||
# Verify the magic link
|
||||
success, user, message = user_service.verify_magic_link(token)
|
||||
|
||||
if not success or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
# Create session
|
||||
session = user_service.create_session(
|
||||
user.email,
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent
|
||||
)
|
||||
|
||||
# Send welcome email to first-time users
|
||||
if is_first_login:
|
||||
email_service.send_welcome_email(user.email, user.credits)
|
||||
|
||||
# Return user info
|
||||
user_public = UserPublic(
|
||||
email=user.email,
|
||||
role=user.role,
|
||||
credits=user.credits,
|
||||
display_name=user.display_name,
|
||||
total_jobs_created=user.total_jobs_created,
|
||||
total_jobs_completed=user.total_jobs_completed,
|
||||
)
|
||||
|
||||
return VerifyMagicLinkResponse(
|
||||
status="success",
|
||||
session_token=session.token,
|
||||
user=user_public,
|
||||
message="Successfully signed in"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/auth/logout", response_model=LogoutResponse)
|
||||
async def logout(
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Logout and invalidate the current session.
|
||||
"""
|
||||
if not authorization:
|
||||
return LogoutResponse(status="success", message="Already logged out")
|
||||
|
||||
# Extract token from "Bearer <token>"
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
|
||||
user_service.revoke_session(token)
|
||||
|
||||
return LogoutResponse(status="success", message="Successfully logged out")
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# User Profile
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/me", response_model=UserProfileResponse)
|
||||
async def get_current_user(
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Get the current user's profile.
|
||||
|
||||
Requires a valid session token.
|
||||
"""
|
||||
if not authorization:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
|
||||
# Extract token
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
|
||||
# Validate session
|
||||
valid, user, message = user_service.validate_session(token)
|
||||
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
user_public = UserPublic(
|
||||
email=user.email,
|
||||
role=user.role,
|
||||
credits=user.credits,
|
||||
display_name=user.display_name,
|
||||
total_jobs_created=user.total_jobs_created,
|
||||
total_jobs_completed=user.total_jobs_completed,
|
||||
)
|
||||
|
||||
return UserProfileResponse(user=user_public, has_session=True)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Credit Packages & Checkout
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/credits/packages", response_model=CreditPackagesResponse)
|
||||
async def list_credit_packages():
|
||||
"""
|
||||
List available credit packages for purchase.
|
||||
|
||||
No authentication required - this is public information.
|
||||
"""
|
||||
packages = [
|
||||
CreditPackage(
|
||||
id=pkg_id,
|
||||
credits=pkg["credits"],
|
||||
price_cents=pkg["price_cents"],
|
||||
name=pkg["name"],
|
||||
description=pkg["description"],
|
||||
)
|
||||
for pkg_id, pkg in CREDIT_PACKAGES.items()
|
||||
]
|
||||
|
||||
return CreditPackagesResponse(packages=packages)
|
||||
|
||||
|
||||
@router.post("/credits/checkout", response_model=CreateCheckoutResponse)
|
||||
async def create_checkout(
|
||||
request: CreateCheckoutRequest,
|
||||
stripe_service: StripeService = Depends(get_stripe_service),
|
||||
):
|
||||
"""
|
||||
Create a Stripe checkout session for purchasing credits.
|
||||
|
||||
Returns a URL to redirect the user to Stripe's hosted checkout page.
|
||||
No authentication required - email is provided in the request.
|
||||
"""
|
||||
if not stripe_service.is_configured():
|
||||
raise HTTPException(status_code=503, detail="Payment processing is not available")
|
||||
|
||||
success, checkout_url, message = stripe_service.create_checkout_session(
|
||||
package_id=request.package_id,
|
||||
user_email=request.email,
|
||||
)
|
||||
|
||||
if not success or not checkout_url:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
return CreateCheckoutResponse(
|
||||
status="success",
|
||||
checkout_url=checkout_url,
|
||||
message=message,
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Stripe Webhooks
|
||||
# =============================================================================
|
||||
|
||||
@router.post("/webhooks/stripe")
|
||||
async def stripe_webhook(
|
||||
request: Request,
|
||||
stripe_signature: str = Header(None, alias="stripe-signature"),
|
||||
stripe_service: StripeService = Depends(get_stripe_service),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Handle Stripe webhook events.
|
||||
|
||||
This endpoint receives events from Stripe about payment status.
|
||||
It verifies the webhook signature and processes the event.
|
||||
"""
|
||||
if not stripe_signature:
|
||||
raise HTTPException(status_code=400, detail="Missing Stripe signature")
|
||||
|
||||
# Get raw body for signature verification
|
||||
payload = await request.body()
|
||||
|
||||
# Verify signature
|
||||
valid, event, message = stripe_service.verify_webhook_signature(payload, stripe_signature)
|
||||
|
||||
if not valid:
|
||||
logger.warning(f"Invalid Stripe webhook signature: {message}")
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
# Handle the event
|
||||
event_type = event.get("type")
|
||||
logger.info(f"Received Stripe webhook: {event_type}")
|
||||
|
||||
if event_type == "checkout.session.completed":
|
||||
session = event["data"]["object"]
|
||||
session_id = session.get("id")
|
||||
|
||||
# Idempotency check: Skip if this session was already processed
|
||||
if session_id and user_service.is_stripe_session_processed(session_id):
|
||||
logger.info(f"Skipping already processed session: {session_id}")
|
||||
return {"status": "received", "type": event_type, "note": "already_processed"}
|
||||
|
||||
# Process the completed checkout
|
||||
success, user_email, credits, _ = stripe_service.handle_checkout_completed(session)
|
||||
|
||||
if success and user_email and credits > 0:
|
||||
# Add credits to user account
|
||||
ok, new_balance, credit_msg = user_service.add_credits(
|
||||
email=user_email,
|
||||
amount=credits,
|
||||
reason="stripe_purchase",
|
||||
stripe_session_id=session_id,
|
||||
)
|
||||
|
||||
if ok:
|
||||
# Send confirmation email
|
||||
email_service.send_credits_added(user_email, credits, new_balance)
|
||||
logger.info(f"Added {credits} credits to {user_email}, new balance: {new_balance}")
|
||||
else:
|
||||
logger.error(f"Failed to add credits: {credit_msg}")
|
||||
|
||||
elif event_type == "checkout.session.expired":
|
||||
logger.info(f"Checkout session expired: {event['data']['object'].get('id')}")
|
||||
|
||||
elif event_type == "payment_intent.payment_failed":
|
||||
logger.warning(f"Payment failed: {event['data']['object'].get('id')}")
|
||||
|
||||
# Return 200 to acknowledge receipt (Stripe will retry on non-2xx)
|
||||
return {"status": "received", "type": event_type}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Beta Tester Program
|
||||
# =============================================================================
|
||||
|
||||
BETA_TESTER_FREE_CREDITS = 1 # Number of free credits for beta testers
|
||||
|
||||
|
||||
@router.post("/beta/enroll", response_model=BetaTesterEnrollResponse)
|
||||
async def enroll_beta_tester(
|
||||
request: BetaTesterEnrollRequest,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Enroll as a beta tester to receive free karaoke credits.
|
||||
|
||||
Requirements:
|
||||
- Accept that there may be work to review/correct lyrics
|
||||
- Promise to provide feedback after using the tool
|
||||
|
||||
Returns free credits and optionally a session token for new users.
|
||||
"""
|
||||
# Check if email service is configured
|
||||
if not email_service.is_configured():
|
||||
logger.error("Email service not configured - cannot send beta welcome emails")
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="Email service is not available. Please contact support."
|
||||
)
|
||||
|
||||
email = request.email.lower()
|
||||
|
||||
# Validate acceptance
|
||||
if not request.accept_corrections_work:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="You must accept that you may need to review/correct lyrics"
|
||||
)
|
||||
|
||||
if len(request.promise_text.strip()) < 10:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="Please write a sentence confirming your promise to provide feedback"
|
||||
)
|
||||
|
||||
# Get or create user
|
||||
user = user_service.get_or_create_user(email)
|
||||
|
||||
# Check if already enrolled as beta tester
|
||||
if user.is_beta_tester:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="You are already enrolled in the beta program"
|
||||
)
|
||||
|
||||
# Get client info
|
||||
ip_address = http_request.client.host if http_request.client else None
|
||||
user_agent = http_request.headers.get("user-agent")
|
||||
|
||||
# Enroll as beta tester
|
||||
from datetime import datetime
|
||||
user_service.update_user(
|
||||
email,
|
||||
is_beta_tester=True,
|
||||
beta_tester_status=BetaTesterStatus.ACTIVE.value,
|
||||
beta_enrolled_at=datetime.utcnow(),
|
||||
beta_promise_text=request.promise_text.strip(),
|
||||
)
|
||||
|
||||
# Add free credits
|
||||
_, new_balance, _ = user_service.add_credits(
|
||||
email=email,
|
||||
amount=BETA_TESTER_FREE_CREDITS,
|
||||
reason="beta_tester_enrollment",
|
||||
)
|
||||
|
||||
# Create session for the user (so they can start using the service immediately)
|
||||
session = user_service.create_session(email, ip_address=ip_address, user_agent=user_agent)
|
||||
|
||||
# Send welcome email
|
||||
email_service.send_beta_welcome_email(email, BETA_TESTER_FREE_CREDITS)
|
||||
|
||||
logger.info(f"Beta tester enrolled: {email}, granted {BETA_TESTER_FREE_CREDITS} credits")
|
||||
|
||||
return BetaTesterEnrollResponse(
|
||||
status="success",
|
||||
message=f"Welcome to the beta program! You have {new_balance} free credits.",
|
||||
credits_granted=BETA_TESTER_FREE_CREDITS,
|
||||
session_token=session.token,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/beta/feedback", response_model=BetaFeedbackResponse)
|
||||
async def submit_beta_feedback(
|
||||
request: BetaFeedbackRequest,
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Submit feedback as a beta tester.
|
||||
|
||||
Requires authentication. Updates beta tester status to completed.
|
||||
May grant bonus credits for detailed feedback.
|
||||
"""
|
||||
if not authorization:
|
||||
raise HTTPException(status_code=401, detail="Authentication required")
|
||||
|
||||
# Extract token
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
|
||||
# Validate session
|
||||
valid, user, message = user_service.validate_session(token)
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
# Check if user is a beta tester
|
||||
if not user.is_beta_tester:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="Only beta testers can submit feedback through this endpoint"
|
||||
)
|
||||
|
||||
# Check if already completed feedback
|
||||
if user.beta_tester_status == BetaTesterStatus.COMPLETED:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="You have already submitted feedback. Thank you!"
|
||||
)
|
||||
|
||||
# Save feedback to Firestore
|
||||
import uuid
|
||||
|
||||
feedback = BetaTesterFeedback(
|
||||
id=str(uuid.uuid4()),
|
||||
user_email=user.email,
|
||||
job_id=request.job_id,
|
||||
overall_rating=request.overall_rating,
|
||||
ease_of_use_rating=request.ease_of_use_rating,
|
||||
lyrics_accuracy_rating=request.lyrics_accuracy_rating,
|
||||
correction_experience_rating=request.correction_experience_rating,
|
||||
what_went_well=request.what_went_well,
|
||||
what_could_improve=request.what_could_improve,
|
||||
additional_comments=request.additional_comments,
|
||||
would_recommend=request.would_recommend,
|
||||
would_use_again=request.would_use_again,
|
||||
submitted_via="web",
|
||||
)
|
||||
|
||||
# Save to Firestore
|
||||
user_service.db.collection("beta_feedback").document(feedback.id).set(
|
||||
feedback.model_dump(mode='json')
|
||||
)
|
||||
|
||||
# Update user status
|
||||
user_service.update_user(
|
||||
user.email,
|
||||
beta_tester_status=BetaTesterStatus.COMPLETED.value,
|
||||
)
|
||||
|
||||
# Calculate bonus credits for detailed feedback
|
||||
bonus_credits = 0
|
||||
has_detailed_feedback = (
|
||||
(request.what_went_well and len(request.what_went_well) > 50) or
|
||||
(request.what_could_improve and len(request.what_could_improve) > 50) or
|
||||
(request.additional_comments and len(request.additional_comments) > 50)
|
||||
)
|
||||
|
||||
if has_detailed_feedback:
|
||||
# Grant bonus credit for detailed feedback
|
||||
bonus_credits = 1
|
||||
user_service.add_credits(
|
||||
email=user.email,
|
||||
amount=bonus_credits,
|
||||
reason="beta_feedback_bonus",
|
||||
)
|
||||
|
||||
logger.info(f"Beta feedback received from {user.email}, bonus: {bonus_credits}")
|
||||
|
||||
return BetaFeedbackResponse(
|
||||
status="success",
|
||||
message="Thank you for your feedback!" + (
|
||||
f" You earned {bonus_credits} bonus credit for your detailed response!"
|
||||
if bonus_credits > 0 else ""
|
||||
),
|
||||
bonus_credits=bonus_credits,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/beta/feedback-form")
|
||||
async def get_feedback_form_data(
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Get data needed to show the feedback form.
|
||||
|
||||
Returns whether the user needs to submit feedback and any job context.
|
||||
"""
|
||||
if not authorization:
|
||||
raise HTTPException(status_code=401, detail="Authentication required")
|
||||
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
valid, user, message = user_service.validate_session(token)
|
||||
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
return {
|
||||
"is_beta_tester": user.is_beta_tester,
|
||||
"beta_status": user.beta_tester_status,
|
||||
"needs_feedback": (
|
||||
user.is_beta_tester and
|
||||
user.beta_tester_status == BetaTesterStatus.PENDING_FEEDBACK.value
|
||||
),
|
||||
"can_submit_feedback": (
|
||||
user.is_beta_tester and
|
||||
user.beta_tester_status != BetaTesterStatus.COMPLETED.value
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Admin Endpoints
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/admin/users", response_model=UserListResponse)
|
||||
async def list_users(
|
||||
limit: int = 100,
|
||||
include_inactive: bool = False,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
List all users (admin only).
|
||||
"""
|
||||
users = user_service.list_users(limit=limit, include_inactive=include_inactive)
|
||||
|
||||
users_public = [
|
||||
UserPublic(
|
||||
email=u.email,
|
||||
role=u.role,
|
||||
credits=u.credits,
|
||||
display_name=u.display_name,
|
||||
total_jobs_created=u.total_jobs_created,
|
||||
total_jobs_completed=u.total_jobs_completed,
|
||||
)
|
||||
for u in users
|
||||
]
|
||||
|
||||
return UserListResponse(users=users_public, total=len(users_public))
|
||||
|
||||
|
||||
@router.post("/admin/credits", response_model=AddCreditsResponse)
|
||||
async def add_credits_to_user(
|
||||
request: AddCreditsRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Add credits to a user's account (admin only).
|
||||
|
||||
Use this to grant free credits to users, e.g., for beta testers or promotions.
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
|
||||
# TODO: Enhance auth system to track admin email identity for better audit trails.
|
||||
# Current token-based admin auth doesn't include email identity.
|
||||
# For now, we log the token prefix for traceability.
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success, new_balance, message = user_service.add_credits(
|
||||
email=request.email,
|
||||
amount=request.amount,
|
||||
reason=request.reason,
|
||||
admin_email=admin_id,
|
||||
)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
# Send notification email
|
||||
if request.amount > 0:
|
||||
email_service.send_credits_added(request.email, request.amount, new_balance)
|
||||
|
||||
return AddCreditsResponse(
|
||||
status="success",
|
||||
email=request.email,
|
||||
credits_added=request.amount,
|
||||
new_balance=new_balance,
|
||||
message=message,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/admin/users/{email}/disable")
|
||||
async def disable_user(
|
||||
email: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Disable a user account (admin only).
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success = user_service.disable_user(email, admin_email=admin_id)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
return {"status": "success", "message": f"User {email} has been disabled"}
|
||||
|
||||
|
||||
@router.post("/admin/users/{email}/enable")
|
||||
async def enable_user(
|
||||
email: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Enable a user account (admin only).
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success = user_service.enable_user(email, admin_email=admin_id)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
return {"status": "success", "message": f"User {email} has been enabled"}
|
||||
|
||||
|
||||
@router.post("/admin/users/{email}/role")
|
||||
async def set_user_role(
|
||||
email: str,
|
||||
role: UserRole,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Set a user's role (admin only).
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success = user_service.set_user_role(email, role, admin_email=admin_id)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
return {"status": "success", "message": f"User {email} role set to {role.value}"}
|
||||
|
||||
|
||||
@router.get("/admin/beta/feedback")
|
||||
async def list_beta_feedback(
|
||||
limit: int = 50,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
List all beta tester feedback (admin only).
|
||||
"""
|
||||
from google.cloud import firestore
|
||||
|
||||
query = user_service.db.collection("beta_feedback")
|
||||
query = query.order_by("created_at", direction=firestore.Query.DESCENDING)
|
||||
query = query.limit(limit)
|
||||
|
||||
docs = query.stream()
|
||||
feedback_list = [doc.to_dict() for doc in docs]
|
||||
|
||||
return {
|
||||
"feedback": feedback_list,
|
||||
"total": len(feedback_list),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/admin/beta/stats")
|
||||
async def get_beta_stats(
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Get beta tester program statistics (admin only).
|
||||
"""
|
||||
from google.cloud.firestore_v1 import FieldFilter
|
||||
from google.cloud.firestore_v1 import aggregation
|
||||
|
||||
# Count beta testers by status using efficient aggregation queries
|
||||
users_collection = user_service.db.collection("users")
|
||||
|
||||
# Helper function to get count using aggregation
|
||||
def get_count(query) -> int:
|
||||
agg_query = aggregation.AggregationQuery(query)
|
||||
agg_query.count(alias="count")
|
||||
results = agg_query.get()
|
||||
return results[0][0].value if results else 0
|
||||
|
||||
total_beta_testers = get_count(
|
||||
users_collection.where(filter=FieldFilter("is_beta_tester", "==", True))
|
||||
)
|
||||
|
||||
active_testers = get_count(
|
||||
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "active"))
|
||||
)
|
||||
|
||||
pending_feedback = get_count(
|
||||
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "pending_feedback"))
|
||||
)
|
||||
|
||||
completed_feedback = get_count(
|
||||
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "completed"))
|
||||
)
|
||||
|
||||
# Get average ratings from feedback
|
||||
feedback_docs = list(user_service.db.collection("beta_feedback").stream())
|
||||
|
||||
avg_overall = 0
|
||||
avg_ease = 0
|
||||
avg_accuracy = 0
|
||||
avg_correction = 0
|
||||
|
||||
if feedback_docs:
|
||||
total = len(feedback_docs)
|
||||
for doc in feedback_docs:
|
||||
data = doc.to_dict()
|
||||
avg_overall += data.get("overall_rating", 0)
|
||||
avg_ease += data.get("ease_of_use_rating", 0)
|
||||
avg_accuracy += data.get("lyrics_accuracy_rating", 0)
|
||||
avg_correction += data.get("correction_experience_rating", 0)
|
||||
|
||||
avg_overall = round(avg_overall / total, 2)
|
||||
avg_ease = round(avg_ease / total, 2)
|
||||
avg_accuracy = round(avg_accuracy / total, 2)
|
||||
avg_correction = round(avg_correction / total, 2)
|
||||
|
||||
return {
|
||||
"total_beta_testers": total_beta_testers,
|
||||
"active_testers": active_testers,
|
||||
"pending_feedback": pending_feedback,
|
||||
"completed_feedback": completed_feedback,
|
||||
"total_feedback_submissions": len(feedback_docs),
|
||||
"average_ratings": {
|
||||
"overall": avg_overall,
|
||||
"ease_of_use": avg_ease,
|
||||
"lyrics_accuracy": avg_accuracy,
|
||||
"correction_experience": avg_correction,
|
||||
},
|
||||
}
|
||||
@@ -40,11 +40,6 @@ class Settings(BaseSettings):
|
||||
# Processing
|
||||
max_concurrent_jobs: int = int(os.getenv("MAX_CONCURRENT_JOBS", "5"))
|
||||
job_timeout_seconds: int = int(os.getenv("JOB_TIMEOUT_SECONDS", "3600"))
|
||||
|
||||
# Agentic AI Correction (for lyrics correction via LLM)
|
||||
# When enabled, uses Gemini via Vertex AI for intelligent lyrics correction
|
||||
use_agentic_ai: bool = os.getenv("USE_AGENTIC_AI", "true").lower() in ("true", "1", "yes")
|
||||
agentic_ai_model: str = os.getenv("AGENTIC_AI_MODEL", "vertexai/gemini-3-flash-preview")
|
||||
|
||||
# Cloud Tasks (for scalable worker coordination)
|
||||
# When enabled, workers are triggered via Cloud Tasks for guaranteed delivery
|
||||
|
||||
+1
-6
@@ -7,10 +7,9 @@ from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from backend.config import settings
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes
|
||||
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
|
||||
from backend.services.structured_logging import setup_structured_logging
|
||||
from backend.middleware.audit_logging import AuditLoggingMiddleware
|
||||
|
||||
|
||||
from backend.version import VERSION
|
||||
@@ -101,9 +100,6 @@ app.add_middleware(
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# Add audit logging middleware (captures all requests with request_id for correlation)
|
||||
app.add_middleware(AuditLoggingMiddleware)
|
||||
|
||||
# Include routers
|
||||
app.include_router(health.router, prefix="/api")
|
||||
app.include_router(jobs.router, prefix="/api")
|
||||
@@ -113,7 +109,6 @@ app.include_router(review.router, prefix="/api") # Review UI compatibility endp
|
||||
app.include_router(auth.router, prefix="/api") # OAuth credential management
|
||||
app.include_router(audio_search.router, prefix="/api") # Audio search (artist+title mode)
|
||||
app.include_router(themes.router, prefix="/api") # Theme selection for styles
|
||||
app.include_router(users.router, prefix="/api") # User auth, credits, and Stripe webhooks
|
||||
|
||||
|
||||
@app.get("/")
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
"""Middleware package for FastAPI application."""
|
||||
|
||||
from backend.middleware.audit_logging import AuditLoggingMiddleware
|
||||
|
||||
__all__ = ["AuditLoggingMiddleware"]
|
||||
@@ -1,124 +0,0 @@
|
||||
"""
|
||||
Audit logging middleware for tracking all HTTP requests.
|
||||
|
||||
This middleware captures request metadata and logs it to Cloud Logging
|
||||
for audit trail and user activity investigation purposes.
|
||||
"""
|
||||
import logging
|
||||
import time
|
||||
import uuid
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
|
||||
|
||||
logger = logging.getLogger("audit")
|
||||
|
||||
# Endpoints to exclude from audit logging (high-frequency health checks)
|
||||
EXCLUDED_PATHS = {
|
||||
"/",
|
||||
"/api/health",
|
||||
"/api/health/detailed",
|
||||
"/api/readiness",
|
||||
"/healthz",
|
||||
"/ready",
|
||||
}
|
||||
|
||||
|
||||
class AuditLoggingMiddleware(BaseHTTPMiddleware):
|
||||
"""
|
||||
Middleware that logs all HTTP requests for audit purposes.
|
||||
|
||||
Captures:
|
||||
- request_id: UUID for correlating with auth logs
|
||||
- method: HTTP method
|
||||
- path: Request path
|
||||
- status_code: Response status
|
||||
- latency_ms: Request duration
|
||||
- client_ip: Client IP (from X-Forwarded-For for proxied requests)
|
||||
- user_agent: Browser/client identifier
|
||||
|
||||
The request_id is stored in request.state and added to response headers,
|
||||
allowing correlation with auth logs that capture user_email.
|
||||
"""
|
||||
|
||||
async def dispatch(self, request: Request, call_next) -> Response:
|
||||
# Skip excluded paths (health checks)
|
||||
if request.url.path in EXCLUDED_PATHS:
|
||||
return await call_next(request)
|
||||
|
||||
# Generate unique request ID for correlation
|
||||
request_id = str(uuid.uuid4())
|
||||
request.state.request_id = request_id
|
||||
|
||||
# Capture start time
|
||||
start_time = time.time()
|
||||
|
||||
# Extract client info
|
||||
client_ip = self._get_client_ip(request)
|
||||
user_agent = request.headers.get("user-agent", "")
|
||||
|
||||
# Process request
|
||||
try:
|
||||
response = await call_next(request)
|
||||
except Exception:
|
||||
# Log failed requests too (exception() auto-includes stack trace)
|
||||
latency_ms = int((time.time() - start_time) * 1000)
|
||||
logger.exception(
|
||||
"request_audit_error",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"method": request.method,
|
||||
"path": request.url.path,
|
||||
"query_string": str(request.query_params) if request.query_params else None,
|
||||
"latency_ms": latency_ms,
|
||||
"client_ip": client_ip,
|
||||
"user_agent": user_agent[:200] if user_agent else None,
|
||||
"audit_type": "http_request",
|
||||
}
|
||||
)
|
||||
raise
|
||||
|
||||
# Calculate latency
|
||||
latency_ms = int((time.time() - start_time) * 1000)
|
||||
|
||||
# Log audit entry (INFO level for successful requests)
|
||||
log_level = logging.WARNING if response.status_code >= 400 else logging.INFO
|
||||
logger.log(
|
||||
log_level,
|
||||
"request_audit",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"method": request.method,
|
||||
"path": request.url.path,
|
||||
"query_string": str(request.query_params) if request.query_params else None,
|
||||
"status_code": response.status_code,
|
||||
"latency_ms": latency_ms,
|
||||
"client_ip": client_ip,
|
||||
"user_agent": user_agent[:200] if user_agent else None,
|
||||
"audit_type": "http_request",
|
||||
}
|
||||
)
|
||||
|
||||
# Add request_id to response headers for debugging/correlation
|
||||
response.headers["X-Request-ID"] = request_id
|
||||
|
||||
return response
|
||||
|
||||
def _get_client_ip(self, request: Request) -> str:
|
||||
"""
|
||||
Extract client IP address, handling proxy scenarios.
|
||||
|
||||
Cloud Run and other proxies set X-Forwarded-For header.
|
||||
"""
|
||||
# Check X-Forwarded-For for proxy scenarios (Cloud Run, load balancers)
|
||||
forwarded = request.headers.get("x-forwarded-for", "")
|
||||
if forwarded:
|
||||
# First IP is the original client
|
||||
return forwarded.split(",")[0].strip()
|
||||
|
||||
# Fall back to direct connection
|
||||
if request.client:
|
||||
return request.client.host
|
||||
|
||||
return "unknown"
|
||||
@@ -1,254 +0,0 @@
|
||||
"""
|
||||
User model for authentication and credits.
|
||||
|
||||
Supports:
|
||||
- Magic link authentication (email-based)
|
||||
- Credit system for karaoke generation
|
||||
- Role-based access control (user/admin)
|
||||
- Stripe integration for payments
|
||||
- Beta tester program with feedback collection
|
||||
"""
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from typing import Optional, List
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class UserRole(str, Enum):
|
||||
"""User roles for access control."""
|
||||
USER = "user"
|
||||
ADMIN = "admin"
|
||||
|
||||
|
||||
class BetaTesterStatus(str, Enum):
|
||||
"""Status of beta tester participation."""
|
||||
ACTIVE = "active" # Enrolled, free credits available
|
||||
PENDING_FEEDBACK = "pending_feedback" # Job completed, awaiting feedback
|
||||
COMPLETED = "completed" # Feedback submitted
|
||||
EXPIRED = "expired" # 24hr deadline passed without feedback
|
||||
|
||||
|
||||
class CreditTransaction(BaseModel):
|
||||
"""Record of a credit transaction."""
|
||||
id: str
|
||||
amount: int # Positive = add, negative = deduct
|
||||
reason: str # e.g., "purchase", "refund", "admin_grant", "job_creation"
|
||||
job_id: Optional[str] = None
|
||||
stripe_session_id: Optional[str] = None
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
created_by: Optional[str] = None # Admin email if granted by admin
|
||||
|
||||
|
||||
class User(BaseModel):
|
||||
"""
|
||||
User model stored in Firestore.
|
||||
|
||||
Users are identified by email address. Authentication is via magic links
|
||||
(no passwords). Credits are consumed when creating karaoke jobs.
|
||||
"""
|
||||
email: str # Primary identifier
|
||||
role: UserRole = UserRole.USER
|
||||
credits: int = 0
|
||||
|
||||
# Stripe integration
|
||||
stripe_customer_id: Optional[str] = None
|
||||
|
||||
# Account state
|
||||
is_active: bool = True
|
||||
email_verified: bool = False
|
||||
|
||||
# Timestamps
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
updated_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
last_login_at: Optional[datetime] = None
|
||||
|
||||
# Credit history (last 100 transactions)
|
||||
credit_transactions: List[CreditTransaction] = Field(default_factory=list)
|
||||
|
||||
# Job tracking
|
||||
total_jobs_created: int = 0
|
||||
total_jobs_completed: int = 0
|
||||
|
||||
# Optional profile fields for future use
|
||||
display_name: Optional[str] = None
|
||||
|
||||
# Beta tester program
|
||||
is_beta_tester: bool = False
|
||||
beta_tester_status: Optional[BetaTesterStatus] = None
|
||||
beta_enrolled_at: Optional[datetime] = None
|
||||
beta_promise_text: Optional[str] = None # User's promise statement
|
||||
beta_feedback_due_at: Optional[datetime] = None # 24hr after job completion
|
||||
beta_feedback_email_sent: bool = False
|
||||
|
||||
|
||||
class MagicLinkToken(BaseModel):
|
||||
"""
|
||||
Magic link token for passwordless authentication.
|
||||
|
||||
Tokens are short-lived (15 minutes) and single-use.
|
||||
Stored in Firestore with automatic TTL.
|
||||
"""
|
||||
token: str # Secure random token
|
||||
email: str
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
expires_at: datetime
|
||||
used: bool = False
|
||||
used_at: Optional[datetime] = None
|
||||
ip_address: Optional[str] = None
|
||||
user_agent: Optional[str] = None
|
||||
|
||||
|
||||
class Session(BaseModel):
|
||||
"""
|
||||
User session for authenticated requests.
|
||||
|
||||
Sessions are created after successful magic link verification.
|
||||
Sessions expire after 7 days of inactivity or 30 days absolute.
|
||||
"""
|
||||
token: str # Secure random session token
|
||||
user_email: str
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
expires_at: datetime
|
||||
last_activity_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
ip_address: Optional[str] = None
|
||||
user_agent: Optional[str] = None
|
||||
is_active: bool = True
|
||||
|
||||
|
||||
# Pydantic models for API requests/responses
|
||||
|
||||
class SendMagicLinkRequest(BaseModel):
|
||||
"""Request to send a magic link email."""
|
||||
email: str
|
||||
|
||||
|
||||
class SendMagicLinkResponse(BaseModel):
|
||||
"""Response after sending magic link."""
|
||||
status: str
|
||||
message: str
|
||||
|
||||
|
||||
class VerifyMagicLinkRequest(BaseModel):
|
||||
"""Request to verify a magic link token."""
|
||||
token: str
|
||||
|
||||
|
||||
class VerifyMagicLinkResponse(BaseModel):
|
||||
"""Response after verifying magic link."""
|
||||
status: str
|
||||
session_token: str
|
||||
user: "UserPublic"
|
||||
message: str
|
||||
|
||||
|
||||
class UserPublic(BaseModel):
|
||||
"""Public user information (safe to expose to frontend)."""
|
||||
email: str
|
||||
role: UserRole
|
||||
credits: int
|
||||
display_name: Optional[str] = None
|
||||
total_jobs_created: int = 0
|
||||
total_jobs_completed: int = 0
|
||||
|
||||
|
||||
class AddCreditsRequest(BaseModel):
|
||||
"""Admin request to add credits to a user."""
|
||||
email: str
|
||||
amount: int
|
||||
reason: str = "admin_grant"
|
||||
|
||||
|
||||
class AddCreditsResponse(BaseModel):
|
||||
"""Response after adding credits."""
|
||||
status: str
|
||||
email: str
|
||||
credits_added: int
|
||||
new_balance: int
|
||||
message: str
|
||||
|
||||
|
||||
class UserListResponse(BaseModel):
|
||||
"""Response for listing users (admin only)."""
|
||||
users: List[UserPublic]
|
||||
total: int
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Beta Tester Program Models
|
||||
# ============================================================================
|
||||
|
||||
class BetaTesterFeedback(BaseModel):
|
||||
"""
|
||||
Feedback from a beta tester after using the service.
|
||||
|
||||
Stored in Firestore 'beta_feedback' collection.
|
||||
"""
|
||||
id: str # Unique feedback ID
|
||||
user_email: str
|
||||
job_id: Optional[str] = None # The job they're providing feedback on
|
||||
|
||||
# Ratings (1-5 scale)
|
||||
overall_rating: int = Field(ge=1, le=5)
|
||||
ease_of_use_rating: int = Field(ge=1, le=5)
|
||||
lyrics_accuracy_rating: int = Field(ge=1, le=5)
|
||||
correction_experience_rating: int = Field(ge=1, le=5)
|
||||
|
||||
# Open-ended feedback
|
||||
what_went_well: Optional[str] = None
|
||||
what_could_improve: Optional[str] = None
|
||||
additional_comments: Optional[str] = None
|
||||
|
||||
# Would they recommend / use again?
|
||||
would_recommend: bool = True
|
||||
would_use_again: bool = True
|
||||
|
||||
# Metadata
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
submitted_via: str = "web" # 'web' or 'email'
|
||||
|
||||
|
||||
class BetaTesterEnrollRequest(BaseModel):
|
||||
"""Request to enroll as a beta tester."""
|
||||
email: str
|
||||
promise_text: str = Field(
|
||||
min_length=10,
|
||||
description="User's promise to provide feedback (min 10 chars)"
|
||||
)
|
||||
accept_corrections_work: bool = Field(
|
||||
description="User accepts they may need to review/correct lyrics"
|
||||
)
|
||||
|
||||
|
||||
class BetaTesterEnrollResponse(BaseModel):
|
||||
"""Response after enrolling as beta tester."""
|
||||
status: str
|
||||
message: str
|
||||
credits_granted: int
|
||||
session_token: Optional[str] = None # If new user, provide session
|
||||
|
||||
|
||||
class BetaFeedbackRequest(BaseModel):
|
||||
"""Request to submit beta tester feedback."""
|
||||
job_id: Optional[str] = None
|
||||
|
||||
# Ratings (1-5 scale)
|
||||
overall_rating: int = Field(ge=1, le=5)
|
||||
ease_of_use_rating: int = Field(ge=1, le=5)
|
||||
lyrics_accuracy_rating: int = Field(ge=1, le=5)
|
||||
correction_experience_rating: int = Field(ge=1, le=5)
|
||||
|
||||
# Open-ended feedback
|
||||
what_went_well: Optional[str] = None
|
||||
what_could_improve: Optional[str] = None
|
||||
additional_comments: Optional[str] = None
|
||||
|
||||
# Would they recommend / use again?
|
||||
would_recommend: bool = True
|
||||
would_use_again: bool = True
|
||||
|
||||
|
||||
class BetaFeedbackResponse(BaseModel):
|
||||
"""Response after submitting feedback."""
|
||||
status: str
|
||||
message: str
|
||||
bonus_credits: int = 0 # Bonus credits for great feedback
|
||||
@@ -3,81 +3,50 @@ Authentication service for karaoke backend.
|
||||
|
||||
Provides token-based authentication with support for:
|
||||
- Admin tokens (hardcoded in environment)
|
||||
- Admin by email domain (@nomadkaraoke.com)
|
||||
- User tokens (stored in Firestore auth_tokens collection)
|
||||
- Session tokens (stored in Firestore sessions collection, from magic link auth)
|
||||
- API keys for business users
|
||||
- User tokens (stored in Firestore)
|
||||
- Usage tracking and limits
|
||||
- Token management API
|
||||
|
||||
This is designed to be extensible for future Stripe integration
|
||||
and admin dashboard functionality.
|
||||
"""
|
||||
import logging
|
||||
import hashlib
|
||||
import time
|
||||
import secrets
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
from typing import Optional, Tuple, Dict, Any, List
|
||||
from typing import Optional, Tuple, Dict, Any
|
||||
from datetime import datetime
|
||||
|
||||
from backend.services.firestore_service import FirestoreService
|
||||
from backend.config import get_settings
|
||||
from backend.models.user import UserRole
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Admin email domains - users with these domains are automatically admin
|
||||
ADMIN_EMAIL_DOMAINS = ["nomadkaraoke.com"]
|
||||
|
||||
|
||||
class UserType(str, Enum):
|
||||
"""User access levels."""
|
||||
ADMIN = "admin" # Unlimited access, can manage tokens
|
||||
UNLIMITED = "unlimited" # Unlimited karaoke generation
|
||||
LIMITED = "limited" # Limited number of uses
|
||||
STRIPE = "stripe" # Paid access via Stripe
|
||||
API_KEY = "api_key" # Business API key access
|
||||
|
||||
|
||||
@dataclass
|
||||
class AuthResult:
|
||||
"""Result of authentication validation."""
|
||||
is_valid: bool
|
||||
user_type: UserType
|
||||
remaining_uses: int # -1 = unlimited, 0 = exhausted, >0 = remaining
|
||||
message: str
|
||||
user_email: Optional[str] = None # Email if authenticated via session/API key
|
||||
is_admin: bool = False # True if admin token or admin email domain
|
||||
api_key_id: Optional[str] = None # API key ID if authenticated via API key
|
||||
|
||||
def __iter__(self):
|
||||
"""Allow unpacking as tuple for backward compatibility."""
|
||||
return iter((self.is_valid, self.user_type, self.remaining_uses, self.message))
|
||||
|
||||
|
||||
def is_admin_email(email: str) -> bool:
|
||||
"""Check if an email belongs to an admin domain."""
|
||||
if not email:
|
||||
return False
|
||||
email_lower = email.lower()
|
||||
return any(email_lower.endswith(f"@{domain}") for domain in ADMIN_EMAIL_DOMAINS)
|
||||
STRIPE = "stripe" # Paid access via Stripe (future)
|
||||
|
||||
|
||||
class AuthService:
|
||||
"""Service for authentication and token management."""
|
||||
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize auth service."""
|
||||
self.firestore = FirestoreService()
|
||||
self.settings = get_settings()
|
||||
self._init_admin_tokens()
|
||||
|
||||
|
||||
def _init_admin_tokens(self):
|
||||
"""Load admin tokens from environment variables."""
|
||||
# Get admin tokens from secret manager or env vars
|
||||
admin_tokens_str = self.settings.admin_tokens or ""
|
||||
self.admin_tokens = [t.strip() for t in admin_tokens_str.split(",") if t.strip()]
|
||||
|
||||
|
||||
if self.admin_tokens:
|
||||
logger.info(f"Loaded {len(self.admin_tokens)} admin token(s)")
|
||||
else:
|
||||
@@ -109,30 +78,10 @@ class AuthService:
|
||||
f"expected len={len(self.admin_tokens[0])}, got len={len(token)}"
|
||||
)
|
||||
|
||||
# Check stored tokens in Firestore (auth_tokens collection)
|
||||
# Check stored tokens in Firestore
|
||||
token_data = self.firestore.get_token(token)
|
||||
|
||||
|
||||
if not token_data:
|
||||
# Fall back to checking session tokens (from magic link auth)
|
||||
# Import here to avoid circular dependency
|
||||
from backend.services.user_service import get_user_service
|
||||
|
||||
user_service = get_user_service()
|
||||
is_valid, user, message = user_service.validate_session(token)
|
||||
|
||||
if is_valid and user:
|
||||
# Session is valid - user is authenticated via magic link
|
||||
# Return their credits as remaining uses
|
||||
credits = user.credits
|
||||
logger.info(f"Session token validated for user {user.email} (credits: {credits})")
|
||||
|
||||
if credits <= 0:
|
||||
# User has no credits but is authenticated
|
||||
return True, UserType.STRIPE, 0, f"Authenticated but no credits remaining"
|
||||
|
||||
return True, UserType.STRIPE, credits, f"Session valid: {credits} credits"
|
||||
|
||||
# Neither auth_token nor session found
|
||||
return False, UserType.LIMITED, 0, "Invalid token"
|
||||
|
||||
# Check if token is active
|
||||
@@ -180,319 +129,35 @@ class AuthService:
|
||||
return True, token_type, -1, "Stripe access granted"
|
||||
|
||||
return False, UserType.LIMITED, 0, "Unknown token type"
|
||||
|
||||
def validate_token_full(self, token: str) -> AuthResult:
|
||||
"""
|
||||
Validate an access token and return full authentication details.
|
||||
|
||||
This is the preferred method for new code as it returns complete
|
||||
authentication context including user email and admin status.
|
||||
|
||||
Returns:
|
||||
AuthResult with full authentication details
|
||||
"""
|
||||
if not token:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType.LIMITED,
|
||||
remaining_uses=0,
|
||||
message="No token provided"
|
||||
)
|
||||
|
||||
# Check for admin tokens (highest priority)
|
||||
# Admin tokens from env var are associated with the system admin email
|
||||
if token in self.admin_tokens:
|
||||
logger.info("Admin token validated")
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.ADMIN,
|
||||
remaining_uses=-1,
|
||||
message="Admin access granted",
|
||||
user_email="admin@nomadkaraoke.com",
|
||||
is_admin=True
|
||||
)
|
||||
|
||||
# Check stored tokens in Firestore (auth_tokens collection)
|
||||
token_data = self.firestore.get_token(token)
|
||||
|
||||
if not token_data:
|
||||
# Fall back to checking session tokens (from magic link auth)
|
||||
from backend.services.user_service import get_user_service
|
||||
|
||||
user_service = get_user_service()
|
||||
is_valid, user, _message = user_service.validate_session(token)
|
||||
|
||||
if is_valid and user:
|
||||
# Session is valid - user is authenticated via magic link
|
||||
credits = user.credits
|
||||
user_email = user.email
|
||||
|
||||
# Check if user is admin by email domain or role
|
||||
user_is_admin = is_admin_email(user_email) or user.role == UserRole.ADMIN
|
||||
|
||||
if user_is_admin:
|
||||
logger.info(f"Admin session validated for {user_email}")
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.ADMIN,
|
||||
remaining_uses=-1,
|
||||
message="Admin access granted (domain)",
|
||||
user_email=user_email,
|
||||
is_admin=True
|
||||
)
|
||||
|
||||
logger.info(f"Session token validated for user {user_email} (credits: {credits})")
|
||||
|
||||
if credits <= 0:
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.STRIPE,
|
||||
remaining_uses=0,
|
||||
message="Authenticated but no credits remaining",
|
||||
user_email=user_email,
|
||||
is_admin=False
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.STRIPE,
|
||||
remaining_uses=credits,
|
||||
message=f"Session valid: {credits} credits",
|
||||
user_email=user_email,
|
||||
is_admin=False
|
||||
)
|
||||
|
||||
# Neither auth_token nor session found
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType.LIMITED,
|
||||
remaining_uses=0,
|
||||
message="Invalid token"
|
||||
)
|
||||
|
||||
# Check if token is active
|
||||
if not token_data.get("active", True):
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType(token_data["type"]),
|
||||
remaining_uses=0,
|
||||
message="Token has been revoked"
|
||||
)
|
||||
|
||||
token_type = UserType(token_data["type"])
|
||||
max_uses = token_data.get("max_uses", -1)
|
||||
# All auth_tokens must have an associated user_email for job ownership
|
||||
token_user_email = token_data.get("user_email")
|
||||
api_key_id = token_data.get("api_key_id")
|
||||
|
||||
# Require user_email on all auth_tokens (no anonymous token auth)
|
||||
if not token_user_email:
|
||||
logger.warning("Auth token missing required user_email field")
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token configuration error: missing user_email. Please contact support."
|
||||
)
|
||||
|
||||
# Check if token's user is an admin (by email domain)
|
||||
token_is_admin = is_admin_email(token_user_email)
|
||||
|
||||
# UNLIMITED tokens: no usage limits
|
||||
if token_type == UserType.UNLIMITED:
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="Unlimited access granted",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
# LIMITED tokens: check usage count
|
||||
if token_type == UserType.LIMITED:
|
||||
if max_uses <= 0: # -1 means unlimited
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="Limited token with unlimited uses",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
current_uses = token_data.get("usage_count", 0)
|
||||
remaining = max_uses - current_uses
|
||||
|
||||
if remaining <= 0:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token usage limit exceeded",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=remaining,
|
||||
message=f"Limited token: {remaining} uses remaining",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
# STRIPE tokens: check expiration and usage
|
||||
if token_type == UserType.STRIPE:
|
||||
expires_at = token_data.get("expires_at")
|
||||
|
||||
if expires_at and time.time() > expires_at:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token has expired",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
if max_uses > 0:
|
||||
current_uses = token_data.get("usage_count", 0)
|
||||
remaining = max_uses - current_uses
|
||||
|
||||
if remaining <= 0:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token usage limit exceeded",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=remaining,
|
||||
message=f"Stripe token: {remaining} uses remaining",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="Stripe access granted",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
# API_KEY tokens
|
||||
if token_type == UserType.API_KEY:
|
||||
if max_uses > 0:
|
||||
current_uses = token_data.get("usage_count", 0)
|
||||
remaining = max_uses - current_uses
|
||||
if remaining <= 0:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="API key usage limit exceeded",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=remaining,
|
||||
message=f"API key valid: {remaining} uses remaining",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="API key access granted",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType.LIMITED,
|
||||
remaining_uses=0,
|
||||
message="Unknown token type"
|
||||
)
|
||||
|
||||
|
||||
def increment_token_usage(self, token: str, job_id: str) -> bool:
|
||||
"""
|
||||
Increment usage count for a token and track the job.
|
||||
|
||||
For session tokens (magic link auth), this deducts a credit from the user.
|
||||
For auth_tokens, this increments the usage count.
|
||||
|
||||
|
||||
Args:
|
||||
token: The access token
|
||||
job_id: The job ID being created
|
||||
|
||||
|
||||
Returns:
|
||||
True if usage was tracked, False otherwise
|
||||
"""
|
||||
# Validate token first
|
||||
is_valid, user_type, remaining_uses, message = self.validate_token(token)
|
||||
|
||||
|
||||
if not is_valid:
|
||||
logger.warning(f"Cannot increment usage for invalid token: {message}")
|
||||
return False
|
||||
|
||||
|
||||
# Don't track usage for admin or unlimited tokens
|
||||
if user_type in [UserType.ADMIN, UserType.UNLIMITED]:
|
||||
logger.debug(f"Skipping usage tracking for {user_type} token")
|
||||
return True
|
||||
|
||||
|
||||
# For admin tokens (not in Firestore), no tracking needed
|
||||
if token in self.admin_tokens:
|
||||
return True
|
||||
|
||||
# Check if this is a session token (STRIPE type from validate_token means it's a session)
|
||||
# Session tokens are not in auth_tokens collection
|
||||
token_data = self.firestore.get_token(token)
|
||||
|
||||
if not token_data:
|
||||
# This is a session token - deduct credit from user
|
||||
from backend.services.user_service import get_user_service
|
||||
user_service = get_user_service()
|
||||
|
||||
# Get the user email from the session
|
||||
is_valid, user, _ = user_service.validate_session(token)
|
||||
if not is_valid or not user:
|
||||
logger.error(f"Session token validation failed during usage increment")
|
||||
return False
|
||||
|
||||
# Deduct one credit
|
||||
success, new_balance, deduct_message = user_service.deduct_credit(
|
||||
user.email, job_id, reason="job_creation"
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.info(f"Deducted credit for user {user.email} (remaining: {new_balance})")
|
||||
return True
|
||||
else:
|
||||
logger.error(f"Failed to deduct credit for user {user.email}: {deduct_message}")
|
||||
return False
|
||||
|
||||
# Regular auth_token - increment usage in Firestore
|
||||
|
||||
# Increment usage in Firestore
|
||||
try:
|
||||
self.firestore.increment_token_usage(token, job_id)
|
||||
logger.info(f"Incremented usage for token (remaining: {remaining_uses - 1})")
|
||||
|
||||
@@ -199,7 +199,7 @@ class DropboxService:
|
||||
|
||||
def upload_folder(self, local_dir: str, remote_path: str) -> None:
|
||||
"""
|
||||
Recursively upload all files and subdirectories to Dropbox folder.
|
||||
Upload all files in a directory to Dropbox folder.
|
||||
|
||||
Args:
|
||||
local_dir: Local directory to upload
|
||||
@@ -212,17 +212,10 @@ class DropboxService:
|
||||
logger.info(f"Uploading folder {local_dir} to {remote_path}")
|
||||
|
||||
uploaded_count = 0
|
||||
for root, _dirs, files in os.walk(local_dir):
|
||||
# Calculate the relative path from local_dir to current root
|
||||
rel_root = os.path.relpath(root, local_dir)
|
||||
if rel_root == ".":
|
||||
current_remote = remote_path
|
||||
else:
|
||||
current_remote = f"{remote_path}/{rel_root}"
|
||||
|
||||
for filename in files:
|
||||
local_file = os.path.join(root, filename)
|
||||
remote_file = f"{current_remote}/{filename}"
|
||||
for filename in os.listdir(local_dir):
|
||||
local_file = os.path.join(local_dir, filename)
|
||||
if os.path.isfile(local_file):
|
||||
remote_file = f"{remote_path}/{filename}"
|
||||
self.upload_file(local_file, remote_file)
|
||||
uploaded_count += 1
|
||||
|
||||
|
||||
@@ -1,829 +0,0 @@
|
||||
"""
|
||||
Email service for sending transactional emails.
|
||||
|
||||
Supports multiple providers:
|
||||
- SendGrid (recommended for production)
|
||||
- Console logging (for development/testing)
|
||||
|
||||
Future providers can be added:
|
||||
- Mailgun
|
||||
- AWS SES
|
||||
- Postmark
|
||||
"""
|
||||
import html
|
||||
import logging
|
||||
import os
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Optional
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class EmailProvider(ABC):
|
||||
"""Abstract base class for email providers."""
|
||||
|
||||
@abstractmethod
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
"""Send an email. Returns True if successful."""
|
||||
pass
|
||||
|
||||
|
||||
class ConsoleEmailProvider(EmailProvider):
|
||||
"""
|
||||
Development email provider that logs to console.
|
||||
|
||||
Useful for local development and testing.
|
||||
"""
|
||||
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
logger.info("=" * 60)
|
||||
logger.info(f"EMAIL TO: {to_email}")
|
||||
logger.info(f"SUBJECT: {subject}")
|
||||
logger.info("-" * 60)
|
||||
logger.info(text_content or html_content)
|
||||
logger.info("=" * 60)
|
||||
return True
|
||||
|
||||
|
||||
class SendGridEmailProvider(EmailProvider):
|
||||
"""
|
||||
SendGrid email provider for production.
|
||||
|
||||
Requires SENDGRID_API_KEY environment variable.
|
||||
"""
|
||||
|
||||
def __init__(self, api_key: str, from_email: str, from_name: str = "Nomad Karaoke"):
|
||||
self.api_key = api_key
|
||||
self.from_email = from_email
|
||||
self.from_name = from_name
|
||||
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
try:
|
||||
# Import here to avoid requiring sendgrid in all environments
|
||||
from sendgrid import SendGridAPIClient
|
||||
from sendgrid.helpers.mail import Mail, Email, To, Content
|
||||
|
||||
sg = SendGridAPIClient(api_key=self.api_key)
|
||||
|
||||
message = Mail(
|
||||
from_email=Email(self.from_email, self.from_name),
|
||||
to_emails=To(to_email),
|
||||
subject=subject,
|
||||
html_content=Content("text/html", html_content)
|
||||
)
|
||||
|
||||
if text_content:
|
||||
message.add_content(Content("text/plain", text_content))
|
||||
|
||||
response = sg.send(message)
|
||||
|
||||
if response.status_code >= 200 and response.status_code < 300:
|
||||
logger.info(f"Email sent to {to_email} via SendGrid")
|
||||
return True
|
||||
else:
|
||||
logger.error(f"SendGrid returned status {response.status_code}")
|
||||
return False
|
||||
|
||||
except Exception:
|
||||
logger.exception("Failed to send email via SendGrid")
|
||||
return False
|
||||
|
||||
|
||||
class EmailService:
|
||||
"""
|
||||
High-level email service for sending transactional emails.
|
||||
|
||||
Automatically selects the appropriate provider based on configuration.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.settings = get_settings()
|
||||
self.provider = self._get_provider()
|
||||
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
|
||||
# After consolidation, buy URL is the same as frontend URL
|
||||
self.buy_url = os.getenv("BUY_URL", self.frontend_url)
|
||||
|
||||
def _get_provider(self) -> EmailProvider:
|
||||
"""Get the configured email provider."""
|
||||
sendgrid_api_key = os.getenv("SENDGRID_API_KEY")
|
||||
from_email = os.getenv("EMAIL_FROM", "gen@nomadkaraoke.com")
|
||||
from_name = os.getenv("EMAIL_FROM_NAME", "Nomad Karaoke")
|
||||
|
||||
if sendgrid_api_key:
|
||||
logger.info("Using SendGrid email provider")
|
||||
return SendGridEmailProvider(sendgrid_api_key, from_email, from_name)
|
||||
else:
|
||||
logger.warning("No email provider configured, using console logging")
|
||||
return ConsoleEmailProvider()
|
||||
|
||||
def is_configured(self) -> bool:
|
||||
"""Check if a real email provider is configured (not just console logging)."""
|
||||
return isinstance(self.provider, SendGridEmailProvider)
|
||||
|
||||
def send_magic_link(self, email: str, token: str) -> bool:
|
||||
"""
|
||||
Send a magic link email for authentication.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
token: Magic link token
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
magic_link_url = f"{self.frontend_url}/auth/verify?token={token}"
|
||||
|
||||
subject = "Sign in to Nomad Karaoke"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.button:hover {{
|
||||
background-color: #2563eb;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
.warning {{
|
||||
background-color: #fef3c7;
|
||||
border: 1px solid #fcd34d;
|
||||
border-radius: 4px;
|
||||
padding: 12px;
|
||||
margin: 20px 0;
|
||||
font-size: 14px;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Hi there,</p>
|
||||
|
||||
<p>Click the button below to sign in to Nomad Karaoke:</p>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{magic_link_url}" class="button">Sign In</a>
|
||||
</p>
|
||||
|
||||
<div class="warning">
|
||||
⏰ This link expires in 15 minutes and can only be used once.
|
||||
</div>
|
||||
|
||||
<p>If the button doesn't work, copy and paste this link into your browser:</p>
|
||||
<p style="word-break: break-all; font-size: 14px; color: #666;">
|
||||
{magic_link_url}
|
||||
</p>
|
||||
|
||||
<p>If you didn't request this email, you can safely ignore it.</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
<p>This is an automated message, please do not reply.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Sign in to Nomad Karaoke
|
||||
========================
|
||||
|
||||
Click this link to sign in:
|
||||
{magic_link_url}
|
||||
|
||||
This link expires in 15 minutes and can only be used once.
|
||||
|
||||
If you didn't request this email, you can safely ignore it.
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_credits_added(self, email: str, credits: int, total_credits: int) -> bool:
|
||||
"""
|
||||
Send notification when credits are added to account.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Number of credits added
|
||||
total_credits: New total credit balance
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = f"🎉 {credits} credits added to your Nomad Karaoke account"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.credits-box {{
|
||||
background-color: #ecfdf5;
|
||||
border: 2px solid #10b981;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.credits-number {{
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
color: #10b981;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Great news! 🎉</p>
|
||||
|
||||
<p><strong>{credits} credits</strong> have been added to your account.</p>
|
||||
|
||||
<div class="credits-box">
|
||||
<div>Your balance:</div>
|
||||
<div class="credits-number">{total_credits}</div>
|
||||
<div>credits</div>
|
||||
</div>
|
||||
|
||||
<p>Each credit lets you create one professional karaoke video with:</p>
|
||||
<ul>
|
||||
<li>AI-powered vocal/instrumental separation</li>
|
||||
<li>Synchronized lyrics with word-level timing</li>
|
||||
<li>4K video output</li>
|
||||
<li>YouTube upload</li>
|
||||
</ul>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Create Karaoke Now</a>
|
||||
</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
{credits} credits added to your Nomad Karaoke account!
|
||||
|
||||
Your new balance: {total_credits} credits
|
||||
|
||||
Each credit lets you create one professional karaoke video.
|
||||
|
||||
Start creating: {self.frontend_url}
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_welcome_email(self, email: str, credits: int = 0) -> bool:
|
||||
"""
|
||||
Send welcome email to new users.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Initial credit balance (if any)
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Welcome to Nomad Karaoke! 🎤"
|
||||
|
||||
credits_text = f"You have <strong>{credits} credits</strong> to get started!" if credits > 0 else ""
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.feature {{
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
margin: 16px 0;
|
||||
}}
|
||||
.feature-icon {{
|
||||
font-size: 24px;
|
||||
margin-right: 12px;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Welcome to Nomad Karaoke!</p>
|
||||
|
||||
<p>Turn any song into a professional karaoke video in minutes. {credits_text}</p>
|
||||
|
||||
<p><strong>Here's how it works:</strong></p>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">🎵</span>
|
||||
<div>
|
||||
<strong>1. Search for a song</strong><br>
|
||||
Enter the artist and title, and we'll find high-quality audio.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">✨</span>
|
||||
<div>
|
||||
<strong>2. AI does the magic</strong><br>
|
||||
We separate vocals, transcribe lyrics, and sync everything perfectly.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">✏️</span>
|
||||
<div>
|
||||
<strong>3. Review & customize</strong><br>
|
||||
Fine-tune the lyrics if needed, choose your instrumental.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">🎬</span>
|
||||
<div>
|
||||
<strong>4. Get your video</strong><br>
|
||||
Download your 4K karaoke video or upload directly to YouTube.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Get Started</a>
|
||||
</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>Questions? Reply to this email and we'll help you out.</p>
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Welcome to Nomad Karaoke!
|
||||
|
||||
Turn any song into a professional karaoke video in minutes.
|
||||
|
||||
Here's how it works:
|
||||
|
||||
1. Search for a song
|
||||
Enter the artist and title, and we'll find high-quality audio.
|
||||
|
||||
2. AI does the magic
|
||||
We separate vocals, transcribe lyrics, and sync everything perfectly.
|
||||
|
||||
3. Review & customize
|
||||
Fine-tune the lyrics if needed, choose your instrumental.
|
||||
|
||||
4. Get your video
|
||||
Download your 4K karaoke video or upload directly to YouTube.
|
||||
|
||||
Get started: {self.frontend_url}
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_beta_welcome_email(self, email: str, credits: int = 1) -> bool:
|
||||
"""
|
||||
Send welcome email to new beta testers.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Initial free credits granted
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Welcome Beta Tester! Free Karaoke Credits Inside 🎤"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.beta-badge {{
|
||||
display: inline-block;
|
||||
background: linear-gradient(135deg, #8b5cf6, #ec4899);
|
||||
color: white;
|
||||
padding: 4px 12px;
|
||||
border-radius: 20px;
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
margin-left: 8px;
|
||||
}}
|
||||
.credits-box {{
|
||||
background-color: #ecfdf5;
|
||||
border: 2px solid #10b981;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.credits-number {{
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
color: #10b981;
|
||||
}}
|
||||
.reminder {{
|
||||
background-color: #fef3c7;
|
||||
border: 1px solid #fcd34d;
|
||||
border-radius: 8px;
|
||||
padding: 16px;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<span class="logo">🎤 Nomad Karaoke</span>
|
||||
<span class="beta-badge">BETA TESTER</span>
|
||||
</div>
|
||||
|
||||
<p>Thank you for joining our beta program! 🎉</p>
|
||||
|
||||
<p>As promised, here's your free credit to create a karaoke video:</p>
|
||||
|
||||
<div class="credits-box">
|
||||
<div>Your balance:</div>
|
||||
<div class="credits-number">{credits}</div>
|
||||
<div>free credit{'' if credits == 1 else 's'}</div>
|
||||
</div>
|
||||
|
||||
<div class="reminder">
|
||||
<strong>📝 Your Promise:</strong><br>
|
||||
Remember, as a beta tester you agreed to:
|
||||
<ul style="margin: 10px 0; padding-left: 20px;">
|
||||
<li>Review and correct any lyrics timing issues</li>
|
||||
<li>Share your honest feedback after trying the tool</li>
|
||||
</ul>
|
||||
We'll send you a quick feedback form after your job completes!
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Create Your Karaoke Video</a>
|
||||
</p>
|
||||
|
||||
<p>Thanks for helping us make Nomad Karaoke better!</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Welcome Beta Tester!
|
||||
|
||||
Thank you for joining our beta program!
|
||||
|
||||
You've been granted {credits} free credit{'' if credits == 1 else 's'} to create karaoke videos.
|
||||
|
||||
YOUR PROMISE:
|
||||
As a beta tester, you agreed to:
|
||||
- Review and correct any lyrics timing issues
|
||||
- Share your honest feedback after trying the tool
|
||||
|
||||
We'll send you a quick feedback form after your job completes!
|
||||
|
||||
Create your karaoke video: {self.frontend_url}
|
||||
|
||||
Thanks for helping us make Nomad Karaoke better!
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_feedback_request_email(self, email: str, feedback_url: str, job_title: Optional[str] = None) -> bool:
|
||||
"""
|
||||
Send feedback request email to beta testers.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
feedback_url: URL to the feedback form
|
||||
job_title: Optional title of the completed job
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Quick feedback on your karaoke experience? 🎤"
|
||||
|
||||
# Escape job_title to prevent XSS in email clients that render HTML
|
||||
safe_job_title = html.escape(job_title) if job_title else ""
|
||||
job_context = f" for <strong>{safe_job_title}</strong>" if job_title else ""
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.feedback-box {{
|
||||
background: linear-gradient(135deg, #f0f9ff, #e0f2fe);
|
||||
border: 2px solid #3b82f6;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.stars {{
|
||||
font-size: 36px;
|
||||
margin: 10px 0;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.time-note {{
|
||||
background-color: #fef3c7;
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
margin: 20px 0;
|
||||
font-size: 14px;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Hi there!</p>
|
||||
|
||||
<p>Hope you enjoyed creating your karaoke video{job_context}! As a beta tester, your feedback is super valuable to us.</p>
|
||||
|
||||
<div class="feedback-box">
|
||||
<div class="stars">⭐⭐⭐⭐⭐</div>
|
||||
<p><strong>How was your experience?</strong></p>
|
||||
<p>Quick 2-minute survey - we'd love to hear your thoughts!</p>
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{feedback_url}" class="button">Share Your Feedback</a>
|
||||
</p>
|
||||
|
||||
<div class="time-note">
|
||||
⏱️ This takes less than 2 minutes and helps us improve the tool for everyone!
|
||||
</div>
|
||||
|
||||
<p>Specifically, we'd love to know:</p>
|
||||
<ul>
|
||||
<li>How easy was it to use?</li>
|
||||
<li>Were the lyrics accurate?</li>
|
||||
<li>How was the correction experience?</li>
|
||||
<li>What could we improve?</li>
|
||||
</ul>
|
||||
|
||||
<p>Thanks for being part of making Nomad Karaoke better! 🙏</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Quick feedback on your karaoke experience?
|
||||
|
||||
Hi there!
|
||||
|
||||
Hope you enjoyed creating your karaoke video{' for ' + safe_job_title if safe_job_title else ''}!
|
||||
|
||||
As a beta tester, your feedback is super valuable to us.
|
||||
|
||||
Share your feedback (2-minute survey): {feedback_url}
|
||||
|
||||
We'd love to know:
|
||||
- How easy was it to use?
|
||||
- Were the lyrics accurate?
|
||||
- How was the correction experience?
|
||||
- What could we improve?
|
||||
|
||||
Thanks for being part of making Nomad Karaoke better!
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def _get_year(self) -> int:
|
||||
"""Get current year for copyright notices."""
|
||||
from datetime import datetime
|
||||
return datetime.now().year
|
||||
|
||||
|
||||
# Global instance
|
||||
_email_service = None
|
||||
|
||||
|
||||
def get_email_service() -> EmailService:
|
||||
"""Get the global email service instance."""
|
||||
global _email_service
|
||||
if _email_service is None:
|
||||
_email_service = EmailService()
|
||||
return _email_service
|
||||
@@ -114,53 +114,47 @@ class FirestoreService:
|
||||
client_id: Optional[str] = None,
|
||||
created_after: Optional[datetime] = None,
|
||||
created_before: Optional[datetime] = None,
|
||||
user_email: Optional[str] = None,
|
||||
limit: int = 100
|
||||
) -> List[Job]:
|
||||
"""
|
||||
List jobs with optional filters.
|
||||
|
||||
|
||||
Args:
|
||||
status: Filter by job status
|
||||
environment: Filter by request_metadata.environment (test/production/development)
|
||||
client_id: Filter by request_metadata.client_id
|
||||
created_after: Filter jobs created after this datetime
|
||||
created_before: Filter jobs created before this datetime
|
||||
user_email: Filter by user_email (owner of the job)
|
||||
limit: Maximum number of jobs to return
|
||||
|
||||
|
||||
Returns:
|
||||
List of Job objects matching filters, ordered by created_at descending
|
||||
"""
|
||||
try:
|
||||
query = self.db.collection(self.collection)
|
||||
|
||||
|
||||
if status:
|
||||
query = query.where(filter=FieldFilter('status', '==', status.value))
|
||||
|
||||
|
||||
# Filter by request_metadata fields using dot notation
|
||||
if environment:
|
||||
query = query.where(filter=FieldFilter('request_metadata.environment', '==', environment))
|
||||
|
||||
|
||||
if client_id:
|
||||
query = query.where(filter=FieldFilter('request_metadata.client_id', '==', client_id))
|
||||
|
||||
# Filter by user_email (job owner)
|
||||
if user_email:
|
||||
query = query.where(filter=FieldFilter('user_email', '==', user_email.lower()))
|
||||
|
||||
|
||||
# Date range filters
|
||||
if created_after:
|
||||
query = query.where(filter=FieldFilter('created_at', '>=', created_after))
|
||||
|
||||
|
||||
if created_before:
|
||||
query = query.where(filter=FieldFilter('created_at', '<=', created_before))
|
||||
|
||||
|
||||
query = query.order_by('created_at', direction=firestore.Query.DESCENDING).limit(limit)
|
||||
|
||||
|
||||
docs = query.stream()
|
||||
jobs = [Job(**doc.to_dict()) for doc in docs]
|
||||
|
||||
|
||||
return jobs
|
||||
except Exception as e:
|
||||
logger.error(f"Error listing jobs: {e}")
|
||||
|
||||
@@ -136,10 +136,8 @@ class GoogleDriveService:
|
||||
logger.info(f"Looking for folder '{folder_name}' in parent {parent_id}")
|
||||
|
||||
# Search for existing folder
|
||||
# Escape single quotes in folder name for Google Drive API query syntax
|
||||
escaped_folder_name = folder_name.replace("'", "\\'")
|
||||
query = (
|
||||
f"name='{escaped_folder_name}' and '{parent_id}' in parents "
|
||||
f"name='{folder_name}' and '{parent_id}' in parents "
|
||||
f"and mimeType='application/vnd.google-apps.folder' and trashed=false"
|
||||
)
|
||||
results = self.service.files().list(q=query, fields="files(id, name)").execute()
|
||||
@@ -205,10 +203,8 @@ class GoogleDriveService:
|
||||
|
||||
# Check for existing file with same name
|
||||
if replace_existing:
|
||||
# Escape single quotes in filename for Google Drive API query syntax
|
||||
escaped_filename = filename.replace("'", "\\'")
|
||||
query = (
|
||||
f"name='{escaped_filename}' and '{parent_id}' in parents and trashed=false"
|
||||
f"name='{filename}' and '{parent_id}' in parents and trashed=false"
|
||||
)
|
||||
results = self.service.files().list(q=query, fields="files(id)").execute()
|
||||
for existing_file in results.get("files", []):
|
||||
|
||||
@@ -61,11 +61,6 @@ class JobManager:
|
||||
discord_webhook_url=job_create.discord_webhook_url,
|
||||
dropbox_path=job_create.dropbox_path,
|
||||
gdrive_folder_id=job_create.gdrive_folder_id,
|
||||
# Theme configuration
|
||||
theme_id=job_create.theme_id,
|
||||
color_overrides=job_create.color_overrides,
|
||||
style_params_gcs_path=job_create.style_params_gcs_path,
|
||||
style_assets=job_create.style_assets,
|
||||
# Two-phase workflow (Batch 6)
|
||||
prep_only=job_create.prep_only,
|
||||
finalise_only=job_create.finalise_only,
|
||||
@@ -111,21 +106,19 @@ class JobManager:
|
||||
client_id: Optional[str] = None,
|
||||
created_after: Optional[datetime] = None,
|
||||
created_before: Optional[datetime] = None,
|
||||
user_email: Optional[str] = None,
|
||||
limit: int = 100
|
||||
) -> List[Job]:
|
||||
"""
|
||||
List jobs with optional filtering.
|
||||
|
||||
|
||||
Args:
|
||||
status: Filter by job status
|
||||
environment: Filter by request_metadata.environment (test/production/development)
|
||||
client_id: Filter by request_metadata.client_id (customer identifier)
|
||||
created_after: Filter jobs created after this datetime
|
||||
created_before: Filter jobs created before this datetime
|
||||
user_email: Filter by user_email (job owner)
|
||||
limit: Maximum number of jobs to return
|
||||
|
||||
|
||||
Returns:
|
||||
List of Job objects matching filters
|
||||
"""
|
||||
@@ -135,7 +128,6 @@ class JobManager:
|
||||
client_id=client_id,
|
||||
created_after=created_after,
|
||||
created_before=created_before,
|
||||
user_email=user_email,
|
||||
limit=limit
|
||||
)
|
||||
|
||||
|
||||
@@ -1,275 +0,0 @@
|
||||
"""
|
||||
Stripe service for payment processing.
|
||||
|
||||
Handles:
|
||||
- Creating checkout sessions for credit purchases
|
||||
- Processing webhook events
|
||||
- Managing customer records
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
from typing import Optional, Dict, Any, Tuple
|
||||
|
||||
import stripe
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Credit packages available for purchase
|
||||
CREDIT_PACKAGES = {
|
||||
"1_credit": {
|
||||
"credits": 1,
|
||||
"price_cents": 500, # $5.00
|
||||
"name": "1 Karaoke Credit",
|
||||
"description": "Create 1 professional karaoke video",
|
||||
},
|
||||
"3_credits": {
|
||||
"credits": 3,
|
||||
"price_cents": 1200, # $12.00 (20% discount)
|
||||
"name": "3 Karaoke Credits",
|
||||
"description": "Create 3 professional karaoke videos (Save 20%)",
|
||||
},
|
||||
"5_credits": {
|
||||
"credits": 5,
|
||||
"price_cents": 1750, # $17.50 (30% discount)
|
||||
"name": "5 Karaoke Credits",
|
||||
"description": "Create 5 professional karaoke videos (Save 30%)",
|
||||
},
|
||||
"10_credits": {
|
||||
"credits": 10,
|
||||
"price_cents": 3000, # $30.00 (40% discount)
|
||||
"name": "10 Karaoke Credits",
|
||||
"description": "Create 10 professional karaoke videos (Save 40%)",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class StripeService:
|
||||
"""Service for Stripe payment processing."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize Stripe with API key."""
|
||||
self.settings = get_settings()
|
||||
self.secret_key = os.getenv("STRIPE_SECRET_KEY")
|
||||
self.webhook_secret = os.getenv("STRIPE_WEBHOOK_SECRET")
|
||||
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
|
||||
# After consolidation, buy URL is the same as frontend URL
|
||||
self.buy_url = os.getenv("BUY_URL", self.frontend_url)
|
||||
|
||||
if self.secret_key:
|
||||
stripe.api_key = self.secret_key
|
||||
logger.info("Stripe initialized with API key")
|
||||
else:
|
||||
logger.warning("STRIPE_SECRET_KEY not set - payments disabled")
|
||||
|
||||
def is_configured(self) -> bool:
|
||||
"""Check if Stripe is properly configured."""
|
||||
return bool(self.secret_key)
|
||||
|
||||
def get_credit_packages(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get available credit packages."""
|
||||
return CREDIT_PACKAGES
|
||||
|
||||
def create_checkout_session(
|
||||
self,
|
||||
package_id: str,
|
||||
user_email: str,
|
||||
success_url: Optional[str] = None,
|
||||
cancel_url: Optional[str] = None,
|
||||
) -> Tuple[bool, Optional[str], str]:
|
||||
"""
|
||||
Create a Stripe Checkout session for purchasing credits.
|
||||
|
||||
Args:
|
||||
package_id: ID of the credit package to purchase
|
||||
user_email: Email of the purchasing user
|
||||
success_url: URL to redirect to on success (optional)
|
||||
cancel_url: URL to redirect to on cancel (optional)
|
||||
|
||||
Returns:
|
||||
(success, checkout_url, message)
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return False, None, "Payment processing is not configured"
|
||||
|
||||
package = CREDIT_PACKAGES.get(package_id)
|
||||
if not package:
|
||||
return False, None, f"Invalid package: {package_id}"
|
||||
|
||||
try:
|
||||
# Default URLs
|
||||
if not success_url:
|
||||
success_url = f"{self.frontend_url}/payment/success?session_id={{CHECKOUT_SESSION_ID}}"
|
||||
if not cancel_url:
|
||||
cancel_url = f"{self.buy_url}?cancelled=true"
|
||||
|
||||
# Create checkout session
|
||||
session = stripe.checkout.Session.create(
|
||||
payment_method_types=['card'],
|
||||
line_items=[{
|
||||
'price_data': {
|
||||
'currency': 'usd',
|
||||
'product_data': {
|
||||
'name': package['name'],
|
||||
'description': package['description'],
|
||||
},
|
||||
'unit_amount': package['price_cents'],
|
||||
},
|
||||
'quantity': 1,
|
||||
}],
|
||||
mode='payment',
|
||||
success_url=success_url,
|
||||
cancel_url=cancel_url,
|
||||
customer_email=user_email,
|
||||
metadata={
|
||||
'package_id': package_id,
|
||||
'credits': str(package['credits']),
|
||||
'user_email': user_email,
|
||||
},
|
||||
# Allow promotion codes
|
||||
allow_promotion_codes=True,
|
||||
)
|
||||
|
||||
logger.info(f"Created checkout session {session.id} for {user_email}, package {package_id}")
|
||||
return True, session.url, "Checkout session created"
|
||||
|
||||
except stripe.error.StripeError as e:
|
||||
logger.error(f"Stripe error creating checkout session: {e}")
|
||||
return False, None, f"Payment error: {str(e)}"
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating checkout session: {e}")
|
||||
return False, None, "Failed to create checkout session"
|
||||
|
||||
def verify_webhook_signature(self, payload: bytes, signature: str) -> Tuple[bool, Optional[Dict], str]:
|
||||
"""
|
||||
Verify a Stripe webhook signature.
|
||||
|
||||
Args:
|
||||
payload: Raw request body
|
||||
signature: Stripe-Signature header value
|
||||
|
||||
Returns:
|
||||
(valid, event_data, message)
|
||||
"""
|
||||
if not self.webhook_secret:
|
||||
logger.error("STRIPE_WEBHOOK_SECRET not configured")
|
||||
return False, None, "Webhook secret not configured"
|
||||
|
||||
try:
|
||||
event = stripe.Webhook.construct_event(
|
||||
payload, signature, self.webhook_secret
|
||||
)
|
||||
return True, event, "Webhook verified"
|
||||
except stripe.error.SignatureVerificationError as e:
|
||||
logger.error(f"Invalid webhook signature: {e}")
|
||||
return False, None, "Invalid signature"
|
||||
except Exception as e:
|
||||
logger.error(f"Error verifying webhook: {e}")
|
||||
return False, None, str(e)
|
||||
|
||||
def handle_checkout_completed(self, session: Dict) -> Tuple[bool, Optional[str], int, str]:
|
||||
"""
|
||||
Handle a completed checkout session.
|
||||
|
||||
Args:
|
||||
session: Stripe checkout session object
|
||||
|
||||
Returns:
|
||||
(success, user_email, credits_to_add, message)
|
||||
"""
|
||||
try:
|
||||
metadata = session.get('metadata', {})
|
||||
user_email = metadata.get('user_email') or session.get('customer_email')
|
||||
try:
|
||||
credits = int(metadata.get('credits', 0))
|
||||
except (ValueError, TypeError):
|
||||
logger.error(f"Invalid credits metadata in session {session.get('id')}: {metadata.get('credits')}")
|
||||
return False, user_email, 0, "Invalid credit amount in session metadata"
|
||||
package_id = metadata.get('package_id')
|
||||
|
||||
if not user_email:
|
||||
logger.error(f"No user email in checkout session {session.get('id')}")
|
||||
return False, None, 0, "No user email found"
|
||||
|
||||
if credits <= 0:
|
||||
logger.error(f"Invalid credits in checkout session {session.get('id')}")
|
||||
return False, user_email, 0, "Invalid credit amount"
|
||||
|
||||
logger.info(
|
||||
f"Checkout completed: {user_email} purchased {credits} credits "
|
||||
f"(package: {package_id}, session: {session.get('id')})"
|
||||
)
|
||||
|
||||
return True, user_email, credits, f"Successfully purchased {credits} credits"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error handling checkout completed: {e}")
|
||||
return False, None, 0, str(e)
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get a checkout session by ID.
|
||||
|
||||
Args:
|
||||
session_id: Stripe checkout session ID
|
||||
|
||||
Returns:
|
||||
Session data or None
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return None
|
||||
|
||||
try:
|
||||
session = stripe.checkout.Session.retrieve(session_id)
|
||||
return dict(session)
|
||||
except Exception as e:
|
||||
logger.error(f"Error retrieving session {session_id}: {e}")
|
||||
return None
|
||||
|
||||
def create_customer(self, email: str, name: Optional[str] = None) -> Optional[str]:
|
||||
"""
|
||||
Create or get a Stripe customer for a user.
|
||||
|
||||
Args:
|
||||
email: User's email
|
||||
name: User's display name (optional)
|
||||
|
||||
Returns:
|
||||
Stripe customer ID or None
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return None
|
||||
|
||||
try:
|
||||
# Check if customer already exists
|
||||
customers = stripe.Customer.list(email=email, limit=1)
|
||||
if customers.data:
|
||||
return customers.data[0].id
|
||||
|
||||
# Create new customer
|
||||
customer = stripe.Customer.create(
|
||||
email=email,
|
||||
name=name,
|
||||
metadata={'source': 'nomad_karaoke'},
|
||||
)
|
||||
logger.info(f"Created Stripe customer {customer.id} for {email}")
|
||||
return customer.id
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating Stripe customer: {e}")
|
||||
return None
|
||||
|
||||
|
||||
# Global instance
|
||||
_stripe_service = None
|
||||
|
||||
|
||||
def get_stripe_service() -> StripeService:
|
||||
"""Get the global Stripe service instance."""
|
||||
global _stripe_service
|
||||
if _stripe_service is None:
|
||||
_stripe_service = StripeService()
|
||||
return _stripe_service
|
||||
@@ -91,15 +91,7 @@ class StructuredFormatter(logging.Formatter):
|
||||
|
||||
# Add custom fields from 'extra' dict
|
||||
# Common fields we want to extract from log records
|
||||
custom_fields = [
|
||||
# Job-related fields
|
||||
"job_id", "worker", "operation", "duration", "status", "error",
|
||||
# Audit logging fields (from middleware and auth)
|
||||
"request_id", "user_email", "client_ip", "latency_ms",
|
||||
"audit_type", "method", "path", "status_code", "query_string",
|
||||
"user_agent", "user_type", "is_admin", "remaining_uses",
|
||||
"auth_message", "token_provided", "token_length", "auth_header_present",
|
||||
]
|
||||
custom_fields = ["job_id", "worker", "operation", "duration", "status", "error"]
|
||||
for field in custom_fields:
|
||||
value = getattr(record, field, None)
|
||||
if value is not None:
|
||||
|
||||
@@ -1,721 +0,0 @@
|
||||
"""
|
||||
User service for authentication, credits, and user management.
|
||||
|
||||
Handles:
|
||||
- Magic link authentication (send, verify)
|
||||
- Session management
|
||||
- Credit operations (add, deduct, refund)
|
||||
- User CRUD operations
|
||||
"""
|
||||
import logging
|
||||
import secrets
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional, List, Tuple
|
||||
|
||||
from google.cloud import firestore
|
||||
from google.cloud.firestore_v1 import FieldFilter
|
||||
from google.cloud.firestore_v1 import Increment
|
||||
|
||||
from backend.config import get_settings
|
||||
from backend.models.user import (
|
||||
User,
|
||||
UserRole,
|
||||
UserPublic,
|
||||
MagicLinkToken,
|
||||
Session,
|
||||
CreditTransaction,
|
||||
)
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Collection names
|
||||
USERS_COLLECTION = "users"
|
||||
MAGIC_LINKS_COLLECTION = "magic_links"
|
||||
SESSIONS_COLLECTION = "sessions"
|
||||
PROCESSED_STRIPE_SESSIONS_COLLECTION = "processed_stripe_sessions"
|
||||
|
||||
# Token/session configuration
|
||||
MAGIC_LINK_EXPIRY_MINUTES = 15
|
||||
SESSION_EXPIRY_DAYS = 7
|
||||
SESSION_ABSOLUTE_EXPIRY_DAYS = 30
|
||||
MAX_CREDIT_TRANSACTIONS = 100 # Keep last N transactions
|
||||
|
||||
|
||||
class UserService:
|
||||
"""Service for user management and authentication."""
|
||||
|
||||
# Number of free credits granted to new users
|
||||
NEW_USER_FREE_CREDITS = 1
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize user service with Firestore client."""
|
||||
self.settings = get_settings()
|
||||
self.db = firestore.Client(project=self.settings.google_cloud_project)
|
||||
|
||||
# =========================================================================
|
||||
# User CRUD Operations
|
||||
# =========================================================================
|
||||
|
||||
def get_user(self, email: str) -> Optional[User]:
|
||||
"""Get a user by email."""
|
||||
try:
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc = doc_ref.get()
|
||||
|
||||
if not doc.exists:
|
||||
return None
|
||||
|
||||
return User(**doc.to_dict())
|
||||
except Exception:
|
||||
logger.exception(f"Error getting user {email}")
|
||||
return None
|
||||
|
||||
def get_or_create_user(self, email: str) -> User:
|
||||
"""
|
||||
Get existing user or create a new one.
|
||||
|
||||
New users receive a welcome credit to try the service.
|
||||
"""
|
||||
email = email.lower()
|
||||
user = self.get_user(email)
|
||||
|
||||
if user:
|
||||
return user
|
||||
|
||||
# Create new user with welcome credit
|
||||
welcome_credit = self.NEW_USER_FREE_CREDITS
|
||||
welcome_transaction = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=welcome_credit,
|
||||
reason="welcome_credit",
|
||||
)
|
||||
|
||||
user = User(
|
||||
email=email,
|
||||
credits=welcome_credit,
|
||||
credit_transactions=[welcome_transaction],
|
||||
)
|
||||
self._save_user(user)
|
||||
logger.info(f"Created new user: {email} with {welcome_credit} welcome credit(s)")
|
||||
return user
|
||||
|
||||
def _save_user(self, user: User) -> None:
|
||||
"""Save user to Firestore."""
|
||||
try:
|
||||
user.updated_at = datetime.utcnow()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(user.email.lower())
|
||||
doc_ref.set(user.model_dump(mode='json'))
|
||||
except Exception:
|
||||
logger.exception(f"Error saving user {user.email}")
|
||||
raise
|
||||
|
||||
def update_user(self, email: str, **updates) -> Optional[User]:
|
||||
"""Update user fields."""
|
||||
try:
|
||||
updates['updated_at'] = datetime.utcnow()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc_ref.update(updates)
|
||||
return self.get_user(email)
|
||||
except Exception:
|
||||
logger.exception(f"Error updating user {email}")
|
||||
return None
|
||||
|
||||
def list_users(self, limit: int = 100, include_inactive: bool = False) -> List[User]:
|
||||
"""List all users (admin only)."""
|
||||
try:
|
||||
query = self.db.collection(USERS_COLLECTION)
|
||||
|
||||
if not include_inactive:
|
||||
query = query.where(filter=FieldFilter('is_active', '==', True))
|
||||
|
||||
query = query.order_by('created_at', direction=firestore.Query.DESCENDING)
|
||||
query = query.limit(limit)
|
||||
|
||||
docs = query.stream()
|
||||
return [User(**doc.to_dict()) for doc in docs]
|
||||
except Exception:
|
||||
logger.exception("Error listing users")
|
||||
return []
|
||||
|
||||
def get_user_public(self, email: str) -> Optional[UserPublic]:
|
||||
"""Get public user info."""
|
||||
user = self.get_user(email)
|
||||
if not user:
|
||||
return None
|
||||
|
||||
return UserPublic(
|
||||
email=user.email,
|
||||
role=user.role,
|
||||
credits=user.credits,
|
||||
display_name=user.display_name,
|
||||
total_jobs_created=user.total_jobs_created,
|
||||
total_jobs_completed=user.total_jobs_completed,
|
||||
)
|
||||
|
||||
# =========================================================================
|
||||
# Magic Link Authentication
|
||||
# =========================================================================
|
||||
|
||||
def create_magic_link(
|
||||
self,
|
||||
email: str,
|
||||
ip_address: Optional[str] = None,
|
||||
user_agent: Optional[str] = None
|
||||
) -> MagicLinkToken:
|
||||
"""
|
||||
Create a magic link token for email authentication.
|
||||
|
||||
Returns the token object. The actual sending of the email
|
||||
should be handled by the caller (or an email service).
|
||||
"""
|
||||
email = email.lower()
|
||||
|
||||
# Ensure user exists
|
||||
self.get_or_create_user(email)
|
||||
|
||||
# Generate secure token
|
||||
token = secrets.token_urlsafe(32)
|
||||
|
||||
magic_link = MagicLinkToken(
|
||||
token=token,
|
||||
email=email,
|
||||
expires_at=datetime.utcnow() + timedelta(minutes=MAGIC_LINK_EXPIRY_MINUTES),
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent,
|
||||
)
|
||||
|
||||
# Save to Firestore
|
||||
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
|
||||
doc_ref.set(magic_link.model_dump(mode='json'))
|
||||
|
||||
logger.info(f"Created magic link for {email}")
|
||||
return magic_link
|
||||
|
||||
def verify_magic_link(self, token: str) -> Tuple[bool, Optional[User], str]:
|
||||
"""
|
||||
Verify a magic link token using a Firestore transaction to prevent race conditions.
|
||||
|
||||
Returns:
|
||||
(success, user, message)
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
|
||||
|
||||
@firestore.transactional
|
||||
def verify_in_transaction(transaction):
|
||||
"""Atomically verify and mark magic link as used."""
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
|
||||
if not doc.exists:
|
||||
return False, None, "Invalid or expired link"
|
||||
|
||||
magic_link = MagicLinkToken(**doc.to_dict())
|
||||
|
||||
# Check if already used
|
||||
if magic_link.used:
|
||||
return False, None, "This link has already been used"
|
||||
|
||||
# Check expiry
|
||||
if datetime.utcnow() > magic_link.expires_at:
|
||||
return False, None, "This link has expired"
|
||||
|
||||
# Mark as used atomically within transaction
|
||||
transaction.update(doc_ref, {
|
||||
'used': True,
|
||||
'used_at': datetime.utcnow()
|
||||
})
|
||||
|
||||
return True, magic_link.email, "Success"
|
||||
|
||||
# Execute the transaction
|
||||
transaction = self.db.transaction()
|
||||
success, email_or_error, message = verify_in_transaction(transaction)
|
||||
|
||||
if not success:
|
||||
return False, None, message
|
||||
|
||||
# Get user and mark email as verified (outside transaction)
|
||||
user = self.get_user(email_or_error)
|
||||
if user:
|
||||
self.update_user(
|
||||
email_or_error,
|
||||
email_verified=True,
|
||||
last_login_at=datetime.utcnow()
|
||||
)
|
||||
user = self.get_user(email_or_error) # Refresh
|
||||
|
||||
logger.info(f"Magic link verified for {email_or_error}")
|
||||
return True, user, "Successfully authenticated"
|
||||
|
||||
except Exception:
|
||||
logger.exception("Error verifying magic link")
|
||||
return False, None, "An error occurred during verification"
|
||||
|
||||
# =========================================================================
|
||||
# Session Management
|
||||
# =========================================================================
|
||||
|
||||
def create_session(
|
||||
self,
|
||||
user_email: str,
|
||||
ip_address: Optional[str] = None,
|
||||
user_agent: Optional[str] = None
|
||||
) -> Session:
|
||||
"""Create a new session for an authenticated user."""
|
||||
token = secrets.token_urlsafe(32)
|
||||
token_prefix = token[:12]
|
||||
|
||||
session = Session(
|
||||
token=token,
|
||||
user_email=user_email.lower(),
|
||||
expires_at=datetime.utcnow() + timedelta(days=SESSION_ABSOLUTE_EXPIRY_DAYS),
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent,
|
||||
)
|
||||
|
||||
# Serialize and write to Firestore
|
||||
session_data = session.model_dump(mode='json')
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc_ref.set(session_data)
|
||||
|
||||
# Verify the write succeeded by reading back
|
||||
verify_doc = doc_ref.get()
|
||||
if not verify_doc.exists:
|
||||
logger.error(f"Session write verification FAILED for {user_email}: {token_prefix}...")
|
||||
raise RuntimeError(f"Failed to persist session for {user_email}")
|
||||
|
||||
logger.info(f"Created and verified session for {user_email}: {token_prefix}... (expires: {session.expires_at})")
|
||||
|
||||
return session
|
||||
|
||||
def validate_session(self, token: str) -> Tuple[bool, Optional[User], str]:
|
||||
"""
|
||||
Validate a session token.
|
||||
|
||||
Returns:
|
||||
(valid, user, message)
|
||||
"""
|
||||
# Log token info for debugging (only prefix for security)
|
||||
token_prefix = token[:12] if token and len(token) >= 12 else token
|
||||
logger.debug(f"Validating session token: {token_prefix}... (len={len(token) if token else 0})")
|
||||
|
||||
try:
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc = doc_ref.get()
|
||||
|
||||
if not doc.exists:
|
||||
logger.warning(f"Session not found in Firestore: {token_prefix}...")
|
||||
return False, None, "Invalid session"
|
||||
|
||||
raw_data = doc.to_dict()
|
||||
logger.debug(f"Session data found for {token_prefix}...: user_email={raw_data.get('user_email')}, is_active={raw_data.get('is_active')}")
|
||||
|
||||
session = Session(**raw_data)
|
||||
|
||||
# Check if active
|
||||
if not session.is_active:
|
||||
logger.warning(f"Session revoked for {token_prefix}... (user: {session.user_email})")
|
||||
return False, None, "Session has been revoked"
|
||||
|
||||
# Use timezone-aware datetime for all comparisons
|
||||
# Firestore returns timezone-aware datetimes, so we must use aware datetimes
|
||||
now = datetime.now(timezone.utc)
|
||||
|
||||
# Normalize session datetimes to be timezone-aware for comparison
|
||||
# (handles legacy naive datetimes that may exist in Firestore)
|
||||
expires_at = session.expires_at
|
||||
if expires_at.tzinfo is None:
|
||||
expires_at = expires_at.replace(tzinfo=timezone.utc)
|
||||
|
||||
last_activity = session.last_activity_at
|
||||
if last_activity.tzinfo is None:
|
||||
last_activity = last_activity.replace(tzinfo=timezone.utc)
|
||||
|
||||
# Check expiry
|
||||
if now > expires_at:
|
||||
logger.warning(f"Session expired for {token_prefix}... (user: {session.user_email}, expired_at: {expires_at}, now: {now})")
|
||||
return False, None, "Session has expired"
|
||||
|
||||
# Check inactivity (7 days)
|
||||
inactivity_limit = now - timedelta(days=SESSION_EXPIRY_DAYS)
|
||||
if last_activity < inactivity_limit:
|
||||
logger.warning(f"Session inactive for {token_prefix}... (user: {session.user_email}, last_activity: {last_activity}, limit: {inactivity_limit})")
|
||||
return False, None, "Session expired due to inactivity"
|
||||
|
||||
# Update last activity (use timezone-aware datetime)
|
||||
doc_ref.update({'last_activity_at': now})
|
||||
|
||||
# Get user
|
||||
user = self.get_user(session.user_email)
|
||||
if not user:
|
||||
logger.warning(f"User not found for session {token_prefix}...: {session.user_email}")
|
||||
return False, None, "User not found"
|
||||
|
||||
if not user.is_active:
|
||||
logger.warning(f"User account disabled for session {token_prefix}...: {session.user_email}")
|
||||
return False, None, "User account is disabled"
|
||||
|
||||
logger.debug(f"Session valid for {token_prefix}... (user: {user.email}, credits: {user.credits})")
|
||||
return True, user, "Valid session"
|
||||
|
||||
except Exception:
|
||||
logger.exception(f"Error validating session {token_prefix}...")
|
||||
return False, None, "An error occurred during validation"
|
||||
|
||||
def revoke_session(self, token: str) -> bool:
|
||||
"""Revoke a session (logout)."""
|
||||
try:
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc_ref.update({'is_active': False})
|
||||
logger.info("Session revoked")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error revoking session")
|
||||
return False
|
||||
|
||||
def revoke_all_sessions(self, user_email: str) -> int:
|
||||
"""Revoke all sessions for a user."""
|
||||
try:
|
||||
query = self.db.collection(SESSIONS_COLLECTION).where(
|
||||
filter=FieldFilter('user_email', '==', user_email.lower())
|
||||
).where(
|
||||
filter=FieldFilter('is_active', '==', True)
|
||||
)
|
||||
|
||||
count = 0
|
||||
for doc in query.stream():
|
||||
doc.reference.update({'is_active': False})
|
||||
count += 1
|
||||
|
||||
logger.info(f"Revoked {count} sessions for {user_email}")
|
||||
return count
|
||||
except Exception:
|
||||
logger.exception("Error revoking sessions")
|
||||
return 0
|
||||
|
||||
def list_sessions_for_user(
|
||||
self,
|
||||
user_email: str,
|
||||
include_revoked: bool = False,
|
||||
limit: int = 50
|
||||
) -> List[Session]:
|
||||
"""
|
||||
List all sessions for a user.
|
||||
|
||||
Args:
|
||||
user_email: User's email address
|
||||
include_revoked: If True, include revoked/inactive sessions
|
||||
limit: Maximum number of sessions to return
|
||||
|
||||
Returns:
|
||||
List of Session objects, ordered by created_at descending
|
||||
"""
|
||||
try:
|
||||
query = self.db.collection(SESSIONS_COLLECTION).where(
|
||||
filter=FieldFilter('user_email', '==', user_email.lower())
|
||||
)
|
||||
|
||||
if not include_revoked:
|
||||
query = query.where(filter=FieldFilter('is_active', '==', True))
|
||||
|
||||
query = query.order_by('created_at', direction=firestore.Query.DESCENDING)
|
||||
query = query.limit(limit)
|
||||
|
||||
sessions = []
|
||||
for doc in query.stream():
|
||||
try:
|
||||
sessions.append(Session(**doc.to_dict()))
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to parse session document: {e}")
|
||||
|
||||
logger.debug(f"Found {len(sessions)} sessions for {user_email}")
|
||||
return sessions
|
||||
except Exception:
|
||||
logger.exception(f"Error listing sessions for {user_email}")
|
||||
return []
|
||||
|
||||
# =========================================================================
|
||||
# Credit Operations
|
||||
# =========================================================================
|
||||
|
||||
def add_credits(
|
||||
self,
|
||||
email: str,
|
||||
amount: int,
|
||||
reason: str,
|
||||
job_id: Optional[str] = None,
|
||||
stripe_session_id: Optional[str] = None,
|
||||
admin_email: Optional[str] = None
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Add credits to a user account.
|
||||
|
||||
Returns:
|
||||
(success, new_balance, message)
|
||||
"""
|
||||
try:
|
||||
# For Stripe sessions, atomically check and mark as processed
|
||||
# Uses create() which fails if document exists - ensures idempotency
|
||||
if stripe_session_id:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
try:
|
||||
doc_ref.create({
|
||||
'stripe_session_id': stripe_session_id,
|
||||
'email': email.lower(),
|
||||
'amount': amount,
|
||||
'processed_at': datetime.utcnow()
|
||||
})
|
||||
except Exception:
|
||||
# Document already exists - this session was already processed
|
||||
logger.info(f"Stripe session {stripe_session_id} already processed (idempotent skip)")
|
||||
return False, 0, "Session already processed"
|
||||
|
||||
email = email.lower()
|
||||
user = self.get_or_create_user(email)
|
||||
|
||||
# Create transaction record
|
||||
transaction = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=amount,
|
||||
reason=reason,
|
||||
job_id=job_id,
|
||||
stripe_session_id=stripe_session_id,
|
||||
created_by=admin_email,
|
||||
)
|
||||
|
||||
# Add to transaction history (keep last N)
|
||||
transactions = user.credit_transactions[-MAX_CREDIT_TRANSACTIONS + 1:]
|
||||
transactions.append(transaction)
|
||||
|
||||
# Update user
|
||||
new_balance = user.credits + amount
|
||||
self.update_user(
|
||||
email,
|
||||
credits=new_balance,
|
||||
credit_transactions=[t.model_dump(mode='json') for t in transactions]
|
||||
)
|
||||
|
||||
logger.info(f"Added {amount} credits to {email} ({reason}). New balance: {new_balance}")
|
||||
return True, new_balance, f"Added {amount} credits"
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error adding credits to {email}")
|
||||
return False, 0, f"Failed to add credits: {e}"
|
||||
|
||||
def deduct_credit(
|
||||
self,
|
||||
email: str,
|
||||
job_id: str,
|
||||
reason: str = "job_creation"
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Deduct one credit from a user account.
|
||||
|
||||
Uses Firestore transaction to prevent race conditions.
|
||||
|
||||
Returns:
|
||||
(success, remaining_credits, message)
|
||||
"""
|
||||
try:
|
||||
email = email.lower()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email)
|
||||
|
||||
@firestore.transactional
|
||||
def deduct_in_transaction(transaction):
|
||||
"""Atomically check and deduct credit."""
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
|
||||
if not doc.exists:
|
||||
return False, 0, "User not found"
|
||||
|
||||
user_data = doc.to_dict()
|
||||
current_credits = user_data.get('credits', 0)
|
||||
|
||||
if current_credits <= 0:
|
||||
return False, 0, "Insufficient credits"
|
||||
|
||||
# Create transaction record
|
||||
credit_txn = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=-1,
|
||||
reason=reason,
|
||||
job_id=job_id,
|
||||
)
|
||||
|
||||
# Get existing transactions and add new one
|
||||
existing_transactions = user_data.get('credit_transactions', [])
|
||||
# Keep last N-1 transactions to make room for new one
|
||||
transactions = existing_transactions[-(MAX_CREDIT_TRANSACTIONS - 1):]
|
||||
transactions.append(credit_txn.model_dump(mode='json'))
|
||||
|
||||
# Calculate new values
|
||||
new_balance = current_credits - 1
|
||||
total_jobs = user_data.get('total_jobs_created', 0) + 1
|
||||
|
||||
# Update atomically within transaction
|
||||
transaction.update(doc_ref, {
|
||||
'credits': new_balance,
|
||||
'credit_transactions': transactions,
|
||||
'total_jobs_created': total_jobs,
|
||||
'updated_at': datetime.utcnow()
|
||||
})
|
||||
|
||||
return True, new_balance, f"Credit deducted. {new_balance} remaining"
|
||||
|
||||
# Execute the transaction
|
||||
fs_transaction = self.db.transaction()
|
||||
success, new_balance, message = deduct_in_transaction(fs_transaction)
|
||||
|
||||
if success:
|
||||
logger.info(f"Deducted 1 credit from {email} for job {job_id}. Remaining: {new_balance}")
|
||||
|
||||
return success, new_balance, message
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error deducting credit from {email}")
|
||||
return False, 0, f"Failed to deduct credit: {e}"
|
||||
|
||||
def refund_credit(
|
||||
self,
|
||||
email: str,
|
||||
job_id: str,
|
||||
reason: str = "job_failed"
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Refund one credit to a user account (e.g., after job failure).
|
||||
|
||||
Returns:
|
||||
(success, new_balance, message)
|
||||
"""
|
||||
return self.add_credits(email, 1, reason, job_id=job_id)
|
||||
|
||||
def check_credits(self, email: str) -> int:
|
||||
"""Check user's credit balance."""
|
||||
user = self.get_user(email)
|
||||
return user.credits if user else 0
|
||||
|
||||
def has_credits(self, email: str) -> bool:
|
||||
"""Check if user has at least one credit."""
|
||||
return self.check_credits(email) > 0
|
||||
|
||||
def is_stripe_session_processed(self, stripe_session_id: str) -> bool:
|
||||
"""
|
||||
Check if a Stripe session ID has already been processed.
|
||||
|
||||
Used to ensure idempotency of webhook processing - prevents
|
||||
duplicate credit additions if Stripe sends the same webhook twice.
|
||||
|
||||
Uses a dedicated collection for O(1) lookup instead of scanning all users.
|
||||
|
||||
Args:
|
||||
stripe_session_id: The Stripe checkout session ID
|
||||
|
||||
Returns:
|
||||
True if this session was already processed
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
doc = doc_ref.get()
|
||||
if doc.exists:
|
||||
logger.info(f"Stripe session {stripe_session_id} already processed")
|
||||
return True
|
||||
return False
|
||||
except Exception:
|
||||
logger.exception(f"Error checking if stripe session {stripe_session_id} was processed")
|
||||
# On error, return False to allow processing (better to risk duplicate than block)
|
||||
return False
|
||||
|
||||
def _mark_stripe_session_processed(self, stripe_session_id: str, email: str, amount: int) -> None:
|
||||
"""
|
||||
Mark a Stripe session as processed.
|
||||
|
||||
Called after successfully adding credits from a Stripe webhook.
|
||||
|
||||
Args:
|
||||
stripe_session_id: The Stripe checkout session ID
|
||||
email: User email who received the credits
|
||||
amount: Number of credits added
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
doc_ref.set({
|
||||
'stripe_session_id': stripe_session_id,
|
||||
'email': email,
|
||||
'amount': amount,
|
||||
'processed_at': datetime.utcnow()
|
||||
})
|
||||
except Exception:
|
||||
# Log but don't fail - the credits were already added
|
||||
logger.exception(f"Error marking stripe session {stripe_session_id} as processed")
|
||||
|
||||
# =========================================================================
|
||||
# Admin Operations
|
||||
# =========================================================================
|
||||
|
||||
def set_user_role(self, email: str, role: UserRole, admin_email: str) -> bool:
|
||||
"""Set a user's role (admin only)."""
|
||||
try:
|
||||
user = self.get_user(email)
|
||||
if not user:
|
||||
return False
|
||||
|
||||
self.update_user(email, role=role.value)
|
||||
logger.info(f"Admin {admin_email} set role for {email} to {role.value}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error setting user role")
|
||||
return False
|
||||
|
||||
def disable_user(self, email: str, admin_email: str) -> bool:
|
||||
"""Disable a user account (admin only)."""
|
||||
try:
|
||||
self.update_user(email, is_active=False)
|
||||
self.revoke_all_sessions(email)
|
||||
logger.info(f"Admin {admin_email} disabled user {email}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error disabling user")
|
||||
return False
|
||||
|
||||
def enable_user(self, email: str, admin_email: str) -> bool:
|
||||
"""Enable a user account (admin only)."""
|
||||
try:
|
||||
self.update_user(email, is_active=True)
|
||||
logger.info(f"Admin {admin_email} enabled user {email}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error enabling user")
|
||||
return False
|
||||
|
||||
def increment_jobs_completed(self, email: str) -> bool:
|
||||
"""Increment the completed jobs counter for a user using atomic increment."""
|
||||
try:
|
||||
# Check user exists first
|
||||
if not self.get_user(email):
|
||||
return False
|
||||
|
||||
# Use atomic increment to prevent race conditions
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc_ref.update({
|
||||
'total_jobs_completed': Increment(1),
|
||||
'updated_at': datetime.utcnow()
|
||||
})
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception(f"Error incrementing jobs completed for {email}")
|
||||
return False
|
||||
|
||||
|
||||
# Global instance
|
||||
_user_service = None
|
||||
|
||||
|
||||
def get_user_service() -> UserService:
|
||||
"""Get the global user service instance."""
|
||||
global _user_service
|
||||
if _user_service is None:
|
||||
_user_service = UserService()
|
||||
return _user_service
|
||||
@@ -330,7 +330,7 @@ class TestReviewEndpoints:
|
||||
|
||||
class TestRenderVideoWorker:
|
||||
"""Test render video worker endpoint."""
|
||||
|
||||
|
||||
def test_render_video_worker_endpoint_exists(self, client, auth_headers):
|
||||
"""Test that render-video worker endpoint exists."""
|
||||
# Create a job
|
||||
@@ -340,9 +340,9 @@ class TestRenderVideoWorker:
|
||||
json={"url": "https://youtube.com/watch?v=render-test"}
|
||||
)
|
||||
job_id = create_resp.json()["job_id"]
|
||||
|
||||
|
||||
time.sleep(0.2)
|
||||
|
||||
|
||||
# Test render-video worker endpoint exists
|
||||
response = client.post(
|
||||
"/api/internal/workers/render-video",
|
||||
|
||||
@@ -798,254 +798,6 @@ class TestRemoteDownloadPath:
|
||||
assert result.filepath == "/tmp/test.flac"
|
||||
|
||||
|
||||
class TestAudioSearchThemeSupport:
|
||||
"""Test theme support in audio search requests.
|
||||
|
||||
These tests verify that theme_id and color_overrides are properly:
|
||||
1. Accepted in AudioSearchRequest model
|
||||
2. Passed through to JobCreate
|
||||
3. Result in correct CDG/TXT defaults when theme is set
|
||||
4. Theme style is PREPARED (copied to job folder) when theme_id is set
|
||||
"""
|
||||
|
||||
def test_audio_search_request_accepts_theme_id(self):
|
||||
"""Test AudioSearchRequest model has theme_id field."""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest
|
||||
|
||||
request = AudioSearchRequest(
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
theme_id="nomad"
|
||||
)
|
||||
|
||||
assert request.theme_id == "nomad"
|
||||
|
||||
def test_audio_search_request_accepts_color_overrides(self):
|
||||
"""Test AudioSearchRequest model has color_overrides field."""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest
|
||||
|
||||
request = AudioSearchRequest(
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
theme_id="nomad",
|
||||
color_overrides={
|
||||
"artist_color": "#ff0000",
|
||||
"title_color": "#00ff00",
|
||||
}
|
||||
)
|
||||
|
||||
assert request.color_overrides["artist_color"] == "#ff0000"
|
||||
assert request.color_overrides["title_color"] == "#00ff00"
|
||||
|
||||
def test_audio_search_request_theme_defaults_cdg_txt(self):
|
||||
"""Test that when theme_id is set, CDG/TXT defaults to enabled.
|
||||
|
||||
This is the key behavior: selecting a theme should automatically
|
||||
enable CDG and TXT output formats.
|
||||
"""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
|
||||
|
||||
# When theme_id is set, enable_cdg/enable_txt should default to True
|
||||
request = AudioSearchRequest(
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
theme_id="nomad"
|
||||
# enable_cdg and enable_txt are None (not specified)
|
||||
)
|
||||
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
request.theme_id, request.enable_cdg, request.enable_txt
|
||||
)
|
||||
|
||||
assert resolved_cdg is True, "CDG should be enabled by default when theme is set"
|
||||
assert resolved_txt is True, "TXT should be enabled by default when theme is set"
|
||||
|
||||
def test_audio_search_request_no_theme_no_cdg_txt(self):
|
||||
"""Test that without theme_id, CDG/TXT defaults to disabled."""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
|
||||
|
||||
request = AudioSearchRequest(
|
||||
artist="Test Artist",
|
||||
title="Test Song"
|
||||
# No theme_id, no enable_cdg, no enable_txt
|
||||
)
|
||||
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
request.theme_id, request.enable_cdg, request.enable_txt
|
||||
)
|
||||
|
||||
assert resolved_cdg is False, "CDG should be disabled by default without theme"
|
||||
assert resolved_txt is False, "TXT should be disabled by default without theme"
|
||||
|
||||
def test_explicit_cdg_txt_overrides_theme_default(self):
|
||||
"""Test that explicit enable_cdg/enable_txt values override theme defaults."""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
|
||||
|
||||
# Theme set (would default to True), but explicitly disabled
|
||||
request = AudioSearchRequest(
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
theme_id="nomad",
|
||||
enable_cdg=False,
|
||||
enable_txt=False,
|
||||
)
|
||||
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
request.theme_id, request.enable_cdg, request.enable_txt
|
||||
)
|
||||
|
||||
assert resolved_cdg is False, "Explicit False should override theme default"
|
||||
assert resolved_txt is False, "Explicit False should override theme default"
|
||||
|
||||
def test_explicit_cdg_txt_enables_without_theme(self):
|
||||
"""Test that explicit True enables CDG/TXT even without theme."""
|
||||
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
|
||||
|
||||
# No theme (would default to False), but explicitly enabled
|
||||
request = AudioSearchRequest(
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
enable_cdg=True,
|
||||
enable_txt=True,
|
||||
)
|
||||
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
request.theme_id, request.enable_cdg, request.enable_txt
|
||||
)
|
||||
|
||||
assert resolved_cdg is True, "Explicit True should enable CDG without theme"
|
||||
assert resolved_txt is True, "Explicit True should enable TXT without theme"
|
||||
|
||||
def test_job_create_receives_theme_from_audio_search(self):
|
||||
"""Test that JobCreate model can receive theme_id and color_overrides."""
|
||||
job_create = JobCreate(
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
theme_id="nomad",
|
||||
color_overrides={
|
||||
"artist_color": "#ff0000",
|
||||
"sung_lyrics_color": "#00ff00"
|
||||
},
|
||||
enable_cdg=True,
|
||||
enable_txt=True,
|
||||
)
|
||||
|
||||
assert job_create.theme_id == "nomad"
|
||||
assert job_create.color_overrides["artist_color"] == "#ff0000"
|
||||
assert job_create.color_overrides["sung_lyrics_color"] == "#00ff00"
|
||||
assert job_create.enable_cdg is True
|
||||
assert job_create.enable_txt is True
|
||||
|
||||
def test_job_model_stores_theme_configuration(self):
|
||||
"""Test Job model stores theme configuration correctly."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.utcnow(),
|
||||
updated_at=datetime.utcnow(),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
theme_id="nomad",
|
||||
color_overrides={
|
||||
"artist_color": "#ff0000"
|
||||
},
|
||||
enable_cdg=True,
|
||||
enable_txt=True,
|
||||
)
|
||||
|
||||
assert job.theme_id == "nomad"
|
||||
assert job.color_overrides["artist_color"] == "#ff0000"
|
||||
assert job.enable_cdg is True
|
||||
assert job.enable_txt is True
|
||||
|
||||
|
||||
class TestAudioSearchThemePreparation:
|
||||
"""Test that audio search endpoint properly prepares theme styles.
|
||||
|
||||
CRITICAL: This test class was added to catch the bug where audio_search.py
|
||||
accepted theme_id but never called prepare_job_style(), resulting in jobs
|
||||
with theme_id set but no style_params_gcs_path or style_assets.
|
||||
|
||||
The symptom: Preview videos show black background instead of themed background.
|
||||
Job ID ffb0b8fa in production demonstrated this issue.
|
||||
"""
|
||||
|
||||
def test_audio_search_with_theme_calls_prepare_job_style(self):
|
||||
"""Test that creating a job via audio search with theme_id prepares the style.
|
||||
|
||||
CRITICAL: This test verifies that when a job is created via the audio-search
|
||||
endpoint with a theme_id (and no custom style files), the code calls
|
||||
prepare_job_style() to set:
|
||||
1. style_params_gcs_path (pointing to the copied style_params.json)
|
||||
2. style_assets (populated with asset mappings)
|
||||
|
||||
Without this, LyricsTranscriber won't have access to the theme's styles
|
||||
and preview videos will have black backgrounds instead of themed ones.
|
||||
"""
|
||||
# Import the endpoint module to access internal functions
|
||||
from backend.api.routes import audio_search as audio_search_module
|
||||
|
||||
# Check if there's an import of prepare_job_style from theme_service
|
||||
source_code_path = audio_search_module.__file__
|
||||
with open(source_code_path, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
# Check for either:
|
||||
# 1. Import of _prepare_theme_for_job from file_upload
|
||||
# 2. Import of prepare_job_style from theme_service
|
||||
# 3. Inline call to theme_service.prepare_job_style
|
||||
has_theme_prep_import = (
|
||||
'_prepare_theme_for_job' in source_code or
|
||||
'prepare_job_style' in source_code
|
||||
)
|
||||
|
||||
assert has_theme_prep_import, (
|
||||
"audio_search.py does not import or call prepare_job_style() or _prepare_theme_for_job(). "
|
||||
"When theme_id is provided without custom style files, the endpoint MUST call "
|
||||
"prepare_job_style() to copy the theme's style_params.json to the job folder. "
|
||||
"Without this, LyricsTranscriber won't have style configuration and preview "
|
||||
"videos will have black backgrounds instead of themed ones."
|
||||
)
|
||||
|
||||
def test_audio_search_code_handles_theme_preparation(self):
|
||||
"""Verify audio_search.py has theme preparation logic similar to file_upload.py.
|
||||
|
||||
This test compares audio_search.py with file_upload.py to ensure they have
|
||||
similar theme preparation patterns. file_upload.py correctly prepares themes,
|
||||
audio_search.py should do the same.
|
||||
"""
|
||||
from backend.api.routes import audio_search as audio_search_module
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
# Read source code of both modules
|
||||
with open(audio_search_module.__file__, 'r') as f:
|
||||
audio_search_code = f.read()
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
file_upload_code = f.read()
|
||||
|
||||
# file_upload.py has this pattern for theme preparation:
|
||||
# if body.theme_id and not has_style_params_upload:
|
||||
# style_params_path, style_assets, youtube_desc = _prepare_theme_for_job(...)
|
||||
assert '_prepare_theme_for_job' in file_upload_code, \
|
||||
"file_upload.py should have _prepare_theme_for_job function"
|
||||
|
||||
# audio_search.py should have similar pattern
|
||||
# Look for either the function or a similar conditional check
|
||||
# STRICT CHECK: Look for actual theme preparation logic being CALLED, not just mentioned
|
||||
has_theme_preparation_call = (
|
||||
# Check for import or call of theme preparation function
|
||||
'_prepare_theme_for_job(' in audio_search_code or
|
||||
'prepare_job_style(' in audio_search_code
|
||||
)
|
||||
|
||||
assert has_theme_preparation_call, (
|
||||
"audio_search.py does not CALL theme preparation logic. "
|
||||
"It accepts theme_id but never prepares the theme style like file_upload.py does. "
|
||||
"Add a call to _prepare_theme_for_job() when theme_id is set and no style files are uploaded."
|
||||
)
|
||||
|
||||
|
||||
class TestFlacfetchIntegration:
|
||||
"""
|
||||
Integration tests that verify flacfetch imports work correctly.
|
||||
|
||||
@@ -114,97 +114,9 @@ class TestWorkerServiceBasics:
|
||||
assert 'http' in url
|
||||
|
||||
|
||||
class TestSessionTokenValidation:
|
||||
"""Tests for session token validation in AuthService.
|
||||
|
||||
PR #120: Fixed authentication to recognize session tokens from magic link auth.
|
||||
Previously, validate_token() only checked admin tokens and auth_tokens collection.
|
||||
Session tokens (created via magic link) are stored in the sessions collection,
|
||||
so they were incorrectly rejected as "Invalid token".
|
||||
"""
|
||||
|
||||
def test_validate_token_checks_session_on_fallback(self):
|
||||
"""Test that validate_token falls back to checking session tokens."""
|
||||
mock_firestore = MagicMock()
|
||||
mock_firestore.get_token.return_value = None # Token not in auth_tokens
|
||||
|
||||
mock_user = MagicMock()
|
||||
mock_user.email = "test@example.com"
|
||||
mock_user.credits = 5
|
||||
|
||||
mock_user_service = MagicMock()
|
||||
mock_user_service.validate_session.return_value = (True, mock_user, "Valid session")
|
||||
|
||||
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
|
||||
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
|
||||
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
|
||||
# Clear the cached instance to get fresh state
|
||||
import backend.services.auth_service
|
||||
backend.services.auth_service._auth_service = None
|
||||
from backend.services.auth_service import AuthService, UserType
|
||||
|
||||
service = AuthService()
|
||||
is_valid, user_type, remaining, msg = service.validate_token("session-token-xyz")
|
||||
|
||||
assert is_valid is True
|
||||
assert user_type == UserType.STRIPE
|
||||
assert remaining == 5 # User's credits
|
||||
mock_user_service.validate_session.assert_called_once_with("session-token-xyz")
|
||||
|
||||
def test_validate_token_returns_invalid_when_session_invalid(self):
|
||||
"""Test that validate_token returns invalid when session validation fails."""
|
||||
mock_firestore = MagicMock()
|
||||
mock_firestore.get_token.return_value = None # Token not in auth_tokens
|
||||
|
||||
mock_user_service = MagicMock()
|
||||
mock_user_service.validate_session.return_value = (False, None, "Invalid session")
|
||||
|
||||
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
|
||||
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
|
||||
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
|
||||
import backend.services.auth_service
|
||||
backend.services.auth_service._auth_service = None
|
||||
from backend.services.auth_service import AuthService, UserType
|
||||
|
||||
service = AuthService()
|
||||
is_valid, user_type, remaining, msg = service.validate_token("invalid-session")
|
||||
|
||||
assert is_valid is False
|
||||
assert remaining == 0
|
||||
assert "Invalid token" in msg
|
||||
|
||||
def test_validate_token_returns_zero_credits_when_user_has_none(self):
|
||||
"""Test that validate_token handles users with 0 credits correctly."""
|
||||
mock_firestore = MagicMock()
|
||||
mock_firestore.get_token.return_value = None
|
||||
|
||||
mock_user = MagicMock()
|
||||
mock_user.email = "test@example.com"
|
||||
mock_user.credits = 0
|
||||
|
||||
mock_user_service = MagicMock()
|
||||
mock_user_service.validate_session.return_value = (True, mock_user, "Valid session")
|
||||
|
||||
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
|
||||
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
|
||||
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
|
||||
import backend.services.auth_service
|
||||
backend.services.auth_service._auth_service = None
|
||||
from backend.services.auth_service import AuthService, UserType
|
||||
|
||||
service = AuthService()
|
||||
is_valid, user_type, remaining, msg = service.validate_token("session-token")
|
||||
|
||||
# User is authenticated but has no credits
|
||||
assert is_valid is True
|
||||
assert user_type == UserType.STRIPE
|
||||
assert remaining == 0
|
||||
assert "no credits" in msg.lower()
|
||||
|
||||
|
||||
class TestJobManagerBasics:
|
||||
"""Basic tests for JobManager."""
|
||||
|
||||
|
||||
def test_job_manager_can_import(self):
|
||||
"""Test JobManager can be imported."""
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
@@ -321,95 +321,31 @@ class TestUploadFile:
|
||||
|
||||
class TestUploadFolder:
|
||||
"""Test upload_folder method."""
|
||||
|
||||
|
||||
@patch("backend.services.dropbox_service.secretmanager.SecretManagerServiceClient")
|
||||
def test_upload_folder(self, mock_sm_client_class, tmp_path):
|
||||
"""Test uploading a folder with multiple files."""
|
||||
from backend.services.dropbox_service import DropboxService
|
||||
|
||||
|
||||
mock_sm_client = Mock()
|
||||
mock_response = Mock()
|
||||
mock_response.payload.data = json.dumps({"access_token": "token"}).encode()
|
||||
mock_sm_client.access_secret_version.return_value = mock_response
|
||||
mock_sm_client_class.return_value = mock_sm_client
|
||||
|
||||
# Create test folder with files (no subdirs for this test)
|
||||
|
||||
# Create test folder with files
|
||||
(tmp_path / "file1.txt").write_text("content1")
|
||||
(tmp_path / "file2.txt").write_text("content2")
|
||||
|
||||
(tmp_path / "subdir").mkdir() # Should be skipped
|
||||
|
||||
service = DropboxService()
|
||||
|
||||
|
||||
with patch.object(service, "upload_file") as mock_upload:
|
||||
service.upload_folder(str(tmp_path), "/Uploads/folder")
|
||||
|
||||
# Should upload 2 files
|
||||
|
||||
# Should upload 2 files (not subdir)
|
||||
assert mock_upload.call_count == 2
|
||||
|
||||
@patch("backend.services.dropbox_service.secretmanager.SecretManagerServiceClient")
|
||||
def test_upload_folder_recursive(self, mock_sm_client_class, tmp_path):
|
||||
"""Test uploading a folder recursively includes subdirectories."""
|
||||
from backend.services.dropbox_service import DropboxService
|
||||
|
||||
mock_sm_client = Mock()
|
||||
mock_response = Mock()
|
||||
mock_response.payload.data = json.dumps({"access_token": "token"}).encode()
|
||||
mock_sm_client.access_secret_version.return_value = mock_response
|
||||
mock_sm_client_class.return_value = mock_sm_client
|
||||
|
||||
# Create test folder with files and subdirectories
|
||||
(tmp_path / "root_file.txt").write_text("root content")
|
||||
(tmp_path / "stems").mkdir()
|
||||
(tmp_path / "stems" / "vocals.flac").write_text("vocals")
|
||||
(tmp_path / "stems" / "instrumental.flac").write_text("instrumental")
|
||||
(tmp_path / "lyrics").mkdir()
|
||||
(tmp_path / "lyrics" / "song.lrc").write_text("lyrics")
|
||||
|
||||
service = DropboxService()
|
||||
|
||||
uploaded_files = []
|
||||
def capture_upload(local_path, remote_path):
|
||||
uploaded_files.append(remote_path)
|
||||
|
||||
with patch.object(service, "upload_file", side_effect=capture_upload) as mock_upload:
|
||||
service.upload_folder(str(tmp_path), "/Uploads/folder")
|
||||
|
||||
# Should upload 4 files (1 root + 2 stems + 1 lyrics)
|
||||
assert mock_upload.call_count == 4
|
||||
|
||||
# Check that subdirectory structure is preserved
|
||||
assert "/Uploads/folder/root_file.txt" in uploaded_files
|
||||
assert "/Uploads/folder/stems/vocals.flac" in uploaded_files
|
||||
assert "/Uploads/folder/stems/instrumental.flac" in uploaded_files
|
||||
assert "/Uploads/folder/lyrics/song.lrc" in uploaded_files
|
||||
|
||||
@patch("backend.services.dropbox_service.secretmanager.SecretManagerServiceClient")
|
||||
def test_upload_folder_deeply_nested(self, mock_sm_client_class, tmp_path):
|
||||
"""Test uploading deeply nested folder structure."""
|
||||
from backend.services.dropbox_service import DropboxService
|
||||
|
||||
mock_sm_client = Mock()
|
||||
mock_response = Mock()
|
||||
mock_response.payload.data = json.dumps({"access_token": "token"}).encode()
|
||||
mock_sm_client.access_secret_version.return_value = mock_response
|
||||
mock_sm_client_class.return_value = mock_sm_client
|
||||
|
||||
# Create deeply nested structure
|
||||
(tmp_path / "level1").mkdir()
|
||||
(tmp_path / "level1" / "level2").mkdir()
|
||||
(tmp_path / "level1" / "level2" / "deep_file.txt").write_text("deep")
|
||||
|
||||
service = DropboxService()
|
||||
|
||||
uploaded_files = []
|
||||
def capture_upload(local_path, remote_path):
|
||||
uploaded_files.append(remote_path)
|
||||
|
||||
with patch.object(service, "upload_file", side_effect=capture_upload):
|
||||
service.upload_folder(str(tmp_path), "/Uploads/folder")
|
||||
|
||||
# Check deeply nested file is uploaded with correct path
|
||||
assert "/Uploads/folder/level1/level2/deep_file.txt" in uploaded_files
|
||||
|
||||
|
||||
class TestCreateSharedLink:
|
||||
"""Test create_shared_link method."""
|
||||
|
||||
@@ -1606,133 +1606,5 @@ class TestIsUrlFunction:
|
||||
assert is_url("") is False
|
||||
|
||||
|
||||
class TestUploadEndpointThemeSupport:
|
||||
"""Test that /jobs/upload endpoint supports theme configuration.
|
||||
|
||||
CRITICAL: These tests verify that the /api/jobs/upload endpoint correctly handles
|
||||
theme_id and color_overrides parameters, ensuring preview videos have themed
|
||||
backgrounds instead of black backgrounds.
|
||||
|
||||
This addresses a bug where:
|
||||
- The frontend sends theme_id and color_overrides when uploading files
|
||||
- But the backend /api/jobs/upload endpoint was ignoring these parameters
|
||||
- Result: Jobs created via file upload had black backgrounds instead of themed ones
|
||||
"""
|
||||
|
||||
def test_upload_endpoint_accepts_theme_id_parameter(self):
|
||||
"""Verify the upload endpoint has theme_id as a form parameter.
|
||||
|
||||
CRITICAL: The frontend sends theme_id when uploading files with a theme.
|
||||
If this parameter is missing from the endpoint, the theme is silently ignored
|
||||
and preview videos will have black backgrounds instead of themed ones.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_theme_id_param = 'theme_id: Optional[str] = Form(' in source_code
|
||||
|
||||
assert has_theme_id_param, (
|
||||
"file_upload.py /jobs/upload endpoint does not have theme_id as a Form parameter. "
|
||||
"The frontend sends theme_id when uploading files with a theme, but the backend "
|
||||
"ignores it. Add: theme_id: Optional[str] = Form(None, description='Theme ID...')"
|
||||
)
|
||||
|
||||
def test_upload_endpoint_accepts_color_overrides_parameter(self):
|
||||
"""Verify the upload endpoint has color_overrides as a form parameter."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_color_overrides_param = 'color_overrides: Optional[str] = Form(' in source_code
|
||||
|
||||
assert has_color_overrides_param, (
|
||||
"file_upload.py /jobs/upload endpoint does not have color_overrides as a Form parameter. "
|
||||
"The frontend sends color_overrides when customizing theme colors."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_calls_prepare_theme_for_job(self):
|
||||
"""Verify the upload endpoint calls _prepare_theme_for_job when theme_id is set.
|
||||
|
||||
CRITICAL: When a job is created via the upload endpoint with a theme_id
|
||||
(and no custom style files), the code must call _prepare_theme_for_job() to set:
|
||||
1. style_params_gcs_path (pointing to the copied style_params.json)
|
||||
2. style_assets (populated with asset mappings)
|
||||
|
||||
Without this, LyricsTranscriber won't have access to the theme's styles
|
||||
and preview videos will have black backgrounds instead of themed ones.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
# The function should be called in upload_and_create_job
|
||||
has_theme_prep_call = '_prepare_theme_for_job(' in source_code
|
||||
|
||||
assert has_theme_prep_call, (
|
||||
"file_upload.py does not call _prepare_theme_for_job(). "
|
||||
"When theme_id is provided to /jobs/upload without custom style files, "
|
||||
"the endpoint MUST call _prepare_theme_for_job() to copy the theme's "
|
||||
"style_params.json to the job folder."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_uses_resolve_cdg_txt_defaults(self):
|
||||
"""Verify the upload endpoint uses _resolve_cdg_txt_defaults for theme-based defaults."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_resolve_call = '_resolve_cdg_txt_defaults(' in source_code
|
||||
|
||||
assert has_resolve_call, (
|
||||
"file_upload.py does not call _resolve_cdg_txt_defaults(). "
|
||||
"When theme_id is set, enable_cdg and enable_txt should default to True."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_has_optional_cdg_txt_params(self):
|
||||
"""Verify enable_cdg and enable_txt are Optional[bool] to support theme defaults.
|
||||
|
||||
CRITICAL: If enable_cdg/enable_txt are bool instead of Optional[bool],
|
||||
they will default to False and override the theme-based defaults.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_optional_cdg = 'enable_cdg: Optional[bool] = Form(' in source_code
|
||||
has_optional_txt = 'enable_txt: Optional[bool] = Form(' in source_code
|
||||
|
||||
assert has_optional_cdg, (
|
||||
"file_upload.py has enable_cdg as bool instead of Optional[bool]. "
|
||||
"This prevents theme-based defaults from working."
|
||||
)
|
||||
assert has_optional_txt, (
|
||||
"file_upload.py has enable_txt as bool instead of Optional[bool]. "
|
||||
"This prevents theme-based defaults from working."
|
||||
)
|
||||
|
||||
def test_job_create_includes_theme_id_and_color_overrides(self):
|
||||
"""Verify JobCreate is called with theme_id and color_overrides."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_theme_id_in_job_create = 'theme_id=theme_id,' in source_code
|
||||
has_color_overrides_in_job_create = 'color_overrides=parsed_color_overrides' in source_code
|
||||
|
||||
assert has_theme_id_in_job_create, (
|
||||
"file_upload.py does not pass theme_id to JobCreate."
|
||||
)
|
||||
assert has_color_overrides_in_job_create, (
|
||||
"file_upload.py does not pass color_overrides to JobCreate."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -54,16 +54,10 @@ def client(mock_job_manager, mock_worker_service):
|
||||
"""Create TestClient with mocked dependencies."""
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
# Create a JobManager class that returns our mock instance
|
||||
def mock_job_manager_factory(*args, **kwargs):
|
||||
return mock_job_manager
|
||||
|
||||
|
||||
# Patch at the module level where jobs.py imports them
|
||||
# Also patch JobManager class used in dependencies.py for auth checks
|
||||
with patch('backend.api.routes.jobs.job_manager', mock_job_manager), \
|
||||
patch('backend.api.routes.jobs.worker_service', mock_worker_service), \
|
||||
patch('backend.services.job_manager.JobManager', mock_job_manager_factory), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
@@ -93,13 +87,9 @@ class TestGetJob:
|
||||
mock_job_manager.get_job.return_value = None
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
def mock_job_manager_factory(*args, **kwargs):
|
||||
return mock_job_manager
|
||||
|
||||
|
||||
with patch('backend.api.routes.jobs.job_manager', mock_job_manager), \
|
||||
patch('backend.api.routes.jobs.worker_service', mock_worker_service), \
|
||||
patch('backend.services.job_manager.JobManager', mock_job_manager_factory), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
@@ -183,17 +173,12 @@ class TestDeleteJob:
|
||||
def test_delete_nonexistent_job(self, mock_worker_service, auth_headers):
|
||||
"""Test deleting non-existent job."""
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = None # Job doesn't exist
|
||||
mock_job_manager.delete_job.return_value = False
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
def mock_job_manager_factory(*args, **kwargs):
|
||||
return mock_job_manager
|
||||
|
||||
|
||||
with patch('backend.api.routes.jobs.job_manager', mock_job_manager), \
|
||||
patch('backend.api.routes.jobs.worker_service', mock_worker_service), \
|
||||
patch('backend.services.job_manager.JobManager', mock_job_manager_factory), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
|
||||
@@ -970,147 +970,10 @@ class TestBackingVocalsAnalysis:
|
||||
def test_analysis_service_can_be_instantiated(self):
|
||||
"""Test that AudioAnalysisService can be instantiated."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
|
||||
mock_storage = MagicMock()
|
||||
service = AudioAnalysisService(storage_service=mock_storage)
|
||||
|
||||
|
||||
assert service is not None
|
||||
assert service.storage_service == mock_storage
|
||||
|
||||
|
||||
class TestModelNamesStorage:
|
||||
"""Tests for model names storage in audio_worker.
|
||||
|
||||
These tests verify that model names are stored in job state_data
|
||||
for use by video_worker in distribution directory preparation.
|
||||
"""
|
||||
|
||||
def test_effective_model_names_defaults(self):
|
||||
"""Test that default model names are used when not specified on job."""
|
||||
from backend.workers.audio_worker import (
|
||||
DEFAULT_CLEAN_MODEL,
|
||||
DEFAULT_BACKING_MODELS,
|
||||
DEFAULT_OTHER_MODELS,
|
||||
)
|
||||
|
||||
# Simulate the logic from process_audio_separation
|
||||
job_clean_model = None
|
||||
job_backing_models = None
|
||||
job_other_models = None
|
||||
|
||||
effective_model_names = {
|
||||
'clean_instrumental_model': job_clean_model or DEFAULT_CLEAN_MODEL,
|
||||
'backing_vocals_models': job_backing_models or DEFAULT_BACKING_MODELS,
|
||||
'other_stems_models': job_other_models or DEFAULT_OTHER_MODELS,
|
||||
}
|
||||
|
||||
assert effective_model_names['clean_instrumental_model'] == DEFAULT_CLEAN_MODEL
|
||||
assert effective_model_names['backing_vocals_models'] == DEFAULT_BACKING_MODELS
|
||||
assert effective_model_names['other_stems_models'] == DEFAULT_OTHER_MODELS
|
||||
|
||||
def test_effective_model_names_custom(self):
|
||||
"""Test that custom model names override defaults."""
|
||||
from backend.workers.audio_worker import (
|
||||
DEFAULT_CLEAN_MODEL,
|
||||
DEFAULT_BACKING_MODELS,
|
||||
DEFAULT_OTHER_MODELS,
|
||||
)
|
||||
|
||||
custom_clean = "custom_clean_model.ckpt"
|
||||
custom_backing = ["custom_backing.ckpt"]
|
||||
custom_other = ["custom_demucs.yaml"]
|
||||
|
||||
# Simulate the logic from process_audio_separation
|
||||
effective_model_names = {
|
||||
'clean_instrumental_model': custom_clean or DEFAULT_CLEAN_MODEL,
|
||||
'backing_vocals_models': custom_backing or DEFAULT_BACKING_MODELS,
|
||||
'other_stems_models': custom_other or DEFAULT_OTHER_MODELS,
|
||||
}
|
||||
|
||||
assert effective_model_names['clean_instrumental_model'] == custom_clean
|
||||
assert effective_model_names['backing_vocals_models'] == custom_backing
|
||||
assert effective_model_names['other_stems_models'] == custom_other
|
||||
|
||||
|
||||
class TestDistributionDirectoryPreparation:
|
||||
"""Tests for _prepare_distribution_directory in video_worker.
|
||||
|
||||
These tests verify that the distribution directory is prepared with:
|
||||
- stems/ subfolder containing all audio stems with model names
|
||||
- lyrics/ subfolder containing intermediate lyrics files
|
||||
- Properly named instrumentals at root level
|
||||
"""
|
||||
|
||||
def test_distribution_directory_creates_stems_folder(self, tmp_path):
|
||||
"""Test that stems directory is created."""
|
||||
stems_dir = tmp_path / "stems"
|
||||
stems_dir.mkdir()
|
||||
|
||||
assert stems_dir.exists()
|
||||
assert stems_dir.is_dir()
|
||||
|
||||
def test_distribution_directory_creates_lyrics_folder(self, tmp_path):
|
||||
"""Test that lyrics directory is created."""
|
||||
lyrics_dir = tmp_path / "lyrics"
|
||||
lyrics_dir.mkdir()
|
||||
|
||||
assert lyrics_dir.exists()
|
||||
assert lyrics_dir.is_dir()
|
||||
|
||||
def test_instrumental_naming_with_model(self):
|
||||
"""Test that instrumental files are named with model names."""
|
||||
base_name = "Artist - Song"
|
||||
clean_model = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
backing_model = "mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"
|
||||
|
||||
clean_instrumental_name = f"{base_name} (Instrumental {clean_model}).flac"
|
||||
backing_instrumental_name = f"{base_name} (Instrumental +BV {backing_model}).flac"
|
||||
|
||||
# Verify expected format
|
||||
assert "model_bs_roformer" in clean_instrumental_name
|
||||
assert "+BV" in backing_instrumental_name
|
||||
assert backing_model in backing_instrumental_name
|
||||
|
||||
def test_stem_naming_convention(self):
|
||||
"""Test that stems are named with proper model suffixes."""
|
||||
base_name = "Artist - Song"
|
||||
clean_model = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
backing_model = "mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"
|
||||
other_model = "htdemucs_6s.yaml"
|
||||
|
||||
# Expected stem filenames
|
||||
expected_stems = {
|
||||
'vocals_clean': f"{base_name} (Vocals {clean_model}).flac",
|
||||
'lead_vocals': f"{base_name} (Lead Vocals {backing_model}).flac",
|
||||
'backing_vocals': f"{base_name} (Backing Vocals {backing_model}).flac",
|
||||
'bass': f"{base_name} (Bass {other_model}).flac",
|
||||
'drums': f"{base_name} (Drums {other_model}).flac",
|
||||
}
|
||||
|
||||
# Verify naming convention
|
||||
for _key, name in expected_stems.items():
|
||||
assert ".flac" in name
|
||||
assert base_name in name
|
||||
|
||||
def test_simplified_instrumental_cleanup(self, tmp_path):
|
||||
"""Test that simplified instrumental names are cleaned up."""
|
||||
base_name = "Artist - Song"
|
||||
|
||||
# Create simplified-named files (as created by _setup_working_directory)
|
||||
simplified_clean = tmp_path / f"{base_name} (Instrumental Clean).flac"
|
||||
simplified_backing = tmp_path / f"{base_name} (Instrumental Backing).flac"
|
||||
simplified_clean.write_text("fake")
|
||||
simplified_backing.write_text("fake")
|
||||
|
||||
# Verify they exist
|
||||
assert simplified_clean.exists()
|
||||
assert simplified_backing.exists()
|
||||
|
||||
# Simulate cleanup
|
||||
simplified_clean.unlink()
|
||||
simplified_backing.unlink()
|
||||
|
||||
# Verify they're removed
|
||||
assert not simplified_clean.exists()
|
||||
assert not simplified_backing.exists()
|
||||
|
||||
@@ -36,12 +36,6 @@ from karaoke_gen.audio_processor import AudioProcessor
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Default model names - used by create_audio_processor and stored in state_data
|
||||
DEFAULT_CLEAN_MODEL = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
DEFAULT_BACKING_MODELS = ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
DEFAULT_OTHER_MODELS = ["htdemucs_6s.yaml"]
|
||||
|
||||
|
||||
# Loggers to capture for audio worker
|
||||
AUDIO_WORKER_LOGGERS = [
|
||||
"karaoke_gen.audio_processor",
|
||||
@@ -187,10 +181,10 @@ def create_audio_processor(
|
||||
audio_logger = logging.getLogger("karaoke_gen.audio_processor")
|
||||
audio_logger.setLevel(logging.INFO)
|
||||
|
||||
# Model configurations - use provided values or defaults from module constants
|
||||
effective_clean_model = clean_instrumental_model or DEFAULT_CLEAN_MODEL
|
||||
effective_backing_models = backing_vocals_models or DEFAULT_BACKING_MODELS
|
||||
effective_other_models = other_stems_models or DEFAULT_OTHER_MODELS # For 6-stem separation (bass, drums, etc.)
|
||||
# Model configurations - use provided values or CLI defaults
|
||||
effective_clean_model = clean_instrumental_model or "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
effective_backing_models = backing_vocals_models or ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
effective_other_models = other_stems_models or ["htdemucs_6s.yaml"] # For 6-stem separation (bass, drums, etc.)
|
||||
|
||||
# FFmpeg command for combining audio files (must be a string, not a list)
|
||||
ffmpeg_base_command = "ffmpeg -hide_banner -loglevel error -nostats -y"
|
||||
@@ -309,17 +303,7 @@ async def process_audio_separation(job_id: str) -> bool:
|
||||
backing_vocals_models=job.backing_vocals_models,
|
||||
other_stems_models=job.other_stems_models
|
||||
)
|
||||
|
||||
# Store effective model names in state_data for video_worker to use in file naming
|
||||
# This ensures output filenames match local CLI behavior (e.g., "Instrumental model_bs_roformer_ep_317_sdr_12.9755.ckpt")
|
||||
effective_model_names = {
|
||||
'clean_instrumental_model': job.clean_instrumental_model or DEFAULT_CLEAN_MODEL,
|
||||
'backing_vocals_models': job.backing_vocals_models or DEFAULT_BACKING_MODELS,
|
||||
'other_stems_models': job.other_stems_models or DEFAULT_OTHER_MODELS,
|
||||
}
|
||||
job_manager.update_state_data(job_id, 'model_names', effective_model_names)
|
||||
job_log.info(f"Stored effective model names: clean={effective_model_names['clean_instrumental_model']}")
|
||||
|
||||
|
||||
# Format artist-title for file naming (matches CLI behavior)
|
||||
artist_title = f"{job.artist} - {job.title}"
|
||||
|
||||
|
||||
@@ -39,30 +39,11 @@ from backend.services.metrics import metrics
|
||||
|
||||
# Import from karaoke_gen package
|
||||
from karaoke_gen.lyrics_processor import LyricsProcessor
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _configure_agentic_ai():
|
||||
"""Configure environment variables for agentic AI correction.
|
||||
|
||||
The lyrics_transcriber library reads these directly from environment.
|
||||
This ensures the settings from backend config are available.
|
||||
"""
|
||||
settings = get_settings()
|
||||
|
||||
# Enable agentic AI if configured
|
||||
if settings.use_agentic_ai:
|
||||
os.environ["USE_AGENTIC_AI"] = "1"
|
||||
os.environ["AGENTIC_AI_MODEL"] = settings.agentic_ai_model
|
||||
logger.info(f"Agentic AI enabled with model: {settings.agentic_ai_model}")
|
||||
else:
|
||||
os.environ["USE_AGENTIC_AI"] = "0"
|
||||
logger.info("Agentic AI disabled")
|
||||
|
||||
|
||||
# Loggers to capture for lyrics worker (top-level package loggers only)
|
||||
# Note: We only capture at top-level package loggers to avoid duplicate log entries.
|
||||
# Logs from sub-modules (e.g., lyrics_transcriber.correction.anchor_sequence) will
|
||||
@@ -138,10 +119,7 @@ async def process_lyrics_transcription(job_id: str) -> bool:
|
||||
start_time = time.time()
|
||||
job_manager = JobManager()
|
||||
storage = StorageService()
|
||||
|
||||
# Configure agentic AI before any lyrics processing
|
||||
_configure_agentic_ai()
|
||||
|
||||
|
||||
# Create job logger for remote debugging FIRST
|
||||
job_log = create_job_logger(job_id, "lyrics")
|
||||
|
||||
|
||||
@@ -268,7 +268,6 @@ async def generate_video(job_id: str) -> bool:
|
||||
job_manager=job_manager,
|
||||
temp_dir=temp_dir,
|
||||
result=result,
|
||||
storage=storage,
|
||||
)
|
||||
|
||||
# Upload generated files to GCS
|
||||
@@ -292,16 +291,11 @@ async def generate_video(job_id: str) -> bool:
|
||||
)
|
||||
|
||||
# Store result metadata in job
|
||||
# NOTE: Must include dropbox_link and gdrive_files from result, since
|
||||
# _handle_native_distribution already saved them but job.state_data
|
||||
# is stale (fetched before distribution ran)
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
'brand_code': result.get('brand_code'),
|
||||
'youtube_url': result.get('youtube_url'),
|
||||
'dropbox_link': result.get('dropbox_link'),
|
||||
'gdrive_files': result.get('gdrive_files'),
|
||||
}
|
||||
})
|
||||
|
||||
@@ -349,17 +343,16 @@ async def _handle_native_distribution(
|
||||
job_manager: JobManager,
|
||||
temp_dir: str,
|
||||
result: Dict[str, Any],
|
||||
storage: StorageService = None,
|
||||
) -> None:
|
||||
"""
|
||||
Handle distribution uploads using native APIs (Dropbox SDK, Google Drive API).
|
||||
|
||||
|
||||
This is used by the remote CLI instead of rclone-based uploads.
|
||||
The native APIs provide:
|
||||
- Better error handling and retry logic
|
||||
- No need for rclone binary in container
|
||||
- Credentials managed via Secret Manager
|
||||
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
job: Job object with dropbox_path and gdrive_folder_id fields
|
||||
@@ -367,7 +360,6 @@ async def _handle_native_distribution(
|
||||
job_manager: Job manager for updating job state
|
||||
temp_dir: Temporary directory with output files
|
||||
result: Result dict from KaraokeFinalise.process()
|
||||
storage: StorageService instance for downloading stems/lyrics
|
||||
"""
|
||||
brand_code = result.get('brand_code')
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
@@ -400,22 +392,10 @@ async def _handle_native_distribution(
|
||||
result['brand_code'] = brand_code
|
||||
job_log.info(f"Generated brand code: {brand_code}")
|
||||
|
||||
# Prepare full distribution directory with stems/ and lyrics/ subfolders
|
||||
# This must be done before upload to ensure complete output structure
|
||||
if storage:
|
||||
await _prepare_distribution_directory(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
storage=storage,
|
||||
temp_dir=temp_dir,
|
||||
base_name=base_name,
|
||||
job_log=job_log,
|
||||
)
|
||||
|
||||
# Create folder name and upload files
|
||||
folder_name = f"{brand_code} - {base_name}"
|
||||
remote_folder = f"{dropbox_path}/{folder_name}"
|
||||
|
||||
|
||||
job_log.info(f"Uploading to Dropbox folder: {remote_folder}")
|
||||
dropbox.upload_folder(temp_dir, remote_folder)
|
||||
|
||||
@@ -621,139 +601,6 @@ async def _setup_working_directory(
|
||||
log_progress("Downloaded end JPG")
|
||||
|
||||
|
||||
async def _prepare_distribution_directory(
|
||||
job_id: str,
|
||||
job,
|
||||
storage: StorageService,
|
||||
temp_dir: str,
|
||||
base_name: str,
|
||||
job_log=None
|
||||
) -> None:
|
||||
"""
|
||||
Prepare the full distribution directory structure matching local CLI output.
|
||||
|
||||
Creates:
|
||||
- stems/ subfolder with all audio stems (proper model names)
|
||||
- lyrics/ subfolder with intermediate lyrics files
|
||||
- Root-level instrumentals with proper model names
|
||||
|
||||
This ensures the Dropbox upload contains the complete output structure.
|
||||
"""
|
||||
def log_progress(message: str):
|
||||
"""Log to both module logger and job logger if available."""
|
||||
logger.info(f"Job {job_id}: {message}")
|
||||
if job_log:
|
||||
job_log.info(message)
|
||||
|
||||
log_progress("Preparing full distribution directory structure")
|
||||
|
||||
# Get model names from state_data (stored by audio_worker)
|
||||
model_names = job.state_data.get('model_names', {})
|
||||
clean_model = model_names.get('clean_instrumental_model', 'model_bs_roformer_ep_317_sdr_12.9755.ckpt')
|
||||
backing_models = model_names.get('backing_vocals_models', ['mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt'])
|
||||
other_models = model_names.get('other_stems_models', ['htdemucs_6s.yaml'])
|
||||
backing_model = backing_models[0] if backing_models else 'mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt'
|
||||
other_model = other_models[0] if other_models else 'htdemucs_6s.yaml'
|
||||
|
||||
# Create stems subdirectory
|
||||
stems_dir = os.path.join(temp_dir, "stems")
|
||||
os.makedirs(stems_dir, exist_ok=True)
|
||||
|
||||
# Create lyrics subdirectory
|
||||
lyrics_dir = os.path.join(temp_dir, "lyrics")
|
||||
os.makedirs(lyrics_dir, exist_ok=True)
|
||||
|
||||
stems = job.file_urls.get('stems', {})
|
||||
lyrics = job.file_urls.get('lyrics', {})
|
||||
|
||||
# --- Download stems to stems/ subfolder with proper model names ---
|
||||
|
||||
# Clean up simplified-named instrumentals from _setup_working_directory
|
||||
# These were needed for KaraokeFinalise but shouldn't go to distribution
|
||||
simplified_instrumental_patterns = [
|
||||
f"{base_name} (Instrumental Clean).flac",
|
||||
f"{base_name} (Instrumental Backing).flac",
|
||||
]
|
||||
for pattern in simplified_instrumental_patterns:
|
||||
simplified_path = os.path.join(temp_dir, pattern)
|
||||
if os.path.exists(simplified_path):
|
||||
os.remove(simplified_path)
|
||||
log_progress(f"Removed simplified instrumental: {pattern}")
|
||||
|
||||
# Clean instrumental (from clean_instrumental_model)
|
||||
if stems.get('instrumental_clean'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Instrumental {clean_model}).flac")
|
||||
storage.download_file(stems['instrumental_clean'], dest_path)
|
||||
log_progress("Downloaded clean instrumental to stems/")
|
||||
|
||||
# Also copy to root level for distribution
|
||||
root_path = os.path.join(temp_dir, f"{base_name} (Instrumental {clean_model}).flac")
|
||||
shutil.copy2(dest_path, root_path)
|
||||
|
||||
# Clean vocals (from clean_instrumental_model)
|
||||
if stems.get('vocals_clean'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Vocals {clean_model}).flac")
|
||||
storage.download_file(stems['vocals_clean'], dest_path)
|
||||
log_progress("Downloaded clean vocals to stems/")
|
||||
|
||||
# Instrumental with backing vocals (from backing_vocals_model)
|
||||
if stems.get('instrumental_with_backing'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Instrumental +BV {backing_model}).flac")
|
||||
storage.download_file(stems['instrumental_with_backing'], dest_path)
|
||||
log_progress("Downloaded instrumental+BV to stems/")
|
||||
|
||||
# Also copy to root level for distribution
|
||||
root_path = os.path.join(temp_dir, f"{base_name} (Instrumental +BV {backing_model}).flac")
|
||||
shutil.copy2(dest_path, root_path)
|
||||
|
||||
# Lead vocals (from backing_vocals_model)
|
||||
if stems.get('lead_vocals'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Lead Vocals {backing_model}).flac")
|
||||
storage.download_file(stems['lead_vocals'], dest_path)
|
||||
log_progress("Downloaded lead vocals to stems/")
|
||||
|
||||
# Backing vocals (from backing_vocals_model)
|
||||
if stems.get('backing_vocals'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Backing Vocals {backing_model}).flac")
|
||||
storage.download_file(stems['backing_vocals'], dest_path)
|
||||
log_progress("Downloaded backing vocals to stems/")
|
||||
|
||||
# Other stems (from other_stems_model - typically htdemucs_6s)
|
||||
other_stem_keys = ['bass', 'drums', 'guitar', 'piano', 'other']
|
||||
for stem_key in other_stem_keys:
|
||||
if stems.get(stem_key):
|
||||
stem_name = stem_key.capitalize()
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} ({stem_name} {other_model}).flac")
|
||||
storage.download_file(stems[stem_key], dest_path)
|
||||
log_progress(f"Downloaded {stem_key} to stems/")
|
||||
|
||||
# --- Download lyrics files to lyrics/ subfolder ---
|
||||
|
||||
# Common lyrics file types
|
||||
lyrics_file_keys = ['lrc', 'ass', 'srt', 'txt', 'json', 'original_json', 'corrected_json']
|
||||
for lyrics_key in lyrics_file_keys:
|
||||
if lyrics.get(lyrics_key):
|
||||
# Determine proper extension
|
||||
ext_map = {
|
||||
'lrc': '.lrc',
|
||||
'ass': '.ass',
|
||||
'srt': '.srt',
|
||||
'txt': '.txt',
|
||||
'json': '.json',
|
||||
'original_json': '_original.json',
|
||||
'corrected_json': '_corrected.json',
|
||||
}
|
||||
ext = ext_map.get(lyrics_key, f'.{lyrics_key}')
|
||||
dest_path = os.path.join(lyrics_dir, f"{base_name} (Karaoke){ext}")
|
||||
try:
|
||||
storage.download_file(lyrics[lyrics_key], dest_path)
|
||||
log_progress(f"Downloaded {lyrics_key} to lyrics/")
|
||||
except Exception as e:
|
||||
log_progress(f"Could not download {lyrics_key}: {e}")
|
||||
|
||||
log_progress("Distribution directory prepared with stems/ and lyrics/ subfolders")
|
||||
|
||||
|
||||
async def _upload_results(
|
||||
job_id: str,
|
||||
job_manager: JobManager,
|
||||
@@ -762,7 +609,7 @@ async def _upload_results(
|
||||
result: Dict[str, Any]
|
||||
) -> None:
|
||||
"""Upload generated files to GCS."""
|
||||
|
||||
|
||||
# Map of result keys to GCS paths
|
||||
file_mappings = [
|
||||
('final_video', 'finals', 'lossless_4k_mp4'),
|
||||
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loaded 100 of 256 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user