mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
Compare commits
138
Commits
No files matched your search
@@ -0,0 +1,109 @@
|
||||
---
|
||||
description: Review and address CodeRabbit comments on current PR
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write, Bash, mcp__github__pull_request_read, mcp__github__get_me, mcp__github__list_pull_requests, mcp__github__add_issue_comment
|
||||
---
|
||||
|
||||
# CodeRabbit Comment Review
|
||||
|
||||
You are reviewing the current PR to address all CodeRabbit automated review comments.
|
||||
|
||||
## Your Task
|
||||
|
||||
### 1. Identify the Current PR
|
||||
|
||||
First, determine the current branch and find the associated PR:
|
||||
```bash
|
||||
git branch --show-current
|
||||
```
|
||||
|
||||
Then use GitHub tools to find the PR for this branch in the `nomadkaraoke/karaoke-gen` repository.
|
||||
|
||||
### 2. Fetch CodeRabbit Comments
|
||||
|
||||
Get all review comments and regular comments on the PR:
|
||||
|
||||
**Review Comments (Potential Issues)**
|
||||
- Use `pull_request_read` with `method: "get_review_comments"` to get all review comment threads
|
||||
- Look for comments from the `coderabbitai` bot user
|
||||
- These are typically marked as "Potential issue" and MUST be resolved before merge
|
||||
|
||||
**PR Comments (Including Nitpicks)**
|
||||
- Use `pull_request_read` with `method: "get_comments"` to get regular PR comments
|
||||
- CodeRabbit posts a summary comment that includes "Nitpick comments" section
|
||||
- These should also be reviewed and considered for action
|
||||
|
||||
### 3. Analyze Each Comment
|
||||
|
||||
For each CodeRabbit comment:
|
||||
1. Read the comment carefully to understand the concern
|
||||
2. Look at the referenced code location
|
||||
3. Decide if a code change is warranted:
|
||||
- **Make changes** for: valid bugs, security issues, clear improvements, style violations
|
||||
- **Skip changes** for: subjective preferences, false positives, already addressed, not applicable
|
||||
4. If skipping, note the reason
|
||||
|
||||
### 4. Make Code Changes
|
||||
|
||||
For each comment requiring action:
|
||||
1. Navigate to the file and line mentioned
|
||||
2. Understand the surrounding context
|
||||
3. Make the minimal change needed to address the concern
|
||||
4. Verify the change doesn't break anything obvious
|
||||
|
||||
### 5. Commit and Push
|
||||
|
||||
After making all changes:
|
||||
```bash
|
||||
# Stage all changes
|
||||
git add -A
|
||||
|
||||
# Commit with descriptive message
|
||||
git commit -m "fix: address CodeRabbit review comments
|
||||
|
||||
- [list each fix briefly]
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
|
||||
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
|
||||
|
||||
# Push to the PR branch
|
||||
git push
|
||||
```
|
||||
|
||||
### 6. Resolve Threads
|
||||
|
||||
After pushing:
|
||||
- Reply to each CodeRabbit review thread indicating what was done
|
||||
- If a comment was addressed, reply with what was changed
|
||||
- If a comment was intentionally skipped, explain why
|
||||
- This helps mark threads as resolved so the PR can be merged
|
||||
|
||||
## Output Format
|
||||
|
||||
Provide a summary report:
|
||||
|
||||
```
|
||||
## CodeRabbit Review Summary
|
||||
|
||||
### Comments Addressed
|
||||
- [ ] File:line - Brief description of issue → What was changed
|
||||
|
||||
### Comments Skipped (with reason)
|
||||
- [ ] File:line - Issue → Reason for skipping
|
||||
|
||||
### Actions Taken
|
||||
- Committed: [commit hash]
|
||||
- Pushed to: [branch name]
|
||||
- Threads replied to: [count]
|
||||
|
||||
### Status
|
||||
Ready for merge: Yes/No (if No, explain why)
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
|
||||
- Focus on code quality improvements, not just making CodeRabbit happy
|
||||
- Don't blindly accept every suggestion - use judgment
|
||||
- Group related fixes into logical commits if needed
|
||||
- Keep changes minimal and focused on the specific issues raised
|
||||
- If a suggestion would require significant refactoring, note it for future work instead
|
||||
@@ -0,0 +1,74 @@
|
||||
---
|
||||
description: Periodic documentation maintenance and organization check
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write, Bash
|
||||
---
|
||||
|
||||
# Documentation Maintenance Check
|
||||
|
||||
You are performing periodic maintenance on the project documentation.
|
||||
|
||||
## Checks to Perform
|
||||
|
||||
### 1. Structure Compliance
|
||||
Verify docs/ follows the correct structure:
|
||||
```
|
||||
docs/
|
||||
├── README.md # Current status + navigation
|
||||
├── ARCHITECTURE.md # System design
|
||||
├── DEVELOPMENT.md # Dev setup, testing, deployment
|
||||
├── API.md # Backend API reference
|
||||
├── LESSONS-LEARNED.md # Accumulated wisdom
|
||||
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
|
||||
```
|
||||
|
||||
Report any files that don't belong or are misplaced.
|
||||
|
||||
### 2. Freshness Check
|
||||
- Read `docs/README.md` - is the status section current?
|
||||
- Check recent git commits - do they suggest docs should be updated?
|
||||
- Look for stale information that contradicts current code
|
||||
|
||||
### 3. Archive Organization
|
||||
- Verify all archive files have YYYY-MM-DD prefix
|
||||
- Check for any docs that should be archived (completed work, old plans)
|
||||
- Remove truly obsolete content (empty files, duplicate info)
|
||||
|
||||
### 4. Cross-Reference Check
|
||||
- Verify CLAUDE.md points to current doc locations
|
||||
- Check that docs reference each other correctly
|
||||
- Ensure no broken internal links
|
||||
|
||||
### 5. Content Quality
|
||||
- Flag any docs over 500 lines (may need splitting)
|
||||
- Identify duplicate information across docs
|
||||
- Note sections that seem outdated
|
||||
|
||||
## Actions to Take
|
||||
|
||||
1. **Fix structural issues** - Move/rename misplaced files
|
||||
2. **Update stale content** - Refresh outdated information
|
||||
3. **Archive completed work** - Move old docs to archive/
|
||||
4. **Report findings** - Summarize what was found and fixed
|
||||
|
||||
## Output Format
|
||||
|
||||
Provide a maintenance report:
|
||||
```
|
||||
## Documentation Maintenance Report - YYYY-MM-DD
|
||||
|
||||
### Issues Found
|
||||
- [list of issues]
|
||||
|
||||
### Actions Taken
|
||||
- [list of fixes made]
|
||||
|
||||
### Recommendations
|
||||
- [suggestions for future attention]
|
||||
|
||||
### Status
|
||||
- [ ] Structure: OK/Fixed
|
||||
- [ ] Freshness: OK/Updated
|
||||
- [ ] Archive: OK/Organized
|
||||
- [ ] Cross-refs: OK/Fixed
|
||||
- [ ] Quality: OK/Flagged
|
||||
```
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
description: Review current session for documentation updates before merging PR
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write
|
||||
---
|
||||
|
||||
# Pre-Merge Documentation Review
|
||||
|
||||
You are reviewing this chat session to identify documentation updates needed before merging the current PR.
|
||||
|
||||
## Your Task
|
||||
|
||||
1. **Analyze the current session** - What work was done? What was learned? What changed?
|
||||
|
||||
2. **Check if docs need updates:**
|
||||
- Does `docs/README.md` status section need updating?
|
||||
- Should anything be added to `docs/LESSONS-LEARNED.md`?
|
||||
- Does `docs/ARCHITECTURE.md` need changes (if architecture changed)?
|
||||
- Does `docs/DEVELOPMENT.md` need changes (if dev workflow changed)?
|
||||
- Does `docs/API.md` need changes (if API changed)?
|
||||
|
||||
3. **For significant completed work:**
|
||||
- Create `docs/archive/YYYY-MM-DD-topic.md` summarizing the work
|
||||
- Use today's date and a descriptive topic name
|
||||
|
||||
4. **Update docs/README.md** with:
|
||||
- Current status if it changed
|
||||
- Any new known issues discovered
|
||||
- Link to new archive doc if created
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **IMPORTANT**: Make all documentation changes on the current feature branch (git worktree), NOT on main. Check `git branch` or the worktree path to ensure you're editing files in the correct location.
|
||||
- Keep updates concise and factual
|
||||
- Focus on information useful for future AI agents
|
||||
- Don't duplicate information that's in the code
|
||||
- Archive docs should capture "what was done and why" not implementation details
|
||||
|
||||
## Output
|
||||
|
||||
After reviewing, either:
|
||||
- Make the necessary documentation updates
|
||||
- Report "No documentation updates needed" with brief explanation why
|
||||
|
||||
Remember: Good documentation helps the next agent continue where you left off.
|
||||
@@ -0,0 +1,83 @@
|
||||
# .dockerignore - Reduce Docker build context size
|
||||
# This is critical for fast builds as it reduces upload time and cache invalidation
|
||||
|
||||
# Git
|
||||
.git/
|
||||
.gitignore
|
||||
.gitmodules
|
||||
|
||||
# Node.js / Frontend (not needed for backend image)
|
||||
node_modules/
|
||||
frontend/
|
||||
frontend-react/
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
|
||||
# Python virtual environments (use Docker's Python instead)
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
*.so
|
||||
.Python
|
||||
.venv/
|
||||
venv/
|
||||
backend/venv/
|
||||
ENV/
|
||||
env/
|
||||
*.egg-info/
|
||||
.eggs/
|
||||
dist/
|
||||
build/
|
||||
*.egg
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
|
||||
# Testing (not needed in production image)
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
coverage.xml
|
||||
*.cover
|
||||
tests/
|
||||
backend/tests/
|
||||
|
||||
# Documentation (not needed in Docker image)
|
||||
docs/
|
||||
*.md
|
||||
!README.md
|
||||
|
||||
# Infrastructure (not needed for Docker build)
|
||||
infrastructure/
|
||||
|
||||
# CI/CD configs
|
||||
.github/
|
||||
|
||||
# Cloud Build configs (keep for reference but not in image)
|
||||
cloudbuild*.yaml
|
||||
|
||||
# Local development files
|
||||
lyrics_transcriber_local/
|
||||
*.local
|
||||
.env
|
||||
.env.*
|
||||
*.bak
|
||||
*.tmp
|
||||
Thumbs.db
|
||||
|
||||
# Large test/sample files
|
||||
tests/data/
|
||||
*.mp3
|
||||
*.wav
|
||||
*.mp4
|
||||
*.mkv
|
||||
*.flac
|
||||
+219
-198
@@ -18,11 +18,13 @@ concurrency:
|
||||
jobs:
|
||||
# ============================================
|
||||
# PATH FILTERING - Determine what changed
|
||||
# Only needed for PRs (main branch skips tests)
|
||||
# ============================================
|
||||
|
||||
changes:
|
||||
name: "Detect Changes"
|
||||
runs-on: ubuntu-latest
|
||||
if: github.ref != 'refs/heads/main'
|
||||
outputs:
|
||||
frontend: ${{ steps.filter.outputs.frontend }}
|
||||
backend: ${{ steps.filter.outputs.backend }}
|
||||
@@ -61,14 +63,14 @@ jobs:
|
||||
echo "| Dependencies | ${{ steps.filter.outputs.deps }} |" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
# ============================================
|
||||
# FRONTEND TESTS
|
||||
# FRONTEND TESTS (PRs only - main skips tests)
|
||||
# ============================================
|
||||
|
||||
frontend-unit-tests:
|
||||
name: "Frontend - Unit Tests"
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
if: ${{ needs.changes.outputs.frontend == 'true' || github.ref == 'refs/heads/main' }}
|
||||
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -95,15 +97,40 @@ jobs:
|
||||
path: frontend/coverage/
|
||||
retention-days: 7
|
||||
|
||||
frontend-build:
|
||||
name: "Frontend - Build"
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Build frontend
|
||||
working-directory: frontend
|
||||
env:
|
||||
NEXT_PUBLIC_API_URL: https://api.nomadkaraoke.com
|
||||
run: npm run build
|
||||
|
||||
# ============================================
|
||||
# PACKAGE TESTS (karaoke-gen Python CLI)
|
||||
# PACKAGE TESTS (PRs only - main skips tests)
|
||||
# ============================================
|
||||
|
||||
package-unit-tests:
|
||||
name: "Package - Unit Tests (70% coverage)"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: ${{ needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -159,9 +186,9 @@ jobs:
|
||||
|
||||
package-integration-tests:
|
||||
name: "Package - Integration Tests"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: ${{ needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -206,11 +233,15 @@ jobs:
|
||||
- name: Run integration tests
|
||||
run: poetry run pytest tests/integration/ -v
|
||||
|
||||
package-build:
|
||||
name: "Package - Build & Install"
|
||||
runs-on: ubuntu-latest
|
||||
package-build-test:
|
||||
name: "Package - Build & Install (Test)"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: [changes, package-unit-tests, package-integration-tests]
|
||||
if: ${{ always() && (needs.package-unit-tests.result == 'success' || needs.package-unit-tests.result == 'skipped') && (needs.package-integration-tests.result == 'success' || needs.package-integration-tests.result == 'skipped') && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main') }}
|
||||
if: >-
|
||||
github.ref != 'refs/heads/main' && always() &&
|
||||
(needs.package-unit-tests.result == 'success' || needs.package-unit-tests.result == 'skipped') &&
|
||||
(needs.package-integration-tests.result == 'success' || needs.package-integration-tests.result == 'skipped') &&
|
||||
(needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -227,49 +258,23 @@ jobs:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-package-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
|
||||
- name: Free disk space
|
||||
uses: jlumbroso/free-disk-space@main
|
||||
with:
|
||||
tool-cache: false
|
||||
android: true
|
||||
haskell: true
|
||||
dotnet: false
|
||||
large-packages: false
|
||||
docker-images: false
|
||||
swap-storage: false
|
||||
|
||||
- name: Test package installation
|
||||
run: |
|
||||
pip install dist/*.whl
|
||||
python -m karaoke_gen.utils.gen_cli --help
|
||||
|
||||
- name: Upload build artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: package-dist
|
||||
path: dist/
|
||||
retention-days: 1
|
||||
|
||||
# ============================================
|
||||
# BACKEND TESTS (Cloud Run API service)
|
||||
# BACKEND TESTS (PRs only - main skips tests)
|
||||
# ============================================
|
||||
|
||||
backend-unit-tests:
|
||||
name: "Backend - Unit Tests"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
@@ -287,14 +292,6 @@ jobs:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-backend-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-backend-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
@@ -312,17 +309,10 @@ jobs:
|
||||
|
||||
backend-emulator-tests:
|
||||
name: "Backend - Emulator Integration Tests"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /opt/ghc
|
||||
sudo rm -rf /usr/local/share/boost
|
||||
df -h
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -339,28 +329,12 @@ jobs:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-backend-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-backend-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Set up Java 21 (required by Firestore emulator)
|
||||
uses: actions/setup-java@v4
|
||||
with:
|
||||
distribution: "temurin"
|
||||
java-version: "21"
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Install Firestore emulator
|
||||
run: gcloud components install beta cloud-firestore-emulator --quiet
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
@@ -382,9 +356,9 @@ jobs:
|
||||
|
||||
backend-lint:
|
||||
name: "Backend - Code Quality"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || github.ref == 'refs/heads/main' }}
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true')
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
@@ -419,9 +393,78 @@ jobs:
|
||||
echo "✅ Basic syntax validation passed"
|
||||
|
||||
# ============================================
|
||||
# DEPLOYMENTS (main branch only)
|
||||
# CI GATE - Required status check for PRs
|
||||
# Only runs on PRs, not on main
|
||||
# ============================================
|
||||
|
||||
ci-gate:
|
||||
name: "CI Gate"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [frontend-unit-tests, frontend-build, package-unit-tests, package-integration-tests, backend-unit-tests, backend-emulator-tests, backend-lint]
|
||||
if: github.ref != 'refs/heads/main' && always()
|
||||
steps:
|
||||
- name: Check results
|
||||
run: |
|
||||
results=(
|
||||
"${{ needs.frontend-unit-tests.result }}"
|
||||
"${{ needs.frontend-build.result }}"
|
||||
"${{ needs.package-unit-tests.result }}"
|
||||
"${{ needs.package-integration-tests.result }}"
|
||||
"${{ needs.backend-unit-tests.result }}"
|
||||
"${{ needs.backend-emulator-tests.result }}"
|
||||
"${{ needs.backend-lint.result }}"
|
||||
)
|
||||
for result in "${results[@]}"; do
|
||||
if [[ "$result" == "failure" || "$result" == "cancelled" ]]; then
|
||||
echo "❌ CI failed"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
echo "✅ CI passed"
|
||||
|
||||
# ============================================
|
||||
# DEPLOYMENTS (main branch only)
|
||||
# Tests already passed on PR, so we skip them here
|
||||
# ============================================
|
||||
|
||||
package-build:
|
||||
name: "Package - Build"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Cache Poetry dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: .venv
|
||||
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
|
||||
restore-keys: |
|
||||
venv-package-${{ runner.os }}-py3.13-
|
||||
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
|
||||
- name: Upload build artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: package-dist
|
||||
path: dist/
|
||||
retention-days: 1
|
||||
|
||||
package-publish:
|
||||
name: "Deploy - Publish to PyPI"
|
||||
runs-on: ubuntu-latest
|
||||
@@ -512,11 +555,7 @@ jobs:
|
||||
deploy-frontend:
|
||||
name: "Deploy - Frontend (GitHub Pages)"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [frontend-unit-tests]
|
||||
if: ${{ always() && github.ref == 'refs/heads/main' && github.event_name == 'push' && (needs.frontend-unit-tests.result == 'success' || needs.frontend-unit-tests.result == 'skipped') }}
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
@@ -587,20 +626,24 @@ jobs:
|
||||
|
||||
deploy-backend:
|
||||
name: "Deploy - Backend (Cloud Run)"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [backend-unit-tests, backend-emulator-tests, backend-lint]
|
||||
if: ${{ always() && github.ref == 'refs/heads/main' && github.event_name == 'push' && (needs.backend-unit-tests.result == 'success' || needs.backend-unit-tests.result == 'skipped') && (needs.backend-emulator-tests.result == 'success' || needs.backend-emulator-tests.result == 'skipped') && (needs.backend-lint.result == 'success' || needs.backend-lint.result == 'skipped') }}
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
env:
|
||||
REGISTRY: us-central1-docker.pkg.dev
|
||||
PROJECT_ID: nomadkaraoke
|
||||
REPOSITORY: karaoke-repo
|
||||
BASE_IMAGE_NAME: karaoke-backend-base
|
||||
APP_IMAGE_NAME: karaoke-backend
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
fetch-depth: 0 # Need full history for tag check
|
||||
|
||||
- name: Set up Python (for version extraction)
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
@@ -608,24 +651,15 @@ jobs:
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
|
||||
- name: Extract version and check if deployment needed
|
||||
- name: Extract version
|
||||
id: version_check
|
||||
run: |
|
||||
VERSION=$(poetry version --short)
|
||||
echo "version=$VERSION" >> $GITHUB_OUTPUT
|
||||
echo "📋 Current version: $VERSION"
|
||||
|
||||
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
|
||||
echo "needs_deploy=false" >> $GITHUB_OUTPUT
|
||||
echo "⏭️ Tag v$VERSION already exists - skipping backend deployment"
|
||||
echo " (Bump version in pyproject.toml to trigger a new deployment)"
|
||||
else
|
||||
echo "needs_deploy=true" >> $GITHUB_OUTPUT
|
||||
echo "✨ Tag v$VERSION does not exist - proceeding with deployment"
|
||||
fi
|
||||
echo "🚀 Deploying on every push to main"
|
||||
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
id: auth
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
@@ -633,26 +667,37 @@ jobs:
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Check if base image needs rebuild
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
id: check-base
|
||||
run: |
|
||||
echo "🔍 Checking if base image needs rebuild..."
|
||||
- name: Configure Docker for Artifact Registry
|
||||
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
|
||||
|
||||
# ============================================
|
||||
# Docker Build with BuildKit + Registry Cache
|
||||
# This replaces expensive Cloud Build with free GitHub Actions
|
||||
# Registry cache persists between builds for fast rebuilds
|
||||
# ============================================
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Calculate base image hash
|
||||
id: base-hash
|
||||
run: |
|
||||
# Hash both poetry.lock AND Dockerfile.base - rebuild if either changes
|
||||
CURRENT_HASH=$(cat poetry.lock backend/Dockerfile.base | sha256sum | cut -d' ' -f1)
|
||||
echo "Current combined hash (poetry.lock + Dockerfile.base): $CURRENT_HASH"
|
||||
echo "hash=$CURRENT_HASH" >> $GITHUB_OUTPUT
|
||||
echo "📋 Base content hash: $CURRENT_HASH"
|
||||
|
||||
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
|
||||
- name: Check if base image exists and is current
|
||||
id: check-base
|
||||
run: |
|
||||
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
|
||||
|
||||
BASE_IMAGE="us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest"
|
||||
# Try to get the existing base image's hash label
|
||||
EXISTING_HASH=""
|
||||
|
||||
if docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
@@ -661,10 +706,10 @@ jobs:
|
||||
fi
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
else
|
||||
echo "⚠️ No existing base image found"
|
||||
echo "⚠️ No existing base image found - will build"
|
||||
fi
|
||||
|
||||
if [ "$CURRENT_HASH" != "$EXISTING_HASH" ]; then
|
||||
if [ "${{ steps.base-hash.outputs.hash }}" != "$EXISTING_HASH" ]; then
|
||||
echo "🔄 Base image needs rebuild (poetry.lock or Dockerfile.base changed)"
|
||||
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
@@ -672,91 +717,73 @@ jobs:
|
||||
echo "needs_rebuild=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Rebuild base image (if needed)
|
||||
if: steps.version_check.outputs.needs_deploy == 'true' && steps.check-base.outputs.needs_rebuild == 'true'
|
||||
- name: Build and push base image (if needed)
|
||||
if: steps.check-base.outputs.needs_rebuild == 'true'
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: backend/Dockerfile.base
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:${{ github.sha }}
|
||||
# Registry cache for ALL layers (mode=max) - persists between builds
|
||||
# ignore-error=true allows build to continue if cache doesn't exist yet
|
||||
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:cache,ignore-error=true
|
||||
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:cache,mode=max,ignore-error=true
|
||||
build-args: |
|
||||
BUILD_DATE=${{ github.run_id }}
|
||||
BUILD_ID=${{ github.sha }}
|
||||
BASE_CONTENT_HASH=${{ steps.base-hash.outputs.hash }}
|
||||
# Provenance disabled for smaller image and faster push
|
||||
provenance: false
|
||||
|
||||
- name: Build and push app image
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: backend/Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:latest
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:${{ github.sha }}
|
||||
${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:v${{ steps.version_check.outputs.version }}
|
||||
# Registry cache for app layers
|
||||
# ignore-error=true allows build to continue if cache doesn't exist yet
|
||||
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:cache,ignore-error=true
|
||||
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:cache,mode=max,ignore-error=true
|
||||
provenance: false
|
||||
|
||||
# ============================================
|
||||
# Deploy to Cloud Run
|
||||
# ============================================
|
||||
|
||||
- name: Deploy to Cloud Run
|
||||
run: |
|
||||
echo "🏗️ Rebuilding base image with updated dependencies..."
|
||||
VERSION="${{ steps.version_check.outputs.version }}"
|
||||
IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:${{ github.sha }}"
|
||||
|
||||
BASE_BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild-base.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
echo "🚀 Deploying to Cloud Run..."
|
||||
echo "Image: $IMAGE"
|
||||
echo "Version: $VERSION"
|
||||
|
||||
echo "Base Build ID: $BASE_BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BASE_BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
echo "Waiting for base image build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BASE_BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Base build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Base image build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Base image build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BASE_BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
gcloud builds log $BASE_BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Trigger Cloud Build
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
run: |
|
||||
echo "🚀 Triggering Cloud Build for deployment..."
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
echo "Version: ${{ steps.version_check.outputs.version }}"
|
||||
|
||||
BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
|
||||
echo "Build ID: $BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
echo "Waiting for build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
echo ""
|
||||
gcloud builds log $BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
gcloud run deploy karaoke-backend \
|
||||
--image $IMAGE \
|
||||
--region us-central1 \
|
||||
--platform managed \
|
||||
--allow-unauthenticated \
|
||||
--memory 16Gi \
|
||||
--cpu 8 \
|
||||
--cpu-boost \
|
||||
--execution-environment gen2 \
|
||||
--timeout 1800 \
|
||||
--concurrency 1 \
|
||||
--max-instances 50 \
|
||||
--min-instances 0 \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest"
|
||||
|
||||
- name: Verify deployment and version
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
run: |
|
||||
EXPECTED_VERSION="${{ steps.version_check.outputs.version }}"
|
||||
SERVICE_URL=$(gcloud run services describe karaoke-backend \
|
||||
@@ -797,19 +824,13 @@ jobs:
|
||||
done
|
||||
|
||||
- name: Summary
|
||||
if: steps.version_check.outputs.needs_deploy == 'true' && success()
|
||||
if: success()
|
||||
run: |
|
||||
echo "🎉 Deployment successful!"
|
||||
echo "Version: ${{ steps.version_check.outputs.version }}"
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
echo "Branch: ${{ github.ref_name }}"
|
||||
echo "Run: ${{ github.run_number }}"
|
||||
|
||||
- name: Skipped deployment summary
|
||||
if: steps.version_check.outputs.needs_deploy == 'false'
|
||||
run: |
|
||||
echo "⏭️ Backend deployment skipped - version ${{ steps.version_check.outputs.version }} already deployed"
|
||||
echo ""
|
||||
echo "To deploy a new version:"
|
||||
echo " 1. Update version in pyproject.toml"
|
||||
echo " 2. Commit and push to main"
|
||||
echo "📊 Built on self-hosted GCP runner (200GB SSD, no disk space issues)"
|
||||
echo " Runner: gcp-runner with 4 vCPU, 16GB RAM"
|
||||
+6
-1
@@ -47,6 +47,7 @@ frontend/coverage/
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
frontend/playwright/.cache/
|
||||
frontend/e2e/screenshots/
|
||||
|
||||
# TypeScript
|
||||
frontend/*.tsbuildinfo
|
||||
@@ -82,8 +83,9 @@ eggs/
|
||||
# Python lib directories (not frontend)
|
||||
lib/
|
||||
lib64/
|
||||
# But allow frontend lib
|
||||
# But allow frontend lib directories
|
||||
!frontend/lib/
|
||||
!buy-site/lib/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
@@ -227,3 +229,6 @@ cython_debug/
|
||||
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
||||
#.idea/
|
||||
flacfetch
|
||||
|
||||
# Test output directories
|
||||
Rick Astley*/
|
||||
@@ -1,131 +1,97 @@
|
||||
# Project Guidelines for AI Assistants
|
||||
# Karaoke-Gen AI Assistant Guidelines
|
||||
|
||||
This document contains guidelines and rules for AI assistants working on this codebase.
|
||||
## Project Overview
|
||||
|
||||
## Core Principles
|
||||
**Karaoke video generation platform** - CLI tool and web service that creates professional karaoke videos with synchronized lyrics.
|
||||
|
||||
**Always be testing.** Proactively evaluate, read, run, fix, and improve tests whenever you touch any part of the system—frontend, backend, infrastructure. Every code change should be thoroughly tested. Leave everything better tested than you found it.
|
||||
- **Production**: <https://gen.nomadkaraoke.com> (frontend), <https://api.nomadkaraoke.com> (backend)
|
||||
- **CLI**: `karaoke-gen` (local), `karaoke-gen-remote` (cloud)
|
||||
- **Repo**: <https://github.com/nomadkaraoke/karaoke-gen>
|
||||
|
||||
**Always be documenting.** Continuously read, write, improve, update, and organize markdown documentation. Reference existing docs and keep them accurate as the codebase evolves.
|
||||
## Quick Reference
|
||||
|
||||
**Build for maintainability.** Keep code SOLID, modular, and clean. Favor simplicity and clarity. Future developers (and AI agents) will thank you.
|
||||
| What | Where |
|
||||
|------|-------|
|
||||
| Current status | `docs/README.md` |
|
||||
| Architecture | `docs/ARCHITECTURE.md` |
|
||||
| Dev setup & testing | `docs/DEVELOPMENT.md` |
|
||||
| API reference | `docs/API.md` |
|
||||
| Past learnings | `docs/LESSONS-LEARNED.md` |
|
||||
|
||||
## Branching Workflow
|
||||
## Tech Stack
|
||||
|
||||
**Never commit directly to `main`.** Always work on a feature branch.
|
||||
- **Backend**: FastAPI on Cloud Run, Firestore, GCS, Secret Manager
|
||||
- **Frontend**: Next.js on Cloudflare Pages
|
||||
- **Processing**: Modal (GPU audio separation), AudioShake/Whisper (transcription)
|
||||
- **Infra**: Pulumi IaC, GitHub Actions CI/CD
|
||||
|
||||
Create descriptive branch names like `feature/add-discord-notifications`, `fix/audio-sync-issue`, or `refactor/cleanup-api-routes`.
|
||||
## Essential Rules
|
||||
|
||||
### Using Git Worktrees
|
||||
|
||||
**For non-trivial changes** (new features, multi-file edits, refactors):
|
||||
- Create a git worktree in the parent folder before making any file changes:
|
||||
### Git Workflow
|
||||
- **Never commit directly to `main`** - always use feature branches
|
||||
- **Use git worktrees** for all changes:
|
||||
```bash
|
||||
git worktree add ../karaoke-gen-feat-xyz feature/xyz
|
||||
git worktree add -b feature/xyz ../karaoke-gen-feat-xyz main
|
||||
```
|
||||
- This keeps the main directory clean on `main` and enables parallel work on multiple features
|
||||
- Name the worktree folder descriptively, e.g., `../karaoke-gen-fix-audio-sync`
|
||||
|
||||
**For trivial changes** (typos, single-line fixes):
|
||||
- Creating a branch in the current directory is acceptable
|
||||
- Use `git checkout -b fix/typo-xyz` before making changes
|
||||
|
||||
**Cleanup:** When done with a worktree, remove it with:
|
||||
```bash
|
||||
git worktree remove ../karaoke-gen-feat-xyz
|
||||
```
|
||||
|
||||
### Creating a Pull Request
|
||||
|
||||
Once all changes are complete, tests pass, and commits are pushed, create a PR with:
|
||||
|
||||
1. **Summary** - Brief overview of the changes
|
||||
2. **Changes Made** - List of specific modifications
|
||||
3. **Testing** - How changes were tested (unit tests, manual testing)
|
||||
4. **Related Issues** - Reference any GitHub issues (e.g., "Fixes #123")
|
||||
|
||||
Only merge after review and CI checks pass.
|
||||
|
||||
## Infrastructure Changes
|
||||
|
||||
All Google Cloud infrastructure (IAM, Cloud Storage, Cloud Run, Secret Manager, Artifact Registry, etc.) must be managed via **Pulumi IaC** in the `infrastructure/` folder.
|
||||
|
||||
### Rules
|
||||
|
||||
- **Reading/querying** with `gcloud` CLI is fine for investigation and debugging
|
||||
- **Temporary changes** via `gcloud` CLI or Console are acceptable for urgent fixes or experimentation, but **must be cleaned up** and properly codified in the IaC afterward
|
||||
- **Permanent changes** must go through Pulumi — never leave `gcloud` or Console changes as the final state
|
||||
|
||||
### Workflow
|
||||
|
||||
1. Make changes in `infrastructure/`
|
||||
2. Run `pulumi preview` and verify the output
|
||||
3. Run `pulumi up` to apply
|
||||
|
||||
### Authentication Issues
|
||||
|
||||
If you encounter any authentication issues with `gcloud` or `pulumi` (e.g., expired tokens, permission errors), **stop and notify the user** so they can run the appropriate auth commands in their external terminal:
|
||||
|
||||
```bash
|
||||
gcloud auth login
|
||||
gcloud auth application-default login
|
||||
```
|
||||
|
||||
Do not attempt to work around auth issues or make assumptions about credentials.
|
||||
|
||||
## Version Management
|
||||
|
||||
When committing and pushing changes, always bump the patch version in `pyproject.toml`.
|
||||
|
||||
The version is located at `tool.poetry.version` in `pyproject.toml`. For example, if the current version is `0.70.3`, bump it to `0.70.4`.
|
||||
|
||||
This ensures every commit pushed to the repository has a unique version number for tracking and deployment purposes.
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
**Before committing and pushing ANY changes, you MUST run the full test suite.**
|
||||
|
||||
### Backend Test Command
|
||||
- Create PR with summary, changes, testing info
|
||||
- Merge only after CI passes
|
||||
|
||||
### Testing (Required Before Commit)
|
||||
```bash
|
||||
# Backend (10 min timeout, limit output)
|
||||
make test 2>&1 | tail -n 500
|
||||
```
|
||||
|
||||
**Important for AI agents:**
|
||||
- Set a **10 minute timeout** when running tests (they can take a while with emulators)
|
||||
- Always pipe output through `tail -n 500` to limit output and preserve context window
|
||||
- The tail output will show test results and any failures at the end
|
||||
|
||||
This runs all backend tests in order and fails fast if any test fails:
|
||||
1. Unit tests (karaoke_gen package) with coverage check
|
||||
2. Backend unit tests
|
||||
3. E2E integration tests with emulators (auto-starts/stops emulators)
|
||||
|
||||
### Frontend Test Command
|
||||
|
||||
When working in the `frontend/` directory, run:
|
||||
|
||||
```bash
|
||||
# Frontend
|
||||
cd frontend && npm run test:all 2>&1 | tail -n 200
|
||||
```
|
||||
|
||||
This runs:
|
||||
1. **Unit tests** (Jest) - Component and utility tests
|
||||
2. **E2E tests** (Playwright) - Browser-based integration tests
|
||||
### Version Bumping
|
||||
- Bump `tool.poetry.version` in `pyproject.toml` for code changes
|
||||
- Skip for docs-only changes
|
||||
|
||||
**Note:** E2E tests require `KARAOKE_ACCESS_TOKEN` environment variable to be set in `frontend/.env.local`.
|
||||
### Infrastructure
|
||||
- All GCP changes via Pulumi in `infrastructure/`
|
||||
- `gcloud` CLI for reading/debugging only
|
||||
- Stop and notify user on auth issues
|
||||
|
||||
Individual test commands:
|
||||
- `npm run test:unit` - Run unit tests only
|
||||
- `npm run test:e2e` - Run E2E tests only
|
||||
- `npm run test:ci` - Run unit tests with coverage
|
||||
## Documentation Maintenance
|
||||
|
||||
### Pre-commit Checklist
|
||||
### Structure
|
||||
```text
|
||||
docs/
|
||||
├── README.md # Current status + navigation (UPDATE THIS)
|
||||
├── ARCHITECTURE.md # System design
|
||||
├── DEVELOPMENT.md # Dev setup, testing, deployment
|
||||
├── API.md # Backend API reference
|
||||
├── LESSONS-LEARNED.md # Key insights for future agents
|
||||
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
|
||||
```
|
||||
|
||||
- [ ] `make test` passes (for backend changes)
|
||||
- [ ] `cd frontend && npm run test:all` passes (for frontend changes)
|
||||
- [ ] Version in `pyproject.toml` has been bumped
|
||||
- [ ] `coderabbit review --plain` has been run to get comprehensive analysis and suggestions for cleaner, more maintainable code. Apply the feedback to improve accessibility, structure, and best practices.
|
||||
### Rules
|
||||
1. **Before merging PRs**: Run `/docs-review` to check if docs need updating
|
||||
2. **Periodically**: Run `/docs-maintain` to verify docs are organized
|
||||
3. **For significant work**: Create `docs/archive/YYYY-MM-DD-topic.md`
|
||||
4. **Add learnings**: Update `docs/LESSONS-LEARNED.md` with insights
|
||||
5. **Keep status current**: Update `docs/README.md` when project state changes
|
||||
|
||||
**Do not commit or push code that fails tests. Fix any test failures before proceeding.**
|
||||
### What Goes Where
|
||||
- **README.md**: Current status, known issues, quick links
|
||||
- **ARCHITECTURE.md**: System design, data flow, tech decisions
|
||||
- **DEVELOPMENT.md**: Setup, testing, deployment, CI/CD
|
||||
- **API.md**: Endpoints, request/response formats
|
||||
- **LESSONS-LEARNED.md**: What worked, what didn't, gotchas
|
||||
- **archive/**: Completed work, session summaries, old plans
|
||||
|
||||
## Related Projects
|
||||
|
||||
### flacfetch
|
||||
Location: `/Users/andrew/Projects/flacfetch`
|
||||
|
||||
Workflow: Make changes on `main`, bump version, push, wait for CI, then `poetry update flacfetch` in karaoke-gen.
|
||||
|
||||
## Pre-Commit Checklist
|
||||
|
||||
- [ ] Tests pass (`make test` / `npm run test:all`)
|
||||
- [ ] Version bumped (if code changed)
|
||||
- [ ] `/docs-review` run (check for doc updates)
|
||||
- [ ] PR created with clear description
|
||||
@@ -71,8 +71,40 @@ export AUDIOSHAKE_API_TOKEN="your_audioshake_token"
|
||||
|
||||
Get an API key at [https://www.audioshake.ai/](https://www.audioshake.ai/) - business only, at time of writing this.
|
||||
|
||||
#### Option 2: Whisper via RunPod
|
||||
Open-source alternative using OpenAI's Whisper model on RunPod infrastructure.
|
||||
#### Option 2: Local Whisper (No Cloud Required)
|
||||
Run Whisper directly on your local machine using whisper-timestamped. Works on CPU, NVIDIA GPU (CUDA), or Apple Silicon.
|
||||
|
||||
```bash
|
||||
# Install with local Whisper support
|
||||
pip install "karaoke-gen[local-whisper]"
|
||||
|
||||
# Optional: Configure model size (tiny, base, small, medium, large)
|
||||
export WHISPER_MODEL_SIZE="medium"
|
||||
|
||||
# Optional: Force specific device (cpu, cuda, mps)
|
||||
export WHISPER_DEVICE="cpu"
|
||||
```
|
||||
|
||||
**Model Size Guide:**
|
||||
| Model | VRAM | Speed | Quality |
|
||||
|-------|------|-------|---------|
|
||||
| tiny | ~1GB | Fast | Lower |
|
||||
| base | ~1GB | Fast | Basic |
|
||||
| small | ~2GB | Medium | Good |
|
||||
| medium | ~5GB | Slower | Better |
|
||||
| large | ~10GB | Slowest | Best |
|
||||
|
||||
**CPU-Only Installation** (no GPU required):
|
||||
```bash
|
||||
# Pre-install CPU-only PyTorch first
|
||||
pip install torch torchaudio --index-url https://download.pytorch.org/whl/cpu
|
||||
pip install "karaoke-gen[local-whisper]"
|
||||
```
|
||||
|
||||
Local Whisper runs automatically as a fallback when no cloud transcription services are configured.
|
||||
|
||||
#### Option 3: Whisper via RunPod
|
||||
Cloud-based alternative using OpenAI's Whisper model on RunPod infrastructure.
|
||||
|
||||
```bash
|
||||
export RUNPOD_API_KEY="your_runpod_key"
|
||||
@@ -574,6 +606,44 @@ If the output video has quality problems:
|
||||
- Check available codecs: `ffmpeg -codecs`
|
||||
- For 4K output, ensure sufficient disk space (10GB+ per track)
|
||||
|
||||
### Local Whisper Issues
|
||||
|
||||
#### GPU Out of Memory
|
||||
If you get CUDA out of memory errors:
|
||||
```bash
|
||||
# Use a smaller model
|
||||
export WHISPER_MODEL_SIZE="small" # or "tiny"
|
||||
|
||||
# Or force CPU mode
|
||||
export WHISPER_DEVICE="cpu"
|
||||
```
|
||||
|
||||
#### Slow Transcription on CPU
|
||||
CPU transcription is significantly slower than GPU. For faster processing:
|
||||
- Use a smaller model (`tiny` or `base`)
|
||||
- Consider using cloud transcription (AudioShake or RunPod)
|
||||
- On Apple Silicon, the `small` model offers good speed/quality balance
|
||||
|
||||
#### Model Download Issues
|
||||
Whisper models are downloaded on first use (~1-3GB depending on size). If downloads fail:
|
||||
- Check your internet connection
|
||||
- Set a custom cache directory: `export WHISPER_CACHE_DIR="/path/with/space"`
|
||||
- Models are cached in `~/.cache/whisper/` by default
|
||||
|
||||
#### whisper-timestamped Not Found
|
||||
If you get "whisper-timestamped is not installed":
|
||||
```bash
|
||||
pip install "karaoke-gen[local-whisper]"
|
||||
# Or install directly:
|
||||
pip install whisper-timestamped
|
||||
```
|
||||
|
||||
#### Disabling Local Whisper
|
||||
If you want to disable local Whisper (e.g., to force cloud transcription):
|
||||
```bash
|
||||
export ENABLE_LOCAL_WHISPER="false"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Development
|
||||
|
||||
+212
-101
@@ -15,6 +15,7 @@ import json
|
||||
import logging
|
||||
import tempfile
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from fastapi import APIRouter, UploadFile, File, Form, HTTPException, BackgroundTasks, Request, Body, Depends
|
||||
from pathlib import Path
|
||||
from typing import Optional, List, Dict, Any, Tuple
|
||||
@@ -275,6 +276,46 @@ def _resolve_cdg_txt_defaults(
|
||||
return resolved_cdg, resolved_txt
|
||||
|
||||
|
||||
@dataclass
|
||||
class EffectiveDistributionSettings:
|
||||
"""Settings with defaults applied from environment variables."""
|
||||
dropbox_path: Optional[str]
|
||||
gdrive_folder_id: Optional[str]
|
||||
discord_webhook_url: Optional[str]
|
||||
brand_prefix: Optional[str]
|
||||
|
||||
|
||||
def _get_effective_distribution_settings(
|
||||
dropbox_path: Optional[str] = None,
|
||||
gdrive_folder_id: Optional[str] = None,
|
||||
discord_webhook_url: Optional[str] = None,
|
||||
brand_prefix: Optional[str] = None,
|
||||
) -> EffectiveDistributionSettings:
|
||||
"""
|
||||
Get distribution settings with defaults applied from environment variables.
|
||||
|
||||
This ensures consistent handling of defaults across all job creation endpoints.
|
||||
Each parameter, if not provided (None), falls back to the corresponding
|
||||
environment variable configured in settings.
|
||||
|
||||
Args:
|
||||
dropbox_path: Explicit Dropbox path or None for default
|
||||
gdrive_folder_id: Explicit Google Drive folder ID or None for default
|
||||
discord_webhook_url: Explicit Discord webhook URL or None for default
|
||||
brand_prefix: Explicit brand prefix or None for default
|
||||
|
||||
Returns:
|
||||
EffectiveDistributionSettings with defaults applied
|
||||
"""
|
||||
settings = get_settings()
|
||||
return EffectiveDistributionSettings(
|
||||
dropbox_path=dropbox_path or settings.default_dropbox_path,
|
||||
gdrive_folder_id=gdrive_folder_id or settings.default_gdrive_folder_id,
|
||||
discord_webhook_url=discord_webhook_url or settings.default_discord_webhook_url,
|
||||
brand_prefix=brand_prefix or settings.default_brand_prefix,
|
||||
)
|
||||
|
||||
|
||||
def _prepare_theme_for_job(
|
||||
job_id: str,
|
||||
theme_id: str,
|
||||
@@ -341,9 +382,12 @@ async def upload_and_create_job(
|
||||
style_cdg_instrumental_background: Optional[UploadFile] = File(None, description="CDG instrumental background"),
|
||||
style_cdg_title_background: Optional[UploadFile] = File(None, description="CDG title screen background"),
|
||||
style_cdg_outro_background: Optional[UploadFile] = File(None, description="CDG outro screen background"),
|
||||
# Processing options (CDG/TXT require style config, disabled by default)
|
||||
enable_cdg: bool = Form(False, description="Generate CDG+MP3 package (requires style config)"),
|
||||
enable_txt: bool = Form(False, description="Generate TXT+MP3 package (requires style config)"),
|
||||
# Theme configuration (use pre-made theme from GCS)
|
||||
theme_id: Optional[str] = Form(None, description="Theme ID to use (e.g., 'nomad', 'default'). If set, CDG/TXT are enabled by default."),
|
||||
color_overrides: Optional[str] = Form(None, description="JSON-encoded color overrides: artist_color, title_color, sung_lyrics_color, unsung_lyrics_color (hex #RRGGBB)"),
|
||||
# Processing options (CDG/TXT require style config or theme, disabled by default unless theme is set)
|
||||
enable_cdg: Optional[bool] = Form(None, description="Generate CDG+MP3 package. Default: True if theme_id set, False otherwise"),
|
||||
enable_txt: Optional[bool] = Form(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise"),
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Form(None, description="Brand code prefix (e.g., NOMAD)"),
|
||||
enable_youtube_upload: bool = Form(False, description="Upload to YouTube"),
|
||||
@@ -432,38 +476,42 @@ async def upload_and_create_job(
|
||||
)
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
if effective_dropbox_path and not dropbox_path:
|
||||
logger.info(f"Using default dropbox_path: {effective_dropbox_path}")
|
||||
if effective_gdrive_folder_id and not gdrive_folder_id:
|
||||
logger.info(f"Using default gdrive_folder_id: {effective_gdrive_folder_id}")
|
||||
if effective_discord_webhook_url and not discord_webhook_url:
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=dropbox_path,
|
||||
gdrive_folder_id=gdrive_folder_id,
|
||||
discord_webhook_url=discord_webhook_url,
|
||||
brand_prefix=brand_prefix,
|
||||
)
|
||||
|
||||
if dist.dropbox_path and not dropbox_path:
|
||||
logger.info(f"Using default dropbox_path: {dist.dropbox_path}")
|
||||
if dist.gdrive_folder_id and not gdrive_folder_id:
|
||||
logger.info(f"Using default gdrive_folder_id: {dist.gdrive_folder_id}")
|
||||
if dist.discord_webhook_url and not discord_webhook_url:
|
||||
logger.info("Using default discord_webhook_url (from env)")
|
||||
|
||||
if dist.brand_prefix and not brand_prefix:
|
||||
logger.info(f"Using default brand_prefix: {dist.brand_prefix}")
|
||||
|
||||
# Validate credentials for requested distribution services (including defaults)
|
||||
# This prevents accepting jobs that will fail later due to missing credentials
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -477,7 +525,35 @@ async def upload_and_create_job(
|
||||
|
||||
# Extract request metadata for tracking and filtering
|
||||
request_metadata = extract_request_metadata(request, created_from="upload")
|
||||
|
||||
|
||||
# Parse color_overrides from JSON if provided
|
||||
parsed_color_overrides: Dict[str, str] = {}
|
||||
if color_overrides:
|
||||
try:
|
||||
parsed_color_overrides = json.loads(color_overrides)
|
||||
except json.JSONDecodeError as e:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid color_overrides JSON: {e}"
|
||||
)
|
||||
|
||||
# Resolve CDG/TXT defaults based on theme
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
theme_id, enable_cdg, enable_txt
|
||||
)
|
||||
|
||||
# Check if any custom style files are being uploaded (overrides theme)
|
||||
has_custom_style_files = any([
|
||||
style_params,
|
||||
style_intro_background,
|
||||
style_karaoke_background,
|
||||
style_end_background,
|
||||
style_font,
|
||||
style_cdg_instrumental_background,
|
||||
style_cdg_title_background,
|
||||
style_cdg_outro_background,
|
||||
])
|
||||
|
||||
# Parse comma-separated model lists into arrays
|
||||
parsed_backing_vocals_models = None
|
||||
if backing_vocals_models:
|
||||
@@ -492,17 +568,19 @@ async def upload_and_create_job(
|
||||
artist=artist,
|
||||
title=title,
|
||||
filename=file.filename,
|
||||
enable_cdg=enable_cdg,
|
||||
enable_txt=enable_txt,
|
||||
brand_prefix=brand_prefix,
|
||||
theme_id=theme_id,
|
||||
color_overrides=parsed_color_overrides,
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=enable_youtube_upload,
|
||||
youtube_description=youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=webhook_url,
|
||||
user_email=user_email,
|
||||
# Native API distribution (preferred for remote CLI)
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
# Legacy rclone distribution (deprecated)
|
||||
organised_dir_rclone_root=organised_dir_rclone_root,
|
||||
# Lyrics configuration (overrides for search/transcription)
|
||||
@@ -521,9 +599,27 @@ async def upload_and_create_job(
|
||||
|
||||
# Record job creation metric
|
||||
metrics.record_job_created(job_id, source="upload")
|
||||
|
||||
|
||||
logger.info(f"Created job {job_id} for {artist} - {title}")
|
||||
|
||||
|
||||
# If theme is set and no custom style files are being uploaded, prepare theme style now
|
||||
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
|
||||
# can access the style configuration for preview videos
|
||||
theme_style_params_path = None
|
||||
theme_style_assets = {}
|
||||
theme_youtube_desc = None
|
||||
if theme_id and not has_custom_style_files:
|
||||
try:
|
||||
theme_style_params_path, theme_style_assets, theme_youtube_desc = _prepare_theme_for_job(
|
||||
job_id, theme_id, parsed_color_overrides or None
|
||||
)
|
||||
logger.info(f"Applied theme '{theme_id}' to job {job_id}")
|
||||
except HTTPException:
|
||||
raise # Re-raise validation errors (e.g., theme not found)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to prepare theme '{theme_id}' for job {job_id}: {e}")
|
||||
# Continue without theme - job can still be processed with defaults
|
||||
|
||||
# Upload main audio file to GCS
|
||||
audio_gcs_path = f"uploads/{job_id}/audio/{file.filename}"
|
||||
logger.info(f"Uploading audio to GCS: {audio_gcs_path}")
|
||||
@@ -592,27 +688,36 @@ async def upload_and_create_job(
|
||||
update_data = {
|
||||
'input_media_gcs_path': audio_gcs_path,
|
||||
'filename': file.filename,
|
||||
'enable_cdg': enable_cdg,
|
||||
'enable_txt': enable_txt,
|
||||
'enable_cdg': resolved_cdg,
|
||||
'enable_txt': resolved_txt,
|
||||
}
|
||||
|
||||
|
||||
# Handle style assets - either from custom uploads or from theme
|
||||
if style_assets:
|
||||
# Custom style files uploaded
|
||||
update_data['style_assets'] = style_assets
|
||||
if 'style_params' in style_assets:
|
||||
update_data['style_params_gcs_path'] = style_assets['style_params']
|
||||
|
||||
if brand_prefix:
|
||||
update_data['brand_prefix'] = brand_prefix
|
||||
if effective_discord_webhook_url:
|
||||
update_data['discord_webhook_url'] = effective_discord_webhook_url
|
||||
if youtube_description:
|
||||
update_data['youtube_description_template'] = youtube_description
|
||||
|
||||
elif theme_style_assets:
|
||||
# Theme style assets (no custom uploads)
|
||||
update_data['style_assets'] = theme_style_assets
|
||||
if theme_style_params_path:
|
||||
update_data['style_params_gcs_path'] = theme_style_params_path
|
||||
|
||||
if dist.brand_prefix:
|
||||
update_data['brand_prefix'] = dist.brand_prefix
|
||||
if dist.discord_webhook_url:
|
||||
update_data['discord_webhook_url'] = dist.discord_webhook_url
|
||||
# Use theme YouTube description if no custom one provided
|
||||
effective_youtube_description = youtube_description or theme_youtube_desc
|
||||
if effective_youtube_description:
|
||||
update_data['youtube_description_template'] = effective_youtube_description
|
||||
|
||||
# Native API distribution (use effective values which include defaults)
|
||||
if effective_dropbox_path:
|
||||
update_data['dropbox_path'] = effective_dropbox_path
|
||||
if effective_gdrive_folder_id:
|
||||
update_data['gdrive_folder_id'] = effective_gdrive_folder_id
|
||||
if dist.dropbox_path:
|
||||
update_data['dropbox_path'] = dist.dropbox_path
|
||||
if dist.gdrive_folder_id:
|
||||
update_data['gdrive_folder_id'] = dist.gdrive_folder_id
|
||||
|
||||
# Legacy rclone distribution (deprecated)
|
||||
if organised_dir_rclone_root:
|
||||
@@ -668,21 +773,21 @@ async def upload_and_create_job(
|
||||
|
||||
# Build distribution services info for response
|
||||
distribution_services: Dict[str, Any] = {}
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
dropbox_result = credential_manager.check_dropbox_credentials()
|
||||
distribution_services["dropbox"] = {
|
||||
"enabled": True,
|
||||
"path": effective_dropbox_path,
|
||||
"path": dist.dropbox_path,
|
||||
"credentials_valid": dropbox_result.status == CredentialStatus.VALID,
|
||||
"using_default": dropbox_path is None,
|
||||
}
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
gdrive_result = credential_manager.check_gdrive_credentials()
|
||||
distribution_services["gdrive"] = {
|
||||
"enabled": True,
|
||||
"folder_id": effective_gdrive_folder_id,
|
||||
"folder_id": dist.gdrive_folder_id,
|
||||
"credentials_valid": gdrive_result.status == CredentialStatus.VALID,
|
||||
"using_default": gdrive_folder_id is None,
|
||||
}
|
||||
@@ -694,7 +799,7 @@ async def upload_and_create_job(
|
||||
"credentials_valid": youtube_result.status == CredentialStatus.VALID,
|
||||
}
|
||||
|
||||
if effective_discord_webhook_url:
|
||||
if dist.discord_webhook_url:
|
||||
distribution_services["discord"] = {
|
||||
"enabled": True,
|
||||
"using_default": discord_webhook_url is None,
|
||||
@@ -911,32 +1016,34 @@ async def create_job_with_upload_urls(
|
||||
status_code=400,
|
||||
detail=f"Invalid file extension '{ext}' for file_type '{file_info.file_type}'. Allowed: {allowed_extensions}"
|
||||
)
|
||||
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -947,7 +1054,7 @@ async def create_job_with_upload_urls(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata for tracking
|
||||
request_metadata = extract_request_metadata(request, created_from="signed_url_upload")
|
||||
|
||||
@@ -971,14 +1078,14 @@ async def create_job_with_upload_urls(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=body.brand_prefix,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=body.webhook_url,
|
||||
user_email=body.user_email,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
organised_dir_rclone_root=body.organised_dir_rclone_root,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
@@ -1322,32 +1429,34 @@ async def create_job_from_url(
|
||||
status_code=400,
|
||||
detail="Invalid URL. Please provide a valid YouTube, Vimeo, SoundCloud, or other supported video URL."
|
||||
)
|
||||
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1358,7 +1467,7 @@ async def create_job_from_url(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata for tracking
|
||||
request_metadata = extract_request_metadata(request, created_from="url")
|
||||
|
||||
@@ -1381,14 +1490,14 @@ async def create_job_from_url(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=body.brand_prefix,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=body.webhook_url,
|
||||
user_email=body.user_email,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
organised_dir_rclone_root=body.organised_dir_rclone_root,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
@@ -1563,30 +1672,32 @@ async def create_finalise_only_job(
|
||||
)
|
||||
|
||||
# Apply default distribution settings
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Validate distribution credentials if services are requested
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1597,7 +1708,7 @@ async def create_finalise_only_job(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata
|
||||
request_metadata = extract_request_metadata(request, created_from="finalise_only_upload")
|
||||
|
||||
@@ -1618,12 +1729,12 @@ async def create_finalise_only_job(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=body.brand_prefix,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
finalise_only=True,
|
||||
keep_brand_code=body.keep_brand_code,
|
||||
request_metadata=request_metadata,
|
||||
|
||||
@@ -0,0 +1,819 @@
|
||||
"""
|
||||
User and authentication API routes.
|
||||
|
||||
Handles:
|
||||
- Magic link authentication (send, verify)
|
||||
- Session management (logout)
|
||||
- User profile and credits
|
||||
- Stripe checkout and webhooks
|
||||
- Admin user management
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional, Tuple
|
||||
from fastapi import APIRouter, HTTPException, Depends, Request, Header
|
||||
from pydantic import BaseModel, EmailStr
|
||||
|
||||
from backend.models.user import (
|
||||
UserRole,
|
||||
UserPublic,
|
||||
SendMagicLinkRequest,
|
||||
SendMagicLinkResponse,
|
||||
VerifyMagicLinkResponse,
|
||||
AddCreditsRequest,
|
||||
AddCreditsResponse,
|
||||
UserListResponse,
|
||||
BetaTesterStatus,
|
||||
BetaTesterEnrollRequest,
|
||||
BetaTesterEnrollResponse,
|
||||
BetaFeedbackRequest,
|
||||
BetaFeedbackResponse,
|
||||
BetaTesterFeedback,
|
||||
)
|
||||
from backend.services.user_service import get_user_service, UserService
|
||||
from backend.services.email_service import get_email_service, EmailService
|
||||
from backend.services.stripe_service import get_stripe_service, StripeService, CREDIT_PACKAGES
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.services.auth_service import UserType
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/users", tags=["users"])
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Request/Response Models
|
||||
# =============================================================================
|
||||
|
||||
class CreateCheckoutRequest(BaseModel):
|
||||
"""Request to create a Stripe checkout session."""
|
||||
package_id: str
|
||||
email: EmailStr
|
||||
|
||||
|
||||
class CreateCheckoutResponse(BaseModel):
|
||||
"""Response with checkout URL."""
|
||||
status: str
|
||||
checkout_url: str
|
||||
message: str
|
||||
|
||||
|
||||
class CreditPackage(BaseModel):
|
||||
"""Credit package information."""
|
||||
id: str
|
||||
credits: int
|
||||
price_cents: int
|
||||
name: str
|
||||
description: str
|
||||
|
||||
|
||||
class CreditPackagesResponse(BaseModel):
|
||||
"""Response listing available credit packages."""
|
||||
packages: list[CreditPackage]
|
||||
|
||||
|
||||
class UserProfileResponse(BaseModel):
|
||||
"""Response with user profile."""
|
||||
user: UserPublic
|
||||
has_session: bool
|
||||
|
||||
|
||||
class LogoutResponse(BaseModel):
|
||||
"""Response after logout."""
|
||||
status: str
|
||||
message: str
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Magic Link Authentication
|
||||
# =============================================================================
|
||||
|
||||
@router.post("/auth/magic-link", response_model=SendMagicLinkResponse)
|
||||
async def send_magic_link(
|
||||
request: SendMagicLinkRequest,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Send a magic link email for passwordless authentication.
|
||||
|
||||
The user will receive an email with a link that logs them in.
|
||||
Links expire after 15 minutes and can only be used once.
|
||||
"""
|
||||
email = request.email.lower()
|
||||
|
||||
# Get client info for security logging
|
||||
ip_address = http_request.client.host if http_request.client else None
|
||||
user_agent = http_request.headers.get("user-agent")
|
||||
|
||||
# Create magic link token
|
||||
magic_link = user_service.create_magic_link(
|
||||
email,
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent
|
||||
)
|
||||
|
||||
# Send email
|
||||
sent = email_service.send_magic_link(email, magic_link.token)
|
||||
|
||||
if not sent:
|
||||
logger.error(f"Failed to send magic link email to {email}")
|
||||
# Don't reveal failure to prevent email enumeration
|
||||
# Still return success
|
||||
|
||||
return SendMagicLinkResponse(
|
||||
status="success",
|
||||
message="If this email is registered, you will receive a sign-in link shortly."
|
||||
)
|
||||
|
||||
|
||||
@router.get("/auth/verify", response_model=VerifyMagicLinkResponse)
|
||||
async def verify_magic_link(
|
||||
token: str,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Verify a magic link token and create a session.
|
||||
|
||||
Returns a session token that should be stored and used for subsequent requests.
|
||||
"""
|
||||
# Get client info
|
||||
ip_address = http_request.client.host if http_request.client else None
|
||||
user_agent = http_request.headers.get("user-agent")
|
||||
|
||||
# Check if this is a first login BEFORE verification (which sets last_login_at)
|
||||
# We need to get the user's state before verify_magic_link updates it
|
||||
from backend.services.user_service import MAGIC_LINKS_COLLECTION
|
||||
magic_link_doc = user_service.db.collection(MAGIC_LINKS_COLLECTION).document(token).get()
|
||||
is_first_login = False
|
||||
if magic_link_doc.exists:
|
||||
magic_link_data = magic_link_doc.to_dict()
|
||||
pre_verify_user = user_service.get_user(magic_link_data.get('email', ''))
|
||||
if pre_verify_user:
|
||||
is_first_login = pre_verify_user.total_jobs_created == 0 and not pre_verify_user.last_login_at
|
||||
|
||||
# Verify the magic link
|
||||
success, user, message = user_service.verify_magic_link(token)
|
||||
|
||||
if not success or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
# Create session
|
||||
session = user_service.create_session(
|
||||
user.email,
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent
|
||||
)
|
||||
|
||||
# Send welcome email to first-time users
|
||||
if is_first_login:
|
||||
email_service.send_welcome_email(user.email, user.credits)
|
||||
|
||||
# Return user info
|
||||
user_public = UserPublic(
|
||||
email=user.email,
|
||||
role=user.role,
|
||||
credits=user.credits,
|
||||
display_name=user.display_name,
|
||||
total_jobs_created=user.total_jobs_created,
|
||||
total_jobs_completed=user.total_jobs_completed,
|
||||
)
|
||||
|
||||
return VerifyMagicLinkResponse(
|
||||
status="success",
|
||||
session_token=session.token,
|
||||
user=user_public,
|
||||
message="Successfully signed in"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/auth/logout", response_model=LogoutResponse)
|
||||
async def logout(
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Logout and invalidate the current session.
|
||||
"""
|
||||
if not authorization:
|
||||
return LogoutResponse(status="success", message="Already logged out")
|
||||
|
||||
# Extract token from "Bearer <token>"
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
|
||||
user_service.revoke_session(token)
|
||||
|
||||
return LogoutResponse(status="success", message="Successfully logged out")
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# User Profile
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/me", response_model=UserProfileResponse)
|
||||
async def get_current_user(
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Get the current user's profile.
|
||||
|
||||
Requires a valid session token.
|
||||
"""
|
||||
if not authorization:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
|
||||
# Extract token
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
|
||||
# Validate session
|
||||
valid, user, message = user_service.validate_session(token)
|
||||
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
user_public = UserPublic(
|
||||
email=user.email,
|
||||
role=user.role,
|
||||
credits=user.credits,
|
||||
display_name=user.display_name,
|
||||
total_jobs_created=user.total_jobs_created,
|
||||
total_jobs_completed=user.total_jobs_completed,
|
||||
)
|
||||
|
||||
return UserProfileResponse(user=user_public, has_session=True)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Credit Packages & Checkout
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/credits/packages", response_model=CreditPackagesResponse)
|
||||
async def list_credit_packages():
|
||||
"""
|
||||
List available credit packages for purchase.
|
||||
|
||||
No authentication required - this is public information.
|
||||
"""
|
||||
packages = [
|
||||
CreditPackage(
|
||||
id=pkg_id,
|
||||
credits=pkg["credits"],
|
||||
price_cents=pkg["price_cents"],
|
||||
name=pkg["name"],
|
||||
description=pkg["description"],
|
||||
)
|
||||
for pkg_id, pkg in CREDIT_PACKAGES.items()
|
||||
]
|
||||
|
||||
return CreditPackagesResponse(packages=packages)
|
||||
|
||||
|
||||
@router.post("/credits/checkout", response_model=CreateCheckoutResponse)
|
||||
async def create_checkout(
|
||||
request: CreateCheckoutRequest,
|
||||
stripe_service: StripeService = Depends(get_stripe_service),
|
||||
):
|
||||
"""
|
||||
Create a Stripe checkout session for purchasing credits.
|
||||
|
||||
Returns a URL to redirect the user to Stripe's hosted checkout page.
|
||||
No authentication required - email is provided in the request.
|
||||
"""
|
||||
if not stripe_service.is_configured():
|
||||
raise HTTPException(status_code=503, detail="Payment processing is not available")
|
||||
|
||||
success, checkout_url, message = stripe_service.create_checkout_session(
|
||||
package_id=request.package_id,
|
||||
user_email=request.email,
|
||||
)
|
||||
|
||||
if not success or not checkout_url:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
return CreateCheckoutResponse(
|
||||
status="success",
|
||||
checkout_url=checkout_url,
|
||||
message=message,
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Stripe Webhooks
|
||||
# =============================================================================
|
||||
|
||||
@router.post("/webhooks/stripe")
|
||||
async def stripe_webhook(
|
||||
request: Request,
|
||||
stripe_signature: str = Header(None, alias="stripe-signature"),
|
||||
stripe_service: StripeService = Depends(get_stripe_service),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Handle Stripe webhook events.
|
||||
|
||||
This endpoint receives events from Stripe about payment status.
|
||||
It verifies the webhook signature and processes the event.
|
||||
"""
|
||||
if not stripe_signature:
|
||||
raise HTTPException(status_code=400, detail="Missing Stripe signature")
|
||||
|
||||
# Get raw body for signature verification
|
||||
payload = await request.body()
|
||||
|
||||
# Verify signature
|
||||
valid, event, message = stripe_service.verify_webhook_signature(payload, stripe_signature)
|
||||
|
||||
if not valid:
|
||||
logger.warning(f"Invalid Stripe webhook signature: {message}")
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
# Handle the event
|
||||
event_type = event.get("type")
|
||||
logger.info(f"Received Stripe webhook: {event_type}")
|
||||
|
||||
if event_type == "checkout.session.completed":
|
||||
session = event["data"]["object"]
|
||||
session_id = session.get("id")
|
||||
|
||||
# Idempotency check: Skip if this session was already processed
|
||||
if session_id and user_service.is_stripe_session_processed(session_id):
|
||||
logger.info(f"Skipping already processed session: {session_id}")
|
||||
return {"status": "received", "type": event_type, "note": "already_processed"}
|
||||
|
||||
# Process the completed checkout
|
||||
success, user_email, credits, _ = stripe_service.handle_checkout_completed(session)
|
||||
|
||||
if success and user_email and credits > 0:
|
||||
# Add credits to user account
|
||||
ok, new_balance, credit_msg = user_service.add_credits(
|
||||
email=user_email,
|
||||
amount=credits,
|
||||
reason="stripe_purchase",
|
||||
stripe_session_id=session_id,
|
||||
)
|
||||
|
||||
if ok:
|
||||
# Send confirmation email
|
||||
email_service.send_credits_added(user_email, credits, new_balance)
|
||||
logger.info(f"Added {credits} credits to {user_email}, new balance: {new_balance}")
|
||||
else:
|
||||
logger.error(f"Failed to add credits: {credit_msg}")
|
||||
|
||||
elif event_type == "checkout.session.expired":
|
||||
logger.info(f"Checkout session expired: {event['data']['object'].get('id')}")
|
||||
|
||||
elif event_type == "payment_intent.payment_failed":
|
||||
logger.warning(f"Payment failed: {event['data']['object'].get('id')}")
|
||||
|
||||
# Return 200 to acknowledge receipt (Stripe will retry on non-2xx)
|
||||
return {"status": "received", "type": event_type}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Beta Tester Program
|
||||
# =============================================================================
|
||||
|
||||
BETA_TESTER_FREE_CREDITS = 1 # Number of free credits for beta testers
|
||||
|
||||
|
||||
@router.post("/beta/enroll", response_model=BetaTesterEnrollResponse)
|
||||
async def enroll_beta_tester(
|
||||
request: BetaTesterEnrollRequest,
|
||||
http_request: Request,
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Enroll as a beta tester to receive free karaoke credits.
|
||||
|
||||
Requirements:
|
||||
- Accept that there may be work to review/correct lyrics
|
||||
- Promise to provide feedback after using the tool
|
||||
|
||||
Returns free credits and optionally a session token for new users.
|
||||
"""
|
||||
email = request.email.lower()
|
||||
|
||||
# Validate acceptance
|
||||
if not request.accept_corrections_work:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="You must accept that you may need to review/correct lyrics"
|
||||
)
|
||||
|
||||
if len(request.promise_text.strip()) < 10:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="Please write a sentence confirming your promise to provide feedback"
|
||||
)
|
||||
|
||||
# Get or create user
|
||||
user = user_service.get_or_create_user(email)
|
||||
|
||||
# Check if already enrolled as beta tester
|
||||
if user.is_beta_tester:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="You are already enrolled in the beta program"
|
||||
)
|
||||
|
||||
# Get client info
|
||||
ip_address = http_request.client.host if http_request.client else None
|
||||
user_agent = http_request.headers.get("user-agent")
|
||||
|
||||
# Enroll as beta tester
|
||||
from datetime import datetime
|
||||
user_service.update_user(
|
||||
email,
|
||||
is_beta_tester=True,
|
||||
beta_tester_status=BetaTesterStatus.ACTIVE.value,
|
||||
beta_enrolled_at=datetime.utcnow(),
|
||||
beta_promise_text=request.promise_text.strip(),
|
||||
)
|
||||
|
||||
# Add free credits
|
||||
_, new_balance, _ = user_service.add_credits(
|
||||
email=email,
|
||||
amount=BETA_TESTER_FREE_CREDITS,
|
||||
reason="beta_tester_enrollment",
|
||||
)
|
||||
|
||||
# Create session for the user (so they can start using the service immediately)
|
||||
session = user_service.create_session(email, ip_address=ip_address, user_agent=user_agent)
|
||||
|
||||
# Send welcome email
|
||||
email_service.send_beta_welcome_email(email, BETA_TESTER_FREE_CREDITS)
|
||||
|
||||
logger.info(f"Beta tester enrolled: {email}, granted {BETA_TESTER_FREE_CREDITS} credits")
|
||||
|
||||
return BetaTesterEnrollResponse(
|
||||
status="success",
|
||||
message=f"Welcome to the beta program! You have {new_balance} free credits.",
|
||||
credits_granted=BETA_TESTER_FREE_CREDITS,
|
||||
session_token=session.token,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/beta/feedback", response_model=BetaFeedbackResponse)
|
||||
async def submit_beta_feedback(
|
||||
request: BetaFeedbackRequest,
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Submit feedback as a beta tester.
|
||||
|
||||
Requires authentication. Updates beta tester status to completed.
|
||||
May grant bonus credits for detailed feedback.
|
||||
"""
|
||||
if not authorization:
|
||||
raise HTTPException(status_code=401, detail="Authentication required")
|
||||
|
||||
# Extract token
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
|
||||
# Validate session
|
||||
valid, user, message = user_service.validate_session(token)
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
# Check if user is a beta tester
|
||||
if not user.is_beta_tester:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="Only beta testers can submit feedback through this endpoint"
|
||||
)
|
||||
|
||||
# Check if already completed feedback
|
||||
if user.beta_tester_status == BetaTesterStatus.COMPLETED:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="You have already submitted feedback. Thank you!"
|
||||
)
|
||||
|
||||
# Save feedback to Firestore
|
||||
import uuid
|
||||
|
||||
feedback = BetaTesterFeedback(
|
||||
id=str(uuid.uuid4()),
|
||||
user_email=user.email,
|
||||
job_id=request.job_id,
|
||||
overall_rating=request.overall_rating,
|
||||
ease_of_use_rating=request.ease_of_use_rating,
|
||||
lyrics_accuracy_rating=request.lyrics_accuracy_rating,
|
||||
correction_experience_rating=request.correction_experience_rating,
|
||||
what_went_well=request.what_went_well,
|
||||
what_could_improve=request.what_could_improve,
|
||||
additional_comments=request.additional_comments,
|
||||
would_recommend=request.would_recommend,
|
||||
would_use_again=request.would_use_again,
|
||||
submitted_via="web",
|
||||
)
|
||||
|
||||
# Save to Firestore
|
||||
user_service.db.collection("beta_feedback").document(feedback.id).set(
|
||||
feedback.model_dump(mode='json')
|
||||
)
|
||||
|
||||
# Update user status
|
||||
user_service.update_user(
|
||||
user.email,
|
||||
beta_tester_status=BetaTesterStatus.COMPLETED.value,
|
||||
)
|
||||
|
||||
# Calculate bonus credits for detailed feedback
|
||||
bonus_credits = 0
|
||||
has_detailed_feedback = (
|
||||
(request.what_went_well and len(request.what_went_well) > 50) or
|
||||
(request.what_could_improve and len(request.what_could_improve) > 50) or
|
||||
(request.additional_comments and len(request.additional_comments) > 50)
|
||||
)
|
||||
|
||||
if has_detailed_feedback:
|
||||
# Grant bonus credit for detailed feedback
|
||||
bonus_credits = 1
|
||||
user_service.add_credits(
|
||||
email=user.email,
|
||||
amount=bonus_credits,
|
||||
reason="beta_feedback_bonus",
|
||||
)
|
||||
|
||||
logger.info(f"Beta feedback received from {user.email}, bonus: {bonus_credits}")
|
||||
|
||||
return BetaFeedbackResponse(
|
||||
status="success",
|
||||
message="Thank you for your feedback!" + (
|
||||
f" You earned {bonus_credits} bonus credit for your detailed response!"
|
||||
if bonus_credits > 0 else ""
|
||||
),
|
||||
bonus_credits=bonus_credits,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/beta/feedback-form")
|
||||
async def get_feedback_form_data(
|
||||
authorization: Optional[str] = Header(None),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Get data needed to show the feedback form.
|
||||
|
||||
Returns whether the user needs to submit feedback and any job context.
|
||||
"""
|
||||
if not authorization:
|
||||
raise HTTPException(status_code=401, detail="Authentication required")
|
||||
|
||||
token = authorization.replace("Bearer ", "") if authorization.startswith("Bearer ") else authorization
|
||||
valid, user, message = user_service.validate_session(token)
|
||||
|
||||
if not valid or not user:
|
||||
raise HTTPException(status_code=401, detail=message)
|
||||
|
||||
return {
|
||||
"is_beta_tester": user.is_beta_tester,
|
||||
"beta_status": user.beta_tester_status,
|
||||
"needs_feedback": (
|
||||
user.is_beta_tester and
|
||||
user.beta_tester_status == BetaTesterStatus.PENDING_FEEDBACK.value
|
||||
),
|
||||
"can_submit_feedback": (
|
||||
user.is_beta_tester and
|
||||
user.beta_tester_status != BetaTesterStatus.COMPLETED.value
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Admin Endpoints
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/admin/users", response_model=UserListResponse)
|
||||
async def list_users(
|
||||
limit: int = 100,
|
||||
include_inactive: bool = False,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
List all users (admin only).
|
||||
"""
|
||||
users = user_service.list_users(limit=limit, include_inactive=include_inactive)
|
||||
|
||||
users_public = [
|
||||
UserPublic(
|
||||
email=u.email,
|
||||
role=u.role,
|
||||
credits=u.credits,
|
||||
display_name=u.display_name,
|
||||
total_jobs_created=u.total_jobs_created,
|
||||
total_jobs_completed=u.total_jobs_completed,
|
||||
)
|
||||
for u in users
|
||||
]
|
||||
|
||||
return UserListResponse(users=users_public, total=len(users_public))
|
||||
|
||||
|
||||
@router.post("/admin/credits", response_model=AddCreditsResponse)
|
||||
async def add_credits_to_user(
|
||||
request: AddCreditsRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
email_service: EmailService = Depends(get_email_service),
|
||||
):
|
||||
"""
|
||||
Add credits to a user's account (admin only).
|
||||
|
||||
Use this to grant free credits to users, e.g., for beta testers or promotions.
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
|
||||
# TODO: Enhance auth system to track admin email identity for better audit trails.
|
||||
# Current token-based admin auth doesn't include email identity.
|
||||
# For now, we log the token prefix for traceability.
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success, new_balance, message = user_service.add_credits(
|
||||
email=request.email,
|
||||
amount=request.amount,
|
||||
reason=request.reason,
|
||||
admin_email=admin_id,
|
||||
)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
# Send notification email
|
||||
if request.amount > 0:
|
||||
email_service.send_credits_added(request.email, request.amount, new_balance)
|
||||
|
||||
return AddCreditsResponse(
|
||||
status="success",
|
||||
email=request.email,
|
||||
credits_added=request.amount,
|
||||
new_balance=new_balance,
|
||||
message=message,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/admin/users/{email}/disable")
|
||||
async def disable_user(
|
||||
email: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Disable a user account (admin only).
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success = user_service.disable_user(email, admin_email=admin_id)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
return {"status": "success", "message": f"User {email} has been disabled"}
|
||||
|
||||
|
||||
@router.post("/admin/users/{email}/enable")
|
||||
async def enable_user(
|
||||
email: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Enable a user account (admin only).
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success = user_service.enable_user(email, admin_email=admin_id)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
return {"status": "success", "message": f"User {email} has been enabled"}
|
||||
|
||||
|
||||
@router.post("/admin/users/{email}/role")
|
||||
async def set_user_role(
|
||||
email: str,
|
||||
role: UserRole,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Set a user's role (admin only).
|
||||
"""
|
||||
admin_token, _, _ = auth_data
|
||||
admin_id = f"admin:{admin_token[:8]}..." if admin_token else "admin:unknown"
|
||||
|
||||
success = user_service.set_user_role(email, role, admin_email=admin_id)
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
return {"status": "success", "message": f"User {email} role set to {role.value}"}
|
||||
|
||||
|
||||
@router.get("/admin/beta/feedback")
|
||||
async def list_beta_feedback(
|
||||
limit: int = 50,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
List all beta tester feedback (admin only).
|
||||
"""
|
||||
from google.cloud import firestore
|
||||
|
||||
query = user_service.db.collection("beta_feedback")
|
||||
query = query.order_by("created_at", direction=firestore.Query.DESCENDING)
|
||||
query = query.limit(limit)
|
||||
|
||||
docs = query.stream()
|
||||
feedback_list = [doc.to_dict() for doc in docs]
|
||||
|
||||
return {
|
||||
"feedback": feedback_list,
|
||||
"total": len(feedback_list),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/admin/beta/stats")
|
||||
async def get_beta_stats(
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Get beta tester program statistics (admin only).
|
||||
"""
|
||||
from google.cloud.firestore_v1 import FieldFilter
|
||||
from google.cloud.firestore_v1 import aggregation
|
||||
|
||||
# Count beta testers by status using efficient aggregation queries
|
||||
users_collection = user_service.db.collection("users")
|
||||
|
||||
# Helper function to get count using aggregation
|
||||
def get_count(query) -> int:
|
||||
agg_query = aggregation.AggregationQuery(query)
|
||||
agg_query.count(alias="count")
|
||||
results = agg_query.get()
|
||||
return results[0][0].value if results else 0
|
||||
|
||||
total_beta_testers = get_count(
|
||||
users_collection.where(filter=FieldFilter("is_beta_tester", "==", True))
|
||||
)
|
||||
|
||||
active_testers = get_count(
|
||||
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "active"))
|
||||
)
|
||||
|
||||
pending_feedback = get_count(
|
||||
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "pending_feedback"))
|
||||
)
|
||||
|
||||
completed_feedback = get_count(
|
||||
users_collection.where(filter=FieldFilter("beta_tester_status", "==", "completed"))
|
||||
)
|
||||
|
||||
# Get average ratings from feedback
|
||||
feedback_docs = list(user_service.db.collection("beta_feedback").stream())
|
||||
|
||||
avg_overall = 0
|
||||
avg_ease = 0
|
||||
avg_accuracy = 0
|
||||
avg_correction = 0
|
||||
|
||||
if feedback_docs:
|
||||
total = len(feedback_docs)
|
||||
for doc in feedback_docs:
|
||||
data = doc.to_dict()
|
||||
avg_overall += data.get("overall_rating", 0)
|
||||
avg_ease += data.get("ease_of_use_rating", 0)
|
||||
avg_accuracy += data.get("lyrics_accuracy_rating", 0)
|
||||
avg_correction += data.get("correction_experience_rating", 0)
|
||||
|
||||
avg_overall = round(avg_overall / total, 2)
|
||||
avg_ease = round(avg_ease / total, 2)
|
||||
avg_accuracy = round(avg_accuracy / total, 2)
|
||||
avg_correction = round(avg_correction / total, 2)
|
||||
|
||||
return {
|
||||
"total_beta_testers": total_beta_testers,
|
||||
"active_testers": active_testers,
|
||||
"pending_feedback": pending_feedback,
|
||||
"completed_feedback": completed_feedback,
|
||||
"total_feedback_submissions": len(feedback_docs),
|
||||
"average_ratings": {
|
||||
"overall": avg_overall,
|
||||
"ease_of_use": avg_ease,
|
||||
"lyrics_accuracy": avg_accuracy,
|
||||
"correction_experience": avg_correction,
|
||||
},
|
||||
}
|
||||
@@ -40,6 +40,11 @@ class Settings(BaseSettings):
|
||||
# Processing
|
||||
max_concurrent_jobs: int = int(os.getenv("MAX_CONCURRENT_JOBS", "5"))
|
||||
job_timeout_seconds: int = int(os.getenv("JOB_TIMEOUT_SECONDS", "3600"))
|
||||
|
||||
# Agentic AI Correction (for lyrics correction via LLM)
|
||||
# When enabled, uses Gemini via Vertex AI for intelligent lyrics correction
|
||||
use_agentic_ai: bool = os.getenv("USE_AGENTIC_AI", "true").lower() in ("true", "1", "yes")
|
||||
agentic_ai_model: str = os.getenv("AGENTIC_AI_MODEL", "vertexai/gemini-3-flash-preview")
|
||||
|
||||
# Cloud Tasks (for scalable worker coordination)
|
||||
# When enabled, workers are triggered via Cloud Tasks for guaranteed delivery
|
||||
|
||||
+2
-1
@@ -7,7 +7,7 @@ from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from backend.config import settings
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users
|
||||
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
|
||||
from backend.services.structured_logging import setup_structured_logging
|
||||
|
||||
@@ -109,6 +109,7 @@ app.include_router(review.router, prefix="/api") # Review UI compatibility endp
|
||||
app.include_router(auth.router, prefix="/api") # OAuth credential management
|
||||
app.include_router(audio_search.router, prefix="/api") # Audio search (artist+title mode)
|
||||
app.include_router(themes.router, prefix="/api") # Theme selection for styles
|
||||
app.include_router(users.router, prefix="/api") # User auth, credits, and Stripe webhooks
|
||||
|
||||
|
||||
@app.get("/")
|
||||
|
||||
@@ -0,0 +1,254 @@
|
||||
"""
|
||||
User model for authentication and credits.
|
||||
|
||||
Supports:
|
||||
- Magic link authentication (email-based)
|
||||
- Credit system for karaoke generation
|
||||
- Role-based access control (user/admin)
|
||||
- Stripe integration for payments
|
||||
- Beta tester program with feedback collection
|
||||
"""
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from typing import Optional, List
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class UserRole(str, Enum):
|
||||
"""User roles for access control."""
|
||||
USER = "user"
|
||||
ADMIN = "admin"
|
||||
|
||||
|
||||
class BetaTesterStatus(str, Enum):
|
||||
"""Status of beta tester participation."""
|
||||
ACTIVE = "active" # Enrolled, free credits available
|
||||
PENDING_FEEDBACK = "pending_feedback" # Job completed, awaiting feedback
|
||||
COMPLETED = "completed" # Feedback submitted
|
||||
EXPIRED = "expired" # 24hr deadline passed without feedback
|
||||
|
||||
|
||||
class CreditTransaction(BaseModel):
|
||||
"""Record of a credit transaction."""
|
||||
id: str
|
||||
amount: int # Positive = add, negative = deduct
|
||||
reason: str # e.g., "purchase", "refund", "admin_grant", "job_creation"
|
||||
job_id: Optional[str] = None
|
||||
stripe_session_id: Optional[str] = None
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
created_by: Optional[str] = None # Admin email if granted by admin
|
||||
|
||||
|
||||
class User(BaseModel):
|
||||
"""
|
||||
User model stored in Firestore.
|
||||
|
||||
Users are identified by email address. Authentication is via magic links
|
||||
(no passwords). Credits are consumed when creating karaoke jobs.
|
||||
"""
|
||||
email: str # Primary identifier
|
||||
role: UserRole = UserRole.USER
|
||||
credits: int = 0
|
||||
|
||||
# Stripe integration
|
||||
stripe_customer_id: Optional[str] = None
|
||||
|
||||
# Account state
|
||||
is_active: bool = True
|
||||
email_verified: bool = False
|
||||
|
||||
# Timestamps
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
updated_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
last_login_at: Optional[datetime] = None
|
||||
|
||||
# Credit history (last 100 transactions)
|
||||
credit_transactions: List[CreditTransaction] = Field(default_factory=list)
|
||||
|
||||
# Job tracking
|
||||
total_jobs_created: int = 0
|
||||
total_jobs_completed: int = 0
|
||||
|
||||
# Optional profile fields for future use
|
||||
display_name: Optional[str] = None
|
||||
|
||||
# Beta tester program
|
||||
is_beta_tester: bool = False
|
||||
beta_tester_status: Optional[BetaTesterStatus] = None
|
||||
beta_enrolled_at: Optional[datetime] = None
|
||||
beta_promise_text: Optional[str] = None # User's promise statement
|
||||
beta_feedback_due_at: Optional[datetime] = None # 24hr after job completion
|
||||
beta_feedback_email_sent: bool = False
|
||||
|
||||
|
||||
class MagicLinkToken(BaseModel):
|
||||
"""
|
||||
Magic link token for passwordless authentication.
|
||||
|
||||
Tokens are short-lived (15 minutes) and single-use.
|
||||
Stored in Firestore with automatic TTL.
|
||||
"""
|
||||
token: str # Secure random token
|
||||
email: str
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
expires_at: datetime
|
||||
used: bool = False
|
||||
used_at: Optional[datetime] = None
|
||||
ip_address: Optional[str] = None
|
||||
user_agent: Optional[str] = None
|
||||
|
||||
|
||||
class Session(BaseModel):
|
||||
"""
|
||||
User session for authenticated requests.
|
||||
|
||||
Sessions are created after successful magic link verification.
|
||||
Sessions expire after 7 days of inactivity or 30 days absolute.
|
||||
"""
|
||||
token: str # Secure random session token
|
||||
user_email: str
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
expires_at: datetime
|
||||
last_activity_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
ip_address: Optional[str] = None
|
||||
user_agent: Optional[str] = None
|
||||
is_active: bool = True
|
||||
|
||||
|
||||
# Pydantic models for API requests/responses
|
||||
|
||||
class SendMagicLinkRequest(BaseModel):
|
||||
"""Request to send a magic link email."""
|
||||
email: str
|
||||
|
||||
|
||||
class SendMagicLinkResponse(BaseModel):
|
||||
"""Response after sending magic link."""
|
||||
status: str
|
||||
message: str
|
||||
|
||||
|
||||
class VerifyMagicLinkRequest(BaseModel):
|
||||
"""Request to verify a magic link token."""
|
||||
token: str
|
||||
|
||||
|
||||
class VerifyMagicLinkResponse(BaseModel):
|
||||
"""Response after verifying magic link."""
|
||||
status: str
|
||||
session_token: str
|
||||
user: "UserPublic"
|
||||
message: str
|
||||
|
||||
|
||||
class UserPublic(BaseModel):
|
||||
"""Public user information (safe to expose to frontend)."""
|
||||
email: str
|
||||
role: UserRole
|
||||
credits: int
|
||||
display_name: Optional[str] = None
|
||||
total_jobs_created: int = 0
|
||||
total_jobs_completed: int = 0
|
||||
|
||||
|
||||
class AddCreditsRequest(BaseModel):
|
||||
"""Admin request to add credits to a user."""
|
||||
email: str
|
||||
amount: int
|
||||
reason: str = "admin_grant"
|
||||
|
||||
|
||||
class AddCreditsResponse(BaseModel):
|
||||
"""Response after adding credits."""
|
||||
status: str
|
||||
email: str
|
||||
credits_added: int
|
||||
new_balance: int
|
||||
message: str
|
||||
|
||||
|
||||
class UserListResponse(BaseModel):
|
||||
"""Response for listing users (admin only)."""
|
||||
users: List[UserPublic]
|
||||
total: int
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Beta Tester Program Models
|
||||
# ============================================================================
|
||||
|
||||
class BetaTesterFeedback(BaseModel):
|
||||
"""
|
||||
Feedback from a beta tester after using the service.
|
||||
|
||||
Stored in Firestore 'beta_feedback' collection.
|
||||
"""
|
||||
id: str # Unique feedback ID
|
||||
user_email: str
|
||||
job_id: Optional[str] = None # The job they're providing feedback on
|
||||
|
||||
# Ratings (1-5 scale)
|
||||
overall_rating: int = Field(ge=1, le=5)
|
||||
ease_of_use_rating: int = Field(ge=1, le=5)
|
||||
lyrics_accuracy_rating: int = Field(ge=1, le=5)
|
||||
correction_experience_rating: int = Field(ge=1, le=5)
|
||||
|
||||
# Open-ended feedback
|
||||
what_went_well: Optional[str] = None
|
||||
what_could_improve: Optional[str] = None
|
||||
additional_comments: Optional[str] = None
|
||||
|
||||
# Would they recommend / use again?
|
||||
would_recommend: bool = True
|
||||
would_use_again: bool = True
|
||||
|
||||
# Metadata
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
submitted_via: str = "web" # 'web' or 'email'
|
||||
|
||||
|
||||
class BetaTesterEnrollRequest(BaseModel):
|
||||
"""Request to enroll as a beta tester."""
|
||||
email: str
|
||||
promise_text: str = Field(
|
||||
min_length=10,
|
||||
description="User's promise to provide feedback (min 10 chars)"
|
||||
)
|
||||
accept_corrections_work: bool = Field(
|
||||
description="User accepts they may need to review/correct lyrics"
|
||||
)
|
||||
|
||||
|
||||
class BetaTesterEnrollResponse(BaseModel):
|
||||
"""Response after enrolling as beta tester."""
|
||||
status: str
|
||||
message: str
|
||||
credits_granted: int
|
||||
session_token: Optional[str] = None # If new user, provide session
|
||||
|
||||
|
||||
class BetaFeedbackRequest(BaseModel):
|
||||
"""Request to submit beta tester feedback."""
|
||||
job_id: Optional[str] = None
|
||||
|
||||
# Ratings (1-5 scale)
|
||||
overall_rating: int = Field(ge=1, le=5)
|
||||
ease_of_use_rating: int = Field(ge=1, le=5)
|
||||
lyrics_accuracy_rating: int = Field(ge=1, le=5)
|
||||
correction_experience_rating: int = Field(ge=1, le=5)
|
||||
|
||||
# Open-ended feedback
|
||||
what_went_well: Optional[str] = None
|
||||
what_could_improve: Optional[str] = None
|
||||
additional_comments: Optional[str] = None
|
||||
|
||||
# Would they recommend / use again?
|
||||
would_recommend: bool = True
|
||||
would_use_again: bool = True
|
||||
|
||||
|
||||
class BetaFeedbackResponse(BaseModel):
|
||||
"""Response after submitting feedback."""
|
||||
status: str
|
||||
message: str
|
||||
bonus_credits: int = 0 # Bonus credits for great feedback
|
||||
@@ -0,0 +1,824 @@
|
||||
"""
|
||||
Email service for sending transactional emails.
|
||||
|
||||
Supports multiple providers:
|
||||
- SendGrid (recommended for production)
|
||||
- Console logging (for development/testing)
|
||||
|
||||
Future providers can be added:
|
||||
- Mailgun
|
||||
- AWS SES
|
||||
- Postmark
|
||||
"""
|
||||
import html
|
||||
import logging
|
||||
import os
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Optional
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class EmailProvider(ABC):
|
||||
"""Abstract base class for email providers."""
|
||||
|
||||
@abstractmethod
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
"""Send an email. Returns True if successful."""
|
||||
pass
|
||||
|
||||
|
||||
class ConsoleEmailProvider(EmailProvider):
|
||||
"""
|
||||
Development email provider that logs to console.
|
||||
|
||||
Useful for local development and testing.
|
||||
"""
|
||||
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
logger.info("=" * 60)
|
||||
logger.info(f"EMAIL TO: {to_email}")
|
||||
logger.info(f"SUBJECT: {subject}")
|
||||
logger.info("-" * 60)
|
||||
logger.info(text_content or html_content)
|
||||
logger.info("=" * 60)
|
||||
return True
|
||||
|
||||
|
||||
class SendGridEmailProvider(EmailProvider):
|
||||
"""
|
||||
SendGrid email provider for production.
|
||||
|
||||
Requires SENDGRID_API_KEY environment variable.
|
||||
"""
|
||||
|
||||
def __init__(self, api_key: str, from_email: str, from_name: str = "Nomad Karaoke"):
|
||||
self.api_key = api_key
|
||||
self.from_email = from_email
|
||||
self.from_name = from_name
|
||||
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
try:
|
||||
# Import here to avoid requiring sendgrid in all environments
|
||||
from sendgrid import SendGridAPIClient
|
||||
from sendgrid.helpers.mail import Mail, Email, To, Content
|
||||
|
||||
sg = SendGridAPIClient(api_key=self.api_key)
|
||||
|
||||
message = Mail(
|
||||
from_email=Email(self.from_email, self.from_name),
|
||||
to_emails=To(to_email),
|
||||
subject=subject,
|
||||
html_content=Content("text/html", html_content)
|
||||
)
|
||||
|
||||
if text_content:
|
||||
message.add_content(Content("text/plain", text_content))
|
||||
|
||||
response = sg.send(message)
|
||||
|
||||
if response.status_code >= 200 and response.status_code < 300:
|
||||
logger.info(f"Email sent to {to_email} via SendGrid")
|
||||
return True
|
||||
else:
|
||||
logger.error(f"SendGrid returned status {response.status_code}")
|
||||
return False
|
||||
|
||||
except Exception:
|
||||
logger.exception("Failed to send email via SendGrid")
|
||||
return False
|
||||
|
||||
|
||||
class EmailService:
|
||||
"""
|
||||
High-level email service for sending transactional emails.
|
||||
|
||||
Automatically selects the appropriate provider based on configuration.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.settings = get_settings()
|
||||
self.provider = self._get_provider()
|
||||
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
|
||||
self.buy_url = os.getenv("BUY_URL", "https://buy.nomadkaraoke.com")
|
||||
|
||||
def _get_provider(self) -> EmailProvider:
|
||||
"""Get the configured email provider."""
|
||||
sendgrid_api_key = os.getenv("SENDGRID_API_KEY")
|
||||
from_email = os.getenv("EMAIL_FROM", "noreply@nomadkaraoke.com")
|
||||
from_name = os.getenv("EMAIL_FROM_NAME", "Nomad Karaoke")
|
||||
|
||||
if sendgrid_api_key:
|
||||
logger.info("Using SendGrid email provider")
|
||||
return SendGridEmailProvider(sendgrid_api_key, from_email, from_name)
|
||||
else:
|
||||
logger.warning("No email provider configured, using console logging")
|
||||
return ConsoleEmailProvider()
|
||||
|
||||
def send_magic_link(self, email: str, token: str) -> bool:
|
||||
"""
|
||||
Send a magic link email for authentication.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
token: Magic link token
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
magic_link_url = f"{self.frontend_url}/auth/verify?token={token}"
|
||||
|
||||
subject = "Sign in to Nomad Karaoke"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.button:hover {{
|
||||
background-color: #2563eb;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
.warning {{
|
||||
background-color: #fef3c7;
|
||||
border: 1px solid #fcd34d;
|
||||
border-radius: 4px;
|
||||
padding: 12px;
|
||||
margin: 20px 0;
|
||||
font-size: 14px;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Hi there,</p>
|
||||
|
||||
<p>Click the button below to sign in to Nomad Karaoke:</p>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{magic_link_url}" class="button">Sign In</a>
|
||||
</p>
|
||||
|
||||
<div class="warning">
|
||||
⏰ This link expires in 15 minutes and can only be used once.
|
||||
</div>
|
||||
|
||||
<p>If the button doesn't work, copy and paste this link into your browser:</p>
|
||||
<p style="word-break: break-all; font-size: 14px; color: #666;">
|
||||
{magic_link_url}
|
||||
</p>
|
||||
|
||||
<p>If you didn't request this email, you can safely ignore it.</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
<p>This is an automated message, please do not reply.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Sign in to Nomad Karaoke
|
||||
========================
|
||||
|
||||
Click this link to sign in:
|
||||
{magic_link_url}
|
||||
|
||||
This link expires in 15 minutes and can only be used once.
|
||||
|
||||
If you didn't request this email, you can safely ignore it.
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_credits_added(self, email: str, credits: int, total_credits: int) -> bool:
|
||||
"""
|
||||
Send notification when credits are added to account.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Number of credits added
|
||||
total_credits: New total credit balance
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = f"🎉 {credits} credits added to your Nomad Karaoke account"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.credits-box {{
|
||||
background-color: #ecfdf5;
|
||||
border: 2px solid #10b981;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.credits-number {{
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
color: #10b981;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Great news! 🎉</p>
|
||||
|
||||
<p><strong>{credits} credits</strong> have been added to your account.</p>
|
||||
|
||||
<div class="credits-box">
|
||||
<div>Your balance:</div>
|
||||
<div class="credits-number">{total_credits}</div>
|
||||
<div>credits</div>
|
||||
</div>
|
||||
|
||||
<p>Each credit lets you create one professional karaoke video with:</p>
|
||||
<ul>
|
||||
<li>AI-powered vocal/instrumental separation</li>
|
||||
<li>Synchronized lyrics with word-level timing</li>
|
||||
<li>4K video output</li>
|
||||
<li>YouTube upload</li>
|
||||
</ul>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Create Karaoke Now</a>
|
||||
</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
{credits} credits added to your Nomad Karaoke account!
|
||||
|
||||
Your new balance: {total_credits} credits
|
||||
|
||||
Each credit lets you create one professional karaoke video.
|
||||
|
||||
Start creating: {self.frontend_url}
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_welcome_email(self, email: str, credits: int = 0) -> bool:
|
||||
"""
|
||||
Send welcome email to new users.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Initial credit balance (if any)
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Welcome to Nomad Karaoke! 🎤"
|
||||
|
||||
credits_text = f"You have <strong>{credits} credits</strong> to get started!" if credits > 0 else ""
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.feature {{
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
margin: 16px 0;
|
||||
}}
|
||||
.feature-icon {{
|
||||
font-size: 24px;
|
||||
margin-right: 12px;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Welcome to Nomad Karaoke!</p>
|
||||
|
||||
<p>Turn any song into a professional karaoke video in minutes. {credits_text}</p>
|
||||
|
||||
<p><strong>Here's how it works:</strong></p>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">🎵</span>
|
||||
<div>
|
||||
<strong>1. Search for a song</strong><br>
|
||||
Enter the artist and title, and we'll find high-quality audio.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">✨</span>
|
||||
<div>
|
||||
<strong>2. AI does the magic</strong><br>
|
||||
We separate vocals, transcribe lyrics, and sync everything perfectly.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">✏️</span>
|
||||
<div>
|
||||
<strong>3. Review & customize</strong><br>
|
||||
Fine-tune the lyrics if needed, choose your instrumental.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">🎬</span>
|
||||
<div>
|
||||
<strong>4. Get your video</strong><br>
|
||||
Download your 4K karaoke video or upload directly to YouTube.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Get Started</a>
|
||||
</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>Questions? Reply to this email and we'll help you out.</p>
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Welcome to Nomad Karaoke!
|
||||
|
||||
Turn any song into a professional karaoke video in minutes.
|
||||
|
||||
Here's how it works:
|
||||
|
||||
1. Search for a song
|
||||
Enter the artist and title, and we'll find high-quality audio.
|
||||
|
||||
2. AI does the magic
|
||||
We separate vocals, transcribe lyrics, and sync everything perfectly.
|
||||
|
||||
3. Review & customize
|
||||
Fine-tune the lyrics if needed, choose your instrumental.
|
||||
|
||||
4. Get your video
|
||||
Download your 4K karaoke video or upload directly to YouTube.
|
||||
|
||||
Get started: {self.frontend_url}
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_beta_welcome_email(self, email: str, credits: int = 1) -> bool:
|
||||
"""
|
||||
Send welcome email to new beta testers.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Initial free credits granted
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Welcome Beta Tester! Free Karaoke Credits Inside 🎤"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.beta-badge {{
|
||||
display: inline-block;
|
||||
background: linear-gradient(135deg, #8b5cf6, #ec4899);
|
||||
color: white;
|
||||
padding: 4px 12px;
|
||||
border-radius: 20px;
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
margin-left: 8px;
|
||||
}}
|
||||
.credits-box {{
|
||||
background-color: #ecfdf5;
|
||||
border: 2px solid #10b981;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.credits-number {{
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
color: #10b981;
|
||||
}}
|
||||
.reminder {{
|
||||
background-color: #fef3c7;
|
||||
border: 1px solid #fcd34d;
|
||||
border-radius: 8px;
|
||||
padding: 16px;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<span class="logo">🎤 Nomad Karaoke</span>
|
||||
<span class="beta-badge">BETA TESTER</span>
|
||||
</div>
|
||||
|
||||
<p>Thank you for joining our beta program! 🎉</p>
|
||||
|
||||
<p>As promised, here's your free credit to create a karaoke video:</p>
|
||||
|
||||
<div class="credits-box">
|
||||
<div>Your balance:</div>
|
||||
<div class="credits-number">{credits}</div>
|
||||
<div>free credit{'' if credits == 1 else 's'}</div>
|
||||
</div>
|
||||
|
||||
<div class="reminder">
|
||||
<strong>📝 Your Promise:</strong><br>
|
||||
Remember, as a beta tester you agreed to:
|
||||
<ul style="margin: 10px 0; padding-left: 20px;">
|
||||
<li>Review and correct any lyrics timing issues</li>
|
||||
<li>Share your honest feedback after trying the tool</li>
|
||||
</ul>
|
||||
We'll send you a quick feedback form after your job completes!
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Create Your Karaoke Video</a>
|
||||
</p>
|
||||
|
||||
<p>Thanks for helping us make Nomad Karaoke better!</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Welcome Beta Tester!
|
||||
|
||||
Thank you for joining our beta program!
|
||||
|
||||
You've been granted {credits} free credit{'' if credits == 1 else 's'} to create karaoke videos.
|
||||
|
||||
YOUR PROMISE:
|
||||
As a beta tester, you agreed to:
|
||||
- Review and correct any lyrics timing issues
|
||||
- Share your honest feedback after trying the tool
|
||||
|
||||
We'll send you a quick feedback form after your job completes!
|
||||
|
||||
Create your karaoke video: {self.frontend_url}
|
||||
|
||||
Thanks for helping us make Nomad Karaoke better!
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_feedback_request_email(self, email: str, feedback_url: str, job_title: Optional[str] = None) -> bool:
|
||||
"""
|
||||
Send feedback request email to beta testers.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
feedback_url: URL to the feedback form
|
||||
job_title: Optional title of the completed job
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Quick feedback on your karaoke experience? 🎤"
|
||||
|
||||
# Escape job_title to prevent XSS in email clients that render HTML
|
||||
safe_job_title = html.escape(job_title) if job_title else ""
|
||||
job_context = f" for <strong>{safe_job_title}</strong>" if job_title else ""
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.feedback-box {{
|
||||
background: linear-gradient(135deg, #f0f9ff, #e0f2fe);
|
||||
border: 2px solid #3b82f6;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.stars {{
|
||||
font-size: 36px;
|
||||
margin: 10px 0;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.time-note {{
|
||||
background-color: #fef3c7;
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
margin: 20px 0;
|
||||
font-size: 14px;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Hi there!</p>
|
||||
|
||||
<p>Hope you enjoyed creating your karaoke video{job_context}! As a beta tester, your feedback is super valuable to us.</p>
|
||||
|
||||
<div class="feedback-box">
|
||||
<div class="stars">⭐⭐⭐⭐⭐</div>
|
||||
<p><strong>How was your experience?</strong></p>
|
||||
<p>Quick 2-minute survey - we'd love to hear your thoughts!</p>
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{feedback_url}" class="button">Share Your Feedback</a>
|
||||
</p>
|
||||
|
||||
<div class="time-note">
|
||||
⏱️ This takes less than 2 minutes and helps us improve the tool for everyone!
|
||||
</div>
|
||||
|
||||
<p>Specifically, we'd love to know:</p>
|
||||
<ul>
|
||||
<li>How easy was it to use?</li>
|
||||
<li>Were the lyrics accurate?</li>
|
||||
<li>How was the correction experience?</li>
|
||||
<li>What could we improve?</li>
|
||||
</ul>
|
||||
|
||||
<p>Thanks for being part of making Nomad Karaoke better! 🙏</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Quick feedback on your karaoke experience?
|
||||
|
||||
Hi there!
|
||||
|
||||
Hope you enjoyed creating your karaoke video{' for ' + safe_job_title if safe_job_title else ''}!
|
||||
|
||||
As a beta tester, your feedback is super valuable to us.
|
||||
|
||||
Share your feedback (2-minute survey): {feedback_url}
|
||||
|
||||
We'd love to know:
|
||||
- How easy was it to use?
|
||||
- Were the lyrics accurate?
|
||||
- How was the correction experience?
|
||||
- What could we improve?
|
||||
|
||||
Thanks for being part of making Nomad Karaoke better!
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def _get_year(self) -> int:
|
||||
"""Get current year for copyright notices."""
|
||||
from datetime import datetime
|
||||
return datetime.now().year
|
||||
|
||||
|
||||
# Global instance
|
||||
_email_service = None
|
||||
|
||||
|
||||
def get_email_service() -> EmailService:
|
||||
"""Get the global email service instance."""
|
||||
global _email_service
|
||||
if _email_service is None:
|
||||
_email_service = EmailService()
|
||||
return _email_service
|
||||
@@ -136,8 +136,10 @@ class GoogleDriveService:
|
||||
logger.info(f"Looking for folder '{folder_name}' in parent {parent_id}")
|
||||
|
||||
# Search for existing folder
|
||||
# Escape single quotes in folder name for Google Drive API query syntax
|
||||
escaped_folder_name = folder_name.replace("'", "\\'")
|
||||
query = (
|
||||
f"name='{folder_name}' and '{parent_id}' in parents "
|
||||
f"name='{escaped_folder_name}' and '{parent_id}' in parents "
|
||||
f"and mimeType='application/vnd.google-apps.folder' and trashed=false"
|
||||
)
|
||||
results = self.service.files().list(q=query, fields="files(id, name)").execute()
|
||||
@@ -203,8 +205,10 @@ class GoogleDriveService:
|
||||
|
||||
# Check for existing file with same name
|
||||
if replace_existing:
|
||||
# Escape single quotes in filename for Google Drive API query syntax
|
||||
escaped_filename = filename.replace("'", "\\'")
|
||||
query = (
|
||||
f"name='{filename}' and '{parent_id}' in parents and trashed=false"
|
||||
f"name='{escaped_filename}' and '{parent_id}' in parents and trashed=false"
|
||||
)
|
||||
results = self.service.files().list(q=query, fields="files(id)").execute()
|
||||
for existing_file in results.get("files", []):
|
||||
|
||||
@@ -0,0 +1,274 @@
|
||||
"""
|
||||
Stripe service for payment processing.
|
||||
|
||||
Handles:
|
||||
- Creating checkout sessions for credit purchases
|
||||
- Processing webhook events
|
||||
- Managing customer records
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
from typing import Optional, Dict, Any, Tuple
|
||||
|
||||
import stripe
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Credit packages available for purchase
|
||||
CREDIT_PACKAGES = {
|
||||
"1_credit": {
|
||||
"credits": 1,
|
||||
"price_cents": 500, # $5.00
|
||||
"name": "1 Karaoke Credit",
|
||||
"description": "Create 1 professional karaoke video",
|
||||
},
|
||||
"3_credits": {
|
||||
"credits": 3,
|
||||
"price_cents": 1200, # $12.00 (20% discount)
|
||||
"name": "3 Karaoke Credits",
|
||||
"description": "Create 3 professional karaoke videos (Save 20%)",
|
||||
},
|
||||
"5_credits": {
|
||||
"credits": 5,
|
||||
"price_cents": 1750, # $17.50 (30% discount)
|
||||
"name": "5 Karaoke Credits",
|
||||
"description": "Create 5 professional karaoke videos (Save 30%)",
|
||||
},
|
||||
"10_credits": {
|
||||
"credits": 10,
|
||||
"price_cents": 3000, # $30.00 (40% discount)
|
||||
"name": "10 Karaoke Credits",
|
||||
"description": "Create 10 professional karaoke videos (Save 40%)",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class StripeService:
|
||||
"""Service for Stripe payment processing."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize Stripe with API key."""
|
||||
self.settings = get_settings()
|
||||
self.secret_key = os.getenv("STRIPE_SECRET_KEY")
|
||||
self.webhook_secret = os.getenv("STRIPE_WEBHOOK_SECRET")
|
||||
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
|
||||
self.buy_url = os.getenv("BUY_URL", "https://buy.nomadkaraoke.com")
|
||||
|
||||
if self.secret_key:
|
||||
stripe.api_key = self.secret_key
|
||||
logger.info("Stripe initialized with API key")
|
||||
else:
|
||||
logger.warning("STRIPE_SECRET_KEY not set - payments disabled")
|
||||
|
||||
def is_configured(self) -> bool:
|
||||
"""Check if Stripe is properly configured."""
|
||||
return bool(self.secret_key)
|
||||
|
||||
def get_credit_packages(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get available credit packages."""
|
||||
return CREDIT_PACKAGES
|
||||
|
||||
def create_checkout_session(
|
||||
self,
|
||||
package_id: str,
|
||||
user_email: str,
|
||||
success_url: Optional[str] = None,
|
||||
cancel_url: Optional[str] = None,
|
||||
) -> Tuple[bool, Optional[str], str]:
|
||||
"""
|
||||
Create a Stripe Checkout session for purchasing credits.
|
||||
|
||||
Args:
|
||||
package_id: ID of the credit package to purchase
|
||||
user_email: Email of the purchasing user
|
||||
success_url: URL to redirect to on success (optional)
|
||||
cancel_url: URL to redirect to on cancel (optional)
|
||||
|
||||
Returns:
|
||||
(success, checkout_url, message)
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return False, None, "Payment processing is not configured"
|
||||
|
||||
package = CREDIT_PACKAGES.get(package_id)
|
||||
if not package:
|
||||
return False, None, f"Invalid package: {package_id}"
|
||||
|
||||
try:
|
||||
# Default URLs
|
||||
if not success_url:
|
||||
success_url = f"{self.frontend_url}/payment/success?session_id={{CHECKOUT_SESSION_ID}}"
|
||||
if not cancel_url:
|
||||
cancel_url = f"{self.buy_url}?cancelled=true"
|
||||
|
||||
# Create checkout session
|
||||
session = stripe.checkout.Session.create(
|
||||
payment_method_types=['card'],
|
||||
line_items=[{
|
||||
'price_data': {
|
||||
'currency': 'usd',
|
||||
'product_data': {
|
||||
'name': package['name'],
|
||||
'description': package['description'],
|
||||
},
|
||||
'unit_amount': package['price_cents'],
|
||||
},
|
||||
'quantity': 1,
|
||||
}],
|
||||
mode='payment',
|
||||
success_url=success_url,
|
||||
cancel_url=cancel_url,
|
||||
customer_email=user_email,
|
||||
metadata={
|
||||
'package_id': package_id,
|
||||
'credits': str(package['credits']),
|
||||
'user_email': user_email,
|
||||
},
|
||||
# Allow promotion codes
|
||||
allow_promotion_codes=True,
|
||||
)
|
||||
|
||||
logger.info(f"Created checkout session {session.id} for {user_email}, package {package_id}")
|
||||
return True, session.url, "Checkout session created"
|
||||
|
||||
except stripe.error.StripeError as e:
|
||||
logger.error(f"Stripe error creating checkout session: {e}")
|
||||
return False, None, f"Payment error: {str(e)}"
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating checkout session: {e}")
|
||||
return False, None, "Failed to create checkout session"
|
||||
|
||||
def verify_webhook_signature(self, payload: bytes, signature: str) -> Tuple[bool, Optional[Dict], str]:
|
||||
"""
|
||||
Verify a Stripe webhook signature.
|
||||
|
||||
Args:
|
||||
payload: Raw request body
|
||||
signature: Stripe-Signature header value
|
||||
|
||||
Returns:
|
||||
(valid, event_data, message)
|
||||
"""
|
||||
if not self.webhook_secret:
|
||||
logger.error("STRIPE_WEBHOOK_SECRET not configured")
|
||||
return False, None, "Webhook secret not configured"
|
||||
|
||||
try:
|
||||
event = stripe.Webhook.construct_event(
|
||||
payload, signature, self.webhook_secret
|
||||
)
|
||||
return True, event, "Webhook verified"
|
||||
except stripe.error.SignatureVerificationError as e:
|
||||
logger.error(f"Invalid webhook signature: {e}")
|
||||
return False, None, "Invalid signature"
|
||||
except Exception as e:
|
||||
logger.error(f"Error verifying webhook: {e}")
|
||||
return False, None, str(e)
|
||||
|
||||
def handle_checkout_completed(self, session: Dict) -> Tuple[bool, Optional[str], int, str]:
|
||||
"""
|
||||
Handle a completed checkout session.
|
||||
|
||||
Args:
|
||||
session: Stripe checkout session object
|
||||
|
||||
Returns:
|
||||
(success, user_email, credits_to_add, message)
|
||||
"""
|
||||
try:
|
||||
metadata = session.get('metadata', {})
|
||||
user_email = metadata.get('user_email') or session.get('customer_email')
|
||||
try:
|
||||
credits = int(metadata.get('credits', 0))
|
||||
except (ValueError, TypeError):
|
||||
logger.error(f"Invalid credits metadata in session {session.get('id')}: {metadata.get('credits')}")
|
||||
return False, user_email, 0, "Invalid credit amount in session metadata"
|
||||
package_id = metadata.get('package_id')
|
||||
|
||||
if not user_email:
|
||||
logger.error(f"No user email in checkout session {session.get('id')}")
|
||||
return False, None, 0, "No user email found"
|
||||
|
||||
if credits <= 0:
|
||||
logger.error(f"Invalid credits in checkout session {session.get('id')}")
|
||||
return False, user_email, 0, "Invalid credit amount"
|
||||
|
||||
logger.info(
|
||||
f"Checkout completed: {user_email} purchased {credits} credits "
|
||||
f"(package: {package_id}, session: {session.get('id')})"
|
||||
)
|
||||
|
||||
return True, user_email, credits, f"Successfully purchased {credits} credits"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error handling checkout completed: {e}")
|
||||
return False, None, 0, str(e)
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get a checkout session by ID.
|
||||
|
||||
Args:
|
||||
session_id: Stripe checkout session ID
|
||||
|
||||
Returns:
|
||||
Session data or None
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return None
|
||||
|
||||
try:
|
||||
session = stripe.checkout.Session.retrieve(session_id)
|
||||
return dict(session)
|
||||
except Exception as e:
|
||||
logger.error(f"Error retrieving session {session_id}: {e}")
|
||||
return None
|
||||
|
||||
def create_customer(self, email: str, name: Optional[str] = None) -> Optional[str]:
|
||||
"""
|
||||
Create or get a Stripe customer for a user.
|
||||
|
||||
Args:
|
||||
email: User's email
|
||||
name: User's display name (optional)
|
||||
|
||||
Returns:
|
||||
Stripe customer ID or None
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return None
|
||||
|
||||
try:
|
||||
# Check if customer already exists
|
||||
customers = stripe.Customer.list(email=email, limit=1)
|
||||
if customers.data:
|
||||
return customers.data[0].id
|
||||
|
||||
# Create new customer
|
||||
customer = stripe.Customer.create(
|
||||
email=email,
|
||||
name=name,
|
||||
metadata={'source': 'nomad_karaoke'},
|
||||
)
|
||||
logger.info(f"Created Stripe customer {customer.id} for {email}")
|
||||
return customer.id
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating Stripe customer: {e}")
|
||||
return None
|
||||
|
||||
|
||||
# Global instance
|
||||
_stripe_service = None
|
||||
|
||||
|
||||
def get_stripe_service() -> StripeService:
|
||||
"""Get the global Stripe service instance."""
|
||||
global _stripe_service
|
||||
if _stripe_service is None:
|
||||
_stripe_service = StripeService()
|
||||
return _stripe_service
|
||||
@@ -0,0 +1,624 @@
|
||||
"""
|
||||
User service for authentication, credits, and user management.
|
||||
|
||||
Handles:
|
||||
- Magic link authentication (send, verify)
|
||||
- Session management
|
||||
- Credit operations (add, deduct, refund)
|
||||
- User CRUD operations
|
||||
"""
|
||||
import logging
|
||||
import secrets
|
||||
import uuid
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Optional, List, Tuple
|
||||
|
||||
from google.cloud import firestore
|
||||
from google.cloud.firestore_v1 import FieldFilter
|
||||
from google.cloud.firestore_v1 import Increment
|
||||
|
||||
from backend.config import get_settings
|
||||
from backend.models.user import (
|
||||
User,
|
||||
UserRole,
|
||||
UserPublic,
|
||||
MagicLinkToken,
|
||||
Session,
|
||||
CreditTransaction,
|
||||
)
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Collection names
|
||||
USERS_COLLECTION = "users"
|
||||
MAGIC_LINKS_COLLECTION = "magic_links"
|
||||
SESSIONS_COLLECTION = "sessions"
|
||||
PROCESSED_STRIPE_SESSIONS_COLLECTION = "processed_stripe_sessions"
|
||||
|
||||
# Token/session configuration
|
||||
MAGIC_LINK_EXPIRY_MINUTES = 15
|
||||
SESSION_EXPIRY_DAYS = 7
|
||||
SESSION_ABSOLUTE_EXPIRY_DAYS = 30
|
||||
MAX_CREDIT_TRANSACTIONS = 100 # Keep last N transactions
|
||||
|
||||
|
||||
class UserService:
|
||||
"""Service for user management and authentication."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize user service with Firestore client."""
|
||||
self.settings = get_settings()
|
||||
self.db = firestore.Client(project=self.settings.google_cloud_project)
|
||||
|
||||
# =========================================================================
|
||||
# User CRUD Operations
|
||||
# =========================================================================
|
||||
|
||||
def get_user(self, email: str) -> Optional[User]:
|
||||
"""Get a user by email."""
|
||||
try:
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc = doc_ref.get()
|
||||
|
||||
if not doc.exists:
|
||||
return None
|
||||
|
||||
return User(**doc.to_dict())
|
||||
except Exception:
|
||||
logger.exception(f"Error getting user {email}")
|
||||
return None
|
||||
|
||||
def get_or_create_user(self, email: str) -> User:
|
||||
"""Get existing user or create a new one."""
|
||||
email = email.lower()
|
||||
user = self.get_user(email)
|
||||
|
||||
if user:
|
||||
return user
|
||||
|
||||
# Create new user
|
||||
user = User(email=email)
|
||||
self._save_user(user)
|
||||
logger.info(f"Created new user: {email}")
|
||||
return user
|
||||
|
||||
def _save_user(self, user: User) -> None:
|
||||
"""Save user to Firestore."""
|
||||
try:
|
||||
user.updated_at = datetime.utcnow()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(user.email.lower())
|
||||
doc_ref.set(user.model_dump(mode='json'))
|
||||
except Exception:
|
||||
logger.exception(f"Error saving user {user.email}")
|
||||
raise
|
||||
|
||||
def update_user(self, email: str, **updates) -> Optional[User]:
|
||||
"""Update user fields."""
|
||||
try:
|
||||
updates['updated_at'] = datetime.utcnow()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc_ref.update(updates)
|
||||
return self.get_user(email)
|
||||
except Exception:
|
||||
logger.exception(f"Error updating user {email}")
|
||||
return None
|
||||
|
||||
def list_users(self, limit: int = 100, include_inactive: bool = False) -> List[User]:
|
||||
"""List all users (admin only)."""
|
||||
try:
|
||||
query = self.db.collection(USERS_COLLECTION)
|
||||
|
||||
if not include_inactive:
|
||||
query = query.where(filter=FieldFilter('is_active', '==', True))
|
||||
|
||||
query = query.order_by('created_at', direction=firestore.Query.DESCENDING)
|
||||
query = query.limit(limit)
|
||||
|
||||
docs = query.stream()
|
||||
return [User(**doc.to_dict()) for doc in docs]
|
||||
except Exception:
|
||||
logger.exception("Error listing users")
|
||||
return []
|
||||
|
||||
def get_user_public(self, email: str) -> Optional[UserPublic]:
|
||||
"""Get public user info."""
|
||||
user = self.get_user(email)
|
||||
if not user:
|
||||
return None
|
||||
|
||||
return UserPublic(
|
||||
email=user.email,
|
||||
role=user.role,
|
||||
credits=user.credits,
|
||||
display_name=user.display_name,
|
||||
total_jobs_created=user.total_jobs_created,
|
||||
total_jobs_completed=user.total_jobs_completed,
|
||||
)
|
||||
|
||||
# =========================================================================
|
||||
# Magic Link Authentication
|
||||
# =========================================================================
|
||||
|
||||
def create_magic_link(
|
||||
self,
|
||||
email: str,
|
||||
ip_address: Optional[str] = None,
|
||||
user_agent: Optional[str] = None
|
||||
) -> MagicLinkToken:
|
||||
"""
|
||||
Create a magic link token for email authentication.
|
||||
|
||||
Returns the token object. The actual sending of the email
|
||||
should be handled by the caller (or an email service).
|
||||
"""
|
||||
email = email.lower()
|
||||
|
||||
# Ensure user exists
|
||||
self.get_or_create_user(email)
|
||||
|
||||
# Generate secure token
|
||||
token = secrets.token_urlsafe(32)
|
||||
|
||||
magic_link = MagicLinkToken(
|
||||
token=token,
|
||||
email=email,
|
||||
expires_at=datetime.utcnow() + timedelta(minutes=MAGIC_LINK_EXPIRY_MINUTES),
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent,
|
||||
)
|
||||
|
||||
# Save to Firestore
|
||||
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
|
||||
doc_ref.set(magic_link.model_dump(mode='json'))
|
||||
|
||||
logger.info(f"Created magic link for {email}")
|
||||
return magic_link
|
||||
|
||||
def verify_magic_link(self, token: str) -> Tuple[bool, Optional[User], str]:
|
||||
"""
|
||||
Verify a magic link token using a Firestore transaction to prevent race conditions.
|
||||
|
||||
Returns:
|
||||
(success, user, message)
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
|
||||
|
||||
@firestore.transactional
|
||||
def verify_in_transaction(transaction):
|
||||
"""Atomically verify and mark magic link as used."""
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
|
||||
if not doc.exists:
|
||||
return False, None, "Invalid or expired link"
|
||||
|
||||
magic_link = MagicLinkToken(**doc.to_dict())
|
||||
|
||||
# Check if already used
|
||||
if magic_link.used:
|
||||
return False, None, "This link has already been used"
|
||||
|
||||
# Check expiry
|
||||
if datetime.utcnow() > magic_link.expires_at:
|
||||
return False, None, "This link has expired"
|
||||
|
||||
# Mark as used atomically within transaction
|
||||
transaction.update(doc_ref, {
|
||||
'used': True,
|
||||
'used_at': datetime.utcnow()
|
||||
})
|
||||
|
||||
return True, magic_link.email, "Success"
|
||||
|
||||
# Execute the transaction
|
||||
transaction = self.db.transaction()
|
||||
success, email_or_error, message = verify_in_transaction(transaction)
|
||||
|
||||
if not success:
|
||||
return False, None, message
|
||||
|
||||
# Get user and mark email as verified (outside transaction)
|
||||
user = self.get_user(email_or_error)
|
||||
if user:
|
||||
self.update_user(
|
||||
email_or_error,
|
||||
email_verified=True,
|
||||
last_login_at=datetime.utcnow()
|
||||
)
|
||||
user = self.get_user(email_or_error) # Refresh
|
||||
|
||||
logger.info(f"Magic link verified for {email_or_error}")
|
||||
return True, user, "Successfully authenticated"
|
||||
|
||||
except Exception:
|
||||
logger.exception("Error verifying magic link")
|
||||
return False, None, "An error occurred during verification"
|
||||
|
||||
# =========================================================================
|
||||
# Session Management
|
||||
# =========================================================================
|
||||
|
||||
def create_session(
|
||||
self,
|
||||
user_email: str,
|
||||
ip_address: Optional[str] = None,
|
||||
user_agent: Optional[str] = None
|
||||
) -> Session:
|
||||
"""Create a new session for an authenticated user."""
|
||||
token = secrets.token_urlsafe(32)
|
||||
|
||||
session = Session(
|
||||
token=token,
|
||||
user_email=user_email.lower(),
|
||||
expires_at=datetime.utcnow() + timedelta(days=SESSION_ABSOLUTE_EXPIRY_DAYS),
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent,
|
||||
)
|
||||
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc_ref.set(session.model_dump(mode='json'))
|
||||
|
||||
logger.info(f"Created session for {user_email}")
|
||||
return session
|
||||
|
||||
def validate_session(self, token: str) -> Tuple[bool, Optional[User], str]:
|
||||
"""
|
||||
Validate a session token.
|
||||
|
||||
Returns:
|
||||
(valid, user, message)
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc = doc_ref.get()
|
||||
|
||||
if not doc.exists:
|
||||
return False, None, "Invalid session"
|
||||
|
||||
session = Session(**doc.to_dict())
|
||||
|
||||
# Check if active
|
||||
if not session.is_active:
|
||||
return False, None, "Session has been revoked"
|
||||
|
||||
# Check expiry
|
||||
if datetime.utcnow() > session.expires_at:
|
||||
return False, None, "Session has expired"
|
||||
|
||||
# Check inactivity (7 days)
|
||||
inactivity_limit = datetime.utcnow() - timedelta(days=SESSION_EXPIRY_DAYS)
|
||||
if session.last_activity_at < inactivity_limit:
|
||||
return False, None, "Session expired due to inactivity"
|
||||
|
||||
# Update last activity
|
||||
doc_ref.update({'last_activity_at': datetime.utcnow()})
|
||||
|
||||
# Get user
|
||||
user = self.get_user(session.user_email)
|
||||
if not user:
|
||||
return False, None, "User not found"
|
||||
|
||||
if not user.is_active:
|
||||
return False, None, "User account is disabled"
|
||||
|
||||
return True, user, "Valid session"
|
||||
|
||||
except Exception:
|
||||
logger.exception("Error validating session")
|
||||
return False, None, "An error occurred during validation"
|
||||
|
||||
def revoke_session(self, token: str) -> bool:
|
||||
"""Revoke a session (logout)."""
|
||||
try:
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc_ref.update({'is_active': False})
|
||||
logger.info("Session revoked")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error revoking session")
|
||||
return False
|
||||
|
||||
def revoke_all_sessions(self, user_email: str) -> int:
|
||||
"""Revoke all sessions for a user."""
|
||||
try:
|
||||
query = self.db.collection(SESSIONS_COLLECTION).where(
|
||||
filter=FieldFilter('user_email', '==', user_email.lower())
|
||||
).where(
|
||||
filter=FieldFilter('is_active', '==', True)
|
||||
)
|
||||
|
||||
count = 0
|
||||
for doc in query.stream():
|
||||
doc.reference.update({'is_active': False})
|
||||
count += 1
|
||||
|
||||
logger.info(f"Revoked {count} sessions for {user_email}")
|
||||
return count
|
||||
except Exception:
|
||||
logger.exception("Error revoking sessions")
|
||||
return 0
|
||||
|
||||
# =========================================================================
|
||||
# Credit Operations
|
||||
# =========================================================================
|
||||
|
||||
def add_credits(
|
||||
self,
|
||||
email: str,
|
||||
amount: int,
|
||||
reason: str,
|
||||
job_id: Optional[str] = None,
|
||||
stripe_session_id: Optional[str] = None,
|
||||
admin_email: Optional[str] = None
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Add credits to a user account.
|
||||
|
||||
Returns:
|
||||
(success, new_balance, message)
|
||||
"""
|
||||
try:
|
||||
# For Stripe sessions, atomically check and mark as processed
|
||||
# Uses create() which fails if document exists - ensures idempotency
|
||||
if stripe_session_id:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
try:
|
||||
doc_ref.create({
|
||||
'stripe_session_id': stripe_session_id,
|
||||
'email': email.lower(),
|
||||
'amount': amount,
|
||||
'processed_at': datetime.utcnow()
|
||||
})
|
||||
except Exception:
|
||||
# Document already exists - this session was already processed
|
||||
logger.info(f"Stripe session {stripe_session_id} already processed (idempotent skip)")
|
||||
return False, 0, "Session already processed"
|
||||
|
||||
email = email.lower()
|
||||
user = self.get_or_create_user(email)
|
||||
|
||||
# Create transaction record
|
||||
transaction = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=amount,
|
||||
reason=reason,
|
||||
job_id=job_id,
|
||||
stripe_session_id=stripe_session_id,
|
||||
created_by=admin_email,
|
||||
)
|
||||
|
||||
# Add to transaction history (keep last N)
|
||||
transactions = user.credit_transactions[-MAX_CREDIT_TRANSACTIONS + 1:]
|
||||
transactions.append(transaction)
|
||||
|
||||
# Update user
|
||||
new_balance = user.credits + amount
|
||||
self.update_user(
|
||||
email,
|
||||
credits=new_balance,
|
||||
credit_transactions=[t.model_dump(mode='json') for t in transactions]
|
||||
)
|
||||
|
||||
logger.info(f"Added {amount} credits to {email} ({reason}). New balance: {new_balance}")
|
||||
return True, new_balance, f"Added {amount} credits"
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error adding credits to {email}")
|
||||
return False, 0, f"Failed to add credits: {e}"
|
||||
|
||||
def deduct_credit(
|
||||
self,
|
||||
email: str,
|
||||
job_id: str,
|
||||
reason: str = "job_creation"
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Deduct one credit from a user account.
|
||||
|
||||
Uses Firestore transaction to prevent race conditions.
|
||||
|
||||
Returns:
|
||||
(success, remaining_credits, message)
|
||||
"""
|
||||
try:
|
||||
email = email.lower()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email)
|
||||
|
||||
@firestore.transactional
|
||||
def deduct_in_transaction(transaction):
|
||||
"""Atomically check and deduct credit."""
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
|
||||
if not doc.exists:
|
||||
return False, 0, "User not found"
|
||||
|
||||
user_data = doc.to_dict()
|
||||
current_credits = user_data.get('credits', 0)
|
||||
|
||||
if current_credits <= 0:
|
||||
return False, 0, "Insufficient credits"
|
||||
|
||||
# Create transaction record
|
||||
credit_txn = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=-1,
|
||||
reason=reason,
|
||||
job_id=job_id,
|
||||
)
|
||||
|
||||
# Get existing transactions and add new one
|
||||
existing_transactions = user_data.get('credit_transactions', [])
|
||||
# Keep last N-1 transactions to make room for new one
|
||||
transactions = existing_transactions[-(MAX_CREDIT_TRANSACTIONS - 1):]
|
||||
transactions.append(credit_txn.model_dump(mode='json'))
|
||||
|
||||
# Calculate new values
|
||||
new_balance = current_credits - 1
|
||||
total_jobs = user_data.get('total_jobs_created', 0) + 1
|
||||
|
||||
# Update atomically within transaction
|
||||
transaction.update(doc_ref, {
|
||||
'credits': new_balance,
|
||||
'credit_transactions': transactions,
|
||||
'total_jobs_created': total_jobs,
|
||||
'updated_at': datetime.utcnow()
|
||||
})
|
||||
|
||||
return True, new_balance, f"Credit deducted. {new_balance} remaining"
|
||||
|
||||
# Execute the transaction
|
||||
fs_transaction = self.db.transaction()
|
||||
success, new_balance, message = deduct_in_transaction(fs_transaction)
|
||||
|
||||
if success:
|
||||
logger.info(f"Deducted 1 credit from {email} for job {job_id}. Remaining: {new_balance}")
|
||||
|
||||
return success, new_balance, message
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error deducting credit from {email}")
|
||||
return False, 0, f"Failed to deduct credit: {e}"
|
||||
|
||||
def refund_credit(
|
||||
self,
|
||||
email: str,
|
||||
job_id: str,
|
||||
reason: str = "job_failed"
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Refund one credit to a user account (e.g., after job failure).
|
||||
|
||||
Returns:
|
||||
(success, new_balance, message)
|
||||
"""
|
||||
return self.add_credits(email, 1, reason, job_id=job_id)
|
||||
|
||||
def check_credits(self, email: str) -> int:
|
||||
"""Check user's credit balance."""
|
||||
user = self.get_user(email)
|
||||
return user.credits if user else 0
|
||||
|
||||
def has_credits(self, email: str) -> bool:
|
||||
"""Check if user has at least one credit."""
|
||||
return self.check_credits(email) > 0
|
||||
|
||||
def is_stripe_session_processed(self, stripe_session_id: str) -> bool:
|
||||
"""
|
||||
Check if a Stripe session ID has already been processed.
|
||||
|
||||
Used to ensure idempotency of webhook processing - prevents
|
||||
duplicate credit additions if Stripe sends the same webhook twice.
|
||||
|
||||
Uses a dedicated collection for O(1) lookup instead of scanning all users.
|
||||
|
||||
Args:
|
||||
stripe_session_id: The Stripe checkout session ID
|
||||
|
||||
Returns:
|
||||
True if this session was already processed
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
doc = doc_ref.get()
|
||||
if doc.exists:
|
||||
logger.info(f"Stripe session {stripe_session_id} already processed")
|
||||
return True
|
||||
return False
|
||||
except Exception:
|
||||
logger.exception(f"Error checking if stripe session {stripe_session_id} was processed")
|
||||
# On error, return False to allow processing (better to risk duplicate than block)
|
||||
return False
|
||||
|
||||
def _mark_stripe_session_processed(self, stripe_session_id: str, email: str, amount: int) -> None:
|
||||
"""
|
||||
Mark a Stripe session as processed.
|
||||
|
||||
Called after successfully adding credits from a Stripe webhook.
|
||||
|
||||
Args:
|
||||
stripe_session_id: The Stripe checkout session ID
|
||||
email: User email who received the credits
|
||||
amount: Number of credits added
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
doc_ref.set({
|
||||
'stripe_session_id': stripe_session_id,
|
||||
'email': email,
|
||||
'amount': amount,
|
||||
'processed_at': datetime.utcnow()
|
||||
})
|
||||
except Exception:
|
||||
# Log but don't fail - the credits were already added
|
||||
logger.exception(f"Error marking stripe session {stripe_session_id} as processed")
|
||||
|
||||
# =========================================================================
|
||||
# Admin Operations
|
||||
# =========================================================================
|
||||
|
||||
def set_user_role(self, email: str, role: UserRole, admin_email: str) -> bool:
|
||||
"""Set a user's role (admin only)."""
|
||||
try:
|
||||
user = self.get_user(email)
|
||||
if not user:
|
||||
return False
|
||||
|
||||
self.update_user(email, role=role.value)
|
||||
logger.info(f"Admin {admin_email} set role for {email} to {role.value}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error setting user role")
|
||||
return False
|
||||
|
||||
def disable_user(self, email: str, admin_email: str) -> bool:
|
||||
"""Disable a user account (admin only)."""
|
||||
try:
|
||||
self.update_user(email, is_active=False)
|
||||
self.revoke_all_sessions(email)
|
||||
logger.info(f"Admin {admin_email} disabled user {email}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error disabling user")
|
||||
return False
|
||||
|
||||
def enable_user(self, email: str, admin_email: str) -> bool:
|
||||
"""Enable a user account (admin only)."""
|
||||
try:
|
||||
self.update_user(email, is_active=True)
|
||||
logger.info(f"Admin {admin_email} enabled user {email}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error enabling user")
|
||||
return False
|
||||
|
||||
def increment_jobs_completed(self, email: str) -> bool:
|
||||
"""Increment the completed jobs counter for a user using atomic increment."""
|
||||
try:
|
||||
# Check user exists first
|
||||
if not self.get_user(email):
|
||||
return False
|
||||
|
||||
# Use atomic increment to prevent race conditions
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc_ref.update({
|
||||
'total_jobs_completed': Increment(1),
|
||||
'updated_at': datetime.utcnow()
|
||||
})
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception(f"Error incrementing jobs completed for {email}")
|
||||
return False
|
||||
|
||||
|
||||
# Global instance
|
||||
_user_service = None
|
||||
|
||||
|
||||
def get_user_service() -> UserService:
|
||||
"""Get the global user service instance."""
|
||||
global _user_service
|
||||
if _user_service is None:
|
||||
_user_service = UserService()
|
||||
return _user_service
|
||||
@@ -1606,5 +1606,133 @@ class TestIsUrlFunction:
|
||||
assert is_url("") is False
|
||||
|
||||
|
||||
class TestUploadEndpointThemeSupport:
|
||||
"""Test that /jobs/upload endpoint supports theme configuration.
|
||||
|
||||
CRITICAL: These tests verify that the /api/jobs/upload endpoint correctly handles
|
||||
theme_id and color_overrides parameters, ensuring preview videos have themed
|
||||
backgrounds instead of black backgrounds.
|
||||
|
||||
This addresses a bug where:
|
||||
- The frontend sends theme_id and color_overrides when uploading files
|
||||
- But the backend /api/jobs/upload endpoint was ignoring these parameters
|
||||
- Result: Jobs created via file upload had black backgrounds instead of themed ones
|
||||
"""
|
||||
|
||||
def test_upload_endpoint_accepts_theme_id_parameter(self):
|
||||
"""Verify the upload endpoint has theme_id as a form parameter.
|
||||
|
||||
CRITICAL: The frontend sends theme_id when uploading files with a theme.
|
||||
If this parameter is missing from the endpoint, the theme is silently ignored
|
||||
and preview videos will have black backgrounds instead of themed ones.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_theme_id_param = 'theme_id: Optional[str] = Form(' in source_code
|
||||
|
||||
assert has_theme_id_param, (
|
||||
"file_upload.py /jobs/upload endpoint does not have theme_id as a Form parameter. "
|
||||
"The frontend sends theme_id when uploading files with a theme, but the backend "
|
||||
"ignores it. Add: theme_id: Optional[str] = Form(None, description='Theme ID...')"
|
||||
)
|
||||
|
||||
def test_upload_endpoint_accepts_color_overrides_parameter(self):
|
||||
"""Verify the upload endpoint has color_overrides as a form parameter."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_color_overrides_param = 'color_overrides: Optional[str] = Form(' in source_code
|
||||
|
||||
assert has_color_overrides_param, (
|
||||
"file_upload.py /jobs/upload endpoint does not have color_overrides as a Form parameter. "
|
||||
"The frontend sends color_overrides when customizing theme colors."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_calls_prepare_theme_for_job(self):
|
||||
"""Verify the upload endpoint calls _prepare_theme_for_job when theme_id is set.
|
||||
|
||||
CRITICAL: When a job is created via the upload endpoint with a theme_id
|
||||
(and no custom style files), the code must call _prepare_theme_for_job() to set:
|
||||
1. style_params_gcs_path (pointing to the copied style_params.json)
|
||||
2. style_assets (populated with asset mappings)
|
||||
|
||||
Without this, LyricsTranscriber won't have access to the theme's styles
|
||||
and preview videos will have black backgrounds instead of themed ones.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
# The function should be called in upload_and_create_job
|
||||
has_theme_prep_call = '_prepare_theme_for_job(' in source_code
|
||||
|
||||
assert has_theme_prep_call, (
|
||||
"file_upload.py does not call _prepare_theme_for_job(). "
|
||||
"When theme_id is provided to /jobs/upload without custom style files, "
|
||||
"the endpoint MUST call _prepare_theme_for_job() to copy the theme's "
|
||||
"style_params.json to the job folder."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_uses_resolve_cdg_txt_defaults(self):
|
||||
"""Verify the upload endpoint uses _resolve_cdg_txt_defaults for theme-based defaults."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_resolve_call = '_resolve_cdg_txt_defaults(' in source_code
|
||||
|
||||
assert has_resolve_call, (
|
||||
"file_upload.py does not call _resolve_cdg_txt_defaults(). "
|
||||
"When theme_id is set, enable_cdg and enable_txt should default to True."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_has_optional_cdg_txt_params(self):
|
||||
"""Verify enable_cdg and enable_txt are Optional[bool] to support theme defaults.
|
||||
|
||||
CRITICAL: If enable_cdg/enable_txt are bool instead of Optional[bool],
|
||||
they will default to False and override the theme-based defaults.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_optional_cdg = 'enable_cdg: Optional[bool] = Form(' in source_code
|
||||
has_optional_txt = 'enable_txt: Optional[bool] = Form(' in source_code
|
||||
|
||||
assert has_optional_cdg, (
|
||||
"file_upload.py has enable_cdg as bool instead of Optional[bool]. "
|
||||
"This prevents theme-based defaults from working."
|
||||
)
|
||||
assert has_optional_txt, (
|
||||
"file_upload.py has enable_txt as bool instead of Optional[bool]. "
|
||||
"This prevents theme-based defaults from working."
|
||||
)
|
||||
|
||||
def test_job_create_includes_theme_id_and_color_overrides(self):
|
||||
"""Verify JobCreate is called with theme_id and color_overrides."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_theme_id_in_job_create = 'theme_id=theme_id,' in source_code
|
||||
has_color_overrides_in_job_create = 'color_overrides=parsed_color_overrides' in source_code
|
||||
|
||||
assert has_theme_id_in_job_create, (
|
||||
"file_upload.py does not pass theme_id to JobCreate."
|
||||
)
|
||||
assert has_color_overrides_in_job_create, (
|
||||
"file_upload.py does not pass color_overrides to JobCreate."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -39,11 +39,30 @@ from backend.services.metrics import metrics
|
||||
|
||||
# Import from karaoke_gen package
|
||||
from karaoke_gen.lyrics_processor import LyricsProcessor
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _configure_agentic_ai():
|
||||
"""Configure environment variables for agentic AI correction.
|
||||
|
||||
The lyrics_transcriber library reads these directly from environment.
|
||||
This ensures the settings from backend config are available.
|
||||
"""
|
||||
settings = get_settings()
|
||||
|
||||
# Enable agentic AI if configured
|
||||
if settings.use_agentic_ai:
|
||||
os.environ["USE_AGENTIC_AI"] = "1"
|
||||
os.environ["AGENTIC_AI_MODEL"] = settings.agentic_ai_model
|
||||
logger.info(f"Agentic AI enabled with model: {settings.agentic_ai_model}")
|
||||
else:
|
||||
os.environ["USE_AGENTIC_AI"] = "0"
|
||||
logger.info("Agentic AI disabled")
|
||||
|
||||
|
||||
# Loggers to capture for lyrics worker (top-level package loggers only)
|
||||
# Note: We only capture at top-level package loggers to avoid duplicate log entries.
|
||||
# Logs from sub-modules (e.g., lyrics_transcriber.correction.anchor_sequence) will
|
||||
@@ -119,7 +138,10 @@ async def process_lyrics_transcription(job_id: str) -> bool:
|
||||
start_time = time.time()
|
||||
job_manager = JobManager()
|
||||
storage = StorageService()
|
||||
|
||||
|
||||
# Configure agentic AI before any lyrics processing
|
||||
_configure_agentic_ai()
|
||||
|
||||
# Create job logger for remote debugging FIRST
|
||||
job_log = create_job_logger(job_id, "lyrics")
|
||||
|
||||
|
||||
@@ -291,11 +291,16 @@ async def generate_video(job_id: str) -> bool:
|
||||
)
|
||||
|
||||
# Store result metadata in job
|
||||
# NOTE: Must include dropbox_link and gdrive_files from result, since
|
||||
# _handle_native_distribution already saved them but job.state_data
|
||||
# is stale (fetched before distribution ran)
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
'brand_code': result.get('brand_code'),
|
||||
'youtube_url': result.get('youtube_url'),
|
||||
'dropbox_link': result.get('dropbox_link'),
|
||||
'gdrive_files': result.get('gdrive_files'),
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
# API URL for the karaoke backend
|
||||
NEXT_PUBLIC_API_URL=https://api.nomadkaraoke.com
|
||||
@@ -0,0 +1,38 @@
|
||||
# Dependencies
|
||||
node_modules/
|
||||
.pnp
|
||||
.pnp.js
|
||||
|
||||
# Build output
|
||||
.next/
|
||||
out/
|
||||
build/
|
||||
|
||||
# Debug
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
|
||||
# Local env files
|
||||
.env
|
||||
.env.local
|
||||
.env.development.local
|
||||
.env.test.local
|
||||
.env.production.local
|
||||
|
||||
# Vercel
|
||||
.vercel
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# TypeScript
|
||||
*.tsbuildinfo
|
||||
next-env.d.ts
|
||||
@@ -0,0 +1,99 @@
|
||||
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&display=swap');
|
||||
|
||||
@tailwind base;
|
||||
@tailwind components;
|
||||
@tailwind utilities;
|
||||
|
||||
:root {
|
||||
--foreground: #f8fafc;
|
||||
--background: #0f172a;
|
||||
}
|
||||
|
||||
body {
|
||||
color: var(--foreground);
|
||||
background: var(--background);
|
||||
font-family: 'Inter', system-ui, sans-serif;
|
||||
}
|
||||
|
||||
/* Smooth scrolling */
|
||||
html {
|
||||
scroll-behavior: smooth;
|
||||
}
|
||||
|
||||
/* Custom scrollbar for dark theme */
|
||||
::-webkit-scrollbar {
|
||||
width: 8px;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-track {
|
||||
background: #1e293b;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb {
|
||||
background: #475569;
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb:hover {
|
||||
background: #64748b;
|
||||
}
|
||||
|
||||
/* Glow effect for primary buttons */
|
||||
.btn-glow {
|
||||
box-shadow: 0 0 20px rgba(59, 130, 246, 0.5);
|
||||
}
|
||||
|
||||
.btn-glow:hover {
|
||||
box-shadow: 0 0 30px rgba(59, 130, 246, 0.7);
|
||||
}
|
||||
|
||||
/* Gradient text */
|
||||
.gradient-text {
|
||||
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ec4899 100%);
|
||||
-webkit-background-clip: text;
|
||||
-webkit-text-fill-color: transparent;
|
||||
background-clip: text;
|
||||
}
|
||||
|
||||
/* Card hover effect */
|
||||
.card-hover {
|
||||
transition: transform 0.2s ease, box-shadow 0.2s ease;
|
||||
}
|
||||
|
||||
.card-hover:hover {
|
||||
transform: translateY(-4px);
|
||||
box-shadow: 0 20px 40px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
/* Animated background gradient */
|
||||
@keyframes gradient-shift {
|
||||
0% {
|
||||
background-position: 0% 50%;
|
||||
}
|
||||
50% {
|
||||
background-position: 100% 50%;
|
||||
}
|
||||
100% {
|
||||
background-position: 0% 50%;
|
||||
}
|
||||
}
|
||||
|
||||
.animated-gradient {
|
||||
background: linear-gradient(-45deg, #0f172a, #1e293b, #1e3a8a, #312e81);
|
||||
background-size: 400% 400%;
|
||||
animation: gradient-shift 15s ease infinite;
|
||||
}
|
||||
|
||||
/* Price highlight animation */
|
||||
@keyframes pulse-subtle {
|
||||
0%, 100% {
|
||||
opacity: 1;
|
||||
}
|
||||
50% {
|
||||
opacity: 0.8;
|
||||
}
|
||||
}
|
||||
|
||||
.pulse-subtle {
|
||||
animation: pulse-subtle 2s ease-in-out infinite;
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
import type { Metadata } from 'next';
|
||||
import './globals.css';
|
||||
|
||||
export const metadata: Metadata = {
|
||||
title: 'Nomad Karaoke - Turn Any Song Into a Karaoke Video',
|
||||
description: 'Create professional karaoke videos in minutes. AI-powered vocal separation, perfect lyrics sync, and 4K video output.',
|
||||
keywords: ['karaoke', 'karaoke video', 'karaoke maker', 'vocal removal', 'lyrics sync'],
|
||||
authors: [{ name: 'Nomad Karaoke' }],
|
||||
openGraph: {
|
||||
title: 'Nomad Karaoke - Turn Any Song Into a Karaoke Video',
|
||||
description: 'Create professional karaoke videos in minutes with AI.',
|
||||
type: 'website',
|
||||
url: 'https://buy.nomadkaraoke.com',
|
||||
images: [
|
||||
{
|
||||
url: 'https://gen.nomadkaraoke.com/nomad-logo.png',
|
||||
width: 512,
|
||||
height: 512,
|
||||
alt: 'Nomad Karaoke',
|
||||
},
|
||||
],
|
||||
},
|
||||
twitter: {
|
||||
card: 'summary_large_image',
|
||||
title: 'Nomad Karaoke - Turn Any Song Into a Karaoke Video',
|
||||
description: 'Create professional karaoke videos in minutes with AI.',
|
||||
},
|
||||
};
|
||||
|
||||
export default function RootLayout({
|
||||
children,
|
||||
}: {
|
||||
children: React.ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<html lang="en">
|
||||
<head>
|
||||
<link rel="icon" href="https://nomadkaraoke.com/favicon.ico" />
|
||||
</head>
|
||||
<body className="min-h-screen animated-gradient">
|
||||
{children}
|
||||
</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,585 @@
|
||||
'use client';
|
||||
|
||||
import { useState } from 'react';
|
||||
import { Mic2, Music, Sparkles, Video, Youtube, Check, ChevronDown, Mail, Loader2, Gift, MessageSquare, CheckCircle2 } from 'lucide-react';
|
||||
import { enrollBetaTester } from '../lib/api';
|
||||
|
||||
// Credit packages (matching backend)
|
||||
const CREDIT_PACKAGES = [
|
||||
{
|
||||
id: '1_credit',
|
||||
credits: 1,
|
||||
price_cents: 500,
|
||||
name: '1 Credit',
|
||||
description: 'Create 1 karaoke video',
|
||||
popular: false,
|
||||
},
|
||||
{
|
||||
id: '3_credits',
|
||||
credits: 3,
|
||||
price_cents: 1200,
|
||||
name: '3 Credits',
|
||||
description: 'Save 20%',
|
||||
popular: false,
|
||||
},
|
||||
{
|
||||
id: '5_credits',
|
||||
credits: 5,
|
||||
price_cents: 1750,
|
||||
name: '5 Credits',
|
||||
description: 'Save 30%',
|
||||
popular: true,
|
||||
},
|
||||
{
|
||||
id: '10_credits',
|
||||
credits: 10,
|
||||
price_cents: 3000,
|
||||
name: '10 Credits',
|
||||
description: 'Save 40%',
|
||||
popular: false,
|
||||
},
|
||||
];
|
||||
|
||||
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'https://api.nomadkaraoke.com';
|
||||
|
||||
function formatPrice(cents: number): string {
|
||||
return `$${(cents / 100).toFixed(0)}`;
|
||||
}
|
||||
|
||||
function pricePerCredit(priceCents: number, credits: number): string {
|
||||
return `$${(priceCents / 100 / credits).toFixed(2)}`;
|
||||
}
|
||||
|
||||
export default function Home() {
|
||||
const [selectedPackage, setSelectedPackage] = useState(CREDIT_PACKAGES[2]); // Default to 5 credits
|
||||
const [email, setEmail] = useState('');
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
const [error, setError] = useState('');
|
||||
const [expandedFaq, setExpandedFaq] = useState<number | null>(null);
|
||||
|
||||
// Beta tester form state
|
||||
const [showBetaForm, setShowBetaForm] = useState(false);
|
||||
const [betaEmail, setBetaEmail] = useState('');
|
||||
const [betaPromise, setBetaPromise] = useState('');
|
||||
const [betaAcceptCorrections, setBetaAcceptCorrections] = useState(false);
|
||||
const [betaLoading, setBetaLoading] = useState(false);
|
||||
const [betaError, setBetaError] = useState('');
|
||||
const [betaSuccess, setBetaSuccess] = useState(false);
|
||||
|
||||
const handleBetaEnroll = async (e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
setBetaError('');
|
||||
|
||||
if (!betaEmail) {
|
||||
setBetaError('Please enter your email');
|
||||
return;
|
||||
}
|
||||
|
||||
if (!betaAcceptCorrections) {
|
||||
setBetaError('Please accept that you may need to review/correct lyrics');
|
||||
return;
|
||||
}
|
||||
|
||||
if (betaPromise.trim().length < 10) {
|
||||
setBetaError('Please write a sentence confirming your promise to provide feedback');
|
||||
return;
|
||||
}
|
||||
|
||||
setBetaLoading(true);
|
||||
|
||||
try {
|
||||
const response = await enrollBetaTester(betaEmail.toLowerCase(), betaPromise, betaAcceptCorrections);
|
||||
|
||||
// Store the session token
|
||||
if (response.session_token) {
|
||||
localStorage.setItem('nomad_karaoke_token', response.session_token);
|
||||
}
|
||||
|
||||
setBetaSuccess(true);
|
||||
|
||||
// Redirect to the main app after a short delay
|
||||
setTimeout(() => {
|
||||
window.location.href = 'https://gen.nomadkaraoke.com';
|
||||
}, 2000);
|
||||
} catch (err) {
|
||||
setBetaError(err instanceof Error ? err.message : 'Something went wrong');
|
||||
setBetaLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
const handleCheckout = async (e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
setError('');
|
||||
|
||||
if (!email) {
|
||||
setError('Please enter your email');
|
||||
return;
|
||||
}
|
||||
|
||||
setIsLoading(true);
|
||||
|
||||
try {
|
||||
const response = await fetch(`${API_BASE_URL}/api/users/credits/checkout`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
package_id: selectedPackage.id,
|
||||
email: email.toLowerCase(),
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const data = await response.json();
|
||||
throw new Error(data.detail || 'Failed to create checkout');
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
window.location.href = data.checkout_url;
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Something went wrong');
|
||||
setIsLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
const faqs = [
|
||||
{
|
||||
q: 'How does it work?',
|
||||
a: 'Just enter a song (artist + title), and our AI finds the audio, removes the vocals, transcribes the lyrics, and creates a professional karaoke video with perfectly synced lyrics.',
|
||||
},
|
||||
{
|
||||
q: 'What songs can I use?',
|
||||
a: "Most popular songs are available. We search multiple high-quality audio sources to find the best version. If we can't find a song, you won't be charged.",
|
||||
},
|
||||
{
|
||||
q: 'What format is the output?',
|
||||
a: 'You get a high-quality MP4 video file (up to 4K) with the instrumental audio and synchronized lyrics. You can also upload directly to YouTube.',
|
||||
},
|
||||
{
|
||||
q: 'What if something goes wrong?',
|
||||
a: "We offer generous refunds. If the quality isn't what you expected or there's an issue with the output, contact us and we'll refund your credit.",
|
||||
},
|
||||
{
|
||||
q: 'Do credits expire?',
|
||||
a: 'No, your credits never expire. Use them whenever you want.',
|
||||
},
|
||||
];
|
||||
|
||||
return (
|
||||
<div className="min-h-screen">
|
||||
{/* Navigation */}
|
||||
<nav className="fixed top-0 left-0 right-0 z-50 bg-dark-900/80 backdrop-blur-md border-b border-dark-700">
|
||||
<div className="max-w-6xl mx-auto px-4 py-4 flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<Mic2 className="w-8 h-8 text-primary-500" />
|
||||
<span className="text-xl font-bold">Nomad Karaoke</span>
|
||||
</div>
|
||||
<a
|
||||
href="https://gen.nomadkaraoke.com"
|
||||
className="text-sm text-dark-300 hover:text-white transition-colors"
|
||||
>
|
||||
Already have credits? Sign in
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
{/* Hero Section */}
|
||||
<section className="pt-32 pb-16 px-4">
|
||||
<div className="max-w-4xl mx-auto text-center">
|
||||
<h1 className="text-4xl sm:text-5xl md:text-6xl font-extrabold mb-6 leading-tight">
|
||||
Turn Any Song Into a{' '}
|
||||
<span className="gradient-text">Karaoke Video</span>
|
||||
</h1>
|
||||
<p className="text-xl text-dark-300 mb-8 max-w-2xl mx-auto">
|
||||
Professional karaoke videos in minutes. AI-powered vocal removal,
|
||||
perfect lyrics sync, and stunning video output.
|
||||
</p>
|
||||
<a
|
||||
href="#pricing"
|
||||
className="inline-flex items-center gap-2 bg-primary-500 hover:bg-primary-600 text-white font-semibold px-8 py-4 rounded-xl transition-all btn-glow"
|
||||
>
|
||||
Get Started
|
||||
<ChevronDown className="w-5 h-5" />
|
||||
</a>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* How It Works */}
|
||||
<section className="py-16 px-4">
|
||||
<div className="max-w-6xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-12">How It Works</h2>
|
||||
<div className="grid md:grid-cols-4 gap-8">
|
||||
{[
|
||||
{
|
||||
icon: Music,
|
||||
title: 'Search',
|
||||
desc: 'Enter artist & song title',
|
||||
},
|
||||
{
|
||||
icon: Sparkles,
|
||||
title: 'AI Magic',
|
||||
desc: 'We remove vocals & sync lyrics',
|
||||
},
|
||||
{
|
||||
icon: Video,
|
||||
title: 'Review',
|
||||
desc: 'Preview and fine-tune',
|
||||
},
|
||||
{
|
||||
icon: Youtube,
|
||||
title: 'Export',
|
||||
desc: 'Download or upload to YouTube',
|
||||
},
|
||||
].map((step, i) => (
|
||||
<div key={i} className="text-center">
|
||||
<div className="w-16 h-16 bg-primary-500/20 rounded-2xl flex items-center justify-center mx-auto mb-4">
|
||||
<step.icon className="w-8 h-8 text-primary-400" />
|
||||
</div>
|
||||
<h3 className="font-semibold text-lg mb-2">{step.title}</h3>
|
||||
<p className="text-dark-400 text-sm">{step.desc}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Features */}
|
||||
<section className="py-16 px-4 bg-dark-800/50">
|
||||
<div className="max-w-6xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-12">What You Get</h2>
|
||||
<div className="grid md:grid-cols-3 gap-6">
|
||||
{[
|
||||
{
|
||||
title: 'Studio-Quality Separation',
|
||||
desc: 'State-of-the-art AI removes vocals cleanly, keeping the instrumental crisp.',
|
||||
},
|
||||
{
|
||||
title: 'Perfect Lyrics Sync',
|
||||
desc: 'Word-by-word timing that highlights exactly when to sing.',
|
||||
},
|
||||
{
|
||||
title: 'Multiple Instrumental Options',
|
||||
desc: 'Choose from different vocal removal levels to get the sound you want.',
|
||||
},
|
||||
{
|
||||
title: '4K Video Output',
|
||||
desc: 'Stunning video quality that looks great on any screen.',
|
||||
},
|
||||
{
|
||||
title: 'YouTube Integration',
|
||||
desc: 'Upload directly to your YouTube channel with one click.',
|
||||
},
|
||||
{
|
||||
title: 'Edit Before Export',
|
||||
desc: 'Fix any lyrics issues before generating your final video.',
|
||||
},
|
||||
].map((feature, i) => (
|
||||
<div
|
||||
key={i}
|
||||
className="bg-dark-800 border border-dark-700 rounded-xl p-6 card-hover"
|
||||
>
|
||||
<h3 className="font-semibold text-lg mb-2">{feature.title}</h3>
|
||||
<p className="text-dark-400 text-sm">{feature.desc}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Beta Tester Promo */}
|
||||
<section className="py-12 px-4">
|
||||
<div className="max-w-2xl mx-auto">
|
||||
{!showBetaForm && !betaSuccess ? (
|
||||
<div className="bg-gradient-to-r from-purple-900/40 to-pink-900/40 border border-purple-500/30 rounded-2xl p-8 text-center">
|
||||
<div className="inline-flex items-center gap-2 bg-purple-500/20 text-purple-300 px-4 py-2 rounded-full text-sm font-medium mb-4">
|
||||
<Gift className="w-4 h-4" />
|
||||
Beta Tester Program
|
||||
</div>
|
||||
<h3 className="text-2xl font-bold mb-3">Try It Free!</h3>
|
||||
<p className="text-dark-300 mb-6">
|
||||
Help us improve by testing the tool and sharing your feedback.
|
||||
Get a <span className="text-green-400 font-semibold">free credit</span> to create your first karaoke video!
|
||||
</p>
|
||||
<button
|
||||
onClick={() => setShowBetaForm(true)}
|
||||
className="inline-flex items-center gap-2 bg-purple-500 hover:bg-purple-600 text-white font-semibold px-6 py-3 rounded-xl transition-all"
|
||||
>
|
||||
Join Beta Program
|
||||
<MessageSquare className="w-5 h-5" />
|
||||
</button>
|
||||
</div>
|
||||
) : betaSuccess ? (
|
||||
<div className="bg-green-900/30 border border-green-500/30 rounded-2xl p-8 text-center">
|
||||
<CheckCircle2 className="w-16 h-16 text-green-400 mx-auto mb-4" />
|
||||
<h3 className="text-2xl font-bold mb-3 text-green-400">Welcome to the Beta!</h3>
|
||||
<p className="text-dark-300 mb-4">
|
||||
You've got 1 free credit! Redirecting you to create your first karaoke video...
|
||||
</p>
|
||||
<Loader2 className="w-6 h-6 animate-spin mx-auto text-green-400" />
|
||||
</div>
|
||||
) : (
|
||||
<div className="bg-dark-800 border border-purple-500/30 rounded-2xl p-8">
|
||||
<div className="flex items-center justify-between mb-6">
|
||||
<div className="inline-flex items-center gap-2 bg-purple-500/20 text-purple-300 px-4 py-2 rounded-full text-sm font-medium">
|
||||
<Gift className="w-4 h-4" />
|
||||
Beta Tester Signup
|
||||
</div>
|
||||
<button
|
||||
onClick={() => setShowBetaForm(false)}
|
||||
className="text-dark-400 hover:text-white transition-colors text-sm"
|
||||
>
|
||||
Cancel
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<h3 className="text-xl font-bold mb-2">Get 1 Free Credit</h3>
|
||||
<p className="text-dark-400 text-sm mb-6">
|
||||
In exchange for your honest feedback after trying the tool.
|
||||
</p>
|
||||
|
||||
<form onSubmit={handleBetaEnroll} className="space-y-4">
|
||||
<div>
|
||||
<label htmlFor="beta-email" className="block text-sm font-medium text-dark-300 mb-2">
|
||||
Email address
|
||||
</label>
|
||||
<div className="relative">
|
||||
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 w-5 h-5 text-dark-500" />
|
||||
<input
|
||||
type="email"
|
||||
id="beta-email"
|
||||
value={betaEmail}
|
||||
onChange={(e) => setBetaEmail(e.target.value)}
|
||||
placeholder="you@example.com"
|
||||
className="w-full pl-10 pr-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 focus:border-transparent text-white placeholder-dark-500"
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="bg-dark-900/50 border border-dark-700 rounded-lg p-4">
|
||||
<h4 className="font-medium mb-3 text-sm">Your Promise</h4>
|
||||
|
||||
<label className="flex items-start gap-3 cursor-pointer mb-4">
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={betaAcceptCorrections}
|
||||
onChange={(e) => setBetaAcceptCorrections(e.target.checked)}
|
||||
className="mt-1 w-4 h-4 rounded border-dark-600 bg-dark-800 text-purple-500 focus:ring-purple-500 focus:ring-offset-dark-900"
|
||||
/>
|
||||
<span className="text-sm text-dark-300">
|
||||
I understand I may need to review and correct lyrics timing to get the best results
|
||||
</span>
|
||||
</label>
|
||||
|
||||
<div>
|
||||
<label htmlFor="promise" className="block text-sm text-dark-400 mb-2">
|
||||
Write a quick sentence promising to share feedback after you try it:
|
||||
</label>
|
||||
<textarea
|
||||
id="promise"
|
||||
value={betaPromise}
|
||||
onChange={(e) => setBetaPromise(e.target.value)}
|
||||
placeholder="I promise to share my honest feedback about my experience..."
|
||||
rows={2}
|
||||
className="w-full px-4 py-3 bg-dark-800 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 focus:border-transparent text-white placeholder-dark-500 text-sm resize-none"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{betaError && (
|
||||
<div className="text-red-400 text-sm bg-red-400/10 border border-red-400/20 rounded-lg p-3">
|
||||
{betaError}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<button
|
||||
type="submit"
|
||||
disabled={betaLoading}
|
||||
className="w-full bg-purple-500 hover:bg-purple-600 disabled:bg-purple-500/50 text-white font-semibold py-4 rounded-xl transition-all flex items-center justify-center gap-2"
|
||||
>
|
||||
{betaLoading ? (
|
||||
<>
|
||||
<Loader2 className="w-5 h-5 animate-spin" />
|
||||
Enrolling...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
Get My Free Credit
|
||||
<Gift className="w-5 h-5" />
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
|
||||
<p className="text-xs text-dark-500 text-center">
|
||||
We'll send you a feedback form after your video is created.
|
||||
Your feedback helps us improve the tool for everyone!
|
||||
</p>
|
||||
</form>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Pricing */}
|
||||
<section id="pricing" className="py-20 px-4">
|
||||
<div className="max-w-4xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-4">Simple Pricing</h2>
|
||||
<p className="text-dark-400 text-center mb-12 max-w-xl mx-auto">
|
||||
One credit = one karaoke video. Buy more to save more.
|
||||
</p>
|
||||
|
||||
{/* Package Selection */}
|
||||
<div className="grid grid-cols-2 md:grid-cols-4 gap-4 mb-8">
|
||||
{CREDIT_PACKAGES.map((pkg) => (
|
||||
<button
|
||||
key={pkg.id}
|
||||
onClick={() => setSelectedPackage(pkg)}
|
||||
className={`relative p-4 rounded-xl border-2 transition-all ${
|
||||
selectedPackage.id === pkg.id
|
||||
? 'border-primary-500 bg-primary-500/10'
|
||||
: 'border-dark-700 hover:border-dark-500 bg-dark-800/50'
|
||||
}`}
|
||||
>
|
||||
{pkg.popular && (
|
||||
<span className="absolute -top-3 left-1/2 -translate-x-1/2 bg-primary-500 text-white text-xs font-semibold px-3 py-1 rounded-full">
|
||||
Best Value
|
||||
</span>
|
||||
)}
|
||||
<div className="text-2xl font-bold mb-1">{pkg.credits}</div>
|
||||
<div className="text-sm text-dark-400 mb-2">
|
||||
{pkg.credits === 1 ? 'credit' : 'credits'}
|
||||
</div>
|
||||
<div className="text-xl font-semibold text-primary-400">
|
||||
{formatPrice(pkg.price_cents)}
|
||||
</div>
|
||||
<div className="text-xs text-dark-500">
|
||||
{pricePerCredit(pkg.price_cents, pkg.credits)}/video
|
||||
</div>
|
||||
{pkg.credits > 1 && (
|
||||
<div className="text-xs text-green-400 mt-1">{pkg.description}</div>
|
||||
)}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
{/* Checkout Form */}
|
||||
<div className="bg-dark-800 border border-dark-700 rounded-2xl p-8 max-w-md mx-auto">
|
||||
<div className="text-center mb-6">
|
||||
<div className="text-sm text-dark-400 mb-1">Selected package</div>
|
||||
<div className="text-2xl font-bold">
|
||||
{selectedPackage.credits} {selectedPackage.credits === 1 ? 'Credit' : 'Credits'}
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-primary-400">
|
||||
{formatPrice(selectedPackage.price_cents)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<form onSubmit={handleCheckout} className="space-y-4">
|
||||
<div>
|
||||
<label htmlFor="email" className="block text-sm font-medium text-dark-300 mb-2">
|
||||
Email address
|
||||
</label>
|
||||
<div className="relative">
|
||||
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 w-5 h-5 text-dark-500" />
|
||||
<input
|
||||
type="email"
|
||||
id="email"
|
||||
value={email}
|
||||
onChange={(e) => setEmail(e.target.value)}
|
||||
placeholder="you@example.com"
|
||||
className="w-full pl-10 pr-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-primary-500 focus:border-transparent text-white placeholder-dark-500"
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
<p className="text-xs text-dark-500 mt-2">
|
||||
We'll send your login link to this email
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{error && (
|
||||
<div className="text-red-400 text-sm bg-red-400/10 border border-red-400/20 rounded-lg p-3">
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<button
|
||||
type="submit"
|
||||
disabled={isLoading}
|
||||
className="w-full bg-primary-500 hover:bg-primary-600 disabled:bg-primary-500/50 text-white font-semibold py-4 rounded-xl transition-all btn-glow flex items-center justify-center gap-2"
|
||||
>
|
||||
{isLoading ? (
|
||||
<>
|
||||
<Loader2 className="w-5 h-5 animate-spin" />
|
||||
Redirecting to checkout...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
Continue to Payment
|
||||
<Check className="w-5 h-5" />
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</form>
|
||||
|
||||
<div className="mt-4 flex items-center justify-center gap-4 text-xs text-dark-500">
|
||||
<span>Secure checkout</span>
|
||||
<span>powered by Stripe</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* FAQ */}
|
||||
<section className="py-16 px-4 bg-dark-800/50">
|
||||
<div className="max-w-2xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-12">Questions?</h2>
|
||||
<div className="space-y-4">
|
||||
{faqs.map((faq, i) => (
|
||||
<div
|
||||
key={i}
|
||||
className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden"
|
||||
>
|
||||
<button
|
||||
onClick={() => setExpandedFaq(expandedFaq === i ? null : i)}
|
||||
className="w-full px-6 py-4 text-left flex items-center justify-between hover:bg-dark-700/50 transition-colors"
|
||||
>
|
||||
<span className="font-medium">{faq.q}</span>
|
||||
<ChevronDown
|
||||
className={`w-5 h-5 text-dark-400 transition-transform ${
|
||||
expandedFaq === i ? 'rotate-180' : ''
|
||||
}`}
|
||||
/>
|
||||
</button>
|
||||
{expandedFaq === i && (
|
||||
<div className="px-6 pb-4 text-dark-400">{faq.a}</div>
|
||||
)}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Footer */}
|
||||
<footer className="py-12 px-4 border-t border-dark-700">
|
||||
<div className="max-w-6xl mx-auto flex flex-col md:flex-row items-center justify-between gap-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<Mic2 className="w-6 h-6 text-primary-500" />
|
||||
<span className="font-semibold">Nomad Karaoke</span>
|
||||
</div>
|
||||
<div className="text-sm text-dark-500">
|
||||
© {new Date().getFullYear()} Nomad Karaoke. All rights reserved.
|
||||
</div>
|
||||
<div className="flex gap-6 text-sm text-dark-400">
|
||||
<a href="mailto:support@nomadkaraoke.com" className="hover:text-white transition-colors">
|
||||
Support
|
||||
</a>
|
||||
<a href="https://nomadkaraoke.com" className="hover:text-white transition-colors">
|
||||
About
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</footer>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
/**
|
||||
* API client for the Nomad Karaoke backend.
|
||||
*/
|
||||
|
||||
const API_BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'https://api.nomadkaraoke.com';
|
||||
|
||||
export interface CreditPackage {
|
||||
id: string;
|
||||
credits: number;
|
||||
price_cents: number;
|
||||
name: string;
|
||||
description: string;
|
||||
}
|
||||
|
||||
export interface CreditPackagesResponse {
|
||||
packages: CreditPackage[];
|
||||
}
|
||||
|
||||
export interface CheckoutResponse {
|
||||
status: string;
|
||||
checkout_url: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch available credit packages.
|
||||
*/
|
||||
export async function getCreditPackages(): Promise<CreditPackage[]> {
|
||||
const response = await fetch(`${API_BASE_URL}/api/users/credits/packages`);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error('Failed to fetch credit packages');
|
||||
}
|
||||
|
||||
const data: CreditPackagesResponse = await response.json();
|
||||
return data.packages;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a Stripe checkout session.
|
||||
*/
|
||||
export async function createCheckout(packageId: string, email: string): Promise<string> {
|
||||
const response = await fetch(`${API_BASE_URL}/api/users/credits/checkout`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
package_id: packageId,
|
||||
email: email,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.json();
|
||||
throw new Error(error.detail || 'Failed to create checkout session');
|
||||
}
|
||||
|
||||
const data: CheckoutResponse = await response.json();
|
||||
return data.checkout_url;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format price from cents to display string.
|
||||
*/
|
||||
export function formatPrice(cents: number): string {
|
||||
return `$${(cents / 100).toFixed(2)}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate per-credit price.
|
||||
*/
|
||||
export function pricePerCredit(pkg: CreditPackage): number {
|
||||
return pkg.price_cents / pkg.credits;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Beta Tester Program
|
||||
// ============================================================================
|
||||
|
||||
export interface BetaEnrollRequest {
|
||||
email: string;
|
||||
promise_text: string;
|
||||
accept_corrections_work: boolean;
|
||||
}
|
||||
|
||||
export interface BetaEnrollResponse {
|
||||
status: string;
|
||||
message: string;
|
||||
credits_granted: number;
|
||||
session_token: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Enroll as a beta tester to receive free credits.
|
||||
*/
|
||||
export async function enrollBetaTester(
|
||||
email: string,
|
||||
promiseText: string,
|
||||
acceptCorrectionsWork: boolean
|
||||
): Promise<BetaEnrollResponse> {
|
||||
const response = await fetch(`${API_BASE_URL}/api/users/beta/enroll`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
email: email,
|
||||
promise_text: promiseText,
|
||||
accept_corrections_work: acceptCorrectionsWork,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const error = await response.json();
|
||||
throw new Error(error.detail || 'Failed to enroll in beta program');
|
||||
}
|
||||
|
||||
return response.json();
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
/** @type {import('next').NextConfig} */
|
||||
const nextConfig = {
|
||||
output: 'export',
|
||||
trailingSlash: true,
|
||||
images: {
|
||||
unoptimized: true,
|
||||
},
|
||||
// For GitHub Pages deployment
|
||||
basePath: '',
|
||||
assetPrefix: '',
|
||||
};
|
||||
|
||||
export default nextConfig;
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"name": "buy-nomadkaraoke",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"description": "Landing page and payment flow for Nomad Karaoke",
|
||||
"scripts": {
|
||||
"dev": "next dev",
|
||||
"build": "next build",
|
||||
"start": "next start",
|
||||
"lint": "next lint"
|
||||
},
|
||||
"dependencies": {
|
||||
"next": "^15.0.0",
|
||||
"react": "^19.0.0",
|
||||
"react-dom": "^19.0.0",
|
||||
"lucide-react": "^0.468.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.0.0",
|
||||
"@types/react": "^19.0.0",
|
||||
"@types/react-dom": "^19.0.0",
|
||||
"typescript": "^5.0.0",
|
||||
"tailwindcss": "^3.4.0",
|
||||
"postcss": "^8.4.0",
|
||||
"autoprefixer": "^10.4.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
/** @type {import('postcss-load-config').Config} */
|
||||
const config = {
|
||||
plugins: {
|
||||
tailwindcss: {},
|
||||
autoprefixer: {},
|
||||
},
|
||||
};
|
||||
|
||||
export default config;
|
||||
@@ -0,0 +1,43 @@
|
||||
/** @type {import('tailwindcss').Config} */
|
||||
module.exports = {
|
||||
content: [
|
||||
'./app/**/*.{js,ts,jsx,tsx,mdx}',
|
||||
'./components/**/*.{js,ts,jsx,tsx,mdx}',
|
||||
],
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
// Match the existing karaoke-gen dark theme
|
||||
primary: {
|
||||
50: '#eff6ff',
|
||||
100: '#dbeafe',
|
||||
200: '#bfdbfe',
|
||||
300: '#93c5fd',
|
||||
400: '#60a5fa',
|
||||
500: '#3b82f6',
|
||||
600: '#2563eb',
|
||||
700: '#1d4ed8',
|
||||
800: '#1e40af',
|
||||
900: '#1e3a8a',
|
||||
},
|
||||
dark: {
|
||||
50: '#f8fafc',
|
||||
100: '#f1f5f9',
|
||||
200: '#e2e8f0',
|
||||
300: '#cbd5e1',
|
||||
400: '#94a3b8',
|
||||
500: '#64748b',
|
||||
600: '#475569',
|
||||
700: '#334155',
|
||||
800: '#1e293b',
|
||||
900: '#0f172a',
|
||||
950: '#020617',
|
||||
},
|
||||
},
|
||||
fontFamily: {
|
||||
sans: ['Inter', 'system-ui', 'sans-serif'],
|
||||
},
|
||||
},
|
||||
},
|
||||
plugins: [],
|
||||
};
|
||||
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"lib": ["dom", "dom.iterable", "esnext"],
|
||||
"allowJs": true,
|
||||
"skipLibCheck": true,
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"esModuleInterop": true,
|
||||
"module": "esnext",
|
||||
"moduleResolution": "bundler",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"jsx": "preserve",
|
||||
"incremental": true,
|
||||
"plugins": [
|
||||
{
|
||||
"name": "next"
|
||||
}
|
||||
],
|
||||
"paths": {
|
||||
"@/*": ["./*"]
|
||||
}
|
||||
},
|
||||
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
|
||||
"exclude": ["node_modules"]
|
||||
}
|
||||
+282
@@ -0,0 +1,282 @@
|
||||
# API Reference
|
||||
|
||||
Base URL: `https://api.nomadkaraoke.com` (production) or `http://localhost:8000` (local)
|
||||
|
||||
## Authentication
|
||||
|
||||
All endpoints except `/health` and `/api/themes` require authentication.
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer YOUR_TOKEN" ...
|
||||
```
|
||||
|
||||
## Endpoints
|
||||
|
||||
### Health
|
||||
|
||||
```http
|
||||
GET /health
|
||||
```
|
||||
|
||||
No auth required. Returns service status.
|
||||
|
||||
### Jobs
|
||||
|
||||
#### Create Job with Upload
|
||||
|
||||
```http
|
||||
POST /api/jobs/upload
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
file: <audio file>
|
||||
artist: "Artist Name"
|
||||
title: "Song Title"
|
||||
```
|
||||
|
||||
Returns job ID. Triggers async processing.
|
||||
|
||||
#### Get Job Status
|
||||
|
||||
```http
|
||||
GET /api/jobs/{job_id}
|
||||
```
|
||||
|
||||
Response includes:
|
||||
- `status` - Current job state
|
||||
- `file_urls` - Signed URLs for outputs (when ready)
|
||||
- `state_data` - Worker progress details
|
||||
- `timeline` - Processing history
|
||||
|
||||
#### List Jobs
|
||||
|
||||
```http
|
||||
GET /api/jobs
|
||||
GET /api/jobs?status=complete
|
||||
GET /api/jobs?limit=10&offset=0
|
||||
```
|
||||
|
||||
#### Delete Job
|
||||
|
||||
```http
|
||||
DELETE /api/jobs/{job_id}
|
||||
```
|
||||
|
||||
### Review
|
||||
|
||||
#### Get Correction Data
|
||||
|
||||
```http
|
||||
GET /api/review/{job_id}/correction-data
|
||||
```
|
||||
|
||||
Returns lyrics correction data for the review UI.
|
||||
|
||||
#### Complete Review
|
||||
|
||||
```http
|
||||
POST /api/review/{job_id}/complete
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"corrections": [...],
|
||||
"corrected_segments": [...]
|
||||
}
|
||||
```
|
||||
|
||||
Saves corrections and triggers video rendering.
|
||||
|
||||
#### Generate Preview
|
||||
|
||||
```http
|
||||
POST /api/review/{job_id}/preview-video
|
||||
```
|
||||
|
||||
Generates preview video during review.
|
||||
|
||||
#### Stream Audio
|
||||
|
||||
```http
|
||||
GET /api/review/{job_id}/audio/{stem_type}
|
||||
```
|
||||
|
||||
Streams audio for review playback.
|
||||
|
||||
### Instrumental Selection
|
||||
|
||||
```http
|
||||
POST /api/jobs/{job_id}/select-instrumental
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"selection": "clean" // or "with_backing"
|
||||
}
|
||||
```
|
||||
|
||||
### Audio Search
|
||||
|
||||
```http
|
||||
GET /api/audio-search?q=song+name
|
||||
```
|
||||
|
||||
Search for songs (flacfetch integration).
|
||||
|
||||
### Themes
|
||||
|
||||
```http
|
||||
GET /api/themes
|
||||
```
|
||||
|
||||
No auth required. Returns available video themes.
|
||||
|
||||
### Internal (Admin Only)
|
||||
|
||||
These endpoints are used by workers and require admin tokens.
|
||||
|
||||
```http
|
||||
POST /api/internal/jobs/{job_id}/trigger-audio
|
||||
POST /api/internal/jobs/{job_id}/trigger-lyrics
|
||||
POST /api/internal/jobs/{job_id}/trigger-screens
|
||||
POST /api/internal/jobs/{job_id}/trigger-render-video
|
||||
POST /api/internal/jobs/{job_id}/trigger-video
|
||||
```
|
||||
|
||||
## Job States
|
||||
|
||||
| State | Description |
|
||||
|-------|-------------|
|
||||
| `pending` | Created, not started |
|
||||
| `downloading` | Processing input |
|
||||
| `separating_stage1` | Audio separation (1/2) |
|
||||
| `separating_stage2` | Audio separation (2/2) |
|
||||
| `transcribing` | Lyrics transcription |
|
||||
| `generating_screens` | Title/end screens |
|
||||
| `awaiting_review` | Waiting for human review |
|
||||
| `in_review` | Human reviewing |
|
||||
| `review_complete` | Review submitted |
|
||||
| `rendering_video` | Generating karaoke video |
|
||||
| `awaiting_instrumental_selection` | Waiting for selection |
|
||||
| `instrumental_selected` | Selection made |
|
||||
| `generating_video` | Final encoding |
|
||||
| `complete` | Done |
|
||||
| `failed` | Error |
|
||||
|
||||
## Error Responses
|
||||
|
||||
```json
|
||||
{
|
||||
"detail": "Error message"
|
||||
}
|
||||
```
|
||||
|
||||
Common status codes:
|
||||
- `400` - Bad request
|
||||
- `401` - Unauthorized
|
||||
- `404` - Job not found
|
||||
- `500` - Server error
|
||||
|
||||
## User Authentication (PR #90)
|
||||
|
||||
### Send Magic Link
|
||||
|
||||
```http
|
||||
POST /api/users/auth/magic-link
|
||||
Content-Type: application/json
|
||||
|
||||
{"email": "user@example.com"}
|
||||
```
|
||||
|
||||
### Verify Magic Link
|
||||
|
||||
```http
|
||||
GET /api/users/auth/verify?token=TOKEN
|
||||
```
|
||||
|
||||
Returns session token and user info.
|
||||
|
||||
### Get Current User
|
||||
|
||||
```http
|
||||
GET /api/users/me
|
||||
Authorization: Bearer SESSION_TOKEN
|
||||
```
|
||||
|
||||
### Logout
|
||||
|
||||
```http
|
||||
POST /api/users/auth/logout
|
||||
Authorization: Bearer SESSION_TOKEN
|
||||
```
|
||||
|
||||
## Credits & Payments (PR #90)
|
||||
|
||||
### List Credit Packages
|
||||
|
||||
```http
|
||||
GET /api/users/credits/packages
|
||||
```
|
||||
|
||||
No auth required. Returns available packages and prices.
|
||||
|
||||
### Create Checkout Session
|
||||
|
||||
```http
|
||||
POST /api/users/credits/checkout
|
||||
Content-Type: application/json
|
||||
|
||||
{"package_id": "5_credits", "email": "user@example.com"}
|
||||
```
|
||||
|
||||
Returns Stripe checkout URL.
|
||||
|
||||
### Stripe Webhook
|
||||
|
||||
```http
|
||||
POST /api/users/webhooks/stripe
|
||||
```
|
||||
|
||||
Handles `checkout.session.completed` events.
|
||||
|
||||
## Beta Tester Program (PR #90)
|
||||
|
||||
### Enroll as Beta Tester
|
||||
|
||||
```http
|
||||
POST /api/users/beta/enroll
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"email": "user@example.com",
|
||||
"promise_text": "I promise to provide feedback...",
|
||||
"accept_corrections_work": true
|
||||
}
|
||||
```
|
||||
|
||||
Returns 1 free credit and session token.
|
||||
|
||||
### Submit Feedback
|
||||
|
||||
```http
|
||||
POST /api/users/beta/feedback
|
||||
Authorization: Bearer SESSION_TOKEN
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"overall_rating": 4,
|
||||
"ease_of_use_rating": 5,
|
||||
"lyrics_accuracy_rating": 4,
|
||||
"correction_experience_rating": 3,
|
||||
"what_went_well": "...",
|
||||
"what_could_improve": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Bonus credit for detailed feedback (50+ chars).
|
||||
|
||||
## Rate Limits
|
||||
|
||||
No rate limits currently implemented.
|
||||
|
||||
## Webhooks
|
||||
|
||||
Stripe webhooks implemented at `/api/users/webhooks/stripe`.
|
||||
Job status webhooks not yet implemented.
|
||||
@@ -0,0 +1,163 @@
|
||||
# Architecture
|
||||
|
||||
## System Overview
|
||||
|
||||
```text
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Frontend (Cloudflare Pages) │
|
||||
│ Next.js + TypeScript │
|
||||
│ https://gen.nomadkaraoke.com │
|
||||
└───────────────────────────┬─────────────────────────────────────┘
|
||||
│ HTTPS
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Backend API (Cloud Run) │
|
||||
│ FastAPI + Python 3.12 │
|
||||
│ https://api.nomadkaraoke.com │
|
||||
│ │
|
||||
│ Workers: │
|
||||
│ • Audio Worker - Stem separation via Modal API │
|
||||
│ • Lyrics Worker - Transcription via AudioShake │
|
||||
│ • Screens Worker - Title/end screen generation │
|
||||
│ • Render Worker - Karaoke video with lyrics │
|
||||
│ • Video Worker - Final assembly and encoding │
|
||||
└───────────────────────────┬─────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────┐ ┌──────────────┐
|
||||
│ Firestore│ │ GCS │ │Secret Manager│
|
||||
│ (jobs) │ │ (files) │ │ (API keys) │
|
||||
└──────────┘ └──────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
## Processing Pipeline
|
||||
|
||||
```text
|
||||
1. UPLOAD
|
||||
POST /api/jobs/upload
|
||||
• Save audio to GCS
|
||||
• Create job in Firestore
|
||||
• Trigger parallel workers
|
||||
|
||||
2. PARALLEL PROCESSING
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Audio Worker │ │ Lyrics Worker │
|
||||
│ Modal API │ │ AudioShake │
|
||||
│ 2-stage sep │ │ Auto-correct │
|
||||
└────────┬────────┘ └────────┬────────┘
|
||||
└──────────┬───────────┘
|
||||
▼
|
||||
3. SCREENS GENERATION
|
||||
• Title screen (5s video)
|
||||
• End screen (5s video)
|
||||
▼
|
||||
4. HUMAN REVIEW (AWAITING_REVIEW)
|
||||
• React UI for lyrics correction
|
||||
• Preview video generation
|
||||
• Submit corrections
|
||||
▼
|
||||
5. VIDEO RENDERING
|
||||
• Merge corrections with original data
|
||||
• Generate karaoke video with lyrics
|
||||
▼
|
||||
6. INSTRUMENTAL SELECTION
|
||||
• User chooses: clean or with_backing
|
||||
▼
|
||||
7. FINAL VIDEO
|
||||
• Remux with selected instrumental
|
||||
• Concatenate: title + karaoke + end
|
||||
• Encode to 4 formats
|
||||
▼
|
||||
8. COMPLETE
|
||||
• User downloads via signed URLs
|
||||
```
|
||||
|
||||
## Job States
|
||||
|
||||
| State | Description | Human Action |
|
||||
|-------|-------------|--------------|
|
||||
| `pending` | Job created | - |
|
||||
| `downloading` | Processing audio | - |
|
||||
| `separating_stage1` | Audio separation stage 1 | - |
|
||||
| `separating_stage2` | Audio separation stage 2 | - |
|
||||
| `transcribing` | Lyrics transcription | - |
|
||||
| `generating_screens` | Creating title/end screens | - |
|
||||
| `awaiting_review` | **Waiting for human** | Review lyrics |
|
||||
| `in_review` | Human actively reviewing | Edit lyrics |
|
||||
| `review_complete` | Review submitted | - |
|
||||
| `rendering_video` | Generating karaoke video | - |
|
||||
| `awaiting_instrumental_selection` | **Waiting for human** | Choose audio |
|
||||
| `instrumental_selected` | Selection made | - |
|
||||
| `generating_video` | Final encoding | - |
|
||||
| `complete` | All done | Download files |
|
||||
| `failed` | Error occurred | - |
|
||||
|
||||
## GCS File Structure
|
||||
|
||||
```text
|
||||
jobs/{job_id}/
|
||||
├── input/
|
||||
│ └── original.flac # Uploaded audio
|
||||
├── stems/
|
||||
│ ├── vocals.flac
|
||||
│ ├── instrumental_clean.flac
|
||||
│ ├── instrumental_with_backing.flac
|
||||
│ └── ...
|
||||
├── lyrics/
|
||||
│ ├── corrections.json # Auto-corrected
|
||||
│ ├── corrections_updated.json # After human review
|
||||
│ └── karaoke.ass
|
||||
├── screens/
|
||||
│ ├── title.mov
|
||||
│ └── end.mov
|
||||
├── videos/
|
||||
│ └── with_vocals.mkv # Karaoke video
|
||||
└── finals/
|
||||
├── lossless_4k.mp4
|
||||
├── lossless_4k.mkv
|
||||
├── lossy_4k.mp4
|
||||
└── lossy_720p.mp4
|
||||
```
|
||||
|
||||
## LyricsTranscriber Integration
|
||||
|
||||
**Key Design Decision**: We use LyricsTranscriber as a **library**, not a server.
|
||||
|
||||
```text
|
||||
CLI Mode (Original): Cloud Mode (Our Approach):
|
||||
──────────────────── ──────────────────────────
|
||||
LyricsTranscriber LyricsTranscriber
|
||||
├── transcribe() ├── transcribe() ✅
|
||||
├── correct() ├── correct() ✅
|
||||
├── ReviewServer.start() │
|
||||
│ └── BLOCKS (waits) │ (Skip server, save JSON)
|
||||
│ │ ... ASYNC GAP ...
|
||||
│ │ (Human reviews via React)
|
||||
└── OutputGenerator │
|
||||
└── OutputGenerator ✅
|
||||
(called after review)
|
||||
```
|
||||
|
||||
**Use**: `LyricsTranscriber`, `CorrectionResult`, `OutputGenerator`, `OutputConfig`
|
||||
**Don't use**: `ReviewServer`, blocking waits
|
||||
|
||||
## External Services
|
||||
|
||||
| Service | Purpose | Required |
|
||||
|---------|---------|----------|
|
||||
| Modal | GPU audio separation | Yes |
|
||||
| AudioShake | Lyrics transcription | Yes |
|
||||
| Vertex AI | Agentic AI correction (Gemini 3 Flash) | Default on |
|
||||
| Genius | Reference lyrics | Yes |
|
||||
| YouTube API | Video upload | Optional |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Backend**: FastAPI, Python 3.12, Cloud Run
|
||||
- **Frontend**: Next.js, TypeScript, Cloudflare Pages
|
||||
- **Database**: Firestore
|
||||
- **Storage**: Google Cloud Storage
|
||||
- **Secrets**: Google Secret Manager
|
||||
- **IaC**: Pulumi
|
||||
- **CI/CD**: GitHub Actions
|
||||
@@ -0,0 +1,216 @@
|
||||
# Development Guide
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.12
|
||||
- Node.js 20+
|
||||
- Docker (for GCS emulator)
|
||||
- Google Cloud SDK with Firestore emulator
|
||||
|
||||
```bash
|
||||
# Install Firestore emulator
|
||||
gcloud components install cloud-firestore-emulator
|
||||
```
|
||||
|
||||
### Backend Setup
|
||||
|
||||
```bash
|
||||
# Create and activate venv
|
||||
python3.12 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install dependencies
|
||||
poetry install
|
||||
|
||||
# Copy environment template
|
||||
cp .env.example .env
|
||||
# Edit .env with required API keys
|
||||
```
|
||||
|
||||
### Frontend Setup
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
cp .env.example .env.local
|
||||
# Edit .env.local with KARAOKE_ACCESS_TOKEN
|
||||
```
|
||||
|
||||
## Running Locally
|
||||
|
||||
### Backend with Emulators
|
||||
|
||||
```bash
|
||||
# Start emulators and backend
|
||||
./scripts/run-backend-local.sh --with-emulators
|
||||
|
||||
# Backend runs at http://localhost:8000
|
||||
# API docs at http://localhost:8000/docs
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm run dev
|
||||
# Runs at http://localhost:3000
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Backend Tests
|
||||
|
||||
```bash
|
||||
# Run all tests (required before commit)
|
||||
make test 2>&1 | tail -n 500
|
||||
|
||||
# This runs in order:
|
||||
# 1. Unit tests with coverage
|
||||
# 2. Backend unit tests
|
||||
# 3. E2E integration tests with emulators
|
||||
```
|
||||
|
||||
**Important**: Set 10 minute timeout. Tests can take a while with emulators.
|
||||
|
||||
### Frontend Tests
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm run test:all 2>&1 | tail -n 200
|
||||
|
||||
# Individual commands:
|
||||
npm run test:unit # Jest unit tests
|
||||
npm run test:e2e # Playwright E2E tests
|
||||
```
|
||||
|
||||
### Test Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
│ Unit Tests (Mocked) │ Fast, isolated
|
||||
│ ~62 tests, <1s │
|
||||
├─────────────────────────────────┤
|
||||
│ Emulator Integration Tests │ Local Firestore/GCS
|
||||
│ ~11 tests, ~2s │
|
||||
├─────────────────────────────────┤
|
||||
│ E2E Tests │ Full API + Frontend
|
||||
│ Requires deployed backend │
|
||||
└─────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Automatic (CI/CD)
|
||||
|
||||
Push to `main` triggers GitHub Actions:
|
||||
1. Run tests
|
||||
2. Build Docker image
|
||||
3. Deploy to Cloud Run
|
||||
4. Deploy frontend to Cloudflare Pages
|
||||
|
||||
### Manual Backend Deploy
|
||||
|
||||
```bash
|
||||
# Preview infrastructure changes
|
||||
cd infrastructure
|
||||
pulumi preview
|
||||
|
||||
# Apply changes
|
||||
pulumi up
|
||||
```
|
||||
|
||||
### Manual Frontend Deploy
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm run build
|
||||
npx wrangler pages deploy out
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Backend (.env)
|
||||
|
||||
```bash
|
||||
# Required
|
||||
AUDIOSHAKE_API_TOKEN=...
|
||||
GENIUS_ACCESS_TOKEN=...
|
||||
AUDIO_SEPARATOR_API_URL=... # Modal API endpoint
|
||||
|
||||
# GCP (auto-configured in Cloud Run)
|
||||
GOOGLE_CLOUD_PROJECT=...
|
||||
GCS_BUCKET_NAME=...
|
||||
|
||||
# Auth
|
||||
ADMIN_TOKENS=token1,token2
|
||||
```
|
||||
|
||||
### Frontend (.env.local)
|
||||
|
||||
```bash
|
||||
NEXT_PUBLIC_API_URL=http://localhost:8000
|
||||
KARAOKE_ACCESS_TOKEN=... # For E2E tests
|
||||
```
|
||||
|
||||
## Key Scripts
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `make test` | Run all backend tests |
|
||||
| `./scripts/run-backend-local.sh` | Start backend with emulators |
|
||||
| `./scripts/start-emulators.sh` | Start emulators only |
|
||||
| `./scripts/stop-emulators.sh` | Stop emulators |
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
karaoke-gen/
|
||||
├── backend/
|
||||
│ ├── api/routes/ # FastAPI endpoints
|
||||
│ ├── models/ # Pydantic models
|
||||
│ ├── services/ # Business logic
|
||||
│ ├── workers/ # Background workers
|
||||
│ └── tests/ # Backend tests
|
||||
├── frontend/
|
||||
│ ├── src/
|
||||
│ │ ├── app/ # Next.js pages
|
||||
│ │ ├── components/ # React components
|
||||
│ │ └── lib/ # Utilities
|
||||
│ └── tests/ # Frontend tests
|
||||
├── infrastructure/ # Pulumi IaC
|
||||
├── karaoke_gen/ # CLI package
|
||||
└── scripts/ # Dev scripts
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Emulators Won't Start
|
||||
|
||||
```bash
|
||||
# Check if already running
|
||||
lsof -i :8080 # Firestore
|
||||
lsof -i :4443 # GCS
|
||||
|
||||
# Stop them
|
||||
./scripts/stop-emulators.sh
|
||||
```
|
||||
|
||||
### Tests Fail with Auth Errors
|
||||
|
||||
Ensure `ADMIN_TOKENS` is set in environment or `.env` file.
|
||||
|
||||
### Docker Issues
|
||||
|
||||
```bash
|
||||
# Ensure Docker is running
|
||||
docker info
|
||||
|
||||
# For GCS emulator
|
||||
docker pull fsouza/fake-gcs-server
|
||||
```
|
||||
|
||||
### Coverage Below Target
|
||||
|
||||
Run `./scripts/run-tests.sh --coverage` and check `htmlcov/index.html` to see uncovered lines.
|
||||
@@ -0,0 +1,310 @@
|
||||
# Lessons Learned
|
||||
|
||||
Key insights for future AI agents working on this codebase.
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
### LyricsTranscriber: Library, Not Server
|
||||
|
||||
**Problem**: LyricsTranscriber's `ReviewServer` blocks waiting for human input - incompatible with async cloud architecture.
|
||||
|
||||
**Solution**: Use LyricsTranscriber as a library:
|
||||
- Call `transcribe()` and `correct()` for processing
|
||||
- Save `corrections.json` to GCS
|
||||
- Skip `ReviewServer` entirely
|
||||
- Call `OutputGenerator` only after human review completes
|
||||
|
||||
**What to use**: `LyricsTranscriber`, `CorrectionResult`, `OutputGenerator`, `OutputConfig`, `CorrectionOperations`
|
||||
|
||||
**What NOT to use**: `ReviewServer`, `server.start()`, blocking waits
|
||||
|
||||
### Video Generation Timing
|
||||
|
||||
**Wrong**: Generate video during lyrics worker, then review
|
||||
**Right**: Lyrics worker saves JSON → Human reviews → Render worker generates video
|
||||
|
||||
Video must include human-corrected lyrics, so generation happens AFTER review.
|
||||
|
||||
### Parallel Processing
|
||||
|
||||
Audio separation and lyrics transcription run in parallel:
|
||||
- Both can fail independently
|
||||
- Progress tracked separately in `state_data`
|
||||
- Screens generation waits for both to complete
|
||||
|
||||
## Common Gotchas
|
||||
|
||||
### Frontend Polling vs Tab Visibility
|
||||
|
||||
**Problem**: After completing review in a new tab (lyrics or instrumental selection), returning to the main frontend showed stale status for up to 10 seconds due to polling interval.
|
||||
|
||||
**Solution**: Use `visibilitychange` event to trigger immediate refresh when tab becomes visible:
|
||||
|
||||
```typescript
|
||||
useEffect(() => {
|
||||
const handleVisibilityChange = () => {
|
||||
if (document.visibilityState === 'visible') {
|
||||
loadJobs() // Immediate refresh
|
||||
}
|
||||
}
|
||||
document.addEventListener('visibilitychange', handleVisibilityChange)
|
||||
return () => document.removeEventListener('visibilitychange', handleVisibilityChange)
|
||||
}, [loadJobs])
|
||||
```
|
||||
|
||||
**Key insight**: Backend status updates are fast (~1 second). If UI appears slow to update after user action in another tab, the issue is likely frontend polling, not backend.
|
||||
|
||||
### Pydantic Model Fields
|
||||
|
||||
**Problem**: Setting a field that doesn't exist in the Pydantic model silently does nothing.
|
||||
|
||||
```python
|
||||
# BAD - fails silently if field doesn't exist
|
||||
job.input_media_gcs_path = "path/to/file"
|
||||
|
||||
# GOOD - explicit field in model
|
||||
class Job(BaseModel):
|
||||
input_media_gcs_path: Optional[str] = None
|
||||
```
|
||||
|
||||
**Lesson**: Always verify Pydantic model has the field before setting it.
|
||||
|
||||
### Styles Configuration
|
||||
|
||||
ASS subtitle generator requires ALL style fields present and properly typed:
|
||||
|
||||
```python
|
||||
required_fields = [
|
||||
"font", "font_path", "ass_name", # ass_name often missed
|
||||
"primary_color", "secondary_color", "outline_color", "back_color",
|
||||
"bold", "italic", "underline", "strike_out",
|
||||
"scale_x", "scale_y", "spacing", "angle",
|
||||
"border_style", "outline", "shadow",
|
||||
"margin_l", "margin_r", "margin_v", "encoding"
|
||||
]
|
||||
|
||||
# Critical: font_path must be string, not None
|
||||
"font_path": "" # Correct
|
||||
"font_path": None # Causes ASS writer to fail
|
||||
```
|
||||
|
||||
### Frontend Data Format
|
||||
|
||||
The React review UI sends partial correction data:
|
||||
|
||||
```python
|
||||
# Frontend sends:
|
||||
{"corrections": [...], "corrected_segments": [...]}
|
||||
|
||||
# Backend must merge with original corrections.json
|
||||
```
|
||||
|
||||
### Firestore Consistency
|
||||
|
||||
**Problem**: Update job, trigger worker immediately, worker reads stale data.
|
||||
|
||||
**Solution**: Verify updates propagated before triggering workers:
|
||||
```python
|
||||
# Update job
|
||||
await job_manager.update_job(job_id, updates)
|
||||
|
||||
# Verify update visible
|
||||
job = await job_manager.get_job(job_id)
|
||||
if job.input_media_gcs_path:
|
||||
await trigger_worker()
|
||||
```
|
||||
|
||||
### Stale Object References After Async Operations
|
||||
|
||||
**Problem**: Fetching a job object, then calling async functions that update the job in the database, then using the original object's data to make another update - this overwrites the async function's changes.
|
||||
|
||||
```python
|
||||
# BAD - loses distribution results
|
||||
job = job_manager.get_job(job_id) # Fetch job
|
||||
await _handle_native_distribution(...) # Updates job.state_data in DB
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {**job.state_data, 'foo': 'bar'} # STALE! Overwrites dropbox_link!
|
||||
})
|
||||
```
|
||||
|
||||
**Solution**: Either re-fetch the job, or include async results explicitly:
|
||||
```python
|
||||
# GOOD - include results from async operation
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
'dropbox_link': result.get('dropbox_link'), # From distribution
|
||||
'gdrive_files': result.get('gdrive_files'),
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
### Google Drive API Query Escaping
|
||||
|
||||
**Problem**: Single quotes in filenames break Google Drive API queries.
|
||||
|
||||
```python
|
||||
# BAD - fails for "Nobody's Home"
|
||||
query = f"name='{filename}' and '{parent_id}' in parents"
|
||||
# Results in: name='NOMAD-1187 - Artist - Nobody's Home.mp4' → 400 Invalid Value
|
||||
```
|
||||
|
||||
**Solution**: Escape single quotes with backslash:
|
||||
```python
|
||||
# GOOD
|
||||
escaped = filename.replace("'", "\\'")
|
||||
query = f"name='{escaped}' and '{parent_id}' in parents"
|
||||
```
|
||||
|
||||
### Duplicate Code Leads to Inconsistent Behavior
|
||||
|
||||
**Problem**: Multiple endpoints computing the same defaults independently. `audio_search.py` had `effective_brand_prefix = body.brand_prefix or settings.default_brand_prefix` but `file_upload.py` was missing it for some endpoints.
|
||||
|
||||
**Result**: Jobs created via audio search had correct brand codes; jobs created via file upload had null brand_prefix, causing Dropbox uploads to fail and Google Drive uploads to use placeholder "TRACK-0000".
|
||||
|
||||
**Solution**: Added `_get_effective_distribution_settings()` helper function in `file_upload.py` to centralize default computation:
|
||||
```python
|
||||
@dataclass
|
||||
class EffectiveDistributionSettings:
|
||||
dropbox_path: Optional[str]
|
||||
gdrive_folder_id: Optional[str]
|
||||
discord_webhook_url: Optional[str]
|
||||
brand_prefix: Optional[str]
|
||||
|
||||
def _get_effective_distribution_settings(...) -> EffectiveDistributionSettings:
|
||||
settings = get_settings()
|
||||
return EffectiveDistributionSettings(
|
||||
dropbox_path=dropbox_path or settings.default_dropbox_path,
|
||||
# ... etc
|
||||
)
|
||||
```
|
||||
|
||||
**Lesson**: When you see the same logic repeated in multiple places, refactor into a helper function immediately. Otherwise bugs will be fixed in one place but not others.
|
||||
|
||||
## Testing Insights
|
||||
|
||||
### Emulator Tests Catch Real Bugs
|
||||
|
||||
Unit tests with mocks didn't catch the `input_media_gcs_path` bug. Emulator integration tests did because they use real Firestore behavior.
|
||||
|
||||
**Lesson**: Run `make test` which includes emulator tests before every commit.
|
||||
|
||||
### Test Isolation
|
||||
|
||||
Emulator tests need their own `conftest.py` that does NOT mock `google.cloud.*`. Mocks from unit tests can leak into integration tests.
|
||||
|
||||
### Playwright: Use Assertions, Not Timeouts
|
||||
|
||||
**Problem**: `waitForTimeout()` is flaky and slow:
|
||||
|
||||
```typescript
|
||||
// BAD - arbitrary wait, still flaky
|
||||
await page.waitForTimeout(2000);
|
||||
const hasContent = await page.locator('.content').count();
|
||||
```
|
||||
|
||||
**Solution**: Use Playwright's auto-retry assertions:
|
||||
|
||||
```typescript
|
||||
// GOOD - waits only until condition met, retries automatically
|
||||
await expect(page.getByText('Hello,')).toBeVisible({ timeout: 5000 });
|
||||
```
|
||||
|
||||
**Why it matters**: Playwright assertions have built-in retry logic and fail fast when conditions aren't met. Arbitrary timeouts either waste time (waiting when condition is already met) or cause flaky tests (not waiting long enough on slow CI).
|
||||
|
||||
## Deployment Notes
|
||||
|
||||
### Cloud Run Timeouts
|
||||
|
||||
Default 5-minute timeout may not be enough for video encoding. Consider:
|
||||
- Cloud Run Jobs for long-running tasks
|
||||
- Breaking encoding into smaller chunks
|
||||
- Increasing timeout for specific operations
|
||||
|
||||
### Secret Manager Access
|
||||
|
||||
Cloud Run service account needs `Secret Manager Secret Accessor` role. This is managed via Pulumi in `infrastructure/`.
|
||||
|
||||
### Vertex AI Authentication
|
||||
|
||||
For Gemini via Vertex AI, use Application Default Credentials (ADC) rather than API keys:
|
||||
- **Cloud Run**: Service account automatically authenticated - just grant `roles/aiplatform.user`
|
||||
- **Local dev**: Run `gcloud auth application-default login`
|
||||
- **Config**: Set `GOOGLE_CLOUD_PROJECT` and optionally `GCP_LOCATION` (defaults to `us-central1`)
|
||||
|
||||
This approach uses GCP's free credits and existing IAM rather than separate API key management.
|
||||
|
||||
### CI/CD: GitHub Actions
|
||||
|
||||
Migrated from Cloud Build to GitHub Actions for:
|
||||
- Simpler configuration
|
||||
- Better visibility
|
||||
- Faster feedback
|
||||
|
||||
### Self-Hosted Runner on GCP
|
||||
|
||||
GitHub-hosted runners have ~14GB disk, which caused failures during Docker builds. Solution: self-hosted runner on GCP with 200GB SSD.
|
||||
|
||||
**Key learnings:**
|
||||
|
||||
1. **GPG in startup scripts**: Always use `gpg --batch` for non-interactive execution
|
||||
2. **setup-python on self-hosted**: Needs Python in tool cache at `$RUNNER/_work/_tool/Python/<version>/x64/` with `x64.complete` marker
|
||||
3. **Debian vs Ubuntu**: PPAs don't work on Debian; use upstream repos (Temurin, NodeSource)
|
||||
4. **pyenv for Python**: More reliable than source builds or distro packages for specific versions
|
||||
|
||||
See `docs/archive/2025-12-28-self-hosted-github-runner.md` for full details.
|
||||
|
||||
## Performance Observations
|
||||
|
||||
| Operation | Duration |
|
||||
|-----------|----------|
|
||||
| Audio separation | 5-8 min |
|
||||
| Lyrics transcription | 2-3 min |
|
||||
| Auto-correction | 30 sec |
|
||||
| Screens generation | 30 sec |
|
||||
| Video rendering | 10-15 min |
|
||||
| Final encoding | 5-10 min |
|
||||
| **Total** | 30-50 min |
|
||||
|
||||
Human review time varies (5-15 min typical).
|
||||
|
||||
### Gitignore Patterns for Monorepos
|
||||
|
||||
**Problem**: Root `.gitignore` had `lib/` to ignore Python lib directories, but this also ignored `buy-site/lib/` (frontend code).
|
||||
|
||||
**Solution**: Use negation patterns for frontend directories:
|
||||
```gitignore
|
||||
# Python lib directories
|
||||
lib/
|
||||
lib64/
|
||||
# But allow frontend lib directories
|
||||
!frontend/lib/
|
||||
!buy-site/lib/
|
||||
```
|
||||
|
||||
**Lesson**: When adding gitignore patterns, consider their effect on subdirectories, especially in monorepos with multiple languages.
|
||||
|
||||
### Firestore Transactions for Race Conditions
|
||||
|
||||
**Problem**: Magic link verification had a race condition - concurrent requests could both verify the same token before it was marked as used.
|
||||
|
||||
**Solution**: Use Firestore transactions for atomic read-check-update operations:
|
||||
```python
|
||||
@firestore.transactional
|
||||
def verify_in_transaction(transaction):
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
# Check conditions...
|
||||
transaction.update(doc_ref, {'used': True})
|
||||
```
|
||||
|
||||
## What We'd Do Differently
|
||||
|
||||
1. **Add Pydantic model fields test first** - Would have caught the silent field issue immediately
|
||||
|
||||
2. **Use emulator tests from day one** - Faster feedback than deploying to test
|
||||
|
||||
3. **Design for async human review upfront** - Avoided rearchitecting after discovering ReviewServer blocks
|
||||
|
||||
4. **Keep docs minimal and current** - Less documentation, but always accurate
|
||||
|
||||
5. **Check gitignore early for new directories** - Especially when adding frontend `lib/` directories
|
||||
@@ -0,0 +1,318 @@
|
||||
# Mobile UX Best Practices
|
||||
|
||||
This document outlines the mobile-first UX principles and implementation patterns used in this project to ensure a great experience across all screen sizes.
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Mobile-First Responsive Design
|
||||
|
||||
Design for the smallest screen first, then progressively enhance for larger screens.
|
||||
|
||||
```css
|
||||
/* Base styles for mobile */
|
||||
.element {
|
||||
width: 100%;
|
||||
padding: 12px;
|
||||
}
|
||||
|
||||
/* Enhance for larger screens */
|
||||
@media (min-width: 640px) {
|
||||
.element {
|
||||
width: 280px;
|
||||
padding: 16px;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. No Horizontal Overflow
|
||||
|
||||
The most critical mobile UX issue is horizontal scrolling. Always ensure content fits within the viewport width.
|
||||
|
||||
**Testing for overflow:**
|
||||
```typescript
|
||||
async function hasNoHorizontalOverflow(page: Page): Promise<boolean> {
|
||||
return await page.evaluate(() => {
|
||||
const docWidth = document.documentElement.scrollWidth;
|
||||
const viewportWidth = window.innerWidth;
|
||||
return docWidth <= viewportWidth + 2; // Small tolerance for rounding
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
**Common causes of horizontal overflow:**
|
||||
- Fixed widths (e.g., `width: 340px`)
|
||||
- Horizontal flex layouts without wrapping
|
||||
- Images without `max-width: 100%`
|
||||
- Tables without horizontal scroll containers
|
||||
|
||||
### 3. Touch Target Sizing
|
||||
|
||||
Interactive elements must be easily tappable. Follow these minimum sizes:
|
||||
|
||||
| Platform | Minimum Size | Recommended |
|
||||
|----------|--------------|-------------|
|
||||
| Apple iOS | 44x44px | 48x48px |
|
||||
| Material Design | 48x48px | 48x48px |
|
||||
| Web (general) | 32x32px | 44x44px |
|
||||
|
||||
**Implementation:**
|
||||
```tsx
|
||||
<Button className="min-h-[44px] px-3">
|
||||
<Icon className="w-4 h-4" />
|
||||
<span className="hidden sm:inline">Label</span>
|
||||
</Button>
|
||||
```
|
||||
|
||||
### 4. Progressive Disclosure
|
||||
|
||||
Show less content on mobile, with clear paths to access more:
|
||||
|
||||
- **Icon-only buttons on mobile, labels on desktop:**
|
||||
```tsx
|
||||
<Icon className="w-4 h-4 sm:mr-2" />
|
||||
<span className="hidden sm:inline">Action Label</span>
|
||||
```
|
||||
|
||||
- **Collapsed navigation on mobile**
|
||||
- **Truncated text with expansion options**
|
||||
|
||||
## Layout Patterns
|
||||
|
||||
### Responsive Grid
|
||||
|
||||
Use responsive grid columns that stack on mobile:
|
||||
|
||||
```tsx
|
||||
// Bad - always 2 columns
|
||||
<div className="grid grid-cols-2 gap-4">
|
||||
|
||||
// Good - stack on mobile, 2 columns on larger screens
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
|
||||
```
|
||||
|
||||
### Flexible Containers
|
||||
|
||||
Avoid fixed widths. Use percentage-based or responsive widths:
|
||||
|
||||
```css
|
||||
/* Bad */
|
||||
.sidebar {
|
||||
width: 340px;
|
||||
}
|
||||
|
||||
/* Good */
|
||||
.sidebar {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
@media (min-width: 768px) {
|
||||
.sidebar {
|
||||
width: 340px;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Flex Wrap for Toolbars
|
||||
|
||||
Allow toolbar items to wrap on smaller screens:
|
||||
|
||||
```tsx
|
||||
<div className="flex flex-wrap gap-2 items-center">
|
||||
{/* Items will wrap naturally */}
|
||||
</div>
|
||||
```
|
||||
|
||||
## Typography
|
||||
|
||||
### Readable Font Sizes
|
||||
|
||||
Minimum font sizes for readability without zooming:
|
||||
|
||||
| Element | Minimum Size | Recommended |
|
||||
|---------|--------------|-------------|
|
||||
| Body text | 14px | 16px |
|
||||
| Labels | 12px | 14px |
|
||||
| Small text/badges | 11px | 12px |
|
||||
|
||||
### Responsive Headings
|
||||
|
||||
```tsx
|
||||
<h1 className="text-base sm:text-xl font-bold truncate">
|
||||
Title Text
|
||||
</h1>
|
||||
```
|
||||
|
||||
## Form Design
|
||||
|
||||
### Input Fields on Mobile
|
||||
|
||||
- Stack form fields vertically on mobile
|
||||
- Use full-width inputs for easier tapping
|
||||
- Ensure proper input type for mobile keyboard optimization
|
||||
|
||||
```tsx
|
||||
<input
|
||||
type="email" // Shows email keyboard
|
||||
inputMode="numeric" // Shows number pad
|
||||
className="w-full min-h-[44px]"
|
||||
/>
|
||||
```
|
||||
|
||||
### Labels and Placeholders
|
||||
|
||||
- Labels should be visible (not just placeholders)
|
||||
- Adequate spacing between label and input
|
||||
- Clear validation feedback
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Automated Testing with Playwright
|
||||
|
||||
Test across multiple viewport sizes representing real devices:
|
||||
|
||||
```typescript
|
||||
const MOBILE_VIEWPORTS = {
|
||||
'iPhone SE': { width: 375, height: 667 },
|
||||
'iPhone 14': { width: 390, height: 844 },
|
||||
'Pixel 7': { width: 412, height: 915 },
|
||||
'iPad Mini': { width: 768, height: 1024 },
|
||||
};
|
||||
|
||||
for (const [name, viewport] of Object.entries(MOBILE_VIEWPORTS)) {
|
||||
test(`page works on ${name}`, async ({ page }) => {
|
||||
await page.setViewportSize(viewport);
|
||||
await page.goto('/');
|
||||
|
||||
// Test for horizontal overflow
|
||||
const hasOverflow = await page.evaluate(() =>
|
||||
document.documentElement.scrollWidth > window.innerWidth + 2
|
||||
);
|
||||
expect(hasOverflow).toBe(false);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### Visual Regression Testing
|
||||
|
||||
Capture screenshots at different viewports to catch visual regressions:
|
||||
|
||||
```typescript
|
||||
await expect(page).toHaveScreenshot(`homepage-${deviceName}.png`, {
|
||||
fullPage: true,
|
||||
maxDiffPixelRatio: 0.1,
|
||||
});
|
||||
```
|
||||
|
||||
### Touch Target Validation
|
||||
|
||||
```typescript
|
||||
const buttons = page.locator('button, a, [role="button"]');
|
||||
for (const button of await buttons.all()) {
|
||||
const box = await button.boundingBox();
|
||||
if (box) {
|
||||
expect(box.width).toBeGreaterThanOrEqual(32);
|
||||
expect(box.height).toBeGreaterThanOrEqual(32);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Framework-Specific Guidelines
|
||||
|
||||
### Tailwind CSS (Next.js Frontend)
|
||||
|
||||
Use responsive prefixes consistently:
|
||||
|
||||
```plaintext
|
||||
sm: 640px - Small tablets and up
|
||||
md: 768px - Tablets and up
|
||||
lg: 1024px - Laptops and up
|
||||
xl: 1280px - Desktops and up
|
||||
```
|
||||
|
||||
**Common patterns:**
|
||||
```tsx
|
||||
// Stack on mobile, side-by-side on larger
|
||||
className="flex flex-col sm:flex-row"
|
||||
|
||||
// Hide on mobile, show on larger
|
||||
className="hidden sm:block"
|
||||
|
||||
// Reduce padding on mobile
|
||||
className="p-3 sm:p-4"
|
||||
|
||||
// Smaller font on mobile
|
||||
className="text-xs sm:text-sm"
|
||||
```
|
||||
|
||||
### Material UI (Lyrics Review UI)
|
||||
|
||||
Use the `useMediaQuery` hook:
|
||||
|
||||
```tsx
|
||||
const theme = useTheme();
|
||||
const isMobile = useMediaQuery(theme.breakpoints.down('md'));
|
||||
|
||||
return (
|
||||
<Box sx={{
|
||||
flexDirection: isMobile ? 'column' : 'row',
|
||||
width: isMobile ? '100%' : '280px',
|
||||
}}>
|
||||
```
|
||||
|
||||
### Vanilla CSS (Instrumental Review)
|
||||
|
||||
Use CSS media queries:
|
||||
|
||||
```css
|
||||
.element {
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
@media (min-width: 768px) {
|
||||
.element {
|
||||
width: 340px;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Checklist for New Features
|
||||
|
||||
Before merging any UI changes, verify:
|
||||
|
||||
- [ ] No horizontal overflow on any mobile viewport (320px - 480px)
|
||||
- [ ] All interactive elements are at least 44x44px
|
||||
- [ ] Text is readable without zooming (minimum 11px)
|
||||
- [ ] Form inputs stack vertically on mobile
|
||||
- [ ] Navigation is accessible on mobile
|
||||
- [ ] Images scale properly with `max-width: 100%`
|
||||
- [ ] Modals/dialogs fit within mobile viewport
|
||||
- [ ] Buttons show icons-only on very small screens
|
||||
- [ ] Playwright mobile tests pass
|
||||
|
||||
## Common Fixes
|
||||
|
||||
### Fix: Horizontal Overflow
|
||||
|
||||
1. **Check for fixed widths** - Replace with responsive values
|
||||
2. **Add flex-wrap** - Allow content to wrap
|
||||
3. **Add overflow-x: hidden** - On containers (use sparingly)
|
||||
4. **Check images** - Add `max-width: 100%`
|
||||
|
||||
### Fix: Touch Target Too Small
|
||||
|
||||
1. **Add minimum height** - `min-h-[44px]`
|
||||
2. **Add padding** - `p-3` or `px-4 py-3`
|
||||
3. **Use icon-only on mobile** - Hide text labels
|
||||
|
||||
### Fix: Text Too Small
|
||||
|
||||
1. **Use responsive font sizes** - `text-xs sm:text-sm`
|
||||
2. **Set minimum body font size** - At least 14px
|
||||
3. **Increase line height** - For better readability
|
||||
|
||||
## Resources
|
||||
|
||||
- [Apple Human Interface Guidelines - Touch Targets](https://developer.apple.com/design/human-interface-guidelines/accessibility)
|
||||
- [Material Design - Touch Targets](https://m3.material.io/foundations/accessible-design/accessibility-basics)
|
||||
- [Tailwind CSS Responsive Design](https://tailwindcss.com/docs/responsive-design)
|
||||
- [Playwright Testing](https://playwright.dev/docs/test-mobile)
|
||||
+37
-177
@@ -1,193 +1,53 @@
|
||||
# Karaoke-Gen Documentation
|
||||
|
||||
This directory contains all documentation for the karaoke-gen project, organized by purpose.
|
||||
> **Status**: Production-ready. First full E2E flow completed 2025-12-28.
|
||||
|
||||
## 📋 Current Status (Updated: 2025-12-08)
|
||||
## What Works
|
||||
|
||||
**Project:** Cloud-hosted karaoke generation with async human review
|
||||
**Backend:** 🔄 Phase 1.3 (85%) - Implementing review architecture
|
||||
**Frontend:** ⏸️ Not started (waiting for backend review flow)
|
||||
**Deployment:** ✅ Cloud Run with custom domain (api.nomadkaraoke.com)
|
||||
**Testing:** 🔄 Local emulator testing in progress
|
||||
- **Web App**: https://gen.nomadkaraoke.com - Upload audio, review lyrics, download videos
|
||||
- **Backend API**: https://api.nomadkaraoke.com - FastAPI on Cloud Run
|
||||
- **CLI Tools**: `karaoke-gen` (local), `karaoke-gen-remote` (cloud)
|
||||
|
||||
### Key Discovery
|
||||
## Current State
|
||||
|
||||
We identified that the `LyricsTranscriber` library's `ReviewServer` **blocks** waiting for human input, which doesn't work in our async cloud architecture. The solution is to:
|
||||
| Component | Status |
|
||||
|-----------|--------|
|
||||
| Audio upload & separation | Working |
|
||||
| Lyrics transcription | Working |
|
||||
| Agentic AI correction | Working (Gemini 3 Flash via Vertex AI) |
|
||||
| Human lyrics review | Working |
|
||||
| Preview video generation | Working |
|
||||
| Instrumental selection | Working |
|
||||
| Multi-format encoding | Working |
|
||||
| Token-based auth | Working |
|
||||
| Magic link auth | In Progress (PR #90) |
|
||||
| Payment flow (Stripe) | In Progress (PR #90) |
|
||||
| Beta tester program | In Progress (PR #90) |
|
||||
| CI/CD self-hosted runner | Working (GCP) |
|
||||
|
||||
1. Use LyricsTranscriber for transcription + auto-correction
|
||||
2. Save corrections to GCS for async human review
|
||||
3. Use `OutputGenerator` to render video **after** review completes
|
||||
## Known Issues
|
||||
|
||||
See `00-current-plan/CURRENT-STATUS.md` for full details.
|
||||
- CDG format generation requires additional style configuration
|
||||
- Long audio files (>10 min) may timeout on some workers
|
||||
|
||||
---
|
||||
## Quick Links
|
||||
|
||||
## 📂 Documentation Structure
|
||||
| Doc | Purpose |
|
||||
|-----|---------|
|
||||
| [ARCHITECTURE.md](ARCHITECTURE.md) | System design, data flow, integrations |
|
||||
| [DEVELOPMENT.md](DEVELOPMENT.md) | Local setup, testing, deployment |
|
||||
| [API.md](API.md) | Backend endpoint reference |
|
||||
| [LESSONS-LEARNED.md](LESSONS-LEARNED.md) | Key insights for future agents |
|
||||
| [archive/](archive/) | Historical documentation |
|
||||
|
||||
### `00-current-plan/` - **START HERE**
|
||||
Active migration plan and current status.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| **`CURRENT-STATUS.md`** | Where we are now, what's working, what's next |
|
||||
| **`WHATS-NEXT.md`** | Detailed plan for review architecture implementation |
|
||||
| **`WORKER-IMPLEMENTATION-PLAN.md`** | Worker responsibilities and implementation details |
|
||||
| `MASTER-PLAN.md` | Overall migration strategy |
|
||||
| `CLOUD-MIGRATION-REQUIREMENTS.md` | Architectural analysis from CLI workflow |
|
||||
|
||||
### `01-reference/` - Reference Documentation
|
||||
Core system documentation.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| **`ARCHITECTURE.md`** | Cloud + CLI architecture with diagrams |
|
||||
| `KARAOKE-GEN-CLI-WORKFLOW.md` | CLI workflow breakdown (8 stages) |
|
||||
| `API-MANUAL-TESTING.md` | How to test backend API with curl |
|
||||
| `AUTHENTICATION.md` | Token-based auth system |
|
||||
| `CLOUD-RUN-ARCHITECTURE.md` | Cloud Run concepts |
|
||||
|
||||
### `02-implementation-history/` - Session Logs
|
||||
Historical implementation records.
|
||||
|
||||
### `03-deployment/` - Deployment Guides
|
||||
How to deploy and operate the system.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| **`EMULATOR-TESTING.md`** | Local testing with Firestore/GCS emulators |
|
||||
| **`SETUP-VENV.md`** | Python venv setup |
|
||||
| **`LOCAL-VALIDATION.md`** | Validate before deploying |
|
||||
| **`OBSERVABILITY-GUIDE.md`** | Debug production issues |
|
||||
|
||||
### `04-testing/` - Testing Guides
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| **`TESTING-GUIDE.md`** | Complete testing guide |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### I want to understand the current state:
|
||||
1. Read `00-current-plan/CURRENT-STATUS.md` - where we are now
|
||||
2. Read `00-current-plan/WHATS-NEXT.md` - what needs to be done
|
||||
3. Read `01-reference/ARCHITECTURE.md` - system architecture
|
||||
|
||||
### I want to work on the backend:
|
||||
1. Read `03-deployment/EMULATOR-TESTING.md` - local testing setup
|
||||
2. Read `00-current-plan/WORKER-IMPLEMENTATION-PLAN.md` - worker details
|
||||
3. Run `./scripts/run-backend-local.sh --with-emulators`
|
||||
|
||||
### I want to test:
|
||||
```bash
|
||||
# Start backend with emulators
|
||||
./scripts/run-backend-local.sh --with-emulators
|
||||
|
||||
# Upload test file
|
||||
curl -X POST http://localhost:8000/api/jobs/upload \
|
||||
-F "file=@tests/data/waterloo10sec.flac" \
|
||||
-F "artist=ABBA" \
|
||||
-F "title=Waterloo"
|
||||
|
||||
# Check status
|
||||
curl http://localhost:8000/api/jobs/{job_id}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Current Phase: Review Architecture
|
||||
|
||||
**Goal:** Implement async human review workflow
|
||||
|
||||
**What Works:**
|
||||
- ✅ File upload → GCS
|
||||
- ✅ Audio separation via Modal API
|
||||
- ✅ Lyrics transcription via AudioShake
|
||||
- ✅ Auto-correction via LyricsTranscriber
|
||||
- ✅ Title/end screen generation
|
||||
- ✅ Instrumental selection endpoint
|
||||
|
||||
**What's Being Implemented:**
|
||||
- 🔄 Review API endpoints (`GET/POST /api/jobs/{id}/review`)
|
||||
- 🔄 Render Video Worker (uses `OutputGenerator` after review)
|
||||
- 🔄 Updated state machine with `RENDERING_VIDEO` state
|
||||
|
||||
**The Key Insight:**
|
||||
```
|
||||
WRONG: Lyrics Worker → Generate Video → Review → ???
|
||||
RIGHT: Lyrics Worker → Save JSON → Human Review → Render Video → Final Assembly
|
||||
```
|
||||
|
||||
Video generation happens AFTER human review, not during the lyrics worker.
|
||||
|
||||
---
|
||||
|
||||
## 📝 Key Technical Decisions
|
||||
|
||||
### Why Not Use LyricsTranscriber's ReviewServer?
|
||||
|
||||
The `ReviewServer` class (in `lyrics_transcriber/review/server.py`):
|
||||
- Starts a local web server on port 8000
|
||||
- Opens a browser window
|
||||
- **BLOCKS** the thread until human submits corrections
|
||||
- Designed for local CLI operation
|
||||
|
||||
This doesn't work in Cloud Run where:
|
||||
- We can't open browsers
|
||||
- Workers must be non-blocking
|
||||
- Human review happens asynchronously via React frontend
|
||||
|
||||
**Solution:** Use the data structures and `OutputGenerator` directly, skip the server.
|
||||
|
||||
### Why Separate Video Rendering?
|
||||
## Workflow Summary
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Lyrics Worker │ ──▶ │ Human Reviews │ ──▶ │ Render Worker │
|
||||
│ │ │ (React UI) │ │ │
|
||||
│ • Transcribe │ │ • Corrects │ │ • OutputGenerator│
|
||||
│ • Auto-correct │ │ • Saves JSON │ │ • with_vocals.mkv│
|
||||
│ • Save JSON │ │ │ │ │
|
||||
│ • NO VIDEO │ │ │ │ │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
Upload Audio → Parallel Processing → Human Review → Video Generation → Download
|
||||
│ │
|
||||
(Modal + AudioShake) (LyricsTranscriber UI)
|
||||
```
|
||||
|
||||
The video must include the human-corrected lyrics, so it can only be generated AFTER review.
|
||||
## For AI Agents
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Job State Flow
|
||||
|
||||
```
|
||||
PENDING → DOWNLOADING → [parallel audio+lyrics] → GENERATING_SCREENS
|
||||
↓
|
||||
AWAITING_REVIEW → IN_REVIEW → REVIEW_COMPLETE
|
||||
↓
|
||||
RENDERING_VIDEO (NEW)
|
||||
↓
|
||||
AWAITING_INSTRUMENTAL_SELECTION → INSTRUMENTAL_SELECTED
|
||||
↓
|
||||
GENERATING_VIDEO → COMPLETE
|
||||
```
|
||||
|
||||
Two human interaction points:
|
||||
1. **AWAITING_REVIEW** - User corrects lyrics
|
||||
2. **AWAITING_INSTRUMENTAL_SELECTION** - User chooses audio track
|
||||
|
||||
---
|
||||
|
||||
## 📞 Getting Help
|
||||
|
||||
1. Check `00-current-plan/CURRENT-STATUS.md` first
|
||||
2. Look for specific guides in `03-deployment/`
|
||||
3. Review session history in `02-implementation-history/`
|
||||
|
||||
### Common Issues
|
||||
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| Emulator not starting | Check ports 8080, 4443 are free |
|
||||
| "Missing lyrics video" | Review hasn't happened yet - expected behavior |
|
||||
| Workers not progressing | Check `state_data` in Firestore |
|
||||
| FFmpeg errors | Ensure FFmpeg is in PATH |
|
||||
Start with `CLAUDE.md` in the repo root for essential rules, then reference these docs as needed. Keep this README current when project state changes.
|
||||
@@ -0,0 +1,195 @@
|
||||
# Stripe Setup Guide
|
||||
|
||||
This guide walks you through setting up Stripe for the Nomad Karaoke payment system.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A Stripe account (sign up at https://stripe.com)
|
||||
- Access to Google Cloud Secret Manager (for secure credential storage)
|
||||
- The Nomad Karaoke backend deployed and running
|
||||
|
||||
## 1. Get Your Stripe API Keys
|
||||
|
||||
1. Log in to your Stripe Dashboard: https://dashboard.stripe.com
|
||||
2. Go to **Developers** > **API keys**
|
||||
3. You'll need two keys:
|
||||
- **Publishable key** (starts with `pk_live_` or `pk_test_`)
|
||||
- **Secret key** (starts with `sk_live_` or `sk_test_`)
|
||||
|
||||
> **Important**: Never commit your secret key to version control. Always use environment variables or Secret Manager.
|
||||
|
||||
## 2. Store Keys in Google Cloud Secret Manager
|
||||
|
||||
Store your Stripe keys securely in Secret Manager:
|
||||
|
||||
```bash
|
||||
# Store the secret key
|
||||
echo -n "sk_live_your_key_here" | gcloud secrets create stripe-secret-key --data-file=-
|
||||
|
||||
# Store the webhook secret (after step 3)
|
||||
echo -n "whsec_your_webhook_secret" | gcloud secrets create stripe-webhook-secret --data-file=-
|
||||
```
|
||||
|
||||
## 3. Set Up Webhook Endpoint
|
||||
|
||||
Webhooks notify your backend when payments complete.
|
||||
|
||||
### In Stripe Dashboard:
|
||||
|
||||
1. Go to **Developers** > **Webhooks**
|
||||
2. Click **Add endpoint**
|
||||
3. Enter your endpoint URL: `https://api.nomadkaraoke.com/api/users/webhooks/stripe`
|
||||
4. Select events to listen to:
|
||||
- `checkout.session.completed` (required - adds credits)
|
||||
- `checkout.session.expired` (optional - logging)
|
||||
- `payment_intent.payment_failed` (optional - logging)
|
||||
5. Click **Add endpoint**
|
||||
6. Copy the **Signing secret** (starts with `whsec_`)
|
||||
7. Store it in Secret Manager (see step 2)
|
||||
|
||||
## 4. Configure Environment Variables
|
||||
|
||||
Set these environment variables for your backend:
|
||||
|
||||
```bash
|
||||
# Required
|
||||
STRIPE_SECRET_KEY=sk_live_your_key_here
|
||||
STRIPE_WEBHOOK_SECRET=whsec_your_webhook_secret
|
||||
|
||||
# Optional (defaults shown)
|
||||
FRONTEND_URL=https://gen.nomadkaraoke.com
|
||||
BUY_URL=https://buy.nomadkaraoke.com
|
||||
```
|
||||
|
||||
### For Cloud Run deployment:
|
||||
|
||||
```bash
|
||||
gcloud run services update karaoke-backend \
|
||||
--set-secrets=STRIPE_SECRET_KEY=stripe-secret-key:latest \
|
||||
--set-secrets=STRIPE_WEBHOOK_SECRET=stripe-webhook-secret:latest
|
||||
```
|
||||
|
||||
## 5. Configure Email Service (SendGrid)
|
||||
|
||||
For sending magic link and purchase confirmation emails:
|
||||
|
||||
1. Sign up for SendGrid: https://sendgrid.com
|
||||
2. Create an API key with "Mail Send" permissions
|
||||
3. Store it in Secret Manager:
|
||||
|
||||
```bash
|
||||
echo -n "SG.your_sendgrid_api_key" | gcloud secrets create sendgrid-api-key --data-file=-
|
||||
```
|
||||
|
||||
4. Set environment variables:
|
||||
|
||||
```bash
|
||||
SENDGRID_API_KEY=SG.your_sendgrid_api_key
|
||||
EMAIL_FROM=noreply@nomadkaraoke.com
|
||||
EMAIL_FROM_NAME=Nomad Karaoke
|
||||
```
|
||||
|
||||
## 6. Test the Integration
|
||||
|
||||
### Test Mode
|
||||
|
||||
Use test keys (starting with `pk_test_` and `sk_test_`) for testing:
|
||||
|
||||
1. Use Stripe's test card: `4242 4242 4242 4242`
|
||||
2. Any future expiration date
|
||||
3. Any 3-digit CVC
|
||||
|
||||
### Verify Webhook Delivery
|
||||
|
||||
1. In Stripe Dashboard, go to **Developers** > **Webhooks**
|
||||
2. Click on your endpoint
|
||||
3. View recent webhook attempts and their status
|
||||
4. Use the "Send test webhook" feature to verify connectivity
|
||||
|
||||
## Credit Packages
|
||||
|
||||
The system comes with these default packages:
|
||||
|
||||
| Package | Credits | Price | Per Credit |
|
||||
|---------|---------|-------|------------|
|
||||
| 1 Credit | 1 | $5.00 | $5.00 |
|
||||
| 3 Credits | 3 | $12.00 | $4.00 (20% off) |
|
||||
| 5 Credits | 5 | $17.50 | $3.50 (30% off) |
|
||||
| 10 Credits | 10 | $30.00 | $3.00 (40% off) |
|
||||
|
||||
To modify packages, edit `backend/services/stripe_service.py`:
|
||||
|
||||
```python
|
||||
CREDIT_PACKAGES = {
|
||||
"1_credit": {
|
||||
"credits": 1,
|
||||
"price_cents": 500,
|
||||
"name": "1 Karaoke Credit",
|
||||
"description": "Create 1 professional karaoke video",
|
||||
},
|
||||
# Add more packages...
|
||||
}
|
||||
```
|
||||
|
||||
## Admin Features
|
||||
|
||||
### Issue Free Credits
|
||||
|
||||
Admins can issue free credits via the API:
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.nomadkaraoke.com/api/users/admin/credits \
|
||||
-H "Authorization: Bearer YOUR_ADMIN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"email": "user@example.com",
|
||||
"amount": 5,
|
||||
"reason": "Beta tester reward"
|
||||
}'
|
||||
```
|
||||
|
||||
### View Users
|
||||
|
||||
```bash
|
||||
curl https://api.nomadkaraoke.com/api/users/admin/users \
|
||||
-H "Authorization: Bearer YOUR_ADMIN_TOKEN"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Webhook Not Receiving Events
|
||||
|
||||
1. Verify the endpoint URL is correct and publicly accessible
|
||||
2. Check the webhook signing secret matches
|
||||
3. Ensure your server returns 2xx status codes
|
||||
4. Check Cloud Run logs for errors
|
||||
|
||||
### Credits Not Adding
|
||||
|
||||
1. Verify `checkout.session.completed` event is being received
|
||||
2. Check that `metadata` contains `user_email` and `credits`
|
||||
3. Review backend logs for errors in `stripe_webhook` handler
|
||||
|
||||
### Email Not Sending
|
||||
|
||||
1. Verify SendGrid API key is valid
|
||||
2. Check sender email is verified in SendGrid
|
||||
3. Review SendGrid activity logs
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Never expose secret keys** - Use environment variables or Secret Manager
|
||||
2. **Always verify webhook signatures** - The code does this automatically
|
||||
3. **Use HTTPS** - All Stripe communication must be over HTTPS
|
||||
4. **Monitor webhook logs** - Watch for failed deliveries or suspicious activity
|
||||
5. **Set up alerts** - Configure Stripe to alert on unusual activity
|
||||
|
||||
## Going Live Checklist
|
||||
|
||||
- [ ] Switch from test to live API keys
|
||||
- [ ] Update webhook endpoint to use live signing secret
|
||||
- [ ] Verify email sending works with real addresses
|
||||
- [ ] Test a real purchase with a small amount
|
||||
- [ ] Set up Stripe Radar for fraud protection
|
||||
- [ ] Configure Stripe tax settings if applicable
|
||||
- [ ] Review and enable appropriate Stripe email receipts
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,80 @@
|
||||
# PR #90: Payment Flow and Beta Tester Program
|
||||
|
||||
**Date**: 2025-12-28
|
||||
**Branch**: `feature/payment-flow`
|
||||
**PR**: https://github.com/nomadkaraoke/karaoke-gen/pull/90
|
||||
|
||||
## Summary
|
||||
|
||||
Implemented customer payment system with Stripe and a beta tester program for low-friction user acquisition.
|
||||
|
||||
## What Was Built
|
||||
|
||||
### Payment System
|
||||
- Magic link (passwordless) authentication
|
||||
- Stripe Checkout integration with credit packages ($5-$30)
|
||||
- Session management (7-day inactivity, 30-day absolute expiry)
|
||||
- SendGrid email integration for transactional emails
|
||||
|
||||
### Beta Tester Program
|
||||
- Free credit in exchange for feedback promise
|
||||
- Users must accept they may need to correct lyrics
|
||||
- Users must write promise text (min 10 chars)
|
||||
- Feedback form with ratings (1-5 scale):
|
||||
- Overall experience
|
||||
- Ease of use
|
||||
- Lyrics accuracy
|
||||
- Correction experience
|
||||
- Bonus credit for detailed feedback (50+ chars)
|
||||
- Admin endpoints for viewing feedback and stats
|
||||
|
||||
### Buy Site (buy.nomadkaraoke.com)
|
||||
- Landing page with credit packages
|
||||
- Purple gradient beta tester signup section
|
||||
- Auto-redirect to main app after enrollment
|
||||
|
||||
## Key Files Changed
|
||||
|
||||
```
|
||||
backend/
|
||||
├── models/user.py # User, Session, MagicLink, Feedback models
|
||||
├── services/user_service.py # Auth, credits, beta enrollment
|
||||
├── services/stripe_service.py # Stripe checkout & webhooks
|
||||
├── services/email_service.py # SendGrid + email templates
|
||||
└── api/routes/users.py # All user/auth/beta endpoints
|
||||
|
||||
buy-site/
|
||||
├── app/page.tsx # Landing page with beta signup
|
||||
└── lib/api.ts # API client for beta enrollment
|
||||
|
||||
frontend/
|
||||
└── lib/auth.ts # Updated for magic link auth
|
||||
```
|
||||
|
||||
## CodeRabbit Fixes Addressed
|
||||
|
||||
1. Added try/except for invalid credits metadata in Stripe webhook
|
||||
2. Fixed magic link race condition with Firestore transaction
|
||||
3. Moved CSS @import before @tailwind directives
|
||||
4. Fixed broken Open Graph image URL
|
||||
5. Removed insecure fallback admin privileges
|
||||
6. Bumped version to 0.76.26
|
||||
|
||||
## Environment Variables Required
|
||||
|
||||
```
|
||||
STRIPE_SECRET_KEY=sk_live_...
|
||||
STRIPE_WEBHOOK_SECRET=whsec_...
|
||||
SENDGRID_API_KEY=SG....
|
||||
EMAIL_FROM=noreply@nomadkaraoke.com
|
||||
FRONTEND_URL=https://gen.nomadkaraoke.com
|
||||
BUY_URL=https://buy.nomadkaraoke.com
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Deploy and test Stripe in live mode
|
||||
- [ ] Configure SendGrid domain verification
|
||||
- [ ] Deploy buy-site to GitHub Pages
|
||||
- [ ] Add feedback request trigger after job completion
|
||||
- [ ] Monitor beta tester metrics via admin dashboard
|
||||
@@ -0,0 +1,80 @@
|
||||
# Self-Hosted GitHub Actions Runner on GCP
|
||||
|
||||
**Date**: 2025-12-28
|
||||
**Status**: Deployed and operational
|
||||
|
||||
## Problem
|
||||
|
||||
GitHub-hosted runners have ~14GB disk space. CI jobs were failing with "No space left on device" during:
|
||||
- Docker builds (multi-stage, large dependencies)
|
||||
- Emulator tests (Docker + Java + Firestore/GCS emulators)
|
||||
- Package builds with poetry
|
||||
|
||||
Workarounds like `jlumbroso/free-disk-space` action helped but weren't reliable.
|
||||
|
||||
## Solution
|
||||
|
||||
Deployed a self-hosted runner on GCP with 200GB SSD via Pulumi IaC.
|
||||
|
||||
### Infrastructure Added
|
||||
|
||||
| Resource | Purpose |
|
||||
|----------|---------|
|
||||
| `github-runner` VM | e2-standard-4, 200GB SSD, Debian 12 |
|
||||
| `github-runner` service account | Minimal permissions for runner |
|
||||
| `github-runner-pat` secret | PAT for runner registration |
|
||||
| IAM bindings | Artifact Registry reader, Secret Manager accessor, logging |
|
||||
|
||||
### Jobs Moved to Self-Hosted
|
||||
|
||||
- `package-build-test`
|
||||
- `backend-unit-tests`
|
||||
- `backend-emulator-tests`
|
||||
- `deploy-backend`
|
||||
|
||||
These use `runs-on: [self-hosted, linux, gcp]` with labels.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
### 1. GPG tty errors in startup script
|
||||
|
||||
**Problem**: `gpg --dearmor` failed with "cannot open /dev/tty" in non-interactive startup.
|
||||
|
||||
**Fix**: Add `--batch` flag to all GPG commands.
|
||||
|
||||
### 2. setup-python can't find Python 3.13 on Debian 12
|
||||
|
||||
**Problem**: `actions/setup-python@v5` couldn't find pre-built Python 3.13 for Debian 12 on self-hosted runners.
|
||||
|
||||
**Fix**:
|
||||
1. Install Python 3.13 via pyenv on the runner
|
||||
2. Copy to tool cache: `_work/_tool/Python/3.13.0/x64/`
|
||||
3. Create marker file: `x64.complete`
|
||||
|
||||
### 3. Java 21 not in Debian repos
|
||||
|
||||
**Problem**: `openjdk-21-jdk` not available in Debian 12 stable.
|
||||
|
||||
**Fix**: Use Temurin (Adoptium) repository directly.
|
||||
|
||||
## Key Files
|
||||
|
||||
- `infrastructure/__main__.py` - Runner VM and IAM setup
|
||||
- `infrastructure/README.md` - Setup instructions
|
||||
- `.github/workflows/ci.yml` - Jobs using self-hosted runner
|
||||
|
||||
## Lessons for Future
|
||||
|
||||
1. **Self-hosted runners need explicit tool setup** - Unlike GitHub-hosted, tools aren't pre-installed in expected locations
|
||||
2. **setup-python tool cache location**: `$RUNNER_DIR/_work/_tool/Python/<version>/x64/`
|
||||
3. **GPG in scripts**: Always use `--batch` for non-interactive execution
|
||||
4. **Debian vs Ubuntu**: Many GitHub Actions assume Ubuntu; PPAs don't work on Debian
|
||||
|
||||
## Cost
|
||||
|
||||
~$50/month for e2-standard-4 running 24/7 (covered by GCP free credits).
|
||||
|
||||
## PRs
|
||||
|
||||
- #95: Initial self-hosted runner setup
|
||||
- #97: Make startup script reproducible (pyenv, tool cache, GPG fixes)
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,197 @@
|
||||
# GitHub Actions Docker Builds - Cost Optimization
|
||||
|
||||
**Date:** 2025-12-28
|
||||
**Problem:** Cloud Build costing $100+/month with slow builds
|
||||
**Solution:** Move Docker builds to GitHub Actions with registry caching
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the migration from Google Cloud Build to GitHub Actions for Docker image builds. This change eliminates Cloud Build costs while improving build speeds through better caching.
|
||||
|
||||
## Cost Comparison
|
||||
|
||||
| Approach | Monthly Cost | Build Speed | Caching |
|
||||
|----------|-------------|-------------|---------|
|
||||
| **Cloud Build (N1_HIGHCPU_32)** | ~$100+ | 5-15 min | Limited |
|
||||
| **GitHub Actions + Registry Cache** | **$0** | 2-5 min | Excellent |
|
||||
|
||||
### Why Cloud Build Was Expensive
|
||||
|
||||
1. **N1_HIGHCPU_32 pricing**: ~$0.064/build-minute (21x more than default)
|
||||
2. **No persistent layer cache**: Each build re-downloaded dependencies
|
||||
3. **Cold start overhead**: Spinning up build VMs added latency
|
||||
4. **Large dependency tree**: torch, transformers, etc. = ~500 packages
|
||||
|
||||
### Why GitHub Actions Is Free
|
||||
|
||||
- **Public repositories**: Unlimited build minutes on GitHub-hosted runners
|
||||
- **Registry cache**: Layers cached in Artifact Registry, persist between builds
|
||||
- **BuildKit mode=max**: Caches ALL intermediate layers, not just final
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ NEW BUILD PIPELINE │
|
||||
├─────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ GitHub Actions (Free) │
|
||||
│ ├── docker/setup-buildx-action (BuildKit) │
|
||||
│ ├── google-github-actions/auth (Workload Identity) │
|
||||
│ └── docker/build-push-action │
|
||||
│ ├── cache-from: type=registry,ref=...karaoke-backend:cache │
|
||||
│ └── cache-to: type=registry,ref=...karaoke-backend:cache,mode=max│
|
||||
│ │
|
||||
│ Artifact Registry │
|
||||
│ ├── karaoke-backend:latest (production image) │
|
||||
│ ├── karaoke-backend:cache (build cache - ALL layers) │
|
||||
│ ├── karaoke-backend-base:latest (base image with deps) │
|
||||
│ └── karaoke-backend-base:cache (base cache) │
|
||||
│ │
|
||||
│ Build Times (Expected): │
|
||||
│ ├── First build (cold): ~8-10 min │
|
||||
│ ├── Code-only changes: ~2-3 min (cache hit) │
|
||||
│ └── Dependency changes: ~6-8 min (partial cache hit) │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
### Two-Stage Build Pattern
|
||||
|
||||
We maintain the existing two-stage build pattern for efficiency:
|
||||
|
||||
1. **Base Image** (`karaoke-backend-base`)
|
||||
- Contains Python 3.11, ffmpeg, system dependencies
|
||||
- All pip dependencies pre-installed
|
||||
- Only rebuilt when `poetry.lock` or `Dockerfile.base` changes
|
||||
- Hash-based change detection via image labels
|
||||
|
||||
2. **App Image** (`karaoke-backend`)
|
||||
- Built FROM the base image
|
||||
- Contains only application code
|
||||
- Fast rebuilds (~2 min) for code-only changes
|
||||
|
||||
### Registry Cache
|
||||
|
||||
The key optimization is using Artifact Registry as a persistent layer cache:
|
||||
|
||||
```yaml
|
||||
- uses: docker/build-push-action@v6
|
||||
with:
|
||||
cache-from: type=registry,ref=.../karaoke-backend:cache
|
||||
cache-to: type=registry,ref=.../karaoke-backend:cache,mode=max
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- `mode=max` caches ALL layers, not just the final image
|
||||
- Cache persists between CI runs (unlike GitHub Actions cache which has 10GB limit)
|
||||
- Layers only re-built when inputs change
|
||||
|
||||
---
|
||||
|
||||
## Files Changed
|
||||
|
||||
### New Files
|
||||
|
||||
- **`.dockerignore`** - Reduces build context from ~1GB to ~10MB
|
||||
|
||||
### Modified Files
|
||||
|
||||
- **`.github/workflows/ci.yml`** - `deploy-backend` job rewritten to use:
|
||||
- `docker/setup-buildx-action@v3`
|
||||
- `docker/build-push-action@v6`
|
||||
- Registry caching to Artifact Registry
|
||||
|
||||
### Kept (as fallback)
|
||||
|
||||
- **`cloudbuild.yaml`** - Still works for manual triggers
|
||||
- **`cloudbuild-base.yaml`** - Still works for manual base image builds
|
||||
|
||||
---
|
||||
|
||||
## Manual Builds (Fallback)
|
||||
|
||||
If GitHub Actions fails, you can still use Cloud Build:
|
||||
|
||||
```bash
|
||||
# Build base image (only if dependencies changed)
|
||||
gcloud builds submit --config=cloudbuild-base.yaml --project=nomadkaraoke
|
||||
|
||||
# Build and deploy app image
|
||||
gcloud builds submit --config=cloudbuild.yaml --project=nomadkaraoke
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Check Build Cache Usage
|
||||
|
||||
View cache images in Artifact Registry:
|
||||
|
||||
```bash
|
||||
gcloud artifacts docker images list \
|
||||
us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo \
|
||||
--include-tags
|
||||
```
|
||||
|
||||
You should see `*:cache` tags for both base and app images.
|
||||
|
||||
### Compare Build Times
|
||||
|
||||
- **GitHub Actions**: Check workflow run times in Actions tab
|
||||
- **Previous Cloud Build**: ~5-15 min per build
|
||||
- **Expected with cache**: ~2-3 min for code changes
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Cache Miss (Slow Build)
|
||||
|
||||
If builds are slow despite no dependency changes:
|
||||
|
||||
1. Check if cache image exists:
|
||||
```bash
|
||||
docker pull us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend:cache
|
||||
```
|
||||
|
||||
2. Force cache rebuild by pushing to `main` with a small change
|
||||
|
||||
### Disk Space Issues
|
||||
|
||||
GitHub Actions runners have limited disk. If base image builds fail:
|
||||
|
||||
1. The workflow includes `jlumbroso/free-disk-space` action
|
||||
2. It removes Android SDK, Haskell, .NET, etc. to free ~20GB
|
||||
|
||||
### Authentication Issues
|
||||
|
||||
If `docker push` fails with permission errors:
|
||||
|
||||
1. Check Workload Identity is configured correctly
|
||||
2. Verify service account has `roles/artifactregistry.writer`
|
||||
|
||||
---
|
||||
|
||||
## Future Improvements
|
||||
|
||||
1. **Parallel base/app builds**: If base is unchanged, start app build immediately
|
||||
2. **GitHub Actions cache fallback**: Use GHA cache when registry is slow
|
||||
3. **Build metrics**: Track build times and cache hit rates
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [Docker Build with GitHub Actions](https://docs.docker.com/build/ci/github-actions/)
|
||||
- [Registry Cache Backend](https://docs.docker.com/build/cache/backends/registry/)
|
||||
- [Cache Management](https://docs.docker.com/build/ci/github-actions/cache/)
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loaded 100 of 198 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user