mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 21:17:56 +09:00
Compare commits
209
Commits
No files matched your search
@@ -1,74 +0,0 @@
|
||||
---
|
||||
description: Periodic documentation maintenance and organization check
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write, Bash
|
||||
---
|
||||
|
||||
# Documentation Maintenance Check
|
||||
|
||||
You are performing periodic maintenance on the project documentation.
|
||||
|
||||
## Checks to Perform
|
||||
|
||||
### 1. Structure Compliance
|
||||
Verify docs/ follows the correct structure:
|
||||
```
|
||||
docs/
|
||||
├── README.md # Current status + navigation
|
||||
├── ARCHITECTURE.md # System design
|
||||
├── DEVELOPMENT.md # Dev setup, testing, deployment
|
||||
├── API.md # Backend API reference
|
||||
├── LESSONS-LEARNED.md # Accumulated wisdom
|
||||
└── archive/ # Historical docs (YYYY-MM-DD-topic.md)
|
||||
```
|
||||
|
||||
Report any files that don't belong or are misplaced.
|
||||
|
||||
### 2. Freshness Check
|
||||
- Read `docs/README.md` - is the status section current?
|
||||
- Check recent git commits - do they suggest docs should be updated?
|
||||
- Look for stale information that contradicts current code
|
||||
|
||||
### 3. Archive Organization
|
||||
- Verify all archive files have YYYY-MM-DD prefix
|
||||
- Check for any docs that should be archived (completed work, old plans)
|
||||
- Remove truly obsolete content (empty files, duplicate info)
|
||||
|
||||
### 4. Cross-Reference Check
|
||||
- Verify CLAUDE.md points to current doc locations
|
||||
- Check that docs reference each other correctly
|
||||
- Ensure no broken internal links
|
||||
|
||||
### 5. Content Quality
|
||||
- Flag any docs over 500 lines (may need splitting)
|
||||
- Identify duplicate information across docs
|
||||
- Note sections that seem outdated
|
||||
|
||||
## Actions to Take
|
||||
|
||||
1. **Fix structural issues** - Move/rename misplaced files
|
||||
2. **Update stale content** - Refresh outdated information
|
||||
3. **Archive completed work** - Move old docs to archive/
|
||||
4. **Report findings** - Summarize what was found and fixed
|
||||
|
||||
## Output Format
|
||||
|
||||
Provide a maintenance report:
|
||||
```
|
||||
## Documentation Maintenance Report - YYYY-MM-DD
|
||||
|
||||
### Issues Found
|
||||
- [list of issues]
|
||||
|
||||
### Actions Taken
|
||||
- [list of fixes made]
|
||||
|
||||
### Recommendations
|
||||
- [suggestions for future attention]
|
||||
|
||||
### Status
|
||||
- [ ] Structure: OK/Fixed
|
||||
- [ ] Freshness: OK/Updated
|
||||
- [ ] Archive: OK/Organized
|
||||
- [ ] Cross-refs: OK/Fixed
|
||||
- [ ] Quality: OK/Flagged
|
||||
```
|
||||
@@ -1,43 +0,0 @@
|
||||
---
|
||||
description: Review current session for documentation updates before merging PR
|
||||
allowed-tools: Read, Glob, Grep, Edit, Write
|
||||
---
|
||||
|
||||
# Pre-Merge Documentation Review
|
||||
|
||||
You are reviewing this chat session to identify documentation updates needed before merging the current PR.
|
||||
|
||||
## Your Task
|
||||
|
||||
1. **Analyze the current session** - What work was done? What was learned? What changed?
|
||||
|
||||
2. **Check if docs need updates:**
|
||||
- Does `docs/README.md` status section need updating?
|
||||
- Should anything be added to `docs/LESSONS-LEARNED.md`?
|
||||
- Does `docs/ARCHITECTURE.md` need changes (if architecture changed)?
|
||||
- Does `docs/DEVELOPMENT.md` need changes (if dev workflow changed)?
|
||||
- Does `docs/API.md` need changes (if API changed)?
|
||||
|
||||
3. **For significant completed work:**
|
||||
- Create `docs/archive/YYYY-MM-DD-topic.md` summarizing the work
|
||||
- Use today's date and a descriptive topic name
|
||||
|
||||
4. **Update docs/README.md** with:
|
||||
- Current status if it changed
|
||||
- Any new known issues discovered
|
||||
- Link to new archive doc if created
|
||||
|
||||
## Guidelines
|
||||
|
||||
- Keep updates concise and factual
|
||||
- Focus on information useful for future AI agents
|
||||
- Don't duplicate information that's in the code
|
||||
- Archive docs should capture "what was done and why" not implementation details
|
||||
|
||||
## Output
|
||||
|
||||
After reviewing, either:
|
||||
- Make the necessary documentation updates
|
||||
- Report "No documentation updates needed" with brief explanation why
|
||||
|
||||
Remember: Good documentation helps the next agent continue where you left off.
|
||||
@@ -0,0 +1,106 @@
|
||||
# Plan: Migrate from MailSlurp to testmail.app
|
||||
|
||||
**Created:** 2026-01-02
|
||||
**Branch:** feat/sess-20260102-2109-migrate-mailslurp-testmail
|
||||
**Status:** Complete
|
||||
|
||||
## Overview
|
||||
|
||||
Replace MailSlurp email testing service with testmail.app for E2E testing of magic link authentication. testmail.app uses a simpler API with namespace-based email addresses that require no inbox creation/deletion - emails just arrive at `{namespace}.{tag}@inbox.testmail.app`.
|
||||
|
||||
## Requirements
|
||||
|
||||
- [x] Remove all code referencing MailSlurp
|
||||
- [x] Remove `mailslurp-client` npm dependency
|
||||
- [x] Implement testmail.app API integration
|
||||
- [x] Update environment variable names (`TESTMAIL_API_KEY`, `TESTMAIL_NAMESPACE`)
|
||||
- [x] Maintain same `EmailHelper` interface for minimal test changes
|
||||
- [x] Update archive docs to note migration (optional - historical accuracy)
|
||||
|
||||
## Technical Approach
|
||||
|
||||
### API Differences
|
||||
|
||||
| Feature | MailSlurp | testmail.app |
|
||||
|---------|-----------|--------------|
|
||||
| Email format | Random `xxx@mailslurp.com` | `{namespace}.{tag}@inbox.testmail.app` |
|
||||
| Inbox creation | Required (`createInbox()`) | Not needed - use any tag |
|
||||
| Inbox deletion | Required (`deleteInbox()`) | Not needed |
|
||||
| Get emails | `waitForLatestEmail(inboxId)` | GET `/api/json?apikey=X&namespace=Y&tag=Z&livequery=true` |
|
||||
| Auth | API key in constructor | API key as query param |
|
||||
| Email body | `email.body` | `email.html` or `email.text` |
|
||||
|
||||
### Implementation Strategy
|
||||
|
||||
1. **Keep same `EmailHelper` interface** - `createInbox()`, `waitForEmail()`, `extractMagicLink()`, `deleteInbox()`
|
||||
2. **Use testmail.app's live query feature** - `livequery=true` waits for email to arrive (up to timeout)
|
||||
3. **Generate unique tags** - Use timestamp + random suffix for each test run to avoid collisions
|
||||
4. **Inbox becomes a simple object** - `{ id: tag, emailAddress: '{namespace}.{tag}@inbox.testmail.app' }`
|
||||
5. **Delete becomes no-op** - testmail.app emails auto-expire, no cleanup needed
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Old | New |
|
||||
|-----|-----|
|
||||
| `MAILSLURP_API_KEY` | `TESTMAIL_API_KEY` |
|
||||
| (none) | `TESTMAIL_NAMESPACE` |
|
||||
|
||||
Values: Set via environment variables (see `frontend/.env.local.example`)
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. [x] **Update `email-testing.ts`** - Replace MailSlurp implementation with testmail.app
|
||||
- Remove `mailslurp-client` import
|
||||
- Add testmail.app API fetch implementation
|
||||
- Use native `fetch` (no new dependencies)
|
||||
- Generate unique tag for each "inbox"
|
||||
- Implement `livequery=true` for waiting
|
||||
- Adapt email body extraction (`html`/`text` instead of `body`)
|
||||
|
||||
2. [x] **Update `full-user-journey.spec.ts`** - Change references
|
||||
- Update doc comments (env var name)
|
||||
- Update error messages
|
||||
- Update console logs
|
||||
|
||||
3. [x] **Update `playwright.record.config.ts`** - Change env var comment
|
||||
|
||||
4. [x] **Update `package.json`** - Remove `mailslurp-client` dependency
|
||||
|
||||
5. [x] **Run `npm install`** - Update package-lock.json
|
||||
|
||||
6. [x] **Create `.env.local.example`** - Document required env vars for local testing
|
||||
|
||||
7. [x] **Update archive docs** - Skipped intentionally; historical docs preserved for accuracy
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
|------|--------|-------------|
|
||||
| `frontend/e2e/helpers/email-testing.ts` | Modify | Replace MailSlurp with testmail.app |
|
||||
| `frontend/e2e/production/full-user-journey.spec.ts` | Modify | Update env var refs and messages |
|
||||
| `frontend/playwright.record.config.ts` | Modify | Update env var comment |
|
||||
| `frontend/package.json` | Modify | Remove `mailslurp-client` |
|
||||
| `frontend/package-lock.json` | Modify | Auto-updated by npm install |
|
||||
| `frontend/.env.local.example` | Create | Document env vars for local testing |
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Manual testing**: Run production E2E test with testmail.app credentials
|
||||
```bash
|
||||
cd frontend
|
||||
TESTMAIL_API_KEY=your-api-key \
|
||||
TESTMAIL_NAMESPACE=your-namespace \
|
||||
npm run test:e2e:prod:headed
|
||||
```
|
||||
- **Verify**: Email received, magic link extracted, auth flow completes
|
||||
- **CI testing**: Env vars already configured in GitHub Actions org secrets
|
||||
|
||||
## Open Questions
|
||||
|
||||
None - requirements are clear.
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
1. Revert the commit
|
||||
2. Run `npm install` to restore mailslurp-client
|
||||
3. Re-add MAILSLURP_API_KEY to secrets if removed
|
||||
+317
-45
@@ -4,6 +4,7 @@ on:
|
||||
push:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
types: [opened, synchronize, reopened, labeled]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
@@ -97,6 +98,155 @@ jobs:
|
||||
path: frontend/coverage/
|
||||
retention-days: 7
|
||||
|
||||
frontend-e2e-smoke:
|
||||
name: "Frontend - E2E Smoke (~2 min)"
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
# Quick smoke tests run on every PR with frontend changes
|
||||
if: github.ref != 'refs/heads/main' && needs.changes.outputs.frontend == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Run smoke tests only
|
||||
working-directory: frontend
|
||||
run: npx playwright test smoke.spec.ts --project=chromium --reporter=line
|
||||
|
||||
- name: Upload Playwright report
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: playwright-smoke-report
|
||||
path: frontend/playwright-report/
|
||||
retention-days: 7
|
||||
|
||||
frontend-e2e-full:
|
||||
name: "Frontend - E2E Full"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
# Full E2E suite only runs when PR has 'e2e' label (on-demand)
|
||||
if: >-
|
||||
github.ref != 'refs/heads/main' &&
|
||||
needs.changes.outputs.frontend == 'true' &&
|
||||
contains(github.event.pull_request.labels.*.name, 'e2e')
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js with caching
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Run full E2E tests
|
||||
working-directory: frontend
|
||||
run: npx playwright test --reporter=list -x
|
||||
|
||||
- name: Upload Playwright report
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: playwright-report
|
||||
path: frontend/playwright-report/
|
||||
retention-days: 7
|
||||
|
||||
# Happy Path Production Test - runs the comprehensive real-user test against prod
|
||||
frontend-e2e-happy-path:
|
||||
name: "Frontend - E2E Happy Path (Production)"
|
||||
runs-on: ubuntu-latest
|
||||
needs: changes
|
||||
# Only runs when PR has 'e2e-happy-path' label
|
||||
if: >-
|
||||
github.ref != 'refs/heads/main' &&
|
||||
contains(github.event.pull_request.labels.*.name, 'e2e-happy-path')
|
||||
timeout-minutes: 90
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Create test-results directory
|
||||
working-directory: frontend
|
||||
run: mkdir -p test-results
|
||||
|
||||
- name: Run Happy Path Production Test
|
||||
working-directory: frontend
|
||||
env:
|
||||
MAILSLURP_API_KEY: ${{ secrets.MAILSLURP_API_KEY }}
|
||||
E2E_TEST_TOKEN: ${{ secrets.E2E_TEST_TOKEN }}
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
run: |
|
||||
echo "Starting E2E Happy Path Test against Production"
|
||||
echo "Time: $(date)"
|
||||
npx playwright test happy-path-real-user.spec.ts \
|
||||
--config=playwright.production.config.ts \
|
||||
--project=chromium \
|
||||
--reporter=list \
|
||||
--timeout=3600000
|
||||
|
||||
- name: Upload test artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-happy-path-results
|
||||
path: |
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
retention-days: 30
|
||||
|
||||
frontend-build:
|
||||
name: "Frontend - Build"
|
||||
runs-on: ubuntu-latest
|
||||
@@ -128,10 +278,27 @@ jobs:
|
||||
|
||||
package-unit-tests:
|
||||
name: "Package - Unit Tests (70% coverage)"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
@@ -186,10 +353,27 @@ jobs:
|
||||
|
||||
package-integration-tests:
|
||||
name: "Package - Integration Tests"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
@@ -243,6 +427,23 @@ jobs:
|
||||
(needs.package-integration-tests.result == 'success' || needs.package-integration-tests.result == 'skipped') &&
|
||||
(needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
@@ -276,6 +477,23 @@ jobs:
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -313,6 +531,23 @@ jobs:
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true')
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -356,10 +591,27 @@ jobs:
|
||||
|
||||
backend-lint:
|
||||
name: "Backend - Code Quality"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
needs: changes
|
||||
if: github.ref != 'refs/heads/main' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true')
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -400,13 +652,16 @@ jobs:
|
||||
ci-gate:
|
||||
name: "CI Gate"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [frontend-unit-tests, frontend-build, package-unit-tests, package-integration-tests, backend-unit-tests, backend-emulator-tests, backend-lint]
|
||||
# Note: frontend-e2e-full is optional (requires 'e2e' label) so not included here
|
||||
# frontend-e2e-smoke IS required - fast smoke tests catch critical regressions
|
||||
needs: [frontend-unit-tests, frontend-e2e-smoke, frontend-build, package-unit-tests, package-integration-tests, backend-unit-tests, backend-emulator-tests, backend-lint]
|
||||
if: github.ref != 'refs/heads/main' && always()
|
||||
steps:
|
||||
- name: Check results
|
||||
run: |
|
||||
results=(
|
||||
"${{ needs.frontend-unit-tests.result }}"
|
||||
"${{ needs.frontend-e2e-smoke.result }}"
|
||||
"${{ needs.frontend-build.result }}"
|
||||
"${{ needs.package-unit-tests.result }}"
|
||||
"${{ needs.package-integration-tests.result }}"
|
||||
@@ -429,9 +684,26 @@ jobs:
|
||||
|
||||
package-build:
|
||||
name: "Package - Build"
|
||||
runs-on: ubuntu-latest
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
steps:
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
@@ -624,6 +896,10 @@ jobs:
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
|
||||
# NOTE: buy-site was consolidated into the main frontend (PR #122, Dec 2024)
|
||||
# Landing page and payment flow are now at gen.nomadkaraoke.com (root)
|
||||
# buy.nomadkaraoke.com subdomain has been deleted
|
||||
|
||||
deploy-backend:
|
||||
name: "Deploy - Backend (Cloud Run)"
|
||||
runs-on: [self-hosted, linux, gcp]
|
||||
@@ -638,14 +914,27 @@ jobs:
|
||||
BASE_IMAGE_NAME: karaoke-backend-base
|
||||
APP_IMAGE_NAME: karaoke-backend
|
||||
steps:
|
||||
# ============================================
|
||||
# Check if deployment is needed FIRST
|
||||
# ============================================
|
||||
- name: Check disk space
|
||||
run: |
|
||||
echo "Checking disk space..."
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "Current disk usage: ${USAGE}%"
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
echo "Disk usage high, running cleanup..."
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
echo "After cleanup: ${USAGE}%"
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage ${USAGE}% exceeds 85% even after cleanup. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
fetch-depth: 0 # Need full history for tag check
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
@@ -655,26 +944,15 @@ jobs:
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
|
||||
- name: Extract version and check if deployment needed
|
||||
- name: Extract version
|
||||
id: version_check
|
||||
run: |
|
||||
VERSION=$(poetry version --short)
|
||||
echo "version=$VERSION" >> $GITHUB_OUTPUT
|
||||
echo "📋 Current version: $VERSION"
|
||||
|
||||
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
|
||||
echo "needs_deploy=false" >> $GITHUB_OUTPUT
|
||||
echo "⏭️ Tag v$VERSION already exists - skipping backend deployment"
|
||||
echo " (Bump version in pyproject.toml to trigger a new deployment)"
|
||||
else
|
||||
echo "needs_deploy=true" >> $GITHUB_OUTPUT
|
||||
echo "✨ Tag v$VERSION does not exist - proceeding with deployment"
|
||||
fi
|
||||
|
||||
# No disk cleanup needed - self-hosted runner has 200GB SSD
|
||||
echo "🚀 Deploying on every push to main"
|
||||
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
id: auth
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
@@ -682,13 +960,11 @@ jobs:
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Configure Docker for Artifact Registry
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
run: gcloud auth configure-docker ${{ env.REGISTRY }} --quiet
|
||||
|
||||
# ============================================
|
||||
@@ -698,11 +974,9 @@ jobs:
|
||||
# ============================================
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Calculate base image hash
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
id: base-hash
|
||||
run: |
|
||||
# Hash both poetry.lock AND Dockerfile.base - rebuild if either changes
|
||||
@@ -711,14 +985,24 @@ jobs:
|
||||
echo "📋 Base content hash: $CURRENT_HASH"
|
||||
|
||||
- name: Check if base image exists and is current
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
id: check-base
|
||||
run: |
|
||||
BASE_IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.BASE_IMAGE_NAME }}:latest"
|
||||
|
||||
# Try to get the existing base image's hash label
|
||||
# First check if image exists locally (fast, no network call)
|
||||
# Only pull from registry if not cached locally
|
||||
EXISTING_HASH=""
|
||||
if docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
if docker image inspect $BASE_IMAGE >/dev/null 2>&1; then
|
||||
echo "✅ Base image found in local cache (skipping slow registry pull)"
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
if [ -z "$EXISTING_HASH" ]; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
|
||||
fi
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
elif docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
echo "📥 Base image pulled from registry"
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
if [ -z "$EXISTING_HASH" ]; then
|
||||
@@ -738,7 +1022,7 @@ jobs:
|
||||
fi
|
||||
|
||||
- name: Build and push base image (if needed)
|
||||
if: steps.version_check.outputs.needs_deploy == 'true' && steps.check-base.outputs.needs_rebuild == 'true'
|
||||
if: steps.check-base.outputs.needs_rebuild == 'true'
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
@@ -759,7 +1043,6 @@ jobs:
|
||||
provenance: false
|
||||
|
||||
- name: Build and push app image
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
@@ -780,7 +1063,6 @@ jobs:
|
||||
# ============================================
|
||||
|
||||
- name: Deploy to Cloud Run
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
run: |
|
||||
VERSION="${{ steps.version_check.outputs.version }}"
|
||||
IMAGE="${{ env.REGISTRY }}/${{ env.PROJECT_ID }}/${{ env.REPOSITORY }}/${{ env.APP_IMAGE_NAME }}:${{ github.sha }}"
|
||||
@@ -801,12 +1083,11 @@ jobs:
|
||||
--timeout 1800 \
|
||||
--concurrency 1 \
|
||||
--max-instances 50 \
|
||||
--min-instances 0 \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest"
|
||||
--min-instances 4 \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=${{ env.PROJECT_ID }},GCS_BUCKET_NAME=karaoke-gen-storage-${{ env.PROJECT_ID }},FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD,USE_GCE_ENCODING=true" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest,SENDGRID_API_KEY=sendgrid-api-key:latest,STRIPE_SECRET_KEY=stripe-secret-key:latest,STRIPE_WEBHOOK_SECRET=stripe-webhook-secret:latest,LANGFUSE_PUBLIC_KEY=langfuse-public-key:latest,LANGFUSE_SECRET_KEY=langfuse-secret-key:latest,LANGFUSE_HOST=langfuse-host:latest,ENCODING_WORKER_URL=encoding-worker-url:latest,ENCODING_WORKER_API_KEY=encoding-worker-api-key:latest"
|
||||
|
||||
- name: Verify deployment and version
|
||||
if: steps.version_check.outputs.needs_deploy == 'true'
|
||||
run: |
|
||||
EXPECTED_VERSION="${{ steps.version_check.outputs.version }}"
|
||||
SERVICE_URL=$(gcloud run services describe karaoke-backend \
|
||||
@@ -847,7 +1128,7 @@ jobs:
|
||||
done
|
||||
|
||||
- name: Summary
|
||||
if: steps.version_check.outputs.needs_deploy == 'true' && success()
|
||||
if: success()
|
||||
run: |
|
||||
echo "🎉 Deployment successful!"
|
||||
echo "Version: ${{ steps.version_check.outputs.version }}"
|
||||
@@ -857,12 +1138,3 @@ jobs:
|
||||
echo ""
|
||||
echo "📊 Built on self-hosted GCP runner (200GB SSD, no disk space issues)"
|
||||
echo " Runner: gcp-runner with 4 vCPU, 16GB RAM"
|
||||
|
||||
- name: Skipped deployment summary
|
||||
if: steps.version_check.outputs.needs_deploy == 'false'
|
||||
run: |
|
||||
echo "⏭️ Backend deployment skipped - version ${{ steps.version_check.outputs.version }} already deployed"
|
||||
echo ""
|
||||
echo "To deploy a new version:"
|
||||
echo " 1. Update version in pyproject.toml"
|
||||
echo " 2. Commit and push to main"
|
||||
@@ -0,0 +1,172 @@
|
||||
name: E2E Happy Path Test
|
||||
|
||||
# Comprehensive end-to-end test that validates the full karaoke generation journey
|
||||
# using ONLY browser interactions (no API shortcuts)
|
||||
#
|
||||
# See docs/E2E-HAPPY-PATH-TEST-SPEC.md for full requirements
|
||||
|
||||
on:
|
||||
# Daily run at 6 AM UTC
|
||||
schedule:
|
||||
- cron: '0 6 * * *'
|
||||
|
||||
# Manual trigger with optional parameters
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
debug_mode:
|
||||
description: 'Enable debug mode (slower, more screenshots)'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: boolean
|
||||
browser:
|
||||
description: 'Browser to test with'
|
||||
required: false
|
||||
default: 'chromium'
|
||||
type: choice
|
||||
options:
|
||||
- chromium
|
||||
- firefox
|
||||
- webkit
|
||||
use_test_token:
|
||||
description: 'Skip signup and use E2E_TEST_TOKEN (for iterations)'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: boolean
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
e2e-happy-path:
|
||||
name: "Full Happy Path (Real User)"
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 90 # 1.5 hours max (test itself is 60 min)
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: frontend
|
||||
run: npm ci --legacy-peer-deps
|
||||
|
||||
- name: Install Playwright browsers
|
||||
working-directory: frontend
|
||||
run: |
|
||||
BROWSER=${{ inputs.browser || 'chromium' }}
|
||||
npx playwright install --with-deps $BROWSER
|
||||
|
||||
- name: Create test-results directory
|
||||
working-directory: frontend
|
||||
run: mkdir -p test-results
|
||||
|
||||
- name: Run E2E Happy Path Test
|
||||
working-directory: frontend
|
||||
env:
|
||||
MAILSLURP_API_KEY: ${{ secrets.MAILSLURP_API_KEY }}
|
||||
E2E_TEST_TOKEN: ${{ inputs.use_test_token == 'true' && secrets.E2E_TEST_TOKEN || '' }}
|
||||
E2E_ADMIN_TOKEN: ${{ secrets.E2E_ADMIN_TOKEN }}
|
||||
DEBUG_MODE: ${{ inputs.debug_mode || 'false' }}
|
||||
run: |
|
||||
echo "Starting E2E Happy Path Test"
|
||||
echo "Browser: ${{ inputs.browser || 'chromium' }}"
|
||||
echo "Debug mode: ${{ inputs.debug_mode || 'false' }}"
|
||||
echo "Use test token: ${{ inputs.use_test_token || 'false' }}"
|
||||
echo "Time: $(date)"
|
||||
echo ""
|
||||
|
||||
# Run the real user happy path test
|
||||
npx playwright test happy-path-real-user.spec.ts \
|
||||
--config=playwright.production.config.ts \
|
||||
--project=${{ inputs.browser || 'chromium' }} \
|
||||
--reporter=list \
|
||||
--timeout=3600000 \
|
||||
2>&1 | tee test-output.log
|
||||
continue-on-error: true
|
||||
|
||||
- name: Check test result
|
||||
id: test_result
|
||||
working-directory: frontend
|
||||
run: |
|
||||
if grep -q "passed" test-output.log && ! grep -q "failed" test-output.log; then
|
||||
echo "result=success" >> $GITHUB_OUTPUT
|
||||
echo "Test PASSED"
|
||||
else
|
||||
echo "result=failure" >> $GITHUB_OUTPUT
|
||||
echo "Test FAILED - see artifacts for details"
|
||||
fi
|
||||
|
||||
- name: Upload test artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-happy-path-results-${{ github.run_number }}
|
||||
path: |
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
frontend/test-output.log
|
||||
retention-days: 30
|
||||
|
||||
- name: Upload video recordings
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-happy-path-videos-${{ github.run_number }}
|
||||
path: frontend/test-results/**/*.webm
|
||||
retention-days: 30
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: Create summary
|
||||
if: always()
|
||||
run: |
|
||||
echo "## E2E Happy Path Test Results" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Property | Value |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "|----------|-------|" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Run | #${{ github.run_number }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Trigger | ${{ github.event_name }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Browser | ${{ inputs.browser || 'chromium' }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "| Result | ${{ steps.test_result.outputs.result || 'unknown' }} |" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
if [ -f frontend/test-output.log ]; then
|
||||
echo "### Test Output (last 100 lines)" >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
|
||||
tail -100 frontend/test-output.log >> $GITHUB_STEP_SUMMARY
|
||||
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
|
||||
- name: Send email notification
|
||||
if: always() && github.event_name == 'schedule'
|
||||
uses: dawidd6/action-send-mail@v3
|
||||
with:
|
||||
server_address: smtp.sendgrid.net
|
||||
server_port: 587
|
||||
username: apikey
|
||||
password: ${{ secrets.SENDGRID_API_KEY }}
|
||||
subject: "E2E Happy Path Test ${{ steps.test_result.outputs.result == 'success' && 'PASSED' || 'FAILED' }} - Run #${{ github.run_number }}"
|
||||
to: andrew@nomadkaraoke.com
|
||||
from: "Karaoke Gen CI <noreply@nomadkaraoke.com>"
|
||||
body: |
|
||||
E2E Happy Path Test Results
|
||||
===========================
|
||||
|
||||
Status: ${{ steps.test_result.outputs.result == 'success' && 'PASSED ✅' || 'FAILED ❌' }}
|
||||
Run: #${{ github.run_number }}
|
||||
Trigger: ${{ github.event_name }}
|
||||
Time: ${{ github.event.repository.updated_at }}
|
||||
|
||||
View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
|
||||
This is an automated notification from the daily E2E happy path test.
|
||||
|
||||
- name: Fail if test failed
|
||||
if: steps.test_result.outputs.result == 'failure'
|
||||
run: exit 1
|
||||
+3
-1
@@ -47,6 +47,7 @@ frontend/coverage/
|
||||
frontend/test-results/
|
||||
frontend/playwright-report/
|
||||
frontend/playwright/.cache/
|
||||
frontend/e2e/screenshots/
|
||||
|
||||
# TypeScript
|
||||
frontend/*.tsbuildinfo
|
||||
@@ -82,8 +83,9 @@ eggs/
|
||||
# Python lib directories (not frontend)
|
||||
lib/
|
||||
lib64/
|
||||
# But allow frontend lib
|
||||
# But allow frontend lib directories
|
||||
!frontend/lib/
|
||||
!buy-site/lib/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
|
||||
| What | Where |
|
||||
|------|-------|
|
||||
| Product vision & goals | `docs/PRODUCT-VISION.md` |
|
||||
| Current status | `docs/README.md` |
|
||||
| Architecture | `docs/ARCHITECTURE.md` |
|
||||
| Dev setup & testing | `docs/DEVELOPMENT.md` |
|
||||
@@ -28,11 +29,8 @@
|
||||
## Essential Rules
|
||||
|
||||
### Git Workflow
|
||||
- **Never commit directly to `main`** - always use feature branches
|
||||
- **Use git worktrees** for all changes:
|
||||
```bash
|
||||
git worktree add -b feature/xyz ../karaoke-gen-feat-xyz main
|
||||
```
|
||||
- **Never commit directly to `main`** - use `/new-worktree <description>` to start
|
||||
- **Follow global workflow** - see `~/.claude/CLAUDE.md` for command sequence
|
||||
- Create PR with summary, changes, testing info
|
||||
- Merge only after CI passes
|
||||
|
||||
@@ -89,9 +87,12 @@ Location: `/Users/andrew/Projects/flacfetch`
|
||||
|
||||
Workflow: Make changes on `main`, bump version, push, wait for CI, then `poetry update flacfetch` in karaoke-gen.
|
||||
|
||||
## Pre-Commit Checklist
|
||||
## Pre-PR Checklist
|
||||
|
||||
- [ ] Tests pass (`make test` / `npm run test:all`)
|
||||
Follow global workflow (`~/.claude/CLAUDE.md`):
|
||||
|
||||
- [ ] Tests pass: `/test` (or `make test` / `npm run test:all`)
|
||||
- [ ] Version bumped (if code changed)
|
||||
- [ ] `/docs-review` run (check for doc updates)
|
||||
- [ ] PR created with clear description
|
||||
- [ ] Docs checked: `/docs-review`
|
||||
- [ ] Code reviewed: `/review` (CodeRabbit CLI)
|
||||
- [ ] PR created: `/pr` (adds @coderabbitai ignore)
|
||||
@@ -0,0 +1,75 @@
|
||||
[
|
||||
{
|
||||
"test_name": "Sound-alike error (wurld -> world)",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "ReplaceWord",
|
||||
"word_id": null,
|
||||
"word_ids": [
|
||||
"w1"
|
||||
],
|
||||
"replacement_text": "world",
|
||||
"confidence": 0.75,
|
||||
"reason": "Sound-alike error. Reference suggests: 'world'. The transcription 'wurld' is a phonetic misspelling or sound-alike of the word 'world' which appears in both reference sources (Genius and LRCLIB).",
|
||||
"gap_category": "GapCategory.SOUND_ALIKE",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"test_name": "Punctuation difference",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "NoAction",
|
||||
"word_id": null,
|
||||
"word_ids": [
|
||||
"w2"
|
||||
],
|
||||
"replacement_text": null,
|
||||
"confidence": 0.99,
|
||||
"reason": "Transcription matches at least one reference source. The gap text 'Hello' followed by the context 'is it me you're looking for' matches the Genius reference lyrics exactly, excluding punctuation. According to the guidelines, if any reference source matches the transcription text (ignoring punctuation), it should be classified as NO_ERROR.",
|
||||
"gap_category": "GapCategory.NO_ERROR",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"test_name": "Extra filler word",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "DeleteWord",
|
||||
"word_id": "w3",
|
||||
"word_ids": null,
|
||||
"replacement_text": null,
|
||||
"confidence": 0.8,
|
||||
"reason": "Extra filler word 'And' at sentence start not in reference. The transcription includes the filler word 'And' at the beginning of the phrase, which is not present in the reference lyrics. This matches the definition of EXTRA_WORDS.",
|
||||
"gap_category": "GapCategory.EXTRA_WORDS",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"test_name": "Complex multi-word error",
|
||||
"success": true,
|
||||
"proposals": [
|
||||
{
|
||||
"action": "NoAction",
|
||||
"word_id": null,
|
||||
"word_ids": [
|
||||
"w7",
|
||||
"w8",
|
||||
"w9",
|
||||
"w10"
|
||||
],
|
||||
"replacement_text": null,
|
||||
"confidence": 0.99,
|
||||
"reason": "Transcription matches at least one reference source. The transcription text 'gonna give you up' matches the Genius reference lyrics exactly when ignoring punctuation and capitalization. In context, the full phrase 'never gonna give you up' is correctly transcribed and aligns perfectly with the known lyrics and title of the song.",
|
||||
"gap_category": "GapCategory.NO_ERROR",
|
||||
"requires_human_review": false
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
+13
-1
@@ -10,14 +10,26 @@ FROM python:3.11-slim
|
||||
# Note: Transmission daemon removed - torrent downloads now handled by
|
||||
# dedicated flacfetch VM service (see docs/00-current-plan/TORRENT-SERVICE-PLAN.md)
|
||||
RUN apt-get update && apt-get install -y \
|
||||
ffmpeg \
|
||||
libsndfile1 \
|
||||
libsox-dev \
|
||||
sox \
|
||||
build-essential \
|
||||
curl \
|
||||
xz-utils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install optimized static FFmpeg build (7.0.2) instead of Debian package (5.1.8)
|
||||
# John Van Sickle's builds are compiled with better optimizations for x86_64
|
||||
# This provides ~20-40% performance improvement for encoding tasks
|
||||
# Source: https://johnvansickle.com/ffmpeg/
|
||||
RUN curl -L https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz -o /tmp/ffmpeg.tar.xz && \
|
||||
tar -xf /tmp/ffmpeg.tar.xz -C /tmp && \
|
||||
cp /tmp/ffmpeg-*-amd64-static/ffmpeg /usr/local/bin/ && \
|
||||
cp /tmp/ffmpeg-*-amd64-static/ffprobe /usr/local/bin/ && \
|
||||
chmod +x /usr/local/bin/ffmpeg /usr/local/bin/ffprobe && \
|
||||
rm -rf /tmp/ffmpeg* && \
|
||||
ffmpeg -version
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
+182
-70
@@ -4,11 +4,11 @@ FastAPI dependencies for authentication and authorization.
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime, timedelta, UTC
|
||||
from typing import Optional, Tuple, Callable
|
||||
from typing import Optional, Tuple, Callable, Union
|
||||
from fastapi import Depends, HTTPException, Header, Query, Request, Path
|
||||
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
|
||||
|
||||
from backend.services.auth_service import get_auth_service, UserType, AuthService
|
||||
from backend.services.auth_service import get_auth_service, UserType, AuthService, AuthResult
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -66,70 +66,130 @@ async def require_auth(
|
||||
auth_service: AuthService = Depends(get_auth_service),
|
||||
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
|
||||
token: Optional[str] = Query(None)
|
||||
) -> Tuple[str, UserType, int]:
|
||||
) -> AuthResult:
|
||||
"""
|
||||
Require authentication for an endpoint.
|
||||
|
||||
|
||||
Returns:
|
||||
(token, user_type, remaining_uses)
|
||||
|
||||
AuthResult with full authentication context including:
|
||||
- is_valid, user_type, remaining_uses, message (backward compatible via tuple unpacking)
|
||||
- user_email: Email of authenticated user (if session/API key auth)
|
||||
- is_admin: Whether user has admin privileges
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if authentication fails
|
||||
"""
|
||||
# Get token from request
|
||||
token_str = await get_token_from_request(request, credentials, token)
|
||||
|
||||
|
||||
if not token_str:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide token via Authorization header or ?token= parameter"
|
||||
)
|
||||
|
||||
# Validate token
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(token_str)
|
||||
|
||||
if not is_valid:
|
||||
# Log more details for debugging token issues
|
||||
|
||||
# Validate token using the full method
|
||||
auth_result = auth_service.validate_token_full(token_str)
|
||||
|
||||
# Get request_id from middleware for correlation
|
||||
request_id = getattr(request.state, "request_id", None)
|
||||
|
||||
if not auth_result.is_valid:
|
||||
# Log auth failure with request_id for correlation
|
||||
auth_header = request.headers.get("Authorization", "")
|
||||
logger.warning(
|
||||
f"Authentication failed: {message}. "
|
||||
f"Token provided: {bool(token_str)}, "
|
||||
f"Token length: {len(token_str) if token_str else 0}, "
|
||||
f"Auth header present: {bool(auth_header)}, "
|
||||
f"Header prefix: {auth_header[:20] if auth_header else 'none'}..."
|
||||
"auth_failed",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"audit_type": "auth_event",
|
||||
"auth_message": auth_result.message,
|
||||
"token_provided": bool(token_str),
|
||||
"token_length": len(token_str) if token_str else 0,
|
||||
"auth_header_present": bool(auth_header),
|
||||
}
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail=f"Authentication failed: {auth_result.message}"
|
||||
)
|
||||
|
||||
# Log successful auth with request_id for correlation with request audit
|
||||
logger.info(
|
||||
"auth_success",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"user_email": auth_result.user_email,
|
||||
"user_type": auth_result.user_type.value if auth_result.user_type else None,
|
||||
"is_admin": auth_result.is_admin,
|
||||
"remaining_uses": auth_result.remaining_uses,
|
||||
"audit_type": "auth_event",
|
||||
}
|
||||
)
|
||||
|
||||
return auth_result
|
||||
|
||||
|
||||
async def require_auth_legacy(
|
||||
request: Request,
|
||||
auth_service: AuthService = Depends(get_auth_service),
|
||||
credentials: Optional[HTTPAuthorizationCredentials] = Depends(security),
|
||||
token: Optional[str] = Query(None)
|
||||
) -> Tuple[str, UserType, int]:
|
||||
"""
|
||||
Legacy authentication dependency that returns tuple.
|
||||
|
||||
DEPRECATED: Use require_auth which returns AuthResult instead.
|
||||
|
||||
Returns:
|
||||
(token, user_type, remaining_uses)
|
||||
"""
|
||||
token_str = await get_token_from_request(request, credentials, token)
|
||||
|
||||
if not token_str:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide token via Authorization header or ?token= parameter"
|
||||
)
|
||||
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(token_str)
|
||||
|
||||
if not is_valid:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail=f"Authentication failed: {message}"
|
||||
)
|
||||
|
||||
logger.info(f"Authenticated as {user_type} (remaining: {remaining_uses})")
|
||||
|
||||
|
||||
return token_str, user_type, remaining_uses
|
||||
|
||||
|
||||
async def require_admin(
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
) -> Tuple[str, UserType, int]:
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> AuthResult:
|
||||
"""
|
||||
Require admin access for an endpoint.
|
||||
|
||||
|
||||
Admin access is granted if:
|
||||
- Using an admin token from ADMIN_TOKENS env var
|
||||
- User email is from admin domain (e.g., @nomadkaraoke.com)
|
||||
- User role is set to ADMIN in database
|
||||
|
||||
Returns:
|
||||
(token, user_type, remaining_uses)
|
||||
|
||||
AuthResult with admin privileges
|
||||
|
||||
Raises:
|
||||
HTTPException: 403 if user is not admin
|
||||
"""
|
||||
token, user_type, remaining_uses = auth_data
|
||||
|
||||
if user_type != UserType.ADMIN:
|
||||
logger.warning(f"Admin access denied for {user_type} user")
|
||||
if not auth_result.is_admin:
|
||||
logger.warning(
|
||||
f"Admin access denied for {auth_result.user_type} user "
|
||||
f"(email: {auth_result.user_email or 'unknown'})"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="Admin access required"
|
||||
)
|
||||
|
||||
return token, user_type, remaining_uses
|
||||
|
||||
return auth_result
|
||||
|
||||
|
||||
def optional_auth(
|
||||
@@ -168,14 +228,14 @@ def optional_auth(
|
||||
def require_review_auth_factory(job_id_param: str = "job_id"):
|
||||
"""
|
||||
Factory to create a review authentication dependency.
|
||||
|
||||
|
||||
Accepts either:
|
||||
1. Full user authentication (admin/user token)
|
||||
1. Full user authentication (admin/user token) - also validates job ownership
|
||||
2. Job-specific review token (only valid for the specific job)
|
||||
|
||||
|
||||
Args:
|
||||
job_id_param: Name of the path parameter containing the job ID
|
||||
|
||||
|
||||
Returns:
|
||||
Dependency function that validates review access
|
||||
"""
|
||||
@@ -189,37 +249,63 @@ def require_review_auth_factory(job_id_param: str = "job_id"):
|
||||
) -> Tuple[str, str]:
|
||||
"""
|
||||
Validate review access for a job.
|
||||
|
||||
|
||||
Returns:
|
||||
(job_id, auth_type) where auth_type is "full" or "review_token"
|
||||
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if authentication fails
|
||||
HTTPException: 403 if user doesn't own the job (for full auth)
|
||||
"""
|
||||
# Import here to avoid circular dependency
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
|
||||
job_manager = JobManager()
|
||||
|
||||
# Try full authentication first
|
||||
full_token = None
|
||||
if credentials:
|
||||
full_token = credentials.credentials
|
||||
elif token:
|
||||
full_token = token
|
||||
|
||||
|
||||
if full_token:
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(full_token)
|
||||
if is_valid:
|
||||
logger.info(f"Review access granted via full auth ({user_type}) for job {job_id}")
|
||||
auth_result = auth_service.validate_token_full(full_token)
|
||||
if auth_result.is_valid:
|
||||
# For full auth, also verify job ownership
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Check ownership: admin can access all, users only their own jobs
|
||||
if not auth_result.is_admin:
|
||||
if auth_result.user_email and job.user_email:
|
||||
if auth_result.user_email.lower() != job.user_email.lower():
|
||||
logger.warning(
|
||||
f"Review access denied: user {auth_result.user_email} "
|
||||
f"tried to access job {job_id} owned by {job.user_email}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's review"
|
||||
)
|
||||
elif job.user_email:
|
||||
# Token auth without email trying to access a job with owner
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's review"
|
||||
)
|
||||
|
||||
logger.info(f"Review access granted via full auth ({auth_result.user_type}) for job {job_id}")
|
||||
return job_id, "full"
|
||||
|
||||
|
||||
# Try review token
|
||||
if review_token:
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
|
||||
# Validate review token matches
|
||||
if job.review_token and secrets.compare_digest(job.review_token, review_token):
|
||||
# Check expiry if set
|
||||
@@ -229,25 +315,25 @@ def require_review_auth_factory(job_id_param: str = "job_id"):
|
||||
expiry = job.review_token_expires_at
|
||||
if expiry.tzinfo is None:
|
||||
expiry = expiry.replace(tzinfo=UTC)
|
||||
|
||||
|
||||
if now > expiry:
|
||||
logger.warning(f"Review token expired for job {job_id}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Review token has expired. Please request a new review link."
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Review access granted via review_token for job {job_id}")
|
||||
return job_id, "review_token"
|
||||
else:
|
||||
logger.warning(f"Invalid review token for job {job_id}")
|
||||
|
||||
|
||||
# No valid authentication
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide either a full access token or a valid review_token for this job."
|
||||
)
|
||||
|
||||
|
||||
return require_review_auth
|
||||
|
||||
|
||||
@@ -258,14 +344,14 @@ require_review_auth = require_review_auth_factory()
|
||||
def require_instrumental_auth_factory(job_id_param: str = "job_id"):
|
||||
"""
|
||||
Factory to create an instrumental review authentication dependency.
|
||||
|
||||
|
||||
Accepts either:
|
||||
1. Full user authentication (admin/user token)
|
||||
1. Full user authentication (admin/user token) - also validates job ownership
|
||||
2. Job-specific instrumental token (only valid for the specific job)
|
||||
|
||||
|
||||
Args:
|
||||
job_id_param: Name of the path parameter containing the job ID
|
||||
|
||||
|
||||
Returns:
|
||||
Dependency function that validates instrumental review access
|
||||
"""
|
||||
@@ -279,37 +365,63 @@ def require_instrumental_auth_factory(job_id_param: str = "job_id"):
|
||||
) -> Tuple[str, str]:
|
||||
"""
|
||||
Validate instrumental review access for a job.
|
||||
|
||||
|
||||
Returns:
|
||||
(job_id, auth_type) where auth_type is "full" or "instrumental_token"
|
||||
|
||||
|
||||
Raises:
|
||||
HTTPException: 401 if authentication fails
|
||||
HTTPException: 403 if user doesn't own the job (for full auth)
|
||||
"""
|
||||
# Import here to avoid circular dependency
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
|
||||
job_manager = JobManager()
|
||||
|
||||
# Try full authentication first
|
||||
full_token = None
|
||||
if credentials:
|
||||
full_token = credentials.credentials
|
||||
elif token:
|
||||
full_token = token
|
||||
|
||||
|
||||
if full_token:
|
||||
is_valid, user_type, remaining_uses, message = auth_service.validate_token(full_token)
|
||||
if is_valid:
|
||||
logger.info(f"Instrumental access granted via full auth ({user_type}) for job {job_id}")
|
||||
auth_result = auth_service.validate_token_full(full_token)
|
||||
if auth_result.is_valid:
|
||||
# For full auth, also verify job ownership
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Check ownership: admin can access all, users only their own jobs
|
||||
if not auth_result.is_admin:
|
||||
if auth_result.user_email and job.user_email:
|
||||
if auth_result.user_email.lower() != job.user_email.lower():
|
||||
logger.warning(
|
||||
f"Instrumental access denied: user {auth_result.user_email} "
|
||||
f"tried to access job {job_id} owned by {job.user_email}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's instrumental review"
|
||||
)
|
||||
elif job.user_email:
|
||||
# Token auth without email trying to access a job with owner
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You don't have permission to access this job's instrumental review"
|
||||
)
|
||||
|
||||
logger.info(f"Instrumental access granted via full auth ({auth_result.user_type}) for job {job_id}")
|
||||
return job_id, "full"
|
||||
|
||||
|
||||
# Try instrumental token
|
||||
if instrumental_token:
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
|
||||
# Validate instrumental token matches
|
||||
if job.instrumental_token and secrets.compare_digest(job.instrumental_token, instrumental_token):
|
||||
# Check expiry if set
|
||||
@@ -319,25 +431,25 @@ def require_instrumental_auth_factory(job_id_param: str = "job_id"):
|
||||
expiry = job.instrumental_token_expires_at
|
||||
if expiry.tzinfo is None:
|
||||
expiry = expiry.replace(tzinfo=UTC)
|
||||
|
||||
|
||||
if now > expiry:
|
||||
logger.warning(f"Instrumental token expired for job {job_id}")
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Instrumental review token has expired. Please request a new review link."
|
||||
)
|
||||
|
||||
|
||||
logger.info(f"Instrumental access granted via instrumental_token for job {job_id}")
|
||||
return job_id, "instrumental_token"
|
||||
else:
|
||||
logger.warning(f"Invalid instrumental token for job {job_id}")
|
||||
|
||||
|
||||
# No valid authentication
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Authentication required. Provide either a full access token or a valid instrumental_token for this job."
|
||||
)
|
||||
|
||||
|
||||
return require_instrumental_auth
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
"""
|
||||
Admin API routes for dashboard statistics and management.
|
||||
|
||||
Handles:
|
||||
- Dashboard overview statistics
|
||||
- System-wide metrics
|
||||
- Admin-only operations
|
||||
"""
|
||||
import logging
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Tuple
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from pydantic import BaseModel
|
||||
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.services.user_service import get_user_service, UserService, USERS_COLLECTION
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.models.job import JobStatus
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/admin", tags=["admin"])
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Response Models
|
||||
# =============================================================================
|
||||
|
||||
class JobsByStatusResponse(BaseModel):
|
||||
"""Breakdown of jobs by status."""
|
||||
pending: int = 0
|
||||
processing: int = 0
|
||||
awaiting_review: int = 0
|
||||
awaiting_instrumental: int = 0
|
||||
complete: int = 0
|
||||
failed: int = 0
|
||||
cancelled: int = 0
|
||||
|
||||
|
||||
class AdminStatsOverview(BaseModel):
|
||||
"""Overview statistics for admin dashboard."""
|
||||
total_users: int
|
||||
active_users_7d: int
|
||||
active_users_30d: int
|
||||
total_jobs: int
|
||||
jobs_last_7d: int
|
||||
jobs_last_30d: int
|
||||
jobs_by_status: JobsByStatusResponse
|
||||
total_credits_issued_30d: int
|
||||
total_beta_testers: int
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Admin Stats Endpoints
|
||||
# =============================================================================
|
||||
|
||||
@router.get("/stats/overview", response_model=AdminStatsOverview)
|
||||
async def get_admin_stats_overview(
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin),
|
||||
user_service: UserService = Depends(get_user_service),
|
||||
):
|
||||
"""
|
||||
Get overview statistics for admin dashboard.
|
||||
|
||||
Includes:
|
||||
- User counts (total, active in 7d, active in 30d)
|
||||
- Job counts (total, by status, recent)
|
||||
- Credit statistics
|
||||
- Beta program stats
|
||||
"""
|
||||
from google.cloud.firestore_v1 import FieldFilter
|
||||
from google.cloud.firestore_v1 import aggregation
|
||||
|
||||
db = user_service.db
|
||||
now = datetime.utcnow()
|
||||
seven_days_ago = now - timedelta(days=7)
|
||||
thirty_days_ago = now - timedelta(days=30)
|
||||
|
||||
# Helper function to get count using aggregation
|
||||
def get_count(query) -> int:
|
||||
try:
|
||||
agg_query = aggregation.AggregationQuery(query)
|
||||
agg_query.count(alias="count")
|
||||
results = agg_query.get()
|
||||
return results[0][0].value if results else 0
|
||||
except Exception as e:
|
||||
logger.warning(f"Aggregation query failed: {e}")
|
||||
return 0
|
||||
|
||||
# User statistics
|
||||
users_collection = db.collection(USERS_COLLECTION)
|
||||
|
||||
total_users = get_count(users_collection)
|
||||
|
||||
active_users_7d = get_count(
|
||||
users_collection.where(filter=FieldFilter("last_login_at", ">=", seven_days_ago))
|
||||
)
|
||||
|
||||
active_users_30d = get_count(
|
||||
users_collection.where(filter=FieldFilter("last_login_at", ">=", thirty_days_ago))
|
||||
)
|
||||
|
||||
total_beta_testers = get_count(
|
||||
users_collection.where(filter=FieldFilter("is_beta_tester", "==", True))
|
||||
)
|
||||
|
||||
# Job statistics
|
||||
jobs_collection = db.collection("jobs")
|
||||
|
||||
total_jobs = get_count(jobs_collection)
|
||||
|
||||
jobs_last_7d = get_count(
|
||||
jobs_collection.where(filter=FieldFilter("created_at", ">=", seven_days_ago))
|
||||
)
|
||||
|
||||
jobs_last_30d = get_count(
|
||||
jobs_collection.where(filter=FieldFilter("created_at", ">=", thirty_days_ago))
|
||||
)
|
||||
|
||||
# Jobs by status - map multiple statuses to simplified categories
|
||||
processing_statuses = [
|
||||
"downloading", "downloading_audio", "searching_audio", "awaiting_audio_selection",
|
||||
"separating_stage1", "separating_stage2", "transcribing", "correcting",
|
||||
"generating_screens", "applying_padding", "rendering_video",
|
||||
"instrumental_selected", "generating_video", "encoding", "packaging",
|
||||
"uploading", "notifying"
|
||||
]
|
||||
|
||||
jobs_by_status = JobsByStatusResponse(
|
||||
pending=get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "pending"))
|
||||
),
|
||||
processing=sum(
|
||||
get_count(jobs_collection.where(filter=FieldFilter("status", "==", status)))
|
||||
for status in processing_statuses
|
||||
),
|
||||
awaiting_review=get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "awaiting_review"))
|
||||
) + get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "in_review"))
|
||||
),
|
||||
awaiting_instrumental=get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "awaiting_instrumental_selection"))
|
||||
),
|
||||
complete=get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "complete"))
|
||||
) + get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "prep_complete"))
|
||||
),
|
||||
failed=get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "failed"))
|
||||
),
|
||||
cancelled=get_count(
|
||||
jobs_collection.where(filter=FieldFilter("status", "==", "cancelled"))
|
||||
),
|
||||
)
|
||||
|
||||
# Credit statistics - sum credits added in last 30 days
|
||||
# This is more expensive, so we'll just estimate from users
|
||||
total_credits_issued_30d = 0
|
||||
try:
|
||||
# Get all users and sum recent credit transactions
|
||||
users_docs = users_collection.limit(500).stream()
|
||||
for user_doc in users_docs:
|
||||
user_data = user_doc.to_dict()
|
||||
transactions = user_data.get("credit_transactions", [])
|
||||
for txn in transactions:
|
||||
txn_date = txn.get("created_at")
|
||||
if txn_date:
|
||||
# Handle both datetime and string formats
|
||||
if isinstance(txn_date, str):
|
||||
try:
|
||||
txn_date = datetime.fromisoformat(txn_date.replace("Z", "+00:00"))
|
||||
except Exception:
|
||||
continue
|
||||
if isinstance(txn_date, datetime):
|
||||
txn_date = txn_date.replace(tzinfo=None)
|
||||
else:
|
||||
continue
|
||||
if txn_date >= thirty_days_ago:
|
||||
amount = txn.get("amount", 0)
|
||||
if amount > 0: # Only count additions, not deductions
|
||||
total_credits_issued_30d += amount
|
||||
except Exception as e:
|
||||
logger.warning(f"Error calculating credits: {e}")
|
||||
|
||||
return AdminStatsOverview(
|
||||
total_users=total_users,
|
||||
active_users_7d=active_users_7d,
|
||||
active_users_30d=active_users_30d,
|
||||
total_jobs=total_jobs,
|
||||
jobs_last_7d=jobs_last_7d,
|
||||
jobs_last_30d=jobs_last_30d,
|
||||
jobs_by_status=jobs_by_status,
|
||||
total_credits_issued_30d=total_credits_issued_30d,
|
||||
total_beta_testers=total_beta_testers,
|
||||
)
|
||||
@@ -35,7 +35,7 @@ from backend.services.audio_search_service import (
|
||||
from backend.config import get_settings
|
||||
from backend.version import VERSION
|
||||
from backend.api.dependencies import require_auth
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.services.auth_service import UserType, AuthResult
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -117,6 +117,9 @@ class AudioSearchRequest(BaseModel):
|
||||
# Style params JSON and related assets (background images, fonts)
|
||||
style_files: Optional[List[StyleFileRequest]] = Field(None, description="Style files to upload (style_params JSON, backgrounds, fonts)")
|
||||
|
||||
# Non-interactive mode
|
||||
non_interactive: bool = Field(False, description="Skip interactive steps (lyrics review, instrumental selection)")
|
||||
|
||||
|
||||
class AudioSearchResultResponse(BaseModel):
|
||||
"""A single audio search result.
|
||||
@@ -372,8 +375,31 @@ async def _download_and_start_processing(
|
||||
target_file=target_file,
|
||||
download_url=download_url,
|
||||
)
|
||||
elif source_name == 'YouTube' and download_url:
|
||||
# For YouTube without remote, download directly using URL from state_data
|
||||
# This avoids relying on in-memory cache which doesn't persist across instances
|
||||
logger.info(f"Downloading YouTube audio directly from URL: {download_url}")
|
||||
from backend.workers.audio_worker import download_from_url
|
||||
|
||||
local_path = await download_from_url(
|
||||
download_url,
|
||||
temp_dir,
|
||||
selected.get('artist'),
|
||||
selected.get('title')
|
||||
)
|
||||
|
||||
if not local_path or not os.path.exists(local_path):
|
||||
raise DownloadError(f"Failed to download from YouTube: {download_url}")
|
||||
|
||||
# Create a result-like object
|
||||
class DownloadResult:
|
||||
def __init__(self, filepath):
|
||||
self.filepath = filepath
|
||||
|
||||
result = DownloadResult(local_path)
|
||||
else:
|
||||
# Fallback to search-based download
|
||||
# Fallback to search-based download (may fail if cache is empty)
|
||||
logger.warning(f"Falling back to cache-based download for {source_name} - this may fail on multi-instance deployments")
|
||||
result = audio_search_service.download(
|
||||
result_index=selection_index,
|
||||
output_dir=temp_dir,
|
||||
@@ -436,7 +462,7 @@ async def search_audio(
|
||||
request: Request,
|
||||
background_tasks: BackgroundTasks,
|
||||
body: AudioSearchRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Search for audio by artist and title, creating a new job.
|
||||
@@ -500,6 +526,9 @@ async def search_audio(
|
||||
body.theme_id, body.enable_cdg, body.enable_txt
|
||||
)
|
||||
|
||||
# Use authenticated user's email
|
||||
effective_user_email = auth_result.user_email
|
||||
|
||||
# Create job
|
||||
job_create = JobCreate(
|
||||
artist=body.artist,
|
||||
@@ -515,6 +544,7 @@ async def search_audio(
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
user_email=effective_user_email,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
subtitle_offset_ms=body.subtitle_offset_ms,
|
||||
@@ -525,10 +555,11 @@ async def search_audio(
|
||||
audio_search_title=body.title,
|
||||
auto_download=body.auto_download,
|
||||
request_metadata=request_metadata,
|
||||
non_interactive=body.non_interactive,
|
||||
)
|
||||
job = job_manager.create_job(job_create)
|
||||
job_id = job.job_id
|
||||
|
||||
|
||||
logger.info(f"Created job {job_id} for audio search: {body.artist} - {body.title}")
|
||||
|
||||
# Update job with audio search fields
|
||||
@@ -766,7 +797,7 @@ async def search_audio(
|
||||
@router.get("/audio-search/{job_id}/results")
|
||||
async def get_audio_search_results(
|
||||
job_id: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Get audio search results for a job.
|
||||
@@ -801,7 +832,7 @@ async def select_audio_source(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
body: AudioSelectRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Select an audio source and start job processing.
|
||||
|
||||
+263
-115
@@ -15,6 +15,7 @@ import json
|
||||
import logging
|
||||
import tempfile
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from fastapi import APIRouter, UploadFile, File, Form, HTTPException, BackgroundTasks, Request, Body, Depends
|
||||
from pathlib import Path
|
||||
from typing import Optional, List, Dict, Any, Tuple
|
||||
@@ -32,7 +33,7 @@ from backend.config import get_settings
|
||||
from backend.version import VERSION
|
||||
from backend.services.metrics import metrics
|
||||
from backend.api.dependencies import require_auth
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.services.auth_service import UserType, AuthResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(tags=["jobs"])
|
||||
@@ -91,6 +92,9 @@ class CreateJobFromUrlRequest(BaseModel):
|
||||
backing_vocals_models: Optional[List[str]] = Field(None, description="Models for backing vocals separation")
|
||||
other_stems_models: Optional[List[str]] = Field(None, description="Models for other stems")
|
||||
|
||||
# Non-interactive mode
|
||||
non_interactive: bool = Field(False, description="Skip interactive steps (lyrics review, instrumental selection)")
|
||||
|
||||
|
||||
class CreateJobFromUrlResponse(BaseModel):
|
||||
"""Response from creating a job from URL."""
|
||||
@@ -149,6 +153,9 @@ class CreateJobWithUploadUrlsRequest(BaseModel):
|
||||
prep_only: bool = Field(False, description="Stop after review phase, don't run finalisation")
|
||||
keep_brand_code: Optional[str] = Field(None, description="Preserve existing brand code instead of generating new one")
|
||||
|
||||
# Non-interactive mode
|
||||
non_interactive: bool = Field(False, description="Skip interactive steps (lyrics review, instrumental selection)")
|
||||
|
||||
|
||||
class SignedUploadUrl(BaseModel):
|
||||
"""Signed URL for uploading a file."""
|
||||
@@ -246,6 +253,16 @@ async def _trigger_workers_parallel(job_id: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
async def _trigger_audio_worker_only(job_id: str) -> None:
|
||||
"""
|
||||
Trigger only the audio worker.
|
||||
|
||||
Used for URL jobs where audio needs to be downloaded first.
|
||||
The audio worker will trigger the lyrics worker after download completes.
|
||||
"""
|
||||
await worker_service.trigger_audio_worker(job_id)
|
||||
|
||||
|
||||
def _resolve_cdg_txt_defaults(
|
||||
theme_id: Optional[str],
|
||||
enable_cdg: Optional[bool],
|
||||
@@ -275,6 +292,46 @@ def _resolve_cdg_txt_defaults(
|
||||
return resolved_cdg, resolved_txt
|
||||
|
||||
|
||||
@dataclass
|
||||
class EffectiveDistributionSettings:
|
||||
"""Settings with defaults applied from environment variables."""
|
||||
dropbox_path: Optional[str]
|
||||
gdrive_folder_id: Optional[str]
|
||||
discord_webhook_url: Optional[str]
|
||||
brand_prefix: Optional[str]
|
||||
|
||||
|
||||
def _get_effective_distribution_settings(
|
||||
dropbox_path: Optional[str] = None,
|
||||
gdrive_folder_id: Optional[str] = None,
|
||||
discord_webhook_url: Optional[str] = None,
|
||||
brand_prefix: Optional[str] = None,
|
||||
) -> EffectiveDistributionSettings:
|
||||
"""
|
||||
Get distribution settings with defaults applied from environment variables.
|
||||
|
||||
This ensures consistent handling of defaults across all job creation endpoints.
|
||||
Each parameter, if not provided (None), falls back to the corresponding
|
||||
environment variable configured in settings.
|
||||
|
||||
Args:
|
||||
dropbox_path: Explicit Dropbox path or None for default
|
||||
gdrive_folder_id: Explicit Google Drive folder ID or None for default
|
||||
discord_webhook_url: Explicit Discord webhook URL or None for default
|
||||
brand_prefix: Explicit brand prefix or None for default
|
||||
|
||||
Returns:
|
||||
EffectiveDistributionSettings with defaults applied
|
||||
"""
|
||||
settings = get_settings()
|
||||
return EffectiveDistributionSettings(
|
||||
dropbox_path=dropbox_path or settings.default_dropbox_path,
|
||||
gdrive_folder_id=gdrive_folder_id or settings.default_gdrive_folder_id,
|
||||
discord_webhook_url=discord_webhook_url or settings.default_discord_webhook_url,
|
||||
brand_prefix=brand_prefix or settings.default_brand_prefix,
|
||||
)
|
||||
|
||||
|
||||
def _prepare_theme_for_job(
|
||||
job_id: str,
|
||||
theme_id: str,
|
||||
@@ -327,7 +384,7 @@ def _prepare_theme_for_job(
|
||||
async def upload_and_create_job(
|
||||
request: Request,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth),
|
||||
auth_result: AuthResult = Depends(require_auth),
|
||||
# Required fields
|
||||
file: UploadFile = File(..., description="Audio file to process"),
|
||||
artist: str = Form(..., description="Artist name"),
|
||||
@@ -341,9 +398,12 @@ async def upload_and_create_job(
|
||||
style_cdg_instrumental_background: Optional[UploadFile] = File(None, description="CDG instrumental background"),
|
||||
style_cdg_title_background: Optional[UploadFile] = File(None, description="CDG title screen background"),
|
||||
style_cdg_outro_background: Optional[UploadFile] = File(None, description="CDG outro screen background"),
|
||||
# Processing options (CDG/TXT require style config, disabled by default)
|
||||
enable_cdg: bool = Form(False, description="Generate CDG+MP3 package (requires style config)"),
|
||||
enable_txt: bool = Form(False, description="Generate TXT+MP3 package (requires style config)"),
|
||||
# Theme configuration (use pre-made theme from GCS)
|
||||
theme_id: Optional[str] = Form(None, description="Theme ID to use (e.g., 'nomad', 'default'). If set, CDG/TXT are enabled by default."),
|
||||
color_overrides: Optional[str] = Form(None, description="JSON-encoded color overrides: artist_color, title_color, sung_lyrics_color, unsung_lyrics_color (hex #RRGGBB)"),
|
||||
# Processing options (CDG/TXT require style config or theme, disabled by default unless theme is set)
|
||||
enable_cdg: Optional[bool] = Form(None, description="Generate CDG+MP3 package. Default: True if theme_id set, False otherwise"),
|
||||
enable_txt: Optional[bool] = Form(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise"),
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Form(None, description="Brand code prefix (e.g., NOMAD)"),
|
||||
enable_youtube_upload: bool = Form(False, description="Upload to YouTube"),
|
||||
@@ -365,6 +425,8 @@ async def upload_and_create_job(
|
||||
clean_instrumental_model: Optional[str] = Form(None, description="Model for clean instrumental separation (e.g., model_bs_roformer_ep_317_sdr_12.9755.ckpt)"),
|
||||
backing_vocals_models: Optional[str] = Form(None, description="Comma-separated list of models for backing vocals separation"),
|
||||
other_stems_models: Optional[str] = Form(None, description="Comma-separated list of models for other stems (bass, drums, guitar, etc.)"),
|
||||
# Non-interactive mode
|
||||
non_interactive: bool = Form(False, description="Skip interactive steps (lyrics review, instrumental selection)"),
|
||||
):
|
||||
"""
|
||||
Upload an audio file and create a karaoke generation job with full style configuration.
|
||||
@@ -432,38 +494,42 @@ async def upload_and_create_job(
|
||||
)
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
if effective_dropbox_path and not dropbox_path:
|
||||
logger.info(f"Using default dropbox_path: {effective_dropbox_path}")
|
||||
if effective_gdrive_folder_id and not gdrive_folder_id:
|
||||
logger.info(f"Using default gdrive_folder_id: {effective_gdrive_folder_id}")
|
||||
if effective_discord_webhook_url and not discord_webhook_url:
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=dropbox_path,
|
||||
gdrive_folder_id=gdrive_folder_id,
|
||||
discord_webhook_url=discord_webhook_url,
|
||||
brand_prefix=brand_prefix,
|
||||
)
|
||||
|
||||
if dist.dropbox_path and not dropbox_path:
|
||||
logger.info(f"Using default dropbox_path: {dist.dropbox_path}")
|
||||
if dist.gdrive_folder_id and not gdrive_folder_id:
|
||||
logger.info(f"Using default gdrive_folder_id: {dist.gdrive_folder_id}")
|
||||
if dist.discord_webhook_url and not discord_webhook_url:
|
||||
logger.info("Using default discord_webhook_url (from env)")
|
||||
|
||||
if dist.brand_prefix and not brand_prefix:
|
||||
logger.info(f"Using default brand_prefix: {dist.brand_prefix}")
|
||||
|
||||
# Validate credentials for requested distribution services (including defaults)
|
||||
# This prevents accepting jobs that will fail later due to missing credentials
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -477,7 +543,35 @@ async def upload_and_create_job(
|
||||
|
||||
# Extract request metadata for tracking and filtering
|
||||
request_metadata = extract_request_metadata(request, created_from="upload")
|
||||
|
||||
|
||||
# Parse color_overrides from JSON if provided
|
||||
parsed_color_overrides: Dict[str, str] = {}
|
||||
if color_overrides:
|
||||
try:
|
||||
parsed_color_overrides = json.loads(color_overrides)
|
||||
except json.JSONDecodeError as e:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid color_overrides JSON: {e}"
|
||||
)
|
||||
|
||||
# Resolve CDG/TXT defaults based on theme
|
||||
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
|
||||
theme_id, enable_cdg, enable_txt
|
||||
)
|
||||
|
||||
# Check if any custom style files are being uploaded (overrides theme)
|
||||
has_custom_style_files = any([
|
||||
style_params,
|
||||
style_intro_background,
|
||||
style_karaoke_background,
|
||||
style_end_background,
|
||||
style_font,
|
||||
style_cdg_instrumental_background,
|
||||
style_cdg_title_background,
|
||||
style_cdg_outro_background,
|
||||
])
|
||||
|
||||
# Parse comma-separated model lists into arrays
|
||||
parsed_backing_vocals_models = None
|
||||
if backing_vocals_models:
|
||||
@@ -487,22 +581,27 @@ async def upload_and_create_job(
|
||||
if other_stems_models:
|
||||
parsed_other_stems_models = [m.strip() for m in other_stems_models.split(',') if m.strip()]
|
||||
|
||||
# Prefer authenticated user's email over form parameter
|
||||
effective_user_email = auth_result.user_email or user_email
|
||||
|
||||
# Create job first to get job_id
|
||||
job_create = JobCreate(
|
||||
artist=artist,
|
||||
title=title,
|
||||
filename=file.filename,
|
||||
enable_cdg=enable_cdg,
|
||||
enable_txt=enable_txt,
|
||||
brand_prefix=brand_prefix,
|
||||
theme_id=theme_id,
|
||||
color_overrides=parsed_color_overrides,
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=enable_youtube_upload,
|
||||
youtube_description=youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=webhook_url,
|
||||
user_email=user_email,
|
||||
user_email=effective_user_email,
|
||||
# Native API distribution (preferred for remote CLI)
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
# Legacy rclone distribution (deprecated)
|
||||
organised_dir_rclone_root=organised_dir_rclone_root,
|
||||
# Lyrics configuration (overrides for search/transcription)
|
||||
@@ -515,15 +614,35 @@ async def upload_and_create_job(
|
||||
other_stems_models=parsed_other_stems_models,
|
||||
# Request metadata for tracking and filtering
|
||||
request_metadata=request_metadata,
|
||||
# Non-interactive mode
|
||||
non_interactive=non_interactive,
|
||||
)
|
||||
job = job_manager.create_job(job_create)
|
||||
job_id = job.job_id
|
||||
|
||||
# Record job creation metric
|
||||
metrics.record_job_created(job_id, source="upload")
|
||||
|
||||
|
||||
logger.info(f"Created job {job_id} for {artist} - {title}")
|
||||
|
||||
|
||||
# If theme is set and no custom style files are being uploaded, prepare theme style now
|
||||
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
|
||||
# can access the style configuration for preview videos
|
||||
theme_style_params_path = None
|
||||
theme_style_assets = {}
|
||||
theme_youtube_desc = None
|
||||
if theme_id and not has_custom_style_files:
|
||||
try:
|
||||
theme_style_params_path, theme_style_assets, theme_youtube_desc = _prepare_theme_for_job(
|
||||
job_id, theme_id, parsed_color_overrides or None
|
||||
)
|
||||
logger.info(f"Applied theme '{theme_id}' to job {job_id}")
|
||||
except HTTPException:
|
||||
raise # Re-raise validation errors (e.g., theme not found)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to prepare theme '{theme_id}' for job {job_id}: {e}")
|
||||
# Continue without theme - job can still be processed with defaults
|
||||
|
||||
# Upload main audio file to GCS
|
||||
audio_gcs_path = f"uploads/{job_id}/audio/{file.filename}"
|
||||
logger.info(f"Uploading audio to GCS: {audio_gcs_path}")
|
||||
@@ -592,27 +711,36 @@ async def upload_and_create_job(
|
||||
update_data = {
|
||||
'input_media_gcs_path': audio_gcs_path,
|
||||
'filename': file.filename,
|
||||
'enable_cdg': enable_cdg,
|
||||
'enable_txt': enable_txt,
|
||||
'enable_cdg': resolved_cdg,
|
||||
'enable_txt': resolved_txt,
|
||||
}
|
||||
|
||||
|
||||
# Handle style assets - either from custom uploads or from theme
|
||||
if style_assets:
|
||||
# Custom style files uploaded
|
||||
update_data['style_assets'] = style_assets
|
||||
if 'style_params' in style_assets:
|
||||
update_data['style_params_gcs_path'] = style_assets['style_params']
|
||||
|
||||
if brand_prefix:
|
||||
update_data['brand_prefix'] = brand_prefix
|
||||
if effective_discord_webhook_url:
|
||||
update_data['discord_webhook_url'] = effective_discord_webhook_url
|
||||
if youtube_description:
|
||||
update_data['youtube_description_template'] = youtube_description
|
||||
|
||||
elif theme_style_assets:
|
||||
# Theme style assets (no custom uploads)
|
||||
update_data['style_assets'] = theme_style_assets
|
||||
if theme_style_params_path:
|
||||
update_data['style_params_gcs_path'] = theme_style_params_path
|
||||
|
||||
if dist.brand_prefix:
|
||||
update_data['brand_prefix'] = dist.brand_prefix
|
||||
if dist.discord_webhook_url:
|
||||
update_data['discord_webhook_url'] = dist.discord_webhook_url
|
||||
# Use theme YouTube description if no custom one provided
|
||||
effective_youtube_description = youtube_description or theme_youtube_desc
|
||||
if effective_youtube_description:
|
||||
update_data['youtube_description_template'] = effective_youtube_description
|
||||
|
||||
# Native API distribution (use effective values which include defaults)
|
||||
if effective_dropbox_path:
|
||||
update_data['dropbox_path'] = effective_dropbox_path
|
||||
if effective_gdrive_folder_id:
|
||||
update_data['gdrive_folder_id'] = effective_gdrive_folder_id
|
||||
if dist.dropbox_path:
|
||||
update_data['dropbox_path'] = dist.dropbox_path
|
||||
if dist.gdrive_folder_id:
|
||||
update_data['gdrive_folder_id'] = dist.gdrive_folder_id
|
||||
|
||||
# Legacy rclone distribution (deprecated)
|
||||
if organised_dir_rclone_root:
|
||||
@@ -668,21 +796,21 @@ async def upload_and_create_job(
|
||||
|
||||
# Build distribution services info for response
|
||||
distribution_services: Dict[str, Any] = {}
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
dropbox_result = credential_manager.check_dropbox_credentials()
|
||||
distribution_services["dropbox"] = {
|
||||
"enabled": True,
|
||||
"path": effective_dropbox_path,
|
||||
"path": dist.dropbox_path,
|
||||
"credentials_valid": dropbox_result.status == CredentialStatus.VALID,
|
||||
"using_default": dropbox_path is None,
|
||||
}
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
gdrive_result = credential_manager.check_gdrive_credentials()
|
||||
distribution_services["gdrive"] = {
|
||||
"enabled": True,
|
||||
"folder_id": effective_gdrive_folder_id,
|
||||
"folder_id": dist.gdrive_folder_id,
|
||||
"credentials_valid": gdrive_result.status == CredentialStatus.VALID,
|
||||
"using_default": gdrive_folder_id is None,
|
||||
}
|
||||
@@ -694,7 +822,7 @@ async def upload_and_create_job(
|
||||
"credentials_valid": youtube_result.status == CredentialStatus.VALID,
|
||||
}
|
||||
|
||||
if effective_discord_webhook_url:
|
||||
if dist.discord_webhook_url:
|
||||
distribution_services["discord"] = {
|
||||
"enabled": True,
|
||||
"using_default": discord_webhook_url is None,
|
||||
@@ -868,7 +996,7 @@ def _get_gcs_path_for_file(job_id: str, file_type: str, filename: str) -> str:
|
||||
async def create_job_with_upload_urls(
|
||||
request: Request,
|
||||
body: CreateJobWithUploadUrlsRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Create a karaoke generation job and return signed URLs for direct file upload to GCS.
|
||||
@@ -911,32 +1039,34 @@ async def create_job_with_upload_urls(
|
||||
status_code=400,
|
||||
detail=f"Invalid file extension '{ext}' for file_type '{file_info.file_type}'. Allowed: {allowed_extensions}"
|
||||
)
|
||||
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -947,7 +1077,7 @@ async def create_job_with_upload_urls(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata for tracking
|
||||
request_metadata = extract_request_metadata(request, created_from="signed_url_upload")
|
||||
|
||||
@@ -962,6 +1092,9 @@ async def create_job_with_upload_urls(
|
||||
# Check if style_params is being uploaded (overrides theme)
|
||||
has_style_params_upload = any(f.file_type == 'style_params' for f in body.files)
|
||||
|
||||
# Prefer authenticated user's email over request body
|
||||
effective_user_email = auth_result.user_email or body.user_email
|
||||
|
||||
# Create job
|
||||
job_create = JobCreate(
|
||||
artist=body.artist,
|
||||
@@ -971,14 +1104,15 @@ async def create_job_with_upload_urls(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=body.brand_prefix,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
youtube_description_template=body.youtube_description, # video_worker reads this field
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=body.webhook_url,
|
||||
user_email=body.user_email,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
user_email=effective_user_email,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
organised_dir_rclone_root=body.organised_dir_rclone_root,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
@@ -987,6 +1121,7 @@ async def create_job_with_upload_urls(
|
||||
backing_vocals_models=body.backing_vocals_models,
|
||||
other_stems_models=body.other_stems_models,
|
||||
request_metadata=request_metadata,
|
||||
non_interactive=body.non_interactive,
|
||||
)
|
||||
job = job_manager.create_job(job_create)
|
||||
job_id = job.job_id
|
||||
@@ -1052,7 +1187,7 @@ async def mark_uploads_complete(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
body: UploadsCompleteRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Mark file uploads as complete and start job processing.
|
||||
@@ -1298,7 +1433,7 @@ async def create_job_from_url(
|
||||
request: Request,
|
||||
background_tasks: BackgroundTasks,
|
||||
body: CreateJobFromUrlRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Create a karaoke generation job from a YouTube or other online video URL.
|
||||
@@ -1322,32 +1457,34 @@ async def create_job_from_url(
|
||||
status_code=400,
|
||||
detail="Invalid URL. Please provide a valid YouTube, Vimeo, SoundCloud, or other supported video URL."
|
||||
)
|
||||
|
||||
|
||||
# Apply default distribution settings from environment if not provided
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Validate credentials for requested distribution services
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1358,7 +1495,7 @@ async def create_job_from_url(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata for tracking
|
||||
request_metadata = extract_request_metadata(request, created_from="url")
|
||||
|
||||
@@ -1371,6 +1508,9 @@ async def create_job_from_url(
|
||||
body.theme_id, body.enable_cdg, body.enable_txt
|
||||
)
|
||||
|
||||
# Prefer authenticated user's email over request body
|
||||
effective_user_email = auth_result.user_email or body.user_email
|
||||
|
||||
# Create job with URL
|
||||
job_create = JobCreate(
|
||||
url=body.url,
|
||||
@@ -1381,14 +1521,14 @@ async def create_job_from_url(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=body.brand_prefix,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
webhook_url=body.webhook_url,
|
||||
user_email=body.user_email,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
user_email=effective_user_email,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
organised_dir_rclone_root=body.organised_dir_rclone_root,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
@@ -1397,6 +1537,7 @@ async def create_job_from_url(
|
||||
backing_vocals_models=body.backing_vocals_models,
|
||||
other_stems_models=body.other_stems_models,
|
||||
request_metadata=request_metadata,
|
||||
non_interactive=body.non_interactive,
|
||||
)
|
||||
job = job_manager.create_job(job_create)
|
||||
job_id = job.job_id
|
||||
@@ -1432,10 +1573,11 @@ async def create_job_from_url(
|
||||
progress=5,
|
||||
message="Starting audio download from URL"
|
||||
)
|
||||
|
||||
# Trigger workers in parallel
|
||||
# The audio worker will handle downloading from URL
|
||||
background_tasks.add_task(_trigger_workers_parallel, job_id)
|
||||
|
||||
# For URL jobs, trigger ONLY audio worker first
|
||||
# The audio worker will trigger lyrics worker after download completes
|
||||
# This prevents race condition where lyrics worker times out waiting for audio
|
||||
background_tasks.add_task(_trigger_audio_worker_only, job_id)
|
||||
|
||||
return CreateJobFromUrlResponse(
|
||||
status="success",
|
||||
@@ -1498,7 +1640,7 @@ def _get_gcs_path_for_finalise_file(job_id: str, file_type: str, filename: str)
|
||||
async def create_finalise_only_job(
|
||||
request: Request,
|
||||
body: CreateFinaliseOnlyJobRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Create a finalise-only job for continuing from a local prep phase.
|
||||
@@ -1563,30 +1705,32 @@ async def create_finalise_only_job(
|
||||
)
|
||||
|
||||
# Apply default distribution settings
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
dist = _get_effective_distribution_settings(
|
||||
dropbox_path=body.dropbox_path,
|
||||
gdrive_folder_id=body.gdrive_folder_id,
|
||||
discord_webhook_url=body.discord_webhook_url,
|
||||
brand_prefix=body.brand_prefix,
|
||||
)
|
||||
|
||||
# Validate distribution credentials if services are requested
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
|
||||
if dist.dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
|
||||
if dist.gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1597,7 +1741,7 @@ async def create_finalise_only_job(
|
||||
"auth_url": "/api/auth/status"
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Extract request metadata
|
||||
request_metadata = extract_request_metadata(request, created_from="finalise_only_upload")
|
||||
|
||||
@@ -1609,6 +1753,9 @@ async def create_finalise_only_job(
|
||||
# Check if style_params is being uploaded (overrides theme)
|
||||
has_style_params_upload = any(f.file_type == 'style_params' for f in body.files)
|
||||
|
||||
# Use authenticated user's email
|
||||
effective_user_email = auth_result.user_email
|
||||
|
||||
# Create job with finalise_only=True
|
||||
job_create = JobCreate(
|
||||
artist=body.artist,
|
||||
@@ -1618,12 +1765,13 @@ async def create_finalise_only_job(
|
||||
color_overrides=body.color_overrides or {},
|
||||
enable_cdg=resolved_cdg,
|
||||
enable_txt=resolved_txt,
|
||||
brand_prefix=body.brand_prefix,
|
||||
brand_prefix=dist.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
discord_webhook_url=dist.discord_webhook_url,
|
||||
dropbox_path=dist.dropbox_path,
|
||||
gdrive_folder_id=dist.gdrive_folder_id,
|
||||
user_email=effective_user_email,
|
||||
finalise_only=True,
|
||||
keep_brand_code=body.keep_brand_code,
|
||||
request_metadata=request_metadata,
|
||||
@@ -1692,7 +1840,7 @@ async def mark_finalise_uploads_complete(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
body: UploadsCompleteRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Mark finalise-only file uploads as complete and start video generation.
|
||||
|
||||
@@ -3,16 +3,123 @@ Health check routes.
|
||||
"""
|
||||
import os
|
||||
import logging
|
||||
import subprocess
|
||||
import platform
|
||||
from fastapi import APIRouter
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
from backend.version import VERSION
|
||||
from backend.services.flacfetch_client import get_flacfetch_client
|
||||
from backend.services.email_service import get_email_service
|
||||
from backend.services.stripe_service import get_stripe_service
|
||||
from backend.services.encoding_service import get_encoding_service
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def get_system_info() -> Dict[str, Any]:
|
||||
"""Get system information for performance analysis."""
|
||||
info = {
|
||||
"platform": platform.system(),
|
||||
"platform_release": platform.release(),
|
||||
"python_version": platform.python_version(),
|
||||
}
|
||||
|
||||
# Get CPU info
|
||||
try:
|
||||
cpu_count = os.cpu_count() or 0
|
||||
info["cpu_count"] = cpu_count
|
||||
|
||||
# Try to get more detailed CPU info on Linux
|
||||
if platform.system() == "Linux":
|
||||
try:
|
||||
with open("/proc/cpuinfo", "r") as f:
|
||||
for line in f:
|
||||
if "model name" in line:
|
||||
info["cpu_model"] = line.split(":")[1].strip()
|
||||
break
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as e:
|
||||
info["cpu_error"] = str(e)
|
||||
|
||||
# Get memory info
|
||||
try:
|
||||
if platform.system() == "Linux":
|
||||
with open("/proc/meminfo", "r") as f:
|
||||
for line in f:
|
||||
if line.startswith("MemTotal:"):
|
||||
mem_kb = int(line.split()[1])
|
||||
info["memory_gb"] = round(mem_kb / (1024 * 1024), 1)
|
||||
break
|
||||
except Exception as e:
|
||||
info["memory_error"] = str(e)
|
||||
|
||||
return info
|
||||
|
||||
|
||||
def get_ffmpeg_info() -> Dict[str, Any]:
|
||||
"""Get FFmpeg version and configuration for debugging encoding performance."""
|
||||
try:
|
||||
# Get FFmpeg version
|
||||
result = subprocess.run(
|
||||
["ffmpeg", "-version"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10
|
||||
)
|
||||
if result.returncode == 0:
|
||||
lines = result.stdout.split("\n")
|
||||
version_line = lines[0] if lines else "unknown"
|
||||
|
||||
# Check for hardware acceleration support
|
||||
hw_accel = {}
|
||||
config_result = subprocess.run(
|
||||
["ffmpeg", "-hwaccels"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10
|
||||
)
|
||||
if config_result.returncode == 0:
|
||||
hw_lines = config_result.stdout.strip().split("\n")
|
||||
hw_accel["available"] = [h.strip() for h in hw_lines[1:] if h.strip()]
|
||||
|
||||
# Check encoder availability
|
||||
encoders = {}
|
||||
encoder_result = subprocess.run(
|
||||
["ffmpeg", "-encoders"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10
|
||||
)
|
||||
if encoder_result.returncode == 0:
|
||||
encoder_output = encoder_result.stdout
|
||||
# Check for specific encoders we care about
|
||||
encoders["libx264"] = "libx264" in encoder_output
|
||||
encoders["h264_nvenc"] = "h264_nvenc" in encoder_output
|
||||
encoders["h264_vaapi"] = "h264_vaapi" in encoder_output
|
||||
encoders["hevc_nvenc"] = "hevc_nvenc" in encoder_output
|
||||
|
||||
return {
|
||||
"available": True,
|
||||
"version": version_line,
|
||||
"hw_acceleration": hw_accel,
|
||||
"encoders": encoders,
|
||||
}
|
||||
else:
|
||||
return {
|
||||
"available": False,
|
||||
"error": result.stderr[:500] if result.stderr else "Unknown error",
|
||||
}
|
||||
except subprocess.TimeoutExpired:
|
||||
return {"available": False, "error": "Timeout getting FFmpeg info"}
|
||||
except FileNotFoundError:
|
||||
return {"available": False, "error": "FFmpeg not found"}
|
||||
except Exception as e:
|
||||
return {"available": False, "error": str(e)}
|
||||
|
||||
|
||||
def check_transmission_status() -> Dict[str, Any]:
|
||||
"""Check if Transmission daemon is available and responsive."""
|
||||
try:
|
||||
@@ -99,34 +206,96 @@ async def check_flacfetch_service_status() -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
async def check_encoding_worker_status() -> Dict[str, Any]:
|
||||
"""Check if GCE encoding worker is available and healthy."""
|
||||
encoding_service = get_encoding_service()
|
||||
|
||||
if not encoding_service.is_configured:
|
||||
return {
|
||||
"configured": False,
|
||||
"enabled": encoding_service.settings.use_gce_encoding,
|
||||
"message": "GCE encoding worker not configured (ENCODING_WORKER_URL or API key not set)",
|
||||
}
|
||||
|
||||
try:
|
||||
health = await encoding_service.health_check()
|
||||
return {
|
||||
"configured": True,
|
||||
"enabled": encoding_service.is_enabled,
|
||||
"available": health.get("status") == "ok",
|
||||
"status": health.get("status"),
|
||||
"active_jobs": health.get("active_jobs", 0),
|
||||
"queue_length": health.get("queue_length", 0),
|
||||
"ffmpeg_version": health.get("ffmpeg_version"),
|
||||
}
|
||||
except Exception as e:
|
||||
return {
|
||||
"configured": True,
|
||||
"enabled": encoding_service.is_enabled,
|
||||
"available": False,
|
||||
"error": str(e),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health/detailed")
|
||||
async def detailed_health_check() -> Dict[str, Any]:
|
||||
"""
|
||||
Detailed health check including dependencies.
|
||||
|
||||
|
||||
Use this to debug issues with Transmission, flacfetch service, etc.
|
||||
"""
|
||||
transmission_status = check_transmission_status()
|
||||
flacfetch_status = await check_flacfetch_service_status()
|
||||
|
||||
encoding_status = await check_encoding_worker_status()
|
||||
|
||||
# Check email service
|
||||
email_service = get_email_service()
|
||||
email_status = {
|
||||
"configured": email_service.is_configured(),
|
||||
"provider": type(email_service.provider).__name__,
|
||||
}
|
||||
|
||||
# Check Stripe service
|
||||
stripe_service = get_stripe_service()
|
||||
stripe_status = {
|
||||
"configured": stripe_service.is_configured(),
|
||||
}
|
||||
|
||||
# Log for debugging
|
||||
if not transmission_status.get("available"):
|
||||
logger.warning(f"Local Transmission not available: {transmission_status.get('error')}")
|
||||
else:
|
||||
logger.info(f"Local Transmission available at {transmission_status.get('host')}:{transmission_status.get('port')}")
|
||||
|
||||
|
||||
if flacfetch_status.get("configured") and not flacfetch_status.get("available"):
|
||||
logger.warning(f"Remote flacfetch service not available: {flacfetch_status.get('error')}")
|
||||
elif flacfetch_status.get("available"):
|
||||
logger.info(f"Remote flacfetch service healthy: {flacfetch_status.get('status')}")
|
||||
|
||||
|
||||
if not email_status["configured"]:
|
||||
logger.warning("Email service not configured - magic links will not work")
|
||||
|
||||
if not stripe_status["configured"]:
|
||||
logger.warning("Stripe service not configured - payments will not work")
|
||||
|
||||
# Get system and FFmpeg info for performance analysis
|
||||
system_info = get_system_info()
|
||||
ffmpeg_info = get_ffmpeg_info()
|
||||
|
||||
return {
|
||||
"status": "healthy",
|
||||
"service": "karaoke-gen-backend",
|
||||
"version": VERSION,
|
||||
"system": system_info,
|
||||
"ffmpeg": ffmpeg_info,
|
||||
"dependencies": {
|
||||
"transmission_local": transmission_status,
|
||||
"flacfetch_remote": flacfetch_status,
|
||||
"encoding_worker": encoding_status,
|
||||
},
|
||||
"services": {
|
||||
"email": email_status,
|
||||
"stripe": stripe_status,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+323
-51
@@ -27,7 +27,7 @@ from backend.services.worker_service import get_worker_service
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.config import get_settings
|
||||
from backend.api.dependencies import require_admin, require_auth, require_instrumental_auth
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.services.auth_service import UserType, AuthResult
|
||||
from backend.services.metrics import metrics
|
||||
|
||||
|
||||
@@ -57,7 +57,7 @@ async def _trigger_workers_parallel(job_id: str) -> None:
|
||||
async def create_job(
|
||||
request: URLSubmissionRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> JobResponse:
|
||||
"""
|
||||
Create a new karaoke generation job from a URL.
|
||||
@@ -69,6 +69,24 @@ async def create_job(
|
||||
4. When both complete, job transitions to AWAITING_REVIEW
|
||||
"""
|
||||
try:
|
||||
# Determine job owner email:
|
||||
# All authentication methods must provide a user_email for job ownership
|
||||
if auth_result.user_email:
|
||||
# Use authenticated user's email (standard case)
|
||||
user_email = auth_result.user_email
|
||||
else:
|
||||
# This should never happen - all auth methods now require user_email
|
||||
logger.error("Authentication succeeded but no user_email provided")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="Authentication error: no user identity available"
|
||||
)
|
||||
|
||||
# Admins can optionally create jobs on behalf of other users
|
||||
if request.user_email and auth_result.is_admin and request.user_email != auth_result.user_email:
|
||||
user_email = request.user_email
|
||||
logger.info(f"Admin {auth_result.user_email} creating job on behalf of {user_email}")
|
||||
|
||||
# Create job with all preferences
|
||||
job_create = JobCreate(
|
||||
url=str(request.url),
|
||||
@@ -79,7 +97,7 @@ async def create_job(
|
||||
enable_youtube_upload=request.enable_youtube_upload,
|
||||
youtube_description=request.youtube_description,
|
||||
webhook_url=request.webhook_url,
|
||||
user_email=request.user_email
|
||||
user_email=user_email
|
||||
)
|
||||
job = job_manager.create_job(job_create)
|
||||
|
||||
@@ -109,20 +127,49 @@ async def create_job(
|
||||
@router.get("/{job_id}", response_model=Job)
|
||||
async def get_job(
|
||||
job_id: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> Job:
|
||||
"""Get job status and details."""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership - users can only see their own jobs, admins can see all
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
# If job is complete, include download URLs
|
||||
if job.status == JobStatus.COMPLETE:
|
||||
job.download_urls = job_manager.get_output_urls(job_id)
|
||||
|
||||
|
||||
return job
|
||||
|
||||
|
||||
def _check_job_ownership(job: Job, auth_result: AuthResult) -> bool:
|
||||
"""
|
||||
Check if the authenticated user owns the job or has admin access.
|
||||
|
||||
Returns:
|
||||
True if user can access the job, False otherwise
|
||||
"""
|
||||
# Admins can access all jobs
|
||||
if auth_result.is_admin:
|
||||
return True
|
||||
|
||||
# Check if user owns the job
|
||||
if auth_result.user_email and job.user_email:
|
||||
return auth_result.user_email.lower() == job.user_email.lower()
|
||||
|
||||
# If no user_email on auth (token auth without email), deny access to jobs with user_email
|
||||
# This prevents token-based auth from accessing user jobs
|
||||
if job.user_email:
|
||||
return False
|
||||
|
||||
# Legacy jobs without user_email - allow access for backward compatibility
|
||||
# TODO: Consider restricting this in the future
|
||||
return True
|
||||
|
||||
|
||||
@router.get("", response_model=List[Job])
|
||||
async def list_jobs(
|
||||
status: Optional[JobStatus] = None,
|
||||
@@ -131,11 +178,13 @@ async def list_jobs(
|
||||
created_after: Optional[str] = None,
|
||||
created_before: Optional[str] = None,
|
||||
limit: int = 100,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> List[Job]:
|
||||
"""
|
||||
List jobs with optional filters.
|
||||
|
||||
|
||||
Regular users only see their own jobs. Admins see all jobs.
|
||||
|
||||
Args:
|
||||
status: Filter by job status (pending, complete, failed, etc.)
|
||||
environment: Filter by request_metadata.environment (test/production/development)
|
||||
@@ -143,37 +192,52 @@ async def list_jobs(
|
||||
created_after: Filter jobs created after this ISO datetime (e.g., 2024-01-01T00:00:00Z)
|
||||
created_before: Filter jobs created before this ISO datetime
|
||||
limit: Maximum number of jobs to return (default 100)
|
||||
|
||||
|
||||
Returns:
|
||||
List of jobs matching filters, ordered by created_at descending
|
||||
"""
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
try:
|
||||
# Parse datetime strings if provided
|
||||
created_after_dt = None
|
||||
created_before_dt = None
|
||||
|
||||
|
||||
if created_after:
|
||||
try:
|
||||
created_after_dt = datetime.fromisoformat(created_after.replace('Z', '+00:00'))
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_after format: {created_after}")
|
||||
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_after format: {created_after}") from e
|
||||
|
||||
if created_before:
|
||||
try:
|
||||
created_before_dt = datetime.fromisoformat(created_before.replace('Z', '+00:00'))
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_before format: {created_before}")
|
||||
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid created_before format: {created_before}") from e
|
||||
|
||||
# Determine user_email filter based on admin status
|
||||
# Admins see all jobs, regular users only see their own
|
||||
user_email_filter = None
|
||||
if not auth_result.is_admin:
|
||||
if auth_result.user_email:
|
||||
user_email_filter = auth_result.user_email
|
||||
logger.debug(f"Filtering jobs for user: {user_email_filter}")
|
||||
else:
|
||||
# Token-based auth without user email - show no jobs for security
|
||||
logger.warning("Non-admin auth without user_email, returning empty job list")
|
||||
return []
|
||||
|
||||
jobs = job_manager.list_jobs(
|
||||
status=status,
|
||||
environment=environment,
|
||||
client_id=client_id,
|
||||
created_after=created_after_dt,
|
||||
created_before=created_before_dt,
|
||||
user_email=user_email_filter,
|
||||
limit=limit
|
||||
)
|
||||
|
||||
logger.debug(f"Listed {len(jobs)} jobs for user={auth_result.user_email}, admin={auth_result.is_admin}")
|
||||
return jobs
|
||||
except HTTPException:
|
||||
raise
|
||||
@@ -186,16 +250,20 @@ async def list_jobs(
|
||||
async def delete_job(
|
||||
job_id: str,
|
||||
delete_files: bool = True,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""Delete a job and optionally its output files."""
|
||||
try:
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership - users can only delete their own jobs
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to delete this job")
|
||||
|
||||
job_manager.delete_job(job_id, delete_files=delete_files)
|
||||
|
||||
|
||||
return {"status": "success", "message": f"Job {job_id} deleted"}
|
||||
except HTTPException:
|
||||
raise
|
||||
@@ -212,7 +280,7 @@ async def bulk_delete_jobs(
|
||||
created_before: Optional[str] = None,
|
||||
delete_files: bool = True,
|
||||
confirm: bool = False,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
|
||||
_auth_result: AuthResult = Depends(require_admin)
|
||||
) -> dict:
|
||||
"""
|
||||
Delete multiple jobs matching filter criteria.
|
||||
@@ -316,18 +384,22 @@ async def bulk_delete_jobs(
|
||||
@router.get("/{job_id}/review-data")
|
||||
async def get_review_data(
|
||||
job_id: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Get data needed for lyrics review interface.
|
||||
|
||||
|
||||
Returns corrections JSON URL and audio URL.
|
||||
Frontend loads these to render the review UI.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -376,13 +448,21 @@ async def get_review_data(
|
||||
async def start_review(
|
||||
job_id: str,
|
||||
request: StartReviewRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Mark job as IN_REVIEW (user opened review interface).
|
||||
|
||||
|
||||
This helps track that the user is actively working on the review.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
success = job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.IN_REVIEW,
|
||||
@@ -400,20 +480,24 @@ async def submit_corrections(
|
||||
job_id: str,
|
||||
submission: CorrectionsSubmission,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Save corrected lyrics during human review.
|
||||
|
||||
|
||||
This endpoint saves review progress but does NOT complete the review.
|
||||
Call POST /{job_id}/complete-review to finish and trigger video rendering.
|
||||
|
||||
|
||||
Can be called multiple times to save progress.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to modify this job")
|
||||
|
||||
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -459,11 +543,11 @@ async def submit_corrections(
|
||||
async def complete_review(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Complete the human review and trigger video rendering.
|
||||
|
||||
|
||||
This is the FIRST critical human-in-the-loop completion point.
|
||||
After this:
|
||||
1. Job transitions to REVIEW_COMPLETE
|
||||
@@ -474,7 +558,11 @@ async def complete_review(
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to modify this job")
|
||||
|
||||
if job.status not in [JobStatus.AWAITING_REVIEW, JobStatus.IN_REVIEW]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -948,18 +1036,22 @@ async def select_instrumental(
|
||||
@router.get("/{job_id}/download-urls")
|
||||
async def get_download_urls(
|
||||
job_id: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Get download URLs for all job output files.
|
||||
|
||||
|
||||
Returns a dictionary mapping file types to download URLs.
|
||||
Uses the streaming download endpoint which proxies through the backend.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
if job.status != JobStatus.COMPLETE:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -990,26 +1082,42 @@ async def get_download_urls(
|
||||
}
|
||||
|
||||
|
||||
# Map file keys to human-readable suffixes (matching Dropbox naming from karaoke_finalise.py)
|
||||
DOWNLOAD_FILENAME_SUFFIXES = {
|
||||
"lossless_4k_mp4": " (Final Karaoke Lossless 4k).mp4",
|
||||
"lossless_4k_mkv": " (Final Karaoke Lossless 4k).mkv",
|
||||
"lossy_4k_mp4": " (Final Karaoke Lossy 4k).mp4",
|
||||
"lossy_720p_mp4": " (Final Karaoke Lossy 720p).mp4",
|
||||
"cdg_zip": " (Final Karaoke CDG).zip",
|
||||
"txt_zip": " (Final Karaoke TXT).zip",
|
||||
"with_vocals": " (With Vocals).mkv",
|
||||
}
|
||||
|
||||
|
||||
@router.get("/{job_id}/download/{category}/{file_key}")
|
||||
async def download_file(
|
||||
job_id: str,
|
||||
category: str,
|
||||
file_key: str,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
):
|
||||
"""
|
||||
Stream download a specific file from a completed job.
|
||||
|
||||
|
||||
This endpoint proxies the file from GCS through the backend,
|
||||
so no client-side authentication is required.
|
||||
"""
|
||||
from fastapi.responses import StreamingResponse
|
||||
import tempfile
|
||||
|
||||
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access this job")
|
||||
|
||||
if job.status != JobStatus.COMPLETE:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1050,8 +1158,28 @@ async def download_file(
|
||||
}
|
||||
content_type = content_types.get(ext, 'application/octet-stream')
|
||||
|
||||
# Get filename for Content-Disposition
|
||||
filename = gcs_path.split('/')[-1]
|
||||
# Build proper filename: "Artist - Title (Final Karaoke Lossy 4k).mp4"
|
||||
# Sanitize artist/title to remove filesystem-unsafe characters
|
||||
def sanitize_filename_part(s: str) -> str:
|
||||
if not s:
|
||||
return ""
|
||||
# Remove/replace characters unsafe for filesystems: / \ : * ? " < > |
|
||||
unsafe_chars = r'[/\\:*?"<>|]'
|
||||
import re
|
||||
sanitized = re.sub(unsafe_chars, '', s)
|
||||
# Collapse multiple spaces and trim
|
||||
sanitized = ' '.join(sanitized.split())
|
||||
# Limit length to prevent overly long filenames
|
||||
return sanitized[:100].strip()
|
||||
|
||||
artist_clean = sanitize_filename_part(job.artist) if job.artist else None
|
||||
title_clean = sanitize_filename_part(job.title) if job.title else None
|
||||
base_name = f"{artist_clean} - {title_clean}" if artist_clean and title_clean else None
|
||||
|
||||
if base_name and file_key in DOWNLOAD_FILENAME_SUFFIXES:
|
||||
filename = f"{base_name}{DOWNLOAD_FILENAME_SUFFIXES[file_key]}"
|
||||
else:
|
||||
filename = gcs_path.split('/')[-1] # Fallback to original
|
||||
|
||||
try:
|
||||
# Download to temp file and stream
|
||||
@@ -1086,13 +1214,21 @@ async def download_file(
|
||||
async def cancel_job(
|
||||
job_id: str,
|
||||
request: CancelJobRequest,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Cancel a job.
|
||||
|
||||
|
||||
Jobs can be cancelled at any stage before completion.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to cancel this job")
|
||||
|
||||
success = job_manager.cancel_job(job_id, reason=request.reason)
|
||||
|
||||
if not success:
|
||||
@@ -1109,7 +1245,7 @@ async def cancel_job(
|
||||
async def retry_job(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""
|
||||
Retry a failed or cancelled job from the last successful checkpoint.
|
||||
@@ -1127,6 +1263,10 @@ async def retry_job(
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Check ownership
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to retry this job")
|
||||
|
||||
if job.status not in [JobStatus.FAILED, JobStatus.CANCELLED]:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
@@ -1301,19 +1441,19 @@ async def get_worker_logs(
|
||||
job_id: str,
|
||||
since_index: int = 0,
|
||||
worker: Optional[str] = None,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_auth)
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Get worker logs for debugging.
|
||||
|
||||
|
||||
This endpoint returns worker logs stored in Firestore.
|
||||
Use `since_index` for efficient polling (returns only new logs).
|
||||
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
since_index: Return only logs after this index (for pagination/polling)
|
||||
worker: Filter by worker name (audio, lyrics, screens, video, render)
|
||||
|
||||
|
||||
Returns:
|
||||
{
|
||||
"logs": [{"timestamp": "...", "level": "INFO", "worker": "audio", "message": "..."}],
|
||||
@@ -1324,13 +1464,145 @@ async def get_worker_logs(
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
|
||||
# Check ownership - users can only see logs for their own jobs
|
||||
if not _check_job_ownership(job, auth_result):
|
||||
raise HTTPException(status_code=403, detail="You don't have permission to access logs for this job")
|
||||
|
||||
logs = job_manager.get_worker_logs(job_id, since_index=since_index, worker=worker)
|
||||
total = len(job.worker_logs) if job.worker_logs else 0
|
||||
|
||||
|
||||
return {
|
||||
"logs": logs,
|
||||
"next_index": total,
|
||||
"total_logs": total
|
||||
}
|
||||
|
||||
|
||||
@router.post("/{job_id}/cleanup-distribution")
|
||||
async def cleanup_distribution(
|
||||
job_id: str,
|
||||
delete_job: bool = True,
|
||||
auth_result: AuthResult = Depends(require_admin)
|
||||
) -> dict:
|
||||
"""
|
||||
Clean up all distributed content for a job (YouTube, Dropbox, Google Drive).
|
||||
|
||||
This admin-only endpoint is designed for E2E test cleanup. It:
|
||||
1. Deletes YouTube video (if uploaded)
|
||||
2. Deletes Dropbox folder (if uploaded)
|
||||
3. Deletes Google Drive files (if uploaded)
|
||||
4. Optionally deletes the job itself
|
||||
|
||||
Args:
|
||||
job_id: Job ID to clean up
|
||||
delete_job: If True, also delete the job after cleaning up distribution
|
||||
|
||||
Returns:
|
||||
Cleanup results for each service
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
state_data = job.state_data or {}
|
||||
results = {
|
||||
"job_id": job_id,
|
||||
"youtube": {"status": "skipped", "reason": "no youtube_url in state_data"},
|
||||
"dropbox": {"status": "skipped", "reason": "no brand_code or dropbox_path"},
|
||||
"gdrive": {"status": "skipped", "reason": "no gdrive_files in state_data"},
|
||||
"job_deleted": False
|
||||
}
|
||||
|
||||
# Clean up YouTube
|
||||
youtube_url = state_data.get('youtube_url')
|
||||
if youtube_url:
|
||||
try:
|
||||
# Extract video ID from URL (format: https://youtu.be/VIDEO_ID or https://www.youtube.com/watch?v=VIDEO_ID)
|
||||
import re
|
||||
video_id_match = re.search(r'(?:youtu\.be/|youtube\.com/watch\?v=)([^&\s]+)', youtube_url)
|
||||
if video_id_match:
|
||||
video_id = video_id_match.group(1)
|
||||
|
||||
# Import and use karaoke_finalise for YouTube deletion
|
||||
from karaoke_gen.karaoke_finalise.karaoke_finalise import KaraokeFinalise
|
||||
from backend.services.youtube_service import get_youtube_service
|
||||
|
||||
youtube_service = get_youtube_service()
|
||||
if youtube_service.is_configured:
|
||||
# Create minimal KaraokeFinalise instance for deletion
|
||||
finalise = KaraokeFinalise(
|
||||
dry_run=False,
|
||||
non_interactive=True,
|
||||
enable_youtube=True,
|
||||
user_youtube_credentials=youtube_service.get_credentials_dict()
|
||||
)
|
||||
success = finalise.delete_youtube_video(video_id)
|
||||
results["youtube"] = {
|
||||
"status": "success" if success else "failed",
|
||||
"video_id": video_id
|
||||
}
|
||||
else:
|
||||
results["youtube"] = {"status": "failed", "reason": "YouTube credentials not configured"}
|
||||
else:
|
||||
results["youtube"] = {"status": "failed", "reason": f"Could not extract video ID from {youtube_url}"}
|
||||
except Exception as e:
|
||||
logger.error(f"Error cleaning up YouTube for job {job_id}: {e}", exc_info=True)
|
||||
results["youtube"] = {"status": "error", "error": str(e)}
|
||||
|
||||
# Clean up Dropbox
|
||||
brand_code = state_data.get('brand_code')
|
||||
dropbox_path = getattr(job, 'dropbox_path', None)
|
||||
if brand_code and dropbox_path:
|
||||
try:
|
||||
from backend.services.dropbox_service import get_dropbox_service
|
||||
dropbox = get_dropbox_service()
|
||||
if dropbox.is_configured:
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
folder_name = f"{brand_code} - {base_name}"
|
||||
full_path = f"{dropbox_path}/{folder_name}"
|
||||
success = dropbox.delete_folder(full_path)
|
||||
results["dropbox"] = {
|
||||
"status": "success" if success else "failed",
|
||||
"path": full_path
|
||||
}
|
||||
else:
|
||||
results["dropbox"] = {"status": "failed", "reason": "Dropbox credentials not configured"}
|
||||
except Exception as e:
|
||||
logger.error(f"Error cleaning up Dropbox for job {job_id}: {e}", exc_info=True)
|
||||
results["dropbox"] = {"status": "error", "error": str(e)}
|
||||
|
||||
# Clean up Google Drive
|
||||
gdrive_files = state_data.get('gdrive_files')
|
||||
if gdrive_files:
|
||||
try:
|
||||
from backend.services.gdrive_service import get_gdrive_service
|
||||
gdrive = get_gdrive_service()
|
||||
if gdrive.is_configured:
|
||||
# gdrive_files is a dict like {"mp4": "file_id", "mp4_720p": "file_id", "cdg": "file_id"}
|
||||
file_ids = list(gdrive_files.values()) if isinstance(gdrive_files, dict) else []
|
||||
delete_results = gdrive.delete_files(file_ids)
|
||||
all_success = all(delete_results.values())
|
||||
results["gdrive"] = {
|
||||
"status": "success" if all_success else "partial",
|
||||
"files": delete_results
|
||||
}
|
||||
else:
|
||||
results["gdrive"] = {"status": "failed", "reason": "Google Drive credentials not configured"}
|
||||
except Exception as e:
|
||||
logger.error(f"Error cleaning up Google Drive for job {job_id}: {e}", exc_info=True)
|
||||
results["gdrive"] = {"status": "error", "error": str(e)}
|
||||
|
||||
# Delete the job if requested
|
||||
if delete_job:
|
||||
try:
|
||||
job_manager.delete_job(job_id, delete_files=True)
|
||||
results["job_deleted"] = True
|
||||
logger.info(f"Deleted job {job_id} after distribution cleanup")
|
||||
except Exception as e:
|
||||
logger.error(f"Error deleting job {job_id}: {e}", exc_info=True)
|
||||
results["job_deleted"] = False
|
||||
results["job_delete_error"] = str(e)
|
||||
|
||||
return results
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+17
-1
@@ -40,7 +40,15 @@ class Settings(BaseSettings):
|
||||
# Processing
|
||||
max_concurrent_jobs: int = int(os.getenv("MAX_CONCURRENT_JOBS", "5"))
|
||||
job_timeout_seconds: int = int(os.getenv("JOB_TIMEOUT_SECONDS", "3600"))
|
||||
|
||||
|
||||
# Agentic AI Correction (for lyrics correction via LLM)
|
||||
# When enabled, uses Gemini via Vertex AI for intelligent lyrics correction
|
||||
use_agentic_ai: bool = os.getenv("USE_AGENTIC_AI", "true").lower() in ("true", "1", "yes")
|
||||
agentic_ai_model: str = os.getenv("AGENTIC_AI_MODEL", "vertexai/gemini-3-flash-preview")
|
||||
# Timeout for agentic correction in seconds. If correction takes longer, abort and
|
||||
# use uncorrected transcription - human review will fix any issues.
|
||||
agentic_correction_timeout_seconds: int = int(os.getenv("AGENTIC_CORRECTION_TIMEOUT_SECONDS", "180"))
|
||||
|
||||
# Cloud Tasks (for scalable worker coordination)
|
||||
# When enabled, workers are triggered via Cloud Tasks for guaranteed delivery
|
||||
# When disabled (default), workers are triggered via direct HTTP (for development)
|
||||
@@ -52,6 +60,14 @@ class Settings(BaseSettings):
|
||||
# instead of Cloud Tasks. This supports encoding times >30 minutes (up to 24 hours).
|
||||
# Default is false - Cloud Tasks is sufficient for most videos (15-20 min).
|
||||
use_cloud_run_jobs_for_video: bool = os.getenv("USE_CLOUD_RUN_JOBS_FOR_VIDEO", "false").lower() in ("true", "1", "yes")
|
||||
|
||||
# GCE Encoding Worker (for high-performance video encoding)
|
||||
# When enabled, video encoding is offloaded to a dedicated C4 GCE instance
|
||||
# with faster CPU (Intel Granite Rapids 3.9 GHz) instead of Cloud Run.
|
||||
# This provides 2-3x faster encoding times for CPU-bound FFmpeg libx264 encoding.
|
||||
use_gce_encoding: bool = os.getenv("USE_GCE_ENCODING", "false").lower() in ("true", "1", "yes")
|
||||
encoding_worker_url: Optional[str] = os.getenv("ENCODING_WORKER_URL") # e.g., http://136.119.50.148:8080
|
||||
encoding_worker_api_key: Optional[str] = os.getenv("ENCODING_WORKER_API_KEY")
|
||||
|
||||
# Storage paths
|
||||
temp_dir: str = os.getenv("TEMP_DIR", "/tmp/karaoke-gen")
|
||||
|
||||
+7
-1
@@ -7,9 +7,10 @@ from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from backend.config import settings
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search, themes, users, admin
|
||||
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
|
||||
from backend.services.structured_logging import setup_structured_logging
|
||||
from backend.middleware.audit_logging import AuditLoggingMiddleware
|
||||
|
||||
|
||||
from backend.version import VERSION
|
||||
@@ -100,6 +101,9 @@ app.add_middleware(
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# Add audit logging middleware (captures all requests with request_id for correlation)
|
||||
app.add_middleware(AuditLoggingMiddleware)
|
||||
|
||||
# Include routers
|
||||
app.include_router(health.router, prefix="/api")
|
||||
app.include_router(jobs.router, prefix="/api")
|
||||
@@ -109,6 +113,8 @@ app.include_router(review.router, prefix="/api") # Review UI compatibility endp
|
||||
app.include_router(auth.router, prefix="/api") # OAuth credential management
|
||||
app.include_router(audio_search.router, prefix="/api") # Audio search (artist+title mode)
|
||||
app.include_router(themes.router, prefix="/api") # Theme selection for styles
|
||||
app.include_router(users.router, prefix="/api") # User auth, credits, and Stripe webhooks
|
||||
app.include_router(admin.router, prefix="/api") # Admin dashboard and management
|
||||
|
||||
|
||||
@app.get("/")
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
"""Middleware package for FastAPI application."""
|
||||
|
||||
from backend.middleware.audit_logging import AuditLoggingMiddleware
|
||||
|
||||
__all__ = ["AuditLoggingMiddleware"]
|
||||
@@ -0,0 +1,124 @@
|
||||
"""
|
||||
Audit logging middleware for tracking all HTTP requests.
|
||||
|
||||
This middleware captures request metadata and logs it to Cloud Logging
|
||||
for audit trail and user activity investigation purposes.
|
||||
"""
|
||||
import logging
|
||||
import time
|
||||
import uuid
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
|
||||
|
||||
logger = logging.getLogger("audit")
|
||||
|
||||
# Endpoints to exclude from audit logging (high-frequency health checks)
|
||||
EXCLUDED_PATHS = {
|
||||
"/",
|
||||
"/api/health",
|
||||
"/api/health/detailed",
|
||||
"/api/readiness",
|
||||
"/healthz",
|
||||
"/ready",
|
||||
}
|
||||
|
||||
|
||||
class AuditLoggingMiddleware(BaseHTTPMiddleware):
|
||||
"""
|
||||
Middleware that logs all HTTP requests for audit purposes.
|
||||
|
||||
Captures:
|
||||
- request_id: UUID for correlating with auth logs
|
||||
- method: HTTP method
|
||||
- path: Request path
|
||||
- status_code: Response status
|
||||
- latency_ms: Request duration
|
||||
- client_ip: Client IP (from X-Forwarded-For for proxied requests)
|
||||
- user_agent: Browser/client identifier
|
||||
|
||||
The request_id is stored in request.state and added to response headers,
|
||||
allowing correlation with auth logs that capture user_email.
|
||||
"""
|
||||
|
||||
async def dispatch(self, request: Request, call_next) -> Response:
|
||||
# Skip excluded paths (health checks)
|
||||
if request.url.path in EXCLUDED_PATHS:
|
||||
return await call_next(request)
|
||||
|
||||
# Generate unique request ID for correlation
|
||||
request_id = str(uuid.uuid4())
|
||||
request.state.request_id = request_id
|
||||
|
||||
# Capture start time
|
||||
start_time = time.time()
|
||||
|
||||
# Extract client info
|
||||
client_ip = self._get_client_ip(request)
|
||||
user_agent = request.headers.get("user-agent", "")
|
||||
|
||||
# Process request
|
||||
try:
|
||||
response = await call_next(request)
|
||||
except Exception:
|
||||
# Log failed requests too (exception() auto-includes stack trace)
|
||||
latency_ms = int((time.time() - start_time) * 1000)
|
||||
logger.exception(
|
||||
"request_audit_error",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"method": request.method,
|
||||
"path": request.url.path,
|
||||
"query_string": str(request.query_params) if request.query_params else None,
|
||||
"latency_ms": latency_ms,
|
||||
"client_ip": client_ip,
|
||||
"user_agent": user_agent[:200] if user_agent else None,
|
||||
"audit_type": "http_request",
|
||||
}
|
||||
)
|
||||
raise
|
||||
|
||||
# Calculate latency
|
||||
latency_ms = int((time.time() - start_time) * 1000)
|
||||
|
||||
# Log audit entry (INFO level for successful requests)
|
||||
log_level = logging.WARNING if response.status_code >= 400 else logging.INFO
|
||||
logger.log(
|
||||
log_level,
|
||||
"request_audit",
|
||||
extra={
|
||||
"request_id": request_id,
|
||||
"method": request.method,
|
||||
"path": request.url.path,
|
||||
"query_string": str(request.query_params) if request.query_params else None,
|
||||
"status_code": response.status_code,
|
||||
"latency_ms": latency_ms,
|
||||
"client_ip": client_ip,
|
||||
"user_agent": user_agent[:200] if user_agent else None,
|
||||
"audit_type": "http_request",
|
||||
}
|
||||
)
|
||||
|
||||
# Add request_id to response headers for debugging/correlation
|
||||
response.headers["X-Request-ID"] = request_id
|
||||
|
||||
return response
|
||||
|
||||
def _get_client_ip(self, request: Request) -> str:
|
||||
"""
|
||||
Extract client IP address, handling proxy scenarios.
|
||||
|
||||
Cloud Run and other proxies set X-Forwarded-For header.
|
||||
"""
|
||||
# Check X-Forwarded-For for proxy scenarios (Cloud Run, load balancers)
|
||||
forwarded = request.headers.get("x-forwarded-for", "")
|
||||
if forwarded:
|
||||
# First IP is the original client
|
||||
return forwarded.split(",")[0].strip()
|
||||
|
||||
# Fall back to direct connection
|
||||
if request.client:
|
||||
return request.client.host
|
||||
|
||||
return "unknown"
|
||||
@@ -204,6 +204,7 @@ class Job(BaseModel):
|
||||
youtube_description: Optional[str] = None # YouTube video description
|
||||
webhook_url: Optional[str] = None # Webhook for notifications
|
||||
user_email: Optional[str] = None # Email for notifications
|
||||
non_interactive: bool = False # Skip interactive steps (lyrics review, instrumental selection)
|
||||
|
||||
# Theme configuration (pre-made themes from GCS)
|
||||
theme_id: Optional[str] = None # Theme identifier (e.g., "nomad", "default")
|
||||
@@ -419,6 +420,7 @@ class JobCreate(BaseModel):
|
||||
youtube_description: Optional[str] = None
|
||||
webhook_url: Optional[str] = None
|
||||
user_email: Optional[str] = None
|
||||
non_interactive: bool = False # Skip interactive steps (lyrics review, instrumental selection)
|
||||
|
||||
# Theme configuration (pre-made themes from GCS)
|
||||
theme_id: Optional[str] = None # Theme identifier (e.g., "nomad", "default")
|
||||
|
||||
@@ -0,0 +1,254 @@
|
||||
"""
|
||||
User model for authentication and credits.
|
||||
|
||||
Supports:
|
||||
- Magic link authentication (email-based)
|
||||
- Credit system for karaoke generation
|
||||
- Role-based access control (user/admin)
|
||||
- Stripe integration for payments
|
||||
- Beta tester program with feedback collection
|
||||
"""
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from typing import Optional, List
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class UserRole(str, Enum):
|
||||
"""User roles for access control."""
|
||||
USER = "user"
|
||||
ADMIN = "admin"
|
||||
|
||||
|
||||
class BetaTesterStatus(str, Enum):
|
||||
"""Status of beta tester participation."""
|
||||
ACTIVE = "active" # Enrolled, free credits available
|
||||
PENDING_FEEDBACK = "pending_feedback" # Job completed, awaiting feedback
|
||||
COMPLETED = "completed" # Feedback submitted
|
||||
EXPIRED = "expired" # 24hr deadline passed without feedback
|
||||
|
||||
|
||||
class CreditTransaction(BaseModel):
|
||||
"""Record of a credit transaction."""
|
||||
id: str
|
||||
amount: int # Positive = add, negative = deduct
|
||||
reason: str # e.g., "purchase", "refund", "admin_grant", "job_creation"
|
||||
job_id: Optional[str] = None
|
||||
stripe_session_id: Optional[str] = None
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
created_by: Optional[str] = None # Admin email if granted by admin
|
||||
|
||||
|
||||
class User(BaseModel):
|
||||
"""
|
||||
User model stored in Firestore.
|
||||
|
||||
Users are identified by email address. Authentication is via magic links
|
||||
(no passwords). Credits are consumed when creating karaoke jobs.
|
||||
"""
|
||||
email: str # Primary identifier
|
||||
role: UserRole = UserRole.USER
|
||||
credits: int = 0
|
||||
|
||||
# Stripe integration
|
||||
stripe_customer_id: Optional[str] = None
|
||||
|
||||
# Account state
|
||||
is_active: bool = True
|
||||
email_verified: bool = False
|
||||
|
||||
# Timestamps
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
updated_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
last_login_at: Optional[datetime] = None
|
||||
|
||||
# Credit history (last 100 transactions)
|
||||
credit_transactions: List[CreditTransaction] = Field(default_factory=list)
|
||||
|
||||
# Job tracking
|
||||
total_jobs_created: int = 0
|
||||
total_jobs_completed: int = 0
|
||||
|
||||
# Optional profile fields for future use
|
||||
display_name: Optional[str] = None
|
||||
|
||||
# Beta tester program
|
||||
is_beta_tester: bool = False
|
||||
beta_tester_status: Optional[BetaTesterStatus] = None
|
||||
beta_enrolled_at: Optional[datetime] = None
|
||||
beta_promise_text: Optional[str] = None # User's promise statement
|
||||
beta_feedback_due_at: Optional[datetime] = None # 24hr after job completion
|
||||
beta_feedback_email_sent: bool = False
|
||||
|
||||
|
||||
class MagicLinkToken(BaseModel):
|
||||
"""
|
||||
Magic link token for passwordless authentication.
|
||||
|
||||
Tokens are short-lived (15 minutes) and single-use.
|
||||
Stored in Firestore with automatic TTL.
|
||||
"""
|
||||
token: str # Secure random token
|
||||
email: str
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
expires_at: datetime
|
||||
used: bool = False
|
||||
used_at: Optional[datetime] = None
|
||||
ip_address: Optional[str] = None
|
||||
user_agent: Optional[str] = None
|
||||
|
||||
|
||||
class Session(BaseModel):
|
||||
"""
|
||||
User session for authenticated requests.
|
||||
|
||||
Sessions are created after successful magic link verification.
|
||||
Sessions expire after 7 days of inactivity or 30 days absolute.
|
||||
"""
|
||||
token: str # Secure random session token
|
||||
user_email: str
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
expires_at: datetime
|
||||
last_activity_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
ip_address: Optional[str] = None
|
||||
user_agent: Optional[str] = None
|
||||
is_active: bool = True
|
||||
|
||||
|
||||
# Pydantic models for API requests/responses
|
||||
|
||||
class SendMagicLinkRequest(BaseModel):
|
||||
"""Request to send a magic link email."""
|
||||
email: str
|
||||
|
||||
|
||||
class SendMagicLinkResponse(BaseModel):
|
||||
"""Response after sending magic link."""
|
||||
status: str
|
||||
message: str
|
||||
|
||||
|
||||
class VerifyMagicLinkRequest(BaseModel):
|
||||
"""Request to verify a magic link token."""
|
||||
token: str
|
||||
|
||||
|
||||
class VerifyMagicLinkResponse(BaseModel):
|
||||
"""Response after verifying magic link."""
|
||||
status: str
|
||||
session_token: str
|
||||
user: "UserPublic"
|
||||
message: str
|
||||
|
||||
|
||||
class UserPublic(BaseModel):
|
||||
"""Public user information (safe to expose to frontend)."""
|
||||
email: str
|
||||
role: UserRole
|
||||
credits: int
|
||||
display_name: Optional[str] = None
|
||||
total_jobs_created: int = 0
|
||||
total_jobs_completed: int = 0
|
||||
|
||||
|
||||
class AddCreditsRequest(BaseModel):
|
||||
"""Admin request to add credits to a user."""
|
||||
email: str
|
||||
amount: int
|
||||
reason: str = "admin_grant"
|
||||
|
||||
|
||||
class AddCreditsResponse(BaseModel):
|
||||
"""Response after adding credits."""
|
||||
status: str
|
||||
email: str
|
||||
credits_added: int
|
||||
new_balance: int
|
||||
message: str
|
||||
|
||||
|
||||
class UserListResponse(BaseModel):
|
||||
"""Response for listing users (admin only)."""
|
||||
users: List[UserPublic]
|
||||
total: int
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Beta Tester Program Models
|
||||
# ============================================================================
|
||||
|
||||
class BetaTesterFeedback(BaseModel):
|
||||
"""
|
||||
Feedback from a beta tester after using the service.
|
||||
|
||||
Stored in Firestore 'beta_feedback' collection.
|
||||
"""
|
||||
id: str # Unique feedback ID
|
||||
user_email: str
|
||||
job_id: Optional[str] = None # The job they're providing feedback on
|
||||
|
||||
# Ratings (1-5 scale)
|
||||
overall_rating: int = Field(ge=1, le=5)
|
||||
ease_of_use_rating: int = Field(ge=1, le=5)
|
||||
lyrics_accuracy_rating: int = Field(ge=1, le=5)
|
||||
correction_experience_rating: int = Field(ge=1, le=5)
|
||||
|
||||
# Open-ended feedback
|
||||
what_went_well: Optional[str] = None
|
||||
what_could_improve: Optional[str] = None
|
||||
additional_comments: Optional[str] = None
|
||||
|
||||
# Would they recommend / use again?
|
||||
would_recommend: bool = True
|
||||
would_use_again: bool = True
|
||||
|
||||
# Metadata
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
submitted_via: str = "web" # 'web' or 'email'
|
||||
|
||||
|
||||
class BetaTesterEnrollRequest(BaseModel):
|
||||
"""Request to enroll as a beta tester."""
|
||||
email: str
|
||||
promise_text: str = Field(
|
||||
min_length=10,
|
||||
description="User's promise to provide feedback (min 10 chars)"
|
||||
)
|
||||
accept_corrections_work: bool = Field(
|
||||
description="User accepts they may need to review/correct lyrics"
|
||||
)
|
||||
|
||||
|
||||
class BetaTesterEnrollResponse(BaseModel):
|
||||
"""Response after enrolling as beta tester."""
|
||||
status: str
|
||||
message: str
|
||||
credits_granted: int
|
||||
session_token: Optional[str] = None # If new user, provide session
|
||||
|
||||
|
||||
class BetaFeedbackRequest(BaseModel):
|
||||
"""Request to submit beta tester feedback."""
|
||||
job_id: Optional[str] = None
|
||||
|
||||
# Ratings (1-5 scale)
|
||||
overall_rating: int = Field(ge=1, le=5)
|
||||
ease_of_use_rating: int = Field(ge=1, le=5)
|
||||
lyrics_accuracy_rating: int = Field(ge=1, le=5)
|
||||
correction_experience_rating: int = Field(ge=1, le=5)
|
||||
|
||||
# Open-ended feedback
|
||||
what_went_well: Optional[str] = None
|
||||
what_could_improve: Optional[str] = None
|
||||
additional_comments: Optional[str] = None
|
||||
|
||||
# Would they recommend / use again?
|
||||
would_recommend: bool = True
|
||||
would_use_again: bool = True
|
||||
|
||||
|
||||
class BetaFeedbackResponse(BaseModel):
|
||||
"""Response after submitting feedback."""
|
||||
status: str
|
||||
message: str
|
||||
bonus_credits: int = 0 # Bonus credits for great feedback
|
||||
@@ -276,20 +276,35 @@ class AudioSearchService:
|
||||
Raises:
|
||||
DownloadError: If download fails or no cached result for index
|
||||
"""
|
||||
if result_index < 0 or result_index >= len(self._cached_results):
|
||||
raise DownloadError(
|
||||
f"No cached result for index {result_index}. "
|
||||
f"Available indices: 0-{len(self._cached_results) - 1}. "
|
||||
"Run search() first."
|
||||
)
|
||||
# Use provided search_id or fall back to service-level cache
|
||||
effective_search_id = remote_search_id or self._remote_search_id
|
||||
|
||||
# Check if we have local cached results
|
||||
has_local_cache = result_index >= 0 and result_index < len(self._cached_results)
|
||||
|
||||
# If no local cache but we have remote search_id, use remote download
|
||||
# This handles multi-instance deployments where the cache doesn't persist
|
||||
if not has_local_cache:
|
||||
if effective_search_id and self._remote_client:
|
||||
logger.info(f"No local cache, using remote download with search_id={effective_search_id}, index={result_index}")
|
||||
return self._download_remote(result_index, output_dir, output_filename, gcs_path, effective_search_id)
|
||||
else:
|
||||
# Provide clear error message based on whether cache is empty or index out of bounds
|
||||
if len(self._cached_results) == 0:
|
||||
raise DownloadError(
|
||||
f"No cached result for index {result_index}. "
|
||||
"No cached results available. Run search() first."
|
||||
)
|
||||
raise DownloadError(
|
||||
f"No cached result for index {result_index}. "
|
||||
f"Available indices: 0-{len(self._cached_results) - 1}. "
|
||||
"Run search() first."
|
||||
)
|
||||
|
||||
result = self._cached_results[result_index]
|
||||
|
||||
logger.info(f"Downloading: {result.artist} - {result.title} from {result.provider}")
|
||||
|
||||
# Use provided search_id or fall back to service-level cache
|
||||
effective_search_id = remote_search_id or self._remote_search_id
|
||||
|
||||
# Check if this was a remote search (torrent sources need remote download)
|
||||
if effective_search_id and self._remote_client:
|
||||
# Check if this is a torrent source that needs remote handling
|
||||
@@ -481,8 +496,12 @@ class AudioSearchService:
|
||||
if not effective_search_id:
|
||||
raise DownloadError("No remote search ID - run search() first")
|
||||
|
||||
result = self._cached_results[result_index]
|
||||
|
||||
# Try to get local result for metadata, but it's optional
|
||||
# The remote service maintains its own cache by search_id
|
||||
result = None
|
||||
if result_index >= 0 and result_index < len(self._cached_results):
|
||||
result = self._cached_results[result_index]
|
||||
|
||||
try:
|
||||
# Enable nested event loops (needed when called from FastAPI async context)
|
||||
try:
|
||||
@@ -529,13 +548,15 @@ class AudioSearchService:
|
||||
raise DownloadError("Remote download completed but no file path returned")
|
||||
|
||||
logger.info(f"Remote download complete: {filepath}")
|
||||
|
||||
|
||||
# Use local result metadata if available, otherwise use empty strings
|
||||
# (the actual download is handled by the remote service using search_id)
|
||||
return AudioDownloadResult(
|
||||
filepath=filepath,
|
||||
artist=result.artist,
|
||||
title=result.title,
|
||||
provider=result.provider,
|
||||
quality=result.quality,
|
||||
artist=result.artist if result else "",
|
||||
title=result.title if result else "",
|
||||
provider=result.provider if result else "remote",
|
||||
quality=result.quality if result else "",
|
||||
)
|
||||
|
||||
except FlacfetchServiceError as e:
|
||||
|
||||
@@ -3,50 +3,81 @@ Authentication service for karaoke backend.
|
||||
|
||||
Provides token-based authentication with support for:
|
||||
- Admin tokens (hardcoded in environment)
|
||||
- User tokens (stored in Firestore)
|
||||
- Admin by email domain (@nomadkaraoke.com)
|
||||
- User tokens (stored in Firestore auth_tokens collection)
|
||||
- Session tokens (stored in Firestore sessions collection, from magic link auth)
|
||||
- API keys for business users
|
||||
- Usage tracking and limits
|
||||
- Token management API
|
||||
|
||||
This is designed to be extensible for future Stripe integration
|
||||
and admin dashboard functionality.
|
||||
"""
|
||||
import logging
|
||||
import hashlib
|
||||
import time
|
||||
import secrets
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
from typing import Optional, Tuple, Dict, Any
|
||||
from typing import Optional, Tuple, Dict, Any, List
|
||||
from datetime import datetime
|
||||
|
||||
from backend.services.firestore_service import FirestoreService
|
||||
from backend.config import get_settings
|
||||
from backend.models.user import UserRole
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Admin email domains - users with these domains are automatically admin
|
||||
ADMIN_EMAIL_DOMAINS = ["nomadkaraoke.com"]
|
||||
|
||||
|
||||
class UserType(str, Enum):
|
||||
"""User access levels."""
|
||||
ADMIN = "admin" # Unlimited access, can manage tokens
|
||||
UNLIMITED = "unlimited" # Unlimited karaoke generation
|
||||
LIMITED = "limited" # Limited number of uses
|
||||
STRIPE = "stripe" # Paid access via Stripe (future)
|
||||
STRIPE = "stripe" # Paid access via Stripe
|
||||
API_KEY = "api_key" # Business API key access
|
||||
|
||||
|
||||
@dataclass
|
||||
class AuthResult:
|
||||
"""Result of authentication validation."""
|
||||
is_valid: bool
|
||||
user_type: UserType
|
||||
remaining_uses: int # -1 = unlimited, 0 = exhausted, >0 = remaining
|
||||
message: str
|
||||
user_email: Optional[str] = None # Email if authenticated via session/API key
|
||||
is_admin: bool = False # True if admin token or admin email domain
|
||||
api_key_id: Optional[str] = None # API key ID if authenticated via API key
|
||||
|
||||
def __iter__(self):
|
||||
"""Allow unpacking as tuple for backward compatibility."""
|
||||
return iter((self.is_valid, self.user_type, self.remaining_uses, self.message))
|
||||
|
||||
|
||||
def is_admin_email(email: str) -> bool:
|
||||
"""Check if an email belongs to an admin domain."""
|
||||
if not email:
|
||||
return False
|
||||
email_lower = email.lower()
|
||||
return any(email_lower.endswith(f"@{domain}") for domain in ADMIN_EMAIL_DOMAINS)
|
||||
|
||||
|
||||
class AuthService:
|
||||
"""Service for authentication and token management."""
|
||||
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize auth service."""
|
||||
self.firestore = FirestoreService()
|
||||
self.settings = get_settings()
|
||||
self._init_admin_tokens()
|
||||
|
||||
|
||||
def _init_admin_tokens(self):
|
||||
"""Load admin tokens from environment variables."""
|
||||
# Get admin tokens from secret manager or env vars
|
||||
admin_tokens_str = self.settings.admin_tokens or ""
|
||||
self.admin_tokens = [t.strip() for t in admin_tokens_str.split(",") if t.strip()]
|
||||
|
||||
|
||||
if self.admin_tokens:
|
||||
logger.info(f"Loaded {len(self.admin_tokens)} admin token(s)")
|
||||
else:
|
||||
@@ -78,10 +109,30 @@ class AuthService:
|
||||
f"expected len={len(self.admin_tokens[0])}, got len={len(token)}"
|
||||
)
|
||||
|
||||
# Check stored tokens in Firestore
|
||||
# Check stored tokens in Firestore (auth_tokens collection)
|
||||
token_data = self.firestore.get_token(token)
|
||||
|
||||
|
||||
if not token_data:
|
||||
# Fall back to checking session tokens (from magic link auth)
|
||||
# Import here to avoid circular dependency
|
||||
from backend.services.user_service import get_user_service
|
||||
|
||||
user_service = get_user_service()
|
||||
is_valid, user, message = user_service.validate_session(token)
|
||||
|
||||
if is_valid and user:
|
||||
# Session is valid - user is authenticated via magic link
|
||||
# Return their credits as remaining uses
|
||||
credits = user.credits
|
||||
logger.info(f"Session token validated for user {user.email} (credits: {credits})")
|
||||
|
||||
if credits <= 0:
|
||||
# User has no credits but is authenticated
|
||||
return True, UserType.STRIPE, 0, f"Authenticated but no credits remaining"
|
||||
|
||||
return True, UserType.STRIPE, credits, f"Session valid: {credits} credits"
|
||||
|
||||
# Neither auth_token nor session found
|
||||
return False, UserType.LIMITED, 0, "Invalid token"
|
||||
|
||||
# Check if token is active
|
||||
@@ -129,35 +180,319 @@ class AuthService:
|
||||
return True, token_type, -1, "Stripe access granted"
|
||||
|
||||
return False, UserType.LIMITED, 0, "Unknown token type"
|
||||
|
||||
|
||||
def validate_token_full(self, token: str) -> AuthResult:
|
||||
"""
|
||||
Validate an access token and return full authentication details.
|
||||
|
||||
This is the preferred method for new code as it returns complete
|
||||
authentication context including user email and admin status.
|
||||
|
||||
Returns:
|
||||
AuthResult with full authentication details
|
||||
"""
|
||||
if not token:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType.LIMITED,
|
||||
remaining_uses=0,
|
||||
message="No token provided"
|
||||
)
|
||||
|
||||
# Check for admin tokens (highest priority)
|
||||
# Admin tokens from env var are associated with the system admin email
|
||||
if token in self.admin_tokens:
|
||||
logger.info("Admin token validated")
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.ADMIN,
|
||||
remaining_uses=-1,
|
||||
message="Admin access granted",
|
||||
user_email="admin@nomadkaraoke.com",
|
||||
is_admin=True
|
||||
)
|
||||
|
||||
# Check stored tokens in Firestore (auth_tokens collection)
|
||||
token_data = self.firestore.get_token(token)
|
||||
|
||||
if not token_data:
|
||||
# Fall back to checking session tokens (from magic link auth)
|
||||
from backend.services.user_service import get_user_service
|
||||
|
||||
user_service = get_user_service()
|
||||
is_valid, user, _message = user_service.validate_session(token)
|
||||
|
||||
if is_valid and user:
|
||||
# Session is valid - user is authenticated via magic link
|
||||
credits = user.credits
|
||||
user_email = user.email
|
||||
|
||||
# Check if user is admin by email domain or role
|
||||
user_is_admin = is_admin_email(user_email) or user.role == UserRole.ADMIN
|
||||
|
||||
if user_is_admin:
|
||||
logger.info(f"Admin session validated for {user_email}")
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.ADMIN,
|
||||
remaining_uses=-1,
|
||||
message="Admin access granted (domain)",
|
||||
user_email=user_email,
|
||||
is_admin=True
|
||||
)
|
||||
|
||||
logger.info(f"Session token validated for user {user_email} (credits: {credits})")
|
||||
|
||||
if credits <= 0:
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.STRIPE,
|
||||
remaining_uses=0,
|
||||
message="Authenticated but no credits remaining",
|
||||
user_email=user_email,
|
||||
is_admin=False
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.STRIPE,
|
||||
remaining_uses=credits,
|
||||
message=f"Session valid: {credits} credits",
|
||||
user_email=user_email,
|
||||
is_admin=False
|
||||
)
|
||||
|
||||
# Neither auth_token nor session found
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType.LIMITED,
|
||||
remaining_uses=0,
|
||||
message="Invalid token"
|
||||
)
|
||||
|
||||
# Check if token is active
|
||||
if not token_data.get("active", True):
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType(token_data["type"]),
|
||||
remaining_uses=0,
|
||||
message="Token has been revoked"
|
||||
)
|
||||
|
||||
token_type = UserType(token_data["type"])
|
||||
max_uses = token_data.get("max_uses", -1)
|
||||
# All auth_tokens must have an associated user_email for job ownership
|
||||
token_user_email = token_data.get("user_email")
|
||||
api_key_id = token_data.get("api_key_id")
|
||||
|
||||
# Require user_email on all auth_tokens (no anonymous token auth)
|
||||
if not token_user_email:
|
||||
logger.warning("Auth token missing required user_email field")
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token configuration error: missing user_email. Please contact support."
|
||||
)
|
||||
|
||||
# Check if token's user is an admin (by email domain)
|
||||
token_is_admin = is_admin_email(token_user_email)
|
||||
|
||||
# UNLIMITED tokens: no usage limits
|
||||
if token_type == UserType.UNLIMITED:
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="Unlimited access granted",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
# LIMITED tokens: check usage count
|
||||
if token_type == UserType.LIMITED:
|
||||
if max_uses <= 0: # -1 means unlimited
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="Limited token with unlimited uses",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
current_uses = token_data.get("usage_count", 0)
|
||||
remaining = max_uses - current_uses
|
||||
|
||||
if remaining <= 0:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token usage limit exceeded",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=remaining,
|
||||
message=f"Limited token: {remaining} uses remaining",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
# STRIPE tokens: check expiration and usage
|
||||
if token_type == UserType.STRIPE:
|
||||
expires_at = token_data.get("expires_at")
|
||||
|
||||
if expires_at and time.time() > expires_at:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token has expired",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
if max_uses > 0:
|
||||
current_uses = token_data.get("usage_count", 0)
|
||||
remaining = max_uses - current_uses
|
||||
|
||||
if remaining <= 0:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="Token usage limit exceeded",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=remaining,
|
||||
message=f"Stripe token: {remaining} uses remaining",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="Stripe access granted",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
# API_KEY tokens
|
||||
if token_type == UserType.API_KEY:
|
||||
if max_uses > 0:
|
||||
current_uses = token_data.get("usage_count", 0)
|
||||
remaining = max_uses - current_uses
|
||||
if remaining <= 0:
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=token_type,
|
||||
remaining_uses=0,
|
||||
message="API key usage limit exceeded",
|
||||
user_email=token_user_email,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=remaining,
|
||||
message=f"API key valid: {remaining} uses remaining",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=token_type,
|
||||
remaining_uses=-1,
|
||||
message="API key access granted",
|
||||
user_email=token_user_email,
|
||||
is_admin=token_is_admin,
|
||||
api_key_id=api_key_id
|
||||
)
|
||||
|
||||
return AuthResult(
|
||||
is_valid=False,
|
||||
user_type=UserType.LIMITED,
|
||||
remaining_uses=0,
|
||||
message="Unknown token type"
|
||||
)
|
||||
|
||||
def increment_token_usage(self, token: str, job_id: str) -> bool:
|
||||
"""
|
||||
Increment usage count for a token and track the job.
|
||||
|
||||
|
||||
For session tokens (magic link auth), this deducts a credit from the user.
|
||||
For auth_tokens, this increments the usage count.
|
||||
|
||||
Args:
|
||||
token: The access token
|
||||
job_id: The job ID being created
|
||||
|
||||
|
||||
Returns:
|
||||
True if usage was tracked, False otherwise
|
||||
"""
|
||||
# Validate token first
|
||||
is_valid, user_type, remaining_uses, message = self.validate_token(token)
|
||||
|
||||
|
||||
if not is_valid:
|
||||
logger.warning(f"Cannot increment usage for invalid token: {message}")
|
||||
return False
|
||||
|
||||
|
||||
# Don't track usage for admin or unlimited tokens
|
||||
if user_type in [UserType.ADMIN, UserType.UNLIMITED]:
|
||||
logger.debug(f"Skipping usage tracking for {user_type} token")
|
||||
return True
|
||||
|
||||
|
||||
# For admin tokens (not in Firestore), no tracking needed
|
||||
if token in self.admin_tokens:
|
||||
return True
|
||||
|
||||
# Increment usage in Firestore
|
||||
|
||||
# Check if this is a session token (STRIPE type from validate_token means it's a session)
|
||||
# Session tokens are not in auth_tokens collection
|
||||
token_data = self.firestore.get_token(token)
|
||||
|
||||
if not token_data:
|
||||
# This is a session token - deduct credit from user
|
||||
from backend.services.user_service import get_user_service
|
||||
user_service = get_user_service()
|
||||
|
||||
# Get the user email from the session
|
||||
is_valid, user, _ = user_service.validate_session(token)
|
||||
if not is_valid or not user:
|
||||
logger.error(f"Session token validation failed during usage increment")
|
||||
return False
|
||||
|
||||
# Deduct one credit
|
||||
success, new_balance, deduct_message = user_service.deduct_credit(
|
||||
user.email, job_id, reason="job_creation"
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.info(f"Deducted credit for user {user.email} (remaining: {new_balance})")
|
||||
return True
|
||||
else:
|
||||
logger.error(f"Failed to deduct credit for user {user.email}: {deduct_message}")
|
||||
return False
|
||||
|
||||
# Regular auth_token - increment usage in Firestore
|
||||
try:
|
||||
self.firestore.increment_token_usage(token, job_id)
|
||||
logger.info(f"Incremented usage for token (remaining: {remaining_uses - 1})")
|
||||
|
||||
@@ -199,7 +199,7 @@ class DropboxService:
|
||||
|
||||
def upload_folder(self, local_dir: str, remote_path: str) -> None:
|
||||
"""
|
||||
Upload all files in a directory to Dropbox folder.
|
||||
Recursively upload all files and subdirectories to Dropbox folder.
|
||||
|
||||
Args:
|
||||
local_dir: Local directory to upload
|
||||
@@ -212,10 +212,17 @@ class DropboxService:
|
||||
logger.info(f"Uploading folder {local_dir} to {remote_path}")
|
||||
|
||||
uploaded_count = 0
|
||||
for filename in os.listdir(local_dir):
|
||||
local_file = os.path.join(local_dir, filename)
|
||||
if os.path.isfile(local_file):
|
||||
remote_file = f"{remote_path}/{filename}"
|
||||
for root, _dirs, files in os.walk(local_dir):
|
||||
# Calculate the relative path from local_dir to current root
|
||||
rel_root = os.path.relpath(root, local_dir)
|
||||
if rel_root == ".":
|
||||
current_remote = remote_path
|
||||
else:
|
||||
current_remote = f"{remote_path}/{rel_root}"
|
||||
|
||||
for filename in files:
|
||||
local_file = os.path.join(root, filename)
|
||||
remote_file = f"{current_remote}/{filename}"
|
||||
self.upload_file(local_file, remote_file)
|
||||
uploaded_count += 1
|
||||
|
||||
@@ -256,6 +263,38 @@ class DropboxService:
|
||||
return links.links[0].url
|
||||
raise
|
||||
|
||||
def delete_folder(self, path: str) -> bool:
|
||||
"""
|
||||
Delete a folder and all its contents from Dropbox.
|
||||
|
||||
Args:
|
||||
path: Dropbox path to delete (e.g., "/Karaoke/Tracks-Organized/NOMAD-1234 - Artist - Title")
|
||||
|
||||
Returns:
|
||||
True if deleted successfully, False otherwise
|
||||
"""
|
||||
from dropbox.exceptions import ApiError
|
||||
|
||||
# Ensure path starts with /
|
||||
if not path.startswith("/"):
|
||||
path = f"/{path}"
|
||||
|
||||
logger.info(f"Deleting Dropbox folder: {path}")
|
||||
|
||||
try:
|
||||
self.client.files_delete_v2(path)
|
||||
logger.info(f"Successfully deleted: {path}")
|
||||
return True
|
||||
except ApiError as e:
|
||||
if e.error.is_path_lookup() and e.error.get_path_lookup().is_not_found():
|
||||
logger.warning(f"Folder not found (already deleted?): {path}")
|
||||
return True # Consider it success if already gone
|
||||
logger.error(f"Failed to delete Dropbox folder: {e}")
|
||||
return False
|
||||
except Exception as e:
|
||||
logger.error(f"Error deleting Dropbox folder: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def get_dropbox_service() -> DropboxService:
|
||||
"""Factory function to get a DropboxService instance."""
|
||||
|
||||
@@ -0,0 +1,829 @@
|
||||
"""
|
||||
Email service for sending transactional emails.
|
||||
|
||||
Supports multiple providers:
|
||||
- SendGrid (recommended for production)
|
||||
- Console logging (for development/testing)
|
||||
|
||||
Future providers can be added:
|
||||
- Mailgun
|
||||
- AWS SES
|
||||
- Postmark
|
||||
"""
|
||||
import html
|
||||
import logging
|
||||
import os
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Optional
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class EmailProvider(ABC):
|
||||
"""Abstract base class for email providers."""
|
||||
|
||||
@abstractmethod
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
"""Send an email. Returns True if successful."""
|
||||
pass
|
||||
|
||||
|
||||
class ConsoleEmailProvider(EmailProvider):
|
||||
"""
|
||||
Development email provider that logs to console.
|
||||
|
||||
Useful for local development and testing.
|
||||
"""
|
||||
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
logger.info("=" * 60)
|
||||
logger.info(f"EMAIL TO: {to_email}")
|
||||
logger.info(f"SUBJECT: {subject}")
|
||||
logger.info("-" * 60)
|
||||
logger.info(text_content or html_content)
|
||||
logger.info("=" * 60)
|
||||
return True
|
||||
|
||||
|
||||
class SendGridEmailProvider(EmailProvider):
|
||||
"""
|
||||
SendGrid email provider for production.
|
||||
|
||||
Requires SENDGRID_API_KEY environment variable.
|
||||
"""
|
||||
|
||||
def __init__(self, api_key: str, from_email: str, from_name: str = "Nomad Karaoke"):
|
||||
self.api_key = api_key
|
||||
self.from_email = from_email
|
||||
self.from_name = from_name
|
||||
|
||||
def send_email(
|
||||
self,
|
||||
to_email: str,
|
||||
subject: str,
|
||||
html_content: str,
|
||||
text_content: Optional[str] = None
|
||||
) -> bool:
|
||||
try:
|
||||
# Import here to avoid requiring sendgrid in all environments
|
||||
from sendgrid import SendGridAPIClient
|
||||
from sendgrid.helpers.mail import Mail, Email, To, Content
|
||||
|
||||
sg = SendGridAPIClient(api_key=self.api_key)
|
||||
|
||||
message = Mail(
|
||||
from_email=Email(self.from_email, self.from_name),
|
||||
to_emails=To(to_email),
|
||||
subject=subject,
|
||||
html_content=Content("text/html", html_content)
|
||||
)
|
||||
|
||||
if text_content:
|
||||
message.add_content(Content("text/plain", text_content))
|
||||
|
||||
response = sg.send(message)
|
||||
|
||||
if response.status_code >= 200 and response.status_code < 300:
|
||||
logger.info(f"Email sent to {to_email} via SendGrid")
|
||||
return True
|
||||
else:
|
||||
logger.error(f"SendGrid returned status {response.status_code}")
|
||||
return False
|
||||
|
||||
except Exception:
|
||||
logger.exception("Failed to send email via SendGrid")
|
||||
return False
|
||||
|
||||
|
||||
class EmailService:
|
||||
"""
|
||||
High-level email service for sending transactional emails.
|
||||
|
||||
Automatically selects the appropriate provider based on configuration.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.settings = get_settings()
|
||||
self.provider = self._get_provider()
|
||||
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
|
||||
# After consolidation, buy URL is the same as frontend URL
|
||||
self.buy_url = os.getenv("BUY_URL", self.frontend_url)
|
||||
|
||||
def _get_provider(self) -> EmailProvider:
|
||||
"""Get the configured email provider."""
|
||||
sendgrid_api_key = os.getenv("SENDGRID_API_KEY")
|
||||
from_email = os.getenv("EMAIL_FROM", "gen@nomadkaraoke.com")
|
||||
from_name = os.getenv("EMAIL_FROM_NAME", "Nomad Karaoke")
|
||||
|
||||
if sendgrid_api_key:
|
||||
logger.info("Using SendGrid email provider")
|
||||
return SendGridEmailProvider(sendgrid_api_key, from_email, from_name)
|
||||
else:
|
||||
logger.warning("No email provider configured, using console logging")
|
||||
return ConsoleEmailProvider()
|
||||
|
||||
def is_configured(self) -> bool:
|
||||
"""Check if a real email provider is configured (not just console logging)."""
|
||||
return isinstance(self.provider, SendGridEmailProvider)
|
||||
|
||||
def send_magic_link(self, email: str, token: str) -> bool:
|
||||
"""
|
||||
Send a magic link email for authentication.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
token: Magic link token
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
magic_link_url = f"{self.frontend_url}/auth/verify?token={token}"
|
||||
|
||||
subject = "Sign in to Nomad Karaoke"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.button:hover {{
|
||||
background-color: #2563eb;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
.warning {{
|
||||
background-color: #fef3c7;
|
||||
border: 1px solid #fcd34d;
|
||||
border-radius: 4px;
|
||||
padding: 12px;
|
||||
margin: 20px 0;
|
||||
font-size: 14px;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Hi there,</p>
|
||||
|
||||
<p>Click the button below to sign in to Nomad Karaoke:</p>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{magic_link_url}" class="button">Sign In</a>
|
||||
</p>
|
||||
|
||||
<div class="warning">
|
||||
⏰ This link expires in 15 minutes and can only be used once.
|
||||
</div>
|
||||
|
||||
<p>If the button doesn't work, copy and paste this link into your browser:</p>
|
||||
<p style="word-break: break-all; font-size: 14px; color: #666;">
|
||||
{magic_link_url}
|
||||
</p>
|
||||
|
||||
<p>If you didn't request this email, you can safely ignore it.</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
<p>This is an automated message, please do not reply.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Sign in to Nomad Karaoke
|
||||
========================
|
||||
|
||||
Click this link to sign in:
|
||||
{magic_link_url}
|
||||
|
||||
This link expires in 15 minutes and can only be used once.
|
||||
|
||||
If you didn't request this email, you can safely ignore it.
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_credits_added(self, email: str, credits: int, total_credits: int) -> bool:
|
||||
"""
|
||||
Send notification when credits are added to account.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Number of credits added
|
||||
total_credits: New total credit balance
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = f"🎉 {credits} credits added to your Nomad Karaoke account"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.credits-box {{
|
||||
background-color: #ecfdf5;
|
||||
border: 2px solid #10b981;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.credits-number {{
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
color: #10b981;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Great news! 🎉</p>
|
||||
|
||||
<p><strong>{credits} credits</strong> have been added to your account.</p>
|
||||
|
||||
<div class="credits-box">
|
||||
<div>Your balance:</div>
|
||||
<div class="credits-number">{total_credits}</div>
|
||||
<div>credits</div>
|
||||
</div>
|
||||
|
||||
<p>Each credit lets you create one professional karaoke video with:</p>
|
||||
<ul>
|
||||
<li>AI-powered vocal/instrumental separation</li>
|
||||
<li>Synchronized lyrics with word-level timing</li>
|
||||
<li>4K video output</li>
|
||||
<li>YouTube upload</li>
|
||||
</ul>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Create Karaoke Now</a>
|
||||
</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
{credits} credits added to your Nomad Karaoke account!
|
||||
|
||||
Your new balance: {total_credits} credits
|
||||
|
||||
Each credit lets you create one professional karaoke video.
|
||||
|
||||
Start creating: {self.frontend_url}
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_welcome_email(self, email: str, credits: int = 0) -> bool:
|
||||
"""
|
||||
Send welcome email to new users.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Initial credit balance (if any)
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Welcome to Nomad Karaoke! 🎤"
|
||||
|
||||
credits_text = f"You have <strong>{credits} credits</strong> to get started!" if credits > 0 else ""
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.feature {{
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
margin: 16px 0;
|
||||
}}
|
||||
.feature-icon {{
|
||||
font-size: 24px;
|
||||
margin-right: 12px;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Welcome to Nomad Karaoke!</p>
|
||||
|
||||
<p>Turn any song into a professional karaoke video in minutes. {credits_text}</p>
|
||||
|
||||
<p><strong>Here's how it works:</strong></p>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">🎵</span>
|
||||
<div>
|
||||
<strong>1. Search for a song</strong><br>
|
||||
Enter the artist and title, and we'll find high-quality audio.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">✨</span>
|
||||
<div>
|
||||
<strong>2. AI does the magic</strong><br>
|
||||
We separate vocals, transcribe lyrics, and sync everything perfectly.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">✏️</span>
|
||||
<div>
|
||||
<strong>3. Review & customize</strong><br>
|
||||
Fine-tune the lyrics if needed, choose your instrumental.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="feature">
|
||||
<span class="feature-icon">🎬</span>
|
||||
<div>
|
||||
<strong>4. Get your video</strong><br>
|
||||
Download your 4K karaoke video or upload directly to YouTube.
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Get Started</a>
|
||||
</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>Questions? Reply to this email and we'll help you out.</p>
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Welcome to Nomad Karaoke!
|
||||
|
||||
Turn any song into a professional karaoke video in minutes.
|
||||
|
||||
Here's how it works:
|
||||
|
||||
1. Search for a song
|
||||
Enter the artist and title, and we'll find high-quality audio.
|
||||
|
||||
2. AI does the magic
|
||||
We separate vocals, transcribe lyrics, and sync everything perfectly.
|
||||
|
||||
3. Review & customize
|
||||
Fine-tune the lyrics if needed, choose your instrumental.
|
||||
|
||||
4. Get your video
|
||||
Download your 4K karaoke video or upload directly to YouTube.
|
||||
|
||||
Get started: {self.frontend_url}
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_beta_welcome_email(self, email: str, credits: int = 1) -> bool:
|
||||
"""
|
||||
Send welcome email to new beta testers.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
credits: Initial free credits granted
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Welcome Beta Tester! Free Karaoke Credits Inside 🎤"
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.beta-badge {{
|
||||
display: inline-block;
|
||||
background: linear-gradient(135deg, #8b5cf6, #ec4899);
|
||||
color: white;
|
||||
padding: 4px 12px;
|
||||
border-radius: 20px;
|
||||
font-size: 12px;
|
||||
font-weight: bold;
|
||||
margin-left: 8px;
|
||||
}}
|
||||
.credits-box {{
|
||||
background-color: #ecfdf5;
|
||||
border: 2px solid #10b981;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.credits-number {{
|
||||
font-size: 48px;
|
||||
font-weight: bold;
|
||||
color: #10b981;
|
||||
}}
|
||||
.reminder {{
|
||||
background-color: #fef3c7;
|
||||
border: 1px solid #fcd34d;
|
||||
border-radius: 8px;
|
||||
padding: 16px;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<span class="logo">🎤 Nomad Karaoke</span>
|
||||
<span class="beta-badge">BETA TESTER</span>
|
||||
</div>
|
||||
|
||||
<p>Thank you for joining our beta program! 🎉</p>
|
||||
|
||||
<p>As promised, here's your free credit to create a karaoke video:</p>
|
||||
|
||||
<div class="credits-box">
|
||||
<div>Your balance:</div>
|
||||
<div class="credits-number">{credits}</div>
|
||||
<div>free credit{'' if credits == 1 else 's'}</div>
|
||||
</div>
|
||||
|
||||
<div class="reminder">
|
||||
<strong>📝 Your Promise:</strong><br>
|
||||
Remember, as a beta tester you agreed to:
|
||||
<ul style="margin: 10px 0; padding-left: 20px;">
|
||||
<li>Review and correct any lyrics timing issues</li>
|
||||
<li>Share your honest feedback after trying the tool</li>
|
||||
</ul>
|
||||
We'll send you a quick feedback form after your job completes!
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{self.frontend_url}" class="button">Create Your Karaoke Video</a>
|
||||
</p>
|
||||
|
||||
<p>Thanks for helping us make Nomad Karaoke better!</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Welcome Beta Tester!
|
||||
|
||||
Thank you for joining our beta program!
|
||||
|
||||
You've been granted {credits} free credit{'' if credits == 1 else 's'} to create karaoke videos.
|
||||
|
||||
YOUR PROMISE:
|
||||
As a beta tester, you agreed to:
|
||||
- Review and correct any lyrics timing issues
|
||||
- Share your honest feedback after trying the tool
|
||||
|
||||
We'll send you a quick feedback form after your job completes!
|
||||
|
||||
Create your karaoke video: {self.frontend_url}
|
||||
|
||||
Thanks for helping us make Nomad Karaoke better!
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def send_feedback_request_email(self, email: str, feedback_url: str, job_title: Optional[str] = None) -> bool:
|
||||
"""
|
||||
Send feedback request email to beta testers.
|
||||
|
||||
Args:
|
||||
email: User's email address
|
||||
feedback_url: URL to the feedback form
|
||||
job_title: Optional title of the completed job
|
||||
|
||||
Returns:
|
||||
True if email was sent successfully
|
||||
"""
|
||||
subject = "Quick feedback on your karaoke experience? 🎤"
|
||||
|
||||
# Escape job_title to prevent XSS in email clients that render HTML
|
||||
safe_job_title = html.escape(job_title) if job_title else ""
|
||||
job_context = f" for <strong>{safe_job_title}</strong>" if job_title else ""
|
||||
|
||||
html_content = f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
body {{
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
||||
line-height: 1.6;
|
||||
color: #333;
|
||||
max-width: 600px;
|
||||
margin: 0 auto;
|
||||
padding: 20px;
|
||||
}}
|
||||
.header {{
|
||||
text-align: center;
|
||||
padding: 20px 0;
|
||||
}}
|
||||
.logo {{
|
||||
font-size: 24px;
|
||||
font-weight: bold;
|
||||
color: #3b82f6;
|
||||
}}
|
||||
.feedback-box {{
|
||||
background: linear-gradient(135deg, #f0f9ff, #e0f2fe);
|
||||
border: 2px solid #3b82f6;
|
||||
border-radius: 12px;
|
||||
padding: 24px;
|
||||
text-align: center;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.stars {{
|
||||
font-size: 36px;
|
||||
margin: 10px 0;
|
||||
}}
|
||||
.button {{
|
||||
display: inline-block;
|
||||
background-color: #3b82f6;
|
||||
color: white;
|
||||
padding: 14px 28px;
|
||||
text-decoration: none;
|
||||
border-radius: 8px;
|
||||
font-weight: 600;
|
||||
margin: 20px 0;
|
||||
}}
|
||||
.time-note {{
|
||||
background-color: #fef3c7;
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
margin: 20px 0;
|
||||
font-size: 14px;
|
||||
}}
|
||||
.footer {{
|
||||
margin-top: 30px;
|
||||
padding-top: 20px;
|
||||
border-top: 1px solid #eee;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="header">
|
||||
<div class="logo">🎤 Nomad Karaoke</div>
|
||||
</div>
|
||||
|
||||
<p>Hi there!</p>
|
||||
|
||||
<p>Hope you enjoyed creating your karaoke video{job_context}! As a beta tester, your feedback is super valuable to us.</p>
|
||||
|
||||
<div class="feedback-box">
|
||||
<div class="stars">⭐⭐⭐⭐⭐</div>
|
||||
<p><strong>How was your experience?</strong></p>
|
||||
<p>Quick 2-minute survey - we'd love to hear your thoughts!</p>
|
||||
</div>
|
||||
|
||||
<p style="text-align: center;">
|
||||
<a href="{feedback_url}" class="button">Share Your Feedback</a>
|
||||
</p>
|
||||
|
||||
<div class="time-note">
|
||||
⏱️ This takes less than 2 minutes and helps us improve the tool for everyone!
|
||||
</div>
|
||||
|
||||
<p>Specifically, we'd love to know:</p>
|
||||
<ul>
|
||||
<li>How easy was it to use?</li>
|
||||
<li>Were the lyrics accurate?</li>
|
||||
<li>How was the correction experience?</li>
|
||||
<li>What could we improve?</li>
|
||||
</ul>
|
||||
|
||||
<p>Thanks for being part of making Nomad Karaoke better! 🙏</p>
|
||||
|
||||
<div class="footer">
|
||||
<p>© {self._get_year()} Nomad Karaoke. All rights reserved.</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
text_content = f"""
|
||||
Quick feedback on your karaoke experience?
|
||||
|
||||
Hi there!
|
||||
|
||||
Hope you enjoyed creating your karaoke video{' for ' + safe_job_title if safe_job_title else ''}!
|
||||
|
||||
As a beta tester, your feedback is super valuable to us.
|
||||
|
||||
Share your feedback (2-minute survey): {feedback_url}
|
||||
|
||||
We'd love to know:
|
||||
- How easy was it to use?
|
||||
- Were the lyrics accurate?
|
||||
- How was the correction experience?
|
||||
- What could we improve?
|
||||
|
||||
Thanks for being part of making Nomad Karaoke better!
|
||||
|
||||
---
|
||||
© {self._get_year()} Nomad Karaoke
|
||||
"""
|
||||
|
||||
return self.provider.send_email(email, subject, html_content, text_content)
|
||||
|
||||
def _get_year(self) -> int:
|
||||
"""Get current year for copyright notices."""
|
||||
from datetime import datetime
|
||||
return datetime.now().year
|
||||
|
||||
|
||||
# Global instance
|
||||
_email_service = None
|
||||
|
||||
|
||||
def get_email_service() -> EmailService:
|
||||
"""Get the global email service instance."""
|
||||
global _email_service
|
||||
if _email_service is None:
|
||||
_email_service = EmailService()
|
||||
return _email_service
|
||||
@@ -0,0 +1,270 @@
|
||||
"""
|
||||
GCE Encoding Worker Service.
|
||||
|
||||
This service dispatches video encoding jobs to a dedicated high-performance
|
||||
GCE instance (C4-standard with Intel Granite Rapids CPU) for faster encoding.
|
||||
|
||||
The GCE worker provides:
|
||||
- 3.9 GHz all-core frequency (vs 3.7 GHz on Cloud Run)
|
||||
- Dedicated vCPUs (no contention)
|
||||
- 2-3x faster FFmpeg libx264 encoding
|
||||
|
||||
Usage:
|
||||
encoding_service = get_encoding_service()
|
||||
if encoding_service.is_configured:
|
||||
result = await encoding_service.encode_videos(job_id, input_gcs_path, config)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Optional, Dict, Any
|
||||
|
||||
import aiohttp
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class EncodingService:
|
||||
"""Service for dispatching encoding jobs to GCE worker."""
|
||||
|
||||
def __init__(self):
|
||||
self.settings = get_settings()
|
||||
self._url = None
|
||||
self._api_key = None
|
||||
self._initialized = False
|
||||
|
||||
def _load_credentials(self):
|
||||
"""Load encoding worker URL and API key from config/secrets."""
|
||||
if self._initialized:
|
||||
return
|
||||
|
||||
# Try environment variables first, then Secret Manager
|
||||
self._url = self.settings.encoding_worker_url
|
||||
self._api_key = self.settings.encoding_worker_api_key
|
||||
|
||||
# Fall back to Secret Manager
|
||||
if not self._url:
|
||||
self._url = self.settings.get_secret("encoding-worker-url")
|
||||
if not self._api_key:
|
||||
self._api_key = self.settings.get_secret("encoding-worker-api-key")
|
||||
|
||||
self._initialized = True
|
||||
|
||||
@property
|
||||
def is_configured(self) -> bool:
|
||||
"""Check if encoding service is configured with URL and API key."""
|
||||
self._load_credentials()
|
||||
return bool(self._url and self._api_key)
|
||||
|
||||
@property
|
||||
def is_enabled(self) -> bool:
|
||||
"""Check if GCE encoding is enabled and configured."""
|
||||
return self.settings.use_gce_encoding and self.is_configured
|
||||
|
||||
async def submit_encoding_job(
|
||||
self,
|
||||
job_id: str,
|
||||
input_gcs_path: str,
|
||||
output_gcs_path: str,
|
||||
encoding_config: Dict[str, Any],
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Submit an encoding job to the GCE worker.
|
||||
|
||||
Args:
|
||||
job_id: Unique job identifier
|
||||
input_gcs_path: GCS path to input files (gs://bucket/path/)
|
||||
output_gcs_path: GCS path for output files (gs://bucket/path/)
|
||||
encoding_config: Configuration for encoding (formats, quality, etc.)
|
||||
|
||||
Returns:
|
||||
Response from the encoding worker
|
||||
|
||||
Raises:
|
||||
Exception: If submission fails
|
||||
"""
|
||||
self._load_credentials()
|
||||
|
||||
if not self.is_configured:
|
||||
raise RuntimeError("Encoding service not configured")
|
||||
|
||||
url = f"{self._url}/encode"
|
||||
headers = {"X-API-Key": self._api_key, "Content-Type": "application/json"}
|
||||
payload = {
|
||||
"job_id": job_id,
|
||||
"input_gcs_path": input_gcs_path,
|
||||
"output_gcs_path": output_gcs_path,
|
||||
"encoding_config": encoding_config,
|
||||
}
|
||||
|
||||
logger.info(f"[job:{job_id}] Submitting encoding job to GCE worker: {url}")
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.post(url, json=payload, headers=headers, timeout=30) as resp:
|
||||
if resp.status == 401:
|
||||
raise RuntimeError("Invalid API key for encoding worker")
|
||||
if resp.status == 409:
|
||||
raise RuntimeError(f"Encoding job {job_id} already exists")
|
||||
if resp.status != 200:
|
||||
text = await resp.text()
|
||||
raise RuntimeError(f"Failed to submit encoding job: {resp.status} - {text}")
|
||||
|
||||
return await resp.json()
|
||||
|
||||
async def get_job_status(self, job_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Get the status of an encoding job.
|
||||
|
||||
Args:
|
||||
job_id: Job identifier
|
||||
|
||||
Returns:
|
||||
Job status including: status, progress, error, output_files
|
||||
"""
|
||||
self._load_credentials()
|
||||
|
||||
if not self.is_configured:
|
||||
raise RuntimeError("Encoding service not configured")
|
||||
|
||||
url = f"{self._url}/status/{job_id}"
|
||||
headers = {"X-API-Key": self._api_key}
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.get(url, headers=headers, timeout=30) as resp:
|
||||
if resp.status == 401:
|
||||
raise RuntimeError("Invalid API key for encoding worker")
|
||||
if resp.status == 404:
|
||||
raise RuntimeError(f"Encoding job {job_id} not found")
|
||||
if resp.status != 200:
|
||||
text = await resp.text()
|
||||
raise RuntimeError(f"Failed to get job status: {resp.status} - {text}")
|
||||
|
||||
return await resp.json()
|
||||
|
||||
async def wait_for_completion(
|
||||
self,
|
||||
job_id: str,
|
||||
poll_interval: float = 10.0,
|
||||
timeout: float = 3600.0,
|
||||
progress_callback=None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Poll for encoding job completion.
|
||||
|
||||
Args:
|
||||
job_id: Job identifier
|
||||
poll_interval: Seconds between status checks
|
||||
timeout: Maximum time to wait (default 1 hour)
|
||||
progress_callback: Optional callback(progress: int) for progress updates
|
||||
|
||||
Returns:
|
||||
Final job status with output files
|
||||
|
||||
Raises:
|
||||
TimeoutError: If job doesn't complete within timeout
|
||||
RuntimeError: If job fails
|
||||
"""
|
||||
logger.info(f"[job:{job_id}] Waiting for GCE encoding to complete...")
|
||||
|
||||
start_time = asyncio.get_event_loop().time()
|
||||
last_progress = 0
|
||||
|
||||
while True:
|
||||
elapsed = asyncio.get_event_loop().time() - start_time
|
||||
if elapsed > timeout:
|
||||
raise TimeoutError(f"Encoding job {job_id} timed out after {timeout}s")
|
||||
|
||||
status = await self.get_job_status(job_id)
|
||||
job_status = status.get("status", "unknown")
|
||||
progress = status.get("progress", 0)
|
||||
|
||||
# Report progress
|
||||
if progress != last_progress:
|
||||
logger.info(f"[job:{job_id}] Encoding progress: {progress}%")
|
||||
last_progress = progress
|
||||
if progress_callback:
|
||||
try:
|
||||
progress_callback(progress)
|
||||
except Exception as e:
|
||||
logger.warning(f"Progress callback failed: {e}")
|
||||
|
||||
if job_status == "complete":
|
||||
logger.info(f"[job:{job_id}] GCE encoding complete in {elapsed:.1f}s")
|
||||
return status
|
||||
|
||||
if job_status == "failed":
|
||||
error = status.get("error", "Unknown error")
|
||||
raise RuntimeError(f"Encoding job {job_id} failed: {error}")
|
||||
|
||||
await asyncio.sleep(poll_interval)
|
||||
|
||||
async def encode_videos(
|
||||
self,
|
||||
job_id: str,
|
||||
input_gcs_path: str,
|
||||
output_gcs_path: str,
|
||||
encoding_config: Optional[Dict[str, Any]] = None,
|
||||
progress_callback=None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Submit encoding job and wait for completion.
|
||||
|
||||
This is a convenience method that combines submit + wait.
|
||||
|
||||
Args:
|
||||
job_id: Unique job identifier
|
||||
input_gcs_path: GCS path to input files
|
||||
output_gcs_path: GCS path for output files
|
||||
encoding_config: Optional encoding configuration
|
||||
progress_callback: Optional callback for progress updates
|
||||
|
||||
Returns:
|
||||
Final job status with output files
|
||||
"""
|
||||
config = encoding_config or {"formats": ["mp4_4k", "mp4_720p"]}
|
||||
|
||||
# Submit the job
|
||||
await self.submit_encoding_job(job_id, input_gcs_path, output_gcs_path, config)
|
||||
|
||||
# Wait for completion
|
||||
return await self.wait_for_completion(
|
||||
job_id, progress_callback=progress_callback
|
||||
)
|
||||
|
||||
async def health_check(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Check the health of the encoding worker.
|
||||
|
||||
Returns:
|
||||
Health status including active jobs and FFmpeg version
|
||||
"""
|
||||
self._load_credentials()
|
||||
|
||||
if not self.is_configured:
|
||||
return {"status": "not_configured"}
|
||||
|
||||
url = f"{self._url}/health"
|
||||
headers = {"X-API-Key": self._api_key}
|
||||
|
||||
try:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.get(url, headers=headers, timeout=10) as resp:
|
||||
if resp.status == 200:
|
||||
return await resp.json()
|
||||
return {"status": "error", "code": resp.status}
|
||||
except Exception as e:
|
||||
return {"status": "error", "error": str(e)}
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_encoding_service: Optional[EncodingService] = None
|
||||
|
||||
|
||||
def get_encoding_service() -> EncodingService:
|
||||
"""Get the singleton encoding service instance."""
|
||||
global _encoding_service
|
||||
if _encoding_service is None:
|
||||
_encoding_service = EncodingService()
|
||||
return _encoding_service
|
||||
@@ -114,47 +114,53 @@ class FirestoreService:
|
||||
client_id: Optional[str] = None,
|
||||
created_after: Optional[datetime] = None,
|
||||
created_before: Optional[datetime] = None,
|
||||
user_email: Optional[str] = None,
|
||||
limit: int = 100
|
||||
) -> List[Job]:
|
||||
"""
|
||||
List jobs with optional filters.
|
||||
|
||||
|
||||
Args:
|
||||
status: Filter by job status
|
||||
environment: Filter by request_metadata.environment (test/production/development)
|
||||
client_id: Filter by request_metadata.client_id
|
||||
created_after: Filter jobs created after this datetime
|
||||
created_before: Filter jobs created before this datetime
|
||||
user_email: Filter by user_email (owner of the job)
|
||||
limit: Maximum number of jobs to return
|
||||
|
||||
|
||||
Returns:
|
||||
List of Job objects matching filters, ordered by created_at descending
|
||||
"""
|
||||
try:
|
||||
query = self.db.collection(self.collection)
|
||||
|
||||
|
||||
if status:
|
||||
query = query.where(filter=FieldFilter('status', '==', status.value))
|
||||
|
||||
|
||||
# Filter by request_metadata fields using dot notation
|
||||
if environment:
|
||||
query = query.where(filter=FieldFilter('request_metadata.environment', '==', environment))
|
||||
|
||||
|
||||
if client_id:
|
||||
query = query.where(filter=FieldFilter('request_metadata.client_id', '==', client_id))
|
||||
|
||||
|
||||
# Filter by user_email (job owner)
|
||||
if user_email:
|
||||
query = query.where(filter=FieldFilter('user_email', '==', user_email.lower()))
|
||||
|
||||
# Date range filters
|
||||
if created_after:
|
||||
query = query.where(filter=FieldFilter('created_at', '>=', created_after))
|
||||
|
||||
|
||||
if created_before:
|
||||
query = query.where(filter=FieldFilter('created_at', '<=', created_before))
|
||||
|
||||
|
||||
query = query.order_by('created_at', direction=firestore.Query.DESCENDING).limit(limit)
|
||||
|
||||
|
||||
docs = query.stream()
|
||||
jobs = [Job(**doc.to_dict()) for doc in docs]
|
||||
|
||||
|
||||
return jobs
|
||||
except Exception as e:
|
||||
logger.error(f"Error listing jobs: {e}")
|
||||
|
||||
@@ -136,8 +136,10 @@ class GoogleDriveService:
|
||||
logger.info(f"Looking for folder '{folder_name}' in parent {parent_id}")
|
||||
|
||||
# Search for existing folder
|
||||
# Escape single quotes in folder name for Google Drive API query syntax
|
||||
escaped_folder_name = folder_name.replace("'", "\\'")
|
||||
query = (
|
||||
f"name='{folder_name}' and '{parent_id}' in parents "
|
||||
f"name='{escaped_folder_name}' and '{parent_id}' in parents "
|
||||
f"and mimeType='application/vnd.google-apps.folder' and trashed=false"
|
||||
)
|
||||
results = self.service.files().list(q=query, fields="files(id, name)").execute()
|
||||
@@ -203,8 +205,10 @@ class GoogleDriveService:
|
||||
|
||||
# Check for existing file with same name
|
||||
if replace_existing:
|
||||
# Escape single quotes in filename for Google Drive API query syntax
|
||||
escaped_filename = filename.replace("'", "\\'")
|
||||
query = (
|
||||
f"name='{filename}' and '{parent_id}' in parents and trashed=false"
|
||||
f"name='{escaped_filename}' and '{parent_id}' in parents and trashed=false"
|
||||
)
|
||||
results = self.service.files().list(q=query, fields="files(id)").execute()
|
||||
for existing_file in results.get("files", []):
|
||||
@@ -296,6 +300,45 @@ class GoogleDriveService:
|
||||
logger.info(f"Public share upload complete: {len(uploaded_files)} files uploaded")
|
||||
return uploaded_files
|
||||
|
||||
def delete_file(self, file_id: str) -> bool:
|
||||
"""
|
||||
Delete a file from Google Drive.
|
||||
|
||||
Args:
|
||||
file_id: Google Drive file ID to delete
|
||||
|
||||
Returns:
|
||||
True if deleted successfully, False otherwise
|
||||
"""
|
||||
logger.info(f"Deleting Google Drive file: {file_id}")
|
||||
|
||||
try:
|
||||
self.service.files().delete(fileId=file_id).execute()
|
||||
logger.info(f"Successfully deleted file: {file_id}")
|
||||
return True
|
||||
except Exception as e:
|
||||
# Check if it's a 404 (already deleted)
|
||||
if hasattr(e, 'resp') and e.resp.status == 404:
|
||||
logger.warning(f"File not found (already deleted?): {file_id}")
|
||||
return True
|
||||
logger.error(f"Failed to delete Google Drive file: {e}")
|
||||
return False
|
||||
|
||||
def delete_files(self, file_ids: list[str]) -> dict[str, bool]:
|
||||
"""
|
||||
Delete multiple files from Google Drive.
|
||||
|
||||
Args:
|
||||
file_ids: List of Google Drive file IDs to delete
|
||||
|
||||
Returns:
|
||||
Dictionary mapping file_id to success status
|
||||
"""
|
||||
results = {}
|
||||
for file_id in file_ids:
|
||||
results[file_id] = self.delete_file(file_id)
|
||||
return results
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_gdrive_service: Optional[GoogleDriveService] = None
|
||||
|
||||
@@ -111,19 +111,21 @@ class JobManager:
|
||||
client_id: Optional[str] = None,
|
||||
created_after: Optional[datetime] = None,
|
||||
created_before: Optional[datetime] = None,
|
||||
user_email: Optional[str] = None,
|
||||
limit: int = 100
|
||||
) -> List[Job]:
|
||||
"""
|
||||
List jobs with optional filtering.
|
||||
|
||||
|
||||
Args:
|
||||
status: Filter by job status
|
||||
environment: Filter by request_metadata.environment (test/production/development)
|
||||
client_id: Filter by request_metadata.client_id (customer identifier)
|
||||
created_after: Filter jobs created after this datetime
|
||||
created_before: Filter jobs created before this datetime
|
||||
user_email: Filter by user_email (job owner)
|
||||
limit: Maximum number of jobs to return
|
||||
|
||||
|
||||
Returns:
|
||||
List of Job objects matching filters
|
||||
"""
|
||||
@@ -133,6 +135,7 @@ class JobManager:
|
||||
client_id=client_id,
|
||||
created_after=created_after,
|
||||
created_before=created_before,
|
||||
user_email=user_email,
|
||||
limit=limit
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
"""
|
||||
LyricsTranscriber cache synchronization with GCS.
|
||||
|
||||
This service persists LyricsTranscriber's cache files to GCS so that
|
||||
cloud workers (Cloud Run instances) can share cache across containers.
|
||||
|
||||
Cache files are stored flat in GCS under lyrics-transcriber-cache/ prefix:
|
||||
- Transcription: {provider}_{audio_hash}_raw.json, {provider}_{audio_hash}_converted.json
|
||||
- Lyrics: {provider}_{artist_title_hash}_raw.json, {provider}_{artist_title_hash}_converted.json
|
||||
|
||||
Hash computation matches LyricsTranscriber's implementation exactly:
|
||||
- Audio hash: MD5 of audio file bytes
|
||||
- Lyrics hash: MD5 of "{artist.lower()}_{title.lower()}"
|
||||
"""
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
from backend.services.storage_service import StorageService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Providers that use audio file hash as cache key
|
||||
TRANSCRIPTION_PROVIDERS = ["audioshake", "whisper", "localwhisper"]
|
||||
|
||||
# Providers that use artist+title hash as cache key
|
||||
LYRICS_PROVIDERS = ["genius", "spotify", "lrclib", "musixmatch"]
|
||||
|
||||
# Cache file suffixes
|
||||
CACHE_SUFFIXES = ["raw", "converted"]
|
||||
|
||||
|
||||
class LyricsCacheService:
|
||||
"""Service to sync LyricsTranscriber cache with GCS."""
|
||||
|
||||
GCS_CACHE_PREFIX = "lyrics-transcriber-cache/"
|
||||
|
||||
def __init__(self, storage: Optional[StorageService] = None):
|
||||
"""Initialize the cache service.
|
||||
|
||||
Args:
|
||||
storage: StorageService instance. If None, creates a new one.
|
||||
"""
|
||||
self.storage = storage or StorageService()
|
||||
|
||||
def compute_audio_hash(self, audio_path: str) -> str:
|
||||
"""Compute MD5 hash of audio file bytes.
|
||||
|
||||
This matches LyricsTranscriber's _get_file_hash() method exactly.
|
||||
|
||||
Args:
|
||||
audio_path: Path to the audio file.
|
||||
|
||||
Returns:
|
||||
MD5 hex digest of the file contents.
|
||||
"""
|
||||
md5_hash = hashlib.md5()
|
||||
with open(audio_path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(4096), b""):
|
||||
md5_hash.update(chunk)
|
||||
return md5_hash.hexdigest()
|
||||
|
||||
def compute_lyrics_hash(self, artist: str, title: str) -> str:
|
||||
"""Compute MD5 hash of artist and title.
|
||||
|
||||
This matches LyricsTranscriber's _get_artist_title_hash() method exactly.
|
||||
|
||||
Args:
|
||||
artist: Artist name.
|
||||
title: Track title.
|
||||
|
||||
Returns:
|
||||
MD5 hex digest of "{artist.lower()}_{title.lower()}".
|
||||
"""
|
||||
combined = f"{artist.lower()}_{title.lower()}"
|
||||
return hashlib.md5(combined.encode()).hexdigest()
|
||||
|
||||
def _get_cache_filenames(
|
||||
self, providers: List[str], hash_value: str
|
||||
) -> List[str]:
|
||||
"""Generate list of possible cache filenames for given providers and hash.
|
||||
|
||||
Args:
|
||||
providers: List of provider names (e.g., ["audioshake", "whisper"]).
|
||||
hash_value: The hash to use in filenames.
|
||||
|
||||
Returns:
|
||||
List of filenames like ["audioshake_abc123_raw.json", ...].
|
||||
"""
|
||||
filenames = []
|
||||
for provider in providers:
|
||||
for suffix in CACHE_SUFFIXES:
|
||||
filenames.append(f"{provider}_{hash_value}_{suffix}.json")
|
||||
return filenames
|
||||
|
||||
def sync_cache_from_gcs(
|
||||
self,
|
||||
local_cache_dir: str,
|
||||
audio_hash: str,
|
||||
lyrics_hash: str,
|
||||
) -> Dict[str, int]:
|
||||
"""Download relevant cache files from GCS to local directory.
|
||||
|
||||
Downloads cache files for both transcription (audio hash) and
|
||||
lyrics (artist+title hash) providers.
|
||||
|
||||
Args:
|
||||
local_cache_dir: Local directory to download cache files to.
|
||||
audio_hash: MD5 hash of audio file.
|
||||
lyrics_hash: MD5 hash of artist+title.
|
||||
|
||||
Returns:
|
||||
Dict with counts: {"downloaded": N, "not_found": M, "errors": E}
|
||||
"""
|
||||
os.makedirs(local_cache_dir, exist_ok=True)
|
||||
|
||||
stats = {"downloaded": 0, "not_found": 0, "errors": 0}
|
||||
|
||||
# Get all possible cache filenames
|
||||
transcription_files = self._get_cache_filenames(
|
||||
TRANSCRIPTION_PROVIDERS, audio_hash
|
||||
)
|
||||
lyrics_files = self._get_cache_filenames(LYRICS_PROVIDERS, lyrics_hash)
|
||||
all_files = transcription_files + lyrics_files
|
||||
|
||||
for filename in all_files:
|
||||
gcs_path = f"{self.GCS_CACHE_PREFIX}{filename}"
|
||||
local_path = os.path.join(local_cache_dir, filename)
|
||||
|
||||
try:
|
||||
if self.storage.file_exists(gcs_path):
|
||||
self.storage.download_file(gcs_path, local_path)
|
||||
logger.info(f"Cache hit: downloaded {filename}")
|
||||
stats["downloaded"] += 1
|
||||
else:
|
||||
logger.debug(f"Cache miss: {filename} not in GCS")
|
||||
stats["not_found"] += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"Error downloading cache file {filename}: {e}")
|
||||
stats["errors"] += 1
|
||||
|
||||
logger.info(
|
||||
f"Cache sync from GCS complete: "
|
||||
f"{stats['downloaded']} downloaded, "
|
||||
f"{stats['not_found']} not found, "
|
||||
f"{stats['errors']} errors"
|
||||
)
|
||||
return stats
|
||||
|
||||
def sync_cache_to_gcs(
|
||||
self,
|
||||
local_cache_dir: str,
|
||||
audio_hash: str,
|
||||
lyrics_hash: str,
|
||||
) -> Dict[str, int]:
|
||||
"""Upload new cache files from local directory to GCS.
|
||||
|
||||
Only uploads files that match expected cache patterns for the given
|
||||
hashes and don't already exist in GCS.
|
||||
|
||||
Args:
|
||||
local_cache_dir: Local directory with cache files.
|
||||
audio_hash: MD5 hash of audio file.
|
||||
lyrics_hash: MD5 hash of artist+title.
|
||||
|
||||
Returns:
|
||||
Dict with counts: {"uploaded": N, "skipped": M, "errors": E}
|
||||
"""
|
||||
stats = {"uploaded": 0, "skipped": 0, "errors": 0}
|
||||
|
||||
if not os.path.exists(local_cache_dir):
|
||||
logger.warning(f"Local cache dir does not exist: {local_cache_dir}")
|
||||
return stats
|
||||
|
||||
# Get all possible cache filenames we're interested in
|
||||
transcription_files = self._get_cache_filenames(
|
||||
TRANSCRIPTION_PROVIDERS, audio_hash
|
||||
)
|
||||
lyrics_files = self._get_cache_filenames(LYRICS_PROVIDERS, lyrics_hash)
|
||||
expected_files = set(transcription_files + lyrics_files)
|
||||
|
||||
# Check each file in local cache dir
|
||||
for filename in os.listdir(local_cache_dir):
|
||||
# Only process files we expect (matching our hash patterns)
|
||||
if filename not in expected_files:
|
||||
continue
|
||||
|
||||
local_path = os.path.join(local_cache_dir, filename)
|
||||
if not os.path.isfile(local_path):
|
||||
continue
|
||||
|
||||
gcs_path = f"{self.GCS_CACHE_PREFIX}{filename}"
|
||||
|
||||
try:
|
||||
# Skip if already exists in GCS (same hash = same content)
|
||||
if self.storage.file_exists(gcs_path):
|
||||
logger.debug(f"Cache file already in GCS: {filename}")
|
||||
stats["skipped"] += 1
|
||||
continue
|
||||
|
||||
self.storage.upload_file(local_path, gcs_path)
|
||||
logger.info(f"Uploaded cache file: {filename}")
|
||||
stats["uploaded"] += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"Error uploading cache file {filename}: {e}")
|
||||
stats["errors"] += 1
|
||||
|
||||
logger.info(
|
||||
f"Cache sync to GCS complete: "
|
||||
f"{stats['uploaded']} uploaded, "
|
||||
f"{stats['skipped']} skipped (already exist), "
|
||||
f"{stats['errors']} errors"
|
||||
)
|
||||
return stats
|
||||
@@ -0,0 +1,275 @@
|
||||
"""
|
||||
Stripe service for payment processing.
|
||||
|
||||
Handles:
|
||||
- Creating checkout sessions for credit purchases
|
||||
- Processing webhook events
|
||||
- Managing customer records
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
from typing import Optional, Dict, Any, Tuple
|
||||
|
||||
import stripe
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Credit packages available for purchase
|
||||
CREDIT_PACKAGES = {
|
||||
"1_credit": {
|
||||
"credits": 1,
|
||||
"price_cents": 500, # $5.00
|
||||
"name": "1 Karaoke Credit",
|
||||
"description": "Create 1 professional karaoke video",
|
||||
},
|
||||
"3_credits": {
|
||||
"credits": 3,
|
||||
"price_cents": 1200, # $12.00 (20% discount)
|
||||
"name": "3 Karaoke Credits",
|
||||
"description": "Create 3 professional karaoke videos (Save 20%)",
|
||||
},
|
||||
"5_credits": {
|
||||
"credits": 5,
|
||||
"price_cents": 1750, # $17.50 (30% discount)
|
||||
"name": "5 Karaoke Credits",
|
||||
"description": "Create 5 professional karaoke videos (Save 30%)",
|
||||
},
|
||||
"10_credits": {
|
||||
"credits": 10,
|
||||
"price_cents": 3000, # $30.00 (40% discount)
|
||||
"name": "10 Karaoke Credits",
|
||||
"description": "Create 10 professional karaoke videos (Save 40%)",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class StripeService:
|
||||
"""Service for Stripe payment processing."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize Stripe with API key."""
|
||||
self.settings = get_settings()
|
||||
self.secret_key = os.getenv("STRIPE_SECRET_KEY")
|
||||
self.webhook_secret = os.getenv("STRIPE_WEBHOOK_SECRET")
|
||||
self.frontend_url = os.getenv("FRONTEND_URL", "https://gen.nomadkaraoke.com")
|
||||
# After consolidation, buy URL is the same as frontend URL
|
||||
self.buy_url = os.getenv("BUY_URL", self.frontend_url)
|
||||
|
||||
if self.secret_key:
|
||||
stripe.api_key = self.secret_key
|
||||
logger.info("Stripe initialized with API key")
|
||||
else:
|
||||
logger.warning("STRIPE_SECRET_KEY not set - payments disabled")
|
||||
|
||||
def is_configured(self) -> bool:
|
||||
"""Check if Stripe is properly configured."""
|
||||
return bool(self.secret_key)
|
||||
|
||||
def get_credit_packages(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get available credit packages."""
|
||||
return CREDIT_PACKAGES
|
||||
|
||||
def create_checkout_session(
|
||||
self,
|
||||
package_id: str,
|
||||
user_email: str,
|
||||
success_url: Optional[str] = None,
|
||||
cancel_url: Optional[str] = None,
|
||||
) -> Tuple[bool, Optional[str], str]:
|
||||
"""
|
||||
Create a Stripe Checkout session for purchasing credits.
|
||||
|
||||
Args:
|
||||
package_id: ID of the credit package to purchase
|
||||
user_email: Email of the purchasing user
|
||||
success_url: URL to redirect to on success (optional)
|
||||
cancel_url: URL to redirect to on cancel (optional)
|
||||
|
||||
Returns:
|
||||
(success, checkout_url, message)
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return False, None, "Payment processing is not configured"
|
||||
|
||||
package = CREDIT_PACKAGES.get(package_id)
|
||||
if not package:
|
||||
return False, None, f"Invalid package: {package_id}"
|
||||
|
||||
try:
|
||||
# Default URLs
|
||||
if not success_url:
|
||||
success_url = f"{self.frontend_url}/payment/success?session_id={{CHECKOUT_SESSION_ID}}"
|
||||
if not cancel_url:
|
||||
cancel_url = f"{self.buy_url}?cancelled=true"
|
||||
|
||||
# Create checkout session
|
||||
session = stripe.checkout.Session.create(
|
||||
payment_method_types=['card'],
|
||||
line_items=[{
|
||||
'price_data': {
|
||||
'currency': 'usd',
|
||||
'product_data': {
|
||||
'name': package['name'],
|
||||
'description': package['description'],
|
||||
},
|
||||
'unit_amount': package['price_cents'],
|
||||
},
|
||||
'quantity': 1,
|
||||
}],
|
||||
mode='payment',
|
||||
success_url=success_url,
|
||||
cancel_url=cancel_url,
|
||||
customer_email=user_email,
|
||||
metadata={
|
||||
'package_id': package_id,
|
||||
'credits': str(package['credits']),
|
||||
'user_email': user_email,
|
||||
},
|
||||
# Allow promotion codes
|
||||
allow_promotion_codes=True,
|
||||
)
|
||||
|
||||
logger.info(f"Created checkout session {session.id} for {user_email}, package {package_id}")
|
||||
return True, session.url, "Checkout session created"
|
||||
|
||||
except stripe.error.StripeError as e:
|
||||
logger.error(f"Stripe error creating checkout session: {e}")
|
||||
return False, None, f"Payment error: {str(e)}"
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating checkout session: {e}")
|
||||
return False, None, "Failed to create checkout session"
|
||||
|
||||
def verify_webhook_signature(self, payload: bytes, signature: str) -> Tuple[bool, Optional[Dict], str]:
|
||||
"""
|
||||
Verify a Stripe webhook signature.
|
||||
|
||||
Args:
|
||||
payload: Raw request body
|
||||
signature: Stripe-Signature header value
|
||||
|
||||
Returns:
|
||||
(valid, event_data, message)
|
||||
"""
|
||||
if not self.webhook_secret:
|
||||
logger.error("STRIPE_WEBHOOK_SECRET not configured")
|
||||
return False, None, "Webhook secret not configured"
|
||||
|
||||
try:
|
||||
event = stripe.Webhook.construct_event(
|
||||
payload, signature, self.webhook_secret
|
||||
)
|
||||
return True, event, "Webhook verified"
|
||||
except stripe.error.SignatureVerificationError as e:
|
||||
logger.error(f"Invalid webhook signature: {e}")
|
||||
return False, None, "Invalid signature"
|
||||
except Exception as e:
|
||||
logger.error(f"Error verifying webhook: {e}")
|
||||
return False, None, str(e)
|
||||
|
||||
def handle_checkout_completed(self, session: Dict) -> Tuple[bool, Optional[str], int, str]:
|
||||
"""
|
||||
Handle a completed checkout session.
|
||||
|
||||
Args:
|
||||
session: Stripe checkout session object
|
||||
|
||||
Returns:
|
||||
(success, user_email, credits_to_add, message)
|
||||
"""
|
||||
try:
|
||||
metadata = session.get('metadata', {})
|
||||
user_email = metadata.get('user_email') or session.get('customer_email')
|
||||
try:
|
||||
credits = int(metadata.get('credits', 0))
|
||||
except (ValueError, TypeError):
|
||||
logger.error(f"Invalid credits metadata in session {session.get('id')}: {metadata.get('credits')}")
|
||||
return False, user_email, 0, "Invalid credit amount in session metadata"
|
||||
package_id = metadata.get('package_id')
|
||||
|
||||
if not user_email:
|
||||
logger.error(f"No user email in checkout session {session.get('id')}")
|
||||
return False, None, 0, "No user email found"
|
||||
|
||||
if credits <= 0:
|
||||
logger.error(f"Invalid credits in checkout session {session.get('id')}")
|
||||
return False, user_email, 0, "Invalid credit amount"
|
||||
|
||||
logger.info(
|
||||
f"Checkout completed: {user_email} purchased {credits} credits "
|
||||
f"(package: {package_id}, session: {session.get('id')})"
|
||||
)
|
||||
|
||||
return True, user_email, credits, f"Successfully purchased {credits} credits"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error handling checkout completed: {e}")
|
||||
return False, None, 0, str(e)
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get a checkout session by ID.
|
||||
|
||||
Args:
|
||||
session_id: Stripe checkout session ID
|
||||
|
||||
Returns:
|
||||
Session data or None
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return None
|
||||
|
||||
try:
|
||||
session = stripe.checkout.Session.retrieve(session_id)
|
||||
return dict(session)
|
||||
except Exception as e:
|
||||
logger.error(f"Error retrieving session {session_id}: {e}")
|
||||
return None
|
||||
|
||||
def create_customer(self, email: str, name: Optional[str] = None) -> Optional[str]:
|
||||
"""
|
||||
Create or get a Stripe customer for a user.
|
||||
|
||||
Args:
|
||||
email: User's email
|
||||
name: User's display name (optional)
|
||||
|
||||
Returns:
|
||||
Stripe customer ID or None
|
||||
"""
|
||||
if not self.is_configured():
|
||||
return None
|
||||
|
||||
try:
|
||||
# Check if customer already exists
|
||||
customers = stripe.Customer.list(email=email, limit=1)
|
||||
if customers.data:
|
||||
return customers.data[0].id
|
||||
|
||||
# Create new customer
|
||||
customer = stripe.Customer.create(
|
||||
email=email,
|
||||
name=name,
|
||||
metadata={'source': 'nomad_karaoke'},
|
||||
)
|
||||
logger.info(f"Created Stripe customer {customer.id} for {email}")
|
||||
return customer.id
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error creating Stripe customer: {e}")
|
||||
return None
|
||||
|
||||
|
||||
# Global instance
|
||||
_stripe_service = None
|
||||
|
||||
|
||||
def get_stripe_service() -> StripeService:
|
||||
"""Get the global Stripe service instance."""
|
||||
global _stripe_service
|
||||
if _stripe_service is None:
|
||||
_stripe_service = StripeService()
|
||||
return _stripe_service
|
||||
@@ -91,7 +91,15 @@ class StructuredFormatter(logging.Formatter):
|
||||
|
||||
# Add custom fields from 'extra' dict
|
||||
# Common fields we want to extract from log records
|
||||
custom_fields = ["job_id", "worker", "operation", "duration", "status", "error"]
|
||||
custom_fields = [
|
||||
# Job-related fields
|
||||
"job_id", "worker", "operation", "duration", "status", "error",
|
||||
# Audit logging fields (from middleware and auth)
|
||||
"request_id", "user_email", "client_ip", "latency_ms",
|
||||
"audit_type", "method", "path", "status_code", "query_string",
|
||||
"user_agent", "user_type", "is_admin", "remaining_uses",
|
||||
"auth_message", "token_provided", "token_length", "auth_header_present",
|
||||
]
|
||||
for field in custom_fields:
|
||||
value = getattr(record, field, None)
|
||||
if value is not None:
|
||||
|
||||
@@ -0,0 +1,721 @@
|
||||
"""
|
||||
User service for authentication, credits, and user management.
|
||||
|
||||
Handles:
|
||||
- Magic link authentication (send, verify)
|
||||
- Session management
|
||||
- Credit operations (add, deduct, refund)
|
||||
- User CRUD operations
|
||||
"""
|
||||
import logging
|
||||
import secrets
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional, List, Tuple
|
||||
|
||||
from google.cloud import firestore
|
||||
from google.cloud.firestore_v1 import FieldFilter
|
||||
from google.cloud.firestore_v1 import Increment
|
||||
|
||||
from backend.config import get_settings
|
||||
from backend.models.user import (
|
||||
User,
|
||||
UserRole,
|
||||
UserPublic,
|
||||
MagicLinkToken,
|
||||
Session,
|
||||
CreditTransaction,
|
||||
)
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Collection names
|
||||
USERS_COLLECTION = "gen_users"
|
||||
MAGIC_LINKS_COLLECTION = "magic_links"
|
||||
SESSIONS_COLLECTION = "sessions"
|
||||
PROCESSED_STRIPE_SESSIONS_COLLECTION = "processed_stripe_sessions"
|
||||
|
||||
# Token/session configuration
|
||||
MAGIC_LINK_EXPIRY_MINUTES = 15
|
||||
SESSION_EXPIRY_DAYS = 7
|
||||
SESSION_ABSOLUTE_EXPIRY_DAYS = 30
|
||||
MAX_CREDIT_TRANSACTIONS = 100 # Keep last N transactions
|
||||
|
||||
|
||||
class UserService:
|
||||
"""Service for user management and authentication."""
|
||||
|
||||
# Number of free credits granted to new users
|
||||
NEW_USER_FREE_CREDITS = 1
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize user service with Firestore client."""
|
||||
self.settings = get_settings()
|
||||
self.db = firestore.Client(project=self.settings.google_cloud_project)
|
||||
|
||||
# =========================================================================
|
||||
# User CRUD Operations
|
||||
# =========================================================================
|
||||
|
||||
def get_user(self, email: str) -> Optional[User]:
|
||||
"""Get a user by email."""
|
||||
try:
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc = doc_ref.get()
|
||||
|
||||
if not doc.exists:
|
||||
return None
|
||||
|
||||
return User(**doc.to_dict())
|
||||
except Exception:
|
||||
logger.exception(f"Error getting user {email}")
|
||||
return None
|
||||
|
||||
def get_or_create_user(self, email: str) -> User:
|
||||
"""
|
||||
Get existing user or create a new one.
|
||||
|
||||
New users receive a welcome credit to try the service.
|
||||
"""
|
||||
email = email.lower()
|
||||
user = self.get_user(email)
|
||||
|
||||
if user:
|
||||
return user
|
||||
|
||||
# Create new user with welcome credit
|
||||
welcome_credit = self.NEW_USER_FREE_CREDITS
|
||||
welcome_transaction = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=welcome_credit,
|
||||
reason="welcome_credit",
|
||||
)
|
||||
|
||||
user = User(
|
||||
email=email,
|
||||
credits=welcome_credit,
|
||||
credit_transactions=[welcome_transaction],
|
||||
)
|
||||
self._save_user(user)
|
||||
logger.info(f"Created new user: {email} with {welcome_credit} welcome credit(s)")
|
||||
return user
|
||||
|
||||
def _save_user(self, user: User) -> None:
|
||||
"""Save user to Firestore."""
|
||||
try:
|
||||
user.updated_at = datetime.utcnow()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(user.email.lower())
|
||||
doc_ref.set(user.model_dump(mode='json'))
|
||||
except Exception:
|
||||
logger.exception(f"Error saving user {user.email}")
|
||||
raise
|
||||
|
||||
def update_user(self, email: str, **updates) -> Optional[User]:
|
||||
"""Update user fields."""
|
||||
try:
|
||||
updates['updated_at'] = datetime.utcnow()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc_ref.update(updates)
|
||||
return self.get_user(email)
|
||||
except Exception:
|
||||
logger.exception(f"Error updating user {email}")
|
||||
return None
|
||||
|
||||
def list_users(self, limit: int = 100, include_inactive: bool = False) -> List[User]:
|
||||
"""List all users (admin only)."""
|
||||
try:
|
||||
query = self.db.collection(USERS_COLLECTION)
|
||||
|
||||
if not include_inactive:
|
||||
query = query.where(filter=FieldFilter('is_active', '==', True))
|
||||
|
||||
query = query.order_by('created_at', direction=firestore.Query.DESCENDING)
|
||||
query = query.limit(limit)
|
||||
|
||||
docs = query.stream()
|
||||
return [User(**doc.to_dict()) for doc in docs]
|
||||
except Exception:
|
||||
logger.exception("Error listing users")
|
||||
return []
|
||||
|
||||
def get_user_public(self, email: str) -> Optional[UserPublic]:
|
||||
"""Get public user info."""
|
||||
user = self.get_user(email)
|
||||
if not user:
|
||||
return None
|
||||
|
||||
return UserPublic(
|
||||
email=user.email,
|
||||
role=user.role,
|
||||
credits=user.credits,
|
||||
display_name=user.display_name,
|
||||
total_jobs_created=user.total_jobs_created,
|
||||
total_jobs_completed=user.total_jobs_completed,
|
||||
)
|
||||
|
||||
# =========================================================================
|
||||
# Magic Link Authentication
|
||||
# =========================================================================
|
||||
|
||||
def create_magic_link(
|
||||
self,
|
||||
email: str,
|
||||
ip_address: Optional[str] = None,
|
||||
user_agent: Optional[str] = None
|
||||
) -> MagicLinkToken:
|
||||
"""
|
||||
Create a magic link token for email authentication.
|
||||
|
||||
Returns the token object. The actual sending of the email
|
||||
should be handled by the caller (or an email service).
|
||||
"""
|
||||
email = email.lower()
|
||||
|
||||
# Ensure user exists
|
||||
self.get_or_create_user(email)
|
||||
|
||||
# Generate secure token
|
||||
token = secrets.token_urlsafe(32)
|
||||
|
||||
magic_link = MagicLinkToken(
|
||||
token=token,
|
||||
email=email,
|
||||
expires_at=datetime.utcnow() + timedelta(minutes=MAGIC_LINK_EXPIRY_MINUTES),
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent,
|
||||
)
|
||||
|
||||
# Save to Firestore
|
||||
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
|
||||
doc_ref.set(magic_link.model_dump(mode='json'))
|
||||
|
||||
logger.info(f"Created magic link for {email}")
|
||||
return magic_link
|
||||
|
||||
def verify_magic_link(self, token: str) -> Tuple[bool, Optional[User], str]:
|
||||
"""
|
||||
Verify a magic link token using a Firestore transaction to prevent race conditions.
|
||||
|
||||
Returns:
|
||||
(success, user, message)
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(MAGIC_LINKS_COLLECTION).document(token)
|
||||
|
||||
@firestore.transactional
|
||||
def verify_in_transaction(transaction):
|
||||
"""Atomically verify and mark magic link as used."""
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
|
||||
if not doc.exists:
|
||||
return False, None, "Invalid or expired link"
|
||||
|
||||
magic_link = MagicLinkToken(**doc.to_dict())
|
||||
|
||||
# Check if already used
|
||||
if magic_link.used:
|
||||
return False, None, "This link has already been used"
|
||||
|
||||
# Check expiry
|
||||
if datetime.utcnow() > magic_link.expires_at:
|
||||
return False, None, "This link has expired"
|
||||
|
||||
# Mark as used atomically within transaction
|
||||
transaction.update(doc_ref, {
|
||||
'used': True,
|
||||
'used_at': datetime.utcnow()
|
||||
})
|
||||
|
||||
return True, magic_link.email, "Success"
|
||||
|
||||
# Execute the transaction
|
||||
transaction = self.db.transaction()
|
||||
success, email_or_error, message = verify_in_transaction(transaction)
|
||||
|
||||
if not success:
|
||||
return False, None, message
|
||||
|
||||
# Get user and mark email as verified (outside transaction)
|
||||
user = self.get_user(email_or_error)
|
||||
if user:
|
||||
self.update_user(
|
||||
email_or_error,
|
||||
email_verified=True,
|
||||
last_login_at=datetime.utcnow()
|
||||
)
|
||||
user = self.get_user(email_or_error) # Refresh
|
||||
|
||||
logger.info(f"Magic link verified for {email_or_error}")
|
||||
return True, user, "Successfully authenticated"
|
||||
|
||||
except Exception:
|
||||
logger.exception("Error verifying magic link")
|
||||
return False, None, "An error occurred during verification"
|
||||
|
||||
# =========================================================================
|
||||
# Session Management
|
||||
# =========================================================================
|
||||
|
||||
def create_session(
|
||||
self,
|
||||
user_email: str,
|
||||
ip_address: Optional[str] = None,
|
||||
user_agent: Optional[str] = None
|
||||
) -> Session:
|
||||
"""Create a new session for an authenticated user."""
|
||||
token = secrets.token_urlsafe(32)
|
||||
token_prefix = token[:12]
|
||||
|
||||
session = Session(
|
||||
token=token,
|
||||
user_email=user_email.lower(),
|
||||
expires_at=datetime.utcnow() + timedelta(days=SESSION_ABSOLUTE_EXPIRY_DAYS),
|
||||
ip_address=ip_address,
|
||||
user_agent=user_agent,
|
||||
)
|
||||
|
||||
# Serialize and write to Firestore
|
||||
session_data = session.model_dump(mode='json')
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc_ref.set(session_data)
|
||||
|
||||
# Verify the write succeeded by reading back
|
||||
verify_doc = doc_ref.get()
|
||||
if not verify_doc.exists:
|
||||
logger.error(f"Session write verification FAILED for {user_email}: {token_prefix}...")
|
||||
raise RuntimeError(f"Failed to persist session for {user_email}")
|
||||
|
||||
logger.info(f"Created and verified session for {user_email}: {token_prefix}... (expires: {session.expires_at})")
|
||||
|
||||
return session
|
||||
|
||||
def validate_session(self, token: str) -> Tuple[bool, Optional[User], str]:
|
||||
"""
|
||||
Validate a session token.
|
||||
|
||||
Returns:
|
||||
(valid, user, message)
|
||||
"""
|
||||
# Log token info for debugging (only prefix for security)
|
||||
token_prefix = token[:12] if token and len(token) >= 12 else token
|
||||
logger.debug(f"Validating session token: {token_prefix}... (len={len(token) if token else 0})")
|
||||
|
||||
try:
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc = doc_ref.get()
|
||||
|
||||
if not doc.exists:
|
||||
logger.warning(f"Session not found in Firestore: {token_prefix}...")
|
||||
return False, None, "Invalid session"
|
||||
|
||||
raw_data = doc.to_dict()
|
||||
logger.debug(f"Session data found for {token_prefix}...: user_email={raw_data.get('user_email')}, is_active={raw_data.get('is_active')}")
|
||||
|
||||
session = Session(**raw_data)
|
||||
|
||||
# Check if active
|
||||
if not session.is_active:
|
||||
logger.warning(f"Session revoked for {token_prefix}... (user: {session.user_email})")
|
||||
return False, None, "Session has been revoked"
|
||||
|
||||
# Use timezone-aware datetime for all comparisons
|
||||
# Firestore returns timezone-aware datetimes, so we must use aware datetimes
|
||||
now = datetime.now(timezone.utc)
|
||||
|
||||
# Normalize session datetimes to be timezone-aware for comparison
|
||||
# (handles legacy naive datetimes that may exist in Firestore)
|
||||
expires_at = session.expires_at
|
||||
if expires_at.tzinfo is None:
|
||||
expires_at = expires_at.replace(tzinfo=timezone.utc)
|
||||
|
||||
last_activity = session.last_activity_at
|
||||
if last_activity.tzinfo is None:
|
||||
last_activity = last_activity.replace(tzinfo=timezone.utc)
|
||||
|
||||
# Check expiry
|
||||
if now > expires_at:
|
||||
logger.warning(f"Session expired for {token_prefix}... (user: {session.user_email}, expired_at: {expires_at}, now: {now})")
|
||||
return False, None, "Session has expired"
|
||||
|
||||
# Check inactivity (7 days)
|
||||
inactivity_limit = now - timedelta(days=SESSION_EXPIRY_DAYS)
|
||||
if last_activity < inactivity_limit:
|
||||
logger.warning(f"Session inactive for {token_prefix}... (user: {session.user_email}, last_activity: {last_activity}, limit: {inactivity_limit})")
|
||||
return False, None, "Session expired due to inactivity"
|
||||
|
||||
# Update last activity (use timezone-aware datetime)
|
||||
doc_ref.update({'last_activity_at': now})
|
||||
|
||||
# Get user
|
||||
user = self.get_user(session.user_email)
|
||||
if not user:
|
||||
logger.warning(f"User not found for session {token_prefix}...: {session.user_email}")
|
||||
return False, None, "User not found"
|
||||
|
||||
if not user.is_active:
|
||||
logger.warning(f"User account disabled for session {token_prefix}...: {session.user_email}")
|
||||
return False, None, "User account is disabled"
|
||||
|
||||
logger.debug(f"Session valid for {token_prefix}... (user: {user.email}, credits: {user.credits})")
|
||||
return True, user, "Valid session"
|
||||
|
||||
except Exception:
|
||||
logger.exception(f"Error validating session {token_prefix}...")
|
||||
return False, None, "An error occurred during validation"
|
||||
|
||||
def revoke_session(self, token: str) -> bool:
|
||||
"""Revoke a session (logout)."""
|
||||
try:
|
||||
doc_ref = self.db.collection(SESSIONS_COLLECTION).document(token)
|
||||
doc_ref.update({'is_active': False})
|
||||
logger.info("Session revoked")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error revoking session")
|
||||
return False
|
||||
|
||||
def revoke_all_sessions(self, user_email: str) -> int:
|
||||
"""Revoke all sessions for a user."""
|
||||
try:
|
||||
query = self.db.collection(SESSIONS_COLLECTION).where(
|
||||
filter=FieldFilter('user_email', '==', user_email.lower())
|
||||
).where(
|
||||
filter=FieldFilter('is_active', '==', True)
|
||||
)
|
||||
|
||||
count = 0
|
||||
for doc in query.stream():
|
||||
doc.reference.update({'is_active': False})
|
||||
count += 1
|
||||
|
||||
logger.info(f"Revoked {count} sessions for {user_email}")
|
||||
return count
|
||||
except Exception:
|
||||
logger.exception("Error revoking sessions")
|
||||
return 0
|
||||
|
||||
def list_sessions_for_user(
|
||||
self,
|
||||
user_email: str,
|
||||
include_revoked: bool = False,
|
||||
limit: int = 50
|
||||
) -> List[Session]:
|
||||
"""
|
||||
List all sessions for a user.
|
||||
|
||||
Args:
|
||||
user_email: User's email address
|
||||
include_revoked: If True, include revoked/inactive sessions
|
||||
limit: Maximum number of sessions to return
|
||||
|
||||
Returns:
|
||||
List of Session objects, ordered by created_at descending
|
||||
"""
|
||||
try:
|
||||
query = self.db.collection(SESSIONS_COLLECTION).where(
|
||||
filter=FieldFilter('user_email', '==', user_email.lower())
|
||||
)
|
||||
|
||||
if not include_revoked:
|
||||
query = query.where(filter=FieldFilter('is_active', '==', True))
|
||||
|
||||
query = query.order_by('created_at', direction=firestore.Query.DESCENDING)
|
||||
query = query.limit(limit)
|
||||
|
||||
sessions = []
|
||||
for doc in query.stream():
|
||||
try:
|
||||
sessions.append(Session(**doc.to_dict()))
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to parse session document: {e}")
|
||||
|
||||
logger.debug(f"Found {len(sessions)} sessions for {user_email}")
|
||||
return sessions
|
||||
except Exception:
|
||||
logger.exception(f"Error listing sessions for {user_email}")
|
||||
return []
|
||||
|
||||
# =========================================================================
|
||||
# Credit Operations
|
||||
# =========================================================================
|
||||
|
||||
def add_credits(
|
||||
self,
|
||||
email: str,
|
||||
amount: int,
|
||||
reason: str,
|
||||
job_id: Optional[str] = None,
|
||||
stripe_session_id: Optional[str] = None,
|
||||
admin_email: Optional[str] = None
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Add credits to a user account.
|
||||
|
||||
Returns:
|
||||
(success, new_balance, message)
|
||||
"""
|
||||
try:
|
||||
# For Stripe sessions, atomically check and mark as processed
|
||||
# Uses create() which fails if document exists - ensures idempotency
|
||||
if stripe_session_id:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
try:
|
||||
doc_ref.create({
|
||||
'stripe_session_id': stripe_session_id,
|
||||
'email': email.lower(),
|
||||
'amount': amount,
|
||||
'processed_at': datetime.utcnow()
|
||||
})
|
||||
except Exception:
|
||||
# Document already exists - this session was already processed
|
||||
logger.info(f"Stripe session {stripe_session_id} already processed (idempotent skip)")
|
||||
return False, 0, "Session already processed"
|
||||
|
||||
email = email.lower()
|
||||
user = self.get_or_create_user(email)
|
||||
|
||||
# Create transaction record
|
||||
transaction = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=amount,
|
||||
reason=reason,
|
||||
job_id=job_id,
|
||||
stripe_session_id=stripe_session_id,
|
||||
created_by=admin_email,
|
||||
)
|
||||
|
||||
# Add to transaction history (keep last N)
|
||||
transactions = user.credit_transactions[-MAX_CREDIT_TRANSACTIONS + 1:]
|
||||
transactions.append(transaction)
|
||||
|
||||
# Update user
|
||||
new_balance = user.credits + amount
|
||||
self.update_user(
|
||||
email,
|
||||
credits=new_balance,
|
||||
credit_transactions=[t.model_dump(mode='json') for t in transactions]
|
||||
)
|
||||
|
||||
logger.info(f"Added {amount} credits to {email} ({reason}). New balance: {new_balance}")
|
||||
return True, new_balance, f"Added {amount} credits"
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error adding credits to {email}")
|
||||
return False, 0, f"Failed to add credits: {e}"
|
||||
|
||||
def deduct_credit(
|
||||
self,
|
||||
email: str,
|
||||
job_id: str,
|
||||
reason: str = "job_creation"
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Deduct one credit from a user account.
|
||||
|
||||
Uses Firestore transaction to prevent race conditions.
|
||||
|
||||
Returns:
|
||||
(success, remaining_credits, message)
|
||||
"""
|
||||
try:
|
||||
email = email.lower()
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email)
|
||||
|
||||
@firestore.transactional
|
||||
def deduct_in_transaction(transaction):
|
||||
"""Atomically check and deduct credit."""
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
|
||||
if not doc.exists:
|
||||
return False, 0, "User not found"
|
||||
|
||||
user_data = doc.to_dict()
|
||||
current_credits = user_data.get('credits', 0)
|
||||
|
||||
if current_credits <= 0:
|
||||
return False, 0, "Insufficient credits"
|
||||
|
||||
# Create transaction record
|
||||
credit_txn = CreditTransaction(
|
||||
id=str(uuid.uuid4()),
|
||||
amount=-1,
|
||||
reason=reason,
|
||||
job_id=job_id,
|
||||
)
|
||||
|
||||
# Get existing transactions and add new one
|
||||
existing_transactions = user_data.get('credit_transactions', [])
|
||||
# Keep last N-1 transactions to make room for new one
|
||||
transactions = existing_transactions[-(MAX_CREDIT_TRANSACTIONS - 1):]
|
||||
transactions.append(credit_txn.model_dump(mode='json'))
|
||||
|
||||
# Calculate new values
|
||||
new_balance = current_credits - 1
|
||||
total_jobs = user_data.get('total_jobs_created', 0) + 1
|
||||
|
||||
# Update atomically within transaction
|
||||
transaction.update(doc_ref, {
|
||||
'credits': new_balance,
|
||||
'credit_transactions': transactions,
|
||||
'total_jobs_created': total_jobs,
|
||||
'updated_at': datetime.utcnow()
|
||||
})
|
||||
|
||||
return True, new_balance, f"Credit deducted. {new_balance} remaining"
|
||||
|
||||
# Execute the transaction
|
||||
fs_transaction = self.db.transaction()
|
||||
success, new_balance, message = deduct_in_transaction(fs_transaction)
|
||||
|
||||
if success:
|
||||
logger.info(f"Deducted 1 credit from {email} for job {job_id}. Remaining: {new_balance}")
|
||||
|
||||
return success, new_balance, message
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error deducting credit from {email}")
|
||||
return False, 0, f"Failed to deduct credit: {e}"
|
||||
|
||||
def refund_credit(
|
||||
self,
|
||||
email: str,
|
||||
job_id: str,
|
||||
reason: str = "job_failed"
|
||||
) -> Tuple[bool, int, str]:
|
||||
"""
|
||||
Refund one credit to a user account (e.g., after job failure).
|
||||
|
||||
Returns:
|
||||
(success, new_balance, message)
|
||||
"""
|
||||
return self.add_credits(email, 1, reason, job_id=job_id)
|
||||
|
||||
def check_credits(self, email: str) -> int:
|
||||
"""Check user's credit balance."""
|
||||
user = self.get_user(email)
|
||||
return user.credits if user else 0
|
||||
|
||||
def has_credits(self, email: str) -> bool:
|
||||
"""Check if user has at least one credit."""
|
||||
return self.check_credits(email) > 0
|
||||
|
||||
def is_stripe_session_processed(self, stripe_session_id: str) -> bool:
|
||||
"""
|
||||
Check if a Stripe session ID has already been processed.
|
||||
|
||||
Used to ensure idempotency of webhook processing - prevents
|
||||
duplicate credit additions if Stripe sends the same webhook twice.
|
||||
|
||||
Uses a dedicated collection for O(1) lookup instead of scanning all users.
|
||||
|
||||
Args:
|
||||
stripe_session_id: The Stripe checkout session ID
|
||||
|
||||
Returns:
|
||||
True if this session was already processed
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
doc = doc_ref.get()
|
||||
if doc.exists:
|
||||
logger.info(f"Stripe session {stripe_session_id} already processed")
|
||||
return True
|
||||
return False
|
||||
except Exception:
|
||||
logger.exception(f"Error checking if stripe session {stripe_session_id} was processed")
|
||||
# On error, return False to allow processing (better to risk duplicate than block)
|
||||
return False
|
||||
|
||||
def _mark_stripe_session_processed(self, stripe_session_id: str, email: str, amount: int) -> None:
|
||||
"""
|
||||
Mark a Stripe session as processed.
|
||||
|
||||
Called after successfully adding credits from a Stripe webhook.
|
||||
|
||||
Args:
|
||||
stripe_session_id: The Stripe checkout session ID
|
||||
email: User email who received the credits
|
||||
amount: Number of credits added
|
||||
"""
|
||||
try:
|
||||
doc_ref = self.db.collection(PROCESSED_STRIPE_SESSIONS_COLLECTION).document(stripe_session_id)
|
||||
doc_ref.set({
|
||||
'stripe_session_id': stripe_session_id,
|
||||
'email': email,
|
||||
'amount': amount,
|
||||
'processed_at': datetime.utcnow()
|
||||
})
|
||||
except Exception:
|
||||
# Log but don't fail - the credits were already added
|
||||
logger.exception(f"Error marking stripe session {stripe_session_id} as processed")
|
||||
|
||||
# =========================================================================
|
||||
# Admin Operations
|
||||
# =========================================================================
|
||||
|
||||
def set_user_role(self, email: str, role: UserRole, admin_email: str) -> bool:
|
||||
"""Set a user's role (admin only)."""
|
||||
try:
|
||||
user = self.get_user(email)
|
||||
if not user:
|
||||
return False
|
||||
|
||||
self.update_user(email, role=role.value)
|
||||
logger.info(f"Admin {admin_email} set role for {email} to {role.value}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error setting user role")
|
||||
return False
|
||||
|
||||
def disable_user(self, email: str, admin_email: str) -> bool:
|
||||
"""Disable a user account (admin only)."""
|
||||
try:
|
||||
self.update_user(email, is_active=False)
|
||||
self.revoke_all_sessions(email)
|
||||
logger.info(f"Admin {admin_email} disabled user {email}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error disabling user")
|
||||
return False
|
||||
|
||||
def enable_user(self, email: str, admin_email: str) -> bool:
|
||||
"""Enable a user account (admin only)."""
|
||||
try:
|
||||
self.update_user(email, is_active=True)
|
||||
logger.info(f"Admin {admin_email} enabled user {email}")
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Error enabling user")
|
||||
return False
|
||||
|
||||
def increment_jobs_completed(self, email: str) -> bool:
|
||||
"""Increment the completed jobs counter for a user using atomic increment."""
|
||||
try:
|
||||
# Check user exists first
|
||||
if not self.get_user(email):
|
||||
return False
|
||||
|
||||
# Use atomic increment to prevent race conditions
|
||||
doc_ref = self.db.collection(USERS_COLLECTION).document(email.lower())
|
||||
doc_ref.update({
|
||||
'total_jobs_completed': Increment(1),
|
||||
'updated_at': datetime.utcnow()
|
||||
})
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception(f"Error incrementing jobs completed for {email}")
|
||||
return False
|
||||
|
||||
|
||||
# Global instance
|
||||
_user_service = None
|
||||
|
||||
|
||||
def get_user_service() -> UserService:
|
||||
"""Get the global user service instance."""
|
||||
global _user_service
|
||||
if _user_service is None:
|
||||
_user_service = UserService()
|
||||
return _user_service
|
||||
@@ -20,6 +20,7 @@ import os
|
||||
import json
|
||||
from typing import Optional
|
||||
import httpx
|
||||
from google.protobuf import duration_pb2
|
||||
|
||||
from backend.config import get_settings
|
||||
from backend.services.tracing import inject_trace_context
|
||||
@@ -38,6 +39,18 @@ WORKER_QUEUES = {
|
||||
"video": "video-worker-queue",
|
||||
}
|
||||
|
||||
# Dispatch deadlines for each worker type (in seconds)
|
||||
# Cloud Tasks max is 1800s (30 min). These set how long Cloud Tasks waits
|
||||
# for the HTTP handler to respond before considering it failed.
|
||||
# Must be >= actual worker timeout + buffer for startup/upload.
|
||||
WORKER_DISPATCH_DEADLINES = {
|
||||
"audio": 1800, # 30 min - Modal separation can take 15-20 min
|
||||
"lyrics": 1500, # 25 min - TRANSCRIPTION_TIMEOUT_SECONDS is 1200s (20 min)
|
||||
"screens": 600, # 10 min - Screen generation is fast
|
||||
"render-video": 1800, # 30 min - Video encoding can be slow
|
||||
"video": 1800, # 30 min - Video encoding can be slow
|
||||
}
|
||||
|
||||
|
||||
class WorkerService:
|
||||
"""
|
||||
@@ -234,6 +247,9 @@ class WorkerService:
|
||||
# This allows worker spans to link back to the original request trace
|
||||
headers = inject_trace_context(headers)
|
||||
|
||||
# Get dispatch deadline for this worker type (how long Cloud Tasks waits for response)
|
||||
dispatch_deadline_seconds = WORKER_DISPATCH_DEADLINES.get(worker_type, 600)
|
||||
|
||||
# Build task payload
|
||||
task = {
|
||||
"http_request": {
|
||||
@@ -247,6 +263,10 @@ class WorkerService:
|
||||
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
|
||||
},
|
||||
},
|
||||
# Set dispatch_deadline - how long Cloud Tasks waits for the handler to respond
|
||||
# Default is 10 min, but audio separation can take 30+ min
|
||||
# Max is 1800s (30 min) for Cloud Tasks
|
||||
"dispatch_deadline": duration_pb2.Duration(seconds=dispatch_deadline_seconds),
|
||||
}
|
||||
|
||||
# Create task
|
||||
@@ -254,7 +274,8 @@ class WorkerService:
|
||||
# This prevents duplicate task errors on retries
|
||||
response = self.tasks_client.create_task(parent=parent, task=task)
|
||||
logger.info(
|
||||
f"[job:{job_id}] Created Cloud Task for {worker_type} worker: {response.name}"
|
||||
f"[job:{job_id}] Created Cloud Task for {worker_type} worker: {response.name} "
|
||||
f"(dispatch_deadline={dispatch_deadline_seconds}s)"
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
@@ -460,16 +460,18 @@ class TestAudioSearchServiceDownload:
|
||||
url="https://youtube.com/watch?v=abc123",
|
||||
index=0,
|
||||
)
|
||||
|
||||
|
||||
service = AudioSearchService(red_api_key=None, red_api_url=None, ops_api_key=None, ops_api_url=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.search.return_value = [mock_result]
|
||||
|
||||
# Ensure no remote client so we test the local cache path
|
||||
service._remote_client = None
|
||||
|
||||
service.search("ABBA", "Waterloo")
|
||||
|
||||
|
||||
with pytest.raises(DownloadError) as exc_info:
|
||||
service.download(99, "/tmp")
|
||||
|
||||
|
||||
assert "No cached result for index 99" in str(exc_info.value)
|
||||
|
||||
|
||||
|
||||
@@ -114,9 +114,97 @@ class TestWorkerServiceBasics:
|
||||
assert 'http' in url
|
||||
|
||||
|
||||
class TestSessionTokenValidation:
|
||||
"""Tests for session token validation in AuthService.
|
||||
|
||||
PR #120: Fixed authentication to recognize session tokens from magic link auth.
|
||||
Previously, validate_token() only checked admin tokens and auth_tokens collection.
|
||||
Session tokens (created via magic link) are stored in the sessions collection,
|
||||
so they were incorrectly rejected as "Invalid token".
|
||||
"""
|
||||
|
||||
def test_validate_token_checks_session_on_fallback(self):
|
||||
"""Test that validate_token falls back to checking session tokens."""
|
||||
mock_firestore = MagicMock()
|
||||
mock_firestore.get_token.return_value = None # Token not in auth_tokens
|
||||
|
||||
mock_user = MagicMock()
|
||||
mock_user.email = "test@example.com"
|
||||
mock_user.credits = 5
|
||||
|
||||
mock_user_service = MagicMock()
|
||||
mock_user_service.validate_session.return_value = (True, mock_user, "Valid session")
|
||||
|
||||
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
|
||||
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
|
||||
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
|
||||
# Clear the cached instance to get fresh state
|
||||
import backend.services.auth_service
|
||||
backend.services.auth_service._auth_service = None
|
||||
from backend.services.auth_service import AuthService, UserType
|
||||
|
||||
service = AuthService()
|
||||
is_valid, user_type, remaining, msg = service.validate_token("session-token-xyz")
|
||||
|
||||
assert is_valid is True
|
||||
assert user_type == UserType.STRIPE
|
||||
assert remaining == 5 # User's credits
|
||||
mock_user_service.validate_session.assert_called_once_with("session-token-xyz")
|
||||
|
||||
def test_validate_token_returns_invalid_when_session_invalid(self):
|
||||
"""Test that validate_token returns invalid when session validation fails."""
|
||||
mock_firestore = MagicMock()
|
||||
mock_firestore.get_token.return_value = None # Token not in auth_tokens
|
||||
|
||||
mock_user_service = MagicMock()
|
||||
mock_user_service.validate_session.return_value = (False, None, "Invalid session")
|
||||
|
||||
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
|
||||
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
|
||||
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
|
||||
import backend.services.auth_service
|
||||
backend.services.auth_service._auth_service = None
|
||||
from backend.services.auth_service import AuthService, UserType
|
||||
|
||||
service = AuthService()
|
||||
is_valid, user_type, remaining, msg = service.validate_token("invalid-session")
|
||||
|
||||
assert is_valid is False
|
||||
assert remaining == 0
|
||||
assert "Invalid token" in msg
|
||||
|
||||
def test_validate_token_returns_zero_credits_when_user_has_none(self):
|
||||
"""Test that validate_token handles users with 0 credits correctly."""
|
||||
mock_firestore = MagicMock()
|
||||
mock_firestore.get_token.return_value = None
|
||||
|
||||
mock_user = MagicMock()
|
||||
mock_user.email = "test@example.com"
|
||||
mock_user.credits = 0
|
||||
|
||||
mock_user_service = MagicMock()
|
||||
mock_user_service.validate_session.return_value = (True, mock_user, "Valid session")
|
||||
|
||||
with patch('backend.services.auth_service.FirestoreService', return_value=mock_firestore), \
|
||||
patch('backend.services.user_service.get_user_service', return_value=mock_user_service), \
|
||||
patch.dict('os.environ', {'ADMIN_TOKENS': 'admin-token-123'}):
|
||||
import backend.services.auth_service
|
||||
backend.services.auth_service._auth_service = None
|
||||
from backend.services.auth_service import AuthService, UserType
|
||||
|
||||
service = AuthService()
|
||||
is_valid, user_type, remaining, msg = service.validate_token("session-token")
|
||||
|
||||
# User is authenticated but has no credits
|
||||
assert is_valid is True
|
||||
assert user_type == UserType.STRIPE
|
||||
assert remaining == 0
|
||||
assert "no credits" in msg.lower()
|
||||
|
||||
|
||||
class TestJobManagerBasics:
|
||||
"""Basic tests for JobManager."""
|
||||
|
||||
|
||||
def test_job_manager_can_import(self):
|
||||
"""Test JobManager can be imported."""
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
@@ -321,31 +321,95 @@ class TestUploadFile:
|
||||
|
||||
class TestUploadFolder:
|
||||
"""Test upload_folder method."""
|
||||
|
||||
|
||||
@patch("backend.services.dropbox_service.secretmanager.SecretManagerServiceClient")
|
||||
def test_upload_folder(self, mock_sm_client_class, tmp_path):
|
||||
"""Test uploading a folder with multiple files."""
|
||||
from backend.services.dropbox_service import DropboxService
|
||||
|
||||
|
||||
mock_sm_client = Mock()
|
||||
mock_response = Mock()
|
||||
mock_response.payload.data = json.dumps({"access_token": "token"}).encode()
|
||||
mock_sm_client.access_secret_version.return_value = mock_response
|
||||
mock_sm_client_class.return_value = mock_sm_client
|
||||
|
||||
# Create test folder with files
|
||||
|
||||
# Create test folder with files (no subdirs for this test)
|
||||
(tmp_path / "file1.txt").write_text("content1")
|
||||
(tmp_path / "file2.txt").write_text("content2")
|
||||
(tmp_path / "subdir").mkdir() # Should be skipped
|
||||
|
||||
|
||||
service = DropboxService()
|
||||
|
||||
|
||||
with patch.object(service, "upload_file") as mock_upload:
|
||||
service.upload_folder(str(tmp_path), "/Uploads/folder")
|
||||
|
||||
# Should upload 2 files (not subdir)
|
||||
|
||||
# Should upload 2 files
|
||||
assert mock_upload.call_count == 2
|
||||
|
||||
@patch("backend.services.dropbox_service.secretmanager.SecretManagerServiceClient")
|
||||
def test_upload_folder_recursive(self, mock_sm_client_class, tmp_path):
|
||||
"""Test uploading a folder recursively includes subdirectories."""
|
||||
from backend.services.dropbox_service import DropboxService
|
||||
|
||||
mock_sm_client = Mock()
|
||||
mock_response = Mock()
|
||||
mock_response.payload.data = json.dumps({"access_token": "token"}).encode()
|
||||
mock_sm_client.access_secret_version.return_value = mock_response
|
||||
mock_sm_client_class.return_value = mock_sm_client
|
||||
|
||||
# Create test folder with files and subdirectories
|
||||
(tmp_path / "root_file.txt").write_text("root content")
|
||||
(tmp_path / "stems").mkdir()
|
||||
(tmp_path / "stems" / "vocals.flac").write_text("vocals")
|
||||
(tmp_path / "stems" / "instrumental.flac").write_text("instrumental")
|
||||
(tmp_path / "lyrics").mkdir()
|
||||
(tmp_path / "lyrics" / "song.lrc").write_text("lyrics")
|
||||
|
||||
service = DropboxService()
|
||||
|
||||
uploaded_files = []
|
||||
def capture_upload(local_path, remote_path):
|
||||
uploaded_files.append(remote_path)
|
||||
|
||||
with patch.object(service, "upload_file", side_effect=capture_upload) as mock_upload:
|
||||
service.upload_folder(str(tmp_path), "/Uploads/folder")
|
||||
|
||||
# Should upload 4 files (1 root + 2 stems + 1 lyrics)
|
||||
assert mock_upload.call_count == 4
|
||||
|
||||
# Check that subdirectory structure is preserved
|
||||
assert "/Uploads/folder/root_file.txt" in uploaded_files
|
||||
assert "/Uploads/folder/stems/vocals.flac" in uploaded_files
|
||||
assert "/Uploads/folder/stems/instrumental.flac" in uploaded_files
|
||||
assert "/Uploads/folder/lyrics/song.lrc" in uploaded_files
|
||||
|
||||
@patch("backend.services.dropbox_service.secretmanager.SecretManagerServiceClient")
|
||||
def test_upload_folder_deeply_nested(self, mock_sm_client_class, tmp_path):
|
||||
"""Test uploading deeply nested folder structure."""
|
||||
from backend.services.dropbox_service import DropboxService
|
||||
|
||||
mock_sm_client = Mock()
|
||||
mock_response = Mock()
|
||||
mock_response.payload.data = json.dumps({"access_token": "token"}).encode()
|
||||
mock_sm_client.access_secret_version.return_value = mock_response
|
||||
mock_sm_client_class.return_value = mock_sm_client
|
||||
|
||||
# Create deeply nested structure
|
||||
(tmp_path / "level1").mkdir()
|
||||
(tmp_path / "level1" / "level2").mkdir()
|
||||
(tmp_path / "level1" / "level2" / "deep_file.txt").write_text("deep")
|
||||
|
||||
service = DropboxService()
|
||||
|
||||
uploaded_files = []
|
||||
def capture_upload(local_path, remote_path):
|
||||
uploaded_files.append(remote_path)
|
||||
|
||||
with patch.object(service, "upload_file", side_effect=capture_upload):
|
||||
service.upload_folder(str(tmp_path), "/Uploads/folder")
|
||||
|
||||
# Check deeply nested file is uploaded with correct path
|
||||
assert "/Uploads/folder/level1/level2/deep_file.txt" in uploaded_files
|
||||
|
||||
|
||||
class TestCreateSharedLink:
|
||||
"""Test create_shared_link method."""
|
||||
|
||||
@@ -1606,5 +1606,133 @@ class TestIsUrlFunction:
|
||||
assert is_url("") is False
|
||||
|
||||
|
||||
class TestUploadEndpointThemeSupport:
|
||||
"""Test that /jobs/upload endpoint supports theme configuration.
|
||||
|
||||
CRITICAL: These tests verify that the /api/jobs/upload endpoint correctly handles
|
||||
theme_id and color_overrides parameters, ensuring preview videos have themed
|
||||
backgrounds instead of black backgrounds.
|
||||
|
||||
This addresses a bug where:
|
||||
- The frontend sends theme_id and color_overrides when uploading files
|
||||
- But the backend /api/jobs/upload endpoint was ignoring these parameters
|
||||
- Result: Jobs created via file upload had black backgrounds instead of themed ones
|
||||
"""
|
||||
|
||||
def test_upload_endpoint_accepts_theme_id_parameter(self):
|
||||
"""Verify the upload endpoint has theme_id as a form parameter.
|
||||
|
||||
CRITICAL: The frontend sends theme_id when uploading files with a theme.
|
||||
If this parameter is missing from the endpoint, the theme is silently ignored
|
||||
and preview videos will have black backgrounds instead of themed ones.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_theme_id_param = 'theme_id: Optional[str] = Form(' in source_code
|
||||
|
||||
assert has_theme_id_param, (
|
||||
"file_upload.py /jobs/upload endpoint does not have theme_id as a Form parameter. "
|
||||
"The frontend sends theme_id when uploading files with a theme, but the backend "
|
||||
"ignores it. Add: theme_id: Optional[str] = Form(None, description='Theme ID...')"
|
||||
)
|
||||
|
||||
def test_upload_endpoint_accepts_color_overrides_parameter(self):
|
||||
"""Verify the upload endpoint has color_overrides as a form parameter."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_color_overrides_param = 'color_overrides: Optional[str] = Form(' in source_code
|
||||
|
||||
assert has_color_overrides_param, (
|
||||
"file_upload.py /jobs/upload endpoint does not have color_overrides as a Form parameter. "
|
||||
"The frontend sends color_overrides when customizing theme colors."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_calls_prepare_theme_for_job(self):
|
||||
"""Verify the upload endpoint calls _prepare_theme_for_job when theme_id is set.
|
||||
|
||||
CRITICAL: When a job is created via the upload endpoint with a theme_id
|
||||
(and no custom style files), the code must call _prepare_theme_for_job() to set:
|
||||
1. style_params_gcs_path (pointing to the copied style_params.json)
|
||||
2. style_assets (populated with asset mappings)
|
||||
|
||||
Without this, LyricsTranscriber won't have access to the theme's styles
|
||||
and preview videos will have black backgrounds instead of themed ones.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
# The function should be called in upload_and_create_job
|
||||
has_theme_prep_call = '_prepare_theme_for_job(' in source_code
|
||||
|
||||
assert has_theme_prep_call, (
|
||||
"file_upload.py does not call _prepare_theme_for_job(). "
|
||||
"When theme_id is provided to /jobs/upload without custom style files, "
|
||||
"the endpoint MUST call _prepare_theme_for_job() to copy the theme's "
|
||||
"style_params.json to the job folder."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_uses_resolve_cdg_txt_defaults(self):
|
||||
"""Verify the upload endpoint uses _resolve_cdg_txt_defaults for theme-based defaults."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_resolve_call = '_resolve_cdg_txt_defaults(' in source_code
|
||||
|
||||
assert has_resolve_call, (
|
||||
"file_upload.py does not call _resolve_cdg_txt_defaults(). "
|
||||
"When theme_id is set, enable_cdg and enable_txt should default to True."
|
||||
)
|
||||
|
||||
def test_upload_endpoint_has_optional_cdg_txt_params(self):
|
||||
"""Verify enable_cdg and enable_txt are Optional[bool] to support theme defaults.
|
||||
|
||||
CRITICAL: If enable_cdg/enable_txt are bool instead of Optional[bool],
|
||||
they will default to False and override the theme-based defaults.
|
||||
"""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_optional_cdg = 'enable_cdg: Optional[bool] = Form(' in source_code
|
||||
has_optional_txt = 'enable_txt: Optional[bool] = Form(' in source_code
|
||||
|
||||
assert has_optional_cdg, (
|
||||
"file_upload.py has enable_cdg as bool instead of Optional[bool]. "
|
||||
"This prevents theme-based defaults from working."
|
||||
)
|
||||
assert has_optional_txt, (
|
||||
"file_upload.py has enable_txt as bool instead of Optional[bool]. "
|
||||
"This prevents theme-based defaults from working."
|
||||
)
|
||||
|
||||
def test_job_create_includes_theme_id_and_color_overrides(self):
|
||||
"""Verify JobCreate is called with theme_id and color_overrides."""
|
||||
from backend.api.routes import file_upload as file_upload_module
|
||||
|
||||
with open(file_upload_module.__file__, 'r') as f:
|
||||
source_code = f.read()
|
||||
|
||||
has_theme_id_in_job_create = 'theme_id=theme_id,' in source_code
|
||||
has_color_overrides_in_job_create = 'color_overrides=parsed_color_overrides' in source_code
|
||||
|
||||
assert has_theme_id_in_job_create, (
|
||||
"file_upload.py does not pass theme_id to JobCreate."
|
||||
)
|
||||
assert has_color_overrides_in_job_create, (
|
||||
"file_upload.py does not pass color_overrides to JobCreate."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
@@ -0,0 +1,583 @@
|
||||
"""
|
||||
Regression tests for job creation issues discovered 2024-12-31.
|
||||
|
||||
These tests ensure:
|
||||
1. user_email is properly extracted from AuthResult and set on jobs
|
||||
2. URL jobs trigger workers sequentially (audio first, then lyrics)
|
||||
3. Audio search downloads work without relying on in-memory cache
|
||||
4. Transcription has proper timeout handling
|
||||
|
||||
See docs/archive/2024-12-31-job-failure-investigation.md for details.
|
||||
"""
|
||||
import asyncio
|
||||
import pytest
|
||||
from unittest.mock import Mock, AsyncMock, patch, MagicMock
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from backend.services.auth_service import AuthResult, UserType
|
||||
|
||||
|
||||
class TestUserEmailExtraction:
|
||||
"""
|
||||
Issue 1: user_email not being set on jobs.
|
||||
|
||||
Root cause: Job creation endpoints used require_auth but didn't extract
|
||||
auth_result.user_email and set it on the job.
|
||||
|
||||
These tests verify that authenticated user's email is properly set on jobs.
|
||||
"""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_auth_result_with_email(self):
|
||||
"""AuthResult for a logged-in user with email."""
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.UNLIMITED, # Session-based users typically have UNLIMITED
|
||||
remaining_uses=-1,
|
||||
message="Valid session",
|
||||
user_email="testuser@example.com",
|
||||
is_admin=False,
|
||||
)
|
||||
|
||||
@pytest.fixture
|
||||
def mock_auth_result_admin_token(self):
|
||||
"""AuthResult for admin token without email."""
|
||||
return AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.ADMIN,
|
||||
remaining_uses=-1,
|
||||
message="Valid admin token",
|
||||
user_email=None,
|
||||
is_admin=True,
|
||||
)
|
||||
|
||||
def test_auth_result_has_user_email_field(self):
|
||||
"""AuthResult must have user_email field for job association."""
|
||||
result = AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.UNLIMITED,
|
||||
remaining_uses=-1,
|
||||
message="test",
|
||||
user_email="user@example.com",
|
||||
)
|
||||
assert hasattr(result, 'user_email')
|
||||
assert result.user_email == "user@example.com"
|
||||
|
||||
def test_auth_result_user_email_can_be_none(self):
|
||||
"""AuthResult user_email can be None for token-based auth."""
|
||||
result = AuthResult(
|
||||
is_valid=True,
|
||||
user_type=UserType.ADMIN,
|
||||
remaining_uses=-1,
|
||||
message="test",
|
||||
user_email=None,
|
||||
)
|
||||
assert result.user_email is None
|
||||
|
||||
def test_upload_endpoint_extracts_user_email_from_auth(self):
|
||||
"""
|
||||
/jobs/upload endpoint must set user_email from authenticated user.
|
||||
|
||||
This verifies the code pattern by inspecting the source.
|
||||
Integration tests verify the full behavior.
|
||||
"""
|
||||
from backend.api.routes import file_upload
|
||||
import inspect
|
||||
|
||||
# Get the source code of upload_and_create_job
|
||||
source = inspect.getsource(file_upload.upload_and_create_job)
|
||||
|
||||
# Verify the endpoint extracts user_email from auth_result
|
||||
assert 'auth_result.user_email' in source, \
|
||||
"upload_and_create_job must extract user_email from auth_result"
|
||||
assert 'effective_user_email' in source, \
|
||||
"upload_and_create_job must use effective_user_email pattern"
|
||||
assert 'user_email=effective_user_email' in source or 'user_email=' in source, \
|
||||
"upload_and_create_job must pass user_email to JobCreate"
|
||||
|
||||
def test_all_job_creation_endpoints_extract_user_email(self):
|
||||
"""
|
||||
All job creation endpoints must extract user_email from AuthResult.
|
||||
|
||||
This is a regression guard for Issue 1 (user_email not being set).
|
||||
"""
|
||||
from backend.api.routes import file_upload, audio_search
|
||||
import inspect
|
||||
|
||||
endpoints_to_check = [
|
||||
(file_upload, 'upload_and_create_job'),
|
||||
(file_upload, 'create_job_from_url'),
|
||||
(audio_search, 'search_audio'),
|
||||
]
|
||||
|
||||
for module, func_name in endpoints_to_check:
|
||||
func = getattr(module, func_name)
|
||||
source = inspect.getsource(func)
|
||||
|
||||
assert 'auth_result.user_email' in source, \
|
||||
f"{func_name} must extract user_email from auth_result"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_create_from_url_endpoint_sets_user_email(self, mock_auth_result_with_email):
|
||||
"""
|
||||
/jobs/create-from-url endpoint must set user_email from authenticated user.
|
||||
"""
|
||||
from backend.api.routes.file_upload import create_job_from_url, CreateJobFromUrlRequest
|
||||
|
||||
mock_request = Mock()
|
||||
mock_request.headers = {}
|
||||
mock_request.client = Mock(host="127.0.0.1")
|
||||
mock_request.url = Mock(path="/api/jobs/create-from-url")
|
||||
|
||||
mock_background_tasks = Mock()
|
||||
body = CreateJobFromUrlRequest(
|
||||
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
|
||||
artist="Rick Astley",
|
||||
title="Never Gonna Give You Up",
|
||||
)
|
||||
|
||||
with patch('backend.api.routes.file_upload.job_manager') as mock_jm, \
|
||||
patch('backend.api.routes.file_upload.worker_service') as mock_worker, \
|
||||
patch('backend.api.routes.file_upload.get_credential_manager') as mock_cred:
|
||||
|
||||
mock_job = Mock()
|
||||
mock_job.job_id = "test-job-456"
|
||||
mock_jm.create_job.return_value = mock_job
|
||||
mock_cred.return_value.check_youtube_credentials.return_value = Mock(status=Mock(value="valid"))
|
||||
|
||||
await create_job_from_url(
|
||||
request=mock_request,
|
||||
background_tasks=mock_background_tasks,
|
||||
body=body,
|
||||
auth_result=mock_auth_result_with_email,
|
||||
)
|
||||
|
||||
mock_jm.create_job.assert_called_once()
|
||||
job_create_arg = mock_jm.create_job.call_args[0][0]
|
||||
assert job_create_arg.user_email == "testuser@example.com", \
|
||||
"create-from-url must set user_email from AuthResult"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_audio_search_endpoint_sets_user_email(self, mock_auth_result_with_email):
|
||||
"""
|
||||
/audio-search/search endpoint must set user_email from authenticated user.
|
||||
"""
|
||||
from backend.api.routes.audio_search import search_audio, AudioSearchRequest
|
||||
|
||||
mock_request = Mock()
|
||||
mock_request.headers = {}
|
||||
mock_request.client = Mock(host="127.0.0.1")
|
||||
mock_request.url = Mock(path="/api/audio-search/search")
|
||||
|
||||
mock_background_tasks = Mock()
|
||||
body = AudioSearchRequest(
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
)
|
||||
|
||||
with patch('backend.api.routes.audio_search.job_manager') as mock_jm, \
|
||||
patch('backend.api.routes.audio_search.get_audio_search_service') as mock_search, \
|
||||
patch('backend.api.routes.audio_search.get_credential_manager') as mock_cred:
|
||||
|
||||
mock_job = Mock()
|
||||
mock_job.job_id = "test-job-789"
|
||||
mock_job.state_data = {}
|
||||
mock_jm.create_job.return_value = mock_job
|
||||
mock_jm.get_job.return_value = mock_job
|
||||
mock_cred.return_value.check_youtube_credentials.return_value = Mock(status=Mock(value="valid"))
|
||||
mock_search.return_value.search.return_value = []
|
||||
|
||||
await search_audio(
|
||||
request=mock_request,
|
||||
background_tasks=mock_background_tasks,
|
||||
body=body,
|
||||
auth_result=mock_auth_result_with_email,
|
||||
)
|
||||
|
||||
mock_jm.create_job.assert_called_once()
|
||||
job_create_arg = mock_jm.create_job.call_args[0][0]
|
||||
assert job_create_arg.user_email == "testuser@example.com", \
|
||||
"audio-search must set user_email from AuthResult"
|
||||
|
||||
def test_effective_user_email_prefers_auth_over_form(self):
|
||||
"""
|
||||
When both auth_result.user_email and form user_email are provided,
|
||||
the authenticated user's email should take precedence.
|
||||
"""
|
||||
auth_email = "authenticated@example.com"
|
||||
form_email = "form@example.com"
|
||||
|
||||
# This is the logic used in the endpoints
|
||||
effective_user_email = auth_email or form_email
|
||||
|
||||
assert effective_user_email == auth_email, \
|
||||
"Authenticated user's email should take precedence over form parameter"
|
||||
|
||||
def test_effective_user_email_falls_back_to_form(self):
|
||||
"""
|
||||
When auth_result.user_email is None (e.g., admin token),
|
||||
form user_email should be used as fallback.
|
||||
"""
|
||||
auth_email = None
|
||||
form_email = "form@example.com"
|
||||
|
||||
effective_user_email = auth_email or form_email
|
||||
|
||||
assert effective_user_email == form_email, \
|
||||
"Form parameter should be used when auth has no email"
|
||||
|
||||
|
||||
class TestUrlJobWorkerSequencing:
|
||||
"""
|
||||
Issue 2: YouTube URL download race condition.
|
||||
|
||||
Root cause: Both audio and lyrics workers were triggered in parallel.
|
||||
For URL jobs, lyrics worker would timeout waiting for audio to download.
|
||||
|
||||
These tests verify that URL jobs trigger workers sequentially.
|
||||
"""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_url_job_triggers_only_audio_worker_initially(self):
|
||||
"""
|
||||
create-from-url must only trigger audio worker, not lyrics worker.
|
||||
The audio worker will trigger lyrics after download completes.
|
||||
"""
|
||||
from backend.api.routes.file_upload import _trigger_audio_worker_only
|
||||
|
||||
with patch('backend.api.routes.file_upload.worker_service') as mock_ws:
|
||||
mock_ws.trigger_audio_worker = AsyncMock()
|
||||
mock_ws.trigger_lyrics_worker = AsyncMock()
|
||||
|
||||
await _trigger_audio_worker_only("test-job-id")
|
||||
|
||||
mock_ws.trigger_audio_worker.assert_called_once_with("test-job-id")
|
||||
mock_ws.trigger_lyrics_worker.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parallel_worker_triggers_both_workers(self):
|
||||
"""
|
||||
_trigger_workers_parallel should trigger both workers (for uploaded files).
|
||||
"""
|
||||
from backend.api.routes.file_upload import _trigger_workers_parallel
|
||||
|
||||
with patch('backend.api.routes.file_upload.worker_service') as mock_ws:
|
||||
mock_ws.trigger_audio_worker = AsyncMock()
|
||||
mock_ws.trigger_lyrics_worker = AsyncMock()
|
||||
|
||||
await _trigger_workers_parallel("test-job-id")
|
||||
|
||||
mock_ws.trigger_audio_worker.assert_called_once_with("test-job-id")
|
||||
mock_ws.trigger_lyrics_worker.assert_called_once_with("test-job-id")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_audio_worker_triggers_lyrics_after_url_download(self):
|
||||
"""
|
||||
Audio worker must trigger lyrics worker after successful URL download.
|
||||
"""
|
||||
from backend.workers.audio_worker import _trigger_lyrics_worker_after_url_download
|
||||
|
||||
# Mock at the source module where it's imported from
|
||||
with patch('backend.services.worker_service.get_worker_service') as mock_get_ws:
|
||||
mock_ws = Mock()
|
||||
mock_ws.trigger_lyrics_worker = AsyncMock()
|
||||
mock_get_ws.return_value = mock_ws
|
||||
|
||||
await _trigger_lyrics_worker_after_url_download("test-job-id")
|
||||
|
||||
mock_ws.trigger_lyrics_worker.assert_called_once_with("test-job-id")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_audio_worker_lyrics_trigger_handles_errors_gracefully(self):
|
||||
"""
|
||||
If lyrics worker trigger fails, audio processing should continue.
|
||||
"""
|
||||
from backend.workers.audio_worker import _trigger_lyrics_worker_after_url_download
|
||||
|
||||
# Mock at the source module where it's imported from
|
||||
with patch('backend.services.worker_service.get_worker_service') as mock_get_ws:
|
||||
mock_ws = Mock()
|
||||
mock_ws.trigger_lyrics_worker = AsyncMock(side_effect=Exception("Network error"))
|
||||
mock_get_ws.return_value = mock_ws
|
||||
|
||||
# Should not raise exception
|
||||
await _trigger_lyrics_worker_after_url_download("test-job-id")
|
||||
|
||||
|
||||
class TestAudioSearchCacheIndependence:
|
||||
"""
|
||||
Issue 3: Audio search cache not persisting across Cloud Run instances.
|
||||
|
||||
Root cause: AudioSearchService used in-memory cache that doesn't persist
|
||||
across horizontally scaled instances.
|
||||
|
||||
These tests verify downloads work using state_data, not in-memory cache.
|
||||
"""
|
||||
|
||||
def test_search_results_stored_in_job_state_data(self):
|
||||
"""
|
||||
Search results must be stored in job.state_data for persistence.
|
||||
This tests the Job model's ability to store search results.
|
||||
"""
|
||||
from backend.models.job import Job, JobStatus
|
||||
|
||||
job = Job(
|
||||
job_id="test-123",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
state_data={
|
||||
'audio_search_results': [
|
||||
{
|
||||
'provider': 'YouTube',
|
||||
'title': 'Test Song',
|
||||
'artist': 'Test Artist',
|
||||
'url': 'https://youtube.com/watch?v=abc123',
|
||||
'source_id': 'abc123',
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
assert 'audio_search_results' in job.state_data
|
||||
assert len(job.state_data['audio_search_results']) == 1
|
||||
assert job.state_data['audio_search_results'][0]['url'] == 'https://youtube.com/watch?v=abc123'
|
||||
|
||||
def test_audio_search_endpoint_stores_results_in_state_data(self):
|
||||
"""
|
||||
Verify that the search_audio code path stores results in job.state_data.
|
||||
|
||||
This tests that the audio_search route has the correct pattern for
|
||||
persisting search results. The actual integration is tested elsewhere.
|
||||
"""
|
||||
# Verify the import and module structure exists
|
||||
from backend.api.routes import audio_search
|
||||
import inspect
|
||||
|
||||
# Get the source code of search_audio
|
||||
source = inspect.getsource(audio_search.search_audio)
|
||||
|
||||
# Verify the endpoint stores results in state_data
|
||||
# The code uses job_manager.update_job with state_data dict
|
||||
assert 'audio_search_results' in source, \
|
||||
"search_audio must store results under 'audio_search_results' key"
|
||||
assert 'state_data' in source, \
|
||||
"search_audio must use state_data for persistence"
|
||||
|
||||
def test_download_code_has_youtube_direct_download_branch(self):
|
||||
"""
|
||||
Verify that _download_and_start_processing has the YouTube direct download path.
|
||||
|
||||
This checks that the code path exists for downloading YouTube audio directly
|
||||
using the URL from state_data, avoiding the in-memory cache dependency.
|
||||
"""
|
||||
from backend.api.routes import audio_search
|
||||
import inspect
|
||||
|
||||
# Get the source of _download_and_start_processing
|
||||
source = inspect.getsource(audio_search._download_and_start_processing)
|
||||
|
||||
# Verify the YouTube direct download branch exists
|
||||
assert "source_name == 'YouTube'" in source, \
|
||||
"_download_and_start_processing must check for YouTube source"
|
||||
assert 'download_url' in source, \
|
||||
"_download_and_start_processing must use download_url from state_data"
|
||||
assert 'download_from_url' in source, \
|
||||
"_download_and_start_processing must call download_from_url for YouTube"
|
||||
|
||||
def test_youtube_download_branch_avoids_cache_dependency(self):
|
||||
"""
|
||||
Verify that the YouTube download path uses URL from state_data,
|
||||
not the in-memory cache that doesn't persist across Cloud Run instances.
|
||||
"""
|
||||
from backend.api.routes import audio_search
|
||||
import inspect
|
||||
|
||||
source = inspect.getsource(audio_search._download_and_start_processing)
|
||||
|
||||
# The fix specifically uses selected.get('url') which comes from state_data
|
||||
# rather than audio_search_service cache
|
||||
assert "selected.get('url')" in source or "download_url" in source, \
|
||||
"YouTube download must use URL from selected result (state_data)"
|
||||
|
||||
# Verify it doesn't rely on cache for YouTube
|
||||
# The branch is: elif source_name == 'YouTube' and download_url:
|
||||
assert "YouTube" in source and "download_url" in source, \
|
||||
"YouTube branch must check for download_url availability"
|
||||
|
||||
|
||||
class TestTranscriptionTimeout:
|
||||
"""
|
||||
Issue 4: Jobs stuck in downloading state.
|
||||
|
||||
Root cause: Lyrics worker's AudioShake transcription could hang forever
|
||||
without proper timeout, leaving jobs stuck.
|
||||
|
||||
These tests verify timeout handling for transcription.
|
||||
"""
|
||||
|
||||
def test_transcription_timeout_constant_exists(self):
|
||||
"""
|
||||
Verify transcription timeout constant is defined in lyrics_worker.
|
||||
"""
|
||||
from backend.workers.lyrics_worker import TRANSCRIPTION_TIMEOUT_SECONDS
|
||||
|
||||
assert TRANSCRIPTION_TIMEOUT_SECONDS > 0
|
||||
assert TRANSCRIPTION_TIMEOUT_SECONDS >= 300, "Timeout should be at least 5 minutes for long songs"
|
||||
assert TRANSCRIPTION_TIMEOUT_SECONDS <= 1500, "Timeout shouldn't be more than 25 minutes"
|
||||
|
||||
def test_transcription_timeout_value(self):
|
||||
"""
|
||||
Verify the specific timeout value (20 minutes = 1200 seconds).
|
||||
|
||||
This accounts for:
|
||||
- Cloud Run cold start / worker initialization (1-5 min)
|
||||
- AudioShake transcription (1-2 min)
|
||||
- spaCy model loading for correction (2-3 min on cold start)
|
||||
- Agentic AI correction (1-3 min)
|
||||
"""
|
||||
from backend.workers.lyrics_worker import TRANSCRIPTION_TIMEOUT_SECONDS
|
||||
|
||||
assert TRANSCRIPTION_TIMEOUT_SECONDS == 1200, "Transcription timeout should be 20 minutes (1200 seconds)"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_asyncio_wait_for_raises_timeout_error(self):
|
||||
"""
|
||||
Verify asyncio.wait_for properly raises TimeoutError.
|
||||
This is a sanity check for the timeout mechanism we use.
|
||||
"""
|
||||
async def slow_operation():
|
||||
await asyncio.sleep(10)
|
||||
return "completed"
|
||||
|
||||
with pytest.raises(asyncio.TimeoutError):
|
||||
await asyncio.wait_for(slow_operation(), timeout=0.1)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_lyrics_worker_timeout_converts_to_descriptive_error(self):
|
||||
"""
|
||||
Verify that timeout is converted to a descriptive exception message.
|
||||
Tests the actual error conversion logic pattern used in lyrics_worker.
|
||||
"""
|
||||
from backend.workers.lyrics_worker import TRANSCRIPTION_TIMEOUT_SECONDS
|
||||
|
||||
# This simulates what happens in lyrics_worker when timeout occurs
|
||||
error_message = None
|
||||
try:
|
||||
# Simulate the timeout handling pattern from lyrics_worker
|
||||
raise asyncio.TimeoutError()
|
||||
except asyncio.TimeoutError:
|
||||
error_message = f"Transcription timed out after {TRANSCRIPTION_TIMEOUT_SECONDS} seconds"
|
||||
|
||||
assert error_message is not None
|
||||
assert "timed out" in error_message.lower()
|
||||
assert "1200" in error_message
|
||||
|
||||
def test_lyrics_worker_exception_marks_job_failed(self):
|
||||
"""
|
||||
Any exception in lyrics worker should mark job as failed.
|
||||
"""
|
||||
from backend.models.job import JobStatus
|
||||
|
||||
# Simulate the exception handling in lyrics_worker
|
||||
job_status = JobStatus.DOWNLOADING
|
||||
error_occurred = True
|
||||
|
||||
if error_occurred:
|
||||
# This is what happens in the except block
|
||||
job_status = JobStatus.FAILED
|
||||
error_message = "Lyrics transcription failed: Transcription timed out after 1200 seconds"
|
||||
|
||||
assert job_status == JobStatus.FAILED
|
||||
assert "timed out" in error_message.lower()
|
||||
|
||||
|
||||
class TestJobOwnershipFiltering:
|
||||
"""
|
||||
Additional tests for job ownership and filtering.
|
||||
|
||||
These verify that jobs are properly associated with users
|
||||
and can be filtered by user_email.
|
||||
"""
|
||||
|
||||
def test_job_model_has_user_email_field(self):
|
||||
"""Job model must have user_email field."""
|
||||
from backend.models.job import Job, JobStatus
|
||||
|
||||
job = Job(
|
||||
job_id="test-123",
|
||||
artist="Test",
|
||||
title="Test",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
user_email="user@example.com",
|
||||
)
|
||||
|
||||
assert hasattr(job, 'user_email')
|
||||
assert job.user_email == "user@example.com"
|
||||
|
||||
def test_job_create_model_has_user_email_field(self):
|
||||
"""JobCreate model must have user_email field."""
|
||||
from backend.models.job import JobCreate
|
||||
|
||||
job_create = JobCreate(
|
||||
artist="Test",
|
||||
title="Test",
|
||||
user_email="user@example.com",
|
||||
)
|
||||
|
||||
assert hasattr(job_create, 'user_email')
|
||||
assert job_create.user_email == "user@example.com"
|
||||
|
||||
def test_job_create_user_email_is_optional(self):
|
||||
"""JobCreate user_email should be optional for backward compatibility."""
|
||||
from backend.models.job import JobCreate
|
||||
|
||||
job_create = JobCreate(
|
||||
artist="Test",
|
||||
title="Test",
|
||||
)
|
||||
|
||||
assert job_create.user_email is None
|
||||
|
||||
def test_jobs_can_be_filtered_by_user_email(self):
|
||||
"""Jobs should be filterable by user_email."""
|
||||
from backend.models.job import Job, JobStatus
|
||||
|
||||
jobs = [
|
||||
Job(
|
||||
job_id="job-1",
|
||||
artist="Test",
|
||||
title="Test 1",
|
||||
status=JobStatus.COMPLETE,
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
user_email="user1@example.com",
|
||||
),
|
||||
Job(
|
||||
job_id="job-2",
|
||||
artist="Test",
|
||||
title="Test 2",
|
||||
status=JobStatus.COMPLETE,
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
user_email="user2@example.com",
|
||||
),
|
||||
Job(
|
||||
job_id="job-3",
|
||||
artist="Test",
|
||||
title="Test 3",
|
||||
status=JobStatus.COMPLETE,
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
user_email="user1@example.com",
|
||||
),
|
||||
]
|
||||
|
||||
user1_jobs = [j for j in jobs if j.user_email == "user1@example.com"]
|
||||
|
||||
assert len(user1_jobs) == 2
|
||||
assert all(j.user_email == "user1@example.com" for j in user1_jobs)
|
||||
@@ -54,10 +54,16 @@ def client(mock_job_manager, mock_worker_service):
|
||||
"""Create TestClient with mocked dependencies."""
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
|
||||
# Create a JobManager class that returns our mock instance
|
||||
def mock_job_manager_factory(*args, **kwargs):
|
||||
return mock_job_manager
|
||||
|
||||
# Patch at the module level where jobs.py imports them
|
||||
# Also patch JobManager class used in dependencies.py for auth checks
|
||||
with patch('backend.api.routes.jobs.job_manager', mock_job_manager), \
|
||||
patch('backend.api.routes.jobs.worker_service', mock_worker_service), \
|
||||
patch('backend.services.job_manager.JobManager', mock_job_manager_factory), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
@@ -87,9 +93,13 @@ class TestGetJob:
|
||||
mock_job_manager.get_job.return_value = None
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
|
||||
def mock_job_manager_factory(*args, **kwargs):
|
||||
return mock_job_manager
|
||||
|
||||
with patch('backend.api.routes.jobs.job_manager', mock_job_manager), \
|
||||
patch('backend.api.routes.jobs.worker_service', mock_worker_service), \
|
||||
patch('backend.services.job_manager.JobManager', mock_job_manager_factory), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
@@ -173,12 +183,17 @@ class TestDeleteJob:
|
||||
def test_delete_nonexistent_job(self, mock_worker_service, auth_headers):
|
||||
"""Test deleting non-existent job."""
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = None # Job doesn't exist
|
||||
mock_job_manager.delete_job.return_value = False
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
|
||||
def mock_job_manager_factory(*args, **kwargs):
|
||||
return mock_job_manager
|
||||
|
||||
with patch('backend.api.routes.jobs.job_manager', mock_job_manager), \
|
||||
patch('backend.api.routes.jobs.worker_service', mock_worker_service), \
|
||||
patch('backend.services.job_manager.JobManager', mock_job_manager_factory), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
|
||||
@@ -970,10 +970,147 @@ class TestBackingVocalsAnalysis:
|
||||
def test_analysis_service_can_be_instantiated(self):
|
||||
"""Test that AudioAnalysisService can be instantiated."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
|
||||
mock_storage = MagicMock()
|
||||
service = AudioAnalysisService(storage_service=mock_storage)
|
||||
|
||||
|
||||
assert service is not None
|
||||
assert service.storage_service == mock_storage
|
||||
|
||||
|
||||
class TestModelNamesStorage:
|
||||
"""Tests for model names storage in audio_worker.
|
||||
|
||||
These tests verify that model names are stored in job state_data
|
||||
for use by video_worker in distribution directory preparation.
|
||||
"""
|
||||
|
||||
def test_effective_model_names_defaults(self):
|
||||
"""Test that default model names are used when not specified on job."""
|
||||
from backend.workers.audio_worker import (
|
||||
DEFAULT_CLEAN_MODEL,
|
||||
DEFAULT_BACKING_MODELS,
|
||||
DEFAULT_OTHER_MODELS,
|
||||
)
|
||||
|
||||
# Simulate the logic from process_audio_separation
|
||||
job_clean_model = None
|
||||
job_backing_models = None
|
||||
job_other_models = None
|
||||
|
||||
effective_model_names = {
|
||||
'clean_instrumental_model': job_clean_model or DEFAULT_CLEAN_MODEL,
|
||||
'backing_vocals_models': job_backing_models or DEFAULT_BACKING_MODELS,
|
||||
'other_stems_models': job_other_models or DEFAULT_OTHER_MODELS,
|
||||
}
|
||||
|
||||
assert effective_model_names['clean_instrumental_model'] == DEFAULT_CLEAN_MODEL
|
||||
assert effective_model_names['backing_vocals_models'] == DEFAULT_BACKING_MODELS
|
||||
assert effective_model_names['other_stems_models'] == DEFAULT_OTHER_MODELS
|
||||
|
||||
def test_effective_model_names_custom(self):
|
||||
"""Test that custom model names override defaults."""
|
||||
from backend.workers.audio_worker import (
|
||||
DEFAULT_CLEAN_MODEL,
|
||||
DEFAULT_BACKING_MODELS,
|
||||
DEFAULT_OTHER_MODELS,
|
||||
)
|
||||
|
||||
custom_clean = "custom_clean_model.ckpt"
|
||||
custom_backing = ["custom_backing.ckpt"]
|
||||
custom_other = ["custom_demucs.yaml"]
|
||||
|
||||
# Simulate the logic from process_audio_separation
|
||||
effective_model_names = {
|
||||
'clean_instrumental_model': custom_clean or DEFAULT_CLEAN_MODEL,
|
||||
'backing_vocals_models': custom_backing or DEFAULT_BACKING_MODELS,
|
||||
'other_stems_models': custom_other or DEFAULT_OTHER_MODELS,
|
||||
}
|
||||
|
||||
assert effective_model_names['clean_instrumental_model'] == custom_clean
|
||||
assert effective_model_names['backing_vocals_models'] == custom_backing
|
||||
assert effective_model_names['other_stems_models'] == custom_other
|
||||
|
||||
|
||||
class TestDistributionDirectoryPreparation:
|
||||
"""Tests for _prepare_distribution_directory in video_worker.
|
||||
|
||||
These tests verify that the distribution directory is prepared with:
|
||||
- stems/ subfolder containing all audio stems with model names
|
||||
- lyrics/ subfolder containing intermediate lyrics files
|
||||
- Properly named instrumentals at root level
|
||||
"""
|
||||
|
||||
def test_distribution_directory_creates_stems_folder(self, tmp_path):
|
||||
"""Test that stems directory is created."""
|
||||
stems_dir = tmp_path / "stems"
|
||||
stems_dir.mkdir()
|
||||
|
||||
assert stems_dir.exists()
|
||||
assert stems_dir.is_dir()
|
||||
|
||||
def test_distribution_directory_creates_lyrics_folder(self, tmp_path):
|
||||
"""Test that lyrics directory is created."""
|
||||
lyrics_dir = tmp_path / "lyrics"
|
||||
lyrics_dir.mkdir()
|
||||
|
||||
assert lyrics_dir.exists()
|
||||
assert lyrics_dir.is_dir()
|
||||
|
||||
def test_instrumental_naming_with_model(self):
|
||||
"""Test that instrumental files are named with model names."""
|
||||
base_name = "Artist - Song"
|
||||
clean_model = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
backing_model = "mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"
|
||||
|
||||
clean_instrumental_name = f"{base_name} (Instrumental {clean_model}).flac"
|
||||
backing_instrumental_name = f"{base_name} (Instrumental +BV {backing_model}).flac"
|
||||
|
||||
# Verify expected format
|
||||
assert "model_bs_roformer" in clean_instrumental_name
|
||||
assert "+BV" in backing_instrumental_name
|
||||
assert backing_model in backing_instrumental_name
|
||||
|
||||
def test_stem_naming_convention(self):
|
||||
"""Test that stems are named with proper model suffixes."""
|
||||
base_name = "Artist - Song"
|
||||
clean_model = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
backing_model = "mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"
|
||||
other_model = "htdemucs_6s.yaml"
|
||||
|
||||
# Expected stem filenames
|
||||
expected_stems = {
|
||||
'vocals_clean': f"{base_name} (Vocals {clean_model}).flac",
|
||||
'lead_vocals': f"{base_name} (Lead Vocals {backing_model}).flac",
|
||||
'backing_vocals': f"{base_name} (Backing Vocals {backing_model}).flac",
|
||||
'bass': f"{base_name} (Bass {other_model}).flac",
|
||||
'drums': f"{base_name} (Drums {other_model}).flac",
|
||||
}
|
||||
|
||||
# Verify naming convention
|
||||
for _key, name in expected_stems.items():
|
||||
assert ".flac" in name
|
||||
assert base_name in name
|
||||
|
||||
def test_simplified_instrumental_cleanup(self, tmp_path):
|
||||
"""Test that simplified instrumental names are cleaned up."""
|
||||
base_name = "Artist - Song"
|
||||
|
||||
# Create simplified-named files (as created by _setup_working_directory)
|
||||
simplified_clean = tmp_path / f"{base_name} (Instrumental Clean).flac"
|
||||
simplified_backing = tmp_path / f"{base_name} (Instrumental Backing).flac"
|
||||
simplified_clean.write_text("fake")
|
||||
simplified_backing.write_text("fake")
|
||||
|
||||
# Verify they exist
|
||||
assert simplified_clean.exists()
|
||||
assert simplified_backing.exists()
|
||||
|
||||
# Simulate cleanup
|
||||
simplified_clean.unlink()
|
||||
simplified_backing.unlink()
|
||||
|
||||
# Verify they're removed
|
||||
assert not simplified_clean.exists()
|
||||
assert not simplified_backing.exists()
|
||||
|
||||
@@ -36,6 +36,35 @@ from karaoke_gen.audio_processor import AudioProcessor
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def _trigger_lyrics_worker_after_url_download(job_id: str) -> None:
|
||||
"""
|
||||
Trigger lyrics worker after URL audio download completes.
|
||||
|
||||
For URL jobs, we use sequential triggering:
|
||||
1. Audio worker downloads and uploads audio to GCS
|
||||
2. Audio worker triggers lyrics worker (this function)
|
||||
3. Both workers then proceed in parallel (audio separation + lyrics transcription)
|
||||
|
||||
This prevents the race condition where lyrics worker times out waiting for audio.
|
||||
"""
|
||||
from backend.services.worker_service import get_worker_service
|
||||
|
||||
try:
|
||||
worker_service = get_worker_service()
|
||||
await worker_service.trigger_lyrics_worker(job_id)
|
||||
logger.info(f"Job {job_id}: Triggered lyrics worker after URL download")
|
||||
except Exception as e:
|
||||
# Log but don't fail - audio processing can still continue
|
||||
# The job will eventually timeout if lyrics worker doesn't run
|
||||
logger.error(f"Job {job_id}: Failed to trigger lyrics worker: {e}")
|
||||
|
||||
|
||||
# Default model names - used by create_audio_processor and stored in state_data
|
||||
DEFAULT_CLEAN_MODEL = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
DEFAULT_BACKING_MODELS = ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
DEFAULT_OTHER_MODELS = ["htdemucs_6s.yaml"]
|
||||
|
||||
|
||||
# Loggers to capture for audio worker
|
||||
AUDIO_WORKER_LOGGERS = [
|
||||
"karaoke_gen.audio_processor",
|
||||
@@ -181,10 +210,10 @@ def create_audio_processor(
|
||||
audio_logger = logging.getLogger("karaoke_gen.audio_processor")
|
||||
audio_logger.setLevel(logging.INFO)
|
||||
|
||||
# Model configurations - use provided values or CLI defaults
|
||||
effective_clean_model = clean_instrumental_model or "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
effective_backing_models = backing_vocals_models or ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
effective_other_models = other_stems_models or ["htdemucs_6s.yaml"] # For 6-stem separation (bass, drums, etc.)
|
||||
# Model configurations - use provided values or defaults from module constants
|
||||
effective_clean_model = clean_instrumental_model or DEFAULT_CLEAN_MODEL
|
||||
effective_backing_models = backing_vocals_models or DEFAULT_BACKING_MODELS
|
||||
effective_other_models = other_stems_models or DEFAULT_OTHER_MODELS # For 6-stem separation (bass, drums, etc.)
|
||||
|
||||
# FFmpeg command for combining audio files (must be a string, not a list)
|
||||
ffmpeg_base_command = "ffmpeg -hide_banner -loglevel error -nostats -y"
|
||||
@@ -303,7 +332,17 @@ async def process_audio_separation(job_id: str) -> bool:
|
||||
backing_vocals_models=job.backing_vocals_models,
|
||||
other_stems_models=job.other_stems_models
|
||||
)
|
||||
|
||||
|
||||
# Store effective model names in state_data for video_worker to use in file naming
|
||||
# This ensures output filenames match local CLI behavior (e.g., "Instrumental model_bs_roformer_ep_317_sdr_12.9755.ckpt")
|
||||
effective_model_names = {
|
||||
'clean_instrumental_model': job.clean_instrumental_model or DEFAULT_CLEAN_MODEL,
|
||||
'backing_vocals_models': job.backing_vocals_models or DEFAULT_BACKING_MODELS,
|
||||
'other_stems_models': job.other_stems_models or DEFAULT_OTHER_MODELS,
|
||||
}
|
||||
job_manager.update_state_data(job_id, 'model_names', effective_model_names)
|
||||
job_log.info(f"Stored effective model names: clean={effective_model_names['clean_instrumental_model']}")
|
||||
|
||||
# Format artist-title for file naming (matches CLI behavior)
|
||||
artist_title = f"{job.artist} - {job.title}"
|
||||
|
||||
@@ -429,29 +468,34 @@ async def download_audio(
|
||||
# Case 3: Fresh URL that needs downloading
|
||||
if job.url:
|
||||
logger.info(f"Job {job_id}: Downloading from URL: {job.url}")
|
||||
|
||||
|
||||
# Use provided job_manager or create new one
|
||||
jm = job_manager_instance or JobManager()
|
||||
|
||||
|
||||
local_path = await download_from_url(
|
||||
job.url,
|
||||
temp_dir,
|
||||
job.artist,
|
||||
job.url,
|
||||
temp_dir,
|
||||
job.artist,
|
||||
job.title,
|
||||
job_manager=jm,
|
||||
job_id=job_id
|
||||
)
|
||||
|
||||
|
||||
if local_path and os.path.exists(local_path):
|
||||
# Upload to GCS and update job
|
||||
gcs_path = f"jobs/{job_id}/input/{os.path.basename(local_path)}"
|
||||
url = storage.upload_file(local_path, gcs_path)
|
||||
|
||||
|
||||
# Update job with GCS path for lyrics worker
|
||||
jm.update_job(job_id, {'input_media_gcs_path': gcs_path})
|
||||
jm.update_file_url(job_id, 'input', 'audio', url)
|
||||
|
||||
|
||||
logger.info(f"Job {job_id}: Downloaded and uploaded audio to GCS: {gcs_path}")
|
||||
|
||||
# For URL jobs, trigger lyrics worker now that audio is available
|
||||
# This is the sequential trigger pattern - audio first, then lyrics
|
||||
await _trigger_lyrics_worker_after_url_download(job_id)
|
||||
|
||||
return local_path
|
||||
else:
|
||||
logger.error(f"Job {job_id}: Failed to download from URL: {job.url}")
|
||||
|
||||
@@ -32,6 +32,7 @@ from backend.models.job import JobStatus
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.services.lyrics_cache_service import LyricsCacheService
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
|
||||
from backend.workers.style_helper import load_style_config
|
||||
from backend.services.tracing import job_span, add_span_event, add_span_attribute
|
||||
@@ -39,10 +40,43 @@ from backend.services.metrics import metrics
|
||||
|
||||
# Import from karaoke_gen package
|
||||
from karaoke_gen.lyrics_processor import LyricsProcessor
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Timeout for entire transcription process including agentic AI correction (20 minutes)
|
||||
# This needs to account for:
|
||||
# - Cloud Run cold start / worker initialization (1-5 min)
|
||||
# - AudioShake transcription (1-2 min)
|
||||
# - spaCy model loading for correction (2-3 min on cold start)
|
||||
# - Agentic AI correction (1-3 min)
|
||||
TRANSCRIPTION_TIMEOUT_SECONDS = 1200
|
||||
|
||||
# Default agentic correction timeout (3 minutes)
|
||||
# Configurable via AGENTIC_CORRECTION_TIMEOUT_SECONDS environment variable
|
||||
# If agentic AI takes longer than this, abort and use uncorrected transcription
|
||||
# Human review will correct any issues
|
||||
DEFAULT_AGENTIC_TIMEOUT_SECONDS = 180
|
||||
|
||||
|
||||
def _configure_agentic_ai():
|
||||
"""Configure environment variables for agentic AI correction.
|
||||
|
||||
The lyrics_transcriber library reads these directly from environment.
|
||||
This ensures the settings from backend config are available.
|
||||
"""
|
||||
settings = get_settings()
|
||||
|
||||
# Enable agentic AI if configured
|
||||
if settings.use_agentic_ai:
|
||||
os.environ["USE_AGENTIC_AI"] = "1"
|
||||
os.environ["AGENTIC_AI_MODEL"] = settings.agentic_ai_model
|
||||
logger.info(f"Agentic AI enabled with model: {settings.agentic_ai_model}")
|
||||
else:
|
||||
os.environ["USE_AGENTIC_AI"] = "0"
|
||||
logger.info("Agentic AI disabled")
|
||||
|
||||
|
||||
# Loggers to capture for lyrics worker (top-level package loggers only)
|
||||
# Note: We only capture at top-level package loggers to avoid duplicate log entries.
|
||||
@@ -119,7 +153,10 @@ async def process_lyrics_transcription(job_id: str) -> bool:
|
||||
start_time = time.time()
|
||||
job_manager = JobManager()
|
||||
storage = StorageService()
|
||||
|
||||
|
||||
# Configure agentic AI before any lyrics processing
|
||||
_configure_agentic_ai()
|
||||
|
||||
# Create job logger for remote debugging FIRST
|
||||
job_log = create_job_logger(job_id, "lyrics")
|
||||
|
||||
@@ -193,7 +230,40 @@ async def process_lyrics_transcription(job_id: str) -> bool:
|
||||
raise Exception("Failed to download audio file")
|
||||
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
|
||||
download_span.set_attribute("audio_file", os.path.basename(audio_path))
|
||||
|
||||
|
||||
# Set up LyricsTranscriber cache directory and sync from GCS
|
||||
# This allows reusing cached AudioShake/lyrics API responses across Cloud Run instances
|
||||
cache_dir = os.path.join(temp_dir, "lyrics-cache")
|
||||
os.makedirs(cache_dir, exist_ok=True)
|
||||
os.environ["LYRICS_TRANSCRIBER_CACHE_DIR"] = cache_dir
|
||||
job_log.info(f"LyricsTranscriber cache dir: {cache_dir}")
|
||||
|
||||
# Sync cache from GCS (download any existing cached API responses)
|
||||
# Note: Cache sync is best-effort - failures should not fail the job
|
||||
cache_service = LyricsCacheService(storage)
|
||||
audio_hash = None
|
||||
lyrics_hash = None
|
||||
with job_span("sync-cache-from-gcs", job_id):
|
||||
try:
|
||||
# Compute cache keys
|
||||
audio_hash = cache_service.compute_audio_hash(audio_path)
|
||||
# Use lyrics_artist/lyrics_title for lyrics hash (these are what LyricsTranscriber uses)
|
||||
lyrics_search_artist = getattr(job, 'lyrics_artist', None) or job.artist
|
||||
lyrics_search_title = getattr(job, 'lyrics_title', None) or job.title
|
||||
lyrics_hash = cache_service.compute_lyrics_hash(lyrics_search_artist, lyrics_search_title)
|
||||
|
||||
job_log.info(f"Cache keys: audio_hash={audio_hash[:8]}..., lyrics_hash={lyrics_hash[:8]}...")
|
||||
add_span_attribute("audio_hash", audio_hash[:8])
|
||||
add_span_attribute("lyrics_hash", lyrics_hash[:8])
|
||||
|
||||
# Download relevant cache files from GCS
|
||||
cache_stats = cache_service.sync_cache_from_gcs(cache_dir, audio_hash, lyrics_hash)
|
||||
job_log.info(f"Cache sync from GCS: {cache_stats['downloaded']} downloaded, {cache_stats['not_found']} not found")
|
||||
add_span_attribute("cache_hits", cache_stats["downloaded"])
|
||||
except Exception as e:
|
||||
job_log.warning(f"Cache sync from GCS failed (non-fatal): {e}", exc_info=True)
|
||||
add_span_attribute("cache_sync_error", str(e)[:100])
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
# The status is managed at a higher level - workers just track their progress
|
||||
job_manager.update_state_data(job_id, 'lyrics_progress', {
|
||||
@@ -263,26 +333,73 @@ async def process_lyrics_transcription(job_id: str) -> bool:
|
||||
if lyrics_search_title != job.title:
|
||||
job_log.info(f" Lyrics search title override: {lyrics_search_title}")
|
||||
logger.info(f"[job:{job_id}] Calling lyrics_processor.transcribe_lyrics()")
|
||||
|
||||
# Run transcription + correction
|
||||
|
||||
# Run transcription + correction with timeout
|
||||
# AudioShake typically takes 1-2 minutes, but can hang.
|
||||
with job_span("audioshake-transcription", job_id) as trans_span:
|
||||
trans_start = time.time()
|
||||
add_span_event("transcription_started")
|
||||
|
||||
# Run transcription in thread pool to avoid blocking the event loop
|
||||
# (transcribe_lyrics is synchronous and takes 1-2 minutes)
|
||||
# Note: transcribe_lyrics internally calls AudioShake and lyrics APIs
|
||||
with metrics.time_external_api("audioshake", job_id):
|
||||
result = await asyncio.to_thread(
|
||||
lyrics_processor.transcribe_lyrics,
|
||||
input_audio_wav=audio_path,
|
||||
artist=job.artist, # Original artist for file naming
|
||||
title=job.title, # Original title for file naming
|
||||
track_output_dir=temp_dir,
|
||||
lyrics_artist=lyrics_search_artist, # Override for lyrics search
|
||||
lyrics_title=lyrics_search_title # Override for lyrics search
|
||||
|
||||
# Calculate deadline for agentic correction
|
||||
# If agentic AI is enabled, the correction loop will check this deadline
|
||||
# and abort early if exceeded, returning uncorrected transcription for human review
|
||||
settings = get_settings()
|
||||
agentic_deadline = None
|
||||
timeout_seconds = settings.agentic_correction_timeout_seconds
|
||||
if settings.use_agentic_ai:
|
||||
agentic_deadline = time.time() + timeout_seconds
|
||||
job_log.info(
|
||||
f"Agentic AI enabled with {timeout_seconds}s timeout"
|
||||
)
|
||||
|
||||
|
||||
# Run transcription in thread pool
|
||||
# When agentic AI is enabled, wrap with outer timeout as safety net
|
||||
# Inner deadline check in corrector.py will break out of gap loop gracefully
|
||||
# Outer timeout catches case where single LLM call hangs for too long
|
||||
transcription_coro = asyncio.to_thread(
|
||||
lyrics_processor.transcribe_lyrics,
|
||||
input_audio_wav=audio_path,
|
||||
artist=job.artist, # Original artist for file naming
|
||||
title=job.title, # Original title for file naming
|
||||
track_output_dir=temp_dir,
|
||||
lyrics_artist=lyrics_search_artist, # Override for lyrics search
|
||||
lyrics_title=lyrics_search_title, # Override for lyrics search
|
||||
agentic_deadline=agentic_deadline, # Deadline for agentic timeout
|
||||
)
|
||||
|
||||
with metrics.time_external_api("audioshake", job_id):
|
||||
if settings.use_agentic_ai:
|
||||
# Outer timeout must be generous to allow for AudioShake time
|
||||
# (can take 2-5+ minutes for long songs) PLUS agentic correction.
|
||||
# The inner deadline check in corrector.py handles the 3-minute
|
||||
# correction limit gracefully by returning partial results.
|
||||
# This outer timeout is a safety net for completely hung LLM calls.
|
||||
outer_timeout = TRANSCRIPTION_TIMEOUT_SECONDS # Same as non-agentic
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
transcription_coro,
|
||||
timeout=outer_timeout
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
# This should rarely trigger since inner deadline check breaks first
|
||||
# But provides hard stop if e.g. single LLM call hangs completely
|
||||
job_log.error(
|
||||
f"HARD TIMEOUT: Transcription exceeded {outer_timeout}s. "
|
||||
"This indicates the inner deadline check failed to trigger."
|
||||
)
|
||||
raise RuntimeError(
|
||||
f"Lyrics transcription timed out after {outer_timeout}s"
|
||||
) from None
|
||||
else:
|
||||
# Non-agentic mode: use general timeout (10 minutes)
|
||||
try:
|
||||
result = await asyncio.wait_for(
|
||||
transcription_coro,
|
||||
timeout=TRANSCRIPTION_TIMEOUT_SECONDS
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
raise Exception(f"Transcription timed out after {TRANSCRIPTION_TIMEOUT_SECONDS} seconds")
|
||||
|
||||
trans_duration = time.time() - trans_start
|
||||
trans_span.set_attribute("duration_seconds", trans_duration)
|
||||
add_span_event("transcription_completed", {"duration_seconds": trans_duration})
|
||||
@@ -297,7 +414,23 @@ async def process_lyrics_transcription(job_id: str) -> bool:
|
||||
|
||||
job_log.info("All lyrics data uploaded successfully")
|
||||
logger.info(f"[job:{job_id}] All lyrics data uploaded successfully")
|
||||
|
||||
|
||||
# Sync new cache files to GCS (upload any newly created cache entries)
|
||||
# This persists cache across Cloud Run instances for future jobs
|
||||
# Note: Cache upload is best-effort - failures should not fail the job
|
||||
with job_span("sync-cache-to-gcs", job_id):
|
||||
try:
|
||||
if audio_hash and lyrics_hash:
|
||||
upload_stats = cache_service.sync_cache_to_gcs(cache_dir, audio_hash, lyrics_hash)
|
||||
job_log.info(f"Cache sync to GCS: {upload_stats['uploaded']} uploaded, {upload_stats['skipped']} already existed")
|
||||
add_span_attribute("cache_uploads", upload_stats["uploaded"])
|
||||
else:
|
||||
job_log.warning("Skipping cache upload: missing audio_hash or lyrics_hash")
|
||||
add_span_attribute("cache_upload_skipped", "missing_hashes")
|
||||
except Exception as e:
|
||||
job_log.warning(f"Cache upload to GCS failed (non-fatal): {e}")
|
||||
add_span_attribute("cache_upload_error", str(e)[:100])
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
job_manager.update_state_data(job_id, 'lyrics_progress', {
|
||||
'stage': 'lyrics_complete',
|
||||
|
||||
+373
-78
@@ -35,6 +35,7 @@ from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.services.rclone_service import get_rclone_service
|
||||
from backend.services.youtube_service import get_youtube_service
|
||||
from backend.services.encoding_service import get_encoding_service
|
||||
from backend.config import get_settings
|
||||
from backend.workers.style_helper import load_style_config
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
|
||||
@@ -55,6 +56,122 @@ VIDEO_WORKER_LOGGERS = [
|
||||
]
|
||||
|
||||
|
||||
async def _encode_via_gce(
|
||||
job_id: str,
|
||||
job,
|
||||
job_manager: JobManager,
|
||||
storage: StorageService,
|
||||
temp_dir: str,
|
||||
base_name: str,
|
||||
job_log,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Encode videos using the high-performance GCE encoding worker.
|
||||
|
||||
This offloads FFmpeg encoding to a dedicated C4 GCE instance with
|
||||
Intel Granite Rapids 3.9 GHz CPU for 2-3x faster encoding.
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
job: Job object
|
||||
job_manager: Job manager instance
|
||||
storage: Storage service instance
|
||||
temp_dir: Temporary directory for output files
|
||||
base_name: Base name for output files (e.g., "Artist - Title")
|
||||
job_log: Job-specific logger
|
||||
|
||||
Returns:
|
||||
Result dict with paths to encoded files
|
||||
"""
|
||||
settings = get_settings()
|
||||
encoding_service = get_encoding_service()
|
||||
|
||||
job_log.info("Using GCE encoding worker for high-performance encoding")
|
||||
logger.info(f"[job:{job_id}] Using GCE encoding worker")
|
||||
|
||||
# Construct GCS paths for the encoding worker
|
||||
bucket_name = settings.gcs_bucket_name
|
||||
input_gcs_path = f"gs://{bucket_name}/jobs/{job_id}/"
|
||||
output_gcs_path = f"gs://{bucket_name}/jobs/{job_id}/encoded/"
|
||||
|
||||
# Determine instrumental selection for encoding config
|
||||
instrumental_selection = job.state_data.get('instrumental_selection', 'clean')
|
||||
existing_instrumental = getattr(job, 'existing_instrumental_gcs_path', None)
|
||||
|
||||
encoding_config = {
|
||||
"formats": ["mp4_4k_lossless", "mp4_4k_lossy", "mp4_720p"],
|
||||
"base_name": base_name,
|
||||
"instrumental_selection": instrumental_selection,
|
||||
"existing_instrumental": existing_instrumental,
|
||||
"ffmpeg_threads": 8, # c4-standard-8 has 8 vCPUs
|
||||
}
|
||||
|
||||
job_log.info(f"Submitting encoding job to GCE worker")
|
||||
job_log.info(f" Input: {input_gcs_path}")
|
||||
job_log.info(f" Output: {output_gcs_path}")
|
||||
|
||||
def progress_callback(progress: int):
|
||||
# Update job progress (encoding is 75-95% of total)
|
||||
scaled_progress = 75 + int(progress * 0.2) # Map 0-100 to 75-95
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.ENCODING,
|
||||
progress=scaled_progress,
|
||||
message=f"Encoding videos ({progress}%)"
|
||||
)
|
||||
|
||||
try:
|
||||
# Submit and wait for encoding
|
||||
with job_span("gce-encoding", job_id) as encoding_span:
|
||||
encode_start = time.time()
|
||||
add_span_event("gce_encoding_started")
|
||||
|
||||
result = await encoding_service.encode_videos(
|
||||
job_id=job_id,
|
||||
input_gcs_path=input_gcs_path,
|
||||
output_gcs_path=output_gcs_path,
|
||||
encoding_config=encoding_config,
|
||||
progress_callback=progress_callback,
|
||||
)
|
||||
|
||||
encode_duration = time.time() - encode_start
|
||||
encoding_span.set_attribute("duration_seconds", encode_duration)
|
||||
add_span_event("gce_encoding_completed", {"duration_seconds": encode_duration})
|
||||
|
||||
job_log.info(f"GCE encoding complete in {encode_duration:.1f}s")
|
||||
logger.info(f"[job:{job_id}] GCE encoding complete in {encode_duration:.1f}s")
|
||||
|
||||
# Download encoded files from GCS to temp_dir
|
||||
output_files = result.get("output_files", [])
|
||||
local_files = {}
|
||||
|
||||
for gcs_path in output_files:
|
||||
filename = os.path.basename(gcs_path)
|
||||
local_path = os.path.join(temp_dir, filename)
|
||||
|
||||
job_log.info(f"Downloading encoded file: {filename}")
|
||||
storage.download_file(gcs_path, local_path)
|
||||
|
||||
# Map to result keys expected by _upload_results
|
||||
if "4K Lossless" in filename or "4k_lossless" in filename.lower():
|
||||
local_files["final_video"] = local_path
|
||||
elif "4K Lossy" in filename or "4k_lossy" in filename.lower():
|
||||
local_files["final_video_lossy"] = local_path
|
||||
elif "720p" in filename:
|
||||
local_files["final_video_720p"] = local_path
|
||||
elif filename.endswith(".mkv"):
|
||||
local_files["final_video_mkv"] = local_path
|
||||
|
||||
job_log.info(f"Downloaded {len(local_files)} encoded files")
|
||||
|
||||
return local_files
|
||||
|
||||
except Exception as e:
|
||||
job_log.error(f"GCE encoding failed: {e}")
|
||||
logger.error(f"[job:{job_id}] GCE encoding failed: {e}")
|
||||
raise
|
||||
|
||||
|
||||
async def generate_video(job_id: str) -> bool:
|
||||
"""
|
||||
Generate final karaoke videos using KaraokeFinalise.
|
||||
@@ -184,80 +301,105 @@ async def generate_video(job_id: str) -> bool:
|
||||
countdown_padding_seconds = lyrics_metadata.get('countdown_padding_seconds', 3.0)
|
||||
job_log.info(f"Countdown padding detected: {countdown_padding_seconds}s - instrumental will be padded if needed")
|
||||
|
||||
# Log finalization parameters
|
||||
job_log.info("Creating KaraokeFinalise with parameters:")
|
||||
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
|
||||
job_log.info(f" enable_txt: {getattr(job, 'enable_txt', False)}")
|
||||
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
|
||||
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
|
||||
job_log.info(f" instrumental source: {instrumental_source}")
|
||||
job_log.info(f" countdown_padding_seconds: {countdown_padding_seconds}")
|
||||
if existing_instrumental_path:
|
||||
job_log.info(f" using user-provided instrumental (selection was: {instrumental_selection})")
|
||||
|
||||
# Create KaraokeFinalise with ALL the parameters from the job
|
||||
# This reuses all existing functionality!
|
||||
|
||||
# Set up YouTube description file if template is provided
|
||||
youtube_desc_path = None
|
||||
if youtube_credentials and getattr(job, 'youtube_description_template', None):
|
||||
youtube_desc_path = os.path.join(temp_dir, "youtube_description.txt")
|
||||
with open(youtube_desc_path, 'w') as f:
|
||||
f.write(job.youtube_description_template)
|
||||
job_log.info("YouTube description template written to temp file")
|
||||
|
||||
finalise = KaraokeFinalise(
|
||||
logger=logger,
|
||||
log_level=logging.INFO,
|
||||
dry_run=False,
|
||||
instrumental_format="flac",
|
||||
# CDG/TXT generation
|
||||
enable_cdg=getattr(job, 'enable_cdg', False),
|
||||
enable_txt=getattr(job, 'enable_txt', False),
|
||||
cdg_styles=cdg_styles,
|
||||
# Brand code and organization (server-side mode uses rclone)
|
||||
brand_prefix=getattr(job, 'brand_prefix', None),
|
||||
organised_dir=None, # Not used in cloud - files stay in GCS
|
||||
organised_dir_rclone_root=getattr(job, 'organised_dir_rclone_root', None),
|
||||
public_share_dir=None, # Not used in cloud
|
||||
# Notifications
|
||||
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
|
||||
# YouTube upload (server-side with pre-loaded credentials)
|
||||
youtube_client_secrets_file=None, # Not used with pre-stored credentials
|
||||
youtube_description_file=youtube_desc_path,
|
||||
user_youtube_credentials=youtube_credentials, # Pre-loaded from Secret Manager
|
||||
rclone_destination=None,
|
||||
email_template_file=None,
|
||||
# Server-side optimizations
|
||||
non_interactive=True,
|
||||
server_side_mode=True,
|
||||
selected_instrumental_file=instrumental_file,
|
||||
# Audio synchronization - ensure instrumental matches vocal padding
|
||||
countdown_padding_seconds=countdown_padding_seconds,
|
||||
)
|
||||
|
||||
# Call process() - this does ALL the work:
|
||||
# - Encodes to 4 video formats
|
||||
# - Generates CDG/TXT packages if enabled
|
||||
# - Posts Discord notification if configured
|
||||
# - Handles brand code generation
|
||||
with job_span("karaoke-finalise", job_id) as finalise_span:
|
||||
finalise_start = time.time()
|
||||
job_log.info("Starting KaraokeFinalise.process() - this may take 15-20 minutes...")
|
||||
logger.info(f"[job:{job_id}] Starting KaraokeFinalise.process()")
|
||||
add_span_event("finalise_started")
|
||||
|
||||
result = finalise.process()
|
||||
|
||||
finalise_duration = time.time() - finalise_start
|
||||
finalise_span.set_attribute("duration_seconds", finalise_duration)
|
||||
add_span_event("finalise_completed", {"duration_seconds": finalise_duration})
|
||||
|
||||
job_log.info("KaraokeFinalise.process() complete!")
|
||||
if result.get('brand_code'):
|
||||
job_log.info(f"Brand code: {result.get('brand_code')}")
|
||||
logger.info(f"[job:{job_id}] KaraokeFinalise.process() complete in {finalise_duration:.1f}s")
|
||||
logger.info(f"[job:{job_id}] Brand code: {result.get('brand_code')}")
|
||||
# Check if GCE encoding is enabled for high-performance encoding
|
||||
encoding_service = get_encoding_service()
|
||||
use_gce_encoding = encoding_service.is_enabled
|
||||
|
||||
if use_gce_encoding:
|
||||
# ============ GCE ENCODING PATH ============
|
||||
# Use dedicated C4 GCE instance for 2-3x faster FFmpeg encoding
|
||||
job_log.info("GCE encoding enabled - using high-performance encoding worker")
|
||||
job_log.info(f" instrumental source: {instrumental_source}")
|
||||
|
||||
result = await _encode_via_gce(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
job_manager=job_manager,
|
||||
storage=storage,
|
||||
temp_dir=temp_dir,
|
||||
base_name=base_name,
|
||||
job_log=job_log,
|
||||
)
|
||||
|
||||
# GCE encoding doesn't generate brand_code, CDG/TXT, or Discord notifications
|
||||
# These will be handled by distribution if needed
|
||||
job_log.info("GCE encoding complete - encoded files downloaded")
|
||||
|
||||
else:
|
||||
# ============ STANDARD ENCODING PATH ============
|
||||
# Use KaraokeFinalise for encoding (runs on Cloud Run)
|
||||
|
||||
# Log finalization parameters
|
||||
job_log.info("Using KaraokeFinalise for encoding (standard path)")
|
||||
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
|
||||
job_log.info(f" enable_txt: {getattr(job, 'enable_txt', False)}")
|
||||
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
|
||||
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
|
||||
job_log.info(f" instrumental source: {instrumental_source}")
|
||||
job_log.info(f" countdown_padding_seconds: {countdown_padding_seconds}")
|
||||
if existing_instrumental_path:
|
||||
job_log.info(f" using user-provided instrumental (selection was: {instrumental_selection})")
|
||||
|
||||
# Set up YouTube description file if template is provided
|
||||
youtube_desc_path = None
|
||||
if youtube_credentials and getattr(job, 'youtube_description_template', None):
|
||||
youtube_desc_path = os.path.join(temp_dir, "youtube_description.txt")
|
||||
with open(youtube_desc_path, 'w') as f:
|
||||
f.write(job.youtube_description_template)
|
||||
job_log.info("YouTube description template written to temp file")
|
||||
|
||||
finalise = KaraokeFinalise(
|
||||
logger=logger,
|
||||
log_level=logging.INFO,
|
||||
dry_run=False,
|
||||
instrumental_format="flac",
|
||||
# CDG/TXT generation
|
||||
enable_cdg=getattr(job, 'enable_cdg', False),
|
||||
enable_txt=getattr(job, 'enable_txt', False),
|
||||
cdg_styles=cdg_styles,
|
||||
# Brand code and organization (server-side mode uses rclone)
|
||||
brand_prefix=getattr(job, 'brand_prefix', None),
|
||||
organised_dir=None, # Not used in cloud - files stay in GCS
|
||||
organised_dir_rclone_root=getattr(job, 'organised_dir_rclone_root', None),
|
||||
public_share_dir=None, # Not used in cloud
|
||||
# Notifications
|
||||
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
|
||||
# YouTube upload (server-side with pre-loaded credentials)
|
||||
youtube_client_secrets_file=None, # Not used with pre-stored credentials
|
||||
youtube_description_file=youtube_desc_path,
|
||||
user_youtube_credentials=youtube_credentials, # Pre-loaded from Secret Manager
|
||||
rclone_destination=None,
|
||||
email_template_file=None,
|
||||
# Server-side optimizations
|
||||
non_interactive=True,
|
||||
server_side_mode=True,
|
||||
selected_instrumental_file=instrumental_file,
|
||||
# Audio synchronization - ensure instrumental matches vocal padding
|
||||
countdown_padding_seconds=countdown_padding_seconds,
|
||||
)
|
||||
|
||||
# Call process() - this does ALL the work:
|
||||
# - Encodes to 4 video formats
|
||||
# - Generates CDG/TXT packages if enabled
|
||||
# - Posts Discord notification if configured
|
||||
# - Handles brand code generation
|
||||
with job_span("karaoke-finalise", job_id) as finalise_span:
|
||||
finalise_start = time.time()
|
||||
job_log.info("Starting KaraokeFinalise.process() - this may take 15-20 minutes...")
|
||||
logger.info(f"[job:{job_id}] Starting KaraokeFinalise.process()")
|
||||
add_span_event("finalise_started")
|
||||
|
||||
result = finalise.process()
|
||||
|
||||
finalise_duration = time.time() - finalise_start
|
||||
finalise_span.set_attribute("duration_seconds", finalise_duration)
|
||||
add_span_event("finalise_completed", {"duration_seconds": finalise_duration})
|
||||
|
||||
job_log.info("KaraokeFinalise.process() complete!")
|
||||
if result.get('brand_code'):
|
||||
job_log.info(f"Brand code: {result.get('brand_code')}")
|
||||
logger.info(f"[job:{job_id}] KaraokeFinalise.process() complete in {finalise_duration:.1f}s")
|
||||
logger.info(f"[job:{job_id}] Brand code: {result.get('brand_code')}")
|
||||
|
||||
# Native API distribution uploads (used by remote CLI instead of rclone)
|
||||
with job_span("distribution", job_id):
|
||||
@@ -268,6 +410,7 @@ async def generate_video(job_id: str) -> bool:
|
||||
job_manager=job_manager,
|
||||
temp_dir=temp_dir,
|
||||
result=result,
|
||||
storage=storage,
|
||||
)
|
||||
|
||||
# Upload generated files to GCS
|
||||
@@ -291,11 +434,16 @@ async def generate_video(job_id: str) -> bool:
|
||||
)
|
||||
|
||||
# Store result metadata in job
|
||||
# NOTE: Must include dropbox_link and gdrive_files from result, since
|
||||
# _handle_native_distribution already saved them but job.state_data
|
||||
# is stale (fetched before distribution ran)
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
'brand_code': result.get('brand_code'),
|
||||
'youtube_url': result.get('youtube_url'),
|
||||
'dropbox_link': result.get('dropbox_link'),
|
||||
'gdrive_files': result.get('gdrive_files'),
|
||||
}
|
||||
})
|
||||
|
||||
@@ -343,16 +491,17 @@ async def _handle_native_distribution(
|
||||
job_manager: JobManager,
|
||||
temp_dir: str,
|
||||
result: Dict[str, Any],
|
||||
storage: StorageService = None,
|
||||
) -> None:
|
||||
"""
|
||||
Handle distribution uploads using native APIs (Dropbox SDK, Google Drive API).
|
||||
|
||||
|
||||
This is used by the remote CLI instead of rclone-based uploads.
|
||||
The native APIs provide:
|
||||
- Better error handling and retry logic
|
||||
- No need for rclone binary in container
|
||||
- Credentials managed via Secret Manager
|
||||
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
job: Job object with dropbox_path and gdrive_folder_id fields
|
||||
@@ -360,6 +509,7 @@ async def _handle_native_distribution(
|
||||
job_manager: Job manager for updating job state
|
||||
temp_dir: Temporary directory with output files
|
||||
result: Result dict from KaraokeFinalise.process()
|
||||
storage: StorageService instance for downloading stems/lyrics
|
||||
"""
|
||||
brand_code = result.get('brand_code')
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
@@ -392,10 +542,22 @@ async def _handle_native_distribution(
|
||||
result['brand_code'] = brand_code
|
||||
job_log.info(f"Generated brand code: {brand_code}")
|
||||
|
||||
# Prepare full distribution directory with stems/ and lyrics/ subfolders
|
||||
# This must be done before upload to ensure complete output structure
|
||||
if storage:
|
||||
await _prepare_distribution_directory(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
storage=storage,
|
||||
temp_dir=temp_dir,
|
||||
base_name=base_name,
|
||||
job_log=job_log,
|
||||
)
|
||||
|
||||
# Create folder name and upload files
|
||||
folder_name = f"{brand_code} - {base_name}"
|
||||
remote_folder = f"{dropbox_path}/{folder_name}"
|
||||
|
||||
|
||||
job_log.info(f"Uploading to Dropbox folder: {remote_folder}")
|
||||
dropbox.upload_folder(temp_dir, remote_folder)
|
||||
|
||||
@@ -601,6 +763,139 @@ async def _setup_working_directory(
|
||||
log_progress("Downloaded end JPG")
|
||||
|
||||
|
||||
async def _prepare_distribution_directory(
|
||||
job_id: str,
|
||||
job,
|
||||
storage: StorageService,
|
||||
temp_dir: str,
|
||||
base_name: str,
|
||||
job_log=None
|
||||
) -> None:
|
||||
"""
|
||||
Prepare the full distribution directory structure matching local CLI output.
|
||||
|
||||
Creates:
|
||||
- stems/ subfolder with all audio stems (proper model names)
|
||||
- lyrics/ subfolder with intermediate lyrics files
|
||||
- Root-level instrumentals with proper model names
|
||||
|
||||
This ensures the Dropbox upload contains the complete output structure.
|
||||
"""
|
||||
def log_progress(message: str):
|
||||
"""Log to both module logger and job logger if available."""
|
||||
logger.info(f"Job {job_id}: {message}")
|
||||
if job_log:
|
||||
job_log.info(message)
|
||||
|
||||
log_progress("Preparing full distribution directory structure")
|
||||
|
||||
# Get model names from state_data (stored by audio_worker)
|
||||
model_names = job.state_data.get('model_names', {})
|
||||
clean_model = model_names.get('clean_instrumental_model', 'model_bs_roformer_ep_317_sdr_12.9755.ckpt')
|
||||
backing_models = model_names.get('backing_vocals_models', ['mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt'])
|
||||
other_models = model_names.get('other_stems_models', ['htdemucs_6s.yaml'])
|
||||
backing_model = backing_models[0] if backing_models else 'mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt'
|
||||
other_model = other_models[0] if other_models else 'htdemucs_6s.yaml'
|
||||
|
||||
# Create stems subdirectory
|
||||
stems_dir = os.path.join(temp_dir, "stems")
|
||||
os.makedirs(stems_dir, exist_ok=True)
|
||||
|
||||
# Create lyrics subdirectory
|
||||
lyrics_dir = os.path.join(temp_dir, "lyrics")
|
||||
os.makedirs(lyrics_dir, exist_ok=True)
|
||||
|
||||
stems = job.file_urls.get('stems', {})
|
||||
lyrics = job.file_urls.get('lyrics', {})
|
||||
|
||||
# --- Download stems to stems/ subfolder with proper model names ---
|
||||
|
||||
# Clean up simplified-named instrumentals from _setup_working_directory
|
||||
# These were needed for KaraokeFinalise but shouldn't go to distribution
|
||||
simplified_instrumental_patterns = [
|
||||
f"{base_name} (Instrumental Clean).flac",
|
||||
f"{base_name} (Instrumental Backing).flac",
|
||||
]
|
||||
for pattern in simplified_instrumental_patterns:
|
||||
simplified_path = os.path.join(temp_dir, pattern)
|
||||
if os.path.exists(simplified_path):
|
||||
os.remove(simplified_path)
|
||||
log_progress(f"Removed simplified instrumental: {pattern}")
|
||||
|
||||
# Clean instrumental (from clean_instrumental_model)
|
||||
if stems.get('instrumental_clean'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Instrumental {clean_model}).flac")
|
||||
storage.download_file(stems['instrumental_clean'], dest_path)
|
||||
log_progress("Downloaded clean instrumental to stems/")
|
||||
|
||||
# Also copy to root level for distribution
|
||||
root_path = os.path.join(temp_dir, f"{base_name} (Instrumental {clean_model}).flac")
|
||||
shutil.copy2(dest_path, root_path)
|
||||
|
||||
# Clean vocals (from clean_instrumental_model)
|
||||
if stems.get('vocals_clean'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Vocals {clean_model}).flac")
|
||||
storage.download_file(stems['vocals_clean'], dest_path)
|
||||
log_progress("Downloaded clean vocals to stems/")
|
||||
|
||||
# Instrumental with backing vocals (from backing_vocals_model)
|
||||
if stems.get('instrumental_with_backing'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Instrumental +BV {backing_model}).flac")
|
||||
storage.download_file(stems['instrumental_with_backing'], dest_path)
|
||||
log_progress("Downloaded instrumental+BV to stems/")
|
||||
|
||||
# Also copy to root level for distribution
|
||||
root_path = os.path.join(temp_dir, f"{base_name} (Instrumental +BV {backing_model}).flac")
|
||||
shutil.copy2(dest_path, root_path)
|
||||
|
||||
# Lead vocals (from backing_vocals_model)
|
||||
if stems.get('lead_vocals'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Lead Vocals {backing_model}).flac")
|
||||
storage.download_file(stems['lead_vocals'], dest_path)
|
||||
log_progress("Downloaded lead vocals to stems/")
|
||||
|
||||
# Backing vocals (from backing_vocals_model)
|
||||
if stems.get('backing_vocals'):
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} (Backing Vocals {backing_model}).flac")
|
||||
storage.download_file(stems['backing_vocals'], dest_path)
|
||||
log_progress("Downloaded backing vocals to stems/")
|
||||
|
||||
# Other stems (from other_stems_model - typically htdemucs_6s)
|
||||
other_stem_keys = ['bass', 'drums', 'guitar', 'piano', 'other']
|
||||
for stem_key in other_stem_keys:
|
||||
if stems.get(stem_key):
|
||||
stem_name = stem_key.capitalize()
|
||||
dest_path = os.path.join(stems_dir, f"{base_name} ({stem_name} {other_model}).flac")
|
||||
storage.download_file(stems[stem_key], dest_path)
|
||||
log_progress(f"Downloaded {stem_key} to stems/")
|
||||
|
||||
# --- Download lyrics files to lyrics/ subfolder ---
|
||||
|
||||
# Common lyrics file types
|
||||
lyrics_file_keys = ['lrc', 'ass', 'srt', 'txt', 'json', 'original_json', 'corrected_json']
|
||||
for lyrics_key in lyrics_file_keys:
|
||||
if lyrics.get(lyrics_key):
|
||||
# Determine proper extension
|
||||
ext_map = {
|
||||
'lrc': '.lrc',
|
||||
'ass': '.ass',
|
||||
'srt': '.srt',
|
||||
'txt': '.txt',
|
||||
'json': '.json',
|
||||
'original_json': '_original.json',
|
||||
'corrected_json': '_corrected.json',
|
||||
}
|
||||
ext = ext_map.get(lyrics_key, f'.{lyrics_key}')
|
||||
dest_path = os.path.join(lyrics_dir, f"{base_name} (Karaoke){ext}")
|
||||
try:
|
||||
storage.download_file(lyrics[lyrics_key], dest_path)
|
||||
log_progress(f"Downloaded {lyrics_key} to lyrics/")
|
||||
except Exception as e:
|
||||
log_progress(f"Could not download {lyrics_key}: {e}")
|
||||
|
||||
log_progress("Distribution directory prepared with stems/ and lyrics/ subfolders")
|
||||
|
||||
|
||||
async def _upload_results(
|
||||
job_id: str,
|
||||
job_manager: JobManager,
|
||||
@@ -609,7 +904,7 @@ async def _upload_results(
|
||||
result: Dict[str, Any]
|
||||
) -> None:
|
||||
"""Upload generated files to GCS."""
|
||||
|
||||
|
||||
# Map of result keys to GCS paths
|
||||
file_mappings = [
|
||||
('final_video', 'finals', 'lossless_4k_mp4'),
|
||||
|
||||
+1
-1
@@ -75,7 +75,7 @@ steps:
|
||||
--concurrency 1 \
|
||||
--max-instances 50 \
|
||||
--min-instances 0 \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=$PROJECT_ID,GCS_BUCKET_NAME=karaoke-gen-storage-$PROJECT_ID,FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD" \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=$PROJECT_ID,GCS_BUCKET_NAME=karaoke-gen-storage-$PROJECT_ID,FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1,DEFAULT_ENABLE_YOUTUBE_UPLOAD=true,DEFAULT_BRAND_PREFIX=NOMAD,USE_CLOUD_RUN_JOBS_FOR_VIDEO=true" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,RED_API_KEY=red-api-key:latest,RED_API_URL=red-api-url:latest,OPS_API_KEY=ops-api-key:latest,OPS_API_URL=ops-api-url:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest"
|
||||
|
||||
images:
|
||||
|
||||
+211
-1
@@ -174,10 +174,220 @@ Common status codes:
|
||||
- `404` - Job not found
|
||||
- `500` - Server error
|
||||
|
||||
## User Authentication
|
||||
|
||||
### Send Magic Link
|
||||
|
||||
```http
|
||||
POST /api/users/auth/magic-link
|
||||
Content-Type: application/json
|
||||
|
||||
{"email": "user@example.com"}
|
||||
```
|
||||
|
||||
### Verify Magic Link
|
||||
|
||||
```http
|
||||
GET /api/users/auth/verify?token=TOKEN
|
||||
```
|
||||
|
||||
Returns session token and user info.
|
||||
|
||||
### Get Current User
|
||||
|
||||
```http
|
||||
GET /api/users/me
|
||||
Authorization: Bearer SESSION_TOKEN
|
||||
```
|
||||
|
||||
### Logout
|
||||
|
||||
```http
|
||||
POST /api/users/auth/logout
|
||||
Authorization: Bearer SESSION_TOKEN
|
||||
```
|
||||
|
||||
## Credits & Payments (PR #90)
|
||||
|
||||
### List Credit Packages
|
||||
|
||||
```http
|
||||
GET /api/users/credits/packages
|
||||
```
|
||||
|
||||
No auth required. Returns available packages and prices.
|
||||
|
||||
### Create Checkout Session
|
||||
|
||||
```http
|
||||
POST /api/users/credits/checkout
|
||||
Content-Type: application/json
|
||||
|
||||
{"package_id": "5_credits", "email": "user@example.com"}
|
||||
```
|
||||
|
||||
Returns Stripe checkout URL.
|
||||
|
||||
### Stripe Webhook
|
||||
|
||||
```http
|
||||
POST /api/users/webhooks/stripe
|
||||
```
|
||||
|
||||
Handles `checkout.session.completed` events.
|
||||
|
||||
## Beta Tester Program (PR #90)
|
||||
|
||||
### Enroll as Beta Tester
|
||||
|
||||
```http
|
||||
POST /api/users/beta/enroll
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"email": "user@example.com",
|
||||
"promise_text": "I promise to provide feedback...",
|
||||
"accept_corrections_work": true
|
||||
}
|
||||
```
|
||||
|
||||
Returns 1 free credit and session token.
|
||||
|
||||
### Submit Feedback
|
||||
|
||||
```http
|
||||
POST /api/users/beta/feedback
|
||||
Authorization: Bearer SESSION_TOKEN
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"overall_rating": 4,
|
||||
"ease_of_use_rating": 5,
|
||||
"lyrics_accuracy_rating": 4,
|
||||
"correction_experience_rating": 3,
|
||||
"what_went_well": "...",
|
||||
"what_could_improve": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Bonus credit for detailed feedback (50+ chars).
|
||||
|
||||
## Admin Endpoints
|
||||
|
||||
All admin endpoints require an admin-role session token.
|
||||
|
||||
### Admin Dashboard Stats
|
||||
|
||||
```http
|
||||
GET /api/admin/stats/overview
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
Returns platform statistics:
|
||||
```json
|
||||
{
|
||||
"total_users": 100,
|
||||
"active_users_7d": 25,
|
||||
"active_users_30d": 60,
|
||||
"total_jobs": 500,
|
||||
"jobs_last_7d": 50,
|
||||
"jobs_last_30d": 150,
|
||||
"jobs_by_status": {
|
||||
"pending": 5,
|
||||
"processing": 3,
|
||||
"awaiting_review": 10,
|
||||
"complete": 400,
|
||||
"failed": 20
|
||||
},
|
||||
"total_credits_issued_30d": 200,
|
||||
"total_beta_testers": 30
|
||||
}
|
||||
```
|
||||
|
||||
### List Users (Admin)
|
||||
|
||||
```http
|
||||
GET /api/users/admin/users
|
||||
GET /api/users/admin/users?search=user@example.com
|
||||
GET /api/users/admin/users?limit=20&offset=0&sort_by=created_at&sort_order=desc
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
Returns paginated user list with total count.
|
||||
|
||||
### User Detail (Admin)
|
||||
|
||||
```http
|
||||
GET /api/users/admin/users/{email}/detail
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
Returns full user profile including:
|
||||
- User info and stats
|
||||
- Recent credit transactions (last 20)
|
||||
- Recent jobs (last 10)
|
||||
- Active sessions count
|
||||
|
||||
### Add Credits (Admin)
|
||||
|
||||
```http
|
||||
POST /api/users/admin/credits
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"email": "user@example.com",
|
||||
"amount": 5,
|
||||
"reason": "Beta reward"
|
||||
}
|
||||
```
|
||||
|
||||
### Set User Role (Admin)
|
||||
|
||||
```http
|
||||
POST /api/users/admin/users/{email}/role
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
Content-Type: application/json
|
||||
|
||||
{"role": "admin"}
|
||||
```
|
||||
|
||||
### Enable/Disable User (Admin)
|
||||
|
||||
```http
|
||||
POST /api/users/admin/users/{email}/enable
|
||||
POST /api/users/admin/users/{email}/disable
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
### Beta Stats (Admin)
|
||||
|
||||
```http
|
||||
GET /api/users/admin/beta/stats
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
### Beta Feedback List (Admin)
|
||||
|
||||
```http
|
||||
GET /api/users/admin/beta/feedback?limit=50
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
### Delete Job (Admin)
|
||||
|
||||
```http
|
||||
DELETE /api/jobs/{job_id}?delete_files=true
|
||||
Authorization: Bearer ADMIN_TOKEN
|
||||
```
|
||||
|
||||
Admins can delete any job. Regular users can only delete their own jobs.
|
||||
|
||||
## Rate Limits
|
||||
|
||||
No rate limits currently implemented.
|
||||
|
||||
## Webhooks
|
||||
|
||||
Not yet implemented. Jobs must be polled for status.
|
||||
Stripe webhooks implemented at `/api/users/webhooks/stripe`.
|
||||
Job status webhooks not yet implemented.
|
||||
+24
-6
@@ -23,12 +23,15 @@
|
||||
│ • Video Worker - Final assembly and encoding │
|
||||
└───────────────────────────┬─────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────┐ ┌──────────────┐
|
||||
│ Firestore│ │ GCS │ │Secret Manager│
|
||||
│ (jobs) │ │ (files) │ │ (API keys) │
|
||||
└──────────┘ └──────────┘ └──────────────┘
|
||||
┌─────────────┼─────────────┬───────────────┐
|
||||
▼ ▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Firestore│ │ GCS │ │Secret Manager│ │GCE Encoding │
|
||||
│ (jobs) │ │ (files) │ │ (API keys) │ │ Worker (*) │
|
||||
└──────────┘ └──────────┘ └──────────────┘ └──────────────┘
|
||||
|
||||
(*) GCE Encoding Worker: C4-standard-8 VM with Intel Granite Rapids for
|
||||
high-performance FFmpeg encoding. Optional - falls back to Cloud Run.
|
||||
```
|
||||
|
||||
## Processing Pipeline
|
||||
@@ -148,9 +151,24 @@ LyricsTranscriber LyricsTranscriber
|
||||
|---------|---------|----------|
|
||||
| Modal | GPU audio separation | Yes |
|
||||
| AudioShake | Lyrics transcription | Yes |
|
||||
| Vertex AI | Agentic AI correction (Gemini 3 Flash) | Default on |
|
||||
| Genius | Reference lyrics | Yes |
|
||||
| YouTube API | Video upload | Optional |
|
||||
|
||||
## Firestore Collections
|
||||
|
||||
karaoke-gen shares a GCP project (`nomadkaraoke`) with karaoke-decide, but uses separate Firestore collections:
|
||||
|
||||
| Collection | Purpose | Key Fields |
|
||||
|------------|---------|------------|
|
||||
| `gen_users` | karaoke-gen user accounts | email, credits, role, is_active |
|
||||
| `jobs` | Karaoke generation jobs | job_id, user_email, status, state_data |
|
||||
| `sessions` | Magic link auth sessions | user_email, token, expires_at |
|
||||
| `magic_links` | Passwordless auth tokens | email, token, expires_at, used |
|
||||
| `beta_feedback` | Beta program feedback | user_email, ratings, comments |
|
||||
|
||||
**Note**: The `users` collection in the same Firestore instance belongs to karaoke-decide (different schema: user_id, is_guest, quiz_* fields). Don't use it for karaoke-gen.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Backend**: FastAPI, Python 3.12, Cloud Run
|
||||
|
||||
+11
-1
@@ -151,7 +151,11 @@ ADMIN_TOKENS=token1,token2
|
||||
|
||||
```bash
|
||||
NEXT_PUBLIC_API_URL=http://localhost:8000
|
||||
KARAOKE_ACCESS_TOKEN=... # For E2E tests
|
||||
|
||||
# For E2E tests (see frontend/.env.local.example)
|
||||
KARAOKE_ACCESS_TOKEN=... # Skip email enrollment
|
||||
TESTMAIL_API_KEY=... # testmail.app API key
|
||||
TESTMAIL_NAMESPACE=... # testmail.app namespace
|
||||
```
|
||||
|
||||
## Key Scripts
|
||||
@@ -162,6 +166,12 @@ KARAOKE_ACCESS_TOKEN=... # For E2E tests
|
||||
| `./scripts/run-backend-local.sh` | Start backend with emulators |
|
||||
| `./scripts/start-emulators.sh` | Start emulators only |
|
||||
| `./scripts/stop-emulators.sh` | Stop emulators |
|
||||
| `./scripts/debug-job.sh <job_id>` | Debug a cloud job (status, logs, files) |
|
||||
| `./scripts/get_job.py <job_id>` | Fetch job data from Firestore |
|
||||
| `./scripts/fetch_job_logs.py <job_id>` | Fetch cloud job logs to local file |
|
||||
| `./scripts/analyze_log_timing.py <log>` | Profile karaoke-gen performance from logs |
|
||||
| `./scripts/benchmark_ffmpeg.py` | Isolated FFmpeg encoding benchmark |
|
||||
| `./scripts/compare_local_vs_remote.py` | Compare local vs cloud karaoke-gen performance |
|
||||
|
||||
## Project Structure
|
||||
|
||||
|
||||
@@ -0,0 +1,234 @@
|
||||
# E2E Happy Path Test Specification
|
||||
|
||||
> **STATUS**: ✅ **PASSING** - Test completes all 12 steps in ~38 minutes. Last verified 2026-01-02.
|
||||
>
|
||||
> **IMPORTANT**: This document defines the requirements for a comprehensive end-to-end test that validates the entire karaoke generation flow with REAL user interactions only. NO API shortcuts allowed.
|
||||
|
||||
## Overview
|
||||
|
||||
**Goal**: A single Playwright test that validates the complete karaoke generation journey from first-time user signup through to distribution, using only browser interactions like a real user would.
|
||||
|
||||
**Environment**: Production (`gen.nomadkaraoke.com` / `api.nomadkaraoke.com`)
|
||||
|
||||
**Automation**: Runs daily via GitHub Actions and can be triggered on-demand.
|
||||
|
||||
## Test Requirements
|
||||
|
||||
### 1. New User Registration (MailSlurp)
|
||||
|
||||
- [ ] Navigate to landing page (`gen.nomadkaraoke.com`)
|
||||
- [ ] Click "Sign In" or "Get Started" button
|
||||
- [ ] Create a NEW MailSlurp inbox (fresh email address for each test run)
|
||||
- [ ] Enter the test email in the beta enrollment form
|
||||
- [ ] Submit the form
|
||||
- [ ] Wait for welcome/magic-link email in MailSlurp inbox
|
||||
- [ ] Extract the magic link from the email
|
||||
- [ ] Click/navigate to the magic link to authenticate
|
||||
- [ ] Verify user is logged in and has credits
|
||||
|
||||
### 2. Create Karaoke Job (UI Only)
|
||||
|
||||
- [ ] Navigate to the karaoke creation page
|
||||
- [ ] Enter artist name: `piri` (test song with cached results)
|
||||
- [ ] Enter song title: `dog`
|
||||
- [ ] Click "Search" or equivalent button
|
||||
- [ ] Wait for search results to appear in the UI
|
||||
|
||||
### 3. Audio Selection (UI Only)
|
||||
|
||||
- [ ] Audio selection dialog/panel should appear
|
||||
- [ ] View available audio options in the UI
|
||||
- [ ] Click to select the first/best audio option
|
||||
- [ ] Confirm the selection via UI button
|
||||
- [ ] Wait for job to transition to processing state
|
||||
|
||||
### 4. Wait for Lyrics Transcription
|
||||
|
||||
- [ ] Monitor job status in the UI (not via API polling)
|
||||
- [ ] Wait for job to reach "awaiting lyrics review" state
|
||||
- [ ] This may take 10-15+ minutes for audio download + transcription
|
||||
|
||||
### 5. Lyrics Review (FULL UI Interaction)
|
||||
|
||||
> **CRITICAL**: This MUST be done through the UI, not via API calls
|
||||
|
||||
- [ ] Click to open the lyrics review interface
|
||||
- [ ] Verify lyrics are displayed in the editor
|
||||
- [ ] Interact with the lyrics editor:
|
||||
- [ ] View synced lyrics with timestamps
|
||||
- [ ] Optionally make a small edit to prove UI interaction works
|
||||
- [ ] Preview the video if preview functionality exists
|
||||
- [ ] Click "Approve" / "Submit" / "Complete Review" button in UI
|
||||
- [ ] Wait for job to transition to next state
|
||||
|
||||
### 6. Instrumental Selection (FULL UI Interaction)
|
||||
|
||||
> **CRITICAL**: This MUST be done through the UI, not via API calls
|
||||
|
||||
- [ ] Wait for instrumental options to become available
|
||||
- [ ] Instrumental selection dialog/panel should appear in UI
|
||||
- [ ] View available instrumental options (e.g., "clean", "with backing vocals")
|
||||
- [ ] Click to select the preferred instrumental option
|
||||
- [ ] Confirm selection via UI button
|
||||
- [ ] Wait for job to transition to video rendering state
|
||||
|
||||
### 7. Wait for Video Rendering & Encoding
|
||||
|
||||
- [ ] Monitor job progress in the UI
|
||||
- [ ] Wait for video rendering to complete
|
||||
- [ ] Wait for final encoding to complete
|
||||
- [ ] Job should reach "complete" status
|
||||
|
||||
### 8. Verify Completion & Downloads
|
||||
|
||||
- [ ] Verify job shows as "complete" in the UI
|
||||
- [ ] Verify download links are visible and clickable
|
||||
- [ ] Click at least one download link to verify it works
|
||||
- [ ] Verify the downloaded file is valid (non-zero size, correct format)
|
||||
|
||||
### 9. Distribution Verification & Cleanup
|
||||
|
||||
> **CRITICAL**: Test MUST clean up after itself to avoid polluting production distribution channels
|
||||
|
||||
- [ ] Verify job.state_data contains distribution results:
|
||||
- [ ] `brand_code` - The generated brand code (e.g., "NOMAD-1234")
|
||||
- [ ] `youtube_url` - YouTube video URL (if YouTube enabled)
|
||||
- [ ] `dropbox_link` - Dropbox shared link (if Dropbox enabled)
|
||||
- [ ] `gdrive_files` - Google Drive file IDs (if GDrive enabled)
|
||||
- [ ] **CLEANUP** - After verification, delete all distributed content:
|
||||
- [ ] Delete YouTube video via API (using stored youtube_url)
|
||||
- [ ] Delete Dropbox folder via API (using brand_code + dropbox_path)
|
||||
- [ ] Delete Google Drive files via API (using gdrive_files IDs)
|
||||
- [ ] Delete the test job from the system (GCS + Firestore)
|
||||
|
||||
## Technical Requirements
|
||||
|
||||
### Timeouts
|
||||
|
||||
| Stage | Timeout |
|
||||
|-------|---------|
|
||||
| Page navigation | 30 seconds |
|
||||
| Email arrival | 120 seconds |
|
||||
| Audio search | 60 seconds |
|
||||
| Lyrics transcription | 20 minutes |
|
||||
| Video rendering | 15 minutes |
|
||||
| Final encoding | 10 minutes |
|
||||
| **Total test** | **60 minutes** |
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
MAILSLURP_API_KEY=xxx # Required - for email testing
|
||||
# No pre-auth tokens - test handles full signup
|
||||
```
|
||||
|
||||
### Test Song
|
||||
|
||||
Use `piri - dog` because:
|
||||
- Has cached flacfetch results (faster audio search)
|
||||
- Known to work reliably
|
||||
- Short song = faster processing
|
||||
|
||||
### Screenshots & Artifacts
|
||||
|
||||
Capture screenshots at each major milestone:
|
||||
1. Landing page loaded
|
||||
2. Beta form submitted
|
||||
3. Email received
|
||||
4. Logged in successfully
|
||||
5. Job created
|
||||
6. Audio selected
|
||||
7. Lyrics review opened
|
||||
8. Lyrics approved
|
||||
9. Instrumental selected
|
||||
10. Job completed
|
||||
11. Download verified
|
||||
|
||||
### Logging
|
||||
|
||||
Log detailed progress at each step with timestamps for debugging.
|
||||
|
||||
## GitHub Actions Workflow
|
||||
|
||||
**File**: `.github/workflows/e2e-happy-path.yml`
|
||||
|
||||
### Triggers
|
||||
|
||||
1. **Daily schedule**: Run at 6 AM UTC every day
|
||||
2. **Manual dispatch**: Can be triggered on-demand with optional parameters
|
||||
3. **Label trigger**: When `e2e-full` label added to PR
|
||||
|
||||
### Secrets Required
|
||||
|
||||
- `MAILSLURP_API_KEY` - For email testing
|
||||
|
||||
### Workflow Steps
|
||||
|
||||
1. Checkout code
|
||||
2. Setup Node.js
|
||||
3. Install dependencies
|
||||
4. Install Playwright browsers
|
||||
5. Run the e2e happy path test
|
||||
6. Upload artifacts (screenshots, videos, logs)
|
||||
7. Report results
|
||||
|
||||
### Success Criteria
|
||||
|
||||
- All UI interactions complete successfully
|
||||
- Job reaches "complete" status
|
||||
- Download link works
|
||||
- No JavaScript errors in console
|
||||
- All screenshots captured
|
||||
|
||||
## What This Test Proves
|
||||
|
||||
When this test passes, it proves:
|
||||
|
||||
1. **User signup works** - New users can register via magic link
|
||||
2. **Authentication works** - Email verification and login flow functional
|
||||
3. **Job creation works** - Can search and create karaoke jobs
|
||||
4. **Audio pipeline works** - Audio search, selection, and download functional
|
||||
5. **Transcription works** - Lyrics transcription completes successfully
|
||||
6. **Lyrics review UI works** - Users can review and approve lyrics
|
||||
7. **Instrumental selection UI works** - Users can select instrumental versions
|
||||
8. **Video rendering works** - Full video generation pipeline functional
|
||||
9. **Downloads work** - Users can download their completed karaoke videos
|
||||
10. **End-to-end integration** - All systems work together correctly
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
1. **NO API calls to complete review** - Must click buttons in UI
|
||||
2. **NO API calls to select instrumental** - Must click buttons in UI
|
||||
3. **NO pre-authenticated tokens** - Must go through full signup
|
||||
4. **NO skipping steps** - Every user-facing interaction must happen
|
||||
5. **NO hardcoded waits** - Use proper Playwright waiting strategies
|
||||
|
||||
## File Location
|
||||
|
||||
**Test file**: `frontend/e2e/production/happy-path-real-user.spec.ts`
|
||||
|
||||
This is separate from the existing `full-user-journey.spec.ts` which uses API shortcuts.
|
||||
|
||||
## Iteration Plan
|
||||
|
||||
1. ~~Create initial test with all steps stubbed~~
|
||||
2. ~~Implement each section one at a time~~
|
||||
3. ~~Run via GHA workflow~~
|
||||
4. ~~Debug failures using artifacts~~
|
||||
5. ~~Iterate until fully passing~~ ✅ Completed 2026-01-02
|
||||
6. Add to required CI checks once stable
|
||||
|
||||
## Key Fixes Applied
|
||||
|
||||
During iteration, three issues were fixed:
|
||||
|
||||
1. **STEP 8 status detection**: "rendering" status comes BEFORE instrumental selection, not after. Changed to only skip on "encoding".
|
||||
2. **Instrumental page navigation**: Changed `waitUntil: 'networkidle'` to `'load'` because audio players prevent networkidle.
|
||||
3. **Download verification**: Changed HEAD to GET request with `maxRedirects: 0` since download endpoint doesn't support HEAD.
|
||||
|
||||
See [LESSONS-LEARNED.md](LESSONS-LEARNED.md) for details.
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-01-02*
|
||||
*Author: Claude (via Andrew's request)*
|
||||
@@ -4,6 +4,25 @@ Key insights for future AI agents working on this codebase.
|
||||
|
||||
## Architecture Decisions
|
||||
|
||||
### Separate Collections for Multi-App Projects
|
||||
|
||||
**Problem**: karaoke-gen and karaoke-decide shared the same `users` Firestore collection with incompatible schemas. karaoke-gen users have `credits`, `role`, `is_active`; karaoke-decide users have `user_id`, `is_guest`, `quiz_*` fields.
|
||||
|
||||
**Symptoms**:
|
||||
- Admin dashboard showed users from both apps mixed together
|
||||
- Potential for data corruption if wrong app writes to wrong user
|
||||
- Hard to run collection-wide queries (indexes, aggregations) for one app
|
||||
|
||||
**Solution**: Migrate karaoke-gen to dedicated `gen_users` collection. Created migration script that:
|
||||
1. Identified karaoke-gen users by schema (has `credits` + `role` + `is_active`, lacks `user_id` + `is_guest`)
|
||||
2. Copied 42 users to `gen_users` with same document IDs (email addresses)
|
||||
3. Updated Pulumi indexes from `users` to `gen_users`
|
||||
4. Left karaoke-decide's `users` collection untouched
|
||||
|
||||
**Key insight**: When multiple apps share a GCP project, use distinct collection names from day one. Prefixing collections with app name (e.g., `gen_users`, `decide_users`) avoids schema conflicts and makes ownership clear.
|
||||
|
||||
**Migration script**: `scripts/migrate_users_to_gen_users.py` - supports `--dry-run` and `--delete` flags.
|
||||
|
||||
### LyricsTranscriber: Library, Not Server
|
||||
|
||||
**Problem**: LyricsTranscriber's `ReviewServer` blocks waiting for human input - incompatible with async cloud architecture.
|
||||
@@ -32,8 +51,75 @@ Audio separation and lyrics transcription run in parallel:
|
||||
- Progress tracked separately in `state_data`
|
||||
- Screens generation waits for both to complete
|
||||
|
||||
### Cloud Run Performance: Cold Starts and FFmpeg
|
||||
|
||||
**Problem**: Cloud karaoke generation was ~3x slower than local CLI (15 min vs 5 min for a typical song).
|
||||
|
||||
**Root causes identified**:
|
||||
|
||||
1. **Cold start penalty**: Cloud Run with `min-instances: 0` caused 77-second cold starts for FFmpeg-heavy operations (title screen generation). Warm instances take only 6 seconds.
|
||||
|
||||
2. **FFmpeg version**: Debian's package (5.1.8) is outdated and built with security-focused `--toolchain=hardened` which reduces performance. Static builds (7.0.2) from johnvansickle.com are ~20-40% faster.
|
||||
|
||||
3. **Generic x86_64 build**: Debian packages use generic CPU instructions, not optimized for Cloud Run's Cascade Lake/Ice Lake processors.
|
||||
|
||||
**Solutions applied**:
|
||||
- Set `min-instances: 4` to keep workers warm (eliminates cold start entirely)
|
||||
- Use John Van Sickle's static FFmpeg 7.0.2 instead of Debian package
|
||||
- Added `/api/health/detailed` endpoint to expose FFmpeg version for debugging
|
||||
|
||||
**Key insight**: For CPU-bound encoding workloads on Cloud Run:
|
||||
- GPU acceleration often doesn't help (filters like libass aren't GPU-accelerated)
|
||||
- Newer FFmpeg versions have significant encoding optimizations
|
||||
- Static builds with CPU-specific tuning outperform generic distribution packages
|
||||
- Cold starts hurt more than expected for initialization-heavy binaries like FFmpeg
|
||||
|
||||
**Debugging**: Use `scripts/compare_local_vs_remote.py` to benchmark local vs cloud performance.
|
||||
|
||||
## Common Gotchas
|
||||
|
||||
### Cross-Domain localStorage Isolation
|
||||
|
||||
**Problem**: Auth tokens stored in localStorage on one subdomain are invisible to other subdomains.
|
||||
|
||||
We had `buy.nomadkaraoke.com` for purchases and `gen.nomadkaraoke.com` for the app. Users who purchased credits on `buy.` appeared logged out when redirected to `gen.` because localStorage is domain-isolated.
|
||||
|
||||
**Symptoms**:
|
||||
- User completes purchase, redirected to app, sees "auth required"
|
||||
- Token exists in browser dev tools for one domain but not the other
|
||||
- Works fine in local dev (same localhost origin)
|
||||
|
||||
**Solution**: Keep auth on a single domain. We consolidated `buy-site/` into the main `frontend/`:
|
||||
- Landing/pricing at `/` (root)
|
||||
- Main app at `/app`
|
||||
- Single localStorage for all auth tokens
|
||||
|
||||
**Alternative** (if subdomains required): Use cookies with `domain=.nomadkaraoke.com` instead of localStorage, or implement a token exchange flow.
|
||||
|
||||
**Lesson**: Before splitting functionality across subdomains, verify your auth storage strategy works cross-domain. E2E tests that exercise the full user journey catch this; unit tests don't.
|
||||
|
||||
See `docs/archive/2025-12-29-buy-site-consolidation.md` for full details.
|
||||
|
||||
### Frontend Polling vs Tab Visibility
|
||||
|
||||
**Problem**: After completing review in a new tab (lyrics or instrumental selection), returning to the main frontend showed stale status for up to 10 seconds due to polling interval.
|
||||
|
||||
**Solution**: Use `visibilitychange` event to trigger immediate refresh when tab becomes visible:
|
||||
|
||||
```typescript
|
||||
useEffect(() => {
|
||||
const handleVisibilityChange = () => {
|
||||
if (document.visibilityState === 'visible') {
|
||||
loadJobs() // Immediate refresh
|
||||
}
|
||||
}
|
||||
document.addEventListener('visibilitychange', handleVisibilityChange)
|
||||
return () => document.removeEventListener('visibilitychange', handleVisibilityChange)
|
||||
}, [loadJobs])
|
||||
```
|
||||
|
||||
**Key insight**: Backend status updates are fast (~1 second). If UI appears slow to update after user action in another tab, the issue is likely frontend polling, not backend.
|
||||
|
||||
### Pydantic Model Fields
|
||||
|
||||
**Problem**: Setting a field that doesn't exist in the Pydantic model silently does nothing.
|
||||
@@ -94,8 +180,136 @@ if job.input_media_gcs_path:
|
||||
await trigger_worker()
|
||||
```
|
||||
|
||||
### Stale Object References After Async Operations
|
||||
|
||||
**Problem**: Fetching a job object, then calling async functions that update the job in the database, then using the original object's data to make another update - this overwrites the async function's changes.
|
||||
|
||||
```python
|
||||
# BAD - loses distribution results
|
||||
job = job_manager.get_job(job_id) # Fetch job
|
||||
await _handle_native_distribution(...) # Updates job.state_data in DB
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {**job.state_data, 'foo': 'bar'} # STALE! Overwrites dropbox_link!
|
||||
})
|
||||
```
|
||||
|
||||
**Solution**: Either re-fetch the job, or include async results explicitly:
|
||||
```python
|
||||
# GOOD - include results from async operation
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
'dropbox_link': result.get('dropbox_link'), # From distribution
|
||||
'gdrive_files': result.get('gdrive_files'),
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
### Google Drive API Query Escaping
|
||||
|
||||
**Problem**: Single quotes in filenames break Google Drive API queries.
|
||||
|
||||
```python
|
||||
# BAD - fails for "Nobody's Home"
|
||||
query = f"name='{filename}' and '{parent_id}' in parents"
|
||||
# Results in: name='NOMAD-1187 - Artist - Nobody's Home.mp4' → 400 Invalid Value
|
||||
```
|
||||
|
||||
**Solution**: Escape single quotes with backslash:
|
||||
```python
|
||||
# GOOD
|
||||
escaped = filename.replace("'", "\\'")
|
||||
query = f"name='{escaped}' and '{parent_id}' in parents"
|
||||
```
|
||||
|
||||
### Duplicate Code Leads to Inconsistent Behavior
|
||||
|
||||
**Problem**: Multiple endpoints computing the same defaults independently. `audio_search.py` had `effective_brand_prefix = body.brand_prefix or settings.default_brand_prefix` but `file_upload.py` was missing it for some endpoints.
|
||||
|
||||
**Result**: Jobs created via audio search had correct brand codes; jobs created via file upload had null brand_prefix, causing Dropbox uploads to fail and Google Drive uploads to use placeholder "TRACK-0000".
|
||||
|
||||
**Solution**: Added `_get_effective_distribution_settings()` helper function in `file_upload.py` to centralize default computation:
|
||||
```python
|
||||
@dataclass
|
||||
class EffectiveDistributionSettings:
|
||||
dropbox_path: Optional[str]
|
||||
gdrive_folder_id: Optional[str]
|
||||
discord_webhook_url: Optional[str]
|
||||
brand_prefix: Optional[str]
|
||||
|
||||
def _get_effective_distribution_settings(...) -> EffectiveDistributionSettings:
|
||||
settings = get_settings()
|
||||
return EffectiveDistributionSettings(
|
||||
dropbox_path=dropbox_path or settings.default_dropbox_path,
|
||||
# ... etc
|
||||
)
|
||||
```
|
||||
|
||||
**Lesson**: When you see the same logic repeated in multiple places, refactor into a helper function immediately. Otherwise bugs will be fixed in one place but not others.
|
||||
|
||||
### Field Name Mismatches Between Endpoints
|
||||
|
||||
**Problem**: Different endpoints setting different field names for the same logical value, where consumers check only one field name.
|
||||
|
||||
Example: `youtube_description` vs `youtube_description_template`
|
||||
- `audio_search.py` endpoint set `youtube_description_template`
|
||||
- `file_upload.py` endpoint set only `youtube_description`
|
||||
- `video_worker.py` checked `youtube_description_template`
|
||||
- Result: YouTube uploads silently failed for remote CLI jobs
|
||||
|
||||
**Symptoms**:
|
||||
- Feature works via one code path but silently fails via another
|
||||
- No errors in logs (the field is just empty/None)
|
||||
- Difficult to debug because both paths appear correct in isolation
|
||||
|
||||
**Solution**: When adding fields that control behavior:
|
||||
1. Grep for all places that read the field
|
||||
2. Grep for all places that write the field
|
||||
3. Ensure all writers set what all readers expect
|
||||
|
||||
**Lesson**: Silent failures from field name mismatches are hard to catch. When a feature works in one flow but not another, check if different endpoints are setting different field names.
|
||||
|
||||
See `docs/archive/2025-12-30-cloud-output-structure-fix.md` for full details.
|
||||
|
||||
## Testing Insights
|
||||
|
||||
### E2E Mock Responses Must Match API Types Exactly
|
||||
|
||||
**Problem**: E2E tests with mocked API responses were failing silently because mock response format didn't match the TypeScript type the app expected.
|
||||
|
||||
The test mocked `/api/users/me` returning:
|
||||
```javascript
|
||||
{ body: { email: 'test@example.com', credits: 5 } }
|
||||
```
|
||||
|
||||
But `UserProfileResponse` type expects:
|
||||
```typescript
|
||||
interface UserProfileResponse {
|
||||
user: UserPublic; // { email, credits, role }
|
||||
has_session: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
**Symptoms**:
|
||||
- Tests passed initial auth check but `/api/jobs` was never called
|
||||
- Mock server showed only 2 of 4 endpoints being called
|
||||
- App redirected to `/` instead of loading job list
|
||||
- No error messages (silent failure)
|
||||
|
||||
**Root cause**: The auth module's `fetchUser()` ran at page load and tried to access `response.user.email`. With the wrong mock format, this threw an error, the catch block called `clearAccessToken()`, and the app page saw no token and redirected.
|
||||
|
||||
**Solution**: Mock must return the exact structure the TypeScript type expects:
|
||||
```javascript
|
||||
{
|
||||
body: {
|
||||
user: { email: 'test@example.com', credits: 5, role: 'user' },
|
||||
has_session: true,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
**Lesson**: When writing E2E test mocks, check the TypeScript interface/type definition for the API response, not just what fields the UI displays. Silent failures from type mismatches are hard to debug.
|
||||
|
||||
### Emulator Tests Catch Real Bugs
|
||||
|
||||
Unit tests with mocks didn't catch the `input_media_gcs_path` bug. Emulator integration tests did because they use real Firestore behavior.
|
||||
@@ -106,6 +320,96 @@ Unit tests with mocks didn't catch the `input_media_gcs_path` bug. Emulator inte
|
||||
|
||||
Emulator tests need their own `conftest.py` that does NOT mock `google.cloud.*`. Mocks from unit tests can leak into integration tests.
|
||||
|
||||
### Playwright: Use Assertions, Not Timeouts
|
||||
|
||||
**Problem**: `waitForTimeout()` is flaky and slow:
|
||||
|
||||
```typescript
|
||||
// BAD - arbitrary wait, still flaky
|
||||
await page.waitForTimeout(2000);
|
||||
const hasContent = await page.locator('.content').count();
|
||||
```
|
||||
|
||||
**Solution**: Use Playwright's auto-retry assertions:
|
||||
|
||||
```typescript
|
||||
// GOOD - waits only until condition met, retries automatically
|
||||
await expect(page.getByText('Hello,')).toBeVisible({ timeout: 5000 });
|
||||
```
|
||||
|
||||
**Why it matters**: Playwright assertions have built-in retry logic and fail fast when conditions aren't met. Arbitrary timeouts either waste time (waiting when condition is already met) or cause flaky tests (not waiting long enough on slow CI).
|
||||
|
||||
### Playwright: Use 'load' Instead of 'networkidle' for Audio-Heavy Pages
|
||||
|
||||
**Problem**: Pages with audio players or ongoing audio network requests never reach `networkidle` state.
|
||||
|
||||
```typescript
|
||||
// BAD - hangs indefinitely on pages with audio players
|
||||
await page.goto(url, { waitUntil: 'networkidle' });
|
||||
```
|
||||
|
||||
**Symptom**: Test times out after 30-45 minutes waiting for navigation to complete, even though the page is fully rendered and usable.
|
||||
|
||||
**Solution**: Use `waitUntil: 'load'` plus a small fixed wait if needed:
|
||||
|
||||
```typescript
|
||||
// GOOD - works with audio-heavy pages
|
||||
await page.goto(url, { waitUntil: 'load' });
|
||||
await page.waitForTimeout(3000); // Allow JS to initialize
|
||||
```
|
||||
|
||||
**Root cause**: Audio players make continuous network requests for streaming/buffering, preventing the network from ever becoming "idle".
|
||||
|
||||
### Playwright: HEAD Requests May Not Be Supported
|
||||
|
||||
**Problem**: Using `page.request.head()` to verify download URLs returns 405 Method Not Allowed on some endpoints.
|
||||
|
||||
```typescript
|
||||
// BAD - many download endpoints don't support HEAD
|
||||
const response = await page.request.head(downloadUrl);
|
||||
// Returns 405 instead of 200
|
||||
```
|
||||
|
||||
**Solution**: Use GET with `maxRedirects: 0` to check accessibility without downloading:
|
||||
|
||||
```typescript
|
||||
// GOOD - works with all endpoints, doesn't follow redirects
|
||||
const response = await page.request.get(downloadUrl, { maxRedirects: 0 });
|
||||
expect(response.status() >= 200 && response.status() < 400).toBe(true);
|
||||
```
|
||||
|
||||
**Why maxRedirects: 0**: Download URLs often redirect to cloud storage (GCS, S3). A 302/307 redirect means the URL is valid. Following the redirect would download the entire file.
|
||||
|
||||
### E2E Tests: Understand Job State Machine Flow
|
||||
|
||||
**Problem**: E2E test skipped instrumental selection because it checked for "rendering" in status, but rendering happens BEFORE instrumental selection, not after.
|
||||
|
||||
```typescript
|
||||
// BAD - assumes rendering means instrumental is done
|
||||
if (status.includes('rendering') || status.includes('encoding')) {
|
||||
console.log('Past instrumental stage');
|
||||
break;
|
||||
}
|
||||
```
|
||||
|
||||
**Actual job flow**:
|
||||
```text
|
||||
downloading → transcribing → in_review → rendering_video →
|
||||
awaiting_instrumental_selection → encoding → complete
|
||||
```
|
||||
|
||||
**Solution**: Only skip on statuses that are definitively past the target stage:
|
||||
|
||||
```typescript
|
||||
// GOOD - encoding is definitely past instrumental
|
||||
if (status.includes('encoding') && !status.includes('awaiting')) {
|
||||
console.log('Past instrumental stage');
|
||||
break;
|
||||
}
|
||||
```
|
||||
|
||||
**Lesson**: Before writing status-checking logic, trace through the actual state machine to understand which states come before/after your target state.
|
||||
|
||||
## Deployment Notes
|
||||
|
||||
### Cloud Run Timeouts
|
||||
@@ -115,10 +419,155 @@ Default 5-minute timeout may not be enough for video encoding. Consider:
|
||||
- Breaking encoding into smaller chunks
|
||||
- Increasing timeout for specific operations
|
||||
|
||||
### Cloud Tasks dispatch_deadline (Separate from Cloud Run Timeout)
|
||||
|
||||
**Problem**: Cloud Tasks HTTP handlers have a separate `dispatch_deadline` (default 10 minutes, max 30 minutes) that controls how long Cloud Tasks waits for the HTTP response. This is DIFFERENT from Cloud Run's service timeout.
|
||||
|
||||
**Symptom**: Worker killed silently after 10 minutes even though Cloud Run timeout was set to 30 minutes. Job appears stuck with no error.
|
||||
|
||||
**Solution**: Set explicit `dispatch_deadline` when creating Cloud Tasks:
|
||||
```python
|
||||
from google.protobuf import duration_pb2
|
||||
|
||||
task = {
|
||||
"http_request": {...},
|
||||
"dispatch_deadline": duration_pb2.Duration(seconds=1800), # 30 min max
|
||||
}
|
||||
```
|
||||
|
||||
**Key insight**: For tasks that might take >30 minutes, Cloud Tasks HTTP targets won't work. Use Cloud Run Jobs instead (supports up to 24-hour execution).
|
||||
|
||||
| Timeout Type | Default | Max | What It Controls |
|
||||
|-------------|---------|-----|------------------|
|
||||
| Cloud Tasks `dispatch_deadline` | 10 min | 30 min | How long Cloud Tasks waits for HTTP response |
|
||||
| Cloud Run service timeout | 5 min | 60 min | How long a single HTTP request can run |
|
||||
| Cloud Run Jobs timeout | 10 min | 24 hr | How long a batch job execution can run |
|
||||
|
||||
See `docs/archive/2026-01-01-worker-timeout-fixes.md` for full details.
|
||||
|
||||
### Two-Layer Timeout Pattern for Long-Running Loops
|
||||
|
||||
**Problem**: A loop processing many items (e.g., 74 LLM calls at 15s each = 18+ minutes) can block indefinitely. Single outer timeout isn't enough because it may interrupt mid-operation and leave state inconsistent.
|
||||
|
||||
**Solution**: Use two layers of timeout protection:
|
||||
|
||||
```python
|
||||
# Layer 1: Inner deadline check (cooperative, clean exit)
|
||||
deadline = time.time() + timeout_seconds
|
||||
for item in items:
|
||||
if time.time() > deadline:
|
||||
logger.warning("Deadline exceeded, breaking early")
|
||||
break # Exit gracefully with partial/no results
|
||||
process(item)
|
||||
|
||||
# Layer 2: Outer asyncio.wait_for (safety net for hung operations)
|
||||
result = await asyncio.wait_for(
|
||||
asyncio.to_thread(run_loop_with_deadline),
|
||||
timeout=timeout_seconds + 60 # Buffer for cleanup
|
||||
)
|
||||
```
|
||||
|
||||
**Why both layers**:
|
||||
- Inner check: Handles normal case (many items). Exits cleanly between iterations.
|
||||
- Outer timeout: Catches edge case (single operation hangs). Hard stop if inner check never runs.
|
||||
|
||||
**Design choice**: Use `break` not `raise` for inner timeout. This returns partial/uncorrected results rather than failing the job, allowing downstream human review to fix issues.
|
||||
|
||||
See `docs/archive/2025-12-31-agentic-timeout-implementation.md` for implementation details.
|
||||
|
||||
### Secret Manager Access
|
||||
|
||||
Cloud Run service account needs `Secret Manager Secret Accessor` role. This is managed via Pulumi in `infrastructure/`.
|
||||
|
||||
### Cloud Run Secret Mounting
|
||||
|
||||
**Problem**: Secrets created in Secret Manager aren't automatically available as environment variables in Cloud Run. The IAM role grants *access*, but secrets must be explicitly *mounted*.
|
||||
|
||||
**Wrong assumption**: "I added the secret to Secret Manager and gave the service account access, so the secret is available."
|
||||
|
||||
**Reality**: Cloud Run needs `--set-secrets` in the deploy command:
|
||||
```bash
|
||||
gcloud run deploy service-name \
|
||||
--set-secrets "ENV_VAR_NAME=secret-name:latest"
|
||||
```
|
||||
|
||||
**Symptoms of missing mount**: Service silently falls back to defaults (e.g., email service uses console logging instead of SendGrid).
|
||||
|
||||
**Solution**: Add all required secrets to the CI deploy command (`.github/workflows/ci.yml`). Manual `gcloud run services update` fixes are overwritten on next deploy.
|
||||
|
||||
### Vertex AI Authentication
|
||||
|
||||
For Gemini via Vertex AI, use Application Default Credentials (ADC) rather than API keys:
|
||||
- **Cloud Run**: Service account automatically authenticated - just grant `roles/aiplatform.user`
|
||||
- **Local dev**: Run `gcloud auth application-default login`
|
||||
- **Config**: Set `GOOGLE_CLOUD_PROJECT` and optionally `GCP_LOCATION` (defaults to `global`)
|
||||
|
||||
This approach uses GCP's free credits and existing IAM rather than separate API key management.
|
||||
|
||||
### Gemini 3 Models Require Global Location
|
||||
|
||||
**Problem**: Gemini 3 Flash (`gemini-3-flash-preview`) returns 404 errors when using regional Vertex AI endpoints.
|
||||
|
||||
```
|
||||
404 Publisher Model `projects/PROJECT/locations/us-central1/publishers/google/models/gemini-3-flash-preview` was not found
|
||||
```
|
||||
|
||||
**Solution**: Use `global` location instead of regional endpoints like `us-central1`:
|
||||
```python
|
||||
gcp_location: str = "global" # Required for Gemini 3 models
|
||||
```
|
||||
|
||||
**Additional gotcha**: Gemini 3 returns multimodal response format:
|
||||
```python
|
||||
# Gemini 2: response.content = "text"
|
||||
# Gemini 3: response.content = [{'type': 'text', 'text': '...'}]
|
||||
```
|
||||
|
||||
Code must handle both formats when invoking LangChain ChatVertexAI.
|
||||
|
||||
See `docs/archive/2025-12-30-gemini3-agentic-correction-fix.md` for full details.
|
||||
|
||||
### LangChain gRPC vs REST Providers
|
||||
|
||||
**Problem**: `langchain-google-vertexai` uses gRPC which can hang indefinitely during connection establishment with no timeout, causing cloud jobs to freeze silently.
|
||||
|
||||
**Symptoms**:
|
||||
- Job hangs at "Initializing model..." with no further logs
|
||||
- No timeout or error - just waits forever
|
||||
- Hard to diagnose because no exception is raised
|
||||
|
||||
**Solution**: Use `langchain-google-genai` (REST-based) instead. This package supports BOTH Vertex AI (service account auth) and Google AI Studio (API key auth):
|
||||
```python
|
||||
# Before (gRPC, can hang)
|
||||
from langchain_google_vertexai import ChatVertexAI
|
||||
model = ChatVertexAI(model=model_name, project=project)
|
||||
|
||||
# After (REST, reliable) - Using Vertex AI with service account/ADC
|
||||
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||
model = ChatGoogleGenerativeAI(model=model_name, project=project_id) # Uses ADC
|
||||
|
||||
# Alternative - Using Google AI Studio with API key
|
||||
model = ChatGoogleGenerativeAI(model=model_name, google_api_key=api_key)
|
||||
```
|
||||
|
||||
**Key insight**: When `project` parameter is provided, `ChatGoogleGenerativeAI` automatically uses the Vertex AI backend with Application Default Credentials (ADC). On Cloud Run, this uses the attached service account - no API key needed.
|
||||
|
||||
**Additional protection**: Wrap model initialization in ThreadPoolExecutor timeout:
|
||||
```python
|
||||
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
|
||||
|
||||
with ThreadPoolExecutor(max_workers=1) as executor:
|
||||
future = executor.submit(create_model)
|
||||
try:
|
||||
model = future.result(timeout=30) # Fail fast
|
||||
except FuturesTimeoutError:
|
||||
raise InitializationTimeoutError("Model init timed out")
|
||||
```
|
||||
|
||||
**Why ThreadPoolExecutor**: Works cross-platform (Windows, Linux, macOS). Python's `signal.alarm` only works on Unix.
|
||||
|
||||
See `docs/archive/2025-12-30-langchain-vertexai-to-genai-migration.md` for full details.
|
||||
|
||||
### CI/CD: GitHub Actions
|
||||
|
||||
Migrated from Cloud Build to GitHub Actions for:
|
||||
@@ -139,6 +588,51 @@ GitHub-hosted runners have ~14GB disk, which caused failures during Docker build
|
||||
|
||||
See `docs/archive/2025-12-28-self-hosted-github-runner.md` for full details.
|
||||
|
||||
### Docker Disk Space Management on Self-Hosted Runners
|
||||
|
||||
**Problem**: Even with 200GB SSDs, self-hosted runners can fill up with Docker images. Each CI build creates new dangling images (~15GB each), and without aggressive cleanup, disk fills in days.
|
||||
|
||||
**Symptoms**:
|
||||
- CI jobs fail with `No space left on device`
|
||||
- Jobs that require Docker (build, deploy, emulator tests) fail
|
||||
- Runners 11-20 at 100% while 1-10 at 30% (newer runners had less cleanup history)
|
||||
|
||||
**Root cause**: Original cleanup cron used `docker system prune -af --filter "until=168h"` which only removes images older than 7 days. With frequent CI builds, most dangling images are < 7 days old and never get cleaned.
|
||||
|
||||
**Solution** (two-layer approach):
|
||||
|
||||
1. **Hourly threshold-based cleanup** (on runners):
|
||||
```bash
|
||||
# /etc/cron.hourly/docker-cleanup
|
||||
THRESHOLD=70
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
if [ "$USAGE" -gt "$THRESHOLD" ]; then
|
||||
docker system prune -af # No age filter!
|
||||
docker builder prune -af
|
||||
fi
|
||||
```
|
||||
|
||||
2. **Pre-job disk check** (in CI workflow):
|
||||
```yaml
|
||||
- name: Check disk space
|
||||
run: |
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
if [ "$USAGE" -gt 70 ]; then
|
||||
docker system prune -af || true
|
||||
docker builder prune -af || true
|
||||
fi
|
||||
if [ "$USAGE" -gt 85 ]; then
|
||||
echo "::error::Disk usage exceeds 85%. Runner needs maintenance."
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
**Key insights**:
|
||||
- Remove the age filter (`--filter "until=168h"`) - dangling images should always be cleaned
|
||||
- Threshold-based cleanup is better than time-based (don't clean if disk is fine)
|
||||
- Pre-job checks provide fail-fast behavior with clear error messages
|
||||
- `docker builder prune -af` catches buildkit cache that `system prune` misses
|
||||
|
||||
## Performance Observations
|
||||
|
||||
| Operation | Duration |
|
||||
@@ -153,6 +647,253 @@ See `docs/archive/2025-12-28-self-hosted-github-runner.md` for full details.
|
||||
|
||||
Human review time varies (5-15 min typical).
|
||||
|
||||
### Gitignore Patterns for Monorepos
|
||||
|
||||
**Problem**: Root `.gitignore` had `lib/` to ignore Python lib directories, but this also ignored `buy-site/lib/` (frontend code).
|
||||
|
||||
**Solution**: Use negation patterns for frontend directories:
|
||||
```gitignore
|
||||
# Python lib directories
|
||||
lib/
|
||||
lib64/
|
||||
# But allow frontend lib directories
|
||||
!frontend/lib/
|
||||
!buy-site/lib/
|
||||
```
|
||||
|
||||
**Lesson**: When adding gitignore patterns, consider their effect on subdirectories, especially in monorepos with multiple languages.
|
||||
|
||||
### Firestore Transactions for Race Conditions
|
||||
|
||||
**Problem**: Magic link verification had a race condition - concurrent requests could both verify the same token before it was marked as used.
|
||||
|
||||
**Solution**: Use Firestore transactions for atomic read-check-update operations:
|
||||
```python
|
||||
@firestore.transactional
|
||||
def verify_in_transaction(transaction):
|
||||
doc = doc_ref.get(transaction=transaction)
|
||||
# Check conditions...
|
||||
transaction.update(doc_ref, {'used': True})
|
||||
```
|
||||
|
||||
### Firestore Returns Timezone-Aware Datetimes
|
||||
|
||||
**Problem**: Firestore stores datetimes as timezone-aware (UTC). Code using `datetime.utcnow()` (naive) for comparisons fails with `TypeError: can't compare offset-naive and offset-aware datetimes`.
|
||||
|
||||
**Symptoms**:
|
||||
- Session validation fails silently (exception caught, returns error)
|
||||
- Session exists and appears valid in database
|
||||
- First request after login may work, subsequent requests fail (because `last_activity_at` is updated to TZ-aware on first access)
|
||||
|
||||
**Solution**: Always use timezone-aware datetimes:
|
||||
```python
|
||||
from datetime import datetime, timezone
|
||||
|
||||
# GOOD
|
||||
now = datetime.now(timezone.utc)
|
||||
|
||||
# Normalize Firestore datetimes that might be naive (legacy data)
|
||||
if session.expires_at.tzinfo is None:
|
||||
expires_at = session.expires_at.replace(tzinfo=timezone.utc)
|
||||
```
|
||||
|
||||
**Note**: `datetime.utcnow()` is deprecated in Python 3.12+. Use `datetime.now(timezone.utc)` instead.
|
||||
|
||||
See `docs/archive/2025-12-30-auth-session-persistence-fix.md` for full details.
|
||||
|
||||
### Frontend Module-Level State in Next.js
|
||||
|
||||
**Problem**: Caching values in module-level variables can cause stale state in Next.js due to module caching and hydration timing.
|
||||
|
||||
```javascript
|
||||
// BAD - may return stale value
|
||||
let accessToken = localStorage.getItem('key');
|
||||
export function getToken() { return accessToken; }
|
||||
|
||||
// GOOD - always fresh
|
||||
export function getToken() {
|
||||
if (typeof window !== 'undefined') {
|
||||
return localStorage.getItem('key');
|
||||
}
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
**Lesson**: For values that can change (auth tokens, user preferences), read fresh from storage on each access rather than caching in module scope.
|
||||
|
||||
### GCP Organization Policies Block Service Account Keys
|
||||
|
||||
**Problem**: GCP organization policies can block service account key creation even when you have Owner access to the project.
|
||||
|
||||
```
|
||||
ERROR: Key creation is not allowed on this service account.
|
||||
violations: constraints/iam.disableServiceAccountKeyCreation
|
||||
```
|
||||
|
||||
**Solution**: If you have Organization Policy Administrator role, temporarily disable the constraint:
|
||||
|
||||
```bash
|
||||
# Disable at organization level (find org ID with: gcloud projects describe PROJECT --format="value(parent)")
|
||||
cat > /tmp/policy.yaml << 'EOF'
|
||||
name: organizations/ORG_ID/policies/iam.disableServiceAccountKeyCreation
|
||||
spec:
|
||||
rules:
|
||||
- enforce: false
|
||||
EOF
|
||||
gcloud org-policies set-policy /tmp/policy.yaml
|
||||
|
||||
# Wait ~30 seconds for propagation
|
||||
sleep 30
|
||||
|
||||
# Create key
|
||||
gcloud iam service-accounts keys create key.json --iam-account=SA_EMAIL
|
||||
|
||||
# Re-enable immediately
|
||||
cat > /tmp/policy.yaml << 'EOF'
|
||||
name: organizations/ORG_ID/policies/iam.disableServiceAccountKeyCreation
|
||||
spec:
|
||||
rules:
|
||||
- enforce: true
|
||||
EOF
|
||||
gcloud org-policies set-policy /tmp/policy.yaml
|
||||
|
||||
# Clean up
|
||||
rm /tmp/policy.yaml key.json # After copying key contents
|
||||
```
|
||||
|
||||
**Note**: Also check for the newer managed constraint `iam.managed.disableServiceAccountKeyCreation` and disable both if needed.
|
||||
|
||||
**Lesson**: Service account keys are a security risk and should be avoided when possible. Use Workload Identity Federation instead. But for third-party services that require JSON keys (like LangFuse), this temporary policy override is the safest approach.
|
||||
|
||||
### Cloud Run Horizontal Scaling and In-Memory Cache
|
||||
|
||||
**Problem**: In-memory caches (singletons, module-level dicts) don't persist across Cloud Run instances. When scaling horizontally, request 1 may hit instance A and populate the cache, but request 2 may hit instance B which has an empty cache.
|
||||
|
||||
**Example**: `AudioSearchService` cached search results in `self._cached_results`. Search hit instance A, but download hit instance B:
|
||||
```
|
||||
Error: No cached result for index 0. Available indices: 0--1. Run search() first.
|
||||
```
|
||||
|
||||
**Solution**: Never rely on in-memory state across requests. Persist to Firestore/GCS:
|
||||
```python
|
||||
# BAD - cache lost if next request hits different instance
|
||||
self._cached_results = search_results
|
||||
result = self._cached_results[selection_index]
|
||||
|
||||
# GOOD - persist in job state_data
|
||||
job_manager.update_job(job_id, {'state_data': {'search_results': results}})
|
||||
# Later...
|
||||
job = job_manager.get_job(job_id)
|
||||
result = job.state_data['search_results'][selection_index]
|
||||
```
|
||||
|
||||
**Lesson**: Design for stateless request handling. Any state that must survive between requests must be persisted externally.
|
||||
|
||||
### GCS Downloads Preserve Directory Structure
|
||||
|
||||
**Problem**: When downloading files from GCS, the directory structure is preserved. Code that uses non-recursive glob patterns won't find files in subdirectories.
|
||||
|
||||
**Example**: GCE encoding worker downloaded from `gs://bucket/jobs/{job_id}/` but files were stored as:
|
||||
- `videos/with_vocals.mkv`
|
||||
- `stems/instrumental_clean.flac`
|
||||
|
||||
The original code used `work_dir.glob("*.mkv")` which only searched the root directory, causing "No video files found" errors.
|
||||
|
||||
**Solution**: Use recursive glob patterns and filter results:
|
||||
```python
|
||||
# BAD - only searches root directory
|
||||
input_files = list(work_dir.glob("*.mkv"))
|
||||
|
||||
# GOOD - searches all subdirectories
|
||||
input_files = list(work_dir.glob("**/*.mkv"))
|
||||
# Filter out outputs directory to avoid re-encoding
|
||||
input_files = [f for f in input_files if "outputs" not in str(f)]
|
||||
```
|
||||
|
||||
**Related gotcha**: Case sensitivity matters. GCS stores filenames as-is, so if files are uploaded as `instrumental_clean.flac` (lowercase), searching for `*Instrumental*.flac` (capital I) won't find them. Use case-insensitive patterns or search for both cases.
|
||||
|
||||
**Lesson**: When writing code that processes files downloaded from GCS, always use recursive glob patterns and verify the expected directory structure matches what's actually stored.
|
||||
|
||||
### Library Caches Need GCS Sync in Cloud Run
|
||||
|
||||
**Problem**: Libraries like LyricsTranscriber cache API responses to local directories (e.g., `~/lyrics-transcriber-cache`). In Cloud Run, each container is ephemeral - the cache is lost on every restart or scale-up, causing redundant API calls.
|
||||
|
||||
**Symptoms**:
|
||||
- Same song processed multiple times always hits external APIs
|
||||
- Higher API costs than expected (AudioShake charges per transcription)
|
||||
- Slower processing even for repeated jobs
|
||||
|
||||
**Solution**: Sync library cache to/from GCS before/after each job:
|
||||
|
||||
```python
|
||||
# Set cache dir via environment variable (library reads this)
|
||||
cache_dir = os.path.join(temp_dir, "lyrics-cache")
|
||||
os.environ["LYRICS_TRANSCRIBER_CACHE_DIR"] = cache_dir
|
||||
|
||||
# Download relevant cache files from GCS before processing
|
||||
cache_service.sync_cache_from_gcs(cache_dir, audio_hash, lyrics_hash)
|
||||
|
||||
# ... run processing ...
|
||||
|
||||
# Upload new cache files to GCS after processing
|
||||
cache_service.sync_cache_to_gcs(cache_dir, audio_hash, lyrics_hash)
|
||||
```
|
||||
|
||||
**Key design decisions**:
|
||||
1. Download only relevant files (by hash) - don't sync entire cache directory
|
||||
2. Skip files that already exist in GCS on upload (same input = same output)
|
||||
3. Use environment variable to configure library cache dir (avoids modifying library code)
|
||||
4. Cache indefinitely (no TTL) - API responses don't change for same inputs
|
||||
|
||||
**Lesson**: When using third-party libraries with local caching in Cloud Run, check if they support configurable cache directories. If so, sync to GCS for persistence across instances.
|
||||
|
||||
### Sequential vs Parallel Worker Triggering
|
||||
|
||||
**Problem**: For URL-based jobs (YouTube), triggering both audio and lyrics workers in parallel caused failures. Lyrics worker needs the audio file, but audio worker hadn't downloaded it yet.
|
||||
|
||||
```python
|
||||
# BAD - for URL jobs, audio isn't ready yet
|
||||
background_tasks.add_task(_trigger_workers_parallel, job_id) # Both start together
|
||||
# Lyrics worker times out waiting for audio
|
||||
```
|
||||
|
||||
**Solution**: For URL jobs, trigger workers sequentially:
|
||||
```python
|
||||
# GOOD - audio downloads first, then triggers lyrics
|
||||
if job.url and not job.input_media_gcs_path:
|
||||
# Only trigger audio worker; it will trigger lyrics after download
|
||||
background_tasks.add_task(worker_service.trigger_audio_worker, job_id)
|
||||
else:
|
||||
# For uploaded files, parallel is fine
|
||||
background_tasks.add_task(_trigger_workers_parallel, job_id)
|
||||
```
|
||||
|
||||
**Lesson**: Understand data dependencies between workers. If worker B needs output from worker A, don't run them in parallel.
|
||||
|
||||
### Always Extract Auth Context Into Jobs
|
||||
|
||||
**Problem**: Job creation endpoints used `require_auth` but never extracted `auth_result.user_email` to store on the job. Jobs were created with `user_email=None`, so users couldn't see their own jobs.
|
||||
|
||||
```python
|
||||
# BAD - auth used for access control but email not captured
|
||||
@router.post("/jobs/upload")
|
||||
async def upload(auth_result = Depends(require_auth)):
|
||||
job = JobCreate(...) # user_email not set!
|
||||
```
|
||||
|
||||
**Solution**: Extract user identity from auth and set on job:
|
||||
```python
|
||||
# GOOD - capture authenticated user's identity
|
||||
@router.post("/jobs/upload")
|
||||
async def upload(auth_result: AuthResult = Depends(require_auth)):
|
||||
effective_user_email = auth_result.user_email or form_user_email
|
||||
job = JobCreate(user_email=effective_user_email, ...)
|
||||
```
|
||||
|
||||
**Lesson**: Auth provides identity, not just access control. If jobs need owner association, explicitly extract and store the user identifier.
|
||||
|
||||
See `docs/archive/2024-12-31-job-failure-investigation.md` for full details.
|
||||
|
||||
## What We'd Do Differently
|
||||
|
||||
1. **Add Pydantic model fields test first** - Would have caught the silent field issue immediately
|
||||
@@ -162,3 +903,5 @@ Human review time varies (5-15 min typical).
|
||||
3. **Design for async human review upfront** - Avoided rearchitecting after discovering ReviewServer blocks
|
||||
|
||||
4. **Keep docs minimal and current** - Less documentation, but always accurate
|
||||
|
||||
5. **Check gitignore early for new directories** - Especially when adding frontend `lib/` directories
|
||||
@@ -0,0 +1,234 @@
|
||||
# Product Vision
|
||||
|
||||
## What is Nomad Karaoke?
|
||||
|
||||
Nomad Karaoke is a karaoke video generation platform that creates professional-quality karaoke videos from any song in under 10 minutes. Founded by Andrew Beveridge and based in South Carolina, it combines state-of-the-art AI for audio separation and lyrics synchronization with human review to produce accurate, polished results.
|
||||
|
||||
**Mission**: Make any song singable by automating the complex, time-consuming process of creating karaoke videos.
|
||||
|
||||
## The Problem
|
||||
|
||||
Creating high-quality karaoke is surprisingly difficult:
|
||||
|
||||
1. **Vocal Removal**: Separating vocals from instrumentals without artifacts requires sophisticated AI models and significant compute power.
|
||||
|
||||
2. **Lyrics Accuracy**: Getting the right lyrics, with correct spelling, formatting, and line breaks, is error-prone when scraping from various sources.
|
||||
|
||||
3. **Timestamp Synchronization**: Aligning each word or syllable to the exact moment in the audio is the hardest part. No existing tool does this reliably for all songs.
|
||||
|
||||
4. **Multiple Formats**: Professional karaoke needs multiple output formats (MP4, MKV, CDG) at various resolutions, with proper encoding.
|
||||
|
||||
5. **Distribution**: Uploading to YouTube with proper metadata, sharing to cloud storage, and notifying users adds complexity.
|
||||
|
||||
Existing solutions either produce poor quality (DIY tools), cost too much time ($50-100/track from manual services), or don't exist at all for many songs.
|
||||
|
||||
## Target Audiences
|
||||
|
||||
### 1. Individual Karaoke Enthusiasts
|
||||
People who want custom karaoke for songs not available in commercial libraries - for home parties, personal practice, or sharing with friends.
|
||||
|
||||
### 2. Professional KJs and Venues
|
||||
Karaoke DJs ("KJs") and bars who need to fulfill on-demand song requests. When a patron asks for a song that isn't in the library, generating it in under 10 minutes means it can be ready within 1-2 performances.
|
||||
|
||||
### 3. Commercial Karaoke Producers (B2B)
|
||||
Companies that produce karaoke tracks at scale for distribution. They have their own instrumentals, branding, and distribution channels - they need automated generation infrastructure that can handle 5,000+ tracks with custom theming.
|
||||
|
||||
## Technical Approach
|
||||
|
||||
The system uses a multi-stage pipeline:
|
||||
|
||||
```
|
||||
Input (audio + metadata)
|
||||
│
|
||||
├──────────────────────────────────────┐
|
||||
│ │
|
||||
v v
|
||||
┌───────────────────┐ ┌───────────────────┐
|
||||
│ Audio Separation │ │ Lyrics Processing │
|
||||
│ (Modal GPU) │ │ (AudioShake API) │
|
||||
│ │ │ │
|
||||
│ • Clean vocal │ │ • Fetch reference │
|
||||
│ removal │ │ lyrics │
|
||||
│ • Backing vocal │ │ • Transcribe with │
|
||||
│ options │ │ timestamps │
|
||||
└─────────┬─────────┘ │ • AI correction │
|
||||
│ │ • HUMAN REVIEW │
|
||||
│ └─────────┬─────────┘
|
||||
│ │
|
||||
└──────────────┬───────────────────┘
|
||||
v
|
||||
┌───────────────────┐
|
||||
│ Video Generation │
|
||||
│ • Title screen │
|
||||
│ • Synced lyrics │
|
||||
│ • End screen │
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
v
|
||||
┌───────────────────┐
|
||||
│ Output & Distro │
|
||||
│ • Multiple formats│
|
||||
│ • YouTube upload │
|
||||
│ • Cloud storage │
|
||||
└───────────────────┘
|
||||
```
|
||||
|
||||
### Why Human Review?
|
||||
|
||||
The hardest unsolved problem in karaoke generation is **reliable lyrics transcription with accurate timestamps**. Current approaches:
|
||||
|
||||
- **Whisper (open source)**: Free, but inconsistent for sung audio. Works well ~60% of the time, poorly the rest.
|
||||
- **AudioShake (paid API)**: Proprietary lyrics transcription model specifically trained for music. Much more accurate/consistent than Whisper.
|
||||
- **Agentic AI Correction**: LLM-based system that compares transcriptions against reference lyrics, fixing errors and improving sync. Costs tokens but catches many issues.
|
||||
|
||||
Even with the best AI, some songs have tricky timing, unusual pronunciations, or ambiguous lyrics. Human review catches these edge cases and ensures quality. The review UI lets users:
|
||||
- Correct misspelled words
|
||||
- Adjust word/line timing
|
||||
- Fix line breaks and formatting
|
||||
- Preview changes in real-time
|
||||
|
||||
## Open Source Philosophy
|
||||
|
||||
All code is open source (MIT license) because:
|
||||
|
||||
1. **Community**: Others can learn from, contribute to, and build upon the work
|
||||
2. **Transparency**: Users can see exactly how their data is processed
|
||||
3. **Self-hosting option**: Technical users can run their own instance
|
||||
|
||||
However, reliable fully-automated generation requires:
|
||||
- **AudioShake API**: ~$0.50-1.00 per track for high-quality transcription
|
||||
- **LLM tokens**: ~$0.50-1.00 for agentic correction workflow
|
||||
- **GPU compute**: ~$0.20-0.50 for audio separation on Modal
|
||||
- **Infrastructure**: Cloud Run, Firestore, GCS, etc.
|
||||
|
||||
**Cost per track: ~$2-3** for fully-automated processing, making a paid service necessary to sustain development.
|
||||
|
||||
The open source CLI (`karaoke-gen`) works with local processing and free APIs for users willing to accept lower reliability and do more manual correction. The hosted service (`gen.nomadkaraoke.com`) provides the premium, reliable experience.
|
||||
|
||||
## Business Model
|
||||
|
||||
### Credit-Based Pricing
|
||||
|
||||
Users purchase credits to generate karaoke videos:
|
||||
|
||||
| Credits | Price | Per Track | Savings |
|
||||
|---------|-------|-----------|---------|
|
||||
| 1 | $5 | $5.00 | - |
|
||||
| 3 | $12 | $4.00 | 20% |
|
||||
| 5 | $18 | $3.50 | 30% |
|
||||
| 10 | $30 | $3.00 | 40% |
|
||||
|
||||
This is significantly cheaper than:
|
||||
- Manual services: $10-50 per track
|
||||
- Hiring a studio: $100+ per track
|
||||
- DIY tools + your time: Hours of frustration
|
||||
|
||||
### Beta Tester Program
|
||||
|
||||
New users can join the beta program to get free credits in exchange for feedback, helping improve the system while building initial user base.
|
||||
|
||||
### B2B / White-Label
|
||||
|
||||
Commercial producers pay for API access or custom integration to generate tracks at scale with their own branding, instrumentals, and distribution.
|
||||
|
||||
## Current State (December 2025)
|
||||
|
||||
### What Works
|
||||
- Audio upload and separation (Modal API)
|
||||
- Lyrics transcription (AudioShake API)
|
||||
- Agentic AI correction (Gemini 3 Flash via Vertex AI)
|
||||
- Human lyrics review (React UI)
|
||||
- Preview video generation
|
||||
- Instrumental selection (clean vs. with backing vocals)
|
||||
- Multi-format encoding (4K, 720p, lossless, lossy)
|
||||
- Token-based authentication
|
||||
- Magic link authentication
|
||||
- Payment flow (Stripe)
|
||||
- Beta tester program
|
||||
- YouTube upload integration
|
||||
- Dropbox/Google Drive distribution
|
||||
|
||||
### Known Limitations
|
||||
- CDG format needs additional style configuration
|
||||
- Very long audio (>10 min) may timeout
|
||||
- Some edge cases in lyrics timing still need manual correction
|
||||
|
||||
## Product Ecosystem
|
||||
|
||||
### Active Products
|
||||
|
||||
1. **gen.nomadkaraoke.com** - Web application for generating karaoke videos (includes landing page, pricing, and payment)
|
||||
2. **karaoke-gen CLI** - Local command-line tool for power users
|
||||
3. **karaoke-gen-remote CLI** - CLI that uses the cloud backend
|
||||
4. **Nomad Karaoke YouTube Channel** - Hosts generated videos for users
|
||||
|
||||
### Future Products
|
||||
|
||||
1. **KaraokeHunt** (revival planned) - Song discovery app that analyzes streaming history to suggest songs you'd enjoy singing. Previously launched as a separate product; will be rebranded and integrated with the karaoke generator.
|
||||
|
||||
2. **New nomadkaraoke.com Homepage** - Replace the legacy static site with a modern landing page showcasing the generator (currently deployed from `nomadkaraoke/public-website` repo).
|
||||
|
||||
## Differentiators
|
||||
|
||||
### vs. DIY Tools (Whisper + Demucs + manual editing)
|
||||
- **Quality**: AudioShake's proprietary model beats open source transcription
|
||||
- **Speed**: 10 minutes vs. hours of manual work
|
||||
- **Convenience**: One-click generation with human review UI
|
||||
|
||||
### vs. Manual Services ($10-50/track)
|
||||
- **Price**: $3-5 per track
|
||||
- **Speed**: Minutes instead of days
|
||||
- **Scale**: Generate as many as you need, when you need them
|
||||
|
||||
### vs. Other Automated Tools
|
||||
- **Accuracy**: Human review catches AI errors
|
||||
- **Output quality**: 4K video, multiple formats, proper encoding
|
||||
- **Distribution**: YouTube upload, cloud storage, notifications built in
|
||||
- **Open source**: Transparency and community contributions
|
||||
|
||||
## Technical Stack
|
||||
|
||||
- **Backend**: FastAPI on Google Cloud Run
|
||||
- **Frontend**: Next.js on Cloudflare Pages
|
||||
- **Database**: Firestore
|
||||
- **Storage**: Google Cloud Storage
|
||||
- **Secrets**: Google Secret Manager
|
||||
- **Audio Separation**: Modal (GPU compute)
|
||||
- **Lyrics Transcription**: AudioShake API
|
||||
- **AI Correction**: Vertex AI (Gemini 3 Flash)
|
||||
- **Payments**: Stripe
|
||||
- **Email**: SendGrid
|
||||
- **IaC**: Pulumi
|
||||
- **CI/CD**: GitHub Actions (self-hosted runner on GCP)
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Near-term
|
||||
- Public launch of gen.nomadkaraoke.com
|
||||
- New marketing homepage at nomadkaraoke.com
|
||||
- B2B API for commercial producer integration
|
||||
- Bulk generation features for high-volume clients
|
||||
|
||||
### Medium-term
|
||||
- KaraokeHunt revival and integration
|
||||
- Mobile-optimized experience
|
||||
- Real-time generation status notifications
|
||||
- Additional output format support
|
||||
|
||||
### Long-term
|
||||
- Karaoke Eternal integration (community request)
|
||||
- On-premises deployment option for venues
|
||||
- Live karaoke event features
|
||||
|
||||
## Links
|
||||
|
||||
- **Web App**: https://gen.nomadkaraoke.com
|
||||
- **API**: https://api.nomadkaraoke.com
|
||||
- **GitHub**: https://github.com/nomadkaraoke/karaoke-gen
|
||||
- **YouTube Channel**: https://www.youtube.com/@nomadkaraoke
|
||||
- **Updates Signup**: https://nomadkaraoke.kit.com/
|
||||
- **Contact**: https://cal.com/beveradb
|
||||
|
||||
## About the Founder
|
||||
|
||||
Andrew Beveridge is a software engineer based in South Carolina who runs a weekly karaoke night and built Nomad Karaoke to solve his own problem: wanting to sing songs that don't exist as karaoke. What started as a personal tool evolved into a platform that can help anyone create professional karaoke for any song.
|
||||
+40
-3
@@ -4,7 +4,10 @@
|
||||
|
||||
## What Works
|
||||
|
||||
- **Web App**: https://gen.nomadkaraoke.com - Upload audio, review lyrics, download videos
|
||||
- **Web App**: https://gen.nomadkaraoke.com
|
||||
- `/` - Landing page with pricing and beta enrollment
|
||||
- `/app` - Main app (upload audio, review lyrics, download videos)
|
||||
- `/admin` - Admin dashboard (user management, job monitoring, beta program)
|
||||
- **Backend API**: https://api.nomadkaraoke.com - FastAPI on Cloud Run
|
||||
- **CLI Tools**: `karaoke-gen` (local), `karaoke-gen-remote` (cloud)
|
||||
|
||||
@@ -14,27 +17,61 @@
|
||||
|-----------|--------|
|
||||
| Audio upload & separation | Working |
|
||||
| Lyrics transcription | Working |
|
||||
| Agentic AI correction | Working (Gemini 2 Flash via Vertex AI) |
|
||||
| Human lyrics review | Working |
|
||||
| Preview video generation | Working |
|
||||
| Instrumental selection | Working |
|
||||
| Multi-format encoding | Working |
|
||||
| Token-based auth | Working |
|
||||
| Payment flow | Not started |
|
||||
| Magic link auth | Working |
|
||||
| Payment flow (Stripe) | Working |
|
||||
| Beta tester program | Working |
|
||||
| Admin dashboard | Working |
|
||||
| CI/CD self-hosted runner | Working (GCP) |
|
||||
| E2E happy path test | Working (38 min full pipeline) |
|
||||
|
||||
## Known Issues
|
||||
|
||||
- CDG format generation requires additional style configuration
|
||||
- Long audio files (>10 min) may timeout on some workers
|
||||
|
||||
## Recent Changes
|
||||
|
||||
- **User Database Separation** (2026-01-03): Migrated karaoke-gen users from shared `users` collection to dedicated `gen_users` collection. This separates karaoke-gen user data from karaoke-decide user data (both apps share the same GCP project/Firestore instance). Migration script in `scripts/migrate_users_to_gen_users.py`. See [ARCHITECTURE.md](ARCHITECTURE.md#firestore-collections) for collection details.
|
||||
|
||||
- **Admin Dashboard** (2026-01-03): Added comprehensive admin dashboard at `/admin` with: stats overview (users, jobs, credits), user management (search, sort, add credits, toggle role, enable/disable), job browser (filter by status/user, view details, delete), and beta program monitoring. Query-param based routing for static export compatibility. See [API.md](API.md#admin-endpoints) for backend endpoints.
|
||||
|
||||
- **GCE Encoding Worker** (2026-01-03): Added optional high-performance encoding on dedicated GCE VM (C4-standard-8 with Intel Granite Rapids 3.9 GHz). Uses `USE_GCE_ENCODING=true` env var. Falls back to Cloud Run if unavailable. See [ARCHITECTURE.md](ARCHITECTURE.md).
|
||||
|
||||
- **Runner Disk Space Auto-Cleanup** (2026-01-03): Fixed CI failures caused by self-hosted runners filling up with Docker images. Changed cleanup from daily/7-day-old to hourly/threshold-based (70%). Added pre-job disk checks to all self-hosted CI jobs. See [LESSONS-LEARNED.md](LESSONS-LEARNED.md#docker-disk-space-management-on-self-hosted-runners)
|
||||
|
||||
- **E2E Happy Path Test** (2026-01-02): Added comprehensive E2E test that validates the complete karaoke generation journey using ONLY browser UI interactions (no API shortcuts). Test covers all 12 steps from landing page through cleanup. Runs daily via GitHub Actions and takes ~38 minutes. See [E2E-HAPPY-PATH-TEST-SPEC.md](E2E-HAPPY-PATH-TEST-SPEC.md)
|
||||
|
||||
- **LyricsTranscriber Cache Persistence** (2026-01-02): Added GCS-backed cache for LyricsTranscriber to avoid redundant API calls. AudioShake transcription and lyrics provider responses (Genius, Spotify, LRCLib, Musixmatch) are now cached to `gs://karaoke-gen-storage/lyrics-transcriber-cache/`. Cache is synced before/after each job, persisting across Cloud Run instances. Significantly reduces API costs and speeds up repeated processing of same songs.
|
||||
|
||||
- **Worker Timeout Fixes** (2026-01-01): Fixed 3 timeout issues blocking job completion: (1) Lyrics transcription timeout increased to 20 min (PR #153), (2) Cloud Tasks dispatch_deadline added for audio worker - default 10 min was killing Modal API calls (PR #154), (3) Enabled Cloud Run Jobs for video encoding - 1-hour timeout vs 30-min Cloud Run service limit (PR #155). Full E2E pipeline now verified working. See [archive/2026-01-01-worker-timeout-fixes.md](archive/2026-01-01-worker-timeout-fixes.md)
|
||||
|
||||
- **Agentic Correction Timeout** (2025-12-31): Added 3-minute configurable timeout for agentic AI lyrics correction. Prevents stuck jobs when songs have many gaps (74+ gaps could take 30+ minutes). On timeout, skips correction and proceeds to human review with raw transcription. See [archive/2025-12-31-agentic-timeout-implementation.md](archive/2025-12-31-agentic-timeout-implementation.md)
|
||||
- **Job Failure Fixes** (2025-12-31): Fixed 4 critical issues causing job failures: (1) user_email not set on jobs - users couldn't see their jobs, (2) YouTube URL race condition - lyrics worker started before audio downloaded, (3) Audio search cache not persisting across Cloud Run instances, (4) Jobs stuck in downloading state - added 10min transcription timeout. See [archive/2024-12-31-job-failure-investigation.md](archive/2024-12-31-job-failure-investigation.md)
|
||||
- **Vertex AI Auth Fix** (2025-12-31): Fixed `ChatGoogleGenerativeAI` to use Vertex AI backend with service account auth on Cloud Run. PR #145 broke cloud deployments by requiring `GOOGLE_API_KEY`. Solution: pass `project` parameter to trigger Vertex AI backend with ADC. See [archive/2025-12-31-vertex-ai-auth-fix.md](archive/2025-12-31-vertex-ai-auth-fix.md)
|
||||
- **LangChain Provider Migration** (2025-12-30): Migrated from `langchain-google-vertexai` (gRPC) to `langchain-google-genai` (REST) to fix silent hangs during model initialization. Added initialization and warm-up timeouts using ThreadPoolExecutor for fail-fast behavior. See [archive/2025-12-30-langchain-vertexai-to-genai-migration.md](archive/2025-12-30-langchain-vertexai-to-genai-migration.md)
|
||||
- **LangFuse Prompt Management** (2025-12-30): Replaced hardcoded LLM prompts with LangFuse-managed prompts for dynamic iteration without redeploying. Added migration script, LangFuse Vertex AI service account for GCP credits. See [archive/2025-12-30-langfuse-prompt-management.md](archive/2025-12-30-langfuse-prompt-management.md)
|
||||
- **Gemini 3 Flash Agentic Correction Fix** (2025-12-30): Fixed agentic correction hanging due to Gemini 3 requiring `global` location instead of regional endpoints, and multimodal response format handling. Added local test script for faster iteration. See [archive/2025-12-30-gemini3-agentic-correction-fix.md](archive/2025-12-30-gemini3-agentic-correction-fix.md)
|
||||
- **Cold Start UX** (2025-12-30): Frontend now shows friendly "Warming up the generator..." messages when backend is slow to respond (cold start after deployment or inactivity). Progressive messages appear after 5s and 15s to reassure users during 30s+ cold starts.
|
||||
- **Cloud Output Structure Fix** (2025-12-30): Fixed remote CLI (`karaoke-gen-remote`) to produce identical output as local CLI: YouTube uploads now work, Dropbox includes stems/ and lyrics/ subfolders, instrumental filenames use actual model names. See [archive/2025-12-30-cloud-output-structure-fix.md](archive/2025-12-30-cloud-output-structure-fix.md)
|
||||
- **Auth Session Persistence Fix** (2025-12-30): Fixed two bugs causing 401 errors despite valid sessions: timezone-aware datetime comparisons in backend, and stale token caching in frontend. Firestore indexes now tracked in Pulumi. See [archive/2025-12-30-auth-session-persistence-fix.md](archive/2025-12-30-auth-session-persistence-fix.md)
|
||||
- **E2E Test Suite Consolidated** (2025-12-29): Tests reorganized into `production/` (real API) and `regression/` (mocked API, CI-safe). See [archive/2025-12-29-e2e-test-consolidation-plan.md](archive/2025-12-29-e2e-test-consolidation-plan.md)
|
||||
|
||||
## Quick Links
|
||||
|
||||
| Doc | Purpose |
|
||||
|-----|---------|
|
||||
| [PRODUCT-VISION.md](PRODUCT-VISION.md) | Product goals, audience, business model |
|
||||
| [ARCHITECTURE.md](ARCHITECTURE.md) | System design, data flow, integrations |
|
||||
| [DEVELOPMENT.md](DEVELOPMENT.md) | Local setup, testing, deployment |
|
||||
| [API.md](API.md) | Backend endpoint reference |
|
||||
| [LESSONS-LEARNED.md](LESSONS-LEARNED.md) | Key insights for future agents |
|
||||
| [STRIPE-SETUP.md](STRIPE-SETUP.md) | Payment integration setup |
|
||||
| [MOBILE-UX-BEST-PRACTICES.md](MOBILE-UX-BEST-PRACTICES.md) | Mobile/responsive design guidelines |
|
||||
| [archive/](archive/) | Historical documentation |
|
||||
|
||||
## Workflow Summary
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
# Stripe Setup Guide
|
||||
|
||||
This guide walks you through setting up Stripe for the Nomad Karaoke payment system.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A Stripe account (sign up at https://stripe.com)
|
||||
- Access to Google Cloud Secret Manager (for secure credential storage)
|
||||
- The Nomad Karaoke backend deployed and running
|
||||
|
||||
## 1. Get Your Stripe API Keys
|
||||
|
||||
1. Log in to your Stripe Dashboard: https://dashboard.stripe.com
|
||||
2. Go to **Developers** > **API keys**
|
||||
3. You'll need two keys:
|
||||
- **Publishable key** (starts with `pk_live_` or `pk_test_`)
|
||||
- **Secret key** (starts with `sk_live_` or `sk_test_`)
|
||||
|
||||
> **Important**: Never commit your secret key to version control. Always use environment variables or Secret Manager.
|
||||
|
||||
## 2. Store Keys in Google Cloud Secret Manager
|
||||
|
||||
The secret containers are already created via Pulumi (in `infrastructure/__main__.py`). You just need to add the secret values:
|
||||
|
||||
```bash
|
||||
# Store the Stripe secret key
|
||||
echo -n "sk_live_your_key_here" | gcloud secrets versions add stripe-secret-key --data-file=-
|
||||
|
||||
# Store the webhook secret (after step 3)
|
||||
echo -n "whsec_your_webhook_secret" | gcloud secrets versions add stripe-webhook-secret --data-file=-
|
||||
```
|
||||
|
||||
> **Note**: If running for the first time after Pulumi creates the secrets, use `gcloud secrets versions add` (not `gcloud secrets create`).
|
||||
|
||||
## 3. Set Up Webhook Endpoint
|
||||
|
||||
Webhooks notify your backend when payments complete.
|
||||
|
||||
### In Stripe Dashboard:
|
||||
|
||||
1. Go to **Developers** > **Webhooks**
|
||||
2. Click **Add endpoint**
|
||||
3. Enter your endpoint URL: `https://api.nomadkaraoke.com/api/users/webhooks/stripe`
|
||||
4. Select events to listen to:
|
||||
- `checkout.session.completed` (required - adds credits)
|
||||
- `checkout.session.expired` (optional - logging)
|
||||
- `payment_intent.payment_failed` (optional - logging)
|
||||
5. Click **Add endpoint**
|
||||
6. Copy the **Signing secret** (starts with `whsec_`)
|
||||
7. Store it in Secret Manager (see step 2)
|
||||
|
||||
## 4. Configure Environment Variables
|
||||
|
||||
Set these environment variables for your backend:
|
||||
|
||||
```bash
|
||||
# Required
|
||||
STRIPE_SECRET_KEY=sk_live_your_key_here
|
||||
STRIPE_WEBHOOK_SECRET=whsec_your_webhook_secret
|
||||
|
||||
# Optional (defaults shown)
|
||||
FRONTEND_URL=https://gen.nomadkaraoke.com
|
||||
# BUY_URL defaults to FRONTEND_URL after site consolidation
|
||||
```
|
||||
|
||||
### For Cloud Run deployment:
|
||||
|
||||
The CI workflow (`.github/workflows/ci.yml`) automatically injects these secrets during deployment. No manual `gcloud` commands needed - just add the secret values to Secret Manager (step 2) and the next backend deployment will pick them up.
|
||||
|
||||
## 5. Configure Email Service (SendGrid)
|
||||
|
||||
For sending magic link and purchase confirmation emails:
|
||||
|
||||
1. Sign up for SendGrid: https://sendgrid.com
|
||||
2. Create an API key with "Mail Send" permissions
|
||||
3. Store it in Secret Manager (secret container created by Pulumi):
|
||||
|
||||
```bash
|
||||
echo -n "SG.your_sendgrid_api_key" | gcloud secrets versions add sendgrid-api-key --data-file=-
|
||||
```
|
||||
|
||||
4. The environment variables are configured in the Cloud Run deployment. Defaults:
|
||||
- `EMAIL_FROM=gen@nomadkaraoke.com`
|
||||
- `EMAIL_FROM_NAME=Nomad Karaoke`
|
||||
|
||||
## 6. Test the Integration
|
||||
|
||||
### Test Mode
|
||||
|
||||
Use test keys (starting with `pk_test_` and `sk_test_`) for testing:
|
||||
|
||||
1. Use Stripe's test card: `4242 4242 4242 4242`
|
||||
2. Any future expiration date
|
||||
3. Any 3-digit CVC
|
||||
|
||||
### Verify Webhook Delivery
|
||||
|
||||
1. In Stripe Dashboard, go to **Developers** > **Webhooks**
|
||||
2. Click on your endpoint
|
||||
3. View recent webhook attempts and their status
|
||||
4. Use the "Send test webhook" feature to verify connectivity
|
||||
|
||||
## Credit Packages
|
||||
|
||||
The system comes with these default packages:
|
||||
|
||||
| Package | Credits | Price | Per Credit |
|
||||
|---------|---------|-------|------------|
|
||||
| 1 Credit | 1 | $5.00 | $5.00 |
|
||||
| 3 Credits | 3 | $12.00 | $4.00 (20% off) |
|
||||
| 5 Credits | 5 | $17.50 | $3.50 (30% off) |
|
||||
| 10 Credits | 10 | $30.00 | $3.00 (40% off) |
|
||||
|
||||
To modify packages, edit `backend/services/stripe_service.py`:
|
||||
|
||||
```python
|
||||
CREDIT_PACKAGES = {
|
||||
"1_credit": {
|
||||
"credits": 1,
|
||||
"price_cents": 500,
|
||||
"name": "1 Karaoke Credit",
|
||||
"description": "Create 1 professional karaoke video",
|
||||
},
|
||||
# Add more packages...
|
||||
}
|
||||
```
|
||||
|
||||
## Admin Features
|
||||
|
||||
### Issue Free Credits
|
||||
|
||||
Admins can issue free credits via the API:
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.nomadkaraoke.com/api/users/admin/credits \
|
||||
-H "Authorization: Bearer YOUR_ADMIN_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"email": "user@example.com",
|
||||
"amount": 5,
|
||||
"reason": "Beta tester reward"
|
||||
}'
|
||||
```
|
||||
|
||||
### View Users
|
||||
|
||||
```bash
|
||||
curl https://api.nomadkaraoke.com/api/users/admin/users \
|
||||
-H "Authorization: Bearer YOUR_ADMIN_TOKEN"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Webhook Not Receiving Events
|
||||
|
||||
1. Verify the endpoint URL is correct and publicly accessible
|
||||
2. Check the webhook signing secret matches
|
||||
3. Ensure your server returns 2xx status codes
|
||||
4. Check Cloud Run logs for errors
|
||||
|
||||
### Credits Not Adding
|
||||
|
||||
1. Verify `checkout.session.completed` event is being received
|
||||
2. Check that `metadata` contains `user_email` and `credits`
|
||||
3. Review backend logs for errors in `stripe_webhook` handler
|
||||
|
||||
### Email Not Sending
|
||||
|
||||
1. Verify SendGrid API key is valid
|
||||
2. Check sender email is verified in SendGrid
|
||||
3. Review SendGrid activity logs
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Never expose secret keys** - Use environment variables or Secret Manager
|
||||
2. **Always verify webhook signatures** - The code does this automatically
|
||||
3. **Use HTTPS** - All Stripe communication must be over HTTPS
|
||||
4. **Monitor webhook logs** - Watch for failed deliveries or suspicious activity
|
||||
5. **Set up alerts** - Configure Stripe to alert on unusual activity
|
||||
|
||||
## Going Live Checklist
|
||||
|
||||
- [ ] Switch from test to live API keys
|
||||
- [ ] Update webhook endpoint to use live signing secret
|
||||
- [ ] Verify email sending works with real addresses
|
||||
- [ ] Test a real purchase with a small amount
|
||||
- [ ] Set up Stripe Radar for fraud protection
|
||||
- [ ] Configure Stripe tax settings if applicable
|
||||
- [ ] Review and enable appropriate Stripe email receipts
|
||||
@@ -0,0 +1,230 @@
|
||||
# API Key Management System - Phase 2 Plan
|
||||
|
||||
**Status:** Planned (Phase 1 complete)
|
||||
**Created:** 2024-12-29
|
||||
|
||||
## Background
|
||||
|
||||
Phase 1 (completed) established that all auth_tokens must have a `user_email` field for job ownership tracking. This eliminated anonymous token authentication and ensures all jobs have proper attribution.
|
||||
|
||||
Phase 2 will add self-service API key management for B2B/bulk karaoke generation users.
|
||||
|
||||
## Current State (After Phase 1)
|
||||
|
||||
- All `auth_tokens` in Firestore require a `user_email` field
|
||||
- Admin tokens (from `ADMIN_TOKENS` env var) are associated with `admin@nomadkaraoke.com`
|
||||
- Session tokens (magic link auth) have user_email from the authenticated user
|
||||
- Jobs created via any auth method are attributed to the token's user_email
|
||||
|
||||
## Phase 2: Self-Service API Key Management
|
||||
|
||||
### New Endpoints
|
||||
|
||||
```
|
||||
POST /api/keys - Generate new API key for current user
|
||||
GET /api/keys - List user's API keys (masked)
|
||||
DELETE /api/keys/{key_id} - Revoke an API key
|
||||
```
|
||||
|
||||
### API Key Schema (auth_tokens collection)
|
||||
|
||||
```python
|
||||
{
|
||||
"token": "sk_live_...", # The API key (document ID)
|
||||
"type": "api_key", # UserType.API_KEY
|
||||
"user_email": "user@example.com", # Owner's email (required)
|
||||
"api_key_id": "key_abc123", # Short ID for display/revocation
|
||||
"name": "Production Server", # User-provided name
|
||||
"max_uses": -1, # -1 = unlimited, or specific limit
|
||||
"usage_count": 0, # Current usage
|
||||
"active": True, # Can be revoked
|
||||
"created_at": 1703894400, # Unix timestamp
|
||||
"last_used": null, # Last usage timestamp
|
||||
"expires_at": null, # Optional expiration
|
||||
"ip_allowlist": [], # Optional IP restrictions
|
||||
"jobs": [] # Job history [{job_id, created_at}]
|
||||
}
|
||||
```
|
||||
|
||||
### Implementation Details
|
||||
|
||||
#### 1. Generate API Key Endpoint
|
||||
|
||||
```python
|
||||
@router.post("/keys", response_model=APIKeyResponse)
|
||||
async def create_api_key(
|
||||
request: CreateAPIKeyRequest,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> APIKeyResponse:
|
||||
"""
|
||||
Generate a new API key for the authenticated user.
|
||||
|
||||
The full key is only shown once at creation time.
|
||||
"""
|
||||
# Require authenticated user (not anonymous)
|
||||
if not auth_result.user_email:
|
||||
raise HTTPException(403, "Must be logged in to create API keys")
|
||||
|
||||
# Check user's API key limit (e.g., max 5 per user)
|
||||
existing_keys = firestore.list_user_api_keys(auth_result.user_email)
|
||||
if len(existing_keys) >= settings.max_api_keys_per_user:
|
||||
raise HTTPException(400, f"Maximum {settings.max_api_keys_per_user} API keys allowed")
|
||||
|
||||
# Generate secure key
|
||||
key_id = shortuuid.uuid()[:8]
|
||||
token = f"sk_{'live' if settings.environment == 'production' else 'test'}_{secrets.token_urlsafe(32)}"
|
||||
|
||||
# Store in Firestore
|
||||
firestore.create_token(token, {
|
||||
"token": token,
|
||||
"type": "api_key",
|
||||
"user_email": auth_result.user_email,
|
||||
"api_key_id": key_id,
|
||||
"name": request.name or f"API Key {key_id}",
|
||||
"max_uses": request.max_uses or -1,
|
||||
"usage_count": 0,
|
||||
"active": True,
|
||||
"created_at": time.time(),
|
||||
"ip_allowlist": request.ip_allowlist or [],
|
||||
"jobs": []
|
||||
})
|
||||
|
||||
return APIKeyResponse(
|
||||
key_id=key_id,
|
||||
token=token, # Only shown once!
|
||||
name=request.name,
|
||||
created_at=datetime.utcnow(),
|
||||
message="Save this key - it won't be shown again"
|
||||
)
|
||||
```
|
||||
|
||||
#### 2. List API Keys Endpoint
|
||||
|
||||
```python
|
||||
@router.get("/keys", response_model=List[APIKeyInfo])
|
||||
async def list_api_keys(
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> List[APIKeyInfo]:
|
||||
"""List user's API keys with masked tokens."""
|
||||
keys = firestore.list_user_api_keys(auth_result.user_email)
|
||||
|
||||
return [
|
||||
APIKeyInfo(
|
||||
key_id=k["api_key_id"],
|
||||
name=k.get("name"),
|
||||
token_prefix=k["token"][:12] + "...", # Masked
|
||||
created_at=datetime.fromtimestamp(k["created_at"]),
|
||||
last_used=datetime.fromtimestamp(k["last_used"]) if k.get("last_used") else None,
|
||||
usage_count=k.get("usage_count", 0),
|
||||
max_uses=k.get("max_uses", -1),
|
||||
is_active=k.get("active", True)
|
||||
)
|
||||
for k in keys
|
||||
]
|
||||
```
|
||||
|
||||
#### 3. Revoke API Key Endpoint
|
||||
|
||||
```python
|
||||
@router.delete("/keys/{key_id}")
|
||||
async def revoke_api_key(
|
||||
key_id: str,
|
||||
auth_result: AuthResult = Depends(require_auth)
|
||||
) -> dict:
|
||||
"""Revoke an API key."""
|
||||
# Find key by key_id
|
||||
key = firestore.get_api_key_by_id(key_id)
|
||||
|
||||
if not key:
|
||||
raise HTTPException(404, "API key not found")
|
||||
|
||||
# Verify ownership
|
||||
if key["user_email"] != auth_result.user_email and not auth_result.is_admin:
|
||||
raise HTTPException(403, "Not authorized to revoke this key")
|
||||
|
||||
# Revoke (soft delete)
|
||||
firestore.update_token(key["token"], {"active": False})
|
||||
|
||||
return {"status": "revoked", "key_id": key_id}
|
||||
```
|
||||
|
||||
### Firestore Indexes Needed
|
||||
|
||||
```json
|
||||
{
|
||||
"collectionGroup": "auth_tokens",
|
||||
"fields": [
|
||||
{ "fieldPath": "user_email", "order": "ASCENDING" },
|
||||
{ "fieldPath": "type", "order": "ASCENDING" },
|
||||
{ "fieldPath": "created_at", "order": "DESCENDING" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Models
|
||||
|
||||
```python
|
||||
class CreateAPIKeyRequest(BaseModel):
|
||||
name: Optional[str] = None
|
||||
max_uses: Optional[int] = None # -1 = unlimited
|
||||
ip_allowlist: Optional[List[str]] = None
|
||||
|
||||
class APIKeyResponse(BaseModel):
|
||||
key_id: str
|
||||
token: str # Full token - only shown at creation
|
||||
name: Optional[str]
|
||||
created_at: datetime
|
||||
message: str
|
||||
|
||||
class APIKeyInfo(BaseModel):
|
||||
key_id: str
|
||||
name: Optional[str]
|
||||
token_prefix: str # Masked token
|
||||
created_at: datetime
|
||||
last_used: Optional[datetime]
|
||||
usage_count: int
|
||||
max_uses: int
|
||||
is_active: bool
|
||||
```
|
||||
|
||||
### Frontend Integration
|
||||
|
||||
Add "API Keys" section to user settings page:
|
||||
|
||||
1. **List view**: Show all keys with name, prefix, usage, created date
|
||||
2. **Create button**: Opens modal to name new key, shows full key once
|
||||
3. **Revoke button**: Confirmation dialog, then revokes key
|
||||
4. **Copy button**: Copy masked key prefix (for reference)
|
||||
|
||||
### Security Considerations
|
||||
|
||||
1. **Rate limiting**: Limit key creation (e.g., 5 keys max per user)
|
||||
2. **Key rotation**: Encourage periodic rotation, show last_used date
|
||||
3. **IP allowlist**: Optional IP restriction for production keys
|
||||
4. **Audit logging**: Log all key creation/revocation events
|
||||
5. **Key prefix**: Use `sk_live_` / `sk_test_` prefix for identification
|
||||
6. **No key retrieval**: Full key only shown at creation, cannot be retrieved
|
||||
|
||||
### Migration
|
||||
|
||||
Existing auth_tokens without `user_email`:
|
||||
- Already handled in Phase 1 - tokens without user_email are rejected
|
||||
- The one existing admin token should be updated in Firestore to have `user_email: "admin@nomadkaraoke.com"`
|
||||
|
||||
### Testing
|
||||
|
||||
1. Create key as authenticated user
|
||||
2. Use key to create job - verify job.user_email matches key owner
|
||||
3. List keys - verify only own keys shown
|
||||
4. Revoke key - verify subsequent requests fail
|
||||
5. Test IP allowlist enforcement (if implemented)
|
||||
|
||||
## Timeline
|
||||
|
||||
Phase 2 can be implemented when there's B2B demand. Estimated effort: 1-2 days.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. Should API keys have expiration dates by default?
|
||||
2. What usage limits should free users have?
|
||||
3. Should we support webhook notifications for key events?
|
||||
@@ -0,0 +1,291 @@
|
||||
# Job Failure Investigation Report
|
||||
|
||||
**Date**: 2024-12-31
|
||||
**Investigated Jobs**: 76296c31, 67bc8ca8, 02207418, cadd7153, 6a461e4e, 14035e2b, 643983e3, 9726a99b
|
||||
**Affected User**: [redacted]@gmail.com (plus E2E test jobs)
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Investigation revealed **4 distinct issues** causing job failures and the user complaint that "jobs don't show up in my account":
|
||||
|
||||
1. **CRITICAL: user_email not set on jobs** - Jobs aren't associated with users
|
||||
2. **YouTube URL download race condition** - Lyrics worker starts before audio is downloaded
|
||||
3. **Audio search cache not shared across instances** - Cloud Run scaling breaks download
|
||||
4. **Lyrics worker stalling silently** - Jobs stuck in "downloading" state forever
|
||||
|
||||
---
|
||||
|
||||
## Issue 1: user_email Not Being Set on Jobs (CRITICAL)
|
||||
|
||||
### Symptoms
|
||||
- User reports: "jobs polling is always returning a []"
|
||||
- User has active account with 2 credits, active session
|
||||
- User investigation shows `Jobs Created: 0` despite creating jobs
|
||||
- All 8 investigated jobs have `user_email: null`
|
||||
|
||||
### Root Cause
|
||||
The job creation endpoints use `require_auth` dependency which returns `AuthResult` containing `user_email`. However, the endpoints **do not extract this email and set it on the job**.
|
||||
|
||||
**In `backend/api/routes/file_upload.py`:**
|
||||
- `user_email` is an optional form field (line 397) that clients must explicitly pass
|
||||
- The `auth_result.user_email` from authentication is ignored
|
||||
- Jobs are created with `user_email=None` unless client explicitly sends it
|
||||
|
||||
**Same issue in:**
|
||||
- `/jobs/upload` endpoint
|
||||
- `/jobs/create-from-url` endpoint
|
||||
- `/jobs/create-with-upload-urls` endpoint
|
||||
- `/api/audio-search/search` endpoint
|
||||
|
||||
### Fix Required
|
||||
Modify all job creation endpoints to:
|
||||
1. Extract `user_email` from `AuthResult` when available
|
||||
2. Prefer authenticated user's email over client-provided value
|
||||
3. Set this on the `JobCreate` model
|
||||
|
||||
**Example fix for `create-from-url`:**
|
||||
```python
|
||||
@router.post("/jobs/create-from-url")
|
||||
async def create_job_from_url(
|
||||
request: Request,
|
||||
body: CreateJobFromUrlRequest,
|
||||
auth_result: AuthResult = Depends(require_auth) # Use AuthResult, not tuple
|
||||
):
|
||||
# Extract user email from authenticated session
|
||||
effective_user_email = auth_result.user_email or body.user_email
|
||||
|
||||
job_create = JobCreate(
|
||||
...
|
||||
user_email=effective_user_email, # Now properly set
|
||||
...
|
||||
)
|
||||
```
|
||||
|
||||
### Priority
|
||||
**CRITICAL** - This is blocking all authenticated users from seeing their jobs.
|
||||
|
||||
---
|
||||
|
||||
## Issue 2: YouTube URL Download Race Condition
|
||||
|
||||
### Symptoms
|
||||
- Jobs cadd7153, 6a461e4e, 14035e2b failed with: `Lyrics transcription failed: Failed to download audio file`
|
||||
- All have `input_media_gcs_path: null`
|
||||
- All were created from YouTube URLs
|
||||
|
||||
### Root Cause
|
||||
In `file_upload.py` at `/jobs/create-from-url`:
|
||||
1. Job is created with `url` but no `input_media_gcs_path`
|
||||
2. Job immediately transitions to `DOWNLOADING` state
|
||||
3. **Both audio and lyrics workers are triggered in parallel** (`_trigger_workers_parallel`)
|
||||
|
||||
The problem:
|
||||
- Audio worker is responsible for downloading from YouTube and uploading to GCS
|
||||
- Lyrics worker needs the audio file from GCS to transcribe
|
||||
- Lyrics worker polls for `input_media_gcs_path` for up to 5 minutes (max_wait_seconds=300)
|
||||
- If YouTube download is slow or fails, lyrics worker times out
|
||||
|
||||
**Additional YouTube download issues:**
|
||||
- `YOUTUBE_COOKIES` env var may not be set, causing "Sign in to confirm you're not a bot" errors
|
||||
- YouTube rate limiting can cause downloads to fail
|
||||
|
||||
### Evidence in `lyrics_worker.py:401-427`
|
||||
```python
|
||||
# For URL jobs, wait for audio worker to download and upload to GCS
|
||||
if job.url:
|
||||
while time.time() - start_time < max_wait_seconds:
|
||||
# Poll for input_media_gcs_path
|
||||
if updated_job.input_media_gcs_path:
|
||||
return local_path
|
||||
await asyncio.sleep(poll_interval)
|
||||
|
||||
logger.error(f"Timed out waiting for audio download")
|
||||
return None
|
||||
```
|
||||
|
||||
### Fix Required
|
||||
Option A (Recommended): **Sequential triggering for URL jobs**
|
||||
```python
|
||||
# In create-from-url endpoint:
|
||||
if job.url and not job.input_media_gcs_path:
|
||||
# Only trigger audio worker first for URL jobs
|
||||
# Lyrics worker triggered after audio is downloaded
|
||||
background_tasks.add_task(worker_service.trigger_audio_worker, job_id)
|
||||
else:
|
||||
# For uploaded files, parallel is fine
|
||||
background_tasks.add_task(_trigger_workers_parallel, job_id)
|
||||
```
|
||||
|
||||
Option B: **Audio worker triggers lyrics worker after download**
|
||||
- Have `audio_worker.download_audio()` trigger lyrics worker upon successful URL download
|
||||
|
||||
### Priority
|
||||
**HIGH** - YouTube/URL-based jobs are failing consistently.
|
||||
|
||||
---
|
||||
|
||||
## Issue 3: Audio Search Cache Not Persisting
|
||||
|
||||
### Symptoms
|
||||
- Job 76296c31 failed with: `Audio download failed: No cached result for index 0. Available indices: 0--1. Run search() first.`
|
||||
- Job was created from `audio_search` mode
|
||||
|
||||
### Root Cause
|
||||
`AudioSearchService` is a singleton with in-memory cache (`self._cached_results`). Cloud Run uses horizontal scaling with multiple instances, so:
|
||||
|
||||
1. Request 1 (search): Hits instance A, caches results in A's memory
|
||||
2. Request 2 (select/download): May hit instance B, which has empty cache
|
||||
3. Download fails because `len(self._cached_results) = 0`
|
||||
|
||||
**In `audio_search_service.py:279-284`:**
|
||||
```python
|
||||
if result_index < 0 or result_index >= len(self._cached_results):
|
||||
raise DownloadError(
|
||||
f"No cached result for index {result_index}. "
|
||||
f"Available indices: 0-{len(self._cached_results) - 1}. " # Shows "0--1" when empty
|
||||
"Run search() first."
|
||||
)
|
||||
```
|
||||
|
||||
### Partial Fix Already in Place
|
||||
The code stores search results in `job.state_data['audio_search_results']` and attempts to use `download_by_id()` which doesn't require cache. However, for YouTube sources, it falls back to cache-based download.
|
||||
|
||||
### Fix Required
|
||||
**Never rely on in-memory cache across requests.** Always use job.state_data.
|
||||
|
||||
In `audio_search.py` `_download_and_start_processing()`:
|
||||
```python
|
||||
# Get the selected result from job state_data (not service cache)
|
||||
search_results = job.state_data.get('audio_search_results', [])
|
||||
selected = search_results[selection_index]
|
||||
|
||||
# For YouTube, construct a fake AudioSearchResult from state_data
|
||||
# and call download_by_id with video URL
|
||||
if selected['provider'] == 'YouTube':
|
||||
result = audio_search_service.download_by_id(
|
||||
source_name='YouTube',
|
||||
source_id=selected.get('source_id'), # Video ID
|
||||
output_dir=temp_dir,
|
||||
download_url=selected.get('url'), # YouTube URL
|
||||
)
|
||||
```
|
||||
|
||||
### Priority
|
||||
**HIGH** - Audio search mode is broken in multi-instance environments.
|
||||
|
||||
---
|
||||
|
||||
## Issue 4: Jobs Stuck in Downloading State
|
||||
|
||||
### Symptoms
|
||||
- Jobs 643983e3, 9726a99b have status `downloading` indefinitely
|
||||
- `state_data.audio_complete: true` - audio worker finished
|
||||
- `state_data.lyrics_progress.stage: "transcribing"` - lyrics worker started
|
||||
- NO `lyrics_complete` flag - lyrics worker never finished
|
||||
- Status never transitions beyond `downloading`
|
||||
|
||||
### Root Cause
|
||||
The lyrics worker appears to crash or timeout during AudioShake transcription without properly failing the job. Possible causes:
|
||||
|
||||
1. **AudioShake API timeout** - transcription takes too long
|
||||
2. **Cloud Run instance killed** - worker running longer than timeout allows
|
||||
3. **Unhandled exception** - error during transcription not caught properly
|
||||
4. **Memory exhaustion** - large audio files causing OOM
|
||||
|
||||
The lyrics worker's exception handling (`lyrics_worker.py:375-383`) should mark job as failed:
|
||||
```python
|
||||
except Exception as e:
|
||||
job_manager.mark_job_failed(
|
||||
job_id=job_id,
|
||||
error_message=f"Lyrics transcription failed: {str(e)}",
|
||||
)
|
||||
```
|
||||
|
||||
But if the Cloud Run instance is killed (e.g., timeout, OOM), this code never runs.
|
||||
|
||||
### Fix Required
|
||||
1. **Add job-level timeouts with cleanup**
|
||||
```python
|
||||
# In job_manager.py - scheduled task to clean up stale jobs
|
||||
async def cleanup_stale_jobs():
|
||||
# Find jobs in 'downloading' state for more than X hours
|
||||
# Check if workers are actually running
|
||||
# Mark as failed if stuck
|
||||
```
|
||||
|
||||
2. **Add heartbeat mechanism**
|
||||
```python
|
||||
# Workers periodically update timestamp
|
||||
job_manager.update_state_data(job_id, 'worker_heartbeat', datetime.now())
|
||||
|
||||
# Background job detects stale heartbeats and fails job
|
||||
```
|
||||
|
||||
3. **Improve AudioShake timeout handling**
|
||||
```python
|
||||
# In lyrics_worker.py, add explicit timeout:
|
||||
async with asyncio.timeout(600): # 10 minute timeout
|
||||
result = await asyncio.to_thread(lyrics_processor.transcribe_lyrics, ...)
|
||||
```
|
||||
|
||||
### Priority
|
||||
**MEDIUM** - Jobs are stuck but not actively failing. Need monitoring/cleanup mechanism.
|
||||
|
||||
---
|
||||
|
||||
## Summary of Fixes
|
||||
|
||||
| Issue | Priority | Fix Complexity | Affected Code |
|
||||
|-------|----------|---------------|---------------|
|
||||
| 1. user_email not set | CRITICAL | Low | `file_upload.py`, `audio_search.py` |
|
||||
| 2. URL download race | HIGH | Medium | `file_upload.py`, `audio_worker.py` |
|
||||
| 3. Audio search cache | HIGH | Medium | `audio_search.py`, `audio_search_service.py` |
|
||||
| 4. Stuck jobs cleanup | MEDIUM | Medium | `job_manager.py`, workers |
|
||||
|
||||
---
|
||||
|
||||
## Implemented Fixes
|
||||
|
||||
### Issue 1: user_email - FIXED
|
||||
**Changes:**
|
||||
- Updated all job creation endpoints to use `AuthResult` type instead of tuple
|
||||
- Extract `auth_result.user_email` and set it on `JobCreate`
|
||||
- Prefer authenticated user's email over client-provided value
|
||||
|
||||
**Files modified:**
|
||||
- `backend/api/routes/file_upload.py` - All job creation endpoints
|
||||
- `backend/api/routes/audio_search.py` - Audio search endpoint
|
||||
|
||||
### Issue 2: URL Download Race Condition - FIXED
|
||||
**Changes:**
|
||||
- For URL jobs, only trigger audio worker initially (not lyrics)
|
||||
- Audio worker triggers lyrics worker after successful URL download
|
||||
- Added `_trigger_audio_worker_only()` function for URL jobs
|
||||
- Added `_trigger_lyrics_worker_after_url_download()` in audio_worker
|
||||
|
||||
**Files modified:**
|
||||
- `backend/api/routes/file_upload.py` - Changed create-from-url to use sequential triggering
|
||||
- `backend/workers/audio_worker.py` - Trigger lyrics worker after URL download
|
||||
|
||||
### Issue 3: Audio Search Cache - FIXED
|
||||
**Changes:**
|
||||
- For YouTube sources without remote flacfetch, download directly using URL from `state_data`
|
||||
- Avoids relying on in-memory cache which doesn't persist across Cloud Run instances
|
||||
- Added fallback warning for cache-based downloads
|
||||
|
||||
**Files modified:**
|
||||
- `backend/api/routes/audio_search.py` - Added direct YouTube download path
|
||||
|
||||
### Issue 4: Stuck Jobs Timeout - FIXED
|
||||
**Changes:**
|
||||
- Added 10-minute timeout around AudioShake transcription call
|
||||
- Uses `asyncio.wait_for` to ensure transcription doesn't hang forever
|
||||
- On timeout, raises exception which marks job as failed
|
||||
|
||||
**Files modified:**
|
||||
- `backend/workers/lyrics_worker.py` - Added timeout handling
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
All 812 backend tests pass after fixes.
|
||||
@@ -0,0 +1,80 @@
|
||||
# PR #90: Payment Flow and Beta Tester Program
|
||||
|
||||
**Date**: 2025-12-28
|
||||
**Branch**: `feature/payment-flow`
|
||||
**PR**: https://github.com/nomadkaraoke/karaoke-gen/pull/90
|
||||
|
||||
## Summary
|
||||
|
||||
Implemented customer payment system with Stripe and a beta tester program for low-friction user acquisition.
|
||||
|
||||
## What Was Built
|
||||
|
||||
### Payment System
|
||||
- Magic link (passwordless) authentication
|
||||
- Stripe Checkout integration with credit packages ($5-$30)
|
||||
- Session management (7-day inactivity, 30-day absolute expiry)
|
||||
- SendGrid email integration for transactional emails
|
||||
|
||||
### Beta Tester Program
|
||||
- Free credit in exchange for feedback promise
|
||||
- Users must accept they may need to correct lyrics
|
||||
- Users must write promise text (min 10 chars)
|
||||
- Feedback form with ratings (1-5 scale):
|
||||
- Overall experience
|
||||
- Ease of use
|
||||
- Lyrics accuracy
|
||||
- Correction experience
|
||||
- Bonus credit for detailed feedback (50+ chars)
|
||||
- Admin endpoints for viewing feedback and stats
|
||||
|
||||
### Buy Site (buy.nomadkaraoke.com)
|
||||
- Landing page with credit packages
|
||||
- Purple gradient beta tester signup section
|
||||
- Auto-redirect to main app after enrollment
|
||||
|
||||
## Key Files Changed
|
||||
|
||||
```
|
||||
backend/
|
||||
├── models/user.py # User, Session, MagicLink, Feedback models
|
||||
├── services/user_service.py # Auth, credits, beta enrollment
|
||||
├── services/stripe_service.py # Stripe checkout & webhooks
|
||||
├── services/email_service.py # SendGrid + email templates
|
||||
└── api/routes/users.py # All user/auth/beta endpoints
|
||||
|
||||
buy-site/
|
||||
├── app/page.tsx # Landing page with beta signup
|
||||
└── lib/api.ts # API client for beta enrollment
|
||||
|
||||
frontend/
|
||||
└── lib/auth.ts # Updated for magic link auth
|
||||
```
|
||||
|
||||
## CodeRabbit Fixes Addressed
|
||||
|
||||
1. Added try/except for invalid credits metadata in Stripe webhook
|
||||
2. Fixed magic link race condition with Firestore transaction
|
||||
3. Moved CSS @import before @tailwind directives
|
||||
4. Fixed broken Open Graph image URL
|
||||
5. Removed insecure fallback admin privileges
|
||||
6. Bumped version to 0.76.26
|
||||
|
||||
## Environment Variables Required
|
||||
|
||||
```
|
||||
STRIPE_SECRET_KEY=sk_live_...
|
||||
STRIPE_WEBHOOK_SECRET=whsec_...
|
||||
SENDGRID_API_KEY=SG....
|
||||
EMAIL_FROM=noreply@nomadkaraoke.com
|
||||
FRONTEND_URL=https://gen.nomadkaraoke.com
|
||||
BUY_URL=https://buy.nomadkaraoke.com
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Deploy and test Stripe in live mode
|
||||
- [ ] Configure SendGrid domain verification
|
||||
- [ ] Deploy buy-site to GitHub Pages
|
||||
- [ ] Add feedback request trigger after job completion
|
||||
- [ ] Monitor beta tester metrics via admin dashboard
|
||||
@@ -0,0 +1,250 @@
|
||||
# Buy Site Architecture Analysis
|
||||
|
||||
**Date**: 2025-12-29
|
||||
**Status**: Analysis Complete - Consolidation Recommended
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The current two-site architecture (buy.nomadkaraoke.com + gen.nomadkaraoke.com) has fundamental issues that break the user authentication flow. **Strong recommendation: Consolidate into a single site** to simplify the architecture, fix auth issues, and improve user experience.
|
||||
|
||||
## Critical Bugs Found
|
||||
|
||||
### 1. Token Storage Key Mismatch
|
||||
|
||||
**Location**: `buy-site/app/page.tsx:95` vs `frontend/lib/api.ts:18`
|
||||
|
||||
```javascript
|
||||
// buy-site stores token as:
|
||||
localStorage.setItem('nomad_karaoke_token', response.session_token);
|
||||
|
||||
// frontend reads token as:
|
||||
localStorage.getItem('karaoke_access_token');
|
||||
```
|
||||
|
||||
Even if the user stays on the same domain, this key mismatch means tokens wouldn't transfer.
|
||||
|
||||
### 2. Cross-Domain localStorage Isolation
|
||||
|
||||
**The fundamental problem**: localStorage is domain-specific. Data stored on `buy.nomadkaraoke.com` is completely inaccessible from `gen.nomadkaraoke.com`.
|
||||
|
||||
This means:
|
||||
- Beta tester enrollment stores a session token on buy.nomadkaraoke.com
|
||||
- Redirect to gen.nomadkaraoke.com loses the token
|
||||
- User appears unauthenticated on the main app
|
||||
|
||||
### 3. Broken Beta Tester Flow
|
||||
|
||||
**Expected user experience**:
|
||||
1. User fills beta form on buy site
|
||||
2. Gets session token
|
||||
3. Redirects to gen site
|
||||
4. Is authenticated and can create karaoke
|
||||
|
||||
**Actual experience**:
|
||||
1. User fills beta form on buy site
|
||||
2. Token stored in buy site's localStorage
|
||||
3. Redirects to gen site (different domain!)
|
||||
4. Token is NOT accessible
|
||||
5. User sees "Authentication Required" message
|
||||
6. User is confused
|
||||
|
||||
## Architecture Comparison
|
||||
|
||||
### Current: Two Sites
|
||||
|
||||
```
|
||||
buy.nomadkaraoke.com (Cloudflare Pages) gen.nomadkaraoke.com (GitHub Pages)
|
||||
├── Landing page ├── Main app
|
||||
├── Pricing ├── Job creation
|
||||
├── Beta enrollment ├── Job management
|
||||
├── Stripe checkout redirect ├── Auth dialog
|
||||
└── localStorage: nomad_karaoke_token └── localStorage: karaoke_access_token
|
||||
↓ ↑
|
||||
└───── REDIRECT (token lost!) ──────────────┘
|
||||
```
|
||||
|
||||
**Problems**:
|
||||
- Cross-domain auth doesn't work with localStorage
|
||||
- Two separate codebases to maintain
|
||||
- Two separate deployments (Cloudflare + GitHub Pages)
|
||||
- Duplicate dependencies (React, Next.js, Tailwind on both)
|
||||
- Confusing user navigation between sites
|
||||
- "Already have credits? Sign in" link goes to unauthenticated state
|
||||
|
||||
### Proposed: Single Site
|
||||
|
||||
```
|
||||
gen.nomadkaraoke.com (GitHub Pages)
|
||||
├── Landing page (new /welcome or / route)
|
||||
├── Pricing (/pricing or landing page section)
|
||||
├── Beta enrollment (modal or dedicated route)
|
||||
├── Stripe checkout redirect
|
||||
├── Main app (/app or /)
|
||||
├── Job creation
|
||||
├── Job management
|
||||
└── localStorage: karaoke_access_token (single key)
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Single domain = single localStorage = no auth issues
|
||||
- One codebase to maintain
|
||||
- One deployment pipeline
|
||||
- Shared components and styles
|
||||
- Clear user navigation
|
||||
- Consistent branding/experience
|
||||
|
||||
## Migration Path
|
||||
|
||||
### Phase 1: Integrate Landing/Pricing into Main Frontend
|
||||
|
||||
1. Create new routes in `frontend/`:
|
||||
- `/welcome` - Landing page (can be set as default for unauthenticated users)
|
||||
- `/pricing` - Pricing section (or integrate into landing)
|
||||
|
||||
2. Move buy-site components to frontend:
|
||||
- Copy pricing UI from `buy-site/app/page.tsx`
|
||||
- Copy beta enrollment form
|
||||
- Reuse existing auth components
|
||||
|
||||
3. Update frontend routing:
|
||||
- Unauthenticated users → Landing page
|
||||
- Authenticated users → Main app
|
||||
|
||||
### Phase 2: Update Auth Flow
|
||||
|
||||
1. Fix token key to be consistent:
|
||||
```javascript
|
||||
localStorage.setItem('karaoke_access_token', token);
|
||||
```
|
||||
|
||||
2. Beta enrollment flow:
|
||||
- Show form on gen.nomadkaraoke.com directly
|
||||
- On success, store token and redirect to app (same domain)
|
||||
- User is immediately authenticated
|
||||
|
||||
### Phase 3: Cleanup
|
||||
|
||||
1. Delete buy-site directory
|
||||
2. Remove Cloudflare Pages deployment
|
||||
3. Delete buy.nomadkaraoke.com DNS record (or redirect to gen)
|
||||
4. Update CI/CD to remove buy-site jobs
|
||||
|
||||
## Files to Delete
|
||||
|
||||
```
|
||||
buy-site/ # Entire directory
|
||||
├── app/
|
||||
│ ├── globals.css
|
||||
│ ├── layout.tsx
|
||||
│ └── page.tsx
|
||||
├── lib/
|
||||
│ └── api.ts
|
||||
├── next.config.mjs
|
||||
├── package.json
|
||||
├── package-lock.json
|
||||
├── postcss.config.mjs
|
||||
├── tailwind.config.js
|
||||
└── tsconfig.json
|
||||
```
|
||||
|
||||
## CI/CD Changes
|
||||
|
||||
In `.github/workflows/ci.yml`:
|
||||
- Remove `buy_site` from path filters
|
||||
- Remove `deploy-buy-site` job
|
||||
|
||||
## DNS/Infrastructure Changes
|
||||
|
||||
- Remove or redirect `buy.nomadkaraoke.com` → `gen.nomadkaraoke.com`
|
||||
- Delete Cloudflare Pages project `nomadkaraoke-buy`
|
||||
|
||||
## Alternative Solutions (Not Recommended)
|
||||
|
||||
### Option A: Shared Auth via URL Token
|
||||
|
||||
Pass token in URL when redirecting:
|
||||
```
|
||||
https://gen.nomadkaraoke.com?token=xxx
|
||||
```
|
||||
|
||||
**Problems**:
|
||||
- Token in URL is a security risk (logs, referrer headers)
|
||||
- Requires frontend to parse and clear URL
|
||||
- Still two codebases to maintain
|
||||
|
||||
### Option B: Cookie with Shared Domain
|
||||
|
||||
Use cookies with `domain=.nomadkaraoke.com`:
|
||||
```javascript
|
||||
document.cookie = `session=${token}; domain=.nomadkaraoke.com; secure; samesite=strict`;
|
||||
```
|
||||
|
||||
**Problems**:
|
||||
- Still two codebases to maintain
|
||||
- Cookie handling adds complexity
|
||||
- CORS issues with API calls
|
||||
|
||||
### Option C: Subdomain with Shared localStorage
|
||||
|
||||
If both sites were on same origin (e.g., `gen.nomadkaraoke.com/buy`), localStorage would be shared.
|
||||
|
||||
**This is essentially the consolidation approach** - just with a different URL structure.
|
||||
|
||||
## Testing the Fix
|
||||
|
||||
Created Playwright e2e test: `frontend/e2e/production-user-journey.spec.ts`
|
||||
|
||||
Key test scenarios:
|
||||
1. Landing page loads correctly
|
||||
2. Beta enrollment validation works
|
||||
3. Token storage is consistent
|
||||
4. Auth persists after redirect
|
||||
5. Cross-domain issues documented (will fail until fixed)
|
||||
|
||||
Run tests:
|
||||
```bash
|
||||
cd frontend
|
||||
npx playwright test production-user-journey --config=playwright.production.config.ts --headed
|
||||
```
|
||||
|
||||
## Email Testing for E2E
|
||||
|
||||
For full end-to-end magic link testing, options evaluated:
|
||||
|
||||
1. **MailSlurp** - Free tier (50 emails/month), good API
|
||||
- Set `MAILSLURP_API_KEY` env var
|
||||
- Creates temporary inbox
|
||||
- Waits for magic link email
|
||||
- Extracts and follows link
|
||||
|
||||
2. **Mailosaur** - Paid, very reliable
|
||||
|
||||
3. **Gmail API** - Works with user's personal Gmail
|
||||
|
||||
For now, tests document the flows but skip actual email verification unless `MAILSLURP_API_KEY` is set.
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Strongly recommend consolidating to a single site** because:
|
||||
|
||||
1. **Fixes fundamental auth bug** - localStorage sharing isn't possible cross-domain
|
||||
2. **Simpler architecture** - One codebase, one deployment
|
||||
3. **Better UX** - No confusing navigation between sites
|
||||
4. **Easier maintenance** - Single set of dependencies
|
||||
5. **Consistent branding** - Same styles/components everywhere
|
||||
6. **Aligns with PRODUCT-VISION.md** - Single gen.nomadkaraoke.com as the product
|
||||
|
||||
## References
|
||||
|
||||
- Product Vision: `docs/PRODUCT-VISION.md`
|
||||
- Frontend auth: `frontend/lib/auth.ts`
|
||||
- Frontend API: `frontend/lib/api.ts`
|
||||
- Buy site page: `buy-site/app/page.tsx`
|
||||
- E2E tests: `frontend/e2e/production-user-journey.spec.ts`
|
||||
|
||||
## Sources
|
||||
|
||||
Research on email testing:
|
||||
- [Mailosaur Playwright guide](https://mailosaur.com/blog/playwright-email-verification)
|
||||
- [MailSlurp Next Auth testing](https://www.mailslurp.com/guides/test-next-auth-magic-links/)
|
||||
- [Better Stack signup testing guide](https://betterstack.com/community/guides/testing/playwright-signup-login/)
|
||||
@@ -0,0 +1,70 @@
|
||||
# Buy-Site Consolidation into Main Frontend
|
||||
|
||||
**Date**: 2025-12-29
|
||||
**PR**: #122
|
||||
**Branch**: `consolidate-buy-site`
|
||||
|
||||
## Summary
|
||||
|
||||
Consolidated the separate `buy-site/` Next.js application into the main `frontend/` application. This eliminates cross-domain issues and simplifies deployment.
|
||||
|
||||
## Problem
|
||||
|
||||
The original architecture had two separate frontends:
|
||||
- `gen.nomadkaraoke.com` - Main karaoke generation app
|
||||
- `buy.nomadkaraoke.com` - Credit purchase and beta enrollment
|
||||
|
||||
This caused a critical bug: **localStorage is domain-isolated**. Auth tokens stored on `buy.nomadkaraoke.com` were invisible to `gen.nomadkaraoke.com`, so users who purchased credits appeared logged out when redirected to the main app.
|
||||
|
||||
## Solution
|
||||
|
||||
Merged all buy-site functionality into the main frontend:
|
||||
- Landing page with pricing, beta enrollment at `/` (root)
|
||||
- Main app (job submission, job list) at `/app`
|
||||
- Payment success page at `/payment/success`
|
||||
|
||||
## Key Changes
|
||||
|
||||
### Route Structure
|
||||
| Before | After |
|
||||
|--------|-------|
|
||||
| `gen.nomadkaraoke.com/` | Main app (auth required) |
|
||||
| `buy.nomadkaraoke.com/` | Landing/pricing |
|
||||
|
||||
| After | Route |
|
||||
|-------|-------|
|
||||
| Landing/pricing | `/` (root) |
|
||||
| Main app | `/app` |
|
||||
| Auth verify | `/auth/verify` |
|
||||
| Payment success | `/payment/success` |
|
||||
|
||||
### Auth Flow
|
||||
- Unauthenticated users see landing page at root
|
||||
- Authenticated users are redirected to `/app`
|
||||
- Magic link verification redirects to `/app`
|
||||
- Payment success redirects to `/app`
|
||||
|
||||
### Files Changed
|
||||
- Created `frontend/app/app/page.tsx` (main app moved from root)
|
||||
- Updated `frontend/app/page.tsx` (landing page from buy-site)
|
||||
- Deleted `frontend/app/welcome/` (no longer needed)
|
||||
- Updated redirect paths in auth/payment pages
|
||||
- Updated E2E tests for new URL structure
|
||||
|
||||
### Deleted
|
||||
- Entire `buy-site/` directory (merged into frontend)
|
||||
- `buy.nomadkaraoke.com` DNS CNAME
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
1. **Cross-domain localStorage isolation** is a common gotcha when splitting apps across subdomains. Unless you need true multi-tenant isolation, keep auth on a single domain.
|
||||
|
||||
2. **Token storage key consistency** matters - we also fixed a mismatch where buy-site stored tokens as `access_token` but frontend expected `auth_token`.
|
||||
|
||||
3. **E2E tests caught the real user journey bug** that unit tests missed - the redirect after payment led to an "auth required" state.
|
||||
|
||||
## Testing
|
||||
|
||||
- Updated `frontend/e2e/production-user-journey.spec.ts` for new URLs
|
||||
- Added MailSlurp integration for email verification testing
|
||||
- All E2E tests pass with new route structure
|
||||
@@ -0,0 +1,484 @@
|
||||
# E2E Test Suite Consolidation Plan
|
||||
|
||||
**Date**: December 29, 2025
|
||||
**Status**: Completed
|
||||
**Author**: Claude (with Andrew's direction)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document outlines a plan to consolidate and improve the frontend E2E test suite. The goal is to create:
|
||||
|
||||
1. **A single, comprehensive production E2E test** that exercises the full user journey with real APIs (manual execution only)
|
||||
2. **A robust regression test suite** using recorded API fixtures that can run in CI without hitting production
|
||||
|
||||
---
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### Test File Inventory
|
||||
|
||||
| File | Lines | Purpose | Test Song | Runs Against |
|
||||
|------|-------|---------|-----------|--------------|
|
||||
| `production-user-journey.spec.ts` | 533 | Landing page, pricing, beta enrollment, auth, token persistence | `piri/dog` (defined but unused) | Production |
|
||||
| `production-e2e.spec.ts` | 319 | Full 8-step karaoke generation flow | `piri/dog` ✅ | Production |
|
||||
| `karaoke-generation.spec.ts` | 570 | App UI, job interactions, search, audio selection | ABBA, LCMDF, Rick Astley ❌ | Local dev (proxy to prod) |
|
||||
| `screenshot-audio-dialog.spec.ts` | 297 | Audio dialog screenshot capture | Radiohead ❌ | Local dev |
|
||||
| `mobile-responsiveness.spec.ts` | 163 | Mobile viewport testing | N/A | Local dev |
|
||||
| `fixtures-example.spec.ts` | 119 | Demo of mock fixture system | N/A (mocked) | Local dev |
|
||||
|
||||
**Total: 6 test files, ~2000 lines of test code**
|
||||
|
||||
### Playwright Configurations
|
||||
|
||||
| Config | Base URL | Test Pattern | Purpose |
|
||||
|--------|----------|--------------|---------|
|
||||
| `playwright.config.ts` | `localhost:3000` | All `e2e/*.spec.ts` | Local dev with proxy to prod API |
|
||||
| `playwright.production.config.ts` | `gen.nomadkaraoke.com` | `production-*.spec.ts` only | Direct production testing |
|
||||
|
||||
### Key Issues Identified
|
||||
|
||||
#### 1. Inconsistent Test Songs
|
||||
Only `production-e2e.spec.ts` uses the designated test song (`piri - dog`). Other files use:
|
||||
- ABBA - Waterloo
|
||||
- LCMDF - Take Me To The Mountains
|
||||
- Rick Astley - Never Gonna Give You Up
|
||||
- Radiohead - Creep
|
||||
|
||||
This causes unnecessary flacfetch API calls and tracker lookups instead of using cached results.
|
||||
|
||||
#### 2. Overlapping Test Coverage
|
||||
- `production-user-journey.spec.ts` and `karaoke-generation.spec.ts` both test homepage/job UI
|
||||
- `production-e2e.spec.ts` duplicates parts of both files
|
||||
- Auth helper functions are copy-pasted across multiple files
|
||||
|
||||
#### 3. Unused Fixture System
|
||||
There's a sophisticated mock/recording system in `fixtures/`:
|
||||
- `recorder.ts` - Records real API calls
|
||||
- `mock-server.ts` - Replays recorded fixtures
|
||||
- `test-helper.ts` - Unified setup interface
|
||||
- `review-cli.ts` - CLI to review/approve recordings
|
||||
|
||||
Only `fixtures-example.spec.ts` uses this system. The original intent was to:
|
||||
1. Run production tests to capture API responses
|
||||
2. Store those responses as fixtures
|
||||
3. Replay them for offline regression testing
|
||||
|
||||
This was never completed because the full production flow wasn't working yet.
|
||||
|
||||
#### 4. CI Only Runs Unit Tests
|
||||
From `.github/workflows/ci.yml`:
|
||||
```yaml
|
||||
name: "Frontend - Unit Tests"
|
||||
run: npm run test:ci # Jest unit tests only
|
||||
```
|
||||
No E2E tests run in CI currently.
|
||||
|
||||
#### 5. Auth Token Configuration
|
||||
- `.env.local` contains `KARAOKE_ACCESS_TOKEN` (session token hash)
|
||||
- Also contains `MAILSLURP_API_KEY` for email testing
|
||||
- Token is loaded by both Playwright configs via custom env file parsing
|
||||
|
||||
### Fixture System Architecture
|
||||
|
||||
```
|
||||
frontend/e2e/fixtures/
|
||||
├── index.ts # Exports all modules
|
||||
├── types.ts # TypeScript interfaces
|
||||
├── recorder.ts # Intercepts & records API calls
|
||||
├── mock-server.ts # Replays recorded fixtures
|
||||
├── test-helper.ts # Unified test setup API
|
||||
└── review-cli.ts # CLI for reviewing recordings
|
||||
|
||||
frontend/e2e/fixtures/data/
|
||||
├── recordings/ # Raw captured API calls (not committed)
|
||||
├── approved/ # Reviewed & approved fixtures (committed)
|
||||
└── sets/ # Named fixture sets for specific scenarios
|
||||
```
|
||||
|
||||
The system supports two modes:
|
||||
- **Recording mode** (`RECORD_FIXTURES=true`): Captures real API responses
|
||||
- **Mock mode** (default): Replays approved fixtures
|
||||
|
||||
---
|
||||
|
||||
## Target Architecture
|
||||
|
||||
### Test Organization
|
||||
|
||||
```
|
||||
frontend/e2e/
|
||||
├── production/ # Real production tests (manual only)
|
||||
│ └── full-user-journey.spec.ts # Single comprehensive test
|
||||
│
|
||||
├── regression/ # Offline tests using fixtures (CI)
|
||||
│ ├── landing-page.spec.ts # Landing, pricing, FAQ
|
||||
│ ├── authentication.spec.ts # Magic link, beta enrollment
|
||||
│ ├── job-management.spec.ts # Create, view, cancel jobs
|
||||
│ ├── karaoke-generation.spec.ts # Audio selection, lyrics review
|
||||
│ └── mobile.spec.ts # Mobile responsiveness
|
||||
│
|
||||
├── fixtures/ # Mock/replay infrastructure
|
||||
│ ├── data/
|
||||
│ │ ├── approved/ # Committed fixture files
|
||||
│ │ └── recordings/ # Temp recordings (gitignored)
|
||||
│ ├── index.ts
|
||||
│ ├── recorder.ts
|
||||
│ ├── mock-server.ts
|
||||
│ └── test-helper.ts
|
||||
│
|
||||
└── helpers/
|
||||
├── auth.ts # Shared auth utilities
|
||||
├── email-testing.ts # MailSlurp helpers
|
||||
└── constants.ts # TEST_ARTIST, TEST_TITLE, URLs
|
||||
```
|
||||
|
||||
### Playwright Configurations
|
||||
|
||||
```
|
||||
frontend/
|
||||
├── playwright.config.ts # CI regression tests (fixtures)
|
||||
├── playwright.production.config.ts # Manual production tests
|
||||
└── playwright.record.config.ts # Fixture recording mode (new)
|
||||
```
|
||||
|
||||
### NPM Scripts
|
||||
|
||||
```json
|
||||
{
|
||||
"test:e2e": "playwright test", // CI regression tests
|
||||
"test:e2e:ui": "playwright test --ui", // Interactive UI
|
||||
"test:e2e:prod": "playwright test --config=playwright.production.config.ts",
|
||||
"test:e2e:prod:headed": "playwright test --config=playwright.production.config.ts --headed",
|
||||
"test:e2e:record": "RECORD_FIXTURES=true playwright test --config=playwright.record.config.ts",
|
||||
"fixtures:review": "npx ts-node e2e/fixtures/review-cli.ts"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Create Production E2E Test (Priority: High)
|
||||
|
||||
**Goal**: Single comprehensive test that validates the entire user journey in production.
|
||||
|
||||
#### Test Flow
|
||||
```
|
||||
1. Landing Page
|
||||
└─ Verify hero, pricing, FAQ sections visible
|
||||
|
||||
2. Beta Enrollment (with MailSlurp)
|
||||
├─ Create test inbox
|
||||
├─ Fill beta form with real feedback promise
|
||||
├─ Submit and verify session token received
|
||||
├─ Verify welcome email received
|
||||
└─ Verify redirected to /app with 2 credits (1 welcome + 1 beta)
|
||||
|
||||
3. Create Karaoke Job
|
||||
├─ Go to Search tab
|
||||
├─ Enter "piri" / "dog"
|
||||
├─ Submit search
|
||||
└─ Verify job created
|
||||
|
||||
4. Audio Selection
|
||||
├─ Wait for audio search to complete
|
||||
├─ Open audio selection dialog
|
||||
├─ Select first result (should be cached)
|
||||
└─ Verify audio download starts
|
||||
|
||||
5. Wait for Processing
|
||||
├─ Poll job status
|
||||
├─ Wait for lyrics transcription (AudioShake)
|
||||
├─ Wait for agentic correction (Vertex AI)
|
||||
└─ Verify job reaches "in_review" status
|
||||
|
||||
6. Lyrics Review
|
||||
├─ Open review UI (new tab)
|
||||
├─ Verify lyrics loaded
|
||||
├─ Click "Preview Video"
|
||||
├─ Wait for preview to generate
|
||||
├─ Click "Complete Review"
|
||||
└─ Verify review completed
|
||||
|
||||
7. Instrumental Selection
|
||||
├─ Wait for job to reach instrumental selection
|
||||
├─ Open instrumental dialog
|
||||
├─ Select "Clean" instrumental
|
||||
└─ Verify selection saved
|
||||
|
||||
8. Wait for Completion
|
||||
├─ Poll job status
|
||||
├─ Wait for video rendering
|
||||
├─ Wait for encoding (4K, 720p, CDG)
|
||||
└─ Verify job status = "completed"
|
||||
|
||||
9. Verify Outputs
|
||||
├─ Check download URLs exist
|
||||
├─ Verify YouTube upload (if enabled)
|
||||
├─ Verify Dropbox upload (if configured)
|
||||
└─ Verify Google Drive upload (if configured)
|
||||
|
||||
10. Cleanup (Optional)
|
||||
└─ Delete test job to avoid clutter
|
||||
```
|
||||
|
||||
#### File: `frontend/e2e/production/full-user-journey.spec.ts`
|
||||
|
||||
Key characteristics:
|
||||
- Uses `piri - dog` consistently
|
||||
- Uses MailSlurp for real email testing
|
||||
- 15-minute timeout (karaoke generation takes time)
|
||||
- Detailed console logging at each step
|
||||
- Screenshots at every milestone
|
||||
- Graceful error handling with diagnostic info
|
||||
|
||||
### Phase 2: Extract Shared Utilities
|
||||
|
||||
**Goal**: Eliminate code duplication and create a single source of truth.
|
||||
|
||||
#### `frontend/e2e/helpers/constants.ts`
|
||||
```typescript
|
||||
export const TEST_SONG = {
|
||||
artist: 'piri',
|
||||
title: 'dog',
|
||||
} as const;
|
||||
|
||||
export const URLS = {
|
||||
production: {
|
||||
frontend: 'https://gen.nomadkaraoke.com',
|
||||
api: 'https://api.nomadkaraoke.com',
|
||||
},
|
||||
local: {
|
||||
frontend: 'http://localhost:3000',
|
||||
api: 'http://localhost:8000',
|
||||
},
|
||||
} as const;
|
||||
|
||||
export const TIMEOUTS = {
|
||||
action: 30_000, // 30s for UI actions
|
||||
expect: 60_000, // 60s for assertions
|
||||
apiCall: 120_000, // 2min for API calls
|
||||
jobProcessing: 600_000, // 10min for job processing
|
||||
fullTest: 900_000, // 15min for full test
|
||||
} as const;
|
||||
```
|
||||
|
||||
#### `frontend/e2e/helpers/auth.ts`
|
||||
```typescript
|
||||
export async function setAuthToken(page: Page, token: string): Promise<void>;
|
||||
export async function clearAuthToken(page: Page): Promise<void>;
|
||||
export async function authenticatePage(page: Page): Promise<boolean>;
|
||||
export async function getAuthToken(): string | undefined;
|
||||
```
|
||||
|
||||
### Phase 3: Record API Fixtures
|
||||
|
||||
**Goal**: Capture real API responses from a successful production run.
|
||||
|
||||
#### Process
|
||||
1. Run the production E2E test with `RECORD_FIXTURES=true`
|
||||
2. The recorder captures all API calls/responses
|
||||
3. Review recordings with `npm run fixtures:review`
|
||||
4. Approve fixtures and commit to `fixtures/data/approved/`
|
||||
|
||||
#### Fixtures to Capture
|
||||
```
|
||||
GET /api/health
|
||||
GET /api/users/credits/packages
|
||||
POST /api/users/beta/enroll
|
||||
GET /api/users/me
|
||||
POST /api/audio-search/search
|
||||
GET /api/jobs
|
||||
GET /api/jobs/:id
|
||||
POST /api/jobs/:id/select-audio
|
||||
GET /api/review/:id/correction-data
|
||||
GET /api/review/:id/audio/:type
|
||||
POST /api/review/:id/preview-video
|
||||
POST /api/review/:id/complete
|
||||
POST /api/jobs/:id/select-instrumental
|
||||
GET /api/jobs/:id/download-urls
|
||||
```
|
||||
|
||||
### Phase 4: Create Regression Test Suite
|
||||
|
||||
**Goal**: Comprehensive offline tests that run in CI.
|
||||
|
||||
#### Test Files
|
||||
|
||||
**`regression/landing-page.spec.ts`**
|
||||
- Hero section displays correctly
|
||||
- Pricing packages show all 4 options
|
||||
- Package selection updates checkout form
|
||||
- FAQ accordion works
|
||||
- Sign in button opens auth dialog
|
||||
- Logged-in users redirect to /app
|
||||
|
||||
**`regression/authentication.spec.ts`**
|
||||
- Magic link dialog opens and validates email
|
||||
- Beta form validates required fields
|
||||
- Beta enrollment success flow (mocked)
|
||||
- Token persistence across navigation
|
||||
- Unauthenticated users redirect to landing
|
||||
|
||||
**`regression/job-management.spec.ts`**
|
||||
- Job list displays correctly
|
||||
- Job cards show status badges
|
||||
- Job details expand on click
|
||||
- Job logs can be viewed
|
||||
- Job can be cancelled
|
||||
- Completed job shows download links
|
||||
|
||||
**`regression/karaoke-generation.spec.ts`**
|
||||
- Search tab shows artist/title inputs
|
||||
- Audio selection dialog opens
|
||||
- Audio options display correctly
|
||||
- Instrumental selection dialog opens
|
||||
- Review UI loads with lyrics data
|
||||
|
||||
**`regression/mobile.spec.ts`**
|
||||
- Pages render without horizontal scroll
|
||||
- Touch targets are adequate size
|
||||
- Forms are usable on mobile
|
||||
- Dialogs fit mobile viewport
|
||||
|
||||
### Phase 5: Update CI Configuration
|
||||
|
||||
**Goal**: Run regression tests on every PR.
|
||||
|
||||
#### `.github/workflows/ci.yml` Changes
|
||||
```yaml
|
||||
frontend-e2e:
|
||||
name: "Frontend - E2E Tests"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [detect-changes]
|
||||
if: needs.detect-changes.outputs.frontend == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
- run: npm ci
|
||||
working-directory: frontend
|
||||
- run: npx playwright install --with-deps chromium
|
||||
working-directory: frontend
|
||||
- run: npm run test:e2e
|
||||
working-directory: frontend
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: failure()
|
||||
with:
|
||||
name: playwright-report
|
||||
path: frontend/playwright-report/
|
||||
```
|
||||
|
||||
### Phase 6: Cleanup Legacy Files
|
||||
|
||||
**Goal**: Remove redundant test files.
|
||||
|
||||
#### Files to Delete
|
||||
- `karaoke-generation.spec.ts` - Functionality merged into regression tests
|
||||
- `screenshot-audio-dialog.spec.ts` - One-off utility, not a test
|
||||
- `production-user-journey.spec.ts` - Merged into production test
|
||||
- `production-e2e.spec.ts` - Merged into production test
|
||||
- `fixtures-example.spec.ts` - Example no longer needed
|
||||
|
||||
#### Files to Keep (Refactored)
|
||||
- `mobile-responsiveness.spec.ts` → `regression/mobile.spec.ts`
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Production Test
|
||||
- [ ] Single test file covers entire user journey
|
||||
- [ ] Uses `piri - dog` exclusively
|
||||
- [ ] MailSlurp validates email delivery
|
||||
- [ ] Verifies all distribution outputs
|
||||
- [ ] Completes in < 15 minutes
|
||||
- [ ] Produces clear diagnostic output on failure
|
||||
|
||||
### Regression Tests
|
||||
- [ ] All tests pass with mocked API
|
||||
- [ ] No network calls to production
|
||||
- [ ] Tests complete in < 2 minutes total
|
||||
- [ ] Mobile tests cover key viewports
|
||||
- [ ] CI runs on every frontend PR
|
||||
|
||||
### Fixture System
|
||||
- [ ] Fixtures captured from real production run
|
||||
- [ ] Fixtures committed and versioned
|
||||
- [ ] Recording mode works for updates
|
||||
- [ ] Review CLI allows fixture management
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
### Production Test Risks
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Flaky due to timing | Generous timeouts, retry logic |
|
||||
| Costs money (AudioShake, Modal) | Only run manually, use cached song |
|
||||
| Creates test data | Use identifiable test data, optional cleanup |
|
||||
| Email delivery delays | MailSlurp has 60s timeout |
|
||||
|
||||
### Regression Test Risks
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Fixtures become stale | Re-record periodically, version control |
|
||||
| API changes break fixtures | Fixture review process catches mismatches |
|
||||
| Mock doesn't match real API | Use exact recorded responses |
|
||||
|
||||
---
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
| Phase | Effort | Dependencies |
|
||||
|-------|--------|--------------|
|
||||
| Phase 1: Production Test | 4-6 hours | None |
|
||||
| Phase 2: Extract Utilities | 1-2 hours | Phase 1 |
|
||||
| Phase 3: Record Fixtures | 1-2 hours | Phase 1 working |
|
||||
| Phase 4: Regression Tests | 4-6 hours | Phase 2, 3 |
|
||||
| Phase 5: CI Configuration | 1-2 hours | Phase 4 |
|
||||
| Phase 6: Cleanup | 1 hour | Phase 4 |
|
||||
|
||||
**Total: 12-19 hours**
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Distribution Testing**: Should the production test verify actual Dropbox/GDrive/YouTube uploads, or just that the job completed successfully?
|
||||
|
||||
2. **Test Data Cleanup**: Should we automatically delete test jobs after the production test runs, or keep them for debugging?
|
||||
|
||||
3. **Fixture Versioning**: Should fixtures be versioned separately from code (e.g., in a fixtures branch)?
|
||||
|
||||
4. **Review UI Testing**: How deeply should we test the lyrics review UI? It opens in a new tab with complex interactions.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Create the production test file (`full-user-journey.spec.ts`)
|
||||
2. Run it against production to identify current bugs
|
||||
3. Fix any bugs discovered
|
||||
4. Once passing, record API fixtures
|
||||
5. Build regression test suite from fixtures
|
||||
6. Add to CI pipeline
|
||||
7. Clean up legacy files
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Current Auth Token
|
||||
|
||||
The `.env.local` file contains:
|
||||
```
|
||||
KARAOKE_ACCESS_TOKEN=9021de39...
|
||||
MAILSLURP_API_KEY=sk_B1mlu...
|
||||
```
|
||||
|
||||
This token should work with the current auth system (session token hash format).
|
||||
@@ -0,0 +1,92 @@
|
||||
# Auth Session Persistence Fix
|
||||
|
||||
**Date**: 2025-12-30
|
||||
**PR**: #130
|
||||
|
||||
## Problem
|
||||
|
||||
Users were being logged out immediately after logging in via magic link:
|
||||
1. User clicks magic link, successfully authenticates
|
||||
2. User is shown as logged in
|
||||
3. On page reload (or even without reload), API calls return 401 Unauthorized
|
||||
4. Session exists and is valid in Firestore, but requests fail
|
||||
|
||||
## Root Causes
|
||||
|
||||
### 1. Frontend Token Caching Bug
|
||||
|
||||
`frontend/lib/api.ts` cached the auth token in a module-level variable:
|
||||
|
||||
```javascript
|
||||
// OLD - Buggy
|
||||
let accessToken: string | null = null;
|
||||
if (typeof window !== 'undefined') {
|
||||
accessToken = localStorage.getItem('karaoke_access_token');
|
||||
}
|
||||
|
||||
export function getAccessToken(): string | null {
|
||||
return accessToken; // Returns cached value, may be stale
|
||||
}
|
||||
```
|
||||
|
||||
**Issue**: In Next.js, if the module initializes before localStorage is properly hydrated, or if there's any module caching behavior, the cached value could be stale or null.
|
||||
|
||||
**Fix**: Always read fresh from localStorage:
|
||||
|
||||
```javascript
|
||||
export function getAccessToken(): string | null {
|
||||
if (typeof window !== 'undefined') {
|
||||
return localStorage.getItem('karaoke_access_token');
|
||||
}
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Backend Timezone Comparison Bug (Main Culprit)
|
||||
|
||||
`backend/services/user_service.py` compared naive and timezone-aware datetimes:
|
||||
|
||||
```python
|
||||
# OLD - Buggy
|
||||
now = datetime.utcnow() # Naive datetime (no timezone)
|
||||
if session.last_activity_at < inactivity_limit: # last_activity_at from Firestore is TZ-aware!
|
||||
# TypeError: can't compare offset-naive and offset-aware datetimes
|
||||
```
|
||||
|
||||
**What happens**: Firestore stores datetimes as timezone-aware. The first time a session is validated, `last_activity_at` is updated in Firestore. On subsequent reads, Firestore returns it as timezone-aware. Comparing this against `datetime.utcnow()` (naive) raises TypeError, caught by the exception handler, returning "An error occurred during validation" → 401.
|
||||
|
||||
**Fix**: Use timezone-aware datetimes throughout:
|
||||
|
||||
```python
|
||||
now = datetime.now(timezone.utc)
|
||||
|
||||
# Normalize session datetimes to be timezone-aware
|
||||
expires_at = session.expires_at
|
||||
if expires_at.tzinfo is None:
|
||||
expires_at = expires_at.replace(tzinfo=timezone.utc)
|
||||
```
|
||||
|
||||
## Additional Changes
|
||||
|
||||
### Firestore Indexes Added to Pulumi
|
||||
|
||||
The `scripts/investigate_user.py` script requires composite indexes. These were created manually and are now tracked in Pulumi IaC:
|
||||
|
||||
- `jobs`: user_email + created_at (user job lists)
|
||||
- `jobs`: status + created_at (admin/system job lists)
|
||||
- `sessions`: is_active + user_email + created_at (active session queries)
|
||||
- `magic_links`: email + created_at (magic link history)
|
||||
|
||||
### Script Deprecation Fix
|
||||
|
||||
Fixed `datetime.utcnow()` deprecation warning in `scripts/investigate_user.py`.
|
||||
|
||||
## Debugging Tools Used
|
||||
|
||||
1. **Direct Firestore query** to verify session exists and is valid
|
||||
2. **Browser DevTools Network tab** to see 401 responses with valid-looking Authorization header
|
||||
3. **`scripts/investigate_user.py`** to check user's sessions, jobs, magic links
|
||||
|
||||
## Key Insight
|
||||
|
||||
The session was valid in the database the entire time. The bug was in the *validation code*, not in session creation or storage. Always check the validation logic when auth fails unexpectedly.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Cloud Output Structure Fix
|
||||
|
||||
**Date**: 2025-12-30
|
||||
|
||||
## Summary
|
||||
|
||||
Fixed three issues causing cloud backend (`karaoke-gen-remote`) output to differ from local CLI (`karaoke-gen`) output:
|
||||
|
||||
1. **YouTube upload silent failure** - Jobs with `--enable_youtube_upload` weren't uploading
|
||||
2. **Missing subdirectories** - Dropbox uploads didn't include stems/ and lyrics/ subfolders
|
||||
3. **Simplified filenames** - Instrumentals named "(Instrumental Clean)" instead of "(Instrumental model_bs_roformer_ep_317_sdr_12.9755.ckpt)"
|
||||
|
||||
## Root Causes
|
||||
|
||||
### YouTube Upload
|
||||
|
||||
The `create-with-upload-urls` endpoint (used by remote CLI) set `youtube_description` but NOT `youtube_description_template`. The video worker checks `youtube_description_template` for YouTube uploads, so they silently failed.
|
||||
|
||||
```python
|
||||
# BEFORE: Only youtube_description was set
|
||||
job_create = JobCreate(
|
||||
youtube_description=body.youtube_description,
|
||||
# youtube_description_template NOT set!
|
||||
)
|
||||
|
||||
# AFTER: Both fields set
|
||||
job_create = JobCreate(
|
||||
youtube_description=body.youtube_description,
|
||||
youtube_description_template=body.youtube_description,
|
||||
)
|
||||
```
|
||||
|
||||
### Missing Subdirectories
|
||||
|
||||
`dropbox_service.upload_folder()` used `os.listdir()` + `os.path.isfile()`, which skipped directories. Changed to `os.walk()` for recursive upload.
|
||||
|
||||
### Simplified Filenames
|
||||
|
||||
Video worker hardcoded "Clean" and "Backing" suffixes instead of using actual model names. Fixed by:
|
||||
|
||||
1. Storing effective model names in job `state_data` during audio separation
|
||||
2. Using those names in `_prepare_distribution_directory()` for proper file naming
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `backend/api/routes/file_upload.py` | Set `youtube_description_template` |
|
||||
| `backend/services/dropbox_service.py` | Recursive `upload_folder()` |
|
||||
| `backend/workers/audio_worker.py` | Store model names in state_data |
|
||||
| `backend/workers/video_worker.py` | Add `_prepare_distribution_directory()` |
|
||||
| `backend/tests/test_dropbox_service.py` | Tests for recursive upload |
|
||||
| `backend/tests/test_workers.py` | Tests for model names and distribution |
|
||||
|
||||
## Expected Output Structure
|
||||
|
||||
After this fix, cloud jobs produce identical output to local CLI:
|
||||
|
||||
```text
|
||||
NOMAD-XXXX - Artist - Title/
|
||||
├── Artist - Title (Final Karaoke Lossless 4k).mp4
|
||||
├── Artist - Title (Final Karaoke Lossless 4k).mkv
|
||||
├── Artist - Title (Instrumental model_bs_roformer_ep_317_sdr_12.9755.ckpt).flac
|
||||
├── Artist - Title (Instrumental +BV mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt).flac
|
||||
├── stems/
|
||||
│ ├── Artist - Title (Instrumental model_bs_roformer_...).flac
|
||||
│ ├── Artist - Title (Vocals model_bs_roformer_...).flac
|
||||
│ ├── Artist - Title (Lead Vocals mel_band_roformer_...).flac
|
||||
│ └── ... (all stems with model names)
|
||||
└── lyrics/
|
||||
└── (intermediate lyrics files)
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
- All 803 existing tests pass
|
||||
- Added new tests for recursive upload and distribution directory preparation
|
||||
@@ -0,0 +1,58 @@
|
||||
# Gemini 3 Flash Agentic Correction Fix
|
||||
|
||||
**Date**: 2025-12-30
|
||||
**PR**: #137
|
||||
|
||||
## Problem
|
||||
|
||||
Karaoke jobs were hanging at the agentic correction stage with no visible errors in Langfuse traces. Cloud Run logs showed:
|
||||
|
||||
```
|
||||
404 Publisher Model `projects/nomadkaraoke/locations/us-central1/publishers/google/models/gemini-3-flash-preview` was not found
|
||||
```
|
||||
|
||||
## Root Cause
|
||||
|
||||
**Gemini 3 models require the `global` location**, not regional endpoints like `us-central1`. This is different from Gemini 2 models which work with regional endpoints.
|
||||
|
||||
Additionally, Gemini 3 returns a **multimodal response format** instead of plain strings:
|
||||
```python
|
||||
# Gemini 2 response.content:
|
||||
"some text response"
|
||||
|
||||
# Gemini 3 response.content:
|
||||
[{'type': 'text', 'text': 'some text response'}]
|
||||
```
|
||||
|
||||
## Solution
|
||||
|
||||
1. Changed default `gcp_location` from `us-central1` to `global` in `config.py`
|
||||
2. Added multimodal response parsing in `langchain_bridge.py`:
|
||||
```python
|
||||
if isinstance(content, list):
|
||||
for part in content:
|
||||
if isinstance(part, dict) and part.get('type') == 'text':
|
||||
return part.get('text', '')
|
||||
```
|
||||
|
||||
## Files Changed
|
||||
|
||||
- `lyrics_transcriber_temp/lyrics_transcriber/correction/agentic/providers/config.py` - Default location
|
||||
- `lyrics_transcriber_temp/lyrics_transcriber/correction/agentic/providers/langchain_bridge.py` - Response parsing
|
||||
- `lyrics_transcriber_temp/lyrics_transcriber/correction/agentic/router.py` - Model name comment
|
||||
|
||||
## Testing
|
||||
|
||||
Created `scripts/test_agentic_correction.py` for local testing without Cloud Run deployment:
|
||||
```bash
|
||||
GOOGLE_CLOUD_PROJECT=nomadkaraoke python scripts/test_agentic_correction.py
|
||||
```
|
||||
|
||||
Four test cases covering sound-alike errors, punctuation, extra words, and no-error scenarios all pass.
|
||||
|
||||
## Key Learnings
|
||||
|
||||
1. **Gemini 3 requires global location** - Not documented prominently; discovered via error message
|
||||
2. **Model response formats differ between versions** - Always handle both string and list content types
|
||||
3. **Local test scripts are essential** - Enabled rapid iteration without Cloud Run deploy cycles
|
||||
4. **Clear LLM cache when debugging** - Cached broken responses can mask fixes: `rm -rf ~/lyrics-transcriber-cache/*.json`
|
||||
@@ -0,0 +1,178 @@
|
||||
# LangChain Vertex AI Migration Plan
|
||||
|
||||
**Date**: 2025-12-30
|
||||
**Status**: Planned
|
||||
**Priority**: Medium (deprecation warning, not yet breaking)
|
||||
|
||||
## Background
|
||||
|
||||
The `ChatVertexAI` class from `langchain-google-vertexai` was deprecated in version 3.2.0 and will be removed in 4.0.0. The recommended replacement is `ChatGoogleGenerativeAI` from the `langchain-google-genai` package.
|
||||
|
||||
Deprecation warning observed in logs:
|
||||
```
|
||||
LangChainDeprecationWarning: The class `ChatVertexAI` was deprecated in LangChain 3.2.0
|
||||
and will be removed in 4.0.0. An updated version of the class exists in the
|
||||
`langchain-google-genai` package and should be used instead.
|
||||
```
|
||||
|
||||
## Current Implementation
|
||||
|
||||
**File**: `lyrics_transcriber_temp/lyrics_transcriber/correction/agentic/providers/model_factory.py`
|
||||
|
||||
```python
|
||||
from langchain_google_vertexai import ChatVertexAI
|
||||
|
||||
model = ChatVertexAI(
|
||||
model=model_name,
|
||||
project=config.gcp_project_id,
|
||||
location=config.gcp_location,
|
||||
timeout=config.request_timeout_seconds,
|
||||
max_retries=config.max_retries,
|
||||
callbacks=callbacks,
|
||||
)
|
||||
```
|
||||
|
||||
## Target Implementation
|
||||
|
||||
```python
|
||||
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||
|
||||
model = ChatGoogleGenerativeAI(
|
||||
model=model_name,
|
||||
vertexai=True, # Keep using Vertex AI authentication
|
||||
project=config.gcp_project_id,
|
||||
location=config.gcp_location,
|
||||
timeout=config.request_timeout_seconds,
|
||||
max_retries=config.max_retries,
|
||||
callbacks=callbacks,
|
||||
)
|
||||
```
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### 1. Update Dependencies
|
||||
|
||||
In `pyproject.toml` (or wherever dependencies are managed):
|
||||
|
||||
```toml
|
||||
# Remove
|
||||
langchain-google-vertexai = "^x.x.x"
|
||||
|
||||
# Add
|
||||
langchain-google-genai = "^4.0.0"
|
||||
```
|
||||
|
||||
### 2. Update model_factory.py
|
||||
|
||||
**File**: `lyrics_transcriber_temp/lyrics_transcriber/correction/agentic/providers/model_factory.py`
|
||||
|
||||
```python
|
||||
# Change import
|
||||
# Old:
|
||||
from langchain_google_vertexai import ChatVertexAI
|
||||
|
||||
# New:
|
||||
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||
```
|
||||
|
||||
Update the `_create_vertexai_model` method:
|
||||
|
||||
```python
|
||||
def _create_vertexai_model(
|
||||
self, model_name: str, callbacks: List[Any], config: ProviderConfig
|
||||
) -> Any:
|
||||
"""Create ChatGoogleGenerativeAI model for Google Gemini via Vertex AI.
|
||||
|
||||
Uses Application Default Credentials (ADC) for authentication.
|
||||
In Cloud Run, this uses the service account automatically.
|
||||
Locally, run: gcloud auth application-default login
|
||||
"""
|
||||
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||
|
||||
model = ChatGoogleGenerativeAI(
|
||||
model=model_name,
|
||||
vertexai=True, # Use Vertex AI backend (not Google AI Studio)
|
||||
project=config.gcp_project_id,
|
||||
location=config.gcp_location,
|
||||
timeout=config.request_timeout_seconds,
|
||||
max_retries=config.max_retries,
|
||||
callbacks=callbacks,
|
||||
)
|
||||
logger.debug(f"🤖 Created Vertex AI model: {model_name} (project={config.gcp_project_id})")
|
||||
return model
|
||||
```
|
||||
|
||||
### 3. Alternative: Environment Variable Approach
|
||||
|
||||
Instead of passing `vertexai=True`, you can set environment variables:
|
||||
|
||||
```bash
|
||||
export GOOGLE_GENAI_USE_VERTEXAI=true
|
||||
export GOOGLE_CLOUD_PROJECT=your-project-id
|
||||
export GOOGLE_CLOUD_LOCATION=global # or us-central1, etc.
|
||||
```
|
||||
|
||||
Then the code simplifies to:
|
||||
|
||||
```python
|
||||
model = ChatGoogleGenerativeAI(
|
||||
model=model_name,
|
||||
timeout=config.request_timeout_seconds,
|
||||
max_retries=config.max_retries,
|
||||
callbacks=callbacks,
|
||||
)
|
||||
```
|
||||
|
||||
## Key Differences
|
||||
|
||||
| Aspect | ChatVertexAI (old) | ChatGoogleGenerativeAI (new) |
|
||||
|--------|-------------------|------------------------------|
|
||||
| Package | `langchain-google-vertexai` | `langchain-google-genai` |
|
||||
| Protocol | gRPC | REST |
|
||||
| Vertex AI Support | Native | Via `vertexai=True` param |
|
||||
| Auth Methods | ADC, Service Account | Same + API keys |
|
||||
| Multimodal | Yes | Yes |
|
||||
|
||||
## Potential Benefits
|
||||
|
||||
1. **REST instead of gRPC**: May resolve the `fork_posix.cc` gRPC warnings we observed:
|
||||
```
|
||||
I0000 00:00:1767135828.790289 30025640 fork_posix.cc:71] Other threads are
|
||||
currently calling into gRPC, skipping fork() handlers
|
||||
```
|
||||
|
||||
2. **Unified SDK**: The `google-genai` SDK is the consolidated approach for all Google AI services.
|
||||
|
||||
3. **Future-proof**: Avoids breaking changes when `langchain-google-vertexai` 4.0 removes `ChatVertexAI`.
|
||||
|
||||
## Potential Risks
|
||||
|
||||
1. **REST vs gRPC latency**: Some users reported slightly slower calls with REST. Monitor LangFuse traces after migration.
|
||||
|
||||
2. **Behavior differences**: Test thoroughly to ensure identical behavior for:
|
||||
- Multimodal response format handling
|
||||
- Timeout behavior
|
||||
- Retry logic
|
||||
- Error messages
|
||||
|
||||
## Testing Plan
|
||||
|
||||
1. Run existing agentic correction unit tests
|
||||
2. Run a full local CLI job with `USE_AGENTIC_AI=1`
|
||||
3. Compare LangFuse traces before/after for:
|
||||
- Response times
|
||||
- Success rates
|
||||
- Error types
|
||||
4. Verify Cloud Run deployment works with new package
|
||||
|
||||
## References
|
||||
|
||||
- [langchain-google-genai 4.0.0 Announcement](https://github.com/langchain-ai/langchain-google/discussions/1422)
|
||||
- [ChatGoogleGenerativeAI Docs](https://reference.langchain.com/python/integrations/langchain_google_genai/)
|
||||
- [LangChain Google Releases](https://github.com/langchain-ai/langchain-google/releases)
|
||||
|
||||
## Notes
|
||||
|
||||
- This migration was identified during investigation of 499 timeout errors in agentic correction (see PR #143)
|
||||
- The gRPC → REST change might actually help with the connection issues we observed
|
||||
- Authentication via Vertex AI (ADC/service accounts) is fully supported in the new package
|
||||
@@ -0,0 +1,95 @@
|
||||
# LangChain Vertex AI to Google GenAI Migration
|
||||
|
||||
**Date**: 2025-12-30
|
||||
**PR**: #145
|
||||
|
||||
## Problem
|
||||
|
||||
Cloud job `b1b2648c` hung silently during agentic correction. No logs appeared after "LangFuse prompt service initialized". Investigation revealed `ChatVertexAI` (langchain-google-vertexai) was hanging during gRPC connection establishment with no timeout.
|
||||
|
||||
## Solution
|
||||
|
||||
Migrated from `langchain-google-vertexai` to `langchain-google-genai`:
|
||||
|
||||
| Before | After |
|
||||
|--------|-------|
|
||||
| `ChatVertexAI` (gRPC-based) | `ChatGoogleGenerativeAI` (REST-based) |
|
||||
| Vertex AI service account auth | GOOGLE_API_KEY (AI Studio) |
|
||||
| No initialization timeout | 30s init + 15s warm-up timeouts |
|
||||
| DEBUG-level logs | INFO-level logs with timing |
|
||||
|
||||
## Key Changes
|
||||
|
||||
### Dependencies
|
||||
```toml
|
||||
# Before
|
||||
langchain-google-vertexai = ">=3.1.1"
|
||||
|
||||
# After
|
||||
langchain-google-genai = ">=2.0.0"
|
||||
```
|
||||
|
||||
### Model Creation
|
||||
```python
|
||||
# Before (gRPC, can hang)
|
||||
from langchain_google_vertexai import ChatVertexAI
|
||||
model = ChatVertexAI(model=model_name, project=project, location=location)
|
||||
|
||||
# After (REST, explicit API key)
|
||||
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||
model = ChatGoogleGenerativeAI(
|
||||
model=model_name,
|
||||
google_api_key=api_key,
|
||||
convert_system_message_to_human=True, # Gemini doesn't support system messages
|
||||
)
|
||||
```
|
||||
|
||||
### Timeout Implementation
|
||||
Used `ThreadPoolExecutor` for cross-platform timeout (works on Windows, Linux, macOS):
|
||||
|
||||
```python
|
||||
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
|
||||
|
||||
with ThreadPoolExecutor(max_workers=1) as executor:
|
||||
future = executor.submit(self._factory.create_chat_model, model, config)
|
||||
try:
|
||||
self._chat_model = future.result(timeout=30)
|
||||
except FuturesTimeoutError:
|
||||
raise InitializationTimeoutError("Model initialization timed out")
|
||||
```
|
||||
|
||||
### Warm-up Pattern
|
||||
Added lightweight "warm-up" request after model creation to establish connection before real work:
|
||||
|
||||
```python
|
||||
warm_up_prompt = 'Respond with exactly: {"status":"ready"}'
|
||||
response = model.invoke([HumanMessage(content=warm_up_prompt)])
|
||||
```
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `pyproject.toml` | Replaced dependency, bumped version |
|
||||
| `lyrics_transcriber_temp/pyproject.toml` | Same |
|
||||
| `.../providers/config.py` | Added `initialization_timeout_seconds`, `warmup_timeout_seconds` |
|
||||
| `.../providers/model_factory.py` | Migrated to `ChatGoogleGenerativeAI`, added timing logs |
|
||||
| `.../providers/langchain_bridge.py` | Added ThreadPoolExecutor timeout wrapper, warm-up, INFO logs |
|
||||
| `.../agent.py` | Added timing logs for classify_gap |
|
||||
|
||||
## Why REST over gRPC
|
||||
|
||||
1. **Simpler debugging** - HTTP requests are easier to trace than gRPC streams
|
||||
2. **No connection pooling issues** - REST is stateless
|
||||
3. **Better timeout behavior** - HTTP timeouts are well-understood
|
||||
4. **Cloud-friendly** - Works better through proxies and load balancers
|
||||
|
||||
## Authentication
|
||||
|
||||
The new approach uses Google AI Studio API keys instead of Vertex AI service accounts:
|
||||
|
||||
- **Get key**: https://aistudio.google.com/app/apikey
|
||||
- **Set env var**: `GOOGLE_API_KEY=<your-key>`
|
||||
- **Benefits**: Simpler setup, no IAM configuration needed
|
||||
|
||||
Note: This uses Google AI Studio quotas, not GCP billing. For high-volume production use, consider keeping Vertex AI with proper timeout handling.
|
||||
@@ -0,0 +1,91 @@
|
||||
# LangFuse Prompt Management Integration
|
||||
|
||||
**Date**: 2025-12-30
|
||||
**PR**: https://github.com/nomadkaraoke/karaoke-gen/pull/140
|
||||
|
||||
## Summary
|
||||
|
||||
Replaced hardcoded LLM prompts in the agentic correction workflow with LangFuse-managed prompts, enabling prompt iteration without redeploying the backend.
|
||||
|
||||
## What Changed
|
||||
|
||||
### New Components
|
||||
|
||||
1. **LangFusePromptService** (`lyrics_transcriber/.../prompts/langfuse_prompts.py`)
|
||||
- Fetches prompts from LangFuse with hardcoded fallback for local development
|
||||
- Fetches few-shot examples from LangFuse datasets
|
||||
- Singleton pattern with fail-fast when LangFuse is configured
|
||||
|
||||
2. **Migration Script** (`scripts/migrate-prompts-to-langfuse.py`)
|
||||
- Uploads `gap-classifier` prompt template to LangFuse
|
||||
- Uploads 9 few-shot examples as `gap-classifier-examples` dataset
|
||||
- Supports `--dry-run`, `--force`, `--verify-only` flags
|
||||
|
||||
3. **LangFuse Vertex AI Service Account** (Pulumi)
|
||||
- `langfuse-vertexai@nomadkaraoke.iam.gserviceaccount.com`
|
||||
- Has `roles/aiplatform.user` for calling Gemini models
|
||||
- JSON key generated for LangFuse to use GCP credits
|
||||
|
||||
### Modified Components
|
||||
|
||||
- `langfuse_integration.py` - Added `fetch_prompt()` and `fetch_dataset()` with label fallback
|
||||
- `classifier.py` - Now uses LangFusePromptService, hardcoded function renamed to `build_classification_prompt_hardcoded`
|
||||
- Default LangFuse host changed to US region (`https://us.cloud.langfuse.com`)
|
||||
|
||||
### Deleted Components
|
||||
|
||||
- `handlers/llm.py` - Legacy LLM handler not being used
|
||||
- `handlers/llm_providers.py` - Legacy provider implementations
|
||||
|
||||
## Configuration
|
||||
|
||||
### LangFuse Credentials (GCP Secret Manager)
|
||||
|
||||
- `langfuse-public-key` - Project: nomadkaraoke
|
||||
- `langfuse-secret-key` - Project: nomadkaraoke
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
LANGFUSE_PUBLIC_KEY=pk-lf-...
|
||||
LANGFUSE_SECRET_KEY=sk-lf-...
|
||||
LANGFUSE_HOST=https://us.cloud.langfuse.com # Optional, this is the default
|
||||
```
|
||||
|
||||
### LangFuse Resources
|
||||
|
||||
- **Prompt**: `gap-classifier` (promote to "production" label for default fetch)
|
||||
- **Dataset**: `gap-classifier-examples` (9 few-shot examples)
|
||||
- **Project**: nomadkaraoke at https://us.cloud.langfuse.com
|
||||
|
||||
## How Prompt Fetching Works
|
||||
|
||||
1. Try to fetch prompt with "production" label (default)
|
||||
2. If not found, fall back to version 1 (for newly created prompts)
|
||||
3. If LangFuse not configured, use hardcoded fallback
|
||||
|
||||
This allows:
|
||||
- New prompts to work immediately after upload
|
||||
- Production prompts to be explicitly promoted
|
||||
- Local development without LangFuse credentials
|
||||
|
||||
## GCP Organization Policy Issue
|
||||
|
||||
Creating service account keys was blocked by organization policy `iam.disableServiceAccountKeyCreation`. Solution:
|
||||
|
||||
1. Temporarily disable policy at organization level
|
||||
2. Create key via `gcloud iam service-accounts keys create`
|
||||
3. Re-enable policy immediately
|
||||
4. Delete local key file after pasting into LangFuse
|
||||
|
||||
This is a one-time operation. The key is now stored in LangFuse, not in any repo or GCP secret.
|
||||
|
||||
## Testing
|
||||
|
||||
All 676 unit tests pass. The prompt service has dedicated tests in `test_langfuse_prompts.py` that mock the LangFuse client.
|
||||
|
||||
## Future Work
|
||||
|
||||
- Add more prompts for other agentic workflows
|
||||
- Set up LangFuse evaluations for prompt quality
|
||||
- Consider A/B testing different prompt versions
|
||||
@@ -0,0 +1,68 @@
|
||||
# Agentic AI Correction Timeout Implementation
|
||||
|
||||
**Date**: 2025-12-31
|
||||
**PR**: #149
|
||||
**Version**: 0.85.0
|
||||
|
||||
## Problem
|
||||
|
||||
Agentic AI lyrics correction was causing stuck jobs in production. Songs with many gaps (74+) could take 30+ minutes to process since each gap takes 10-30 seconds for LLM inference. This blocked the entire job pipeline.
|
||||
|
||||
Example from production logs:
|
||||
```
|
||||
Processing 22/74 gaps (~15s each) = ~18 minutes elapsed
|
||||
Estimated total: 74 gaps × 15s = 18+ minutes
|
||||
```
|
||||
|
||||
## Solution
|
||||
|
||||
Implemented a configurable 3-minute timeout (default) with two layers:
|
||||
|
||||
### Layer 1: Inner Deadline Check (Cooperative)
|
||||
In `corrector.py`, check deadline before processing each gap:
|
||||
```python
|
||||
if deadline and time.time() > deadline:
|
||||
self.logger.warning("AGENTIC TIMEOUT: Deadline exceeded...")
|
||||
break # Return uncorrected transcription for human review
|
||||
```
|
||||
|
||||
### Layer 2: Outer asyncio.wait_for (Safety Net)
|
||||
In `lyrics_worker.py`, wrap the entire transcription call:
|
||||
```python
|
||||
result = await asyncio.wait_for(
|
||||
asyncio.to_thread(lyrics_processor.transcribe_lyrics, ...),
|
||||
timeout=timeout_seconds + 60 # Extra buffer
|
||||
)
|
||||
```
|
||||
|
||||
## Behavior
|
||||
|
||||
| Scenario | What Happens |
|
||||
|----------|--------------|
|
||||
| Completes in <3 min | Normal flow, corrections applied |
|
||||
| Gap loop exceeds deadline | Inner check breaks, returns uncorrected |
|
||||
| LLM hangs indefinitely | Outer timeout fires (rare), job fails |
|
||||
| Any timeout | Job proceeds to human review with raw transcription |
|
||||
|
||||
## Files Changed
|
||||
|
||||
- `backend/config.py` - Added `agentic_correction_timeout_seconds` config
|
||||
- `backend/workers/lyrics_worker.py` - Deadline calculation, outer timeout wrapper
|
||||
- `karaoke_gen/lyrics_processor.py` - Pass deadline to controller
|
||||
- `lyrics_transcriber_temp/lyrics_transcriber/core/controller.py` - Accept deadline parameter
|
||||
- `lyrics_transcriber_temp/lyrics_transcriber/correction/corrector.py` - Deadline check in gap loop
|
||||
|
||||
## Configuration
|
||||
|
||||
Environment variable: `AGENTIC_CORRECTION_TIMEOUT_SECONDS` (default: 180)
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
1. **Break, don't raise**: Inner timeout uses `break` to exit the gap loop gracefully, returning whatever we have (uncorrected). This is better than raising an exception because:
|
||||
- No retry logic needed
|
||||
- Job continues to human review
|
||||
- User can manually correct any issues
|
||||
|
||||
2. **Two-layer protection**: The inner check handles normal cases (many gaps). The outer timeout catches edge cases (single LLM call hanging). Both are needed for reliability.
|
||||
|
||||
3. **Configurable timeout**: Default 3 minutes balances correction quality vs. user experience. It can be adjusted via environment variable per deployment.
|
||||
@@ -0,0 +1,72 @@
|
||||
# ChatGoogleGenerativeAI Vertex AI Auth Fix
|
||||
|
||||
**Date**: 2025-12-31
|
||||
**PR**: #147
|
||||
**Related**: PR #145 (LangChain provider migration)
|
||||
|
||||
## Background
|
||||
|
||||
PR #145 migrated from `ChatVertexAI` (gRPC) to `ChatGoogleGenerativeAI` (REST) to fix silent hangs during model initialization. However, the migration only configured API key authentication, breaking Cloud Run deployments that use service account credentials.
|
||||
|
||||
## Problem
|
||||
|
||||
After deploying PR #145, Cloud Run lyrics worker failed with:
|
||||
```
|
||||
🤖 Failed to initialize chat model: GOOGLE_API_KEY environment variable is required for Google/Gemini models
|
||||
```
|
||||
|
||||
The code required `GOOGLE_API_KEY` even though:
|
||||
- Cloud Run already has `GOOGLE_CLOUD_PROJECT` set
|
||||
- Service account credentials are available via ADC (Application Default Credentials)
|
||||
- Vertex AI IAM permissions were already configured
|
||||
|
||||
## Root Cause
|
||||
|
||||
`ChatGoogleGenerativeAI` supports TWO authentication backends:
|
||||
|
||||
| Backend | Trigger | Auth Method |
|
||||
|---------|---------|-------------|
|
||||
| Vertex AI | `project` param set | Service account / ADC |
|
||||
| Google AI Studio | Only `google_api_key` | API key |
|
||||
|
||||
PR #145 only passed `google_api_key`, forcing the Google AI Studio backend which doesn't support service accounts.
|
||||
|
||||
## Solution
|
||||
|
||||
Pass `project` parameter when `GOOGLE_CLOUD_PROJECT` is set to trigger Vertex AI backend:
|
||||
|
||||
```python
|
||||
# Before (broken)
|
||||
model = ChatGoogleGenerativeAI(model=model_name, google_api_key=api_key)
|
||||
|
||||
# After (works with service account)
|
||||
model_kwargs = {"model": model_name, ...}
|
||||
if project:
|
||||
model_kwargs["project"] = project # Triggers Vertex AI backend
|
||||
if api_key:
|
||||
model_kwargs["google_api_key"] = api_key # Optional fallback
|
||||
model = ChatGoogleGenerativeAI(**model_kwargs)
|
||||
```
|
||||
|
||||
## Key Insight
|
||||
|
||||
`ChatGoogleGenerativeAI` from `langchain-google-genai` is a unified client that supports both Vertex AI and Google AI Studio APIs. The `project` parameter is the key discriminator - when present, it uses Vertex AI with ADC auth.
|
||||
|
||||
## Testing
|
||||
|
||||
Verified that on Cloud Run with `GOOGLE_CLOUD_PROJECT=nomadkaraoke`:
|
||||
- No `GOOGLE_API_KEY` required
|
||||
- Uses attached service account via ADC
|
||||
- REST API (avoids gRPC hang issues from PR #145)
|
||||
|
||||
## Files Changed
|
||||
|
||||
- `lyrics_transcriber_temp/.../providers/model_factory.py` - Added project param logic
|
||||
- `docs/LESSONS-LEARNED.md` - Updated documentation
|
||||
|
||||
## Timing Analysis Context
|
||||
|
||||
This fix was discovered during a timing analysis comparing local vs remote job performance. Key findings:
|
||||
- Remote jobs had 38+ minute delays in lyrics worker
|
||||
- Root cause: gRPC hangs (fixed by PR #145) + auth misconfiguration (fixed by this PR)
|
||||
- With both fixes, remote processing should match local performance
|
||||
@@ -0,0 +1,78 @@
|
||||
# Worker Timeout Fixes
|
||||
|
||||
**Date**: 2026-01-01
|
||||
**PRs**: #153, #154, #155
|
||||
**Status**: Complete
|
||||
|
||||
## Problem
|
||||
|
||||
Jobs were failing at different stages due to worker timeouts:
|
||||
|
||||
1. **Lyrics worker** - Timeout after 600 seconds (10 minutes) during transcription
|
||||
2. **Audio worker** - Killed after 10 minutes by Cloud Tasks default `dispatch_deadline`
|
||||
3. **Video worker** - Timeout after 30 minutes (Cloud Run service timeout)
|
||||
|
||||
## Root Causes
|
||||
|
||||
### Lyrics Transcription Timeout (PR #153)
|
||||
- `TRANSCRIPTION_TIMEOUT_SECONDS` was 600 (10 minutes)
|
||||
- AudioShake transcription + agentic correction can take 15+ minutes for complex songs
|
||||
- **Fix**: Increased to 1200 seconds (20 minutes)
|
||||
|
||||
### Cloud Tasks dispatch_deadline (PR #154)
|
||||
- Cloud Tasks HTTP handlers have a default `dispatch_deadline` of 10 minutes
|
||||
- Audio separation via Modal API takes 15-20+ minutes
|
||||
- Worker was killed before Modal could complete, but no error surfaced
|
||||
- **Symptom**: Job stuck at `downloading` with `audio_complete: false`
|
||||
- **Fix**: Added explicit `dispatch_deadline` per worker type:
|
||||
```python
|
||||
WORKER_DISPATCH_DEADLINES = {
|
||||
"audio": 1800, # 30 min
|
||||
"lyrics": 1500, # 25 min
|
||||
"screens": 600, # 10 min
|
||||
"render-video": 1800, # 30 min
|
||||
"video": 1800, # 30 min
|
||||
}
|
||||
```
|
||||
|
||||
### Video Encoding Timeout (PR #155)
|
||||
- Cloud Run service timeout is 1800 seconds (30 minutes)
|
||||
- `KaraokeFinalise.process()` takes 30-40+ minutes for average songs
|
||||
- Video encoding includes: 4 format encodings, CDG generation, TXT packages
|
||||
- **Symptom**: Job stuck at `encoding` with `updated_at` frozen
|
||||
- **Fix**: Enabled `USE_CLOUD_RUN_JOBS_FOR_VIDEO=true` in `cloudbuild.yaml`
|
||||
- Cloud Run Jobs support up to 1-hour execution (configured in `infrastructure/__main__.py`)
|
||||
|
||||
## Key Insight: dispatch_deadline vs Cloud Run Timeout
|
||||
|
||||
These are different timeouts:
|
||||
|
||||
| Timeout | Default | Max | Controls |
|
||||
|---------|---------|-----|----------|
|
||||
| Cloud Tasks `dispatch_deadline` | 10 min | 30 min | How long Cloud Tasks waits for HTTP response |
|
||||
| Cloud Run service timeout | 5 min | 60 min | How long Cloud Run allows request to run |
|
||||
| Cloud Run Jobs timeout | 10 min | 24 hr | How long a batch job can run |
|
||||
|
||||
For worker tasks that take >30 minutes, Cloud Tasks HTTP targets can't work. Use Cloud Run Jobs instead.
|
||||
|
||||
## Verification
|
||||
|
||||
Job `eb725b2c` completed successfully through the full pipeline:
|
||||
- Audio separation: Complete
|
||||
- Lyrics transcription: Complete
|
||||
- Video encoding: Complete
|
||||
- All 4 video formats generated
|
||||
- All stems, lyrics, and packages available
|
||||
|
||||
## Files Changed
|
||||
|
||||
- `backend/services/worker_service.py` - Added `dispatch_deadline` to Cloud Tasks
|
||||
- `cloudbuild.yaml` - Enabled `USE_CLOUD_RUN_JOBS_FOR_VIDEO=true`
|
||||
- `pyproject.toml` - Version bumps (0.86.4, 0.86.5)
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
1. **Cloud Tasks dispatch_deadline is separate from Cloud Run timeout** - Both need to be configured for long-running workers
|
||||
2. **Default dispatch_deadline (10 min) is often insufficient** - Modal API, video encoding, and other heavy processing need more time
|
||||
3. **Cloud Run Jobs for truly long tasks** - For anything that might take >30 minutes, Cloud Tasks HTTP targets won't work
|
||||
4. **E2E tests reveal timeout issues** - Production E2E tests caught these issues that unit tests missed
|
||||
@@ -0,0 +1,19 @@
|
||||
# Environment variables for local development and testing
|
||||
# Copy this file to .env.local and fill in your values
|
||||
|
||||
# =============================================================================
|
||||
# E2E Testing - Email (for beta enrollment flow)
|
||||
# =============================================================================
|
||||
|
||||
# testmail.app credentials for email testing
|
||||
# Get these from https://testmail.app dashboard
|
||||
TESTMAIL_API_KEY=your-api-key-here
|
||||
TESTMAIL_NAMESPACE=your-namespace-here
|
||||
|
||||
# =============================================================================
|
||||
# E2E Testing - Authentication (optional)
|
||||
# =============================================================================
|
||||
|
||||
# Skip email enrollment by providing an existing access token
|
||||
# Useful for faster local testing when you already have credits
|
||||
# KARAOKE_ACCESS_TOKEN=your-token-here
|
||||
@@ -0,0 +1,286 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import { adminApi, AdminBetaStats, AdminBetaFeedback } from "@/lib/api"
|
||||
import { StatsCard, StatsGrid } from "@/components/admin/stats-card"
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow,
|
||||
} from "@/components/ui/table"
|
||||
import {
|
||||
Users,
|
||||
MessageSquare,
|
||||
Star,
|
||||
RefreshCw,
|
||||
Loader2,
|
||||
ThumbsUp,
|
||||
ThumbsDown,
|
||||
} from "lucide-react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { useToast } from "@/hooks/use-toast"
|
||||
|
||||
export default function AdminBetaPage() {
|
||||
const { toast } = useToast()
|
||||
const [stats, setStats] = useState<AdminBetaStats | null>(null)
|
||||
const [feedback, setFeedback] = useState<AdminBetaFeedback[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
|
||||
const loadData = async () => {
|
||||
try {
|
||||
setLoading(true)
|
||||
const [statsData, feedbackData] = await Promise.all([
|
||||
adminApi.getBetaStats(),
|
||||
adminApi.getBetaFeedback(50),
|
||||
])
|
||||
setStats(statsData)
|
||||
setFeedback(feedbackData.feedback || [])
|
||||
} catch (err: any) {
|
||||
console.error("Failed to load beta data:", err)
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to load beta program data",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
loadData()
|
||||
}, [])
|
||||
|
||||
const renderStars = (rating?: number) => {
|
||||
if (!rating) return "—"
|
||||
return (
|
||||
<div className="flex gap-0.5">
|
||||
{[1, 2, 3, 4, 5].map((star) => (
|
||||
<Star
|
||||
key={star}
|
||||
className={`w-4 h-4 ${
|
||||
star <= rating ? "fill-yellow-400 text-yellow-400" : "text-muted"
|
||||
}`}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
const formatDate = (dateStr?: string) => {
|
||||
if (!dateStr) return "—"
|
||||
const date = new Date(dateStr)
|
||||
if (isNaN(date.getTime())) return "—"
|
||||
return date.toLocaleDateString()
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold tracking-tight">Beta Program</h1>
|
||||
<p className="text-muted-foreground">
|
||||
Manage beta testers and view feedback
|
||||
</p>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" onClick={loadData} disabled={loading}>
|
||||
<RefreshCw className={`w-4 h-4 mr-2 ${loading ? "animate-spin" : ""}`} />
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Stats Grid */}
|
||||
<StatsGrid>
|
||||
<StatsCard
|
||||
title="Total Beta Testers"
|
||||
value={stats?.total_beta_testers ?? 0}
|
||||
icon={Users}
|
||||
loading={loading}
|
||||
/>
|
||||
<StatsCard
|
||||
title="Active Testers"
|
||||
value={stats?.active_testers ?? 0}
|
||||
icon={Users}
|
||||
loading={loading}
|
||||
/>
|
||||
<StatsCard
|
||||
title="Pending Feedback"
|
||||
value={stats?.pending_feedback ?? 0}
|
||||
icon={MessageSquare}
|
||||
loading={loading}
|
||||
/>
|
||||
<StatsCard
|
||||
title="Total Feedback"
|
||||
value={stats?.total_feedback_submissions ?? 0}
|
||||
icon={MessageSquare}
|
||||
loading={loading}
|
||||
/>
|
||||
</StatsGrid>
|
||||
|
||||
{/* Average Ratings */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-base">Average Ratings</CardTitle>
|
||||
<CardDescription>Overall feedback scores from beta testers</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{loading ? (
|
||||
<div className="flex items-center justify-center py-8">
|
||||
<Loader2 className="w-6 h-6 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
) : stats?.average_ratings ? (
|
||||
<div className="grid gap-4 md:grid-cols-4">
|
||||
<div className="rounded-lg border p-4">
|
||||
<p className="text-sm text-muted-foreground">Overall</p>
|
||||
<div className="flex items-center gap-2 mt-1">
|
||||
<span className="text-2xl font-bold">
|
||||
{stats.average_ratings.overall?.toFixed(1) || "—"}
|
||||
</span>
|
||||
<span className="text-muted-foreground">/ 5</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="rounded-lg border p-4">
|
||||
<p className="text-sm text-muted-foreground">Ease of Use</p>
|
||||
<div className="flex items-center gap-2 mt-1">
|
||||
<span className="text-2xl font-bold">
|
||||
{stats.average_ratings.ease_of_use?.toFixed(1) || "—"}
|
||||
</span>
|
||||
<span className="text-muted-foreground">/ 5</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="rounded-lg border p-4">
|
||||
<p className="text-sm text-muted-foreground">Lyrics Accuracy</p>
|
||||
<div className="flex items-center gap-2 mt-1">
|
||||
<span className="text-2xl font-bold">
|
||||
{stats.average_ratings.lyrics_accuracy?.toFixed(1) || "—"}
|
||||
</span>
|
||||
<span className="text-muted-foreground">/ 5</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="rounded-lg border p-4">
|
||||
<p className="text-sm text-muted-foreground">Correction Experience</p>
|
||||
<div className="flex items-center gap-2 mt-1">
|
||||
<span className="text-2xl font-bold">
|
||||
{stats.average_ratings.correction_experience?.toFixed(1) || "—"}
|
||||
</span>
|
||||
<span className="text-muted-foreground">/ 5</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<p className="text-muted-foreground">No ratings available yet</p>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Feedback List */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-base">Recent Feedback</CardTitle>
|
||||
<CardDescription>Feedback submissions from beta testers</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{loading ? (
|
||||
<div className="flex items-center justify-center py-8">
|
||||
<Loader2 className="w-6 h-6 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
) : feedback.length === 0 ? (
|
||||
<p className="text-muted-foreground text-center py-8">No feedback submissions yet</p>
|
||||
) : (
|
||||
<div className="space-y-4">
|
||||
{feedback.map((item) => (
|
||||
<div key={item.id} className="border rounded-lg p-4">
|
||||
<div className="flex items-start justify-between gap-4">
|
||||
<div className="flex-1">
|
||||
<div className="flex items-center gap-2 mb-2">
|
||||
<span className="font-medium text-sm">{item.user_email}</span>
|
||||
{item.job_id && (
|
||||
<Badge variant="outline" className="text-xs">
|
||||
Job: {item.job_id}
|
||||
</Badge>
|
||||
)}
|
||||
<span className="text-xs text-muted-foreground">
|
||||
{formatDate(item.created_at)}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{/* Ratings */}
|
||||
<div className="flex flex-wrap gap-4 mb-3 text-sm">
|
||||
<div className="flex items-center gap-1">
|
||||
<span className="text-muted-foreground">Overall:</span>
|
||||
{renderStars(item.overall_rating)}
|
||||
</div>
|
||||
{item.ease_of_use_rating && (
|
||||
<div className="flex items-center gap-1">
|
||||
<span className="text-muted-foreground">Ease:</span>
|
||||
{renderStars(item.ease_of_use_rating)}
|
||||
</div>
|
||||
)}
|
||||
{item.lyrics_accuracy_rating && (
|
||||
<div className="flex items-center gap-1">
|
||||
<span className="text-muted-foreground">Accuracy:</span>
|
||||
{renderStars(item.lyrics_accuracy_rating)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Comments */}
|
||||
{item.what_went_well && (
|
||||
<div className="mb-2">
|
||||
<p className="text-xs text-muted-foreground">What went well:</p>
|
||||
<p className="text-sm">{item.what_went_well}</p>
|
||||
</div>
|
||||
)}
|
||||
{item.what_could_improve && (
|
||||
<div className="mb-2">
|
||||
<p className="text-xs text-muted-foreground">What could improve:</p>
|
||||
<p className="text-sm">{item.what_could_improve}</p>
|
||||
</div>
|
||||
)}
|
||||
{item.additional_comments && (
|
||||
<div>
|
||||
<p className="text-xs text-muted-foreground">Additional comments:</p>
|
||||
<p className="text-sm">{item.additional_comments}</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Recommend/Use Again */}
|
||||
<div className="flex flex-col gap-1 text-sm">
|
||||
{item.would_recommend !== undefined && (
|
||||
<div className="flex items-center gap-1">
|
||||
{item.would_recommend ? (
|
||||
<ThumbsUp className="w-4 h-4 text-green-500" />
|
||||
) : (
|
||||
<ThumbsDown className="w-4 h-4 text-red-500" />
|
||||
)}
|
||||
<span className="text-xs text-muted-foreground">Recommend</span>
|
||||
</div>
|
||||
)}
|
||||
{item.would_use_again !== undefined && (
|
||||
<div className="flex items-center gap-1">
|
||||
{item.would_use_again ? (
|
||||
<ThumbsUp className="w-4 h-4 text-green-500" />
|
||||
) : (
|
||||
<ThumbsDown className="w-4 h-4 text-red-500" />
|
||||
)}
|
||||
<span className="text-xs text-muted-foreground">Use Again</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,428 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import { useSearchParams, useRouter } from "next/navigation"
|
||||
import { api, adminApi, Job } from "@/lib/api"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import {
|
||||
Accordion,
|
||||
AccordionContent,
|
||||
AccordionItem,
|
||||
AccordionTrigger,
|
||||
} from "@/components/ui/accordion"
|
||||
import {
|
||||
AlertDialog,
|
||||
AlertDialogAction,
|
||||
AlertDialogCancel,
|
||||
AlertDialogContent,
|
||||
AlertDialogDescription,
|
||||
AlertDialogFooter,
|
||||
AlertDialogHeader,
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog"
|
||||
import {
|
||||
ArrowLeft,
|
||||
RefreshCw,
|
||||
Loader2,
|
||||
Trash2,
|
||||
User,
|
||||
Clock,
|
||||
FileText,
|
||||
XCircle,
|
||||
} from "lucide-react"
|
||||
import { useToast } from "@/hooks/use-toast"
|
||||
|
||||
export default function AdminJobDetailPage() {
|
||||
const searchParams = useSearchParams()
|
||||
const router = useRouter()
|
||||
const { toast } = useToast()
|
||||
const jobId = searchParams.get("id") || ""
|
||||
|
||||
const [job, setJob] = useState<Job | null>(null)
|
||||
const [logs, setLogs] = useState<any[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [logsLoading, setLogsLoading] = useState(false)
|
||||
|
||||
// Delete dialog
|
||||
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false)
|
||||
const [deleting, setDeleting] = useState(false)
|
||||
|
||||
const loadJob = async () => {
|
||||
if (!jobId) {
|
||||
setLoading(false)
|
||||
return
|
||||
}
|
||||
try {
|
||||
setLoading(true)
|
||||
const data = await api.getJob(jobId)
|
||||
setJob(data)
|
||||
} catch (err: any) {
|
||||
console.error("Failed to load job:", err)
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to load job",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
const loadLogs = async () => {
|
||||
if (!jobId) return
|
||||
try {
|
||||
setLogsLoading(true)
|
||||
const data = await api.getJobLogs(jobId, 200)
|
||||
setLogs(data)
|
||||
} catch (err: any) {
|
||||
console.error("Failed to load logs:", err)
|
||||
} finally {
|
||||
setLogsLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
loadJob()
|
||||
loadLogs()
|
||||
}, [jobId])
|
||||
|
||||
const handleDelete = async () => {
|
||||
try {
|
||||
setDeleting(true)
|
||||
await adminApi.deleteJob(jobId)
|
||||
toast({
|
||||
title: "Job Deleted",
|
||||
description: `Job ${jobId} has been deleted`,
|
||||
})
|
||||
router.push("/admin/jobs")
|
||||
} catch (err: any) {
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to delete job",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setDeleting(false)
|
||||
}
|
||||
}
|
||||
|
||||
const getStatusVariant = (status: string) => {
|
||||
if (status === "complete" || status === "prep_complete") return "default"
|
||||
if (status === "failed") return "destructive"
|
||||
if (status === "cancelled") return "outline"
|
||||
if (status.includes("awaiting")) return "secondary"
|
||||
return "secondary"
|
||||
}
|
||||
|
||||
const getLogLevelColor = (level: string) => {
|
||||
switch (level?.toUpperCase()) {
|
||||
case "ERROR":
|
||||
return "text-red-500"
|
||||
case "WARNING":
|
||||
return "text-yellow-500"
|
||||
case "INFO":
|
||||
return "text-blue-500"
|
||||
case "DEBUG":
|
||||
return "text-gray-500"
|
||||
default:
|
||||
return "text-foreground"
|
||||
}
|
||||
}
|
||||
|
||||
const formatDate = (dateStr?: string) => {
|
||||
if (!dateStr) return "—"
|
||||
return new Date(dateStr).toLocaleString()
|
||||
}
|
||||
|
||||
if (!jobId) {
|
||||
return (
|
||||
<div className="text-center py-12">
|
||||
<p className="text-muted-foreground">No job ID provided</p>
|
||||
<Button variant="outline" className="mt-4" onClick={() => router.push("/admin/jobs")}>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Jobs
|
||||
</Button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
if (loading) {
|
||||
return (
|
||||
<div className="flex items-center justify-center py-12">
|
||||
<Loader2 className="w-8 h-8 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
if (!job) {
|
||||
return (
|
||||
<div className="text-center py-12">
|
||||
<p className="text-muted-foreground">Job not found</p>
|
||||
<Button variant="outline" className="mt-4" onClick={() => router.push("/admin/jobs")}>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Jobs
|
||||
</Button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-4">
|
||||
<Button variant="ghost" size="icon" onClick={() => router.push("/admin/jobs")}>
|
||||
<ArrowLeft className="w-4 h-4" />
|
||||
</Button>
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold tracking-tight flex items-center gap-2">
|
||||
Job {job.job_id}
|
||||
<Badge variant={getStatusVariant(job.status)}>
|
||||
{job.status.replace(/_/g, " ")}
|
||||
</Badge>
|
||||
</h1>
|
||||
{job.artist && job.title && (
|
||||
<p className="text-muted-foreground">{job.artist} - {job.title}</p>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex gap-2">
|
||||
<Button variant="outline" size="sm" onClick={() => { loadJob(); loadLogs(); }} disabled={loading}>
|
||||
<RefreshCw className={`w-4 h-4 mr-2 ${loading ? "animate-spin" : ""}`} />
|
||||
Refresh
|
||||
</Button>
|
||||
<Button
|
||||
variant="destructive"
|
||||
size="sm"
|
||||
onClick={() => setDeleteDialogOpen(true)}
|
||||
>
|
||||
<Trash2 className="w-4 h-4 mr-2" />
|
||||
Delete
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Overview Cards */}
|
||||
<div className="grid gap-4 md:grid-cols-4">
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
<User className="w-4 h-4" />
|
||||
User
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-sm truncate">
|
||||
{job.user_email || "Unknown"}
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
<Clock className="w-4 h-4" />
|
||||
Created
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-sm">{formatDate(job.created_at)}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
<Clock className="w-4 h-4" />
|
||||
Updated
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-sm">{formatDate(job.updated_at)}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
Progress
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="flex items-center gap-2">
|
||||
<div className="flex-1 h-2 bg-muted rounded-full overflow-hidden">
|
||||
<div
|
||||
className="h-full bg-primary transition-all"
|
||||
style={{ width: `${job.progress}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="text-sm font-medium">{job.progress}%</span>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
|
||||
{/* Error Message */}
|
||||
{job.error_message && (
|
||||
<Card className="border-destructive">
|
||||
<CardHeader>
|
||||
<CardTitle className="text-destructive flex items-center gap-2">
|
||||
<XCircle className="w-5 h-5" />
|
||||
Error
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-sm">{job.error_message}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Accordion Sections */}
|
||||
<Accordion type="multiple" className="w-full" defaultValue={["timeline", "logs"]}>
|
||||
{/* Timeline */}
|
||||
<AccordionItem value="timeline">
|
||||
<AccordionTrigger>
|
||||
<div className="flex items-center gap-2">
|
||||
<Clock className="w-4 h-4" />
|
||||
Timeline ({job.timeline?.length || 0} events)
|
||||
</div>
|
||||
</AccordionTrigger>
|
||||
<AccordionContent>
|
||||
<div className="space-y-2 max-h-[400px] overflow-y-auto">
|
||||
{job.timeline && job.timeline.length > 0 ? (
|
||||
job.timeline.map((event, i) => (
|
||||
<div
|
||||
key={i}
|
||||
className="flex items-start gap-3 text-sm border-l-2 border-muted pl-4 py-1"
|
||||
>
|
||||
<div className="flex-1">
|
||||
<div className="flex items-center gap-2">
|
||||
<Badge variant="outline" className="text-xs">
|
||||
{event.status}
|
||||
</Badge>
|
||||
{event.progress !== undefined && (
|
||||
<span className="text-muted-foreground">
|
||||
{event.progress}%
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
{event.message && (
|
||||
<p className="text-muted-foreground mt-1">{event.message}</p>
|
||||
)}
|
||||
</div>
|
||||
<span className="text-xs text-muted-foreground whitespace-nowrap">
|
||||
{formatDate(event.timestamp)}
|
||||
</span>
|
||||
</div>
|
||||
))
|
||||
) : (
|
||||
<p className="text-muted-foreground text-sm">No timeline events</p>
|
||||
)}
|
||||
</div>
|
||||
</AccordionContent>
|
||||
</AccordionItem>
|
||||
|
||||
{/* Worker Logs */}
|
||||
<AccordionItem value="logs">
|
||||
<AccordionTrigger>
|
||||
<div className="flex items-center gap-2">
|
||||
<FileText className="w-4 h-4" />
|
||||
Worker Logs ({logs.length} entries)
|
||||
{logsLoading && <Loader2 className="w-4 h-4 animate-spin ml-2" />}
|
||||
</div>
|
||||
</AccordionTrigger>
|
||||
<AccordionContent>
|
||||
<div className="space-y-1 max-h-[500px] overflow-y-auto font-mono text-xs">
|
||||
{logs.length > 0 ? (
|
||||
logs.map((log, i) => (
|
||||
<div key={i} className="flex gap-2 py-0.5 border-b border-muted/50">
|
||||
<span className="text-muted-foreground whitespace-nowrap">
|
||||
{log.timestamp ? new Date(log.timestamp).toLocaleTimeString() : "—"}
|
||||
</span>
|
||||
<span className={`w-16 ${getLogLevelColor(log.level)}`}>
|
||||
[{log.level}]
|
||||
</span>
|
||||
<span className="text-muted-foreground w-20">
|
||||
{log.worker || "—"}
|
||||
</span>
|
||||
<span className="flex-1 break-all">{log.message}</span>
|
||||
</div>
|
||||
))
|
||||
) : (
|
||||
<p className="text-muted-foreground">No logs available</p>
|
||||
)}
|
||||
</div>
|
||||
</AccordionContent>
|
||||
</AccordionItem>
|
||||
|
||||
{/* State Data */}
|
||||
<AccordionItem value="state">
|
||||
<AccordionTrigger>
|
||||
<div className="flex items-center gap-2">
|
||||
<FileText className="w-4 h-4" />
|
||||
State Data
|
||||
</div>
|
||||
</AccordionTrigger>
|
||||
<AccordionContent>
|
||||
<pre className="text-xs bg-muted p-4 rounded-md overflow-x-auto max-h-[400px]">
|
||||
{JSON.stringify(job.state_data || {}, null, 2)}
|
||||
</pre>
|
||||
</AccordionContent>
|
||||
</AccordionItem>
|
||||
|
||||
{/* File URLs */}
|
||||
<AccordionItem value="files">
|
||||
<AccordionTrigger>
|
||||
<div className="flex items-center gap-2">
|
||||
<FileText className="w-4 h-4" />
|
||||
File URLs
|
||||
</div>
|
||||
</AccordionTrigger>
|
||||
<AccordionContent>
|
||||
<pre className="text-xs bg-muted p-4 rounded-md overflow-x-auto max-h-[400px]">
|
||||
{JSON.stringify(job.file_urls || {}, null, 2)}
|
||||
</pre>
|
||||
</AccordionContent>
|
||||
</AccordionItem>
|
||||
|
||||
{/* Raw Job Data */}
|
||||
<AccordionItem value="raw">
|
||||
<AccordionTrigger>
|
||||
<div className="flex items-center gap-2">
|
||||
<FileText className="w-4 h-4" />
|
||||
Raw Job Data
|
||||
</div>
|
||||
</AccordionTrigger>
|
||||
<AccordionContent>
|
||||
<pre className="text-xs bg-muted p-4 rounded-md overflow-x-auto max-h-[500px]">
|
||||
{JSON.stringify(job, null, 2)}
|
||||
</pre>
|
||||
</AccordionContent>
|
||||
</AccordionItem>
|
||||
</Accordion>
|
||||
|
||||
{/* Delete Confirmation Dialog */}
|
||||
<AlertDialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogTitle>Delete Job</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
Are you sure you want to delete job {jobId}? This action cannot be
|
||||
undone and will also delete all associated files.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel>Cancel</AlertDialogCancel>
|
||||
<AlertDialogAction
|
||||
onClick={handleDelete}
|
||||
disabled={deleting}
|
||||
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
|
||||
>
|
||||
{deleting && <Loader2 className="w-4 h-4 mr-2 animate-spin" />}
|
||||
Delete
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,327 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState, useCallback } from "react"
|
||||
import { useRouter } from "next/navigation"
|
||||
import { adminApi, Job } from "@/lib/api"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow,
|
||||
} from "@/components/ui/table"
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select"
|
||||
import {
|
||||
AlertDialog,
|
||||
AlertDialogAction,
|
||||
AlertDialogCancel,
|
||||
AlertDialogContent,
|
||||
AlertDialogDescription,
|
||||
AlertDialogFooter,
|
||||
AlertDialogHeader,
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog"
|
||||
import {
|
||||
Search,
|
||||
RefreshCw,
|
||||
Loader2,
|
||||
Trash2,
|
||||
ExternalLink,
|
||||
} from "lucide-react"
|
||||
import { useToast } from "@/hooks/use-toast"
|
||||
|
||||
const statusOptions = [
|
||||
{ value: "all", label: "All Statuses" },
|
||||
{ value: "pending", label: "Pending" },
|
||||
{ value: "downloading", label: "Downloading" },
|
||||
{ value: "separating_stage1", label: "Separating (Stage 1)" },
|
||||
{ value: "separating_stage2", label: "Separating (Stage 2)" },
|
||||
{ value: "transcribing", label: "Transcribing" },
|
||||
{ value: "correcting", label: "Correcting" },
|
||||
{ value: "awaiting_review", label: "Awaiting Review" },
|
||||
{ value: "in_review", label: "In Review" },
|
||||
{ value: "rendering_video", label: "Rendering Video" },
|
||||
{ value: "awaiting_instrumental_selection", label: "Awaiting Instrumental" },
|
||||
{ value: "generating_video", label: "Generating Video" },
|
||||
{ value: "encoding", label: "Encoding" },
|
||||
{ value: "complete", label: "Complete" },
|
||||
{ value: "failed", label: "Failed" },
|
||||
{ value: "cancelled", label: "Cancelled" },
|
||||
]
|
||||
|
||||
export default function AdminJobsPage() {
|
||||
const router = useRouter()
|
||||
const { toast } = useToast()
|
||||
const [jobs, setJobs] = useState<Job[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [statusFilter, setStatusFilter] = useState("all")
|
||||
const [userEmailFilter, setUserEmailFilter] = useState("")
|
||||
const [searchInput, setSearchInput] = useState("")
|
||||
|
||||
// Delete dialog
|
||||
const [deleteDialogOpen, setDeleteDialogOpen] = useState(false)
|
||||
const [jobToDelete, setJobToDelete] = useState<Job | null>(null)
|
||||
const [deleting, setDeleting] = useState(false)
|
||||
|
||||
const loadJobs = useCallback(async () => {
|
||||
try {
|
||||
setLoading(true)
|
||||
const data = await adminApi.listAllJobs({
|
||||
status: statusFilter !== "all" ? statusFilter : undefined,
|
||||
user_email: userEmailFilter || undefined,
|
||||
limit: 100,
|
||||
})
|
||||
setJobs(data)
|
||||
} catch (err: any) {
|
||||
console.error("Failed to load jobs:", err)
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to load jobs",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}, [statusFilter, userEmailFilter, toast])
|
||||
|
||||
useEffect(() => {
|
||||
loadJobs()
|
||||
}, [loadJobs])
|
||||
|
||||
const handleSearch = () => {
|
||||
setUserEmailFilter(searchInput)
|
||||
}
|
||||
|
||||
const handleKeyPress = (e: React.KeyboardEvent) => {
|
||||
if (e.key === "Enter") {
|
||||
handleSearch()
|
||||
}
|
||||
}
|
||||
|
||||
const handleDelete = async () => {
|
||||
if (!jobToDelete) return
|
||||
|
||||
try {
|
||||
setDeleting(true)
|
||||
await adminApi.deleteJob(jobToDelete.job_id)
|
||||
toast({
|
||||
title: "Job Deleted",
|
||||
description: `Job ${jobToDelete.job_id} has been deleted`,
|
||||
})
|
||||
setDeleteDialogOpen(false)
|
||||
setJobToDelete(null)
|
||||
loadJobs()
|
||||
} catch (err: any) {
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to delete job",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setDeleting(false)
|
||||
}
|
||||
}
|
||||
|
||||
const getStatusVariant = (status: string) => {
|
||||
if (status === "complete" || status === "prep_complete") return "default"
|
||||
if (status === "failed") return "destructive"
|
||||
if (status === "cancelled") return "outline"
|
||||
if (status.includes("awaiting")) return "secondary"
|
||||
return "secondary"
|
||||
}
|
||||
|
||||
const formatDate = (dateStr?: string) => {
|
||||
if (!dateStr) return "—"
|
||||
return new Date(dateStr).toLocaleString()
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold tracking-tight">Jobs</h1>
|
||||
<p className="text-muted-foreground">
|
||||
View and manage all jobs in the system
|
||||
</p>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" onClick={loadJobs} disabled={loading}>
|
||||
<RefreshCw className={`w-4 h-4 mr-2 ${loading ? "animate-spin" : ""}`} />
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Filters */}
|
||||
<div className="flex flex-col sm:flex-row gap-4">
|
||||
<div className="flex-1 flex gap-2">
|
||||
<Input
|
||||
placeholder="Filter by user email..."
|
||||
value={searchInput}
|
||||
onChange={(e) => setSearchInput(e.target.value)}
|
||||
onKeyDown={handleKeyPress}
|
||||
className="max-w-sm"
|
||||
/>
|
||||
<Button variant="secondary" onClick={handleSearch}>
|
||||
<Search className="w-4 h-4" />
|
||||
</Button>
|
||||
</div>
|
||||
<Select value={statusFilter} onValueChange={setStatusFilter}>
|
||||
<SelectTrigger className="w-[200px]">
|
||||
<SelectValue placeholder="Filter by status" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
{statusOptions.map((option) => (
|
||||
<SelectItem key={option.value} value={option.value}>
|
||||
{option.label}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
|
||||
{/* Table */}
|
||||
<div className="rounded-md border">
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>Job ID</TableHead>
|
||||
<TableHead>User</TableHead>
|
||||
<TableHead>Artist / Title</TableHead>
|
||||
<TableHead>Status</TableHead>
|
||||
<TableHead>Progress</TableHead>
|
||||
<TableHead>Created</TableHead>
|
||||
<TableHead className="text-right">Actions</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{loading ? (
|
||||
<TableRow>
|
||||
<TableCell colSpan={7} className="text-center py-8">
|
||||
<Loader2 className="w-6 h-6 animate-spin mx-auto text-muted-foreground" />
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
) : jobs.length === 0 ? (
|
||||
<TableRow>
|
||||
<TableCell colSpan={7} className="text-center py-8 text-muted-foreground">
|
||||
No jobs found
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
) : (
|
||||
jobs.map((job) => (
|
||||
<TableRow
|
||||
key={job.job_id}
|
||||
className="cursor-pointer hover:bg-muted/50"
|
||||
onClick={() => router.push(`/admin/jobs/${job.job_id}`)}
|
||||
>
|
||||
<TableCell className="font-mono text-sm">{job.job_id}</TableCell>
|
||||
<TableCell className="text-sm text-muted-foreground max-w-[150px] truncate">
|
||||
{job.user_email || "—"}
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
{job.artist && job.title ? (
|
||||
<span className="max-w-[200px] truncate block">
|
||||
{job.artist} - {job.title}
|
||||
</span>
|
||||
) : job.filename ? (
|
||||
<span className="max-w-[200px] truncate block text-muted-foreground">
|
||||
{job.filename}
|
||||
</span>
|
||||
) : (
|
||||
"—"
|
||||
)}
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Badge variant={getStatusVariant(job.status)}>
|
||||
{job.status.replace(/_/g, " ")}
|
||||
</Badge>
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<div className="flex items-center gap-2">
|
||||
<div className="w-16 h-2 bg-muted rounded-full overflow-hidden">
|
||||
<div
|
||||
className="h-full bg-primary transition-all"
|
||||
style={{ width: `${job.progress}%` }}
|
||||
/>
|
||||
</div>
|
||||
<span className="text-xs text-muted-foreground">
|
||||
{job.progress}%
|
||||
</span>
|
||||
</div>
|
||||
</TableCell>
|
||||
<TableCell className="text-sm">
|
||||
{formatDate(job.created_at)}
|
||||
</TableCell>
|
||||
<TableCell className="text-right">
|
||||
<div className="flex justify-end gap-1" onClick={(e) => e.stopPropagation()}>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={() => router.push(`/admin/jobs/${job.job_id}`)}
|
||||
title="View details"
|
||||
>
|
||||
<ExternalLink className="w-4 h-4" />
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={() => {
|
||||
setJobToDelete(job)
|
||||
setDeleteDialogOpen(true)
|
||||
}}
|
||||
title="Delete job"
|
||||
className="text-destructive hover:text-destructive"
|
||||
>
|
||||
<Trash2 className="w-4 h-4" />
|
||||
</Button>
|
||||
</div>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
))
|
||||
)}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</div>
|
||||
|
||||
{/* Summary */}
|
||||
<p className="text-sm text-muted-foreground">
|
||||
Showing {jobs.length} jobs
|
||||
{statusFilter !== "all" && ` with status "${statusFilter}"`}
|
||||
{userEmailFilter && ` for user "${userEmailFilter}"`}
|
||||
</p>
|
||||
|
||||
{/* Delete Confirmation Dialog */}
|
||||
<AlertDialog open={deleteDialogOpen} onOpenChange={setDeleteDialogOpen}>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogTitle>Delete Job</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
Are you sure you want to delete job {jobToDelete?.job_id}? This action
|
||||
cannot be undone and will also delete all associated files.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel>Cancel</AlertDialogCancel>
|
||||
<AlertDialogAction
|
||||
onClick={handleDelete}
|
||||
disabled={deleting}
|
||||
className="bg-destructive text-destructive-foreground hover:bg-destructive/90"
|
||||
>
|
||||
{deleting && <Loader2 className="w-4 h-4 mr-2 animate-spin" />}
|
||||
Delete
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
"use client"
|
||||
|
||||
import React, { useEffect, useState } from "react"
|
||||
import { useRouter } from "next/navigation"
|
||||
import { getAccessToken } from "@/lib/api"
|
||||
import { useAuth } from "@/lib/auth"
|
||||
import { Loader2 } from "lucide-react"
|
||||
import { SidebarProvider, SidebarInset, SidebarTrigger } from "@/components/ui/sidebar"
|
||||
import { AdminSidebar } from "@/components/admin/admin-sidebar"
|
||||
import { Separator } from "@/components/ui/separator"
|
||||
import {
|
||||
Breadcrumb,
|
||||
BreadcrumbItem,
|
||||
BreadcrumbLink,
|
||||
BreadcrumbList,
|
||||
BreadcrumbPage,
|
||||
BreadcrumbSeparator,
|
||||
} from "@/components/ui/breadcrumb"
|
||||
import { usePathname } from "next/navigation"
|
||||
|
||||
function AdminBreadcrumb() {
|
||||
const pathname = usePathname()
|
||||
|
||||
const getBreadcrumbs = () => {
|
||||
const segments = pathname.split("/").filter(Boolean)
|
||||
const breadcrumbs: { label: string; href?: string }[] = []
|
||||
|
||||
// Always start with Admin
|
||||
if (segments[0] === "admin") {
|
||||
if (segments.length === 1) {
|
||||
breadcrumbs.push({ label: "Dashboard" })
|
||||
} else {
|
||||
breadcrumbs.push({ label: "Admin", href: "/admin" })
|
||||
|
||||
if (segments[1] === "users") {
|
||||
if (segments.length === 2) {
|
||||
breadcrumbs.push({ label: "Users" })
|
||||
} else {
|
||||
breadcrumbs.push({ label: "Users", href: "/admin/users" })
|
||||
breadcrumbs.push({ label: decodeURIComponent(segments[2]) })
|
||||
}
|
||||
} else if (segments[1] === "jobs") {
|
||||
if (segments.length === 2) {
|
||||
breadcrumbs.push({ label: "Jobs" })
|
||||
} else if (segments[2]) {
|
||||
breadcrumbs.push({ label: "Jobs", href: "/admin/jobs" })
|
||||
breadcrumbs.push({ label: decodeURIComponent(segments[2]) })
|
||||
}
|
||||
} else if (segments[1] === "beta") {
|
||||
breadcrumbs.push({ label: "Beta Program" })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return breadcrumbs
|
||||
}
|
||||
|
||||
const breadcrumbs = getBreadcrumbs()
|
||||
|
||||
return (
|
||||
<Breadcrumb>
|
||||
<BreadcrumbList>
|
||||
{breadcrumbs.map((crumb, index) => (
|
||||
<React.Fragment key={`${crumb.href || ''}-${crumb.label}`}>
|
||||
{index > 0 && <BreadcrumbSeparator />}
|
||||
<BreadcrumbItem>
|
||||
{crumb.href ? (
|
||||
<BreadcrumbLink href={crumb.href}>{crumb.label}</BreadcrumbLink>
|
||||
) : (
|
||||
<BreadcrumbPage>{crumb.label}</BreadcrumbPage>
|
||||
)}
|
||||
</BreadcrumbItem>
|
||||
</React.Fragment>
|
||||
))}
|
||||
</BreadcrumbList>
|
||||
</Breadcrumb>
|
||||
)
|
||||
}
|
||||
|
||||
export default function AdminLayout({
|
||||
children,
|
||||
}: {
|
||||
children: React.ReactNode
|
||||
}) {
|
||||
const router = useRouter()
|
||||
const { user } = useAuth()
|
||||
const [isChecking, setIsChecking] = useState(true)
|
||||
|
||||
useEffect(() => {
|
||||
const token = getAccessToken()
|
||||
if (!token) {
|
||||
router.replace("/")
|
||||
return
|
||||
}
|
||||
|
||||
// Check if user is admin (either by role or @nomadkaraoke.com email)
|
||||
if (user) {
|
||||
const isAdmin = user.role === "admin" || user.email?.endsWith("@nomadkaraoke.com")
|
||||
if (!isAdmin) {
|
||||
router.replace("/app")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// If we have a token but no user yet, wait for auth to load
|
||||
if (!user) {
|
||||
return
|
||||
}
|
||||
|
||||
setIsChecking(false)
|
||||
}, [router, user])
|
||||
|
||||
// Show loading while checking auth
|
||||
if (isChecking || !user) {
|
||||
return (
|
||||
<div
|
||||
className="min-h-screen flex items-center justify-center"
|
||||
style={{ background: "var(--bg)" }}
|
||||
>
|
||||
<Loader2 className="w-8 h-8 animate-spin" style={{ color: "var(--text-muted)" }} />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<SidebarProvider>
|
||||
<AdminSidebar />
|
||||
<SidebarInset>
|
||||
<header className="flex h-16 shrink-0 items-center gap-2 border-b px-4">
|
||||
<SidebarTrigger className="-ml-1" />
|
||||
<Separator orientation="vertical" className="mr-2 h-4" />
|
||||
<AdminBreadcrumb />
|
||||
</header>
|
||||
<main className="flex-1 p-4 md:p-6">{children}</main>
|
||||
</SidebarInset>
|
||||
</SidebarProvider>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,223 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import { adminApi, AdminStatsOverview } from "@/lib/api"
|
||||
import { StatsCard, StatsGrid } from "@/components/admin/stats-card"
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import {
|
||||
Users,
|
||||
Briefcase,
|
||||
CreditCard,
|
||||
TestTube2,
|
||||
Clock,
|
||||
CheckCircle,
|
||||
XCircle,
|
||||
AlertCircle,
|
||||
Loader2,
|
||||
RefreshCw,
|
||||
} from "lucide-react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
|
||||
export default function AdminDashboardPage() {
|
||||
const [stats, setStats] = useState<AdminStatsOverview | null>(null)
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const loadStats = async () => {
|
||||
try {
|
||||
setLoading(true)
|
||||
setError(null)
|
||||
const data = await adminApi.getStats()
|
||||
setStats(data)
|
||||
} catch (err: unknown) {
|
||||
const message = err instanceof Error ? err.message : "Failed to load statistics"
|
||||
setError(message)
|
||||
console.error("Failed to load admin stats:", err)
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
loadStats()
|
||||
}, [])
|
||||
|
||||
if (error) {
|
||||
return (
|
||||
<div className="flex flex-col items-center justify-center py-12">
|
||||
<XCircle className="w-12 h-12 text-destructive mb-4" />
|
||||
<h2 className="text-lg font-semibold mb-2">Failed to load statistics</h2>
|
||||
<p className="text-muted-foreground mb-4">{error}</p>
|
||||
<Button onClick={loadStats}>
|
||||
<RefreshCw className="w-4 h-4 mr-2" />
|
||||
Retry
|
||||
</Button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold tracking-tight">Dashboard</h1>
|
||||
<p className="text-muted-foreground">
|
||||
Overview of your karaoke platform
|
||||
</p>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" onClick={loadStats} disabled={loading}>
|
||||
<RefreshCw className={`w-4 h-4 mr-2 ${loading ? "animate-spin" : ""}`} />
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Main Stats Grid */}
|
||||
<StatsGrid>
|
||||
<StatsCard
|
||||
title="Total Users"
|
||||
value={stats?.total_users ?? 0}
|
||||
description={`${stats?.active_users_7d ?? 0} active in last 7 days`}
|
||||
icon={Users}
|
||||
loading={loading}
|
||||
/>
|
||||
<StatsCard
|
||||
title="Total Jobs"
|
||||
value={stats?.total_jobs ?? 0}
|
||||
description={`${stats?.jobs_last_7d ?? 0} in last 7 days`}
|
||||
icon={Briefcase}
|
||||
loading={loading}
|
||||
/>
|
||||
<StatsCard
|
||||
title="Credits Issued (30d)"
|
||||
value={stats?.total_credits_issued_30d ?? 0}
|
||||
description="Credits added to accounts"
|
||||
icon={CreditCard}
|
||||
loading={loading}
|
||||
/>
|
||||
<StatsCard
|
||||
title="Beta Testers"
|
||||
value={stats?.total_beta_testers ?? 0}
|
||||
description="Active beta participants"
|
||||
icon={TestTube2}
|
||||
loading={loading}
|
||||
/>
|
||||
</StatsGrid>
|
||||
|
||||
{/* Secondary Stats */}
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
{/* Job Status Breakdown */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-base">Job Status Breakdown</CardTitle>
|
||||
<CardDescription>Current status of all jobs in the system</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{loading ? (
|
||||
<div className="flex items-center justify-center py-8">
|
||||
<Loader2 className="w-6 h-6 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<Clock className="w-4 h-4 text-yellow-500" />
|
||||
<span className="text-sm">Pending</span>
|
||||
</div>
|
||||
<Badge variant="secondary">{stats?.jobs_by_status?.pending ?? 0}</Badge>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<Loader2 className="w-4 h-4 text-blue-500" />
|
||||
<span className="text-sm">Processing</span>
|
||||
</div>
|
||||
<Badge variant="secondary">{stats?.jobs_by_status?.processing ?? 0}</Badge>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<AlertCircle className="w-4 h-4 text-orange-500" />
|
||||
<span className="text-sm">Awaiting Review</span>
|
||||
</div>
|
||||
<Badge variant="secondary">{stats?.jobs_by_status?.awaiting_review ?? 0}</Badge>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<AlertCircle className="w-4 h-4 text-purple-500" />
|
||||
<span className="text-sm">Awaiting Instrumental</span>
|
||||
</div>
|
||||
<Badge variant="secondary">{stats?.jobs_by_status?.awaiting_instrumental ?? 0}</Badge>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<CheckCircle className="w-4 h-4 text-green-500" />
|
||||
<span className="text-sm">Complete</span>
|
||||
</div>
|
||||
<Badge variant="secondary">{stats?.jobs_by_status?.complete ?? 0}</Badge>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<XCircle className="w-4 h-4 text-red-500" />
|
||||
<span className="text-sm">Failed</span>
|
||||
</div>
|
||||
<Badge variant="destructive">{stats?.jobs_by_status?.failed ?? 0}</Badge>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<XCircle className="w-4 h-4 text-gray-500" />
|
||||
<span className="text-sm">Cancelled</span>
|
||||
</div>
|
||||
<Badge variant="outline">{stats?.jobs_by_status?.cancelled ?? 0}</Badge>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Activity Summary */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-base">Activity Summary</CardTitle>
|
||||
<CardDescription>User and job activity over time</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{loading ? (
|
||||
<div className="flex items-center justify-center py-8">
|
||||
<Loader2 className="w-6 h-6 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-4">
|
||||
<div>
|
||||
<h4 className="text-sm font-medium mb-2">Users</h4>
|
||||
<div className="grid grid-cols-2 gap-4">
|
||||
<div className="rounded-lg border p-3">
|
||||
<p className="text-2xl font-bold">{stats?.active_users_7d ?? 0}</p>
|
||||
<p className="text-xs text-muted-foreground">Active (7d)</p>
|
||||
</div>
|
||||
<div className="rounded-lg border p-3">
|
||||
<p className="text-2xl font-bold">{stats?.active_users_30d ?? 0}</p>
|
||||
<p className="text-xs text-muted-foreground">Active (30d)</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<h4 className="text-sm font-medium mb-2">Jobs</h4>
|
||||
<div className="grid grid-cols-2 gap-4">
|
||||
<div className="rounded-lg border p-3">
|
||||
<p className="text-2xl font-bold">{stats?.jobs_last_7d ?? 0}</p>
|
||||
<p className="text-xs text-muted-foreground">Created (7d)</p>
|
||||
</div>
|
||||
<div className="rounded-lg border p-3">
|
||||
<p className="text-2xl font-bold">{stats?.jobs_last_30d ?? 0}</p>
|
||||
<p className="text-xs text-muted-foreground">Created (30d)</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,485 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import { useSearchParams, useRouter } from "next/navigation"
|
||||
import { adminApi, AdminUserDetail } from "@/lib/api"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow,
|
||||
} from "@/components/ui/table"
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { Label } from "@/components/ui/label"
|
||||
import {
|
||||
ArrowLeft,
|
||||
RefreshCw,
|
||||
Loader2,
|
||||
Plus,
|
||||
Shield,
|
||||
ShieldOff,
|
||||
UserX,
|
||||
UserCheck,
|
||||
CreditCard,
|
||||
Briefcase,
|
||||
Clock,
|
||||
} from "lucide-react"
|
||||
import { useToast } from "@/hooks/use-toast"
|
||||
|
||||
export default function AdminUserDetailPage() {
|
||||
const searchParams = useSearchParams()
|
||||
const router = useRouter()
|
||||
const { toast } = useToast()
|
||||
const email = searchParams.get("email") || ""
|
||||
|
||||
const [user, setUser] = useState<AdminUserDetail | null>(null)
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [actionLoading, setActionLoading] = useState(false)
|
||||
|
||||
// Credit dialog
|
||||
const [creditDialogOpen, setCreditDialogOpen] = useState(false)
|
||||
const [creditAmount, setCreditAmount] = useState("")
|
||||
const [creditReason, setCreditReason] = useState("")
|
||||
|
||||
const loadUser = async () => {
|
||||
if (!email) {
|
||||
setLoading(false)
|
||||
return
|
||||
}
|
||||
try {
|
||||
setLoading(true)
|
||||
const data = await adminApi.getUserDetail(email)
|
||||
setUser(data)
|
||||
} catch (err: any) {
|
||||
console.error("Failed to load user:", err)
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to load user",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
loadUser()
|
||||
}, [email])
|
||||
|
||||
const handleAddCredits = async () => {
|
||||
if (!creditAmount || !creditReason) return
|
||||
|
||||
const amount = parseInt(creditAmount.trim(), 10)
|
||||
if (!Number.isInteger(amount) || amount < 1 || amount > 100000) {
|
||||
toast({
|
||||
title: "Invalid Amount",
|
||||
description: "Please enter a valid amount between 1 and 100,000",
|
||||
variant: "destructive",
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
setActionLoading(true)
|
||||
await adminApi.addCredits(email, amount, creditReason)
|
||||
toast({
|
||||
title: "Credits Added",
|
||||
description: `Added ${creditAmount} credits`,
|
||||
})
|
||||
setCreditDialogOpen(false)
|
||||
setCreditAmount("")
|
||||
setCreditReason("")
|
||||
loadUser()
|
||||
} catch (err: any) {
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to add credits",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setActionLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
const handleToggleRole = async () => {
|
||||
if (!user) return
|
||||
const newRole = user.role === "admin" ? "user" : "admin"
|
||||
|
||||
try {
|
||||
setActionLoading(true)
|
||||
await adminApi.setUserRole(email, newRole)
|
||||
toast({
|
||||
title: "Role Updated",
|
||||
description: `User is now ${newRole}`,
|
||||
})
|
||||
loadUser()
|
||||
} catch (err: any) {
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to update role",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setActionLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
const handleToggleStatus = async () => {
|
||||
if (!user) return
|
||||
|
||||
try {
|
||||
setActionLoading(true)
|
||||
if (user.is_active) {
|
||||
await adminApi.disableUser(email)
|
||||
toast({
|
||||
title: "User Disabled",
|
||||
description: "User has been disabled",
|
||||
})
|
||||
} else {
|
||||
await adminApi.enableUser(email)
|
||||
toast({
|
||||
title: "User Enabled",
|
||||
description: "User has been enabled",
|
||||
})
|
||||
}
|
||||
loadUser()
|
||||
} catch (err: any) {
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to update user status",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setActionLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
const formatDate = (dateStr?: string) => {
|
||||
if (!dateStr) return "Never"
|
||||
return new Date(dateStr).toLocaleString()
|
||||
}
|
||||
|
||||
if (!email) {
|
||||
return (
|
||||
<div className="text-center py-12">
|
||||
<p className="text-muted-foreground">No user email provided</p>
|
||||
<Button variant="outline" className="mt-4" onClick={() => router.push("/admin/users")}>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Users
|
||||
</Button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
if (loading) {
|
||||
return (
|
||||
<div className="flex items-center justify-center py-12">
|
||||
<Loader2 className="w-8 h-8 animate-spin text-muted-foreground" />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
if (!user) {
|
||||
return (
|
||||
<div className="text-center py-12">
|
||||
<p className="text-muted-foreground">User not found</p>
|
||||
<Button variant="outline" className="mt-4" onClick={() => router.push("/admin/users")}>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Users
|
||||
</Button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-4">
|
||||
<Button variant="ghost" size="icon" onClick={() => router.push("/admin/users")}>
|
||||
<ArrowLeft className="w-4 h-4" />
|
||||
</Button>
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold tracking-tight flex items-center gap-2">
|
||||
{user.display_name || user.email}
|
||||
<Badge variant={user.role === "admin" ? "default" : "secondary"}>
|
||||
{user.role}
|
||||
</Badge>
|
||||
{!user.is_active && <Badge variant="destructive">Disabled</Badge>}
|
||||
</h1>
|
||||
<p className="text-muted-foreground">{user.email}</p>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex gap-2">
|
||||
<Button variant="outline" size="sm" onClick={loadUser} disabled={loading}>
|
||||
<RefreshCw className={`w-4 h-4 mr-2 ${loading ? "animate-spin" : ""}`} />
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Actions */}
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button onClick={() => setCreditDialogOpen(true)} disabled={actionLoading}>
|
||||
<Plus className="w-4 h-4 mr-2" />
|
||||
Add Credits
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
onClick={handleToggleRole}
|
||||
disabled={actionLoading}
|
||||
>
|
||||
{user.role === "admin" ? (
|
||||
<>
|
||||
<ShieldOff className="w-4 h-4 mr-2" />
|
||||
Remove Admin
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<Shield className="w-4 h-4 mr-2" />
|
||||
Make Admin
|
||||
</>
|
||||
)}
|
||||
</Button>
|
||||
<Button
|
||||
variant={user.is_active ? "destructive" : "outline"}
|
||||
onClick={handleToggleStatus}
|
||||
disabled={actionLoading}
|
||||
>
|
||||
{user.is_active ? (
|
||||
<>
|
||||
<UserX className="w-4 h-4 mr-2" />
|
||||
Disable User
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<UserCheck className="w-4 h-4 mr-2" />
|
||||
Enable User
|
||||
</>
|
||||
)}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Stats Grid */}
|
||||
<div className="grid gap-4 md:grid-cols-4">
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
<CreditCard className="w-4 h-4" />
|
||||
Credits
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-2xl font-bold">{user.credits}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
<Briefcase className="w-4 h-4" />
|
||||
Jobs Created
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-2xl font-bold">{user.total_jobs_created}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
<Briefcase className="w-4 h-4" />
|
||||
Jobs Completed
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-2xl font-bold">{user.total_jobs_completed}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
<Card>
|
||||
<CardHeader className="pb-2">
|
||||
<CardTitle className="text-sm font-medium flex items-center gap-2">
|
||||
<Clock className="w-4 h-4" />
|
||||
Active Sessions
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-2xl font-bold">{user.active_sessions_count}</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
|
||||
<div className="grid gap-6 md:grid-cols-2">
|
||||
{/* Account Info */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Account Information</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-3">
|
||||
<div className="flex justify-between">
|
||||
<span className="text-muted-foreground">Email Verified</span>
|
||||
<Badge variant={user.email_verified ? "default" : "secondary"}>
|
||||
{user.email_verified ? "Yes" : "No"}
|
||||
</Badge>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span className="text-muted-foreground">Beta Tester</span>
|
||||
<Badge variant={user.is_beta_tester ? "default" : "secondary"}>
|
||||
{user.is_beta_tester ? user.beta_tester_status || "Yes" : "No"}
|
||||
</Badge>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span className="text-muted-foreground">Created</span>
|
||||
<span className="text-sm">{formatDate(user.created_at)}</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span className="text-muted-foreground">Last Login</span>
|
||||
<span className="text-sm">{formatDate(user.last_login_at)}</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span className="text-muted-foreground">Last Updated</span>
|
||||
<span className="text-sm">{formatDate(user.updated_at)}</span>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Credit Transactions */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Recent Credit Transactions</CardTitle>
|
||||
<CardDescription>Last 20 transactions</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{user.credit_transactions.length === 0 ? (
|
||||
<p className="text-muted-foreground text-sm">No transactions</p>
|
||||
) : (
|
||||
<div className="space-y-2 max-h-[300px] overflow-y-auto">
|
||||
{user.credit_transactions.slice().reverse().map((txn, i) => (
|
||||
<div key={txn.id || i} className="flex justify-between items-center text-sm border-b pb-2">
|
||||
<div>
|
||||
<span className={txn.amount > 0 ? "text-green-500" : "text-red-500"}>
|
||||
{txn.amount > 0 ? "+" : ""}{txn.amount}
|
||||
</span>
|
||||
<span className="text-muted-foreground ml-2">{txn.reason}</span>
|
||||
</div>
|
||||
<span className="text-xs text-muted-foreground">
|
||||
{new Date(txn.created_at).toLocaleDateString()}
|
||||
</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
|
||||
{/* Recent Jobs */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Recent Jobs</CardTitle>
|
||||
<CardDescription>Last 10 jobs by this user</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{user.recent_jobs.length === 0 ? (
|
||||
<p className="text-muted-foreground text-sm">No jobs</p>
|
||||
) : (
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>Job ID</TableHead>
|
||||
<TableHead>Artist / Title</TableHead>
|
||||
<TableHead>Status</TableHead>
|
||||
<TableHead>Created</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{user.recent_jobs.map((job) => (
|
||||
<TableRow
|
||||
key={job.job_id}
|
||||
className="cursor-pointer hover:bg-muted/50"
|
||||
onClick={() => router.push(`/admin/jobs/detail?id=${job.job_id}`)}
|
||||
>
|
||||
<TableCell className="font-mono text-sm">{job.job_id}</TableCell>
|
||||
<TableCell>
|
||||
{job.artist && job.title ? `${job.artist} - ${job.title}` : "—"}
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Badge variant={
|
||||
job.status === "complete" ? "default" :
|
||||
job.status === "failed" ? "destructive" :
|
||||
"secondary"
|
||||
}>
|
||||
{job.status}
|
||||
</Badge>
|
||||
</TableCell>
|
||||
<TableCell className="text-sm">
|
||||
{job.created_at ? new Date(job.created_at).toLocaleString() : "—"}
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
))}
|
||||
</TableBody>
|
||||
</Table>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Add Credits Dialog */}
|
||||
<Dialog open={creditDialogOpen} onOpenChange={setCreditDialogOpen}>
|
||||
<DialogContent>
|
||||
<DialogHeader>
|
||||
<DialogTitle>Add Credits</DialogTitle>
|
||||
<DialogDescription>
|
||||
Add credits to {user.email}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<div className="space-y-4 py-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="amount">Amount</Label>
|
||||
<Input
|
||||
id="amount"
|
||||
type="number"
|
||||
min="1"
|
||||
placeholder="Enter amount"
|
||||
value={creditAmount}
|
||||
onChange={(e) => setCreditAmount(e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="reason">Reason</Label>
|
||||
<Input
|
||||
id="reason"
|
||||
placeholder="e.g., Beta reward, Support credit"
|
||||
value={creditReason}
|
||||
onChange={(e) => setCreditReason(e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<DialogFooter>
|
||||
<Button variant="outline" onClick={() => setCreditDialogOpen(false)}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
onClick={handleAddCredits}
|
||||
disabled={!creditAmount || !creditReason || actionLoading}
|
||||
>
|
||||
{actionLoading && <Loader2 className="w-4 h-4 mr-2 animate-spin" />}
|
||||
Add Credits
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,375 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState, useCallback } from "react"
|
||||
import { useRouter } from "next/navigation"
|
||||
import { adminApi, AdminUser, AdminUserListResponse } from "@/lib/api"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import {
|
||||
Table,
|
||||
TableBody,
|
||||
TableCell,
|
||||
TableHead,
|
||||
TableHeader,
|
||||
TableRow,
|
||||
} from "@/components/ui/table"
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select"
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
DialogTrigger,
|
||||
} from "@/components/ui/dialog"
|
||||
import { Label } from "@/components/ui/label"
|
||||
import {
|
||||
Search,
|
||||
RefreshCw,
|
||||
ChevronLeft,
|
||||
ChevronRight,
|
||||
Plus,
|
||||
Loader2,
|
||||
CreditCard,
|
||||
Shield,
|
||||
ShieldOff,
|
||||
UserX,
|
||||
UserCheck,
|
||||
} from "lucide-react"
|
||||
import { useToast } from "@/hooks/use-toast"
|
||||
|
||||
export default function AdminUsersPage() {
|
||||
const router = useRouter()
|
||||
const { toast } = useToast()
|
||||
const [users, setUsers] = useState<AdminUser[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [total, setTotal] = useState(0)
|
||||
const [offset, setOffset] = useState(0)
|
||||
const [limit] = useState(20)
|
||||
const [hasMore, setHasMore] = useState(false)
|
||||
const [search, setSearch] = useState("")
|
||||
const [searchInput, setSearchInput] = useState("")
|
||||
const [sortBy, setSortBy] = useState<"created_at" | "last_login_at" | "credits" | "email">("created_at")
|
||||
const [sortOrder, setSortOrder] = useState<"asc" | "desc">("desc")
|
||||
|
||||
// Credit dialog state
|
||||
const [creditDialogOpen, setCreditDialogOpen] = useState(false)
|
||||
const [selectedUser, setSelectedUser] = useState<AdminUser | null>(null)
|
||||
const [creditAmount, setCreditAmount] = useState("")
|
||||
const [creditReason, setCreditReason] = useState("")
|
||||
const [addingCredits, setAddingCredits] = useState(false)
|
||||
|
||||
const loadUsers = useCallback(async () => {
|
||||
try {
|
||||
setLoading(true)
|
||||
const data = await adminApi.listUsers({
|
||||
limit,
|
||||
offset,
|
||||
search: search || undefined,
|
||||
sort_by: sortBy,
|
||||
sort_order: sortOrder,
|
||||
})
|
||||
setUsers(data.users)
|
||||
setTotal(data.total)
|
||||
setHasMore(data.has_more)
|
||||
} catch (err: any) {
|
||||
console.error("Failed to load users:", err)
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to load users",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}, [limit, offset, search, sortBy, sortOrder, toast])
|
||||
|
||||
useEffect(() => {
|
||||
loadUsers()
|
||||
}, [loadUsers])
|
||||
|
||||
const handleSearch = () => {
|
||||
setOffset(0)
|
||||
setSearch(searchInput)
|
||||
}
|
||||
|
||||
const handleKeyPress = (e: React.KeyboardEvent) => {
|
||||
if (e.key === "Enter") {
|
||||
handleSearch()
|
||||
}
|
||||
}
|
||||
|
||||
const handleAddCredits = async () => {
|
||||
if (!selectedUser || !creditAmount || !creditReason) return
|
||||
|
||||
const amount = parseInt(creditAmount.trim(), 10)
|
||||
if (!Number.isInteger(amount) || amount < 1 || amount > 100000) {
|
||||
toast({
|
||||
title: "Invalid Amount",
|
||||
description: "Please enter a valid amount between 1 and 100,000",
|
||||
variant: "destructive",
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
setAddingCredits(true)
|
||||
await adminApi.addCredits(selectedUser.email, amount, creditReason)
|
||||
toast({
|
||||
title: "Credits Added",
|
||||
description: `Added ${creditAmount} credits to ${selectedUser.email}`,
|
||||
})
|
||||
setCreditDialogOpen(false)
|
||||
setCreditAmount("")
|
||||
setCreditReason("")
|
||||
setSelectedUser(null)
|
||||
loadUsers()
|
||||
} catch (err: any) {
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to add credits",
|
||||
variant: "destructive",
|
||||
})
|
||||
} finally {
|
||||
setAddingCredits(false)
|
||||
}
|
||||
}
|
||||
|
||||
const handleToggleRole = async (user: AdminUser) => {
|
||||
const newRole = user.role === "admin" ? "user" : "admin"
|
||||
try {
|
||||
await adminApi.setUserRole(user.email, newRole)
|
||||
toast({
|
||||
title: "Role Updated",
|
||||
description: `${user.email} is now ${newRole}`,
|
||||
})
|
||||
loadUsers()
|
||||
} catch (err: any) {
|
||||
toast({
|
||||
title: "Error",
|
||||
description: err.message || "Failed to update role",
|
||||
variant: "destructive",
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold tracking-tight">Users</h1>
|
||||
<p className="text-muted-foreground">
|
||||
Manage user accounts and credits
|
||||
</p>
|
||||
</div>
|
||||
<Button variant="outline" size="sm" onClick={loadUsers} disabled={loading}>
|
||||
<RefreshCw className={`w-4 h-4 mr-2 ${loading ? "animate-spin" : ""}`} />
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Filters */}
|
||||
<div className="flex flex-col sm:flex-row gap-4">
|
||||
<div className="flex-1 flex gap-2">
|
||||
<Input
|
||||
placeholder="Search by email..."
|
||||
value={searchInput}
|
||||
onChange={(e) => setSearchInput(e.target.value)}
|
||||
onKeyPress={handleKeyPress}
|
||||
className="max-w-sm"
|
||||
/>
|
||||
<Button variant="secondary" onClick={handleSearch}>
|
||||
<Search className="w-4 h-4" />
|
||||
</Button>
|
||||
</div>
|
||||
<div className="flex gap-2">
|
||||
<Select value={sortBy} onValueChange={(v) => setSortBy(v as any)}>
|
||||
<SelectTrigger className="w-[140px]">
|
||||
<SelectValue placeholder="Sort by" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="created_at">Created</SelectItem>
|
||||
<SelectItem value="last_login_at">Last Login</SelectItem>
|
||||
<SelectItem value="credits">Credits</SelectItem>
|
||||
<SelectItem value="email">Email</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
<Select value={sortOrder} onValueChange={(v) => setSortOrder(v as any)}>
|
||||
<SelectTrigger className="w-[100px]">
|
||||
<SelectValue placeholder="Order" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="desc">Desc</SelectItem>
|
||||
<SelectItem value="asc">Asc</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Table */}
|
||||
<div className="rounded-md border">
|
||||
<Table>
|
||||
<TableHeader>
|
||||
<TableRow>
|
||||
<TableHead>Email</TableHead>
|
||||
<TableHead>Role</TableHead>
|
||||
<TableHead className="text-right">Credits</TableHead>
|
||||
<TableHead className="text-right">Jobs</TableHead>
|
||||
<TableHead className="text-right">Actions</TableHead>
|
||||
</TableRow>
|
||||
</TableHeader>
|
||||
<TableBody>
|
||||
{loading ? (
|
||||
<TableRow>
|
||||
<TableCell colSpan={5} className="text-center py-8">
|
||||
<Loader2 className="w-6 h-6 animate-spin mx-auto text-muted-foreground" />
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
) : users.length === 0 ? (
|
||||
<TableRow>
|
||||
<TableCell colSpan={5} className="text-center py-8 text-muted-foreground">
|
||||
No users found
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
) : (
|
||||
users.map((user) => (
|
||||
<TableRow
|
||||
key={user.email}
|
||||
className="cursor-pointer hover:bg-muted/50"
|
||||
onClick={() => router.push(`/admin/users/detail?email=${encodeURIComponent(user.email)}`)}
|
||||
>
|
||||
<TableCell className="font-medium">
|
||||
{user.display_name || user.email}
|
||||
{user.display_name && (
|
||||
<span className="text-muted-foreground text-sm block">
|
||||
{user.email}
|
||||
</span>
|
||||
)}
|
||||
</TableCell>
|
||||
<TableCell>
|
||||
<Badge variant={user.role === "admin" ? "default" : "secondary"}>
|
||||
{user.role}
|
||||
</Badge>
|
||||
</TableCell>
|
||||
<TableCell className="text-right">{user.credits}</TableCell>
|
||||
<TableCell className="text-right">
|
||||
{user.total_jobs_created ?? 0} / {user.total_jobs_completed ?? 0}
|
||||
</TableCell>
|
||||
<TableCell className="text-right">
|
||||
<div className="flex justify-end gap-1" onClick={(e) => e.stopPropagation()}>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={() => {
|
||||
setSelectedUser(user)
|
||||
setCreditDialogOpen(true)
|
||||
}}
|
||||
title="Add credits"
|
||||
>
|
||||
<Plus className="w-4 h-4" />
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={() => handleToggleRole(user)}
|
||||
title={user.role === "admin" ? "Remove admin" : "Make admin"}
|
||||
>
|
||||
{user.role === "admin" ? (
|
||||
<ShieldOff className="w-4 h-4" />
|
||||
) : (
|
||||
<Shield className="w-4 h-4" />
|
||||
)}
|
||||
</Button>
|
||||
</div>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
))
|
||||
)}
|
||||
</TableBody>
|
||||
</Table>
|
||||
</div>
|
||||
|
||||
{/* Pagination */}
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="text-sm text-muted-foreground">
|
||||
{total === 0 ? "No users found" : `Showing ${offset + 1}-${Math.min(offset + users.length, total)} of ${total} users`}
|
||||
</p>
|
||||
<div className="flex gap-2">
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => setOffset(Math.max(0, offset - limit))}
|
||||
disabled={offset === 0}
|
||||
>
|
||||
<ChevronLeft className="w-4 h-4" />
|
||||
Previous
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => setOffset(offset + limit)}
|
||||
disabled={!hasMore}
|
||||
>
|
||||
Next
|
||||
<ChevronRight className="w-4 h-4" />
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Add Credits Dialog */}
|
||||
<Dialog open={creditDialogOpen} onOpenChange={setCreditDialogOpen}>
|
||||
<DialogContent>
|
||||
<DialogHeader>
|
||||
<DialogTitle>Add Credits</DialogTitle>
|
||||
<DialogDescription>
|
||||
Add credits to {selectedUser?.email}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<div className="space-y-4 py-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="amount">Amount</Label>
|
||||
<Input
|
||||
id="amount"
|
||||
type="number"
|
||||
min="1"
|
||||
placeholder="Enter amount"
|
||||
value={creditAmount}
|
||||
onChange={(e) => setCreditAmount(e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="reason">Reason</Label>
|
||||
<Input
|
||||
id="reason"
|
||||
placeholder="e.g., Beta reward, Support credit"
|
||||
value={creditReason}
|
||||
onChange={(e) => setCreditReason(e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<DialogFooter>
|
||||
<Button variant="outline" onClick={() => setCreditDialogOpen(false)}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
onClick={handleAddCredits}
|
||||
disabled={!creditAmount || !creditReason || addingCredits}
|
||||
>
|
||||
{addingCredits && <Loader2 className="w-4 h-4 mr-2 animate-spin" />}
|
||||
Add Credits
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
"use client"
|
||||
|
||||
import { useState, useEffect, useCallback } from "react"
|
||||
import { useRouter } from "next/navigation"
|
||||
import { api, Job, getAccessToken } from "@/lib/api"
|
||||
import { useJobNotifications, useVisibilityRefresh } from "@/hooks/use-notifications"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import { Music2, RefreshCw, Loader2, Moon, Sun } from "lucide-react"
|
||||
import { WarmingUpLoader } from "@/components/WarmingUpLoader"
|
||||
import { JobCard } from "@/components/job"
|
||||
import { JobSubmission } from "@/components/job/JobSubmission"
|
||||
import { AuthStatus } from "@/components/auth"
|
||||
import { AutoProcessor } from "@/components/AutoProcessor"
|
||||
import { VersionFooter } from "@/components/version-footer"
|
||||
import { useTheme } from "@/lib/theme"
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
TooltipProvider,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip"
|
||||
|
||||
export default function AppPage() {
|
||||
const router = useRouter()
|
||||
const [jobs, setJobs] = useState<Job[]>([])
|
||||
const [isLoadingJobs, setIsLoadingJobs] = useState(true)
|
||||
const [isInitialLoad, setIsInitialLoad] = useState(true)
|
||||
const [isAuthenticated, setIsAuthenticated] = useState<boolean | null>(null)
|
||||
const { isDarkMode, toggleTheme, mounted } = useTheme()
|
||||
|
||||
// Memoize loadJobs for use with visibility refresh
|
||||
const loadJobs = useCallback(async () => {
|
||||
if (!getAccessToken()) {
|
||||
setIsLoadingJobs(false)
|
||||
setIsInitialLoad(false)
|
||||
return
|
||||
}
|
||||
try {
|
||||
const data = await api.listJobs({ limit: 20 })
|
||||
setJobs(data)
|
||||
} catch (err: any) {
|
||||
// Don't log auth errors - user just needs to authenticate
|
||||
if (err?.status !== 401) {
|
||||
console.error("Failed to load jobs:", err)
|
||||
}
|
||||
} finally {
|
||||
setIsLoadingJobs(false)
|
||||
setIsInitialLoad(false)
|
||||
}
|
||||
}, [])
|
||||
|
||||
// Enable notifications for job status changes (sound + title animation)
|
||||
useJobNotifications(jobs)
|
||||
|
||||
// Refresh jobs immediately when tab becomes visible (after returning from review UIs)
|
||||
useVisibilityRefresh(loadJobs, isAuthenticated === true)
|
||||
|
||||
// Check auth status on mount and redirect if not authenticated
|
||||
useEffect(() => {
|
||||
const token = getAccessToken()
|
||||
if (!token) {
|
||||
// Redirect to landing page if not authenticated
|
||||
router.replace('/')
|
||||
return
|
||||
}
|
||||
setIsAuthenticated(true)
|
||||
}, [router])
|
||||
|
||||
// Load jobs on mount (only if authenticated)
|
||||
useEffect(() => {
|
||||
if (isAuthenticated !== true) {
|
||||
setIsLoadingJobs(false)
|
||||
return
|
||||
}
|
||||
loadJobs()
|
||||
// Poll for updates every 10 seconds
|
||||
const interval = setInterval(loadJobs, 10000)
|
||||
return () => clearInterval(interval)
|
||||
}, [isAuthenticated, loadJobs])
|
||||
|
||||
// Show nothing while checking auth (prevents flash)
|
||||
if (isAuthenticated === null) {
|
||||
return (
|
||||
<div className="min-h-screen flex items-center justify-center" style={{ background: 'var(--bg)' }}>
|
||||
<Loader2 className="w-8 h-8 animate-spin" style={{ color: 'var(--text-muted)' }} />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// Show warming up loader during initial job load (cold start scenario)
|
||||
if (isInitialLoad && isLoadingJobs) {
|
||||
return (
|
||||
<div className="min-h-screen flex items-center justify-center" style={{ background: 'var(--bg)' }}>
|
||||
<WarmingUpLoader spinnerClassName="w-10 h-10" />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="min-h-screen" style={{ background: 'var(--bg)' }}>
|
||||
{/* AutoProcessor - handles non-interactive mode for jobs with that flag */}
|
||||
<AutoProcessor jobs={jobs} onJobsChanged={loadJobs} />
|
||||
|
||||
{/* Header */}
|
||||
<header className="border-b backdrop-blur-sm sticky top-0 z-10" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
|
||||
<div className="px-3 sm:px-4 py-2 sm:py-3 flex items-center justify-between gap-2">
|
||||
<div className="flex items-center gap-2 sm:gap-3 min-w-0">
|
||||
<img src="/nomad-karaoke-logo.svg" alt="Nomad Karaoke" className="h-8 sm:h-10 shrink-0" />
|
||||
<div className="min-w-0">
|
||||
<h1 className="text-base sm:text-xl font-bold truncate" style={{ color: 'var(--text)' }}>Karaoke Generator</h1>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center gap-1 sm:gap-2 shrink-0">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={loadJobs}
|
||||
disabled={isLoadingJobs || !isAuthenticated}
|
||||
className="min-h-[40px] px-2 sm:px-3"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
>
|
||||
<RefreshCw className={`w-4 h-4 sm:mr-2 ${isLoadingJobs ? 'animate-spin' : ''}`} />
|
||||
<span className="hidden sm:inline">Refresh</span>
|
||||
</Button>
|
||||
<AuthStatus />
|
||||
{mounted && (
|
||||
<TooltipProvider>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={toggleTheme}
|
||||
className="min-h-[40px] px-2 sm:px-3"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
>
|
||||
{isDarkMode ? (
|
||||
<Sun className="w-4 h-4" />
|
||||
) : (
|
||||
<Moon className="w-4 h-4" />
|
||||
)}
|
||||
</Button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom">
|
||||
<p>{isDarkMode ? "Switch to light mode" : "Switch to dark mode"}</p>
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</TooltipProvider>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main className="px-4 py-8 space-y-6">
|
||||
<div className="grid lg:grid-cols-2 gap-8">
|
||||
{/* Submit Job Card */}
|
||||
<Card className="backdrop-blur" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
|
||||
<CardHeader>
|
||||
<CardTitle style={{ color: 'var(--text)' }}>Create Karaoke Video</CardTitle>
|
||||
<CardDescription style={{ color: 'var(--text-muted)' }}>
|
||||
Upload an audio file, provide a YouTube URL, or search for audio by artist & title
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<JobSubmission onJobCreated={loadJobs} />
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Jobs List Card */}
|
||||
<Card className="backdrop-blur" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center justify-between" style={{ color: 'var(--text)' }}>
|
||||
Recent Jobs
|
||||
{isLoadingJobs && <Loader2 className="w-4 h-4 animate-spin" style={{ color: 'var(--text-muted)' }} />}
|
||||
</CardTitle>
|
||||
<CardDescription style={{ color: 'var(--text-muted)' }}>
|
||||
{jobs.length} job{jobs.length !== 1 ? 's' : ''} found
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{jobs.length === 0 ? (
|
||||
<div className="text-center py-8" style={{ color: 'var(--text-muted)' }}>
|
||||
<Music2 className="w-12 h-12 mx-auto mb-3 opacity-50" />
|
||||
<p>No jobs yet. Create one to get started!</p>
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-3 max-h-[500px] overflow-y-auto pr-2">
|
||||
{jobs.map((job) => (
|
||||
<JobCard key={job.job_id} job={job} onRefresh={loadJobs} />
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
</main>
|
||||
|
||||
<VersionFooter />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
"use client"
|
||||
|
||||
import { Suspense, useEffect, useState, useRef } from "react"
|
||||
import { useRouter, useSearchParams } from "next/navigation"
|
||||
import { Loader2, CheckCircle, XCircle } from "lucide-react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { useAuth } from "@/lib/auth"
|
||||
|
||||
type VerifyState = "loading" | "success" | "error"
|
||||
|
||||
function VerifyMagicLinkContent() {
|
||||
const router = useRouter()
|
||||
const searchParams = useSearchParams()
|
||||
const [state, setState] = useState<VerifyState>("loading")
|
||||
const [errorMessage, setErrorMessage] = useState("")
|
||||
const hasVerified = useRef(false)
|
||||
|
||||
const { verifyMagicLink, user, error } = useAuth()
|
||||
|
||||
useEffect(() => {
|
||||
// Prevent duplicate verification attempts
|
||||
if (hasVerified.current) return
|
||||
hasVerified.current = true
|
||||
|
||||
const token = searchParams.get("token")
|
||||
|
||||
if (!token) {
|
||||
setState("error")
|
||||
setErrorMessage("No verification token provided")
|
||||
return
|
||||
}
|
||||
|
||||
// Verify the token
|
||||
const verify = async () => {
|
||||
const success = await verifyMagicLink(token)
|
||||
if (success) {
|
||||
setState("success")
|
||||
// Redirect to main app after short delay
|
||||
setTimeout(() => {
|
||||
router.push("/app")
|
||||
}, 2000)
|
||||
} else {
|
||||
setState("error")
|
||||
// Get error from auth store at this point
|
||||
const authStore = useAuth.getState()
|
||||
setErrorMessage(authStore.error || "Invalid or expired link")
|
||||
}
|
||||
}
|
||||
|
||||
verify()
|
||||
}, [searchParams, verifyMagicLink, router])
|
||||
|
||||
return (
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
{state === "loading" && (
|
||||
<>
|
||||
<Loader2 className="w-12 h-12 text-blue-400 animate-spin mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
Verifying your sign-in link...
|
||||
</h1>
|
||||
<p className="text-slate-400">
|
||||
Please wait while we sign you in.
|
||||
</p>
|
||||
</>
|
||||
)}
|
||||
|
||||
{state === "success" && (
|
||||
<>
|
||||
<CheckCircle className="w-12 h-12 text-green-400 mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
Successfully signed in!
|
||||
</h1>
|
||||
<p className="text-slate-400 mb-4">
|
||||
Welcome back{user?.email ? `, ${user.email}` : ""}!
|
||||
</p>
|
||||
<p className="text-sm text-slate-500">
|
||||
Redirecting you to the app...
|
||||
</p>
|
||||
</>
|
||||
)}
|
||||
|
||||
{state === "error" && (
|
||||
<>
|
||||
<XCircle className="w-12 h-12 text-red-400 mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
Sign-in failed
|
||||
</h1>
|
||||
<p className="text-slate-400 mb-6">
|
||||
{errorMessage || "The link may have expired or already been used."}
|
||||
</p>
|
||||
<div className="space-y-2">
|
||||
<Button
|
||||
onClick={() => router.push("/app")}
|
||||
className="w-full bg-blue-600 hover:bg-blue-500"
|
||||
>
|
||||
Go to App
|
||||
</Button>
|
||||
<p className="text-xs text-slate-500">
|
||||
You can request a new sign-in link from the app.
|
||||
</p>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function VerifyLoadingFallback() {
|
||||
return (
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
<Loader2 className="w-12 h-12 text-blue-400 animate-spin mx-auto mb-4" />
|
||||
<h1 className="text-xl font-semibold text-white mb-2">
|
||||
Loading...
|
||||
</h1>
|
||||
<p className="text-slate-400">
|
||||
Please wait.
|
||||
</p>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export default function VerifyMagicLinkPage() {
|
||||
return (
|
||||
<div className="min-h-screen bg-slate-950 flex items-center justify-center p-4">
|
||||
<Suspense fallback={<VerifyLoadingFallback />}>
|
||||
<VerifyMagicLinkContent />
|
||||
</Suspense>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -96,3 +96,66 @@
|
||||
@apply bg-background text-foreground;
|
||||
}
|
||||
}
|
||||
|
||||
/* Landing page utilities */
|
||||
@layer utilities {
|
||||
/* Glow effect for primary buttons */
|
||||
.btn-glow {
|
||||
box-shadow: 0 0 20px rgba(59, 130, 246, 0.5);
|
||||
}
|
||||
.btn-glow:hover {
|
||||
box-shadow: 0 0 30px rgba(59, 130, 246, 0.7);
|
||||
}
|
||||
|
||||
/* Gradient text */
|
||||
.gradient-text {
|
||||
background: linear-gradient(135deg, #3b82f6 0%, #8b5cf6 50%, #ec4899 100%);
|
||||
-webkit-background-clip: text;
|
||||
-webkit-text-fill-color: transparent;
|
||||
background-clip: text;
|
||||
}
|
||||
|
||||
/* Card hover effect */
|
||||
.card-hover {
|
||||
transition: transform 0.2s ease, box-shadow 0.2s ease;
|
||||
}
|
||||
.card-hover:hover {
|
||||
transform: translateY(-4px);
|
||||
box-shadow: 0 20px 40px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
/* Animated background gradient */
|
||||
@keyframes gradient-shift {
|
||||
0% { background-position: 0% 50%; }
|
||||
50% { background-position: 100% 50%; }
|
||||
100% { background-position: 0% 50%; }
|
||||
}
|
||||
.animated-gradient {
|
||||
background: linear-gradient(-45deg, #0f172a, #1e293b, #1e3a8a, #312e81);
|
||||
background-size: 400% 400%;
|
||||
animation: gradient-shift 15s ease infinite;
|
||||
}
|
||||
|
||||
/* Dark theme color utilities for landing page */
|
||||
.text-dark-300 { color: #d1d5db; }
|
||||
.text-dark-400 { color: #9ca3af; }
|
||||
.text-dark-500 { color: #6b7280; }
|
||||
.bg-dark-700 { background-color: #374151; }
|
||||
.bg-dark-800 { background-color: #1f2937; }
|
||||
.bg-dark-900 { background-color: #111827; }
|
||||
.border-dark-600 { border-color: #4b5563; }
|
||||
.border-dark-700 { border-color: #374151; }
|
||||
.text-primary-400 { color: #60a5fa; }
|
||||
.text-primary-500 { color: #3b82f6; }
|
||||
.bg-primary-500 { background-color: #3b82f6; }
|
||||
.hover\:bg-primary-600:hover { background-color: #2563eb; }
|
||||
|
||||
/* Fade in animation for cold start messages */
|
||||
@keyframes fade-in {
|
||||
from { opacity: 0; transform: translateY(-4px); }
|
||||
to { opacity: 1; transform: translateY(0); }
|
||||
}
|
||||
.animate-fade-in {
|
||||
animation: fade-in 0.3s ease-out forwards;
|
||||
}
|
||||
}
|
||||
+530
-218
@@ -1,245 +1,557 @@
|
||||
"use client"
|
||||
'use client';
|
||||
|
||||
import { useState, useEffect } from "react"
|
||||
import { api, Job, getAccessToken } from "@/lib/api"
|
||||
import { useAutoMode, getAutoModeFromUrl } from "@/lib/auto-mode"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import { Music2, RefreshCw, Loader2, Zap, ZapOff, KeyRound, Moon, Sun } from "lucide-react"
|
||||
import { JobCard } from "@/components/job"
|
||||
import { JobSubmission } from "@/components/job/JobSubmission"
|
||||
import { AuthStatus } from "@/components/auth"
|
||||
import { AutoProcessor } from "@/components/AutoProcessor"
|
||||
import { useTheme } from "@/lib/theme"
|
||||
import { useState, useEffect, useRef } from 'react';
|
||||
import { useRouter } from 'next/navigation';
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
TooltipProvider,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip"
|
||||
MicVocal,
|
||||
Music,
|
||||
Sparkles,
|
||||
Video,
|
||||
Youtube,
|
||||
ChevronDown,
|
||||
Check,
|
||||
Gift,
|
||||
MessageSquare,
|
||||
Mail,
|
||||
Loader2,
|
||||
} from 'lucide-react';
|
||||
import { api, setAccessToken, getAccessToken, CreditPackage, BetaEnrollResponse } from '@/lib/api';
|
||||
import { AuthDialog } from '@/components/auth/AuthDialog';
|
||||
import { WarmingUpLoader } from '@/components/WarmingUpLoader';
|
||||
|
||||
export default function HomePage() {
|
||||
const [jobs, setJobs] = useState<Job[]>([])
|
||||
const [isLoadingJobs, setIsLoadingJobs] = useState(true)
|
||||
const [isAuthenticated, setIsAuthenticated] = useState(false)
|
||||
const { enabled: autoModeEnabled, setEnabled: setAutoMode, toggle: toggleAutoMode } = useAutoMode()
|
||||
const { isDarkMode, toggleTheme, mounted } = useTheme()
|
||||
// FAQ data
|
||||
const faqs = [
|
||||
{
|
||||
question: 'How does it work?',
|
||||
answer:
|
||||
'Enter the artist and song title, and our AI finds the audio, removes the vocals, transcribes and syncs the lyrics, then generates a professional karaoke video. You can review and edit the lyrics before final export.',
|
||||
},
|
||||
{
|
||||
question: 'What songs can I use?',
|
||||
answer:
|
||||
'Any song! Our system searches multiple audio sources to find high-quality versions. If a song exists online, we can likely create a karaoke video for it.',
|
||||
},
|
||||
{
|
||||
question: 'What format is the output?',
|
||||
answer:
|
||||
'You get a 4K MP4 video file with perfectly synced lyrics. We also offer direct YouTube upload and cloud storage integration.',
|
||||
},
|
||||
{
|
||||
question: 'What if something goes wrong?',
|
||||
answer:
|
||||
"If the AI makes mistakes, you can edit the lyrics and timing before generating the final video. If there's a technical issue, contact us and we'll make it right.",
|
||||
},
|
||||
{
|
||||
question: 'Do credits expire?',
|
||||
answer:
|
||||
'No! Your credits never expire. Use them whenever you want.',
|
||||
},
|
||||
];
|
||||
|
||||
// Check auth status on mount
|
||||
export default function LandingPage() {
|
||||
const router = useRouter();
|
||||
const [isCheckingAuth, setIsCheckingAuth] = useState(true);
|
||||
const [packages, setPackages] = useState<CreditPackage[]>([]);
|
||||
const [packagesLoading, setPackagesLoading] = useState(true);
|
||||
const [packagesError, setPackagesError] = useState<string | null>(null);
|
||||
const [selectedPackage, setSelectedPackage] = useState<CreditPackage | null>(null);
|
||||
const [email, setEmail] = useState('');
|
||||
const [isLoading, setIsLoading] = useState(false);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
// Beta form state
|
||||
const [showBetaForm, setShowBetaForm] = useState(false);
|
||||
const [betaEmail, setBetaEmail] = useState('');
|
||||
const [betaPromise, setBetaPromise] = useState('');
|
||||
const [betaAccept, setBetaAccept] = useState(false);
|
||||
const [betaLoading, setBetaLoading] = useState(false);
|
||||
const [betaError, setBetaError] = useState<string | null>(null);
|
||||
const [betaSuccess, setBetaSuccess] = useState(false);
|
||||
const [betaWillRedirect, setBetaWillRedirect] = useState(false);
|
||||
|
||||
// FAQ expansion state
|
||||
const [expandedFaq, setExpandedFaq] = useState<number | null>(null);
|
||||
|
||||
// Auth dialog state
|
||||
const [showAuthDialog, setShowAuthDialog] = useState(false);
|
||||
|
||||
// Ref for redirect timeout cleanup
|
||||
const redirectTimeoutRef = useRef<NodeJS.Timeout | null>(null);
|
||||
|
||||
// Cleanup timeout on unmount
|
||||
useEffect(() => {
|
||||
setIsAuthenticated(!!getAccessToken())
|
||||
}, [])
|
||||
|
||||
// Initialize auto-mode from URL parameter on mount
|
||||
useEffect(() => {
|
||||
const urlAutoMode = getAutoModeFromUrl()
|
||||
if (urlAutoMode) {
|
||||
setAutoMode(true)
|
||||
}
|
||||
}, [setAutoMode])
|
||||
|
||||
// Load jobs on mount (only if authenticated)
|
||||
useEffect(() => {
|
||||
if (!isAuthenticated) {
|
||||
setIsLoadingJobs(false)
|
||||
return
|
||||
}
|
||||
loadJobs()
|
||||
// Poll for updates every 10 seconds
|
||||
const interval = setInterval(loadJobs, 10000)
|
||||
return () => clearInterval(interval)
|
||||
}, [isAuthenticated])
|
||||
|
||||
async function loadJobs() {
|
||||
if (!getAccessToken()) {
|
||||
setIsLoadingJobs(false)
|
||||
return
|
||||
}
|
||||
try {
|
||||
const data = await api.listJobs({ limit: 20 })
|
||||
setJobs(data)
|
||||
} catch (err: any) {
|
||||
// Don't log auth errors - user just needs to authenticate
|
||||
if (err?.status !== 401) {
|
||||
console.error("Failed to load jobs:", err)
|
||||
return () => {
|
||||
if (redirectTimeoutRef.current) {
|
||||
clearTimeout(redirectTimeoutRef.current);
|
||||
}
|
||||
} finally {
|
||||
setIsLoadingJobs(false)
|
||||
};
|
||||
}, []);
|
||||
|
||||
// Check if already logged in and redirect to app
|
||||
useEffect(() => {
|
||||
const token = getAccessToken();
|
||||
if (token) {
|
||||
router.replace('/app');
|
||||
return;
|
||||
}
|
||||
setIsCheckingAuth(false);
|
||||
}, [router]);
|
||||
|
||||
// Handle successful auth from dialog
|
||||
const handleAuthSuccess = () => {
|
||||
setShowAuthDialog(false);
|
||||
router.push('/app');
|
||||
};
|
||||
|
||||
// Load credit packages on mount
|
||||
useEffect(() => {
|
||||
if (isCheckingAuth) return;
|
||||
|
||||
async function loadPackages() {
|
||||
try {
|
||||
const pkgs = await api.getCreditPackages();
|
||||
setPackages(pkgs);
|
||||
// Default to 5 credits (best value)
|
||||
const bestValue = pkgs.find((p) => p.credits === 5);
|
||||
setSelectedPackage(bestValue || pkgs[0]);
|
||||
} catch (err) {
|
||||
console.error('Failed to load packages:', err);
|
||||
setPackagesError('Failed to load pricing. Please refresh the page.');
|
||||
} finally {
|
||||
setPackagesLoading(false);
|
||||
}
|
||||
}
|
||||
loadPackages();
|
||||
}, [isCheckingAuth]);
|
||||
|
||||
// Handle checkout
|
||||
const handleCheckout = async (e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
if (!selectedPackage || !email) return;
|
||||
|
||||
setIsLoading(true);
|
||||
setError(null);
|
||||
|
||||
try {
|
||||
const checkoutUrl = await api.createCheckout(selectedPackage.id, email);
|
||||
// Validate that the URL is a Stripe checkout URL
|
||||
if (!checkoutUrl.startsWith('https://checkout.stripe.com/')) {
|
||||
throw new Error('Invalid checkout URL received');
|
||||
}
|
||||
window.location.href = checkoutUrl;
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : 'Failed to create checkout session');
|
||||
setIsLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
// Handle beta enrollment
|
||||
const handleBetaEnroll = async (e: React.FormEvent) => {
|
||||
e.preventDefault();
|
||||
setBetaError(null);
|
||||
|
||||
// Validation
|
||||
if (!betaEmail) {
|
||||
setBetaError('Please enter your email address');
|
||||
return;
|
||||
}
|
||||
if (!betaAccept) {
|
||||
setBetaError('Please accept the corrections work agreement');
|
||||
return;
|
||||
}
|
||||
if (!betaPromise || betaPromise.length < 10) {
|
||||
setBetaError('Please write a sentence about a song you want to make into karaoke');
|
||||
return;
|
||||
}
|
||||
|
||||
setBetaLoading(true);
|
||||
|
||||
try {
|
||||
const response: BetaEnrollResponse = await api.enrollBetaTester(
|
||||
betaEmail,
|
||||
betaPromise,
|
||||
betaAccept
|
||||
);
|
||||
|
||||
// If we got a session token, store it and redirect to main app
|
||||
if (response.session_token) {
|
||||
setAccessToken(response.session_token);
|
||||
setBetaSuccess(true);
|
||||
setBetaWillRedirect(true);
|
||||
// Redirect to main app after showing success message
|
||||
redirectTimeoutRef.current = setTimeout(() => {
|
||||
router.push('/app');
|
||||
}, 2000);
|
||||
} else {
|
||||
// No token means email verification needed
|
||||
setBetaSuccess(true);
|
||||
setBetaWillRedirect(false);
|
||||
}
|
||||
} catch (err) {
|
||||
setBetaError(err instanceof Error ? err.message : 'Failed to enroll in beta program');
|
||||
} finally {
|
||||
setBetaLoading(false);
|
||||
}
|
||||
};
|
||||
|
||||
const formatPrice = (cents: number) => `$${(cents / 100).toFixed(0)}`;
|
||||
const pricePerVideo = (pkg: CreditPackage) =>
|
||||
`$${(pkg.price_cents / 100 / pkg.credits).toFixed(2)}`;
|
||||
|
||||
// Show loading while checking auth
|
||||
if (isCheckingAuth) {
|
||||
return (
|
||||
<div className="min-h-screen flex items-center justify-center animated-gradient">
|
||||
<Loader2 className="w-8 h-8 animate-spin text-white" />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="min-h-screen" style={{ background: 'var(--bg)' }}>
|
||||
{/* AutoProcessor - handles non-interactive mode */}
|
||||
<AutoProcessor jobs={jobs} onJobsChanged={loadJobs} />
|
||||
|
||||
{/* Header */}
|
||||
<header className="border-b backdrop-blur-sm sticky top-0 z-10" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
|
||||
<div className="px-3 sm:px-4 py-2 sm:py-3 flex items-center justify-between gap-2">
|
||||
<div className="flex items-center gap-2 sm:gap-3 min-w-0">
|
||||
<img src="/nomad-karaoke-logo.svg" alt="Nomad Karaoke" className="h-8 sm:h-10 shrink-0" />
|
||||
<div className="min-w-0">
|
||||
<h1 className="text-base sm:text-xl font-bold truncate" style={{ color: 'var(--text)' }}>Karaoke Generator</h1>
|
||||
</div>
|
||||
<div className="min-h-screen animated-gradient">
|
||||
{/* Navigation */}
|
||||
<nav className="fixed top-0 left-0 right-0 z-50 bg-dark-900/80 backdrop-blur-md border-b border-dark-700">
|
||||
<div className="max-w-6xl mx-auto px-4 py-4 flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<MicVocal className="w-8 h-8 text-primary-500" />
|
||||
<span className="text-xl font-bold">Nomad Karaoke</span>
|
||||
</div>
|
||||
<div className="flex items-center gap-1 sm:gap-2 shrink-0">
|
||||
<TooltipProvider>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={toggleAutoMode}
|
||||
className={`min-h-[40px] px-2 sm:px-3 ${autoModeEnabled ? "text-amber-400 hover:text-amber-300 bg-amber-500/10" : ""}`}
|
||||
style={!autoModeEnabled ? { color: 'var(--text-muted)' } : undefined}
|
||||
>
|
||||
{autoModeEnabled ? (
|
||||
<Zap className="w-4 h-4 sm:mr-2" />
|
||||
) : (
|
||||
<ZapOff className="w-4 h-4 sm:mr-2" />
|
||||
)}
|
||||
<span className="hidden sm:inline">Auto</span>
|
||||
</Button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom" className="max-w-xs">
|
||||
<p className="font-medium">
|
||||
{autoModeEnabled ? "Auto Mode Enabled" : "Auto Mode Disabled"}
|
||||
</p>
|
||||
<p className="text-xs mt-1" style={{ color: 'var(--text-muted)' }}>
|
||||
{autoModeEnabled
|
||||
? "Jobs will auto-complete review and select clean instrumental (like -y flag)"
|
||||
: "Click to enable non-interactive mode"}
|
||||
</p>
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</TooltipProvider>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={loadJobs}
|
||||
disabled={isLoadingJobs || !isAuthenticated}
|
||||
className="min-h-[40px] px-2 sm:px-3"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
>
|
||||
<RefreshCw className={`w-4 h-4 sm:mr-2 ${isLoadingJobs ? 'animate-spin' : ''}`} />
|
||||
<span className="hidden sm:inline">Refresh</span>
|
||||
</Button>
|
||||
<AuthStatus />
|
||||
{mounted && (
|
||||
<TooltipProvider>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={toggleTheme}
|
||||
className="min-h-[40px] px-2 sm:px-3"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
>
|
||||
{isDarkMode ? (
|
||||
<Sun className="w-4 h-4" />
|
||||
) : (
|
||||
<Moon className="w-4 h-4" />
|
||||
)}
|
||||
</Button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom">
|
||||
<p>{isDarkMode ? "Switch to light mode" : "Switch to dark mode"}</p>
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</TooltipProvider>
|
||||
<button
|
||||
onClick={() => setShowAuthDialog(true)}
|
||||
className="text-sm text-dark-300 hover:text-white transition-colors"
|
||||
>
|
||||
Already have credits? Sign in
|
||||
</button>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
{/* Hero Section */}
|
||||
<section className="pt-32 pb-16 px-4">
|
||||
<div className="max-w-4xl mx-auto text-center">
|
||||
<h1 className="text-4xl sm:text-5xl md:text-6xl font-extrabold mb-6 leading-tight">
|
||||
Turn Any Song Into a{' '}
|
||||
<span className="gradient-text">Karaoke Video</span>
|
||||
</h1>
|
||||
<p className="text-xl text-dark-300 mb-8 max-w-2xl mx-auto">
|
||||
Professional karaoke videos in minutes. AI-powered vocal removal,
|
||||
perfect lyrics sync, and stunning video output.
|
||||
</p>
|
||||
<a
|
||||
href="#pricing"
|
||||
className="inline-flex items-center gap-2 bg-primary-500 hover:bg-primary-600 text-white font-semibold px-8 py-4 rounded-xl transition-all btn-glow"
|
||||
>
|
||||
Get Started
|
||||
<ChevronDown className="w-5 h-5" />
|
||||
</a>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* How It Works */}
|
||||
<section className="py-16 px-4">
|
||||
<div className="max-w-6xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-12">How It Works</h2>
|
||||
<div className="grid md:grid-cols-4 gap-8">
|
||||
{[
|
||||
{ icon: Music, title: 'Search', desc: 'Enter artist & song title' },
|
||||
{ icon: Sparkles, title: 'AI Magic', desc: 'We remove vocals & sync lyrics' },
|
||||
{ icon: Video, title: 'Review', desc: 'Preview and fine-tune' },
|
||||
{ icon: Youtube, title: 'Export', desc: 'Download or upload to YouTube' },
|
||||
].map((step, i) => (
|
||||
<div key={i} className="text-center">
|
||||
<div className="w-16 h-16 bg-primary-500/20 rounded-2xl flex items-center justify-center mx-auto mb-4">
|
||||
<step.icon className="w-8 h-8 text-primary-400" />
|
||||
</div>
|
||||
<h3 className="font-semibold text-lg mb-2">{step.title}</h3>
|
||||
<p className="text-dark-400 text-sm">{step.desc}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Features */}
|
||||
<section className="py-16 px-4 bg-dark-800/50">
|
||||
<div className="max-w-6xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-12">What You Get</h2>
|
||||
<div className="grid md:grid-cols-3 gap-6">
|
||||
{[
|
||||
{ title: 'Studio-Quality Separation', desc: 'State-of-the-art AI removes vocals cleanly, keeping the instrumental crisp.' },
|
||||
{ title: 'Perfect Lyrics Sync', desc: 'Word-by-word timing that highlights exactly when to sing.' },
|
||||
{ title: 'Multiple Instrumental Options', desc: 'Choose from different vocal removal levels to get the sound you want.' },
|
||||
{ title: '4K Video Output', desc: 'Stunning video quality that looks great on any screen.' },
|
||||
{ title: 'YouTube Integration', desc: 'Upload directly to your YouTube channel with one click.' },
|
||||
{ title: 'Edit Before Export', desc: 'Fix any lyrics issues before generating your final video.' },
|
||||
].map((feature, i) => (
|
||||
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl p-6 card-hover">
|
||||
<h3 className="font-semibold text-lg mb-2">{feature.title}</h3>
|
||||
<p className="text-dark-400 text-sm">{feature.desc}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Beta Tester Program */}
|
||||
<section className="py-12 px-4">
|
||||
<div className="max-w-2xl mx-auto">
|
||||
<div className="bg-gradient-to-r from-purple-900/40 to-pink-900/40 border border-purple-500/30 rounded-2xl p-8 text-center">
|
||||
<div className="inline-flex items-center gap-2 bg-purple-500/20 text-purple-300 px-4 py-2 rounded-full text-sm font-medium mb-4">
|
||||
<Gift className="w-4 h-4" />
|
||||
Beta Tester Program
|
||||
</div>
|
||||
<h3 className="text-2xl font-bold mb-3">Try It Free!</h3>
|
||||
<p className="text-dark-300 mb-6">
|
||||
Help us improve by testing the tool and sharing your feedback. Get a{' '}
|
||||
<span className="text-green-400 font-semibold">free credit</span> to
|
||||
create your first karaoke video!
|
||||
</p>
|
||||
|
||||
{!showBetaForm && !betaSuccess && (
|
||||
<button
|
||||
onClick={() => setShowBetaForm(true)}
|
||||
className="inline-flex items-center gap-2 bg-purple-500 hover:bg-purple-600 text-white font-semibold px-6 py-3 rounded-xl transition-all"
|
||||
>
|
||||
Join Beta Program
|
||||
<MessageSquare className="w-5 h-5" />
|
||||
</button>
|
||||
)}
|
||||
|
||||
{showBetaForm && !betaSuccess && (
|
||||
<form onSubmit={handleBetaEnroll} className="mt-6 text-left space-y-4">
|
||||
<div>
|
||||
<label htmlFor="beta-email" className="block text-sm font-medium text-dark-300 mb-2">
|
||||
Email address
|
||||
</label>
|
||||
<input
|
||||
type="email"
|
||||
id="beta-email"
|
||||
value={betaEmail}
|
||||
onChange={(e) => setBetaEmail(e.target.value)}
|
||||
placeholder="you@example.com"
|
||||
className="w-full px-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 text-white placeholder-dark-500"
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<label htmlFor="beta-promise" className="block text-sm font-medium text-dark-300 mb-2">
|
||||
What song do you want to make into karaoke?
|
||||
</label>
|
||||
<textarea
|
||||
id="beta-promise"
|
||||
value={betaPromise}
|
||||
onChange={(e) => setBetaPromise(e.target.value)}
|
||||
placeholder="Tell us about a song you'd love to sing..."
|
||||
rows={3}
|
||||
className="w-full px-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-purple-500 text-white placeholder-dark-500 resize-none"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="flex items-start gap-3">
|
||||
<input
|
||||
type="checkbox"
|
||||
id="beta-accept"
|
||||
checked={betaAccept}
|
||||
onChange={(e) => setBetaAccept(e.target.checked)}
|
||||
className="mt-1 w-4 h-4 rounded border-dark-600 bg-dark-900 text-purple-500 focus:ring-purple-500"
|
||||
/>
|
||||
<label htmlFor="beta-accept" className="text-sm text-dark-300">
|
||||
I understand that beta testers help improve lyrics accuracy by reviewing and correcting AI-generated lyrics
|
||||
</label>
|
||||
</div>
|
||||
|
||||
{betaError && (
|
||||
<p className="text-red-400 text-sm">{betaError}</p>
|
||||
)}
|
||||
|
||||
<button
|
||||
type="submit"
|
||||
disabled={betaLoading}
|
||||
className="w-full bg-purple-500 hover:bg-purple-600 disabled:bg-purple-500/50 text-white font-semibold py-3 rounded-xl transition-all flex items-center justify-center gap-2"
|
||||
>
|
||||
{betaLoading ? 'Enrolling...' : 'Get My Free Credit'}
|
||||
{!betaLoading && <Gift className="w-5 h-5" />}
|
||||
</button>
|
||||
</form>
|
||||
)}
|
||||
|
||||
{betaSuccess && (
|
||||
<div className="mt-6 p-4 bg-green-500/20 border border-green-500/30 rounded-xl">
|
||||
<p className="text-green-400 font-semibold">
|
||||
Welcome to the beta program!{' '}
|
||||
{betaWillRedirect
|
||||
? 'Redirecting to the app...'
|
||||
: 'Check your email for a sign-in link.'}
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
</section>
|
||||
|
||||
<main className="px-4 py-8 space-y-6">
|
||||
{/* Auto Mode Banner */}
|
||||
{autoModeEnabled && (
|
||||
<div className="bg-amber-500/10 border border-amber-500/30 rounded-lg p-4 flex items-center gap-3">
|
||||
<Zap className="w-5 h-5 text-amber-400 flex-shrink-0" />
|
||||
<div className="flex-1">
|
||||
<p className="text-sm text-amber-200 font-medium">Non-Interactive Mode Active</p>
|
||||
<p className="text-xs text-amber-200/70 mt-0.5">
|
||||
Jobs will automatically accept lyrics and select clean instrumental (equivalent to -y flag)
|
||||
</p>
|
||||
{/* Pricing */}
|
||||
<section id="pricing" className="py-20 px-4">
|
||||
<div className="max-w-4xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-4">Simple Pricing</h2>
|
||||
<p className="text-dark-400 text-center mb-12 max-w-xl mx-auto">
|
||||
One credit = one karaoke video. Buy more to save more.
|
||||
</p>
|
||||
|
||||
{/* Package Selection */}
|
||||
{packagesLoading && (
|
||||
<WarmingUpLoader className="py-8" />
|
||||
)}
|
||||
{packagesError && (
|
||||
<div className="text-center py-8 text-red-400">
|
||||
{packagesError}
|
||||
</div>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={toggleAutoMode}
|
||||
className="text-amber-200 hover:text-amber-100 hover:bg-amber-500/20"
|
||||
>
|
||||
Disable
|
||||
</Button>
|
||||
)}
|
||||
<div className="grid grid-cols-2 md:grid-cols-4 gap-4 mb-8">
|
||||
{packages.map((pkg) => {
|
||||
const isSelected = selectedPackage?.id === pkg.id;
|
||||
const isBestValue = pkg.credits === 5;
|
||||
const savings = pkg.credits > 1 ? Math.round((1 - pkg.price_cents / (pkg.credits * 500)) * 100) : 0;
|
||||
|
||||
return (
|
||||
<button
|
||||
key={pkg.id}
|
||||
onClick={() => setSelectedPackage(pkg)}
|
||||
className={`relative p-4 rounded-xl border-2 transition-all ${
|
||||
isSelected
|
||||
? 'border-primary-500 bg-primary-500/10'
|
||||
: 'border-dark-700 hover:border-dark-500 bg-dark-800/50'
|
||||
}`}
|
||||
>
|
||||
{isBestValue && (
|
||||
<span className="absolute -top-3 left-1/2 -translate-x-1/2 bg-primary-500 text-white text-xs font-semibold px-3 py-1 rounded-full">
|
||||
Best Value
|
||||
</span>
|
||||
)}
|
||||
<div className="text-2xl font-bold mb-1">{pkg.credits}</div>
|
||||
<div className="text-sm text-dark-400 mb-2">
|
||||
{pkg.credits === 1 ? 'credit' : 'credits'}
|
||||
</div>
|
||||
<div className="text-xl font-semibold text-primary-400">
|
||||
{formatPrice(pkg.price_cents)}
|
||||
</div>
|
||||
<div className="text-xs text-dark-500">{pricePerVideo(pkg)}/video</div>
|
||||
{savings > 0 && (
|
||||
<div className="text-xs text-green-400 mt-1">Save {savings}%</div>
|
||||
)}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Unauthenticated state */}
|
||||
{!isAuthenticated && (
|
||||
<div className="bg-amber-500/10 border border-amber-500/30 rounded-lg p-6 text-center">
|
||||
<KeyRound className="w-8 h-8 text-amber-400 mx-auto mb-3" />
|
||||
<h3 className="text-lg font-medium mb-2" style={{ color: 'var(--text)' }}>Authentication Required</h3>
|
||||
<p className="text-sm mb-4" style={{ color: 'var(--text-muted)' }}>
|
||||
Click the Login button in the header to enter your access token
|
||||
</p>
|
||||
{/* Checkout Form */}
|
||||
<div className="bg-dark-800 border border-dark-700 rounded-2xl p-8 max-w-md mx-auto">
|
||||
<div className="text-center mb-6">
|
||||
<div className="text-sm text-dark-400 mb-1">Selected package</div>
|
||||
<div className="text-2xl font-bold">
|
||||
{selectedPackage?.credits} Credits
|
||||
</div>
|
||||
<div className="text-3xl font-bold text-primary-400">
|
||||
{selectedPackage && formatPrice(selectedPackage.price_cents)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<form onSubmit={handleCheckout} className="space-y-4">
|
||||
<div>
|
||||
<label htmlFor="email" className="block text-sm font-medium text-dark-300 mb-2">
|
||||
Email address
|
||||
</label>
|
||||
<div className="relative">
|
||||
<Mail className="absolute left-3 top-1/2 -translate-y-1/2 w-5 h-5 text-dark-500" />
|
||||
<input
|
||||
type="email"
|
||||
id="email"
|
||||
value={email}
|
||||
onChange={(e) => setEmail(e.target.value)}
|
||||
placeholder="you@example.com"
|
||||
className="w-full pl-10 pr-4 py-3 bg-dark-900 border border-dark-600 rounded-lg focus:outline-none focus:ring-2 focus:ring-primary-500 focus:border-transparent text-white placeholder-dark-500"
|
||||
required
|
||||
/>
|
||||
</div>
|
||||
<p className="text-xs text-dark-500 mt-2">
|
||||
We'll send your login link to this email
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{error && <p className="text-red-400 text-sm">{error}</p>}
|
||||
|
||||
<button
|
||||
type="submit"
|
||||
disabled={isLoading || !selectedPackage}
|
||||
className="w-full bg-primary-500 hover:bg-primary-600 disabled:bg-primary-500/50 text-white font-semibold py-4 rounded-xl transition-all btn-glow flex items-center justify-center gap-2"
|
||||
>
|
||||
{isLoading ? 'Redirecting...' : 'Continue to Payment'}
|
||||
{!isLoading && <Check className="w-5 h-5" />}
|
||||
</button>
|
||||
</form>
|
||||
|
||||
<div className="mt-4 flex items-center justify-center gap-4 text-xs text-dark-500">
|
||||
<span>Secure checkout</span>
|
||||
<span>powered by Stripe</span>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="grid lg:grid-cols-2 gap-8">
|
||||
{/* Submit Job Card */}
|
||||
<Card className="backdrop-blur" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
|
||||
<CardHeader>
|
||||
<CardTitle style={{ color: 'var(--text)' }}>Create Karaoke Video</CardTitle>
|
||||
<CardDescription style={{ color: 'var(--text-muted)' }}>
|
||||
Upload an audio file, provide a YouTube URL, or search for audio by artist & title
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<JobSubmission onJobCreated={loadJobs} />
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Jobs List Card */}
|
||||
<Card className="backdrop-blur" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--card)' }}>
|
||||
<CardHeader>
|
||||
<CardTitle className="flex items-center justify-between" style={{ color: 'var(--text)' }}>
|
||||
Recent Jobs
|
||||
{isLoadingJobs && <Loader2 className="w-4 h-4 animate-spin" style={{ color: 'var(--text-muted)' }} />}
|
||||
</CardTitle>
|
||||
<CardDescription style={{ color: 'var(--text-muted)' }}>
|
||||
{jobs.length} job{jobs.length !== 1 ? 's' : ''} found
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{jobs.length === 0 ? (
|
||||
<div className="text-center py-8" style={{ color: 'var(--text-muted)' }}>
|
||||
<Music2 className="w-12 h-12 mx-auto mb-3 opacity-50" />
|
||||
<p>No jobs yet. Create one to get started!</p>
|
||||
</div>
|
||||
) : (
|
||||
<div className="space-y-3 max-h-[500px] overflow-y-auto pr-2">
|
||||
{jobs.map((job) => (
|
||||
<JobCard key={job.job_id} job={job} onRefresh={loadJobs} />
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
</main>
|
||||
</section>
|
||||
|
||||
{/* FAQ */}
|
||||
<section className="py-16 px-4 bg-dark-800/50">
|
||||
<div className="max-w-2xl mx-auto">
|
||||
<h2 className="text-3xl font-bold text-center mb-12">Questions?</h2>
|
||||
<div className="space-y-4">
|
||||
{faqs.map((faq, i) => (
|
||||
<div key={i} className="bg-dark-800 border border-dark-700 rounded-xl overflow-hidden">
|
||||
<button
|
||||
onClick={() => setExpandedFaq(expandedFaq === i ? null : i)}
|
||||
aria-expanded={expandedFaq === i}
|
||||
className="w-full px-6 py-4 text-left flex items-center justify-between hover:bg-dark-700/50 transition-colors"
|
||||
>
|
||||
<span className="font-medium">{faq.question}</span>
|
||||
<ChevronDown
|
||||
className={`w-5 h-5 text-dark-400 transition-transform ${
|
||||
expandedFaq === i ? 'rotate-180' : ''
|
||||
}`}
|
||||
/>
|
||||
</button>
|
||||
{expandedFaq === i && (
|
||||
<div className="px-6 pb-4 text-dark-300 text-sm">{faq.answer}</div>
|
||||
)}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{/* Footer */}
|
||||
<footer className="border-t mt-12 py-6" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<div className="px-4 text-center text-sm" style={{ color: 'var(--text-muted)' }}>
|
||||
<p>
|
||||
Powered by{" "}
|
||||
<a href="https://github.com/nomadkaraoke/karaoke-gen" className="text-amber-500 hover:underline">
|
||||
karaoke-gen
|
||||
<footer className="py-12 px-4 border-t border-dark-700">
|
||||
<div className="max-w-6xl mx-auto flex flex-col md:flex-row items-center justify-between gap-4">
|
||||
<div className="flex items-center gap-2">
|
||||
<MicVocal className="w-6 h-6 text-primary-500" />
|
||||
<span className="font-semibold">Nomad Karaoke</span>
|
||||
</div>
|
||||
<div className="text-sm text-dark-500">
|
||||
© {new Date().getFullYear()} Nomad Karaoke. All rights reserved.
|
||||
</div>
|
||||
<div className="flex gap-6 text-sm text-dark-400">
|
||||
<a href="mailto:support@nomadkaraoke.com" className="hover:text-white transition-colors">
|
||||
Support
|
||||
</a>
|
||||
</p>
|
||||
<a href="https://nomadkaraoke.com" className="hover:text-white transition-colors">
|
||||
About
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
{/* Auth Dialog for sign-in */}
|
||||
<AuthDialog
|
||||
open={showAuthDialog}
|
||||
onClose={() => setShowAuthDialog(false)}
|
||||
onSuccess={handleAuthSuccess}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
"use client"
|
||||
|
||||
import { Suspense, useEffect, useState } from "react"
|
||||
import { useRouter, useSearchParams } from "next/navigation"
|
||||
import { CheckCircle, Loader2, Coins } from "lucide-react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { useAuth } from "@/lib/auth"
|
||||
|
||||
function PaymentSuccessContent() {
|
||||
const router = useRouter()
|
||||
const searchParams = useSearchParams()
|
||||
const [isLoading, setIsLoading] = useState(true)
|
||||
|
||||
const { user, fetchUser } = useAuth()
|
||||
|
||||
useEffect(() => {
|
||||
// Refresh user data to get updated credits
|
||||
const refreshUser = async () => {
|
||||
await fetchUser()
|
||||
setIsLoading(false)
|
||||
}
|
||||
refreshUser()
|
||||
}, [fetchUser])
|
||||
|
||||
const sessionId = searchParams.get("session_id")
|
||||
|
||||
return (
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
<CheckCircle className="w-16 h-16 text-green-400 mx-auto mb-4" />
|
||||
|
||||
<h1 className="text-2xl font-bold text-white mb-2">
|
||||
Payment Successful!
|
||||
</h1>
|
||||
|
||||
<p className="text-slate-400 mb-6">
|
||||
Thank you for your purchase. Your credits have been added to your account.
|
||||
</p>
|
||||
|
||||
{isLoading ? (
|
||||
<div className="flex items-center justify-center gap-2 text-slate-400 mb-6">
|
||||
<Loader2 className="w-4 h-4 animate-spin" />
|
||||
Loading your balance...
|
||||
</div>
|
||||
) : user ? (
|
||||
<div className="bg-slate-800 rounded-lg p-4 mb-6">
|
||||
<div className="flex items-center justify-center gap-2 text-amber-400">
|
||||
<Coins className="w-5 h-5" />
|
||||
<span className="text-2xl font-bold">{user.credits}</span>
|
||||
<span className="text-slate-400">credits available</span>
|
||||
</div>
|
||||
</div>
|
||||
) : null}
|
||||
|
||||
<div className="space-y-3">
|
||||
<Button
|
||||
onClick={() => router.push("/app")}
|
||||
className="w-full bg-blue-600 hover:bg-blue-500"
|
||||
>
|
||||
Start Creating Karaoke
|
||||
</Button>
|
||||
|
||||
<p className="text-xs text-slate-500">
|
||||
A confirmation email has been sent to your email address.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export default function PaymentSuccessPage() {
|
||||
return (
|
||||
<div className="min-h-screen bg-slate-950 flex items-center justify-center p-4">
|
||||
<Suspense
|
||||
fallback={
|
||||
<div className="max-w-md w-full bg-slate-900 border border-slate-700 rounded-xl p-8 text-center">
|
||||
<Loader2 className="w-16 h-16 text-blue-400 mx-auto mb-4 animate-spin" />
|
||||
<p className="text-slate-400">Loading...</p>
|
||||
</div>
|
||||
}
|
||||
>
|
||||
<PaymentSuccessContent />
|
||||
</Suspense>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
import { useEffect, useCallback, useRef } from "react"
|
||||
import { api, Job } from "@/lib/api"
|
||||
import { useAutoMode } from "@/lib/auto-mode"
|
||||
|
||||
interface AutoProcessorProps {
|
||||
/** List of jobs to monitor */
|
||||
@@ -13,27 +12,27 @@ interface AutoProcessorProps {
|
||||
|
||||
/**
|
||||
* AutoProcessor component that automatically handles interactive states
|
||||
* when auto-mode is enabled.
|
||||
* for jobs that have non_interactive mode enabled.
|
||||
*
|
||||
* This is equivalent to the -y flag in karaoke-gen-remote CLI:
|
||||
* - Auto-completes lyrics review (accepts auto-corrected lyrics)
|
||||
* - Auto-selects "clean" instrumental
|
||||
* - Auto-selects instrumental based on backing vocals analyzer recommendation
|
||||
* - Auto-selects first/best audio search result
|
||||
*/
|
||||
export function AutoProcessor({ jobs, onJobsChanged }: AutoProcessorProps) {
|
||||
const { enabled, markProcessing, unmarkProcessing, isProcessing } = useAutoMode()
|
||||
const processedRef = useRef<Set<string>>(new Set())
|
||||
const processingRef = useRef<Set<string>>(new Set())
|
||||
|
||||
const processJob = useCallback(async (job: Job, action: string) => {
|
||||
const processingKey = `${job.job_id}-${action}`
|
||||
|
||||
// Skip if already being processed
|
||||
if (isProcessing(processingKey) || processedRef.current.has(processingKey)) {
|
||||
// Skip if already being processed or already processed
|
||||
if (processingRef.current.has(processingKey) || processedRef.current.has(processingKey)) {
|
||||
return
|
||||
}
|
||||
|
||||
// Mark as processing
|
||||
markProcessing(processingKey)
|
||||
processingRef.current.add(processingKey)
|
||||
processedRef.current.add(processingKey)
|
||||
|
||||
console.log(`[AutoProcessor] Auto-processing ${action} for job ${job.job_id}`)
|
||||
@@ -46,11 +45,16 @@ export function AutoProcessor({ jobs, onJobsChanged }: AutoProcessorProps) {
|
||||
console.log(`[AutoProcessor] Auto-completed review for job ${job.job_id}`)
|
||||
break
|
||||
|
||||
case 'instrumental':
|
||||
// Auto-select "clean" instrumental
|
||||
await api.selectInstrumental(job.job_id, 'clean')
|
||||
console.log(`[AutoProcessor] Auto-selected clean instrumental for job ${job.job_id}`)
|
||||
case 'instrumental': {
|
||||
// Use backing vocals analyzer recommendation (default to clean if review needed or missing)
|
||||
const analysis = job.state_data?.backing_vocals_analysis
|
||||
const recommendation = analysis?.recommended_selection
|
||||
// Only use 'with_backing' if explicitly recommended by analyzer, otherwise clean (safe default)
|
||||
const selection = recommendation === 'with_backing' ? 'with_backing' : 'clean'
|
||||
await api.selectInstrumental(job.job_id, selection)
|
||||
console.log(`[AutoProcessor] Auto-selected ${selection} instrumental for job ${job.job_id} (analyzer recommendation: ${recommendation || 'none'})`)
|
||||
break
|
||||
}
|
||||
|
||||
case 'audio':
|
||||
// Auto-select first audio search result (index 0)
|
||||
@@ -66,15 +70,18 @@ export function AutoProcessor({ jobs, onJobsChanged }: AutoProcessorProps) {
|
||||
// Remove from processed so it can be retried
|
||||
processedRef.current.delete(processingKey)
|
||||
} finally {
|
||||
unmarkProcessing(processingKey)
|
||||
processingRef.current.delete(processingKey)
|
||||
}
|
||||
}, [isProcessing, markProcessing, unmarkProcessing, onJobsChanged])
|
||||
}, [onJobsChanged])
|
||||
|
||||
useEffect(() => {
|
||||
if (!enabled) return
|
||||
|
||||
// Find jobs that need auto-processing
|
||||
// Find jobs that need auto-processing (only jobs with non_interactive flag)
|
||||
for (const job of jobs) {
|
||||
// Only process jobs with non_interactive mode enabled
|
||||
if (!job.non_interactive) {
|
||||
continue
|
||||
}
|
||||
|
||||
switch (job.status) {
|
||||
case 'awaiting_review':
|
||||
processJob(job, 'review')
|
||||
@@ -87,7 +94,7 @@ export function AutoProcessor({ jobs, onJobsChanged }: AutoProcessorProps) {
|
||||
break
|
||||
}
|
||||
}
|
||||
}, [enabled, jobs, processJob])
|
||||
}, [jobs, processJob])
|
||||
|
||||
// This component doesn't render anything - it's purely for side effects
|
||||
return null
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
'use client';
|
||||
|
||||
import { useState, useEffect } from 'react';
|
||||
import { Loader2 } from 'lucide-react';
|
||||
|
||||
interface WarmingUpLoaderProps {
|
||||
/** Custom class name for the container */
|
||||
className?: string;
|
||||
/** Custom class name for the spinner */
|
||||
spinnerClassName?: string;
|
||||
/** Time in ms before showing "warming up" message (default: 5000) */
|
||||
warmUpDelay?: number;
|
||||
/** Time in ms before showing "almost there" message (default: 15000) */
|
||||
almostThereDelay?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* A loading component that gracefully handles cold start delays.
|
||||
* Shows increasingly reassuring messages as load time increases.
|
||||
*/
|
||||
export function WarmingUpLoader({
|
||||
className = '',
|
||||
spinnerClassName = 'w-8 h-8',
|
||||
warmUpDelay = 5000,
|
||||
almostThereDelay = 15000,
|
||||
}: WarmingUpLoaderProps) {
|
||||
const [phase, setPhase] = useState<'initial' | 'warming' | 'almost'>('initial');
|
||||
|
||||
useEffect(() => {
|
||||
const warmUpTimer = setTimeout(() => {
|
||||
setPhase('warming');
|
||||
}, warmUpDelay);
|
||||
|
||||
const almostThereTimer = setTimeout(() => {
|
||||
setPhase('almost');
|
||||
}, almostThereDelay);
|
||||
|
||||
return () => {
|
||||
clearTimeout(warmUpTimer);
|
||||
clearTimeout(almostThereTimer);
|
||||
};
|
||||
}, [warmUpDelay, almostThereDelay]);
|
||||
|
||||
return (
|
||||
<div className={`flex flex-col items-center justify-center gap-3 ${className}`}>
|
||||
<Loader2
|
||||
className={`animate-spin ${spinnerClassName}`}
|
||||
style={{ color: 'var(--primary)' }}
|
||||
/>
|
||||
{phase === 'warming' && (
|
||||
<p className="text-sm animate-fade-in" style={{ color: 'var(--text-muted)' }}>
|
||||
Warming up the generator...
|
||||
</p>
|
||||
)}
|
||||
{phase === 'almost' && (
|
||||
<div className="text-center animate-fade-in">
|
||||
<p className="text-sm" style={{ color: 'var(--text-muted)' }}>Almost there...</p>
|
||||
<p className="text-xs mt-1" style={{ color: 'var(--text-muted)', opacity: 0.7 }}>
|
||||
First load can take up to 30 seconds
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -5,18 +5,14 @@
|
||||
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
|
||||
import { AuthStatus } from '../auth/AuthStatus'
|
||||
import { AuthDialog } from '../auth/AuthDialog'
|
||||
import { getAccessToken, setAccessToken, clearAccessToken } from '@/lib/api'
|
||||
import { useAuth } from '@/lib/auth'
|
||||
|
||||
// Mock the api module
|
||||
jest.mock('@/lib/api', () => ({
|
||||
getAccessToken: jest.fn(),
|
||||
setAccessToken: jest.fn(),
|
||||
clearAccessToken: jest.fn(),
|
||||
// Mock the auth module
|
||||
jest.mock('@/lib/auth', () => ({
|
||||
useAuth: jest.fn(),
|
||||
}))
|
||||
|
||||
const mockGetAccessToken = getAccessToken as jest.Mock
|
||||
const mockSetAccessToken = setAccessToken as jest.Mock
|
||||
const mockClearAccessToken = clearAccessToken as jest.Mock
|
||||
const mockUseAuth = useAuth as jest.Mock
|
||||
|
||||
// Mock window.location.reload
|
||||
const mockReload = jest.fn()
|
||||
@@ -31,49 +27,75 @@ describe('AuthStatus', () => {
|
||||
})
|
||||
|
||||
it('shows Login button when not authenticated', () => {
|
||||
mockGetAccessToken.mockReturnValue(null)
|
||||
mockUseAuth.mockReturnValue({
|
||||
user: null,
|
||||
logout: jest.fn(),
|
||||
})
|
||||
render(<AuthStatus />)
|
||||
|
||||
expect(screen.getByRole('button', { name: /login/i })).toBeInTheDocument()
|
||||
expect(screen.queryByRole('button', { name: /logout/i })).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows Logout button when authenticated', () => {
|
||||
mockGetAccessToken.mockReturnValue('test-token')
|
||||
it('shows user menu when authenticated', () => {
|
||||
mockUseAuth.mockReturnValue({
|
||||
user: { email: 'test@example.com', credits: 5, display_name: null },
|
||||
logout: jest.fn(),
|
||||
})
|
||||
render(<AuthStatus />)
|
||||
|
||||
expect(screen.getByRole('button', { name: /logout/i })).toBeInTheDocument()
|
||||
expect(screen.queryByRole('button', { name: /login/i })).not.toBeInTheDocument()
|
||||
// Should show user email and credits
|
||||
expect(screen.getByText('test@example.com')).toBeInTheDocument()
|
||||
expect(screen.getByText('5')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('opens auth dialog when Login is clicked', () => {
|
||||
mockGetAccessToken.mockReturnValue(null)
|
||||
mockUseAuth.mockReturnValue({
|
||||
user: null,
|
||||
logout: jest.fn(),
|
||||
sendMagicLink: jest.fn(),
|
||||
loginWithToken: jest.fn(),
|
||||
isLoading: false,
|
||||
error: null,
|
||||
clearError: jest.fn(),
|
||||
})
|
||||
render(<AuthStatus />)
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /login/i }))
|
||||
|
||||
// Dialog should appear with "Authentication Required" title
|
||||
expect(screen.getByText('Authentication Required')).toBeInTheDocument()
|
||||
// Dialog should appear with "Sign In" title
|
||||
expect(screen.getByText('Sign In')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('clears token and reloads when Logout is clicked', () => {
|
||||
mockGetAccessToken.mockReturnValue('test-token')
|
||||
it('renders dropdown trigger button when authenticated', async () => {
|
||||
const mockLogout = jest.fn().mockResolvedValue(undefined)
|
||||
mockUseAuth.mockReturnValue({
|
||||
user: { email: 'test@example.com', credits: 5, display_name: null },
|
||||
logout: mockLogout,
|
||||
})
|
||||
render(<AuthStatus />)
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /logout/i }))
|
||||
// Wait for component to mount
|
||||
const menuButton = await screen.findByRole('button')
|
||||
|
||||
expect(mockClearAccessToken).toHaveBeenCalled()
|
||||
expect(mockReload).toHaveBeenCalled()
|
||||
// Verify the dropdown trigger has correct attributes
|
||||
expect(menuButton).toHaveAttribute('aria-haspopup', 'menu')
|
||||
expect(menuButton).toHaveAttribute('data-slot', 'dropdown-menu-trigger')
|
||||
})
|
||||
|
||||
it('calls onAuthChange callback when logging out', () => {
|
||||
mockGetAccessToken.mockReturnValue('test-token')
|
||||
it('passes onAuthChange prop correctly', async () => {
|
||||
const mockLogout = jest.fn().mockResolvedValue(undefined)
|
||||
const mockOnAuthChange = jest.fn()
|
||||
mockUseAuth.mockReturnValue({
|
||||
user: { email: 'test@example.com', credits: 5, display_name: null },
|
||||
logout: mockLogout,
|
||||
})
|
||||
|
||||
// Component should render without errors when onAuthChange is provided
|
||||
render(<AuthStatus onAuthChange={mockOnAuthChange} />)
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /logout/i }))
|
||||
|
||||
expect(mockOnAuthChange).toHaveBeenCalled()
|
||||
// Wait for component to mount and verify it renders the user menu
|
||||
const menuButton = await screen.findByRole('button')
|
||||
expect(menuButton).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -83,80 +105,145 @@ describe('AuthDialog', () => {
|
||||
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks()
|
||||
mockUseAuth.mockReturnValue({
|
||||
sendMagicLink: jest.fn(),
|
||||
loginWithToken: jest.fn(),
|
||||
isLoading: false,
|
||||
error: null,
|
||||
clearError: jest.fn(),
|
||||
})
|
||||
})
|
||||
|
||||
it('renders when open is true', () => {
|
||||
it('renders email step when open', () => {
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
expect(screen.getByText('Authentication Required')).toBeInTheDocument()
|
||||
expect(screen.getByPlaceholderText('Enter access token')).toBeInTheDocument()
|
||||
expect(screen.getByText('Sign In')).toBeInTheDocument()
|
||||
expect(screen.getByPlaceholderText('you@example.com')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('does not render when open is false', () => {
|
||||
render(<AuthDialog open={false} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
expect(screen.queryByText('Authentication Required')).not.toBeInTheDocument()
|
||||
expect(screen.queryByText('Sign In')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('calls onClose when Cancel is clicked', () => {
|
||||
it('disables Send button when email is empty', () => {
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /cancel/i }))
|
||||
|
||||
expect(mockOnClose).toHaveBeenCalled()
|
||||
const sendButton = screen.getByRole('button', { name: /send sign-in link/i })
|
||||
expect(sendButton).toBeDisabled()
|
||||
})
|
||||
|
||||
it('disables Authenticate button when token is empty', () => {
|
||||
it('enables Send button when email is entered', () => {
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
const authenticateBtn = screen.getByRole('button', { name: /authenticate/i })
|
||||
expect(authenticateBtn).toBeDisabled()
|
||||
const input = screen.getByPlaceholderText('you@example.com')
|
||||
fireEvent.change(input, { target: { value: 'test@example.com' } })
|
||||
|
||||
const sendButton = screen.getByRole('button', { name: /send sign-in link/i })
|
||||
expect(sendButton).not.toBeDisabled()
|
||||
})
|
||||
|
||||
it('enables Authenticate button when token is entered', () => {
|
||||
it('calls sendMagicLink when form is submitted', async () => {
|
||||
const mockSendMagicLink = jest.fn().mockResolvedValue(true)
|
||||
mockUseAuth.mockReturnValue({
|
||||
sendMagicLink: mockSendMagicLink,
|
||||
loginWithToken: jest.fn(),
|
||||
isLoading: false,
|
||||
error: null,
|
||||
clearError: jest.fn(),
|
||||
})
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
const input = screen.getByPlaceholderText('Enter access token')
|
||||
fireEvent.change(input, { target: { value: 'my-token' } })
|
||||
const input = screen.getByPlaceholderText('you@example.com')
|
||||
fireEvent.change(input, { target: { value: 'test@example.com' } })
|
||||
|
||||
const authenticateBtn = screen.getByRole('button', { name: /authenticate/i })
|
||||
expect(authenticateBtn).not.toBeDisabled()
|
||||
const sendButton = screen.getByRole('button', { name: /send sign-in link/i })
|
||||
fireEvent.click(sendButton)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockSendMagicLink).toHaveBeenCalledWith('test@example.com')
|
||||
})
|
||||
})
|
||||
|
||||
it('sets token and calls onSuccess when form is submitted', async () => {
|
||||
it('shows token step when Use Access Token is clicked', () => {
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /use access token instead/i }))
|
||||
|
||||
expect(screen.getByText('Access Token')).toBeInTheDocument()
|
||||
expect(screen.getByPlaceholderText('Enter access token')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('calls loginWithToken when token is submitted', async () => {
|
||||
const mockLoginWithToken = jest.fn().mockResolvedValue(true)
|
||||
mockUseAuth.mockReturnValue({
|
||||
sendMagicLink: jest.fn(),
|
||||
loginWithToken: mockLoginWithToken,
|
||||
isLoading: false,
|
||||
error: null,
|
||||
clearError: jest.fn(),
|
||||
})
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
// Switch to token step
|
||||
fireEvent.click(screen.getByRole('button', { name: /use access token instead/i }))
|
||||
|
||||
const input = screen.getByPlaceholderText('Enter access token')
|
||||
fireEvent.change(input, { target: { value: 'my-token' } })
|
||||
|
||||
const authenticateBtn = screen.getByRole('button', { name: /authenticate/i })
|
||||
fireEvent.click(authenticateBtn)
|
||||
|
||||
expect(mockSetAccessToken).toHaveBeenCalledWith('my-token')
|
||||
expect(mockOnSuccess).toHaveBeenCalled()
|
||||
await waitFor(() => {
|
||||
expect(mockLoginWithToken).toHaveBeenCalledWith('my-token')
|
||||
})
|
||||
})
|
||||
|
||||
it('trims whitespace from token', () => {
|
||||
it('calls onSuccess after successful token login', async () => {
|
||||
const mockLoginWithToken = jest.fn().mockResolvedValue(true)
|
||||
mockUseAuth.mockReturnValue({
|
||||
sendMagicLink: jest.fn(),
|
||||
loginWithToken: mockLoginWithToken,
|
||||
isLoading: false,
|
||||
error: null,
|
||||
clearError: jest.fn(),
|
||||
})
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
// Switch to token step
|
||||
fireEvent.click(screen.getByRole('button', { name: /use access token instead/i }))
|
||||
|
||||
const input = screen.getByPlaceholderText('Enter access token')
|
||||
fireEvent.change(input, { target: { value: ' my-token ' } })
|
||||
fireEvent.change(input, { target: { value: 'my-token' } })
|
||||
|
||||
const authenticateBtn = screen.getByRole('button', { name: /authenticate/i })
|
||||
fireEvent.click(authenticateBtn)
|
||||
|
||||
expect(mockSetAccessToken).toHaveBeenCalledWith('my-token')
|
||||
await waitFor(() => {
|
||||
expect(mockOnSuccess).toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
it('shows error for empty token submission', () => {
|
||||
it('shows email sent step after successful magic link send', async () => {
|
||||
const mockSendMagicLink = jest.fn().mockResolvedValue(true)
|
||||
mockUseAuth.mockReturnValue({
|
||||
sendMagicLink: mockSendMagicLink,
|
||||
loginWithToken: jest.fn(),
|
||||
isLoading: false,
|
||||
error: null,
|
||||
clearError: jest.fn(),
|
||||
})
|
||||
render(<AuthDialog open={true} onClose={mockOnClose} onSuccess={mockOnSuccess} />)
|
||||
|
||||
// Token with only whitespace
|
||||
const input = screen.getByPlaceholderText('Enter access token')
|
||||
fireEvent.change(input, { target: { value: ' ' } })
|
||||
const input = screen.getByPlaceholderText('you@example.com')
|
||||
fireEvent.change(input, { target: { value: 'test@example.com' } })
|
||||
|
||||
// Button should still be disabled for whitespace-only input
|
||||
const authenticateBtn = screen.getByRole('button', { name: /authenticate/i })
|
||||
expect(authenticateBtn).toBeDisabled()
|
||||
const sendButton = screen.getByRole('button', { name: /send sign-in link/i })
|
||||
fireEvent.click(sendButton)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Check Your Email')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -7,14 +7,6 @@
|
||||
import { render, screen } from '@testing-library/react'
|
||||
import { JobCard } from '../job/JobCard'
|
||||
import { Job } from '@/lib/api'
|
||||
import { useAutoMode } from '@/lib/auto-mode'
|
||||
|
||||
// Mock the auto-mode hook
|
||||
jest.mock('@/lib/auto-mode', () => ({
|
||||
useAutoMode: jest.fn()
|
||||
}))
|
||||
|
||||
const mockUseAutoMode = useAutoMode as jest.Mock
|
||||
|
||||
// Mock the child components
|
||||
jest.mock('../job/JobActions', () => ({
|
||||
@@ -48,11 +40,6 @@ describe('JobCard', () => {
|
||||
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks()
|
||||
// Default mock: auto-mode disabled
|
||||
mockUseAutoMode.mockReturnValue({
|
||||
enabled: false,
|
||||
isProcessing: jest.fn().mockReturnValue(false)
|
||||
})
|
||||
})
|
||||
|
||||
it('renders job information', () => {
|
||||
@@ -104,14 +91,7 @@ describe('JobCard', () => {
|
||||
expect(card).toHaveClass('border-red-500/30')
|
||||
})
|
||||
|
||||
describe('when auto-mode is disabled', () => {
|
||||
beforeEach(() => {
|
||||
mockUseAutoMode.mockReturnValue({
|
||||
enabled: false,
|
||||
isProcessing: jest.fn().mockReturnValue(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('regular jobs (non_interactive: false)', () => {
|
||||
it('shows lyrics review link for awaiting_review status', () => {
|
||||
const reviewJob = { ...mockJob, status: 'awaiting_review', review_token: 'test-token-123' }
|
||||
render(<JobCard job={reviewJob} onRefresh={mockOnRefresh} />)
|
||||
@@ -146,18 +126,24 @@ describe('JobCard', () => {
|
||||
expect(href).toContain('baseApiUrl=')
|
||||
expect(href).toContain('instrumentalToken=test-instrumental-token')
|
||||
})
|
||||
|
||||
it('does not show Auto badge for regular jobs', () => {
|
||||
render(<JobCard job={mockJob} onRefresh={mockOnRefresh} />)
|
||||
|
||||
expect(screen.queryByText('Auto')).not.toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
describe('when auto-mode is enabled', () => {
|
||||
beforeEach(() => {
|
||||
mockUseAutoMode.mockReturnValue({
|
||||
enabled: true,
|
||||
isProcessing: jest.fn().mockReturnValue(false)
|
||||
})
|
||||
describe('non-interactive jobs (non_interactive: true)', () => {
|
||||
it('shows Auto badge for non-interactive jobs', () => {
|
||||
const nonInteractiveJob = { ...mockJob, non_interactive: true }
|
||||
render(<JobCard job={nonInteractiveJob} onRefresh={mockOnRefresh} />)
|
||||
|
||||
expect(screen.getByText('Auto')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows auto-accept message for awaiting_review status', () => {
|
||||
const reviewJob = { ...mockJob, status: 'awaiting_review' }
|
||||
const reviewJob = { ...mockJob, status: 'awaiting_review', non_interactive: true }
|
||||
render(<JobCard job={reviewJob} onRefresh={mockOnRefresh} />)
|
||||
|
||||
expect(screen.getByText(/Will auto-accept/)).toBeInTheDocument()
|
||||
@@ -165,32 +151,20 @@ describe('JobCard', () => {
|
||||
})
|
||||
|
||||
it('shows auto-select message for awaiting_instrumental_selection status', () => {
|
||||
const instrumentalJob = { ...mockJob, status: 'awaiting_instrumental_selection' }
|
||||
const instrumentalJob = { ...mockJob, status: 'awaiting_instrumental_selection', non_interactive: true }
|
||||
render(<JobCard job={instrumentalJob} onRefresh={mockOnRefresh} />)
|
||||
|
||||
expect(screen.getByText(/Will auto-select clean/)).toBeInTheDocument()
|
||||
expect(screen.getByText(/Will auto-select/)).toBeInTheDocument()
|
||||
expect(screen.queryByText('Select Instrumental')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows auto-select message for awaiting_audio_selection status', () => {
|
||||
const audioJob = { ...mockJob, status: 'awaiting_audio_selection' }
|
||||
it('shows auto-select first message for awaiting_audio_selection status', () => {
|
||||
const audioJob = { ...mockJob, status: 'awaiting_audio_selection', non_interactive: true }
|
||||
render(<JobCard job={audioJob} onRefresh={mockOnRefresh} />)
|
||||
|
||||
expect(screen.getByText(/Will auto-select first/)).toBeInTheDocument()
|
||||
expect(screen.queryByRole('button', { name: 'Select Audio' })).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows auto-processing indicator when processing', () => {
|
||||
mockUseAutoMode.mockReturnValue({
|
||||
enabled: true,
|
||||
isProcessing: jest.fn().mockReturnValue(true)
|
||||
})
|
||||
|
||||
const reviewJob = { ...mockJob, status: 'awaiting_review' }
|
||||
render(<JobCard job={reviewJob} onRefresh={mockOnRefresh} />)
|
||||
|
||||
expect(screen.getByText(/Auto-processing.../)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('shows output links for complete jobs', () => {
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
"use client"
|
||||
|
||||
import Link from "next/link"
|
||||
import { usePathname } from "next/navigation"
|
||||
import {
|
||||
LayoutDashboard,
|
||||
Users,
|
||||
Briefcase,
|
||||
TestTube2,
|
||||
ArrowLeft,
|
||||
Settings,
|
||||
} from "lucide-react"
|
||||
|
||||
import {
|
||||
Sidebar,
|
||||
SidebarContent,
|
||||
SidebarFooter,
|
||||
SidebarGroup,
|
||||
SidebarGroupContent,
|
||||
SidebarGroupLabel,
|
||||
SidebarHeader,
|
||||
SidebarMenu,
|
||||
SidebarMenuButton,
|
||||
SidebarMenuItem,
|
||||
SidebarSeparator,
|
||||
} from "@/components/ui/sidebar"
|
||||
|
||||
const navItems = [
|
||||
{
|
||||
title: "Dashboard",
|
||||
href: "/admin",
|
||||
icon: LayoutDashboard,
|
||||
},
|
||||
{
|
||||
title: "Users",
|
||||
href: "/admin/users",
|
||||
icon: Users,
|
||||
},
|
||||
{
|
||||
title: "Jobs",
|
||||
href: "/admin/jobs",
|
||||
icon: Briefcase,
|
||||
},
|
||||
{
|
||||
title: "Beta Program",
|
||||
href: "/admin/beta",
|
||||
icon: TestTube2,
|
||||
},
|
||||
]
|
||||
|
||||
export function AdminSidebar() {
|
||||
const pathname = usePathname()
|
||||
|
||||
const isActive = (href: string) => {
|
||||
if (href === "/admin") {
|
||||
return pathname === "/admin"
|
||||
}
|
||||
return pathname.startsWith(href)
|
||||
}
|
||||
|
||||
return (
|
||||
<Sidebar>
|
||||
<SidebarHeader className="border-b border-sidebar-border">
|
||||
<div className="flex items-center gap-2 px-2 py-2">
|
||||
<Settings className="w-6 h-6 text-primary" />
|
||||
<span className="font-semibold text-lg">Admin Panel</span>
|
||||
</div>
|
||||
</SidebarHeader>
|
||||
|
||||
<SidebarContent>
|
||||
<SidebarGroup>
|
||||
<SidebarGroupLabel>Navigation</SidebarGroupLabel>
|
||||
<SidebarGroupContent>
|
||||
<SidebarMenu>
|
||||
{navItems.map((item) => (
|
||||
<SidebarMenuItem key={item.href}>
|
||||
<SidebarMenuButton asChild isActive={isActive(item.href)}>
|
||||
<Link href={item.href}>
|
||||
<item.icon className="w-4 h-4" />
|
||||
<span>{item.title}</span>
|
||||
</Link>
|
||||
</SidebarMenuButton>
|
||||
</SidebarMenuItem>
|
||||
))}
|
||||
</SidebarMenu>
|
||||
</SidebarGroupContent>
|
||||
</SidebarGroup>
|
||||
</SidebarContent>
|
||||
|
||||
<SidebarSeparator />
|
||||
|
||||
<SidebarFooter>
|
||||
<SidebarMenu>
|
||||
<SidebarMenuItem>
|
||||
<SidebarMenuButton asChild>
|
||||
<Link href="/app">
|
||||
<ArrowLeft className="w-4 h-4" />
|
||||
<span>Back to App</span>
|
||||
</Link>
|
||||
</SidebarMenuButton>
|
||||
</SidebarMenuItem>
|
||||
</SidebarMenu>
|
||||
</SidebarFooter>
|
||||
</Sidebar>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,2 @@
|
||||
export { AdminSidebar } from "./admin-sidebar"
|
||||
export { StatsCard, StatsGrid } from "./stats-card"
|
||||
@@ -0,0 +1,87 @@
|
||||
"use client"
|
||||
|
||||
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import { Skeleton } from "@/components/ui/skeleton"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { LucideIcon } from "lucide-react"
|
||||
|
||||
interface StatsCardProps {
|
||||
title: string
|
||||
value: string | number
|
||||
description?: string
|
||||
icon?: LucideIcon
|
||||
trend?: {
|
||||
value: number
|
||||
label: string
|
||||
positive?: boolean
|
||||
}
|
||||
loading?: boolean
|
||||
className?: string
|
||||
}
|
||||
|
||||
export function StatsCard({
|
||||
title,
|
||||
value,
|
||||
description,
|
||||
icon: Icon,
|
||||
trend,
|
||||
loading,
|
||||
className,
|
||||
}: StatsCardProps) {
|
||||
if (loading) {
|
||||
return (
|
||||
<Card className={cn("", className)}>
|
||||
<CardHeader className="flex flex-row items-center justify-between space-y-0 pb-2">
|
||||
<CardTitle className="text-sm font-medium">
|
||||
<Skeleton className="h-4 w-24" />
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<Skeleton className="h-8 w-16" />
|
||||
<Skeleton className="h-3 w-32 mt-2" />
|
||||
</CardContent>
|
||||
</Card>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<Card className={cn("", className)}>
|
||||
<CardHeader className="flex flex-row items-center justify-between space-y-0 pb-2">
|
||||
<CardTitle className="text-sm font-medium text-muted-foreground">
|
||||
{title}
|
||||
</CardTitle>
|
||||
{Icon && <Icon className="h-4 w-4 text-muted-foreground" />}
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="text-2xl font-bold">{value}</div>
|
||||
{description && (
|
||||
<p className="text-xs text-muted-foreground mt-1">{description}</p>
|
||||
)}
|
||||
{trend && (
|
||||
<p className={cn(
|
||||
"text-xs mt-1",
|
||||
trend.positive === true ? "text-green-500" : trend.positive === false ? "text-red-500" : "text-muted-foreground"
|
||||
)}>
|
||||
{trend.positive === true ? "+" : ""}{trend.value}% {trend.label}
|
||||
</p>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
)
|
||||
}
|
||||
|
||||
interface StatsGridProps {
|
||||
children: React.ReactNode
|
||||
className?: string
|
||||
}
|
||||
|
||||
export function StatsGrid({ children, className }: StatsGridProps) {
|
||||
return (
|
||||
<div className={cn(
|
||||
"grid gap-4 md:grid-cols-2 lg:grid-cols-4",
|
||||
className
|
||||
)}>
|
||||
{children}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
"use client"
|
||||
|
||||
import { useState } from "react"
|
||||
import { KeyRound } from "lucide-react"
|
||||
import { Mail, KeyRound, ArrowLeft, Loader2, CheckCircle } from "lucide-react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import {
|
||||
@@ -11,7 +11,7 @@ import {
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog"
|
||||
import { setAccessToken } from "@/lib/api"
|
||||
import { useAuth } from "@/lib/auth"
|
||||
|
||||
interface AuthDialogProps {
|
||||
open: boolean
|
||||
@@ -19,72 +19,229 @@ interface AuthDialogProps {
|
||||
onSuccess: () => void
|
||||
}
|
||||
|
||||
export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
const [token, setToken] = useState("")
|
||||
const [error, setError] = useState("")
|
||||
type AuthStep = "email" | "sent" | "token"
|
||||
|
||||
const handleSubmit = (e: React.FormEvent) => {
|
||||
export function AuthDialog({ open, onClose, onSuccess }: AuthDialogProps) {
|
||||
const [step, setStep] = useState<AuthStep>("email")
|
||||
const [email, setEmail] = useState("")
|
||||
const [token, setToken] = useState("")
|
||||
const [localError, setLocalError] = useState("")
|
||||
|
||||
const { sendMagicLink, loginWithToken, isLoading, error, clearError } = useAuth()
|
||||
|
||||
const handleSendMagicLink = async (e: React.FormEvent) => {
|
||||
e.preventDefault()
|
||||
if (token.trim()) {
|
||||
setAccessToken(token.trim())
|
||||
setError("")
|
||||
setToken("") // Clear token from state for security
|
||||
onSuccess()
|
||||
} else {
|
||||
setError("Please enter a valid token")
|
||||
setLocalError("")
|
||||
|
||||
if (!email.trim()) {
|
||||
setLocalError("Please enter your email")
|
||||
return
|
||||
}
|
||||
|
||||
// Simple email validation
|
||||
if (!email.includes("@") || !email.includes(".")) {
|
||||
setLocalError("Please enter a valid email")
|
||||
return
|
||||
}
|
||||
|
||||
const success = await sendMagicLink(email.trim().toLowerCase())
|
||||
if (success) {
|
||||
setStep("sent")
|
||||
}
|
||||
}
|
||||
|
||||
const handleTokenSubmit = async (e: React.FormEvent) => {
|
||||
e.preventDefault()
|
||||
setLocalError("")
|
||||
|
||||
if (!token.trim()) {
|
||||
setLocalError("Please enter your access token")
|
||||
return
|
||||
}
|
||||
|
||||
const success = await loginWithToken(token.trim())
|
||||
if (success) {
|
||||
setToken("") // Clear for security
|
||||
onSuccess()
|
||||
}
|
||||
}
|
||||
|
||||
const handleClose = () => {
|
||||
setStep("email")
|
||||
setEmail("")
|
||||
setToken("")
|
||||
setLocalError("")
|
||||
clearError()
|
||||
onClose()
|
||||
}
|
||||
|
||||
const displayError = localError || error
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(open) => !open && onClose()}>
|
||||
<Dialog open={open} onOpenChange={(open) => !open && handleClose()}>
|
||||
<DialogContent className="sm:max-w-md bg-slate-900 border-slate-700">
|
||||
<DialogHeader>
|
||||
<DialogTitle className="text-white flex items-center gap-2">
|
||||
<KeyRound className="w-5 h-5 text-amber-400" />
|
||||
Authentication Required
|
||||
</DialogTitle>
|
||||
<DialogDescription className="text-slate-400">
|
||||
Enter your access token to use the Karaoke Generator
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<form onSubmit={handleSubmit} className="space-y-4">
|
||||
<div>
|
||||
<Input
|
||||
type="password"
|
||||
placeholder="Enter access token"
|
||||
value={token}
|
||||
onChange={(e) => {
|
||||
setToken(e.target.value)
|
||||
setError("")
|
||||
}}
|
||||
className="bg-slate-800 border-slate-600 text-white"
|
||||
autoFocus
|
||||
/>
|
||||
{error && (
|
||||
<p className="text-xs text-red-400 mt-1">{error}</p>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex justify-end gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
onClick={onClose}
|
||||
className="text-slate-400"
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
type="submit"
|
||||
disabled={!token.trim()}
|
||||
className="bg-amber-600 hover:bg-amber-500"
|
||||
>
|
||||
Authenticate
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-xs text-slate-500 text-center">
|
||||
Contact an admin to get an access token
|
||||
</p>
|
||||
</form>
|
||||
{step === "email" && (
|
||||
<>
|
||||
<DialogHeader>
|
||||
<DialogTitle className="text-white flex items-center gap-2">
|
||||
<Mail className="w-5 h-5 text-blue-400" />
|
||||
Sign In
|
||||
</DialogTitle>
|
||||
<DialogDescription className="text-slate-400">
|
||||
Enter your email to receive a sign-in link
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<form onSubmit={handleSendMagicLink} className="space-y-4">
|
||||
<div>
|
||||
<Input
|
||||
type="email"
|
||||
placeholder="you@example.com"
|
||||
value={email}
|
||||
onChange={(e) => {
|
||||
setEmail(e.target.value)
|
||||
setLocalError("")
|
||||
clearError()
|
||||
}}
|
||||
className="bg-slate-800 border-slate-600 text-white"
|
||||
autoFocus
|
||||
disabled={isLoading}
|
||||
/>
|
||||
{displayError && (
|
||||
<p className="text-xs text-red-400 mt-1">{displayError}</p>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex flex-col gap-2">
|
||||
<Button
|
||||
type="submit"
|
||||
disabled={!email.trim() || isLoading}
|
||||
className="w-full bg-blue-600 hover:bg-blue-500"
|
||||
>
|
||||
{isLoading ? (
|
||||
<>
|
||||
<Loader2 className="w-4 h-4 mr-2 animate-spin" />
|
||||
Sending...
|
||||
</>
|
||||
) : (
|
||||
"Send Sign-In Link"
|
||||
)}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
onClick={() => setStep("token")}
|
||||
className="text-slate-400 hover:text-white"
|
||||
>
|
||||
<KeyRound className="w-4 h-4 mr-2" />
|
||||
Use Access Token Instead
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-xs text-slate-500 text-center">
|
||||
No account? Just enter your email to get started.
|
||||
</p>
|
||||
</form>
|
||||
</>
|
||||
)}
|
||||
|
||||
{step === "sent" && (
|
||||
<>
|
||||
<DialogHeader>
|
||||
<DialogTitle className="text-white flex items-center gap-2">
|
||||
<CheckCircle className="w-5 h-5 text-green-400" />
|
||||
Check Your Email
|
||||
</DialogTitle>
|
||||
</DialogHeader>
|
||||
<div className="space-y-4 py-4">
|
||||
<p className="text-slate-300">
|
||||
We sent a sign-in link to:
|
||||
</p>
|
||||
<p className="text-white font-medium bg-slate-800 px-4 py-2 rounded-md">
|
||||
{email}
|
||||
</p>
|
||||
<p className="text-sm text-slate-400">
|
||||
Click the link in the email to sign in. The link expires in 15 minutes.
|
||||
</p>
|
||||
<div className="pt-4 border-t border-slate-700 space-y-2">
|
||||
<Button
|
||||
variant="outline"
|
||||
onClick={() => setStep("email")}
|
||||
className="w-full border-slate-600 text-slate-300 hover:text-white"
|
||||
>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Try a Different Email
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
onClick={handleClose}
|
||||
className="w-full text-slate-400"
|
||||
>
|
||||
Close
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
{step === "token" && (
|
||||
<>
|
||||
<DialogHeader>
|
||||
<DialogTitle className="text-white flex items-center gap-2">
|
||||
<KeyRound className="w-5 h-5 text-amber-400" />
|
||||
Access Token
|
||||
</DialogTitle>
|
||||
<DialogDescription className="text-slate-400">
|
||||
Enter your access token to authenticate
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<form onSubmit={handleTokenSubmit} className="space-y-4">
|
||||
<div>
|
||||
<Input
|
||||
type="password"
|
||||
placeholder="Enter access token"
|
||||
value={token}
|
||||
onChange={(e) => {
|
||||
setToken(e.target.value)
|
||||
setLocalError("")
|
||||
clearError()
|
||||
}}
|
||||
className="bg-slate-800 border-slate-600 text-white"
|
||||
autoFocus
|
||||
disabled={isLoading}
|
||||
/>
|
||||
{displayError && (
|
||||
<p className="text-xs text-red-400 mt-1">{displayError}</p>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex flex-col gap-2">
|
||||
<Button
|
||||
type="submit"
|
||||
disabled={!token.trim() || isLoading}
|
||||
className="w-full bg-amber-600 hover:bg-amber-500"
|
||||
>
|
||||
{isLoading ? (
|
||||
<>
|
||||
<Loader2 className="w-4 h-4 mr-2 animate-spin" />
|
||||
Authenticating...
|
||||
</>
|
||||
) : (
|
||||
"Authenticate"
|
||||
)}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
onClick={() => setStep("email")}
|
||||
className="text-slate-400 hover:text-white"
|
||||
>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Email Sign-In
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-xs text-slate-500 text-center">
|
||||
Contact an admin to get an access token
|
||||
</p>
|
||||
</form>
|
||||
</>
|
||||
)}
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
)
|
||||
|
||||
@@ -1,9 +1,17 @@
|
||||
"use client"
|
||||
|
||||
import { useState, useEffect } from "react"
|
||||
import { KeyRound, LogOut } from "lucide-react"
|
||||
import { User, LogOut, CreditCard, Coins, KeyRound } from "lucide-react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { getAccessToken, clearAccessToken } from "@/lib/api"
|
||||
import {
|
||||
DropdownMenu,
|
||||
DropdownMenuContent,
|
||||
DropdownMenuItem,
|
||||
DropdownMenuLabel,
|
||||
DropdownMenuSeparator,
|
||||
DropdownMenuTrigger,
|
||||
} from "@/components/ui/dropdown-menu"
|
||||
import { useAuth } from "@/lib/auth"
|
||||
import { AuthDialog } from "./AuthDialog"
|
||||
|
||||
interface AuthStatusProps {
|
||||
@@ -11,38 +19,94 @@ interface AuthStatusProps {
|
||||
}
|
||||
|
||||
export function AuthStatus({ onAuthChange }: AuthStatusProps) {
|
||||
const [isAuthenticated, setIsAuthenticated] = useState(false)
|
||||
const { user, logout } = useAuth()
|
||||
const [showAuthDialog, setShowAuthDialog] = useState(false)
|
||||
const [mounted, setMounted] = useState(false)
|
||||
|
||||
// Avoid hydration mismatch
|
||||
useEffect(() => {
|
||||
setIsAuthenticated(!!getAccessToken())
|
||||
setMounted(true)
|
||||
}, [])
|
||||
|
||||
const handleLogout = () => {
|
||||
clearAccessToken()
|
||||
setIsAuthenticated(false)
|
||||
const handleLogout = async () => {
|
||||
await logout()
|
||||
onAuthChange?.()
|
||||
window.location.reload()
|
||||
}
|
||||
|
||||
const handleAuthSuccess = () => {
|
||||
setIsAuthenticated(true)
|
||||
setShowAuthDialog(false)
|
||||
onAuthChange?.()
|
||||
window.location.reload()
|
||||
}
|
||||
|
||||
if (isAuthenticated) {
|
||||
const handleBuyCredits = () => {
|
||||
// Navigate to landing page pricing section with email prefilled if available
|
||||
const buyUrl = new URL('/#pricing', window.location.origin)
|
||||
if (user?.email) {
|
||||
buyUrl.searchParams.set("email", user.email)
|
||||
}
|
||||
window.location.href = buyUrl.toString()
|
||||
}
|
||||
|
||||
// Don't render until mounted (avoids hydration issues)
|
||||
if (!mounted) {
|
||||
return null
|
||||
}
|
||||
|
||||
if (user) {
|
||||
return (
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={handleLogout}
|
||||
className="text-slate-400 hover:text-white min-h-[40px] px-2 sm:px-3"
|
||||
>
|
||||
<LogOut className="w-4 h-4 sm:mr-2" />
|
||||
<span className="hidden sm:inline">Logout</span>
|
||||
</Button>
|
||||
<DropdownMenu>
|
||||
<DropdownMenuTrigger asChild>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="text-slate-300 hover:text-white flex items-center gap-2 min-h-[40px] px-2 sm:px-3"
|
||||
>
|
||||
<User className="w-4 h-4" />
|
||||
<span className="hidden sm:inline max-w-[150px] truncate">
|
||||
{user.display_name || user.email}
|
||||
</span>
|
||||
<span className="flex items-center gap-1 text-amber-400 font-medium">
|
||||
<Coins className="w-3 h-3" />
|
||||
{user.credits}
|
||||
</span>
|
||||
</Button>
|
||||
</DropdownMenuTrigger>
|
||||
<DropdownMenuContent align="end" className="w-56 bg-slate-900 border-slate-700">
|
||||
<DropdownMenuLabel className="text-slate-400 font-normal">
|
||||
<div className="flex flex-col space-y-1">
|
||||
<p className="text-sm font-medium text-white">
|
||||
{user.display_name || "Karaoke Fan"}
|
||||
</p>
|
||||
<p className="text-xs text-slate-500 truncate">{user.email}</p>
|
||||
</div>
|
||||
</DropdownMenuLabel>
|
||||
<DropdownMenuSeparator className="bg-slate-700" />
|
||||
<DropdownMenuItem
|
||||
className="text-slate-300 focus:text-white focus:bg-slate-800 cursor-default"
|
||||
disabled
|
||||
>
|
||||
<Coins className="w-4 h-4 mr-2 text-amber-400" />
|
||||
<span>{user.credits} credits available</span>
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuItem
|
||||
onClick={handleBuyCredits}
|
||||
className="text-slate-300 focus:text-white focus:bg-slate-800"
|
||||
>
|
||||
<CreditCard className="w-4 h-4 mr-2" />
|
||||
<span>Buy More Credits</span>
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuSeparator className="bg-slate-700" />
|
||||
<DropdownMenuItem
|
||||
onClick={handleLogout}
|
||||
className="text-red-400 focus:text-red-300 focus:bg-slate-800"
|
||||
>
|
||||
<LogOut className="w-4 h-4 mr-2" />
|
||||
<span>Sign Out</span>
|
||||
</DropdownMenuItem>
|
||||
</DropdownMenuContent>
|
||||
</DropdownMenu>
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
@@ -31,14 +31,15 @@ function ColorField({ id, label, description, value, onChange, disabled }: Color
|
||||
return (
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between">
|
||||
<Label htmlFor={id} className="text-sm text-slate-300">
|
||||
<Label htmlFor={id} className="text-sm" style={{ color: 'var(--text)' }}>
|
||||
{label}
|
||||
</Label>
|
||||
{value && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onChange(undefined)}
|
||||
className="text-xs text-slate-500 hover:text-slate-300"
|
||||
className="text-xs"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
disabled={disabled}
|
||||
>
|
||||
Reset
|
||||
@@ -47,8 +48,8 @@ function ColorField({ id, label, description, value, onChange, disabled }: Color
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<div
|
||||
className="w-8 h-8 rounded border border-slate-600 flex-shrink-0"
|
||||
style={{ backgroundColor: value || "#333333" }}
|
||||
className="w-8 h-8 rounded border flex-shrink-0"
|
||||
style={{ backgroundColor: value || "#333333", borderColor: 'var(--card-border)' }}
|
||||
/>
|
||||
<Input
|
||||
id={id}
|
||||
@@ -64,7 +65,8 @@ function ColorField({ id, label, description, value, onChange, disabled }: Color
|
||||
}
|
||||
}}
|
||||
disabled={disabled}
|
||||
className="bg-slate-800 border-slate-700 text-white placeholder:text-slate-500 font-mono text-sm"
|
||||
className="font-mono text-sm"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
<Input
|
||||
type="color"
|
||||
@@ -74,7 +76,7 @@ function ColorField({ id, label, description, value, onChange, disabled }: Color
|
||||
className="w-10 h-8 p-0 border-0 bg-transparent cursor-pointer"
|
||||
/>
|
||||
</div>
|
||||
<p className="text-xs text-slate-500">{description}</p>
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>{description}</p>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -106,7 +108,8 @@ export function ColorOverridesPanel({ value, onChange, disabled }: ColorOverride
|
||||
<Button
|
||||
variant="ghost"
|
||||
type="button"
|
||||
className="w-full justify-between px-3 py-2 h-auto text-slate-300 hover:text-white hover:bg-slate-700/50"
|
||||
className="w-full justify-between px-3 py-2 h-auto"
|
||||
style={{ color: 'var(--text)' }}
|
||||
>
|
||||
<span className="flex items-center gap-2">
|
||||
<Paintbrush className="w-4 h-4" />
|
||||
@@ -126,9 +129,9 @@ export function ColorOverridesPanel({ value, onChange, disabled }: ColorOverride
|
||||
</CollapsibleTrigger>
|
||||
|
||||
<CollapsibleContent className="mt-2">
|
||||
<div className="space-y-4 p-4 rounded-lg border border-slate-700 bg-slate-800/50">
|
||||
<div className="space-y-4 p-4 rounded-lg border" style={{ borderColor: 'var(--card-border)', backgroundColor: 'var(--secondary)' }}>
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="text-sm text-slate-400">
|
||||
<p className="text-sm" style={{ color: 'var(--text-muted)' }}>
|
||||
Override theme colors (optional)
|
||||
</p>
|
||||
{hasAnyOverrides && (
|
||||
@@ -138,7 +141,7 @@ export function ColorOverridesPanel({ value, onChange, disabled }: ColorOverride
|
||||
size="sm"
|
||||
onClick={handleResetAll}
|
||||
disabled={disabled}
|
||||
className="text-slate-400 hover:text-white"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
>
|
||||
<RotateCcw className="w-3 h-3 mr-1" />
|
||||
Reset All
|
||||
|
||||
@@ -124,7 +124,8 @@ export function JobActions({ job, onRefresh, showLogs, onToggleLogs }: JobAction
|
||||
size="sm"
|
||||
variant="ghost"
|
||||
onClick={onToggleLogs}
|
||||
className="text-xs text-slate-400 hover:text-white"
|
||||
className="text-xs"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
>
|
||||
<FileText className="w-3 h-3 mr-1" />
|
||||
{showLogs ? "Hide" : "Show"} Logs
|
||||
|
||||
@@ -2,9 +2,8 @@
|
||||
|
||||
import { useState } from "react"
|
||||
import { Job } from "@/lib/api"
|
||||
import { useAutoMode } from "@/lib/auto-mode"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Loader2, ExternalLink, Zap } from "lucide-react"
|
||||
import { ExternalLink, Zap } from "lucide-react"
|
||||
import { JobActions } from "./JobActions"
|
||||
import { JobLogs } from "./JobLogs"
|
||||
import { OutputLinks } from "./OutputLinks"
|
||||
@@ -60,7 +59,7 @@ function ProgressBar({ job }: { job: Job }) {
|
||||
const barColor = barColorMap[color] || "bg-blue-400"
|
||||
|
||||
return (
|
||||
<div className="w-full h-1 bg-slate-700/50 rounded-full mt-2 overflow-hidden">
|
||||
<div className="w-full h-1 rounded-full mt-2 overflow-hidden" style={{ backgroundColor: 'var(--secondary)' }}>
|
||||
<div
|
||||
className={`h-full ${barColor} rounded-full transition-all duration-500 ease-out`}
|
||||
style={{ width: `${progressPercent}%` }}
|
||||
@@ -77,7 +76,6 @@ interface JobCardProps {
|
||||
export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
const [showLogs, setShowLogs] = useState(false)
|
||||
const [showAudioSearch, setShowAudioSearch] = useState(false)
|
||||
const { enabled: autoModeEnabled, isProcessing: isAutoProcessing } = useAutoMode()
|
||||
|
||||
const createdAt = new Date(job.created_at).toLocaleString()
|
||||
const isInteractive = job.status === "awaiting_review" ||
|
||||
@@ -86,14 +84,6 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
job.status === "awaiting_audio_selection"
|
||||
const isComplete = job.status === "complete"
|
||||
const isFailed = job.status === "failed"
|
||||
const isProcessing = !isComplete && !isFailed && job.status !== "cancelled"
|
||||
|
||||
// Check if this job is being auto-processed
|
||||
const isBeingAutoProcessed = autoModeEnabled && isInteractive && (
|
||||
isAutoProcessing(`${job.job_id}-review`) ||
|
||||
isAutoProcessing(`${job.job_id}-instrumental`) ||
|
||||
isAutoProcessing(`${job.job_id}-audio`)
|
||||
)
|
||||
|
||||
// Build review URL for awaiting_review status
|
||||
const getReviewUrl = () => {
|
||||
@@ -125,22 +115,14 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
|
||||
// Render primary action button based on status
|
||||
const renderPrimaryAction = () => {
|
||||
if (autoModeEnabled && isInteractive) {
|
||||
// For non-interactive jobs in interactive states, show auto-processing message
|
||||
if (job.non_interactive && isInteractive) {
|
||||
return (
|
||||
<div className="flex items-center gap-1 text-xs text-amber-400">
|
||||
{isBeingAutoProcessed ? (
|
||||
<>
|
||||
<Loader2 className="w-3 h-3 animate-spin" />
|
||||
Auto-processing...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<Zap className="w-3 h-3" />
|
||||
Will auto-{job.status === "awaiting_review" || job.status === "in_review" ? "accept" :
|
||||
job.status === "awaiting_instrumental_selection" ? "select clean" :
|
||||
"select first"}
|
||||
</>
|
||||
)}
|
||||
<Zap className="w-3 h-3" />
|
||||
Will auto-{job.status === "awaiting_review" || job.status === "in_review" ? "accept" :
|
||||
job.status === "awaiting_instrumental_selection" ? "select" :
|
||||
"select first"}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -190,18 +172,30 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
|
||||
return (
|
||||
<div
|
||||
className={`rounded-lg border p-3 transition-colors border-slate-700 bg-slate-800/30
|
||||
className={`rounded-lg border p-3 transition-colors
|
||||
${isComplete ? "border-green-500/30" : ""}
|
||||
${isFailed ? "border-red-500/30" : ""}`}
|
||||
style={{
|
||||
borderColor: isComplete || isFailed ? undefined : 'var(--card-border)',
|
||||
backgroundColor: 'var(--card)',
|
||||
}}
|
||||
>
|
||||
{/* Header row with title */}
|
||||
<p className="font-medium text-white truncate">
|
||||
{job.artist || "Unknown"} - {job.title || "Unknown"}
|
||||
</p>
|
||||
{/* Header row with title and non-interactive badge */}
|
||||
<div className="flex items-start justify-between gap-2">
|
||||
<p className="font-medium truncate" style={{ color: 'var(--text)' }}>
|
||||
{job.artist || "Unknown"} - {job.title || "Unknown"}
|
||||
</p>
|
||||
{job.non_interactive && (
|
||||
<span className="shrink-0 inline-flex items-center gap-1 text-[10px] px-1.5 py-0.5 rounded bg-amber-500/20 text-amber-400 font-medium">
|
||||
<Zap className="w-2.5 h-2.5" />
|
||||
Auto
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Meta row: ID, date, status */}
|
||||
<p className="text-xs text-slate-500 mt-1">
|
||||
<span className="text-slate-600">ID:</span> {job.job_id} <span className="text-slate-600">•</span> {createdAt} <span className="text-slate-600">•</span> <StatusIndicator job={job} />
|
||||
<p className="text-xs mt-1" style={{ color: 'var(--text-muted)' }}>
|
||||
<span style={{ opacity: 0.7 }}>ID:</span> {job.job_id} <span style={{ opacity: 0.7 }}>•</span> {createdAt} <span style={{ opacity: 0.7 }}>•</span> <StatusIndicator job={job} />
|
||||
</p>
|
||||
|
||||
{/* Progress bar for active jobs */}
|
||||
@@ -250,4 +244,3 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -34,7 +34,7 @@ export function JobLogs({ jobId, limit = 50 }: JobLogsProps) {
|
||||
|
||||
if (isLoading) {
|
||||
return (
|
||||
<div className="flex items-center justify-center py-4 text-slate-400">
|
||||
<div className="flex items-center justify-center py-4" style={{ color: 'var(--text-muted)' }}>
|
||||
<Loader2 className="w-4 h-4 animate-spin mr-2" />
|
||||
Loading logs...
|
||||
</div>
|
||||
@@ -52,14 +52,14 @@ export function JobLogs({ jobId, limit = 50 }: JobLogsProps) {
|
||||
|
||||
if (logs.length === 0) {
|
||||
return (
|
||||
<div className="text-center py-4 text-slate-500 text-sm">
|
||||
<div className="text-center py-4 text-sm" style={{ color: 'var(--text-muted)' }}>
|
||||
No logs available
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="bg-slate-900 rounded border border-slate-700 p-3 max-h-64 overflow-y-auto">
|
||||
<div className="rounded border p-3 max-h-64 overflow-y-auto" style={{ backgroundColor: 'var(--bg)', borderColor: 'var(--card-border)' }}>
|
||||
<div className="space-y-1 font-mono text-xs">
|
||||
{logs.map((log, index) => (
|
||||
<div
|
||||
@@ -68,10 +68,11 @@ export function JobLogs({ jobId, limit = 50 }: JobLogsProps) {
|
||||
log.level === "ERROR" ? "text-red-400" :
|
||||
log.level === "WARNING" ? "text-yellow-400" :
|
||||
log.level === "INFO" ? "text-blue-400" :
|
||||
"text-slate-400"
|
||||
""
|
||||
}`}
|
||||
style={log.level !== "ERROR" && log.level !== "WARNING" && log.level !== "INFO" ? { color: 'var(--text-muted)' } : undefined}
|
||||
>
|
||||
<span className="text-slate-600 shrink-0">
|
||||
<span className="shrink-0" style={{ color: 'var(--text-muted)', opacity: 0.7 }}>
|
||||
{new Date(log.timestamp).toLocaleTimeString()}
|
||||
</span>
|
||||
<span className="shrink-0 font-semibold w-12">
|
||||
|
||||
@@ -17,7 +17,7 @@ interface JobSubmissionProps {
|
||||
}
|
||||
|
||||
export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
const [activeTab, setActiveTab] = useState("upload")
|
||||
const [activeTab, setActiveTab] = useState("search")
|
||||
const [error, setError] = useState("")
|
||||
const [isSubmitting, setIsSubmitting] = useState(false)
|
||||
|
||||
@@ -39,6 +39,9 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
const [selectedTheme, setSelectedTheme] = useState<string | undefined>()
|
||||
const [colorOverrides, setColorOverrides] = useState<ColorOverrides>({})
|
||||
|
||||
// Non-interactive mode (shared across all tabs)
|
||||
const [nonInteractive, setNonInteractive] = useState(false)
|
||||
|
||||
async function handleUploadSubmit(e: React.FormEvent) {
|
||||
e.preventDefault()
|
||||
setError("")
|
||||
@@ -57,6 +60,7 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
await api.uploadJob(uploadFile, uploadArtist.trim(), uploadTitle.trim(), {
|
||||
theme_id: selectedTheme,
|
||||
color_overrides: cleanColorOverrides(colorOverrides),
|
||||
non_interactive: nonInteractive,
|
||||
})
|
||||
setUploadFile(null)
|
||||
setUploadArtist("")
|
||||
@@ -91,6 +95,7 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
{
|
||||
theme_id: selectedTheme,
|
||||
color_overrides: cleanColorOverrides(colorOverrides),
|
||||
non_interactive: nonInteractive,
|
||||
}
|
||||
)
|
||||
setYoutubeUrl("")
|
||||
@@ -122,6 +127,7 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
await api.searchAudio(searchArtist.trim(), searchTitle.trim(), false, {
|
||||
theme_id: selectedTheme,
|
||||
color_overrides: cleanColorOverrides(colorOverrides),
|
||||
non_interactive: nonInteractive,
|
||||
})
|
||||
setSearchArtist("")
|
||||
setSearchTitle("")
|
||||
@@ -147,28 +153,43 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
|
||||
return (
|
||||
<Tabs value={activeTab} onValueChange={setActiveTab}>
|
||||
<TabsList className="grid w-full grid-cols-3 bg-slate-800 h-auto">
|
||||
<TabsTrigger value="upload" className="gap-1.5 sm:gap-2 py-3 min-h-[44px] text-xs sm:text-sm data-[state=active]:bg-slate-700">
|
||||
<TabsList className="grid w-full grid-cols-3 h-auto rounded-lg p-1" style={{ backgroundColor: 'var(--secondary)' }}>
|
||||
<TabsTrigger
|
||||
value="search"
|
||||
className="gap-1.5 sm:gap-2 py-3 min-h-[44px] text-xs sm:text-sm rounded-md data-[state=active]:!bg-amber-600 data-[state=active]:!text-white data-[state=active]:shadow-md data-[state=inactive]:hover:bg-amber-600/10"
|
||||
style={{ color: activeTab === 'search' ? 'white' : 'var(--text-muted)' }}
|
||||
>
|
||||
<Music className="w-4 h-4 shrink-0" />
|
||||
<span className="truncate">Search</span>
|
||||
</TabsTrigger>
|
||||
<TabsTrigger
|
||||
value="upload"
|
||||
className="gap-1.5 sm:gap-2 py-3 min-h-[44px] text-xs sm:text-sm rounded-md data-[state=active]:!bg-amber-600 data-[state=active]:!text-white data-[state=active]:shadow-md data-[state=inactive]:hover:bg-amber-600/10"
|
||||
style={{ color: activeTab === 'upload' ? 'white' : 'var(--text-muted)' }}
|
||||
>
|
||||
<Upload className="w-4 h-4 shrink-0" />
|
||||
<span className="truncate">Upload</span>
|
||||
</TabsTrigger>
|
||||
<TabsTrigger value="url" className="gap-1.5 sm:gap-2 py-3 min-h-[44px] text-xs sm:text-sm data-[state=active]:bg-slate-700">
|
||||
<TabsTrigger
|
||||
value="url"
|
||||
className="gap-1.5 sm:gap-2 py-3 min-h-[44px] text-xs sm:text-sm rounded-md data-[state=active]:!bg-amber-600 data-[state=active]:!text-white data-[state=active]:shadow-md data-[state=inactive]:hover:bg-amber-600/10"
|
||||
style={{ color: activeTab === 'url' ? 'white' : 'var(--text-muted)' }}
|
||||
>
|
||||
<Youtube className="w-4 h-4 shrink-0" />
|
||||
<span className="truncate">URL</span>
|
||||
</TabsTrigger>
|
||||
<TabsTrigger value="search" className="gap-1.5 sm:gap-2 py-3 min-h-[44px] text-xs sm:text-sm data-[state=active]:bg-slate-700">
|
||||
<Music className="w-4 h-4 shrink-0" />
|
||||
<span className="truncate">Search</span>
|
||||
</TabsTrigger>
|
||||
</TabsList>
|
||||
|
||||
<TabsContent value="upload" className="mt-4">
|
||||
<form onSubmit={handleUploadSubmit} className="space-y-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="audio-file" className="text-slate-200">Audio File</Label>
|
||||
<Label htmlFor="audio-file" style={{ color: 'var(--text)' }}>Audio File</Label>
|
||||
<div
|
||||
className={`relative border-2 border-dashed rounded-lg p-6 text-center transition-colors cursor-pointer
|
||||
${uploadFile ? "border-amber-500/50 bg-amber-500/5" : "border-slate-700 hover:border-slate-600 bg-slate-800/50"}`}
|
||||
className="relative border-2 border-dashed rounded-lg p-6 text-center transition-colors cursor-pointer"
|
||||
style={{
|
||||
borderColor: uploadFile ? 'rgb(245 158 11 / 0.5)' : 'var(--card-border)',
|
||||
backgroundColor: uploadFile ? 'rgb(245 158 11 / 0.05)' : 'var(--secondary)',
|
||||
}}
|
||||
>
|
||||
<Input
|
||||
id="audio-file"
|
||||
@@ -178,16 +199,16 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
className="absolute inset-0 w-full h-full opacity-0 cursor-pointer"
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<Upload className="w-8 h-8 mx-auto mb-2 text-slate-500" />
|
||||
<Upload className="w-8 h-8 mx-auto mb-2" style={{ color: 'var(--text-muted)' }} />
|
||||
{uploadFile ? (
|
||||
<div>
|
||||
<p className="font-medium text-white">{uploadFile.name}</p>
|
||||
<p className="text-sm text-slate-400">{(uploadFile.size / 1024 / 1024).toFixed(2)} MB</p>
|
||||
<p className="font-medium" style={{ color: 'var(--text)' }}>{uploadFile.name}</p>
|
||||
<p className="text-sm" style={{ color: 'var(--text-muted)' }}>{(uploadFile.size / 1024 / 1024).toFixed(2)} MB</p>
|
||||
</div>
|
||||
) : (
|
||||
<div>
|
||||
<p className="font-medium text-slate-300 mb-1">Click to upload</p>
|
||||
<p className="text-sm text-slate-500">MP3, WAV, FLAC, M4A, or OGG</p>
|
||||
<p className="font-medium mb-1" style={{ color: 'var(--text)' }}>Click to upload</p>
|
||||
<p className="text-sm" style={{ color: 'var(--text-muted)' }}>MP3, WAV, FLAC, M4A, or OGG</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
@@ -195,31 +216,34 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="upload-artist" className="text-slate-200">Artist</Label>
|
||||
<Label htmlFor="upload-artist" style={{ color: 'var(--text)' }}>Artist</Label>
|
||||
<Input
|
||||
id="upload-artist"
|
||||
placeholder="Artist name"
|
||||
value={uploadArtist}
|
||||
onChange={(e) => setUploadArtist(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
className="bg-slate-800 border-slate-700 text-white placeholder:text-slate-500"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="upload-title" className="text-slate-200">Title</Label>
|
||||
<Label htmlFor="upload-title" style={{ color: 'var(--text)' }}>Title</Label>
|
||||
<Input
|
||||
id="upload-title"
|
||||
placeholder="Song title"
|
||||
value={uploadTitle}
|
||||
onChange={(e) => setUploadTitle(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
className="bg-slate-800 border-slate-700 text-white placeholder:text-slate-500"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>
|
||||
<span className="text-amber-500 font-medium">Note:</span> Format these exactly as you want them on the title card and video filename.
|
||||
</p>
|
||||
|
||||
{/* Theme Selection */}
|
||||
<div className="space-y-4 pt-4 border-t border-slate-700">
|
||||
<div className="space-y-4 pt-4 border-t" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<ThemeSelector
|
||||
value={selectedTheme}
|
||||
onChange={setSelectedTheme}
|
||||
@@ -232,6 +256,29 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Non-interactive mode */}
|
||||
<div className="space-y-2 pt-4 border-t" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<div className="flex items-start gap-3">
|
||||
<input
|
||||
type="checkbox"
|
||||
id="non-interactive-upload"
|
||||
checked={nonInteractive}
|
||||
onChange={(e) => setNonInteractive(e.target.checked)}
|
||||
disabled={isSubmitting}
|
||||
className="mt-1 w-4 h-4 rounded border-gray-600 bg-gray-700 text-amber-500 focus:ring-amber-500 focus:ring-offset-gray-800"
|
||||
/>
|
||||
<div>
|
||||
<Label htmlFor="non-interactive-upload" className="cursor-pointer" style={{ color: 'var(--text)' }}>
|
||||
Skip lyrics review (non-interactive)
|
||||
</Label>
|
||||
<p className="text-xs mt-1" style={{ color: 'var(--text-muted)' }}>
|
||||
<span className="text-amber-500 font-medium">Warning:</span> AI lyrics correction isn't perfect.
|
||||
Skipping review will likely result in timing errors and incorrect words in your karaoke video.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{error && activeTab === "upload" && (
|
||||
<p className="text-sm text-red-400 bg-red-500/10 rounded p-2">{error}</p>
|
||||
)}
|
||||
@@ -256,16 +303,17 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
<TabsContent value="url" className="mt-4">
|
||||
<form onSubmit={handleUrlSubmit} className="space-y-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="youtube-url" className="text-slate-200">YouTube URL</Label>
|
||||
<Label htmlFor="youtube-url" style={{ color: 'var(--text)' }}>YouTube URL</Label>
|
||||
<div className="relative">
|
||||
<Youtube className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-slate-500" />
|
||||
<Youtube className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4" style={{ color: 'var(--text-muted)' }} />
|
||||
<Input
|
||||
id="youtube-url"
|
||||
type="url"
|
||||
placeholder="https://youtube.com/watch?v=..."
|
||||
value={youtubeUrl}
|
||||
onChange={(e) => setYoutubeUrl(e.target.value)}
|
||||
className="pl-10 bg-slate-800 border-slate-700 text-white placeholder:text-slate-500"
|
||||
className="pl-10"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</div>
|
||||
@@ -273,31 +321,34 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="youtube-artist" className="text-slate-200">Artist (optional)</Label>
|
||||
<Label htmlFor="youtube-artist" style={{ color: 'var(--text)' }}>Artist (optional)</Label>
|
||||
<Input
|
||||
id="youtube-artist"
|
||||
placeholder="Auto-detected"
|
||||
value={youtubeArtist}
|
||||
onChange={(e) => setYoutubeArtist(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
className="bg-slate-800 border-slate-700 text-white placeholder:text-slate-500"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="youtube-title" className="text-slate-200">Title (optional)</Label>
|
||||
<Label htmlFor="youtube-title" style={{ color: 'var(--text)' }}>Title (optional)</Label>
|
||||
<Input
|
||||
id="youtube-title"
|
||||
placeholder="Auto-detected"
|
||||
value={youtubeTitle}
|
||||
onChange={(e) => setYoutubeTitle(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
className="bg-slate-800 border-slate-700 text-white placeholder:text-slate-500"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>
|
||||
<span className="text-amber-500 font-medium">Note:</span> Override auto-detected values if needed. These appear on the title card and video filename.
|
||||
</p>
|
||||
|
||||
{/* Theme Selection */}
|
||||
<div className="space-y-4 pt-4 border-t border-slate-700">
|
||||
<div className="space-y-4 pt-4 border-t" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<ThemeSelector
|
||||
value={selectedTheme}
|
||||
onChange={setSelectedTheme}
|
||||
@@ -310,6 +361,29 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Non-interactive mode */}
|
||||
<div className="space-y-2 pt-4 border-t" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<div className="flex items-start gap-3">
|
||||
<input
|
||||
type="checkbox"
|
||||
id="non-interactive-url"
|
||||
checked={nonInteractive}
|
||||
onChange={(e) => setNonInteractive(e.target.checked)}
|
||||
disabled={isSubmitting}
|
||||
className="mt-1 w-4 h-4 rounded border-gray-600 bg-gray-700 text-amber-500 focus:ring-amber-500 focus:ring-offset-gray-800"
|
||||
/>
|
||||
<div>
|
||||
<Label htmlFor="non-interactive-url" className="cursor-pointer" style={{ color: 'var(--text)' }}>
|
||||
Skip lyrics review (non-interactive)
|
||||
</Label>
|
||||
<p className="text-xs mt-1" style={{ color: 'var(--text-muted)' }}>
|
||||
<span className="text-amber-500 font-medium">Warning:</span> AI lyrics correction isn't perfect.
|
||||
Skipping review will likely result in timing errors and incorrect words in your karaoke video.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{error && activeTab === "url" && (
|
||||
<p className="text-sm text-red-400 bg-red-500/10 rounded p-2">{error}</p>
|
||||
)}
|
||||
@@ -335,31 +409,34 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
<form onSubmit={handleSearchSubmit} className="space-y-4">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 gap-3 sm:gap-4">
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="search-artist" className="text-slate-200">Artist</Label>
|
||||
<Label htmlFor="search-artist" style={{ color: 'var(--text)' }}>Artist</Label>
|
||||
<Input
|
||||
id="search-artist"
|
||||
placeholder="Artist name"
|
||||
value={searchArtist}
|
||||
onChange={(e) => setSearchArtist(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
className="bg-slate-800 border-slate-700 text-white placeholder:text-slate-500"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="search-title" className="text-slate-200">Title</Label>
|
||||
<Label htmlFor="search-title" style={{ color: 'var(--text)' }}>Title</Label>
|
||||
<Input
|
||||
id="search-title"
|
||||
placeholder="Song title"
|
||||
value={searchTitle}
|
||||
onChange={(e) => setSearchTitle(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
className="bg-slate-800 border-slate-700 text-white placeholder:text-slate-500"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>
|
||||
<span className="text-amber-500 font-medium">Note:</span> Format these exactly as you want them on the title card and video filename.
|
||||
</p>
|
||||
|
||||
{/* Theme Selection */}
|
||||
<div className="space-y-4 pt-4 border-t border-slate-700">
|
||||
<div className="space-y-4 pt-4 border-t" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<ThemeSelector
|
||||
value={selectedTheme}
|
||||
onChange={setSelectedTheme}
|
||||
@@ -372,6 +449,29 @@ export function JobSubmission({ onJobCreated }: JobSubmissionProps) {
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Non-interactive mode */}
|
||||
<div className="space-y-2 pt-4 border-t" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<div className="flex items-start gap-3">
|
||||
<input
|
||||
type="checkbox"
|
||||
id="non-interactive-search"
|
||||
checked={nonInteractive}
|
||||
onChange={(e) => setNonInteractive(e.target.checked)}
|
||||
disabled={isSubmitting}
|
||||
className="mt-1 w-4 h-4 rounded border-gray-600 bg-gray-700 text-amber-500 focus:ring-amber-500 focus:ring-offset-gray-800"
|
||||
/>
|
||||
<div>
|
||||
<Label htmlFor="non-interactive-search" className="cursor-pointer" style={{ color: 'var(--text)' }}>
|
||||
Skip lyrics review (non-interactive)
|
||||
</Label>
|
||||
<p className="text-xs mt-1" style={{ color: 'var(--text-muted)' }}>
|
||||
<span className="text-amber-500 font-medium">Warning:</span> AI lyrics correction isn't perfect.
|
||||
Skipping review will likely result in timing errors and incorrect words in your karaoke video.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{error && activeTab === "search" && (
|
||||
<p className="text-sm text-red-400 bg-red-500/10 rounded p-2">{error}</p>
|
||||
)}
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import { useEffect, useState, useRef, useCallback } from "react"
|
||||
import { api } from "@/lib/api"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Download, Loader2, ExternalLink, FolderOpen } from "lucide-react"
|
||||
import { Download, Loader2, ExternalLink, FolderOpen, Copy } from "lucide-react"
|
||||
|
||||
interface OutputLinksProps {
|
||||
jobId: string
|
||||
@@ -14,24 +14,35 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
const [youtubeUrl, setYoutubeUrl] = useState<string | null>(null)
|
||||
const [dropboxUrl, setDropboxUrl] = useState<string | null>(null)
|
||||
const [isLoading, setIsLoading] = useState(false)
|
||||
const [copiedUrl, setCopiedUrl] = useState<string | null>(null)
|
||||
const copyTimeoutRef = useRef<NodeJS.Timeout | null>(null)
|
||||
|
||||
useEffect(() => {
|
||||
loadOutputLinks()
|
||||
}, [jobId])
|
||||
|
||||
// Cleanup timeout on unmount
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (copyTimeoutRef.current) {
|
||||
clearTimeout(copyTimeoutRef.current)
|
||||
}
|
||||
}
|
||||
}, [])
|
||||
|
||||
async function loadOutputLinks() {
|
||||
setIsLoading(true)
|
||||
try {
|
||||
const result = await api.getDownloadUrls(jobId)
|
||||
setDownloadUrls(result.download_urls)
|
||||
|
||||
|
||||
// Extract YouTube and Dropbox URLs from state_data if available
|
||||
const job = await api.getJob(jobId)
|
||||
if (job.state_data?.youtube_url) {
|
||||
setYoutubeUrl(job.state_data.youtube_url)
|
||||
}
|
||||
if (job.state_data?.dropbox_folder_url) {
|
||||
setDropboxUrl(job.state_data.dropbox_folder_url)
|
||||
if (job.state_data?.dropbox_link) {
|
||||
setDropboxUrl(job.state_data.dropbox_link)
|
||||
}
|
||||
} catch (err) {
|
||||
console.error("Failed to load output links:", err)
|
||||
@@ -40,58 +51,100 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
}
|
||||
}
|
||||
|
||||
const copyToClipboard = useCallback(async (url: string) => {
|
||||
// Check if clipboard API is available
|
||||
if (!navigator?.clipboard?.writeText) {
|
||||
console.error("Clipboard API not available")
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
await navigator.clipboard.writeText(url)
|
||||
setCopiedUrl(url)
|
||||
|
||||
// Clear any existing timeout
|
||||
if (copyTimeoutRef.current) {
|
||||
clearTimeout(copyTimeoutRef.current)
|
||||
}
|
||||
|
||||
// Set new timeout
|
||||
copyTimeoutRef.current = setTimeout(() => {
|
||||
setCopiedUrl(null)
|
||||
}, 2000)
|
||||
} catch (err) {
|
||||
console.error("Failed to copy to clipboard:", err)
|
||||
}
|
||||
}, [])
|
||||
|
||||
if (isLoading) {
|
||||
return (
|
||||
<div className="flex items-center gap-2 text-xs text-slate-400">
|
||||
<div className="flex items-center gap-2 text-xs" style={{ color: 'var(--text-muted)' }}>
|
||||
<Loader2 className="w-3 h-3 animate-spin" />
|
||||
Loading downloads...
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
const hasOutputs = youtubeUrl || dropboxUrl || (downloadUrls && Object.keys(downloadUrls).length > 0)
|
||||
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
{/* Distribution Links */}
|
||||
{(youtubeUrl || dropboxUrl) && (
|
||||
{/* Combined Output Links */}
|
||||
{hasOutputs && (
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs text-slate-400 font-medium">Distribution:</p>
|
||||
<p className="text-xs font-medium" style={{ color: 'var(--text-muted)' }}>Outputs:</p>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{/* Distribution Links */}
|
||||
{youtubeUrl && (
|
||||
<a
|
||||
href={youtubeUrl}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-red-600 hover:bg-red-500 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<ExternalLink className="w-3 h-3" />
|
||||
View on YouTube
|
||||
</a>
|
||||
<div className="flex gap-1">
|
||||
<a
|
||||
href={youtubeUrl}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded-l bg-red-600 hover:bg-red-500 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<ExternalLink className="w-3 h-3" />
|
||||
View on YouTube
|
||||
</a>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => { e.stopPropagation(); copyToClipboard(youtubeUrl) }}
|
||||
className="inline-flex items-center text-xs px-2 py-1.5 rounded-r bg-red-700 hover:bg-red-600 text-white"
|
||||
title="Copy YouTube URL"
|
||||
aria-label={copiedUrl === youtubeUrl ? "YouTube URL copied" : "Copy YouTube URL"}
|
||||
>
|
||||
{copiedUrl === youtubeUrl ? <span className="text-[10px]">Copied!</span> : <Copy className="w-3 h-3" />}
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
{dropboxUrl && (
|
||||
<a
|
||||
href={dropboxUrl}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-blue-600 hover:bg-blue-500 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<FolderOpen className="w-3 h-3" />
|
||||
Open Dropbox Folder
|
||||
</a>
|
||||
<div className="flex gap-1">
|
||||
<a
|
||||
href={dropboxUrl}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded-l bg-blue-600 hover:bg-blue-500 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<FolderOpen className="w-3 h-3" />
|
||||
Dropbox Folder
|
||||
</a>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => { e.stopPropagation(); copyToClipboard(dropboxUrl) }}
|
||||
className="inline-flex items-center text-xs px-2 py-1.5 rounded-r bg-blue-700 hover:bg-blue-600 text-white"
|
||||
title="Copy Dropbox URL"
|
||||
aria-label={copiedUrl === dropboxUrl ? "Dropbox URL copied" : "Copy Dropbox URL"}
|
||||
>
|
||||
{copiedUrl === dropboxUrl ? <span className="text-[10px]">Copied!</span> : <Copy className="w-3 h-3" />}
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Download Links */}
|
||||
{downloadUrls && Object.keys(downloadUrls).length > 0 && (
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs text-slate-400 font-medium">Downloads:</p>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{downloadUrls.finals?.lossy_720p && (
|
||||
{/* Download Links */}
|
||||
{downloadUrls?.finals?.lossy_720p_mp4 && (
|
||||
<a
|
||||
href={api.getDownloadUrl(jobId, "finals", "lossy_720p")}
|
||||
href={api.getDownloadUrl(jobId, "finals", "lossy_720p_mp4")}
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-green-600 hover:bg-green-500 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
@@ -99,7 +152,7 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
720p Video
|
||||
</a>
|
||||
)}
|
||||
{downloadUrls.finals?.lossy_4k_mp4 && (
|
||||
{downloadUrls?.finals?.lossy_4k_mp4 && (
|
||||
<a
|
||||
href={api.getDownloadUrl(jobId, "finals", "lossy_4k_mp4")}
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-green-600 hover:bg-green-500 text-white"
|
||||
@@ -109,20 +162,30 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
4K Video
|
||||
</a>
|
||||
)}
|
||||
{downloadUrls.packages?.cdg_zip && (
|
||||
{downloadUrls?.videos?.with_vocals && (
|
||||
<a
|
||||
href={api.getDownloadUrl(jobId, "videos", "with_vocals")}
|
||||
className="inline-flex items-center gap-1 text-xs px-3 py-1.5 rounded bg-purple-600 hover:bg-purple-500 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
With Vocals
|
||||
</a>
|
||||
)}
|
||||
{downloadUrls?.packages?.cdg_zip && (
|
||||
<a
|
||||
href={api.getDownloadUrl(jobId, "packages", "cdg_zip")}
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-slate-600 hover:bg-slate-500 text-white"
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-gray-700 hover:bg-gray-600 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
CDG+MP3
|
||||
</a>
|
||||
)}
|
||||
{downloadUrls.packages?.txt_zip && (
|
||||
{downloadUrls?.packages?.txt_zip && (
|
||||
<a
|
||||
href={api.getDownloadUrl(jobId, "packages", "txt_zip")}
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-slate-600 hover:bg-slate-500 text-white"
|
||||
className="inline-flex items-center gap-1 text-xs px-2 py-1.5 rounded bg-gray-700 hover:bg-gray-600 text-white"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
@@ -133,10 +196,9 @@ export function OutputLinks({ jobId }: OutputLinksProps) {
|
||||
</div>
|
||||
)}
|
||||
|
||||
{!downloadUrls && !youtubeUrl && !dropboxUrl && (
|
||||
<p className="text-xs text-slate-500">No outputs available yet</p>
|
||||
{!hasOutputs && (
|
||||
<p className="text-xs" style={{ color: 'var(--text-muted)' }}>No outputs available yet</p>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -6,7 +6,8 @@ import type { VideoThemeSummary } from "@/lib/video-themes"
|
||||
import { Label } from "@/components/ui/label"
|
||||
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select"
|
||||
import { Skeleton } from "@/components/ui/skeleton"
|
||||
import { Palette, AlertCircle } from "lucide-react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Palette, AlertCircle, Eye, EyeOff } from "lucide-react"
|
||||
|
||||
interface ThemeSelectorProps {
|
||||
value?: string
|
||||
@@ -18,6 +19,7 @@ export function ThemeSelector({ value, onChange, disabled }: ThemeSelectorProps)
|
||||
const [themes, setThemes] = useState<VideoThemeSummary[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const [showPreview, setShowPreview] = useState(false)
|
||||
|
||||
useEffect(() => {
|
||||
async function loadThemes() {
|
||||
@@ -52,11 +54,11 @@ export function ThemeSelector({ value, onChange, disabled }: ThemeSelectorProps)
|
||||
if (loading) {
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
<Label className="text-slate-200 flex items-center gap-2">
|
||||
<Label className="flex items-center gap-2" style={{ color: 'var(--text)' }}>
|
||||
<Palette className="w-4 h-4" />
|
||||
Video Theme
|
||||
</Label>
|
||||
<Skeleton className="h-10 w-full bg-slate-700" />
|
||||
<Skeleton className="h-10 w-full" style={{ backgroundColor: 'var(--secondary)' }} />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -64,7 +66,7 @@ export function ThemeSelector({ value, onChange, disabled }: ThemeSelectorProps)
|
||||
if (error || themes.length === 0) {
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
<Label className="text-slate-200 flex items-center gap-2">
|
||||
<Label className="flex items-center gap-2" style={{ color: 'var(--text)' }}>
|
||||
<Palette className="w-4 h-4" />
|
||||
Video Theme
|
||||
</Label>
|
||||
@@ -78,7 +80,7 @@ export function ThemeSelector({ value, onChange, disabled }: ThemeSelectorProps)
|
||||
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
<Label htmlFor="theme-select" className="text-slate-200 flex items-center gap-2">
|
||||
<Label htmlFor="theme-select" className="flex items-center gap-2" style={{ color: 'var(--text)' }}>
|
||||
<Palette className="w-4 h-4" />
|
||||
Video Theme
|
||||
</Label>
|
||||
@@ -90,16 +92,16 @@ export function ThemeSelector({ value, onChange, disabled }: ThemeSelectorProps)
|
||||
>
|
||||
<SelectTrigger
|
||||
id="theme-select"
|
||||
className="bg-slate-800 border-slate-700 text-white"
|
||||
style={{ backgroundColor: 'var(--secondary)', borderColor: 'var(--card-border)', color: 'var(--text)' }}
|
||||
>
|
||||
<SelectValue placeholder="Select a theme..." />
|
||||
</SelectTrigger>
|
||||
<SelectContent className="bg-slate-800 border-slate-700">
|
||||
<SelectContent style={{ backgroundColor: 'var(--card)', borderColor: 'var(--card-border)' }}>
|
||||
{themes.map((theme) => (
|
||||
<SelectItem
|
||||
key={theme.id}
|
||||
value={theme.id}
|
||||
className="text-white hover:bg-slate-700 focus:bg-slate-700"
|
||||
style={{ color: 'var(--text)' }}
|
||||
>
|
||||
<div className="flex items-center gap-2">
|
||||
<span>{theme.name}</span>
|
||||
@@ -112,13 +114,26 @@ export function ThemeSelector({ value, onChange, disabled }: ThemeSelectorProps)
|
||||
</SelectContent>
|
||||
</Select>
|
||||
|
||||
{/* Theme description and preview */}
|
||||
{selectedTheme && (
|
||||
{/* Theme preview toggle */}
|
||||
{selectedTheme?.thumbnail_url && (
|
||||
<div className="mt-2 space-y-2">
|
||||
<p className="text-sm text-slate-400">{selectedTheme.description}</p>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={() => setShowPreview(!showPreview)}
|
||||
className="px-0"
|
||||
style={{ color: 'var(--text-muted)' }}
|
||||
>
|
||||
{showPreview ? (
|
||||
<><EyeOff className="w-4 h-4 mr-1.5" />Hide preview</>
|
||||
) : (
|
||||
<><Eye className="w-4 h-4 mr-1.5" />Show preview</>
|
||||
)}
|
||||
</Button>
|
||||
|
||||
{selectedTheme.thumbnail_url && (
|
||||
<div className="relative w-full aspect-video max-w-xs rounded-lg overflow-hidden border border-slate-700">
|
||||
{showPreview && (
|
||||
<div className="relative w-full aspect-video max-w-xs rounded-lg overflow-hidden border" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<img
|
||||
src={selectedTheme.thumbnail_url}
|
||||
alt={`${selectedTheme.name} preview`}
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect, useState } from "react";
|
||||
import { api } from "@/lib/api";
|
||||
|
||||
export function VersionFooter() {
|
||||
const [backendVersion, setBackendVersion] = useState<string | null>(null);
|
||||
// Frontend version is injected at build time from pyproject.toml
|
||||
const frontendVersion = process.env.NEXT_PUBLIC_APP_VERSION || "dev";
|
||||
|
||||
useEffect(() => {
|
||||
const fetchBackendVersion = async () => {
|
||||
try {
|
||||
const info = await api.getBackendInfo();
|
||||
setBackendVersion(info.version);
|
||||
} catch (error) {
|
||||
console.error("Failed to fetch backend version:", error);
|
||||
setBackendVersion(null);
|
||||
}
|
||||
};
|
||||
|
||||
fetchBackendVersion();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<footer className="border-t mt-12 py-6" style={{ borderColor: 'var(--card-border)' }}>
|
||||
<div className="px-4 text-center text-sm" style={{ color: 'var(--text-muted)' }}>
|
||||
<p>
|
||||
Powered by{" "}
|
||||
<a href="https://github.com/nomadkaraoke/karaoke-gen" className="text-amber-500 hover:underline">
|
||||
karaoke-gen
|
||||
</a>
|
||||
</p>
|
||||
<p className="mt-1 text-xs opacity-75">
|
||||
Frontend: v{frontendVersion}
|
||||
{backendVersion && (
|
||||
<>
|
||||
{" | "}
|
||||
Backend: v{backendVersion}
|
||||
</>
|
||||
)}
|
||||
</p>
|
||||
</div>
|
||||
</footer>
|
||||
);
|
||||
}
|
||||
@@ -1,118 +0,0 @@
|
||||
/**
|
||||
* Example E2E test demonstrating the API fixture system
|
||||
*
|
||||
* This test shows how to use the fixture system to run tests against
|
||||
* mock data instead of a real backend.
|
||||
*
|
||||
* Run in recording mode to capture new fixtures:
|
||||
* RECORD_FIXTURES=true npm run test:e2e -- fixtures-example.spec.ts
|
||||
*
|
||||
* Then review and approve fixtures:
|
||||
* npm run fixtures:review
|
||||
*/
|
||||
|
||||
import { test, expect } from '@playwright/test';
|
||||
import { setupApiFixtures, setupMocks, setAuthToken } from './fixtures';
|
||||
|
||||
test.describe('API Fixtures Example', () => {
|
||||
test('shows empty state when no jobs exist', async ({ page }) => {
|
||||
// Set up mock API responses (frontend uses /api/jobs, not /api/v1/jobs)
|
||||
await setupMocks(page, [
|
||||
{
|
||||
method: 'GET',
|
||||
path: '/api/jobs',
|
||||
response: { body: [] },
|
||||
},
|
||||
]);
|
||||
|
||||
// Set auth token before navigating
|
||||
await setAuthToken(page, 'test-token');
|
||||
|
||||
await page.goto('/');
|
||||
|
||||
// Verify the empty state message
|
||||
await expect(page.getByText(/No jobs yet/i)).toBeVisible();
|
||||
});
|
||||
|
||||
test('shows jobs when they exist', async ({ page }) => {
|
||||
// Set up mock with sample job data (frontend uses /api/jobs)
|
||||
await setupMocks(page, [
|
||||
{
|
||||
method: 'GET',
|
||||
path: '/api/jobs',
|
||||
response: {
|
||||
body: [
|
||||
{
|
||||
job_id: 'job-123',
|
||||
title: 'Test Song',
|
||||
artist: 'Test Artist',
|
||||
status: 'completed',
|
||||
created_at: '2024-01-15T10:00:00Z',
|
||||
updated_at: '2024-01-15T10:05:00Z',
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
await setAuthToken(page, 'test-token');
|
||||
await page.goto('/');
|
||||
|
||||
// Verify job is displayed
|
||||
await expect(page.getByText('Test Song')).toBeVisible();
|
||||
await expect(page.getByText('Test Artist')).toBeVisible();
|
||||
});
|
||||
|
||||
test.skip('shows error when API fails', async ({ page }) => {
|
||||
// NOTE: Currently the frontend doesn't display API errors visibly
|
||||
// This test is skipped until error handling UI is added
|
||||
await setupMocks(page, [
|
||||
{
|
||||
method: 'GET',
|
||||
path: '/api/jobs',
|
||||
response: {
|
||||
status: 500,
|
||||
body: { detail: 'Internal server error' },
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
await setAuthToken(page, 'test-token');
|
||||
await page.goto('/');
|
||||
|
||||
// This would require the frontend to display API errors
|
||||
await expect(page.getByText(/error/i)).toBeVisible();
|
||||
});
|
||||
|
||||
test('shows login prompt when not authenticated', async ({ page }) => {
|
||||
// No mock needed - just don't set auth token
|
||||
await page.goto('/');
|
||||
|
||||
// Verify login prompt is shown (actual UI text)
|
||||
await expect(page.getByText(/Authentication Required/i)).toBeVisible();
|
||||
});
|
||||
|
||||
test.skip('records real API responses when RECORD_FIXTURES=true', async ({ page }) => {
|
||||
// This test only runs in recording mode
|
||||
// It will capture actual API responses for review
|
||||
|
||||
const api = await setupApiFixtures(page, {
|
||||
testFile: 'fixtures-example.spec.ts',
|
||||
});
|
||||
|
||||
if (!api.isRecording) {
|
||||
test.skip();
|
||||
return;
|
||||
}
|
||||
|
||||
await setAuthToken(page, process.env.TEST_ACCESS_TOKEN || 'your-token');
|
||||
await page.goto('/');
|
||||
|
||||
// Perform actions that trigger API calls
|
||||
await page.waitForLoadState('networkidle');
|
||||
|
||||
// Stop recording and show captured fixtures
|
||||
const fixtures = api.stopRecording();
|
||||
console.log(`Captured ${fixtures.length} API calls`);
|
||||
});
|
||||
});
|
||||
Loaded 100 of 185 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user