Compare commits

..
Author SHA1 Message Date
Andrew BeveridgeandClaude Opus 4.5 5c71d58a8d Fix progress display to show step 4 when workers are active during download
When the backend status is "downloading" but parallel workers (audio/lyrics)
are actively running, the frontend now correctly displays step 4 progress
instead of appearing stuck at step 3 "Downloading".

This improves UX by showing detailed worker progress (e.g., "Audio 1/2 + Transcribing")
when both workers are active, making the job appear responsive.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 22:26:26 -05:00
Andrew BeveridgeandClaude Opus 4.5 78d7eedeee Add tests for YouTube URL support
Add comprehensive test coverage for the new download_from_url()
functionality:

- FlacFetchAudioFetcher.download_from_url() tests:
  - YouTube URL download success
  - youtu.be short URL support
  - Embed URL support
  - Filename generation from artist/title
  - Filename fallback to video ID
  - Error handling for download failures
  - Output directory creation

- RemoteFlacFetchAudioFetcher.download_from_url() tests:
  - Verifies local flacfetch is used (not remote API)
  - Video ID extraction
  - Error handling

- KaraokePrep URL handling tests:
  - process() with YouTube URL
  - process() with URL but no artist/title
  - process() with youtu.be short URL
  - _is_url() helper method edge cases

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 22:18:17 -05:00
Andrew Beveridge 6fa81c1b69 Updated dependencies 2025-12-27 22:10:59 -05:00
Andrew BeveridgeandClaude Opus 4.5 706e6f7527 Fix YouTube URL support in karaoke-gen CLI
Restore YouTube URL input functionality that was broken during
flacfetch integration. Now users can pass a YouTube URL as the
first argument to karaoke-gen again.

Changes:
- Add download_from_url() abstract method to AudioFetcher base class
- Implement download_from_url() in FlacFetchAudioFetcher using
  flacfetch's download_by_id() method for direct YouTube downloads
- Implement download_from_url() in RemoteFlacFetchAudioFetcher
  using local flacfetch for YouTube (no remote API needed)
- Update KaraokePrep.process() to detect URLs and route them to
  the new download_from_url() method instead of flacfetch search
- Add _is_url() helper method for URL detection
- Add _use_url_download flag to differentiate URL downloads from
  artist+title searches

Fixes #67

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 22:08:19 -05:00
Andrew Beveridge ☄️ 8bbb1351f8 Merge pull request #73 from nomadkaraoke/feature/ci-improvements
ci: Add path filtering, caching, and frontend tests to GHA workflow
2025-12-27 21:50:35 -05:00
Andrew BeveridgeandClaude Opus 4.5 18e97dc0a0 ci: Use poetry.lock exclusively for deps filter
Remove pyproject.toml from the deps filter since poetry.lock is
already used as the source of truth for dependency changes elsewhere
(e.g., base image rebuild triggers). Version bumps alone shouldn't
trigger full test suites.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 21:48:50 -05:00
Andrew BeveridgeandClaude Opus 4.5 642c80164c ci: Add path filtering, caching, and frontend tests to GHA workflow
- Add `changes` job using dorny/paths-filter@v3 to detect what changed
- Add frontend-unit-tests job (Jest with coverage)
- Make all test jobs conditional based on path changes:
  - Frontend tests: run only if frontend/** changed
  - Backend tests: run if backend/** or package or deps changed
  - Package tests: run if karaoke_gen/** or tests/** or deps changed
- Add Poetry dependency caching using actions/cache@v4:
  - Cache .venv directory keyed on poetry.lock hash
  - Shared cache between package, backend, and lint jobs
- Add npm caching using actions/setup-node's built-in cache feature
- Add Yarn caching for lyrics frontend in deploy-frontend job
- Add spaCy model caching
- Jobs always run on main branch to ensure full validation before deploy
- Add summary table to changes job showing what areas changed

This significantly reduces CI time for focused PRs (e.g., frontend-only
changes skip all backend tests) while ensuring comprehensive testing
before merging to main.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 21:40:56 -05:00
Andrew Beveridge ☄️andClaude Opus 4.5 46eca212fe Fix: Prepare theme styles when creating job via audio search (#72)
* Fix: Prepare theme styles when creating job via audio search

When creating a job via the audio search endpoint with a theme_id,
the theme styles were not being prepared. This resulted in jobs that
had theme_id set but no style_params_gcs_path or style_assets, causing
preview videos to have black backgrounds instead of themed backgrounds.

The fix adds a call to _prepare_theme_for_job() in audio_search.py
when theme_id is provided and no custom style files are being uploaded.
This matches the behavior of file_upload.py.

Also adds regression tests to catch this issue:
- test_audio_search_with_theme_calls_prepare_job_style
- test_audio_search_code_handles_theme_preparation

Fixes issue observed in production job ffb0b8fa.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove unused variable from test (CodeRabbit feedback)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 21:38:05 -05:00
Andrew BeveridgeandClaude Opus 4.5 8d8e64ccc0 docs: Add job debugging guide with tested gcloud commands
Add comprehensive documentation for debugging jobs in production:
- Firestore job data access via Python script
- Cloud Logging queries for job-specific logs
- GCS file listing and inspection
- Cloud Run service health checks
- Cloud Tasks queue monitoring
- Common debugging scenarios

Also adds scripts/get_job.py helper for quick job lookups.

All commands tested against real job ID ffb0b8fa.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 21:26:10 -05:00
Andrew Beveridge ☄️ 50f895e88a Merge pull request #71 from nomadkaraoke/feature/improve-job-status-display
feat: Improve job status display with step-based progress
2025-12-27 21:24:44 -05:00
Andrew BeveridgeandClaude Opus 4.5 7218b4d176 feat: Add step-based job status display with progress indicator
- Create job-status.ts utility that maps 28+ backend statuses to a 10-step user-friendly system
- Display status as "[X/10] Status..." format for better progress visibility
- Add visual progress bar that matches status color
- Show "Action needed" badge for blocking states (review, instrumental selection)
- Handle parallel processing (audio + lyrics) with combined status display
- Add comprehensive unit tests (40+ test cases) for all status mappings
- Update JobCard to use new StatusIndicator and ProgressBar components

This improves user experience by providing clearer feedback about job progress
instead of showing static labels like "Downloading..." for extended periods.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 21:17:44 -05:00
Andrew BeveridgeandClaude Opus 4.5 c052fec326 Fix E2E test API request timeouts
Increase timeout for Playwright API requests from default 60s to 120s
to handle slower API responses during job polling.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 20:54:56 -05:00
Andrew BeveridgeandClaude Opus 4.5 70eca20c7a Consolidate production E2E tests into single comprehensive flow
Merged production-full-journey.spec.ts and production-theme-system.spec.ts
into production-e2e.spec.ts. The combined test:

- Uses cached song (piri/dog) for fast flacfetch responses
- Tests theme selection as part of the core journey
- Validates theme_id is correctly set on jobs
- Follows the complete flow: search -> audio -> review -> instrumental -> complete

This eliminates redundancy and ensures theme testing is integrated with
the main E2E flow rather than being a separate concern.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 20:35:03 -05:00
Andrew BeveridgeandClaude Opus 4.5 0e6a0b43ce Update production E2E tests to use real cached song
Use "piri" / "dog" for all production E2E tests instead of fake
test data. This leverages flacfetch's new caching system for fast
search responses without repeated API calls to trackers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 20:31:50 -05:00
Andrew BeveridgeandClaude Opus 4.5 3cbfbd14d1 Fix: Pass theme configuration from JobCreate to Job
The create_job() function was not passing theme_id, color_overrides,
style_params_gcs_path, or style_assets from JobCreate to the Job
constructor. This caused jobs created via audio search to have null
theme_id even when a theme was specified.

Also enables video recording for all Playwright tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 20:17:42 -05:00
Andrew Beveridge ☄️ 2f4265e9d0 Fix: Add theme support to audio search endpoint (#70)
Fixes bug where themes selected in the frontend were not being passed through when creating jobs via the Search tab. Audio search jobs now correctly receive theme_id and color_overrides, and CDG/TXT outputs are automatically enabled when a theme is selected.

- Add theme_id and color_overrides fields to AudioSearchRequest
- Add _resolve_cdg_txt_defaults() helper for smart CDG/TXT defaults
- Add comprehensive backend tests for theme support
- Add E2E Playwright tests for production theme system with full UI flow
2025-12-27 19:15:28 -05:00
Andrew Beveridge ☄️ 4a74a2cc3a Add theme system for pre-made video styles (#69)
Adds a comprehensive theme system for karaoke video styles:

- Theme selection and customizable color overrides for karaoke jobs
- Public Themes API for listing, previewing, and fetching theme details
- Theme selector and color-overrides UI in job submission flows
- CLI option to specify a theme when preparing/submitting jobs
- Thread-safe singleton initialization for ThemeService

🤖 Generated with Claude Code
2025-12-27 17:55:47 -05:00
21 changed files with 3052 additions and 967 deletions

No files matched your search

+171 -10
View File
@@ -16,6 +16,85 @@ concurrency:
cancel-in-progress: true
jobs:
# ============================================
# PATH FILTERING - Determine what changed
# ============================================
changes:
name: "Detect Changes"
runs-on: ubuntu-latest
outputs:
frontend: ${{ steps.filter.outputs.frontend }}
backend: ${{ steps.filter.outputs.backend }}
package: ${{ steps.filter.outputs.package }}
infra: ${{ steps.filter.outputs.infra }}
deps: ${{ steps.filter.outputs.deps }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: filter
with:
filters: |
frontend:
- 'frontend/**'
backend:
- 'backend/**'
package:
- 'karaoke_gen/**'
- 'lyrics_transcriber_temp/**'
- 'tests/**'
infra:
- 'infrastructure/**'
deps:
- 'poetry.lock'
- name: Summary
run: |
echo "### Change Detection Results" >> $GITHUB_STEP_SUMMARY
echo "| Area | Changed |" >> $GITHUB_STEP_SUMMARY
echo "|------|---------|" >> $GITHUB_STEP_SUMMARY
echo "| Frontend | ${{ steps.filter.outputs.frontend }} |" >> $GITHUB_STEP_SUMMARY
echo "| Backend | ${{ steps.filter.outputs.backend }} |" >> $GITHUB_STEP_SUMMARY
echo "| Package | ${{ steps.filter.outputs.package }} |" >> $GITHUB_STEP_SUMMARY
echo "| Infrastructure | ${{ steps.filter.outputs.infra }} |" >> $GITHUB_STEP_SUMMARY
echo "| Dependencies | ${{ steps.filter.outputs.deps }} |" >> $GITHUB_STEP_SUMMARY
# ============================================
# FRONTEND TESTS
# ============================================
frontend-unit-tests:
name: "Frontend - Unit Tests"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.frontend == 'true' || github.ref == 'refs/heads/main' }}
steps:
- uses: actions/checkout@v4
- name: Setup Node.js with caching
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Install dependencies
working-directory: frontend
run: npm ci --legacy-peer-deps
- name: Run unit tests with coverage
working-directory: frontend
run: npm run test:ci
- name: Upload coverage report
if: always()
uses: actions/upload-artifact@v4
with:
name: frontend-coverage
path: frontend/coverage/
retention-days: 7
# ============================================
# PACKAGE TESTS (karaoke-gen Python CLI)
# ============================================
@@ -23,6 +102,8 @@ jobs:
package-unit-tests:
name: "Package - Unit Tests (70% coverage)"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
steps:
- uses: actions/checkout@v4
with:
@@ -39,6 +120,14 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-package-${{ runner.os }}-py3.13-
- name: Install dependencies
run: poetry install --with dev
@@ -47,6 +136,12 @@ jobs:
sudo apt-get update
sudo apt-get install -y ffmpeg
- name: Cache spaCy model
uses: actions/cache@v4
with:
path: ~/.cache/spacy
key: spacy-en_core_web_sm-${{ runner.os }}
- name: Install spaCy language models
run: poetry run python -m spacy download en_core_web_sm
@@ -65,6 +160,8 @@ jobs:
package-integration-tests:
name: "Package - Integration Tests"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
steps:
- uses: actions/checkout@v4
with:
@@ -81,6 +178,14 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-package-${{ runner.os }}-py3.13-
- name: Install dependencies
run: poetry install --with dev
@@ -89,6 +194,12 @@ jobs:
sudo apt-get update
sudo apt-get install -y ffmpeg
- name: Cache spaCy model
uses: actions/cache@v4
with:
path: ~/.cache/spacy
key: spacy-en_core_web_sm-${{ runner.os }}
- name: Install spaCy language models
run: poetry run python -m spacy download en_core_web_sm
@@ -98,7 +209,8 @@ jobs:
package-build:
name: "Package - Build & Install"
runs-on: ubuntu-latest
needs: [package-unit-tests, package-integration-tests]
needs: [changes, package-unit-tests, package-integration-tests]
if: ${{ always() && (needs.package-unit-tests.result == 'success' || needs.package-unit-tests.result == 'skipped') && (needs.package-integration-tests.result == 'success' || needs.package-integration-tests.result == 'skipped') && (needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main') }}
steps:
- uses: actions/checkout@v4
with:
@@ -115,6 +227,14 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-package-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-package-${{ runner.os }}-py3.13-
- name: Build package
run: poetry build
@@ -148,6 +268,8 @@ jobs:
backend-unit-tests:
name: "Backend - Unit Tests"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -165,6 +287,14 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-backend-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-backend-${{ runner.os }}-py3.13-
- name: Install dependencies
run: poetry install
@@ -183,6 +313,8 @@ jobs:
backend-emulator-tests:
name: "Backend - Emulator Integration Tests"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || needs.changes.outputs.deps == 'true' || github.ref == 'refs/heads/main' }}
steps:
- name: Free disk space
run: |
@@ -207,6 +339,14 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-backend-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-backend-${{ runner.os }}-py3.13-
- name: Set up Java 21 (required by Firestore emulator)
uses: actions/setup-java@v4
with:
@@ -243,6 +383,8 @@ jobs:
backend-lint:
name: "Backend - Code Quality"
runs-on: ubuntu-latest
needs: changes
if: ${{ needs.changes.outputs.backend == 'true' || needs.changes.outputs.package == 'true' || github.ref == 'refs/heads/main' }}
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -260,6 +402,14 @@ jobs:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Cache Poetry dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-lint-${{ runner.os }}-py3.13-${{ hashFiles('poetry.lock') }}
restore-keys: |
venv-lint-${{ runner.os }}-py3.13-
- name: Install dependencies
run: poetry install
@@ -362,7 +512,8 @@ jobs:
deploy-frontend:
name: "Deploy - Frontend (GitHub Pages)"
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
needs: [frontend-unit-tests]
if: ${{ always() && github.ref == 'refs/heads/main' && github.event_name == 'push' && (needs.frontend-unit-tests.result == 'success' || needs.frontend-unit-tests.result == 'skipped') }}
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
@@ -370,10 +521,12 @@ jobs:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node.js
- name: Setup Node.js with caching
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Enable Yarn (for lyrics frontend)
run: corepack enable
@@ -390,6 +543,14 @@ jobs:
run: npm run build
# ===== Build Lyrics Review Frontend (Vite) =====
- name: Cache Yarn dependencies (lyrics frontend)
uses: actions/cache@v4
with:
path: lyrics_transcriber_temp/lyrics_transcriber/frontend/node_modules
key: yarn-lyrics-${{ runner.os }}-${{ hashFiles('lyrics_transcriber_temp/lyrics_transcriber/frontend/yarn.lock') }}
restore-keys: |
yarn-lyrics-${{ runner.os }}-
- name: Install lyrics frontend dependencies
working-directory: lyrics_transcriber_temp/lyrics_transcriber/frontend
run: yarn install --frozen-lockfile
@@ -428,7 +589,7 @@ jobs:
name: "Deploy - Backend (Cloud Run)"
runs-on: ubuntu-latest
needs: [backend-unit-tests, backend-emulator-tests, backend-lint]
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
if: ${{ always() && github.ref == 'refs/heads/main' && github.event_name == 'push' && (needs.backend-unit-tests.result == 'success' || needs.backend-unit-tests.result == 'skipped') && (needs.backend-emulator-tests.result == 'success' || needs.backend-emulator-tests.result == 'skipped') && (needs.backend-lint.result == 'success' || needs.backend-lint.result == 'skipped') }}
permissions:
contents: read
id-token: write
@@ -453,7 +614,7 @@ jobs:
VERSION=$(poetry version --short)
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "📋 Current version: $VERSION"
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
echo "needs_deploy=false" >> $GITHUB_OUTPUT
echo "⏭️ Tag v$VERSION already exists - skipping backend deployment"
@@ -604,22 +765,22 @@ jobs:
echo "Service URL: $SERVICE_URL"
echo "Expected version: $EXPECTED_VERSION"
# Wait for new revision to be ready
echo "Waiting for new revision to be ready..."
sleep 15
# Basic health check
curl -f "$SERVICE_URL/api/health" || exit 1
echo "✅ Service is healthy"
# Version verification with retries (Cloud Run may take time to route to new revision)
MAX_RETRIES=6
RETRY_DELAY=10
for i in $(seq 1 $MAX_RETRIES); do
echo "Checking deployed version (attempt $i/$MAX_RETRIES)..."
DEPLOYED_VERSION=$(curl -sf "$SERVICE_URL/api/health/detailed" | jq -r '.version // empty')
if [ "$DEPLOYED_VERSION" = "$EXPECTED_VERSION" ]; then
echo "✅ Version verified: $DEPLOYED_VERSION"
break
@@ -643,7 +804,7 @@ jobs:
echo "Commit: ${{ github.sha }}"
echo "Branch: ${{ github.ref_name }}"
echo "Run: ${{ github.run_number }}"
- name: Skipped deployment summary
if: steps.version_check.outputs.needs_deploy == 'false'
run: |
+3
View File
@@ -12,6 +12,9 @@ tests/outputs
/input
/output/
# Cloud Infra
.emulator-logs
# Next.js / React Frontend (./frontend)
node_modules/
frontend/node_modules/
+68 -7
View File
@@ -85,9 +85,13 @@ class AudioSearchRequest(BaseModel):
# Auto-download mode
auto_download: bool = Field(False, description="Automatically select best result and download")
# Processing options (CDG/TXT require style config, disabled by default)
enable_cdg: bool = Field(False, description="Generate CDG+MP3 package (requires style config)")
enable_txt: bool = Field(False, description="Generate TXT+MP3 package (requires style config)")
# Theme configuration
theme_id: Optional[str] = Field(None, description="Theme ID to use (e.g., 'nomad', 'default'). If set, CDG/TXT are enabled by default.")
color_overrides: Optional[Dict[str, str]] = Field(None, description="Color overrides: artist_color, title_color, sung_lyrics_color, unsung_lyrics_color (hex #RRGGBB)")
# Processing options (CDG/TXT require style config or theme, disabled by default unless theme is set)
enable_cdg: Optional[bool] = Field(None, description="Generate CDG+MP3 package. Default: True if theme_id set, False otherwise")
enable_txt: Optional[bool] = Field(None, description="Generate TXT+MP3 package. Default: True if theme_id set, False otherwise")
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
@@ -180,6 +184,35 @@ class AudioSelectResponse(BaseModel):
selected_provider: str
def _resolve_cdg_txt_defaults(
theme_id: Optional[str],
enable_cdg: Optional[bool],
enable_txt: Optional[bool]
) -> Tuple[bool, bool]:
"""
Resolve CDG/TXT settings based on theme and explicit settings.
When a theme is selected, CDG and TXT are enabled by default.
Explicit True/False values always override the default.
Args:
theme_id: Theme identifier (if any)
enable_cdg: Explicit CDG setting (None means use default)
enable_txt: Explicit TXT setting (None means use default)
Returns:
Tuple of (resolved_enable_cdg, resolved_enable_txt)
"""
# Default based on whether theme is set
default_enabled = theme_id is not None
# Explicit values override defaults, None uses default
resolved_cdg = enable_cdg if enable_cdg is not None else default_enabled
resolved_txt = enable_txt if enable_txt is not None else default_enabled
return resolved_cdg, resolved_txt
def extract_request_metadata(request: Request, created_from: str = "audio_search") -> Dict[str, Any]:
"""Extract metadata from request for job tracking."""
headers = dict(request.headers)
@@ -461,13 +494,20 @@ async def search_audio(
# Extract request metadata
request_metadata = extract_request_metadata(request, created_from="audio_search")
# Resolve CDG/TXT defaults based on theme
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
body.theme_id, body.enable_cdg, body.enable_txt
)
# Create job
job_create = JobCreate(
artist=body.artist,
title=body.title,
enable_cdg=body.enable_cdg,
enable_txt=body.enable_txt,
theme_id=body.theme_id,
color_overrides=body.color_overrides or {},
enable_cdg=resolved_cdg,
enable_txt=resolved_txt,
brand_prefix=effective_brand_prefix,
enable_youtube_upload=effective_enable_youtube_upload,
youtube_description=effective_youtube_description,
@@ -497,7 +537,28 @@ async def search_audio(
'audio_search_title': body.title,
'auto_download': body.auto_download,
})
# If theme is set and no custom style files are being uploaded, prepare theme style now
# This copies the theme's style_params.json to the job folder so LyricsTranscriber
# can access the style configuration for preview videos
if body.theme_id and not body.style_files:
from backend.api.routes.file_upload import _prepare_theme_for_job
try:
style_params_path, theme_style_assets, youtube_desc = _prepare_theme_for_job(
job_id, body.theme_id, body.color_overrides
)
theme_update = {
'style_params_gcs_path': style_params_path,
'style_assets': theme_style_assets,
}
if youtube_desc and not effective_youtube_description:
theme_update['youtube_description_template'] = youtube_desc
job_manager.update_job(job_id, theme_update)
logger.info(f"Applied theme '{body.theme_id}' to job {job_id}")
except Exception as e:
logger.warning(f"Failed to prepare theme '{body.theme_id}' for job {job_id}: {e}")
# Continue without theme - job can still be processed with defaults
# Handle style file uploads if provided
style_upload_urls: List[StyleUploadUrl] = []
style_assets = {}
+5
View File
@@ -61,6 +61,11 @@ class JobManager:
discord_webhook_url=job_create.discord_webhook_url,
dropbox_path=job_create.dropbox_path,
gdrive_folder_id=job_create.gdrive_folder_id,
# Theme configuration
theme_id=job_create.theme_id,
color_overrides=job_create.color_overrides,
style_params_gcs_path=job_create.style_params_gcs_path,
style_assets=job_create.style_assets,
# Two-phase workflow (Batch 6)
prep_only=job_create.prep_only,
finalise_only=job_create.finalise_only,
@@ -330,7 +330,7 @@ class TestReviewEndpoints:
class TestRenderVideoWorker:
"""Test render video worker endpoint."""
def test_render_video_worker_endpoint_exists(self, client, auth_headers):
"""Test that render-video worker endpoint exists."""
# Create a job
@@ -340,9 +340,9 @@ class TestRenderVideoWorker:
json={"url": "https://youtube.com/watch?v=render-test"}
)
job_id = create_resp.json()["job_id"]
time.sleep(0.2)
# Test render-video worker endpoint exists
response = client.post(
"/api/internal/workers/render-video",
+248
View File
@@ -798,6 +798,254 @@ class TestRemoteDownloadPath:
assert result.filepath == "/tmp/test.flac"
class TestAudioSearchThemeSupport:
"""Test theme support in audio search requests.
These tests verify that theme_id and color_overrides are properly:
1. Accepted in AudioSearchRequest model
2. Passed through to JobCreate
3. Result in correct CDG/TXT defaults when theme is set
4. Theme style is PREPARED (copied to job folder) when theme_id is set
"""
def test_audio_search_request_accepts_theme_id(self):
"""Test AudioSearchRequest model has theme_id field."""
from backend.api.routes.audio_search import AudioSearchRequest
request = AudioSearchRequest(
artist="Test Artist",
title="Test Song",
theme_id="nomad"
)
assert request.theme_id == "nomad"
def test_audio_search_request_accepts_color_overrides(self):
"""Test AudioSearchRequest model has color_overrides field."""
from backend.api.routes.audio_search import AudioSearchRequest
request = AudioSearchRequest(
artist="Test Artist",
title="Test Song",
theme_id="nomad",
color_overrides={
"artist_color": "#ff0000",
"title_color": "#00ff00",
}
)
assert request.color_overrides["artist_color"] == "#ff0000"
assert request.color_overrides["title_color"] == "#00ff00"
def test_audio_search_request_theme_defaults_cdg_txt(self):
"""Test that when theme_id is set, CDG/TXT defaults to enabled.
This is the key behavior: selecting a theme should automatically
enable CDG and TXT output formats.
"""
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
# When theme_id is set, enable_cdg/enable_txt should default to True
request = AudioSearchRequest(
artist="Test Artist",
title="Test Song",
theme_id="nomad"
# enable_cdg and enable_txt are None (not specified)
)
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
request.theme_id, request.enable_cdg, request.enable_txt
)
assert resolved_cdg is True, "CDG should be enabled by default when theme is set"
assert resolved_txt is True, "TXT should be enabled by default when theme is set"
def test_audio_search_request_no_theme_no_cdg_txt(self):
"""Test that without theme_id, CDG/TXT defaults to disabled."""
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
request = AudioSearchRequest(
artist="Test Artist",
title="Test Song"
# No theme_id, no enable_cdg, no enable_txt
)
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
request.theme_id, request.enable_cdg, request.enable_txt
)
assert resolved_cdg is False, "CDG should be disabled by default without theme"
assert resolved_txt is False, "TXT should be disabled by default without theme"
def test_explicit_cdg_txt_overrides_theme_default(self):
"""Test that explicit enable_cdg/enable_txt values override theme defaults."""
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
# Theme set (would default to True), but explicitly disabled
request = AudioSearchRequest(
artist="Test Artist",
title="Test Song",
theme_id="nomad",
enable_cdg=False,
enable_txt=False,
)
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
request.theme_id, request.enable_cdg, request.enable_txt
)
assert resolved_cdg is False, "Explicit False should override theme default"
assert resolved_txt is False, "Explicit False should override theme default"
def test_explicit_cdg_txt_enables_without_theme(self):
"""Test that explicit True enables CDG/TXT even without theme."""
from backend.api.routes.audio_search import AudioSearchRequest, _resolve_cdg_txt_defaults
# No theme (would default to False), but explicitly enabled
request = AudioSearchRequest(
artist="Test Artist",
title="Test Song",
enable_cdg=True,
enable_txt=True,
)
resolved_cdg, resolved_txt = _resolve_cdg_txt_defaults(
request.theme_id, request.enable_cdg, request.enable_txt
)
assert resolved_cdg is True, "Explicit True should enable CDG without theme"
assert resolved_txt is True, "Explicit True should enable TXT without theme"
def test_job_create_receives_theme_from_audio_search(self):
"""Test that JobCreate model can receive theme_id and color_overrides."""
job_create = JobCreate(
artist="Test Artist",
title="Test Song",
theme_id="nomad",
color_overrides={
"artist_color": "#ff0000",
"sung_lyrics_color": "#00ff00"
},
enable_cdg=True,
enable_txt=True,
)
assert job_create.theme_id == "nomad"
assert job_create.color_overrides["artist_color"] == "#ff0000"
assert job_create.color_overrides["sung_lyrics_color"] == "#00ff00"
assert job_create.enable_cdg is True
assert job_create.enable_txt is True
def test_job_model_stores_theme_configuration(self):
"""Test Job model stores theme configuration correctly."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.utcnow(),
updated_at=datetime.utcnow(),
artist="Test Artist",
title="Test Song",
theme_id="nomad",
color_overrides={
"artist_color": "#ff0000"
},
enable_cdg=True,
enable_txt=True,
)
assert job.theme_id == "nomad"
assert job.color_overrides["artist_color"] == "#ff0000"
assert job.enable_cdg is True
assert job.enable_txt is True
class TestAudioSearchThemePreparation:
"""Test that audio search endpoint properly prepares theme styles.
CRITICAL: This test class was added to catch the bug where audio_search.py
accepted theme_id but never called prepare_job_style(), resulting in jobs
with theme_id set but no style_params_gcs_path or style_assets.
The symptom: Preview videos show black background instead of themed background.
Job ID ffb0b8fa in production demonstrated this issue.
"""
def test_audio_search_with_theme_calls_prepare_job_style(self):
"""Test that creating a job via audio search with theme_id prepares the style.
CRITICAL: This test verifies that when a job is created via the audio-search
endpoint with a theme_id (and no custom style files), the code calls
prepare_job_style() to set:
1. style_params_gcs_path (pointing to the copied style_params.json)
2. style_assets (populated with asset mappings)
Without this, LyricsTranscriber won't have access to the theme's styles
and preview videos will have black backgrounds instead of themed ones.
"""
# Import the endpoint module to access internal functions
from backend.api.routes import audio_search as audio_search_module
# Check if there's an import of prepare_job_style from theme_service
source_code_path = audio_search_module.__file__
with open(source_code_path, 'r') as f:
source_code = f.read()
# Check for either:
# 1. Import of _prepare_theme_for_job from file_upload
# 2. Import of prepare_job_style from theme_service
# 3. Inline call to theme_service.prepare_job_style
has_theme_prep_import = (
'_prepare_theme_for_job' in source_code or
'prepare_job_style' in source_code
)
assert has_theme_prep_import, (
"audio_search.py does not import or call prepare_job_style() or _prepare_theme_for_job(). "
"When theme_id is provided without custom style files, the endpoint MUST call "
"prepare_job_style() to copy the theme's style_params.json to the job folder. "
"Without this, LyricsTranscriber won't have style configuration and preview "
"videos will have black backgrounds instead of themed ones."
)
def test_audio_search_code_handles_theme_preparation(self):
"""Verify audio_search.py has theme preparation logic similar to file_upload.py.
This test compares audio_search.py with file_upload.py to ensure they have
similar theme preparation patterns. file_upload.py correctly prepares themes,
audio_search.py should do the same.
"""
from backend.api.routes import audio_search as audio_search_module
from backend.api.routes import file_upload as file_upload_module
# Read source code of both modules
with open(audio_search_module.__file__, 'r') as f:
audio_search_code = f.read()
with open(file_upload_module.__file__, 'r') as f:
file_upload_code = f.read()
# file_upload.py has this pattern for theme preparation:
# if body.theme_id and not has_style_params_upload:
# style_params_path, style_assets, youtube_desc = _prepare_theme_for_job(...)
assert '_prepare_theme_for_job' in file_upload_code, \
"file_upload.py should have _prepare_theme_for_job function"
# audio_search.py should have similar pattern
# Look for either the function or a similar conditional check
# STRICT CHECK: Look for actual theme preparation logic being CALLED, not just mentioned
has_theme_preparation_call = (
# Check for import or call of theme preparation function
'_prepare_theme_for_job(' in audio_search_code or
'prepare_job_style(' in audio_search_code
)
assert has_theme_preparation_call, (
"audio_search.py does not CALL theme preparation logic. "
"It accepts theme_id but never prepares the theme style like file_upload.py does. "
"Add a call to _prepare_theme_for_job() when theme_id is set and no style files are uploaded."
)
class TestFlacfetchIntegration:
"""
Integration tests that verify flacfetch imports work correctly.
@@ -0,0 +1,233 @@
# Job Status Display Improvement Plan
**Status:** Planned
**Created:** 2025-12-27
**Priority:** HIGH - User experience improvement
## Problem Statement
Users see generic status messages like "Downloading..." that persist for too long, even after the download is complete and other processing stages are happening. This makes it hard to understand:
1. What stage the job is actually at
2. How much progress has been made
3. How much longer the job might take
## Current Implementation
### Backend Status System
The backend has 28 distinct job statuses organized into 8 processing stages:
```
Stage 1: Setup (PENDING)
Stage 2: Audio Search (SEARCHING_AUDIO, AWAITING_AUDIO_SELECTION, DOWNLOADING_AUDIO)
Stage 3: Download (DOWNLOADING)
Stage 4: Parallel Processing
- Audio: SEPARATING_STAGE1 → SEPARATING_STAGE2 → AUDIO_COMPLETE
- Lyrics: TRANSCRIBING → CORRECTING → LYRICS_COMPLETE
Stage 5: Screen Generation (GENERATING_SCREENS, APPLYING_PADDING)
Stage 6: Lyrics Review (AWAITING_REVIEW, IN_REVIEW, REVIEW_COMPLETE) ⚠️ BLOCKING
Stage 7: Video Rendering (RENDERING_VIDEO)
Stage 8: Instrumental Selection (AWAITING_INSTRUMENTAL_SELECTION) ⚠️ BLOCKING
Stage 9: Final Processing (INSTRUMENTAL_SELECTED, GENERATING_VIDEO, ENCODING, PACKAGING)
Stage 10: Distribution (UPLOADING, NOTIFYING, COMPLETE)
```
### Key Findings
1. **Progress field exists but unused**: The `Job` model has a `progress: number` field (0-100) that's returned by the API but NOT displayed in the frontend
2. **Parallel processing complicates status**: Audio and lyrics workers run in parallel, updating `state_data.audio_progress` and `state_data.lyrics_progress` independently
3. **Status labels are static**: Frontend shows "Downloading..." but doesn't reflect that parallel workers are running
4. **10-second polling**: Frontend polls every 10 seconds, so status updates are not real-time
## Proposed Solution
### 1. Step-Based Progress Display
Show a `[X/10] Status...` format that gives users a clear sense of overall progress:
```
[1/10] Setting up...
[2/10] Searching for audio...
[3/10] Downloading audio...
[4/10] Processing audio & lyrics... ← Parallel stage
[5/10] Generating screens...
[6/10] Needs review ← Blocking
[7/10] Rendering video...
[8/10] Select instrumental ← Blocking
[9/10] Encoding final video...
[10/10] Uploading...
```
### 2. Status-to-Step Mapping
```typescript
const STATUS_STEP_MAP: Record<string, { step: number; total: number; label: string }> = {
// Step 1: Setup
pending: { step: 1, total: 10, label: "Setting up" },
// Step 2: Audio Search (optional)
searching_audio: { step: 2, total: 10, label: "Searching for audio" },
awaiting_audio_selection: { step: 2, total: 10, label: "Select audio source" },
// Step 3: Download
downloading_audio: { step: 3, total: 10, label: "Downloading audio" },
downloading: { step: 3, total: 10, label: "Downloading" },
// Step 4: Parallel Processing (Audio + Lyrics)
separating_stage1: { step: 4, total: 10, label: "Separating audio (1/2)" },
separating_stage2: { step: 4, total: 10, label: "Separating audio (2/2)" },
audio_complete: { step: 4, total: 10, label: "Audio ready" },
transcribing: { step: 4, total: 10, label: "Transcribing lyrics" },
correcting: { step: 4, total: 10, label: "Correcting lyrics" },
lyrics_complete: { step: 4, total: 10, label: "Lyrics ready" },
// Step 5: Screen Generation
generating_screens: { step: 5, total: 10, label: "Generating screens" },
applying_padding: { step: 5, total: 10, label: "Syncing countdown" },
// Step 6: Review (BLOCKING)
awaiting_review: { step: 6, total: 10, label: "Needs review" },
in_review: { step: 6, total: 10, label: "In review" },
// Step 7: Video Rendering
review_complete: { step: 7, total: 10, label: "Review complete" },
rendering_video: { step: 7, total: 10, label: "Rendering video" },
// Step 8: Instrumental Selection (BLOCKING)
awaiting_instrumental_selection: { step: 8, total: 10, label: "Select instrumental" },
// Step 9: Final Encoding
instrumental_selected: { step: 9, total: 10, label: "Instrumental selected" },
generating_video: { step: 9, total: 10, label: "Generating final video" },
encoding: { step: 9, total: 10, label: "Encoding video" },
packaging: { step: 9, total: 10, label: "Packaging files" },
// Step 10: Distribution
uploading: { step: 10, total: 10, label: "Uploading" },
notifying: { step: 10, total: 10, label: "Sending notifications" },
complete: { step: 10, total: 10, label: "Complete" },
// Terminal states
failed: { step: 0, total: 10, label: "Failed" },
cancelled: { step: 0, total: 10, label: "Cancelled" },
prep_complete: { step: 10, total: 10, label: "Prep complete" },
};
```
### 3. Enhanced Parallel Processing Display
When audio and lyrics are processing in parallel, show both activities:
```typescript
// Check state_data for parallel progress
if (job.state_data?.audio_progress && job.state_data?.lyrics_progress) {
const audioStatus = job.state_data.audio_progress.stage;
const lyricsStatus = job.state_data.lyrics_progress.stage;
// Show combined status like "Audio: separating, Lyrics: transcribing"
if (audioStatus !== 'audio_complete' || lyricsStatus !== 'lyrics_complete') {
return {
step: 4,
total: 10,
label: buildParallelLabel(audioStatus, lyricsStatus)
};
}
}
```
### 4. Visual Progress Bar (Optional Enhancement)
Add a thin progress bar below the status text:
```tsx
<div className="w-full h-1 bg-slate-700 rounded mt-1">
<div
className="h-full bg-blue-500 rounded transition-all"
style={{ width: `${(step / total) * 100}%` }}
/>
</div>
```
## Implementation Steps
### Phase 1: Core Changes (MVP)
1. **Create status utility** (`frontend/lib/job-status.ts`):
- Define `STATUS_STEP_MAP` constant
- Create `getJobStep(job: Job)` function
- Create `formatStepStatus(step, total, label)` function
2. **Update JobCard component**:
- Import status utility
- Replace static `StatusText` with step-based format
- Keep existing color coding from `statusConfig`
3. **Add unit tests** (`frontend/__tests__/job-status.test.ts`):
- Test all status mappings
- Test parallel processing detection
- Test edge cases (unknown status, null values)
### Phase 2: Enhanced Parallel Processing (Optional)
1. **Add parallel progress detection**:
- Check `state_data.audio_progress` and `state_data.lyrics_progress`
- Build combined label showing both activities
2. **Add visual progress bar**:
- Thin progress indicator based on step number
- Smooth transitions between steps
### Phase 3: Backend Improvements (Future)
1. **Add estimated time remaining**:
- Track average duration per stage from historical data
- Return ETA in API response
2. **WebSocket for real-time updates**:
- Replace polling with push notifications
- Smoother progress updates during long operations
## Files to Modify
| File | Changes |
|------|---------|
| `frontend/lib/job-status.ts` | NEW: Status utility functions |
| `frontend/components/job/JobCard.tsx` | Update status display |
| `frontend/__tests__/job-status.test.ts` | NEW: Unit tests |
| `frontend/lib/api.ts` | Add `state_data` types (if needed) |
## Testing Plan
1. **Unit tests**:
- All status mappings return correct step/label
- Unknown statuses handled gracefully
- Parallel progress detection works
2. **Integration tests** (E2E):
- Verify status displays correctly at each stage
- Verify step numbers increment as job progresses
3. **Manual testing**:
- Submit a job and observe status progression
- Verify parallel processing shows meaningful status
## Success Criteria
- [ ] All statuses show `[X/10] Label` format
- [ ] Step numbers progress logically (never decrease except on retry)
- [ ] Parallel processing shows combined activity
- [ ] Blocking statuses clearly indicated for user action
- [ ] All existing tests pass
- [ ] New unit tests have >80% coverage
## Rollback Plan
The changes are frontend-only and purely cosmetic. If issues arise:
1. Revert the JobCard changes to use the old `statusConfig` approach
2. The backend API remains unchanged
## Notes
- The 10-step system is a simplification of the 28 backend statuses for user comprehension
- Parallel processing (step 4) groups multiple backend statuses into one user-facing step
- Blocking steps (6, 8) require user action and should be visually distinct
- The `progress` field from backend could be used for sub-step progress bars in the future
+485
View File
@@ -0,0 +1,485 @@
# Job Debugging Guide
This guide explains how to access debug info, observability, metrics, logs, and data about a given job ID in the karaoke-gen cloud backend.
**Last Updated:** 2025-12-28
---
## Prerequisites
### Service Account Authentication
For automated access, set the service account impersonation variable:
```bash
export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=claude-automation@nomadkaraoke.iam.gserviceaccount.com
```
This service account has permissions to:
- Read Firestore documents
- Read Cloud Storage files
- Query Cloud Logging
- Describe Cloud Run services
- List Cloud Tasks queues
### Required Tools
- `gcloud` CLI (authenticated)
- Python 3.10+ with `google-cloud-firestore` installed
- Access to the `nomadkaraoke` GCP project
---
## Quick Reference
| Data Type | Location | Access Method |
|-----------|----------|---------------|
| Job state & metadata | Firestore `jobs/{job_id}` | Python script |
| Processing logs | Cloud Logging | `gcloud logging read` |
| Audio/video files | GCS bucket | `gcloud storage` |
| Service health | Cloud Run | `gcloud run services describe` |
| Task queues | Cloud Tasks | `gcloud tasks queues list` |
---
## 1. Fetching Job Data from Firestore
Firestore stores all job state including status, timeline, worker logs, and file URLs.
### Using Python (Recommended)
A helper script is included at `scripts/get_job.py`. You can also create your own:
```python
#!/usr/bin/env python3
"""Fetch and display job data from Firestore."""
import sys
import json
from google.cloud import firestore
def get_job(job_id: str):
db = firestore.Client(project='nomadkaraoke')
doc = db.collection('jobs').document(job_id).get()
if doc.exists:
data = doc.to_dict()
print(json.dumps(data, indent=2, default=str))
else:
print(f"Job {job_id} not found")
sys.exit(1)
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: python get_job.py <job_id>")
sys.exit(1)
get_job(sys.argv[1])
```
Run it:
```bash
python get_job.py ffb0b8fa
```
### Example Output
```json
{
"status": "in_review",
"artist": "piri",
"title": "dog",
"created_at": "2025-12-28T01:55:23.914454",
"audio_search_artist": "piri",
"audio_search_title": "dog",
"timeline": [
{
"timestamp": "2025-12-28T01:55:23.995388",
"message": "Searching for: piri - dog",
"progress": 5,
"status": "searching_audio"
},
{
"timestamp": "2025-12-28T01:55:24.249594",
"message": "Found 20 audio sources. Waiting for selection.",
"progress": 10,
"status": "awaiting_audio_selection"
}
// ... more timeline entries
],
"worker_logs": [
{
"level": "INFO",
"message": "=== AUDIO WORKER STARTED ===",
"timestamp": "2025-12-28T01:55:37.559644Z",
"worker": "audio"
}
// ... more worker logs
]
}
```
### Key Fields to Check
| Field | Description |
|-------|-------------|
| `status` | Current job status (e.g., `in_review`, `complete`, `failed`) |
| `error_message` | Error description if job failed |
| `timeline` | Ordered list of status transitions with timestamps |
| `worker_logs` | Logs captured from worker processes |
| `file_urls` | Signed URLs for generated files |
| `state_data` | Worker-specific progress data |
---
## 2. Viewing Cloud Run Logs
Logs are stored in Cloud Logging and can be queried with `gcloud logging read`.
### Search Logs for a Job ID
```bash
gcloud logging read \
'resource.type="cloud_run_revision" AND resource.labels.service_name="karaoke-backend" AND "ffb0b8fa"' \
--project=nomadkaraoke \
--limit=50 \
--format='table(timestamp,severity,jsonPayload.message,jsonPayload.logger)'
```
### Filter by Severity (Errors Only)
```bash
gcloud logging read \
'resource.type="cloud_run_revision" AND resource.labels.service_name="karaoke-backend" AND "ffb0b8fa" AND severity>=ERROR' \
--project=nomadkaraoke \
--limit=20 \
--format=json
```
### Filter by Time Window
```bash
gcloud logging read \
'resource.type="cloud_run_revision" AND resource.labels.service_name="karaoke-backend" AND "ffb0b8fa" AND timestamp>="2025-12-28T01:55:00Z" AND timestamp<="2025-12-28T02:05:00Z"' \
--project=nomadkaraoke \
--limit=100 \
--order=asc \
--format='table(timestamp,severity,jsonPayload.message)'
```
### Search for Exceptions
```bash
gcloud logging read \
'resource.type="cloud_run_revision" AND resource.labels.service_name="karaoke-backend" AND (jsonPayload.message:"Exception" OR jsonPayload.message:"Traceback")' \
--project=nomadkaraoke \
--limit=10 \
--format='table(timestamp,severity,jsonPayload.message)'
```
### View HTTP Request Logs
```bash
gcloud logging read \
'resource.type="cloud_run_revision" AND resource.labels.service_name="karaoke-backend" AND httpRequest.requestUrl:"ffb0b8fa"' \
--project=nomadkaraoke \
--limit=20 \
--format='table(timestamp,httpRequest.status,httpRequest.requestMethod,httpRequest.requestUrl)'
```
---
## 3. Accessing Job Files in Cloud Storage
Files are stored in `gs://karaoke-gen-storage-nomadkaraoke/jobs/{job_id}/`.
### List All Files for a Job
```bash
gcloud storage ls -r gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/ --project=nomadkaraoke
```
Example output:
```
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/:
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/lyrics/:
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/lyrics/corrections.json
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/lyrics/karaoke.lrc
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/lyrics/piri - dog (Lyrics Genius).txt
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/lyrics/uncorrected.txt
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/previews/:
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/previews/86b528e04907.mp4
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/screens/:
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/screens/end.jpg
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/screens/end.mov
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/screens/title.jpg
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/screens/title.mov
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/:
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/backing_vocals.flac
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/instrumental_clean.flac
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/instrumental_with_backing.flac
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/lead_vocals.flac
gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/vocals_clean.flac
```
### View File Content (Small Files)
```bash
gcloud storage cat gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/lyrics/corrections.json --project=nomadkaraoke | head -50
```
### Get File Metadata
```bash
gcloud storage objects describe gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/instrumental_clean.flac --project=nomadkaraoke
```
Example output:
```yaml
bucket: karaoke-gen-storage-nomadkaraoke
content_type: application/octet-stream
crc32c_hash: /uXURg==
creation_time: 2025-12-28T01:57:44+0000
size: 18115881
storage_class: STANDARD
```
### Download a File
```bash
gcloud storage cp gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/stems/instrumental_clean.flac ./instrumental.flac --project=nomadkaraoke
```
### GCS Folder Structure
```
jobs/{job_id}/
├── lyrics/
│ ├── corrections.json # Lyrics with timing corrections
│ ├── karaoke.lrc # LRC format lyrics
│ ├── uncorrected.txt # Raw transcription
│ └── *.txt # Reference lyrics from sources
├── previews/
│ └── {hash}.mp4 # Preview video for lyrics review
├── screens/
│ ├── title.mov/.jpg/.png # Title screen
│ └── end.mov/.jpg/.png # End screen
├── stems/
│ ├── instrumental_clean.flac
│ ├── instrumental_with_backing.flac
│ ├── vocals_clean.flac
│ ├── lead_vocals.flac
│ └── backing_vocals.flac
├── videos/
│ └── with_vocals.mkv # Karaoke video with lyrics overlay
└── finals/
├── lossless_4k.mp4
├── lossless_4k.mkv
├── lossy_4k.mp4
└── lossy_720p.mp4
```
---
## 4. Cloud Run Service Status
### Check Service Health
```bash
gcloud run services describe karaoke-backend \
--project=nomadkaraoke \
--region=us-central1 \
--format='yaml(status.conditions,status.latestReadyRevisionName,status.url)'
```
Example output:
```yaml
status:
conditions:
- lastTransitionTime: '2025-12-28T01:29:43.465349Z'
status: 'True'
type: Ready
- lastTransitionTime: '2025-12-28T01:29:42.192338Z'
status: 'True'
type: ConfigurationsReady
- lastTransitionTime: '2025-12-28T01:29:43.430737Z'
status: 'True'
type: RoutesReady
latestReadyRevisionName: karaoke-backend-00203-c7q
url: https://karaoke-backend-ipzqd2k4yq-uc.a.run.app
```
### List Recent Revisions
```bash
gcloud run revisions list \
--service=karaoke-backend \
--project=nomadkaraoke \
--region=us-central1 \
--limit=5 \
--format='table(name,status.conditions.type,status.conditions.status,createTime)'
```
### View Revision Details
```bash
gcloud run revisions describe karaoke-backend-00203-c7q \
--project=nomadkaraoke \
--region=us-central1 \
--format='yaml(spec.containerConcurrency,spec.containers[0].resources)'
```
---
## 5. Cloud Tasks Queues
The backend uses Cloud Tasks for async worker processing.
### List All Queues
```bash
gcloud tasks queues list --project=nomadkaraoke --location=us-central1
```
Example output:
```
QUEUE_NAME STATE MAX_NUM_OF_TASKS MAX_RATE (/sec) MAX_ATTEMPTS
audio-worker-queue RUNNING 50 10.0 3
lyrics-worker-queue RUNNING 50 10.0 3
render-worker-queue RUNNING 20 5.0 2
screens-worker-queue RUNNING 100 50.0 3
video-worker-queue RUNNING 10 3.0 2
```
### View Queue Details
```bash
gcloud tasks queues describe audio-worker-queue \
--project=nomadkaraoke \
--location=us-central1
```
### List Tasks in a Queue
```bash
gcloud tasks list \
--queue=audio-worker-queue \
--project=nomadkaraoke \
--location=us-central1 \
--limit=10
```
---
## 6. Common Debugging Scenarios
### Scenario: Job Stuck in Status
1. **Check job state:**
```bash
python get_job.py ffb0b8fa | jq '.status, .error_message, .timeline[-3:]'
```
2. **Check for errors in logs:**
```bash
gcloud logging read 'resource.labels.service_name="karaoke-backend" AND "ffb0b8fa" AND severity>=ERROR' --project=nomadkaraoke --limit=5 --format=json
```
3. **Check if worker task is queued:**
```bash
gcloud tasks list --queue=audio-worker-queue --project=nomadkaraoke --location=us-central1
```
### Scenario: Job Failed
1. **Get error message from Firestore:**
```bash
python get_job.py ffb0b8fa | jq '.error_message, .error_details'
```
2. **Search for exception stack trace:**
```bash
gcloud logging read 'resource.labels.service_name="karaoke-backend" AND "ffb0b8fa" AND jsonPayload.message:"Traceback"' --project=nomadkaraoke --limit=3 --format='value(jsonPayload.message)'
```
### Scenario: Missing Output Files
1. **List all generated files:**
```bash
gcloud storage ls -r gs://karaoke-gen-storage-nomadkaraoke/jobs/ffb0b8fa/ --project=nomadkaraoke
```
2. **Check worker logs for upload failures:**
```bash
gcloud logging read 'resource.labels.service_name="karaoke-backend" AND "ffb0b8fa" AND (jsonPayload.message:"upload" OR jsonPayload.message:"GCS")' --project=nomadkaraoke --limit=20 --format='table(timestamp,jsonPayload.message)'
```
### Scenario: Audio Separation Slow/Failing
1. **Check audio worker logs:**
```bash
gcloud logging read 'resource.labels.service_name="karaoke-backend" AND "ffb0b8fa" AND jsonPayload.worker:"audio"' --project=nomadkaraoke --limit=30 --format='table(timestamp,jsonPayload.message)'
```
2. **Check Modal API status in worker logs:**
```bash
python get_job.py ffb0b8fa | jq '.worker_logs | map(select(.worker == "audio"))'
```
---
## 7. Useful Aliases
Add these to your shell profile for quick access:
```bash
# Job debugging aliases (update paths as needed)
alias kj='python scripts/get_job.py'
alias klogs='gcloud logging read "resource.labels.service_name=\"karaoke-backend\"" --project=nomadkaraoke --limit=50 --format="table(timestamp,severity,jsonPayload.message)"'
alias kfiles='gcloud storage ls -r gs://karaoke-gen-storage-nomadkaraoke/jobs/'
alias kservice='gcloud run services describe karaoke-backend --project=nomadkaraoke --region=us-central1'
alias kqueues='gcloud tasks queues list --project=nomadkaraoke --location=us-central1'
# Usage:
# kj ffb0b8fa # Get job data
# klogs | grep ffb0b8fa # Search logs
# kfiles ffb0b8fa/ # List job files
```
---
## 8. API Access (Alternative)
The backend API also provides job information if you have an access token.
### Get Job via API
```bash
export TOKEN="your-access-token"
curl -s -H "Authorization: Bearer $TOKEN" \
"https://api.nomadkaraoke.com/api/jobs/ffb0b8fa" | jq
```
### List Jobs
```bash
curl -s -H "Authorization: Bearer $TOKEN" \
"https://api.nomadkaraoke.com/api/jobs?limit=10" | jq
```
---
## Related Documentation
- [Architecture Overview](./ARCHITECTURE.md) - System architecture and data flow
- [Frontend Guide](./FRONTEND-GUIDE.md) - Web UI documentation
- [Backend Feature Parity](../00-current-plan/BACKEND-FEATURE-PARITY-PLAN.md) - Feature status
- [Scalable Architecture Plan](../00-current-plan/SCALABLE-ARCHITECTURE-PLAN.md) - Cloud Tasks and scaling
+459
View File
@@ -0,0 +1,459 @@
/**
* Unit tests for job-status.ts utility functions
*/
import { getJobStep, formatStepIndicator, isBlockingStatus, getJobProgressPercent, JobStep } from '../lib/job-status';
import type { Job } from '../lib/api';
// Helper to create a minimal Job object for testing
function createJob(status: string, stateData?: Record<string, any>): Job {
return {
job_id: 'test-job-123',
status,
progress: 0,
created_at: new Date().toISOString(),
updated_at: new Date().toISOString(),
state_data: stateData,
};
}
describe('getJobStep', () => {
describe('Step 1: Setup', () => {
it('returns step 1 for pending status', () => {
const job = createJob('pending');
const result = getJobStep(job);
expect(result.step).toBe(1);
expect(result.total).toBe(10);
expect(result.label).toBe('Setting up');
expect(result.isBlocking).toBe(false);
expect(result.color).toBe('text-slate-400');
});
});
describe('Step 2: Audio Search', () => {
it('returns step 2 for searching_audio status', () => {
const job = createJob('searching_audio');
const result = getJobStep(job);
expect(result.step).toBe(2);
expect(result.label).toBe('Searching for audio');
expect(result.isBlocking).toBe(false);
});
it('returns step 2 with blocking for awaiting_audio_selection', () => {
const job = createJob('awaiting_audio_selection');
const result = getJobStep(job);
expect(result.step).toBe(2);
expect(result.label).toBe('Select audio source');
expect(result.isBlocking).toBe(true);
expect(result.color).toBe('text-amber-400');
});
});
describe('Step 3: Download', () => {
it('returns step 3 for downloading_audio status', () => {
const job = createJob('downloading_audio');
const result = getJobStep(job);
expect(result.step).toBe(3);
expect(result.label).toBe('Downloading audio');
});
it('returns step 3 for downloading status without worker progress', () => {
const job = createJob('downloading');
const result = getJobStep(job);
expect(result.step).toBe(3);
expect(result.label).toBe('Downloading');
});
it('returns step 3 for downloading with empty state_data', () => {
const job = createJob('downloading', {});
const result = getJobStep(job);
expect(result.step).toBe(3);
expect(result.label).toBe('Downloading');
});
});
describe('Step 3→4 Transition: Downloading with active workers', () => {
// When backend status is "downloading" but workers are actively running,
// the frontend should show step 4 (Processing) to avoid appearing stuck
it('returns step 4 when downloading with active audio worker', () => {
const job = createJob('downloading', {
audio_progress: { stage: 'separating_stage1', progress: 10 },
});
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.color).toBe('text-purple-400');
});
it('returns step 4 when downloading with active lyrics worker', () => {
const job = createJob('downloading', {
lyrics_progress: { stage: 'transcribing', progress: 10 },
});
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.color).toBe('text-purple-400');
});
it('returns step 4 with combined label when both workers active', () => {
const job = createJob('downloading', {
audio_progress: { stage: 'separating_stage1', progress: 10 },
lyrics_progress: { stage: 'transcribing', progress: 10 },
});
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.label).toMatch(/Audio.*\+.*Transcribing|Transcribing.*\+.*Audio/);
});
it('shows only active worker when one is complete', () => {
const job = createJob('downloading', {
audio_progress: { stage: 'audio_complete' },
lyrics_progress: { stage: 'transcribing' },
audio_complete: true,
});
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.label).toBe('Transcribing');
});
it('shows processing complete when both workers done', () => {
const job = createJob('downloading', {
audio_progress: { stage: 'audio_complete' },
lyrics_progress: { stage: 'lyrics_complete' },
audio_complete: true,
lyrics_complete: true,
});
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.label).toBe('Processing complete');
});
it('returns step 3 when state_data has no stage info', () => {
// state_data exists but no actual progress stage set
const job = createJob('downloading', {
audio_progress: {},
lyrics_progress: {},
});
const result = getJobStep(job);
expect(result.step).toBe(3);
expect(result.label).toBe('Downloading');
});
});
describe('Step 4: Parallel Processing', () => {
it('returns step 4 for separating_stage1', () => {
const job = createJob('separating_stage1');
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.label).toBe('Separating audio (1/2)');
});
it('returns step 4 for separating_stage2', () => {
const job = createJob('separating_stage2');
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.label).toBe('Separating audio (2/2)');
});
it('returns step 4 for transcribing', () => {
const job = createJob('transcribing');
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.label).toBe('Transcribing lyrics');
});
it('returns step 4 for correcting', () => {
const job = createJob('correcting');
const result = getJobStep(job);
expect(result.step).toBe(4);
expect(result.label).toBe('Correcting lyrics');
});
it('shows combined progress when both audio and lyrics are in progress', () => {
const job = createJob('separating_stage1', {
audio_progress: { stage: 'separating_stage1' },
lyrics_progress: { stage: 'transcribing' },
});
const result = getJobStep(job);
expect(result.step).toBe(4);
// Should show combined label
expect(result.label).toMatch(/Audio|Transcribing/);
});
it('shows combined status when audio done but lyrics in progress', () => {
const job = createJob('lyrics_complete', {
audio_progress: { stage: 'audio_complete' },
lyrics_progress: { stage: 'transcribing' },
audio_complete: true,
});
const result = getJobStep(job);
expect(result.step).toBe(4);
});
});
describe('Step 5: Screen Generation', () => {
it('returns step 5 for generating_screens', () => {
const job = createJob('generating_screens');
const result = getJobStep(job);
expect(result.step).toBe(5);
expect(result.label).toBe('Generating screens');
});
it('returns step 5 for applying_padding', () => {
const job = createJob('applying_padding');
const result = getJobStep(job);
expect(result.step).toBe(5);
expect(result.label).toBe('Syncing countdown');
});
});
describe('Step 6: Lyrics Review (BLOCKING)', () => {
it('returns step 6 with blocking for awaiting_review', () => {
const job = createJob('awaiting_review');
const result = getJobStep(job);
expect(result.step).toBe(6);
expect(result.label).toBe('Review lyrics');
expect(result.isBlocking).toBe(true);
expect(result.color).toBe('text-amber-400');
});
it('returns step 6 for in_review', () => {
const job = createJob('in_review');
const result = getJobStep(job);
expect(result.step).toBe(6);
expect(result.label).toBe('In review');
expect(result.isBlocking).toBe(true);
});
});
describe('Step 7: Video Rendering', () => {
it('returns step 7 for review_complete', () => {
const job = createJob('review_complete');
const result = getJobStep(job);
expect(result.step).toBe(7);
expect(result.label).toBe('Starting render');
});
it('returns step 7 for rendering_video', () => {
const job = createJob('rendering_video');
const result = getJobStep(job);
expect(result.step).toBe(7);
expect(result.label).toBe('Rendering video');
});
});
describe('Step 8: Instrumental Selection (BLOCKING)', () => {
it('returns step 8 with blocking for awaiting_instrumental_selection', () => {
const job = createJob('awaiting_instrumental_selection');
const result = getJobStep(job);
expect(result.step).toBe(8);
expect(result.label).toBe('Select instrumental');
expect(result.isBlocking).toBe(true);
expect(result.color).toBe('text-amber-400');
});
});
describe('Step 9: Final Encoding', () => {
it('returns step 9 for instrumental_selected', () => {
const job = createJob('instrumental_selected');
const result = getJobStep(job);
expect(result.step).toBe(9);
expect(result.label).toBe('Starting final encode');
});
it('returns step 9 for generating_video', () => {
const job = createJob('generating_video');
const result = getJobStep(job);
expect(result.step).toBe(9);
expect(result.label).toBe('Generating final video');
});
it('returns step 9 for encoding', () => {
const job = createJob('encoding');
const result = getJobStep(job);
expect(result.step).toBe(9);
expect(result.label).toBe('Encoding video');
});
it('returns step 9 for packaging', () => {
const job = createJob('packaging');
const result = getJobStep(job);
expect(result.step).toBe(9);
expect(result.label).toBe('Packaging files');
});
});
describe('Step 10: Distribution / Complete', () => {
it('returns step 10 for uploading', () => {
const job = createJob('uploading');
const result = getJobStep(job);
expect(result.step).toBe(10);
expect(result.label).toBe('Uploading');
expect(result.color).toBe('text-green-400');
});
it('returns step 10 for notifying', () => {
const job = createJob('notifying');
const result = getJobStep(job);
expect(result.step).toBe(10);
expect(result.label).toBe('Sending notifications');
});
it('returns step 10 for complete', () => {
const job = createJob('complete');
const result = getJobStep(job);
expect(result.step).toBe(10);
expect(result.label).toBe('Complete');
expect(result.color).toBe('text-green-400');
});
it('returns step 10 for prep_complete', () => {
const job = createJob('prep_complete');
const result = getJobStep(job);
expect(result.step).toBe(10);
expect(result.label).toBe('Prep complete');
});
});
describe('Terminal states', () => {
it('returns step 0 for failed status', () => {
const job = createJob('failed');
const result = getJobStep(job);
expect(result.step).toBe(0);
expect(result.label).toBe('Failed');
expect(result.color).toBe('text-red-400');
});
it('returns step 0 for cancelled status', () => {
const job = createJob('cancelled');
const result = getJobStep(job);
expect(result.step).toBe(0);
expect(result.label).toBe('Cancelled');
expect(result.color).toBe('text-slate-500');
});
});
describe('Unknown status handling', () => {
it('returns step 0 with humanized label for unknown status', () => {
const job = createJob('some_unknown_status');
const result = getJobStep(job);
expect(result.step).toBe(0);
expect(result.label).toBe('some unknown status');
expect(result.isBlocking).toBe(false);
expect(result.color).toBe('text-slate-400');
});
it('handles null status gracefully', () => {
const job = createJob(null as unknown as string);
const result = getJobStep(job);
expect(result.step).toBe(1); // Falls back to 'pending'
});
it('handles undefined status gracefully', () => {
const job = createJob(undefined as unknown as string);
const result = getJobStep(job);
expect(result.step).toBe(1); // Falls back to 'pending'
});
});
describe('Case insensitivity', () => {
it('handles uppercase status', () => {
const job = createJob('PENDING');
const result = getJobStep(job);
expect(result.step).toBe(1);
});
it('handles mixed case status', () => {
const job = createJob('Downloading');
const result = getJobStep(job);
expect(result.step).toBe(3);
});
});
});
describe('formatStepIndicator', () => {
it('formats active step correctly', () => {
expect(formatStepIndicator(4, 10, 'Processing')).toBe('[4/10] Processing');
});
it('omits step number for terminal states (step 0)', () => {
expect(formatStepIndicator(0, 10, 'Failed')).toBe('Failed');
});
it('formats first step correctly', () => {
expect(formatStepIndicator(1, 10, 'Setting up')).toBe('[1/10] Setting up');
});
it('formats last step correctly', () => {
expect(formatStepIndicator(10, 10, 'Complete')).toBe('[10/10] Complete');
});
});
describe('isBlockingStatus', () => {
it('returns true for awaiting_review', () => {
expect(isBlockingStatus('awaiting_review')).toBe(true);
});
it('returns true for in_review', () => {
expect(isBlockingStatus('in_review')).toBe(true);
});
it('returns true for awaiting_audio_selection', () => {
expect(isBlockingStatus('awaiting_audio_selection')).toBe(true);
});
it('returns true for awaiting_instrumental_selection', () => {
expect(isBlockingStatus('awaiting_instrumental_selection')).toBe(true);
});
it('returns false for non-blocking statuses', () => {
expect(isBlockingStatus('pending')).toBe(false);
expect(isBlockingStatus('downloading')).toBe(false);
expect(isBlockingStatus('complete')).toBe(false);
expect(isBlockingStatus('failed')).toBe(false);
});
it('returns false for unknown statuses', () => {
expect(isBlockingStatus('unknown_status')).toBe(false);
});
it('handles case insensitivity', () => {
expect(isBlockingStatus('AWAITING_REVIEW')).toBe(true);
});
it('handles null/undefined gracefully', () => {
expect(isBlockingStatus(null as unknown as string)).toBe(false);
expect(isBlockingStatus(undefined as unknown as string)).toBe(false);
});
});
describe('getJobProgressPercent', () => {
it('returns 0 for terminal states (step 0)', () => {
const job = createJob('failed');
expect(getJobProgressPercent(job)).toBe(0);
});
it('returns 10% for step 1', () => {
const job = createJob('pending');
expect(getJobProgressPercent(job)).toBe(10);
});
it('returns 40% for step 4', () => {
const job = createJob('transcribing');
expect(getJobProgressPercent(job)).toBe(40);
});
it('returns 60% for step 6', () => {
const job = createJob('awaiting_review');
expect(getJobProgressPercent(job)).toBe(60);
});
it('returns 100% for step 10', () => {
const job = createJob('complete');
expect(getJobProgressPercent(job)).toBe(100);
});
it('returns 0 for unknown status', () => {
const job = createJob('unknown_status_xyz');
expect(getJobProgressPercent(job)).toBe(0);
});
});
+59 -31
View File
@@ -10,39 +10,64 @@ import { JobLogs } from "./JobLogs"
import { OutputLinks } from "./OutputLinks"
import { InstrumentalSelector } from "./InstrumentalSelector"
import { AudioSearchDialog } from "../audio-search/AudioSearchDialog"
import { getJobStep, formatStepIndicator, getJobProgressPercent } from "@/lib/job-status"
// Status text styling (subtle colored text instead of badges)
const statusConfig: Record<string, { textColor: string; label: string }> = {
pending: { textColor: "text-slate-400", label: "Pending" },
searching_audio: { textColor: "text-blue-400", label: "Searching..." },
awaiting_audio_selection: { textColor: "text-amber-400", label: "Needs audio" },
downloading: { textColor: "text-blue-400", label: "Downloading..." },
downloading_audio: { textColor: "text-blue-400", label: "Downloading..." },
separating_stage1: { textColor: "text-purple-400", label: "Separating 1/2..." },
separating_stage2: { textColor: "text-purple-400", label: "Separating 2/2..." },
audio_complete: { textColor: "text-purple-400", label: "Audio ready" },
transcribing: { textColor: "text-blue-400", label: "Transcribing..." },
correcting: { textColor: "text-blue-400", label: "Correcting..." },
lyrics_complete: { textColor: "text-teal-400", label: "Lyrics ready" },
generating_screens: { textColor: "text-cyan-400", label: "Generating screens..." },
awaiting_review: { textColor: "text-amber-400", label: "Needs review" },
in_review: { textColor: "text-blue-400", label: "In review..." },
review_complete: { textColor: "text-teal-400", label: "Review complete" },
rendering_video: { textColor: "text-indigo-400", label: "Rendering..." },
awaiting_instrumental_selection: { textColor: "text-amber-400", label: "Needs instrumental" },
instrumental_selected: { textColor: "text-pink-400", label: "Instrumental selected" },
generating_video: { textColor: "text-violet-400", label: "Generating video..." },
encoding: { textColor: "text-violet-400", label: "Encoding..." },
packaging: { textColor: "text-violet-400", label: "Packaging..." },
uploading: { textColor: "text-green-400", label: "Uploading..." },
complete: { textColor: "text-green-400", label: "Complete" },
failed: { textColor: "text-red-400", label: "Failed" },
cancelled: { textColor: "text-slate-500", label: "Cancelled" },
/**
* StatusIndicator component - Shows step-based progress with visual indicator
*
* Displays: "[4/10] Processing..." with colored text and optional progress bar
*/
function StatusIndicator({ job }: { job: Job }) {
const { step, total, label, isBlocking, color } = getJobStep(job)
const formattedStatus = formatStepIndicator(step, total, label)
return (
<span className="inline-flex items-center gap-1.5">
<span className={color}>{formattedStatus}</span>
{isBlocking && (
<span className="text-amber-400 text-[10px] font-medium uppercase tracking-wide">
Action needed
</span>
)}
</span>
)
}
function StatusText({ status }: { status: string }) {
const config = statusConfig[status] || { textColor: "text-slate-400", label: status }
return <span className={config.textColor}>{config.label}</span>
/**
* Progress bar component - Visual indicator of job progress
*/
function ProgressBar({ job }: { job: Job }) {
const progressPercent = getJobProgressPercent(job)
const { step, color } = getJobStep(job)
// Don't show progress bar for terminal states
if (step === 0 || job.status === "complete" || job.status === "prep_complete") {
return null
}
// Map color class to background color for the bar
const barColorMap: Record<string, string> = {
"text-slate-400": "bg-slate-400",
"text-blue-400": "bg-blue-400",
"text-purple-400": "bg-purple-400",
"text-teal-400": "bg-teal-400",
"text-cyan-400": "bg-cyan-400",
"text-amber-400": "bg-amber-400",
"text-indigo-400": "bg-indigo-400",
"text-pink-400": "bg-pink-400",
"text-violet-400": "bg-violet-400",
"text-green-400": "bg-green-400",
}
const barColor = barColorMap[color] || "bg-blue-400"
return (
<div className="w-full h-1 bg-slate-700/50 rounded-full mt-2 overflow-hidden">
<div
className={`h-full ${barColor} rounded-full transition-all duration-500 ease-out`}
style={{ width: `${progressPercent}%` }}
/>
</div>
)
}
interface JobCardProps {
@@ -164,9 +189,12 @@ export function JobCard({ job, onRefresh }: JobCardProps) {
{/* Meta row: ID, date, status */}
<p className="text-xs text-slate-500 mt-1">
<span className="text-slate-600">ID:</span> {job.job_id} <span className="text-slate-600">•</span> {createdAt} <span className="text-slate-600">•</span> <StatusText status={job.status} />
<span className="text-slate-600">ID:</span> {job.job_id} <span className="text-slate-600">•</span> {createdAt} <span className="text-slate-600">•</span> <StatusIndicator job={job} />
</p>
{/* Progress bar for active jobs */}
<ProgressBar job={job} />
{/* Error message if any */}
{job.error_message && (
<div className="mt-2 text-xs text-red-400 bg-red-500/10 rounded p-2 break-words">
+318
View File
@@ -0,0 +1,318 @@
import { test, expect, Page } from '@playwright/test';
/**
* Production E2E test - Single complete karaoke generation flow.
*
* This test runs against gen.nomadkaraoke.com and creates a REAL karaoke video.
*
* Run with:
* npx playwright test --config=playwright.production.config.ts --headed
*
* The test follows the complete user journey:
* 1. Select a theme
* 2. Enter artist/title and search
* 3. Select an audio source
* 4. Wait for processing
* 5. Complete lyrics review
* 6. Select instrumental
* 7. Wait for final render
* 8. Verify completion
*/
const ACCESS_TOKEN = process.env.KARAOKE_ACCESS_TOKEN;
const PROD_URL = 'https://gen.nomadkaraoke.com';
const API_URL = 'https://api.nomadkaraoke.com';
// Real song - uses cached flacfetch results
const TEST_ARTIST = 'piri';
const TEST_TITLE = 'dog';
async function authenticatePage(page: Page) {
if (!ACCESS_TOKEN) {
throw new Error('KARAOKE_ACCESS_TOKEN environment variable is required');
}
await page.addInitScript((token) => {
localStorage.setItem('karaoke_access_token', token);
}, ACCESS_TOKEN);
}
test.describe('Production E2E', () => {
test('Complete karaoke generation flow', async ({ page, request }) => {
test.setTimeout(900000); // 15 minutes
await authenticatePage(page);
// ========== STEP 1: Go to homepage ==========
console.log('\n=== STEP 1: Loading homepage ===');
await page.goto(PROD_URL);
await page.waitForLoadState('networkidle');
await expect(page.locator('h1')).toContainText('Karaoke Generator');
console.log('Homepage loaded');
// ========== STEP 2: Go to Search tab and select theme ==========
console.log('\n=== STEP 2: Search tab + theme selection ===');
await page.getByRole('tab', { name: /search/i }).click();
await page.waitForTimeout(1000);
// Select theme (first one in dropdown)
const themeSelect = page.locator('button[role="combobox"]').first();
if (await themeSelect.isVisible({ timeout: 3000 })) {
const currentTheme = await themeSelect.textContent();
console.log(`Current theme: ${currentTheme}`);
if (currentTheme?.toLowerCase().includes('select')) {
await themeSelect.click();
await page.waitForTimeout(500);
await page.locator('[role="option"]').first().click();
await page.waitForTimeout(500);
console.log('Selected first theme');
}
}
// ========== STEP 3: Fill in artist/title and search ==========
console.log('\n=== STEP 3: Filling form and searching ===');
await page.getByLabel('Artist').fill(TEST_ARTIST);
await page.getByLabel('Title').fill(TEST_TITLE);
console.log(`Searching for: ${TEST_ARTIST} - ${TEST_TITLE}`);
await page.screenshot({ path: 'test-results/step3-before-search.png' });
await page.getByRole('button', { name: /search.*create/i }).click();
// Wait for search to complete (loading spinner -> results)
console.log('Waiting for search results...');
await page.waitForTimeout(10000);
await page.screenshot({ path: 'test-results/step3-after-search.png' });
// ========== STEP 4: Select audio source ==========
console.log('\n=== STEP 4: Selecting audio source ===');
// The search creates a job and shows audio options
// Look for "Select Audio" button on the job card
const selectAudioBtn = page.getByRole('button', { name: /select audio/i }).first();
if (await selectAudioBtn.isVisible({ timeout: 30000 })) {
console.log('Found "Select Audio" button, clicking...');
await selectAudioBtn.click();
await page.waitForTimeout(2000);
// A dialog should open with audio options
const dialog = page.locator('[role="dialog"]');
if (await dialog.isVisible({ timeout: 5000 })) {
await page.screenshot({ path: 'test-results/step4-audio-dialog.png' });
// Find all "Select" buttons in the dialog
const selectButtons = dialog.getByRole('button', { name: /^select$/i });
const count = await selectButtons.count();
console.log(`Found ${count} audio options`);
if (count > 0) {
console.log('Clicking first audio option...');
await selectButtons.first().click();
// Wait for dialog to close and download to start
await page.waitForTimeout(5000);
console.log('Audio selected, download should start');
} else {
console.log('ERROR: No Select buttons found in dialog');
const dialogContent = await dialog.textContent();
console.log('Dialog content:', dialogContent?.substring(0, 500));
}
} else {
console.log('ERROR: Dialog did not open');
}
} else {
console.log('ERROR: "Select Audio" button not found');
// Maybe job auto-started? Check job status
const pageContent = await page.locator('body').textContent();
console.log('Page contains:', pageContent?.substring(0, 500));
}
await page.screenshot({ path: 'test-results/step4-after-audio-select.png' });
// ========== STEP 5: Wait for processing and lyrics review ==========
console.log('\n=== STEP 5: Waiting for job to reach review stage ===');
// Poll the jobs API to check status
let jobId: string | null = null;
let jobStatus: string | null = null;
for (let i = 0; i < 60; i++) { // 5 minutes of polling
const response = await request.get(`${API_URL}/api/jobs`, {
headers: { 'Authorization': `Bearer ${ACCESS_TOKEN}` },
timeout: 120000 // 2 minute timeout for API calls
});
if (response.ok()) {
const data = await response.json();
const jobs = data.jobs || data;
// Find our job
const ourJob = Array.isArray(jobs)
? jobs.find((j: any) =>
j.artist?.toLowerCase() === TEST_ARTIST.toLowerCase() &&
j.title?.toLowerCase() === TEST_TITLE.toLowerCase())
: null;
if (ourJob) {
jobId = ourJob.job_id;
jobStatus = ourJob.status;
console.log(`Job ${jobId}: ${jobStatus}`);
// Check if we've reached a milestone
if (jobStatus === 'in_review') {
console.log('Job reached lyrics review stage!');
break;
} else if (jobStatus === 'awaiting_instrumental_selection') {
console.log('Job skipped review, at instrumental selection');
break;
} else if (jobStatus === 'completed') {
console.log('Job already completed!');
break;
} else if (jobStatus === 'failed') {
throw new Error(`Job failed: ${ourJob.error_message || 'Unknown error'}`);
}
}
}
// Refresh the page periodically
if (i % 6 === 0) {
await page.getByRole('button', { name: /refresh/i }).click().catch(() => {});
}
await page.waitForTimeout(5000);
}
if (!jobId) {
throw new Error('Could not find our job after 5 minutes');
}
await page.screenshot({ path: 'test-results/step5-job-status.png' });
// ========== STEP 6: Complete lyrics review (if needed) ==========
if (jobStatus === 'in_review') {
console.log('\n=== STEP 6: Completing lyrics review ===');
await page.reload();
await page.waitForLoadState('networkidle');
const reviewBtn = page.getByRole('button', { name: /review.*lyrics/i }).first();
if (await reviewBtn.isVisible({ timeout: 10000 })) {
console.log('Opening lyrics review...');
// This opens a new tab
const [popup] = await Promise.all([
page.context().waitForEvent('page', { timeout: 15000 }).catch(() => null),
reviewBtn.click()
]);
if (popup) {
await popup.waitForLoadState('networkidle');
await popup.waitForTimeout(5000);
await popup.screenshot({ path: 'test-results/step6-review-ui.png', fullPage: true });
// Click Preview Video
const previewBtn = popup.getByRole('button', { name: /preview.*video/i }).first();
if (await previewBtn.isVisible({ timeout: 5000 })) {
console.log('Clicking Preview Video...');
await previewBtn.click();
await popup.waitForTimeout(30000); // Wait for preview to generate
// Click Complete Review
const completeBtn = popup.getByRole('button', { name: /complete.*review/i }).first();
if (await completeBtn.isVisible({ timeout: 60000 })) {
console.log('Clicking Complete Review...');
await completeBtn.click();
await popup.waitForTimeout(3000);
}
}
if (!popup.isClosed()) {
await popup.close();
}
}
}
}
// ========== STEP 7: Handle instrumental selection (if needed) ==========
console.log('\n=== STEP 7: Checking for instrumental selection ===');
// Wait and check status
for (let i = 0; i < 12; i++) { // 1 minute
const response = await request.get(`${API_URL}/api/jobs/${jobId}`, {
headers: { 'Authorization': `Bearer ${ACCESS_TOKEN}` },
timeout: 120000
});
if (response.ok()) {
const job = await response.json();
console.log(`Job status: ${job.status}`);
if (job.status === 'awaiting_instrumental_selection') {
console.log('Handling instrumental selection...');
await page.reload();
await page.waitForLoadState('networkidle');
const instrumentalBtn = page.getByRole('button', { name: /select.*instrumental/i }).first();
if (await instrumentalBtn.isVisible({ timeout: 10000 })) {
await instrumentalBtn.click();
await page.waitForTimeout(2000);
const dialog = page.locator('[role="dialog"]');
if (await dialog.isVisible()) {
await page.screenshot({ path: 'test-results/step7-instrumental.png' });
// Select clean instrumental
const cleanBtn = dialog.getByRole('button').first();
await cleanBtn.click();
await page.waitForTimeout(3000);
console.log('Selected instrumental');
}
}
break;
} else if (job.status === 'completed' || job.status === 'processing') {
break;
}
}
await page.waitForTimeout(5000);
}
// ========== STEP 8: Wait for completion ==========
console.log('\n=== STEP 8: Waiting for completion ===');
for (let i = 0; i < 60; i++) { // 5 minutes
const response = await request.get(`${API_URL}/api/jobs/${jobId}`, {
headers: { 'Authorization': `Bearer ${ACCESS_TOKEN}` },
timeout: 120000
});
if (response.ok()) {
const job = await response.json();
console.log(`Job status: ${job.status}`);
if (job.status === 'completed') {
console.log('\n🎉 JOB COMPLETED SUCCESSFULLY!');
console.log(`Theme ID: ${job.theme_id}`);
console.log(`YouTube URL: ${job.youtube_url || 'N/A'}`);
console.log(`Dropbox URL: ${job.dropbox_url || 'N/A'}`);
// Final verification
expect(job.theme_id).toBeTruthy();
await page.reload();
await page.screenshot({ path: 'test-results/step8-completed.png', fullPage: true });
return; // SUCCESS
} else if (job.status === 'failed') {
throw new Error(`Job failed: ${job.error_message || 'Unknown error'}`);
}
}
await page.waitForTimeout(5000);
}
throw new Error('Job did not complete within timeout');
});
});
@@ -1,874 +0,0 @@
import { test, expect, Page, ConsoleMessage, Request, Response } from '@playwright/test';
/**
* Production E2E tests for the complete karaoke generation journey.
*
* These tests run directly against gen.nomadkaraoke.com (production).
*
* IMPORTANT: Set KARAOKE_ACCESS_TOKEN environment variable:
* KARAOKE_ACCESS_TOKEN=your-token npx playwright test --config=playwright.production.config.ts
*
* The complete user journey:
* 1. Enter artist name and song title
* 2. Select audio source from search results
* 3. Wait for audio download and preparation
* 4. Review/correct lyrics in the review UI
* 5. Submit the review
* 6. Wait for render stage
* 7. Select instrumental (clean or with backing vocals)
* 8. Wait for final render and distribution
* 9. Verify YouTube upload and Dropbox folder links
*/
const ACCESS_TOKEN = process.env.KARAOKE_ACCESS_TOKEN;
const PROD_URL = 'https://gen.nomadkaraoke.com';
const API_URL = 'https://api.nomadkaraoke.com';
// Test song - using something short and reliable
const TEST_ARTIST = 'LCMDF';
const TEST_TITLE = 'Take Me To The Mountains';
// Helper to authenticate the page by setting localStorage token
async function authenticatePage(page: Page) {
if (!ACCESS_TOKEN) {
throw new Error('KARAOKE_ACCESS_TOKEN environment variable is required for production tests');
}
// Set the token in localStorage before navigating
await page.addInitScript((token) => {
localStorage.setItem('karaoke_access_token', token);
}, ACCESS_TOKEN);
return true;
}
// Network and console logging for debugging
function setupLogging(page: Page) {
const logs: { type: string; url?: string; status?: number; text?: string; body?: any }[] = [];
page.on('request', (request: Request) => {
if (request.url().includes('/api/')) {
console.log(`>> ${request.method()} ${request.url()}`);
logs.push({ type: 'request', url: request.url() });
}
});
page.on('response', async (response: Response) => {
if (response.url().includes('/api/')) {
console.log(`<< ${response.status()} ${response.url()}`);
logs.push({ type: 'response', url: response.url(), status: response.status() });
if (response.status() >= 400) {
try {
const body = await response.json();
console.log(' Error:', JSON.stringify(body));
logs.push({ type: 'error', body });
} catch { /* ignore */ }
}
}
});
page.on('console', (msg: ConsoleMessage) => {
if (msg.type() === 'error') {
console.log(`[Console Error] ${msg.text()}`);
logs.push({ type: 'console-error', text: msg.text() });
}
});
return logs;
}
// Wait for a job to reach a specific status with progress logging
async function waitForJobStatus(
page: Page,
targetStatuses: string[],
timeoutMs: number = 600000, // 10 minutes default
pollIntervalMs: number = 5000
): Promise<{ status: string; jobId?: string }> {
const startTime = Date.now();
let lastStatus = '';
while (Date.now() - startTime < timeoutMs) {
// Find job cards with status badges
const badges = await page.locator('[class*="rounded-lg"][class*="border"]').locator('span, [class*="Badge"]').allTextContents();
const allText = badges.join(' ').toLowerCase();
// Check for target statuses
for (const status of targetStatuses) {
if (allText.includes(status.toLowerCase())) {
console.log(`Found target status: ${status}`);
return { status };
}
}
// Check for failure states
if (allText.includes('failed')) {
// Get error details
const errorDetails = await page.locator('[class*="text-red"]').allTextContents();
throw new Error(`Job failed: ${errorDetails.join(', ')}`);
}
if (allText.includes('cancelled')) {
throw new Error('Job was cancelled');
}
// Log progress updates
const currentBadges = badges.filter(b => b.trim()).join(', ');
if (currentBadges !== lastStatus) {
console.log(`Current status: ${currentBadges}`);
lastStatus = currentBadges;
}
// Click refresh and wait
const refreshBtn = page.getByRole('button', { name: /refresh/i });
if (await refreshBtn.isVisible()) {
await refreshBtn.click();
}
await page.waitForTimeout(pollIntervalMs);
}
throw new Error(`Timeout waiting for status: ${targetStatuses.join(' or ')}`);
}
test.describe('Production Full Karaoke Generation Journey', () => {
test.beforeEach(async ({ page }, testInfo) => {
// Ensure we have authentication
await authenticatePage(page);
setupLogging(page);
});
test('Step 1: Homepage loads and shows interface', async ({ page }) => {
await page.goto(PROD_URL);
await page.waitForLoadState('networkidle');
// Screenshot for visual verification
await page.screenshot({ path: 'test-results/prod-01-homepage.png', fullPage: true });
// Verify main elements
await expect(page.locator('h1')).toContainText('Karaoke Generator');
await expect(page.getByRole('tab', { name: /search/i })).toBeVisible();
await expect(page.locator('text=Recent Jobs')).toBeVisible();
console.log('Homepage loaded successfully');
});
test('Step 2: Can search for audio and see results', async ({ page }) => {
test.setTimeout(180000); // 3 minutes - search can be slow
await page.goto(PROD_URL);
await page.waitForLoadState('networkidle');
// Click Search tab
await page.getByRole('tab', { name: /search/i }).click();
await page.waitForTimeout(500);
// Fill in artist and title
await page.getByLabel('Artist').fill(TEST_ARTIST);
await page.getByLabel('Title').fill(TEST_TITLE);
await page.screenshot({ path: 'test-results/prod-02-search-form.png' });
// Submit search
console.log(`Searching for: ${TEST_ARTIST} - ${TEST_TITLE}`);
const searchButton = page.getByRole('button', { name: /search.*create/i });
await searchButton.click();
// Wait for response - could be quick or take a while
await page.waitForTimeout(5000);
// Check for loading state or error
const errorDiv = page.locator('[class*="text-red"]').first();
const hasError = await errorDiv.isVisible().catch(() => false);
if (hasError) {
const errorText = await errorDiv.textContent();
console.log(`Search error: ${errorText}`);
await page.screenshot({ path: 'test-results/prod-02-search-error.png' });
// Don't fail - document the error state
} else {
// Should see either results or job created
await page.screenshot({ path: 'test-results/prod-02-search-submitted.png' });
console.log('Search request submitted');
}
});
test('Step 3: Full journey - Search to Audio Selection', async ({ page }) => {
test.setTimeout(300000); // 5 minutes
await page.goto(PROD_URL);
await page.waitForLoadState('networkidle');
// First check if there's already a job awaiting audio selection
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(2000);
let jobCard = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /select audio/i
}).first();
if (!await jobCard.isVisible()) {
// Create new job via search
console.log('Creating new job via search...');
await page.getByRole('tab', { name: /search/i }).click();
await page.getByLabel('Artist').fill(TEST_ARTIST);
await page.getByLabel('Title').fill(TEST_TITLE);
await page.getByRole('button', { name: /search.*create/i }).click();
// Wait for job to appear and reach audio selection state
await page.waitForTimeout(10000);
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(3000);
jobCard = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /select audio/i
}).first();
}
if (await jobCard.isVisible()) {
console.log('Found job awaiting audio selection');
await jobCard.click();
await page.waitForTimeout(1000);
await page.screenshot({ path: 'test-results/prod-03-job-expanded.png' });
// Open audio selection dialog
const selectAudioBtn = page.getByRole('button', { name: /select audio/i });
if (await selectAudioBtn.isVisible()) {
await selectAudioBtn.click();
await page.waitForTimeout(2000);
const dialog = page.locator('[role="dialog"]');
if (await dialog.isVisible()) {
await page.screenshot({ path: 'test-results/prod-03-audio-dialog.png', fullPage: true });
// Count audio options
const selectButtons = dialog.getByRole('button', { name: /^select$/i });
const optionCount = await selectButtons.count();
console.log(`Found ${optionCount} audio options`);
if (optionCount > 0) {
// Select first option
console.log('Selecting first audio option...');
await selectButtons.first().click();
// Wait for dialog to close (download may take time)
const closed = await dialog.waitFor({ state: 'hidden', timeout: 120000 }).then(() => true).catch(() => false);
if (closed) {
console.log('Audio selection completed');
} else {
console.log('Dialog still open - may be processing');
await page.keyboard.press('Escape');
}
await page.screenshot({ path: 'test-results/prod-03-after-selection.png' });
}
}
}
} else {
console.log('No job awaiting audio selection');
await page.screenshot({ path: 'test-results/prod-03-no-audio-job.png' });
}
});
test('Step 4: Full journey - Lyrics Review', async ({ page }) => {
test.setTimeout(600000); // 10 minutes - review can take time
await page.goto(PROD_URL);
await page.waitForLoadState('networkidle');
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(2000);
// Look for job awaiting review
const reviewJob = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /awaiting review|review lyrics/i
}).first();
if (await reviewJob.isVisible()) {
console.log('Found job awaiting lyrics review');
await reviewJob.click();
await page.waitForTimeout(1000);
await page.screenshot({ path: 'test-results/prod-04-review-job.png' });
// Click review lyrics button (use .first() since there may be multiple jobs with this button)
const reviewBtn = page.getByRole('button', { name: /review lyrics/i }).or(page.getByRole('link', { name: /review lyrics/i })).first();
if (await reviewBtn.isVisible()) {
console.log('Opening lyrics review UI...');
// The review button opens a new tab with the LyricsTranscriber UI
const [popup] = await Promise.all([
page.context().waitForEvent('page', { timeout: 15000 }).catch(() => null),
reviewBtn.click()
]);
if (popup) {
await popup.waitForLoadState('networkidle', { timeout: 60000 }).catch(() => {
console.log('Review UI network idle timeout - continuing anyway');
});
await popup.screenshot({ path: 'test-results/prod-04-review-ui.png', fullPage: true });
console.log(`Review UI opened: ${popup.url()}`);
// Wait for the review UI to fully load
await popup.waitForTimeout(5000);
// The LyricsTranscriber review UI should have:
// - A preview video player
// - Lyrics display with timing
// - Edit controls
// - A submit/complete button
// First, let's see what buttons are available
const allButtons = await popup.getByRole('button').allTextContents();
console.log(`Available buttons in review UI: ${allButtons.join(', ')}`);
// Take a screenshot after loading
await popup.screenshot({ path: 'test-results/prod-04-review-ui-loaded.png', fullPage: true });
// The review flow in LyricsTranscriber is:
// 1. Click "Preview Video" button to generate preview
// 2. Wait for preview to generate (this opens a modal)
// 3. Click "Complete Review" in the modal to submit
// Step 1: Click "Preview Video" button
const previewBtn = popup.locator('button:has-text("Preview Video")').first();
if (await previewBtn.isVisible({ timeout: 5000 }).catch(() => false)) {
console.log('Found "Preview Video" button, clicking...');
await previewBtn.click();
// Wait for preview to generate (can take up to 30-60 seconds)
console.log('Waiting for preview video to generate...');
await popup.waitForTimeout(10000);
await popup.screenshot({ path: 'test-results/prod-04-preview-generating.png' });
// Step 2: Look for "Complete Review" button (appears after preview is ready or in modal)
let submitted = false;
const maxWaitAttempts = 12; // Wait up to 2 minutes for preview
for (let i = 0; i < maxWaitAttempts && !submitted; i++) {
// Check if popup is still open
if (popup.isClosed()) {
console.log('Popup closed - review may have been auto-submitted');
submitted = true;
break;
}
// Look for Complete Review button
const completeBtn = popup.locator('button:has-text("Complete Review")').first();
if (await completeBtn.isVisible({ timeout: 5000 }).catch(() => false)) {
console.log('Found "Complete Review" button, clicking...');
await completeBtn.click();
// Clicking Complete Review closes the popup, so catch any errors
await popup.waitForTimeout(3000).catch(() => {
console.log('Popup closed after clicking Complete Review (expected)');
});
submitted = true;
console.log('Review submitted successfully!');
} else {
console.log(`Waiting for Complete Review button... (attempt ${i + 1}/${maxWaitAttempts})`);
await popup.waitForTimeout(10000).catch(() => {});
await popup.screenshot({ path: `test-results/prod-04-waiting-${i}.png` }).catch(() => {});
}
}
if (!submitted) {
console.log('Could not find "Complete Review" button after waiting');
const allButtons = await popup.getByRole('button').allTextContents().catch(() => []);
console.log(`Available buttons: ${allButtons.filter(b => b.trim()).join(', ')}`);
}
} else {
console.log('Preview Video button not found, trying direct Complete Review...');
// Fallback: Maybe the modal is already open
const completeBtn = popup.locator('button:has-text("Complete Review")').first();
if (await completeBtn.isVisible({ timeout: 5000 }).catch(() => false)) {
await completeBtn.click();
await popup.waitForTimeout(3000);
console.log('Review submitted via direct Complete Review button');
await popup.screenshot({ path: 'test-results/prod-04-after-submit.png' });
} else {
console.log('Could not find Complete Review button');
const allButtons = await popup.getByRole('button').allTextContents();
console.log(`Available buttons: ${allButtons.filter(b => b.trim()).join(', ')}`);
}
}
// Close popup if still open
if (!popup.isClosed()) {
await popup.close().catch(() => {});
}
// Refresh main page to check status
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(3000);
await page.screenshot({ path: 'test-results/prod-04-after-review.png' });
// Check if job progressed past review
const newStatus = await page.locator('[class*="rounded-lg"][class*="border"]').first().locator('span').allTextContents();
console.log(`Job status after review: ${newStatus.filter(s => s.trim()).join(', ')}`);
// Verify the job is no longer in "awaiting review" state
const pageTextAfter = await page.locator('body').textContent();
if (pageTextAfter?.toLowerCase().includes('awaiting review')) {
console.log('WARNING: Job still shows "awaiting review" after submission');
} else if (pageTextAfter?.toLowerCase().includes('review complete') ||
pageTextAfter?.toLowerCase().includes('rendering') ||
pageTextAfter?.toLowerCase().includes('select instrumental')) {
console.log('SUCCESS: Job progressed past review stage!');
}
} else {
// No popup - might have navigated in same page
console.log('Review button did not open popup, checking current page...');
await page.waitForTimeout(2000);
await page.screenshot({ path: 'test-results/prod-04-review-same-page.png' });
}
} else {
console.log('Review button not visible');
}
} else {
console.log('No jobs awaiting lyrics review');
await page.screenshot({ path: 'test-results/prod-04-no-review-job.png' });
}
});
test('Step 5: Full journey - Instrumental Selection', async ({ page }) => {
test.setTimeout(300000); // 5 minutes
await page.goto(PROD_URL);
await page.waitForLoadState('networkidle');
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(2000);
// Look for job awaiting instrumental selection
const instrumentalJob = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /select instrumental/i
}).first();
if (await instrumentalJob.isVisible()) {
console.log('Found job awaiting instrumental selection');
await instrumentalJob.click();
await page.waitForTimeout(1000);
await page.screenshot({ path: 'test-results/prod-05-instrumental-job.png' });
// Click select instrumental button (use .first() since there may be multiple jobs with this button)
const selectBtn = page.getByRole('button', { name: /select instrumental/i }).first();
if (await selectBtn.isVisible()) {
await selectBtn.click();
await page.waitForTimeout(2000);
const dialog = page.locator('[role="dialog"]');
if (await dialog.isVisible()) {
await page.screenshot({ path: 'test-results/prod-05-instrumental-dialog.png' });
// Find and click "Clean" option (usually preferred)
const cleanOption = dialog.getByRole('button', { name: /clean|select/i }).first();
if (await cleanOption.isVisible()) {
console.log('Selecting clean instrumental...');
await cleanOption.click();
await dialog.waitFor({ state: 'hidden', timeout: 30000 }).catch(() => {
console.log('Dialog still open after selection');
});
await page.screenshot({ path: 'test-results/prod-05-after-instrumental.png' });
console.log('Instrumental selection completed');
}
}
}
} else {
console.log('No jobs awaiting instrumental selection');
await page.screenshot({ path: 'test-results/prod-05-no-instrumental-job.png' });
}
});
test('Step 6: Verify completed job - Downloads and Distribution', async ({ page }) => {
test.setTimeout(180000); // 3 minutes
await page.goto(PROD_URL);
await page.waitForLoadState('networkidle');
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(2000);
// Look for completed job
const completedJob = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /complete/i
}).first();
if (await completedJob.isVisible()) {
console.log('Found completed job');
await completedJob.click();
await page.waitForTimeout(2000);
await page.screenshot({ path: 'test-results/prod-06-completed-job.png', fullPage: true });
// Look for download links
const downloadSection = page.locator('text=Download').first();
if (await downloadSection.isVisible()) {
console.log('Downloads section visible');
}
// Look for YouTube link
const youtubeLink = page.locator('a[href*="youtube.com"], a[href*="youtu.be"]').first();
if (await youtubeLink.isVisible()) {
const href = await youtubeLink.getAttribute('href');
console.log(`YouTube link found: ${href}`);
} else {
console.log('No YouTube link found');
}
// Look for Dropbox link
const dropboxLink = page.locator('a[href*="dropbox.com"]').first();
if (await dropboxLink.isVisible()) {
const href = await dropboxLink.getAttribute('href');
console.log(`Dropbox link found: ${href}`);
} else {
console.log('No Dropbox link found');
}
// Check for video download buttons
const videoDownloads = page.locator('a:has-text("720p"), a:has-text("4K"), button:has-text("Download")');
const downloadCount = await videoDownloads.count();
console.log(`Found ${downloadCount} download options`);
} else {
console.log('No completed jobs found');
await page.screenshot({ path: 'test-results/prod-06-no-completed-job.png' });
}
});
test('Full E2E Journey - Complete track generation', async ({ page }) => {
/**
* FULL END-TO-END TEST
*
* This test goes through the entire journey:
* 1. Search for a song
* 2. Select audio
* 3. Wait for preparation
* 4. Review lyrics (or auto-accept in auto mode)
* 5. Select instrumental
* 6. Wait for render
* 7. Verify distribution (YouTube, Dropbox)
*
* WARNING: This creates a real karaoke track on production!
* Only run when you want to test the full system.
*/
test.setTimeout(900000); // 15 minutes - full generation takes time
// Use auto mode for non-interactive flow
await page.goto(`${PROD_URL}?auto=true`);
await page.waitForLoadState('networkidle');
// Verify auto mode banner
const autoBanner = page.locator('text=Non-Interactive Mode');
if (await autoBanner.isVisible()) {
console.log('Auto mode enabled');
} else {
console.log('Auto mode may not have activated, continuing...');
}
await page.screenshot({ path: 'test-results/prod-e2e-01-start.png' });
// Step 1: Search for song
console.log('=== Step 1: Searching for song ===');
await page.getByRole('tab', { name: /search/i }).click();
await page.getByLabel('Artist').fill(TEST_ARTIST);
await page.getByLabel('Title').fill(TEST_TITLE);
// Generate unique identifier to track our job
const timestamp = Date.now();
console.log(`Test timestamp: ${timestamp}`);
await page.getByRole('button', { name: /search.*create/i }).click();
console.log('Search submitted, waiting for job creation...');
// Wait for job to appear
await page.waitForTimeout(15000);
await page.click('button:has-text("Refresh")');
await page.screenshot({ path: 'test-results/prod-e2e-02-after-search.png' });
// Step 2-3: Audio selection (may happen automatically with search results, or need manual)
console.log('=== Step 2: Audio selection ===');
let needsAudioSelection = true;
let attempts = 0;
const maxAttempts = 30; // 2.5 minutes of checking
while (needsAudioSelection && attempts < maxAttempts) {
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(5000);
// Check current state
const pageText = await page.locator('body').textContent();
if (pageText?.toLowerCase().includes('select audio')) {
console.log('Job needs audio selection');
// Find and click the job card
const jobCard = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /select audio/i
}).first();
if (await jobCard.isVisible()) {
await jobCard.click();
await page.waitForTimeout(1000);
const selectAudioBtn = page.getByRole('button', { name: /select audio/i });
if (await selectAudioBtn.isVisible()) {
await selectAudioBtn.click();
await page.waitForTimeout(2000);
const dialog = page.locator('[role="dialog"]');
if (await dialog.isVisible()) {
const selectButtons = dialog.getByRole('button', { name: /^select$/i });
if (await selectButtons.count() > 0) {
console.log('Selecting first audio option');
await selectButtons.first().click();
await page.waitForTimeout(5000);
}
}
}
}
needsAudioSelection = false;
} else if (
pageText?.toLowerCase().includes('downloading') ||
pageText?.toLowerCase().includes('separating') ||
pageText?.toLowerCase().includes('transcribing') ||
pageText?.toLowerCase().includes('awaiting review') ||
pageText?.toLowerCase().includes('select instrumental') ||
pageText?.toLowerCase().includes('complete')
) {
console.log('Job has progressed past audio selection');
needsAudioSelection = false;
}
attempts++;
}
await page.screenshot({ path: 'test-results/prod-e2e-03-after-audio.png' });
// Step 4: Wait for processing and lyrics review
console.log('=== Step 3-4: Waiting for processing and review ===');
// Monitor progress and handle review stage
attempts = 0;
const maxProcessingAttempts = 120; // 10 minutes of processing
let reviewAttempts = 0;
const maxReviewAttempts = 3; // Try manual review completion up to 3 times
while (attempts < maxProcessingAttempts) {
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(5000);
const pageText = await page.locator('body').textContent();
if (pageText?.toLowerCase().includes('select instrumental')) {
console.log('Ready for instrumental selection');
break;
} else if (pageText?.toLowerCase().includes('complete')) {
console.log('Job completed!');
break;
} else if (pageText?.toLowerCase().includes('failed')) {
await page.screenshot({ path: 'test-results/prod-e2e-failed.png' });
throw new Error('Job failed during processing');
} else if (pageText?.toLowerCase().includes('awaiting review') || pageText?.toLowerCase().includes('review lyrics')) {
// Job is awaiting review - auto-mode should handle this, but let's verify
console.log('Job awaiting review - checking if auto-processing is working...');
await page.screenshot({ path: 'test-results/prod-e2e-awaiting-review.png' });
// Wait a bit for auto-processor to kick in
await page.waitForTimeout(10000);
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(2000);
// Check if still awaiting review
const stillAwaiting = await page.locator('body').textContent();
if (stillAwaiting?.toLowerCase().includes('awaiting review') && reviewAttempts < maxReviewAttempts) {
console.log(`Auto-processing not completing review. Attempting manual completion (attempt ${reviewAttempts + 1}/${maxReviewAttempts})...`);
// Find the job card and try to get the job ID
const jobCard = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /awaiting review/i
}).first();
if (await jobCard.isVisible()) {
// Click to expand the job
await jobCard.click();
await page.waitForTimeout(1000);
// Try to find and click "Review Lyrics" button to open review UI
const reviewBtn = page.getByRole('button', { name: /review lyrics/i }).or(page.getByRole('link', { name: /review lyrics/i })).first();
if (await reviewBtn.isVisible()) {
console.log('Opening review UI...');
// The review button opens a new tab with the LyricsTranscriber UI
const [popup] = await Promise.all([
page.context().waitForEvent('page', { timeout: 15000 }).catch(() => null),
reviewBtn.click()
]);
if (popup) {
console.log(`Review UI opened: ${popup.url()}`);
await popup.waitForLoadState('networkidle', { timeout: 30000 }).catch(() => {});
await popup.screenshot({ path: `test-results/prod-e2e-review-ui-${reviewAttempts}.png`, fullPage: true });
// Wait for the review UI to fully load
await popup.waitForTimeout(5000);
// The review flow: Click "Preview Video" -> Wait -> Click "Complete Review"
const previewBtn = popup.locator('button:has-text("Preview Video")').first();
if (await previewBtn.isVisible({ timeout: 5000 }).catch(() => false)) {
console.log('Clicking "Preview Video" button...');
await previewBtn.click();
await popup.waitForTimeout(10000);
// Wait for "Complete Review" button to appear
let reviewCompleted = false;
for (let i = 0; i < 6 && !reviewCompleted; i++) {
const completeBtn = popup.locator('button:has-text("Complete Review")').first();
if (await completeBtn.isVisible({ timeout: 5000 }).catch(() => false)) {
console.log('Clicking "Complete Review" button...');
await completeBtn.click();
await popup.waitForTimeout(3000);
reviewCompleted = true;
console.log('Review submitted via UI');
} else {
console.log(`Waiting for Complete Review... (${i + 1}/6)`);
await popup.waitForTimeout(10000);
}
}
if (!reviewCompleted) {
console.log('Could not complete review via UI');
const allButtons = await popup.getByRole('button').allTextContents();
console.log(`Available buttons: ${allButtons.filter(b => b.trim()).join(', ')}`);
}
} else {
console.log('Preview Video button not found');
}
await popup.close();
} else {
// No popup - might have navigated in same page
console.log('Review button did not open popup, checking current page...');
await page.waitForTimeout(2000);
}
}
}
reviewAttempts++;
}
}
// Log current stage
const badges = await page.locator('[class*="rounded-lg"][class*="border"]').first().locator('span').allTextContents();
console.log(`Current status: ${badges.filter(b => b.trim()).join(', ')}`);
attempts++;
}
await page.screenshot({ path: 'test-results/prod-e2e-04-after-processing.png' });
// Step 5: Instrumental selection
console.log('=== Step 5: Instrumental selection ===');
const instrumentalJob = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /select instrumental/i
}).first();
if (await instrumentalJob.isVisible()) {
await instrumentalJob.click();
await page.waitForTimeout(1000);
const selectBtn = page.getByRole('button', { name: /select instrumental/i }).first();
if (await selectBtn.isVisible()) {
await selectBtn.click();
await page.waitForTimeout(2000);
const dialog = page.locator('[role="dialog"]');
if (await dialog.isVisible()) {
// Select clean instrumental
const cleanBtn = dialog.locator('button:has-text("Clean"), button:has-text("Select")').first();
if (await cleanBtn.isVisible()) {
console.log('Selecting clean instrumental');
await cleanBtn.click();
await page.waitForTimeout(3000);
}
}
}
}
await page.screenshot({ path: 'test-results/prod-e2e-05-after-instrumental.png' });
// Step 6: Wait for final render and distribution
console.log('=== Step 6: Waiting for final render ===');
attempts = 0;
const maxRenderAttempts = 60; // 5 minutes
while (attempts < maxRenderAttempts) {
await page.click('button:has-text("Refresh")');
await page.waitForTimeout(5000);
const pageText = await page.locator('body').textContent();
if (pageText?.toLowerCase().includes('complete')) {
console.log('Job completed!');
break;
} else if (pageText?.toLowerCase().includes('failed')) {
await page.screenshot({ path: 'test-results/prod-e2e-render-failed.png' });
throw new Error('Job failed during rendering');
}
const badges = await page.locator('[class*="rounded-lg"][class*="border"]').first().locator('span').allTextContents();
console.log(`Render status: ${badges.filter(b => b.trim()).join(', ')}`);
attempts++;
}
await page.screenshot({ path: 'test-results/prod-e2e-06-final.png', fullPage: true });
// Step 7: Verify results
console.log('=== Step 7: Verifying results ===');
const completedJob = page.locator('[class*="rounded-lg"][class*="border"]').filter({
hasText: /complete/i
}).first();
if (await completedJob.isVisible()) {
await completedJob.click();
await page.waitForTimeout(2000);
await page.screenshot({ path: 'test-results/prod-e2e-07-completed-details.png', fullPage: true });
// Check for YouTube link
const youtubeLink = page.locator('a[href*="youtube.com"], a[href*="youtu.be"]').first();
if (await youtubeLink.isVisible()) {
const href = await youtubeLink.getAttribute('href');
console.log(`SUCCESS: YouTube link found: ${href}`);
} else {
console.log('WARNING: No YouTube link found');
}
// Check for Dropbox link
const dropboxLink = page.locator('a[href*="dropbox.com"]').first();
if (await dropboxLink.isVisible()) {
const href = await dropboxLink.getAttribute('href');
console.log(`SUCCESS: Dropbox link found: ${href}`);
} else {
console.log('WARNING: No Dropbox link found');
}
// Count download options
const downloads = page.locator('a:has-text("Download"), button:has-text("Download"), a:has-text("720p"), a:has-text("4K")');
const downloadCount = await downloads.count();
console.log(`Downloads available: ${downloadCount}`);
console.log('=== E2E TEST COMPLETE ===');
} else {
throw new Error('Job did not complete - check screenshots for debugging');
}
});
});
// API health check - quick sanity test
test('Production API health check', async ({ request }) => {
const response = await request.get(`${API_URL}/api/health`);
expect(response.ok()).toBeTruthy();
const data = await response.json();
console.log('API Health:', data);
expect(data.status).toBe('healthy');
});
+266
View File
@@ -0,0 +1,266 @@
/**
* Job Status Display Utilities
*
* Maps backend job statuses to user-friendly step-based progress indicators.
* The 10-step system simplifies 28+ backend statuses for better user comprehension.
*/
import type { Job } from "./api";
export interface JobStep {
step: number;
total: number;
label: string;
isBlocking: boolean;
color: string;
}
/**
* Status-to-step mapping configuration.
*
* Steps are organized as:
* 1. Setup
* 2. Audio Search (optional)
* 3. Download
* 4. Processing (Audio + Lyrics in parallel)
* 5. Screen Generation
* 6. Lyrics Review (BLOCKING)
* 7. Video Rendering
* 8. Instrumental Selection (BLOCKING)
* 9. Final Encoding
* 10. Distribution / Complete
*/
const STATUS_CONFIG: Record<
string,
{ step: number; label: string; isBlocking: boolean; color: string }
> = {
// Step 1: Setup
pending: { step: 1, label: "Setting up", isBlocking: false, color: "text-slate-400" },
// Step 2: Audio Search (optional path)
searching_audio: { step: 2, label: "Searching for audio", isBlocking: false, color: "text-blue-400" },
awaiting_audio_selection: { step: 2, label: "Select audio source", isBlocking: true, color: "text-amber-400" },
// Step 3: Download
downloading_audio: { step: 3, label: "Downloading audio", isBlocking: false, color: "text-blue-400" },
downloading: { step: 3, label: "Downloading", isBlocking: false, color: "text-blue-400" },
// Step 4: Parallel Processing (Audio + Lyrics)
separating_stage1: { step: 4, label: "Separating audio (1/2)", isBlocking: false, color: "text-purple-400" },
separating_stage2: { step: 4, label: "Separating audio (2/2)", isBlocking: false, color: "text-purple-400" },
audio_complete: { step: 4, label: "Audio ready, processing lyrics", isBlocking: false, color: "text-purple-400" },
transcribing: { step: 4, label: "Transcribing lyrics", isBlocking: false, color: "text-blue-400" },
correcting: { step: 4, label: "Correcting lyrics", isBlocking: false, color: "text-blue-400" },
lyrics_complete: { step: 4, label: "Lyrics ready, processing audio", isBlocking: false, color: "text-teal-400" },
// Step 5: Screen Generation
generating_screens: { step: 5, label: "Generating screens", isBlocking: false, color: "text-cyan-400" },
applying_padding: { step: 5, label: "Syncing countdown", isBlocking: false, color: "text-cyan-400" },
// Step 6: Review (BLOCKING - requires user action)
awaiting_review: { step: 6, label: "Review lyrics", isBlocking: true, color: "text-amber-400" },
in_review: { step: 6, label: "In review", isBlocking: true, color: "text-blue-400" },
// Step 7: Video Rendering
review_complete: { step: 7, label: "Starting render", isBlocking: false, color: "text-teal-400" },
rendering_video: { step: 7, label: "Rendering video", isBlocking: false, color: "text-indigo-400" },
// Step 8: Instrumental Selection (BLOCKING - requires user action)
awaiting_instrumental_selection: { step: 8, label: "Select instrumental", isBlocking: true, color: "text-amber-400" },
// Step 9: Final Encoding
instrumental_selected: { step: 9, label: "Starting final encode", isBlocking: false, color: "text-pink-400" },
generating_video: { step: 9, label: "Generating final video", isBlocking: false, color: "text-violet-400" },
encoding: { step: 9, label: "Encoding video", isBlocking: false, color: "text-violet-400" },
packaging: { step: 9, label: "Packaging files", isBlocking: false, color: "text-violet-400" },
// Step 10: Distribution / Complete
uploading: { step: 10, label: "Uploading", isBlocking: false, color: "text-green-400" },
notifying: { step: 10, label: "Sending notifications", isBlocking: false, color: "text-green-400" },
complete: { step: 10, label: "Complete", isBlocking: false, color: "text-green-400" },
prep_complete: { step: 10, label: "Prep complete", isBlocking: false, color: "text-green-400" },
// Terminal states (no step progression)
failed: { step: 0, label: "Failed", isBlocking: false, color: "text-red-400" },
cancelled: { step: 0, label: "Cancelled", isBlocking: false, color: "text-slate-500" },
};
const TOTAL_STEPS = 10;
/**
* Check if parallel workers (audio/lyrics) are actively running.
* This is indicated by state_data containing audio_progress or lyrics_progress.
*/
function isParallelProcessingActive(job: Job): boolean {
if (!job.state_data) return false;
const audioProgress = job.state_data.audio_progress as { stage?: string } | undefined;
const lyricsProgress = job.state_data.lyrics_progress as { stage?: string } | undefined;
// Workers are active if we have progress data with a stage
return !!(audioProgress?.stage || lyricsProgress?.stage);
}
/**
* Get the step information for a job based on its status.
*
* @param job - The job object with status and optional state_data
* @returns Step information including step number, label, and display properties
*/
export function getJobStep(job: Job): JobStep {
const status = job.status?.toLowerCase() || "pending";
const config = STATUS_CONFIG[status];
if (!config) {
// Unknown status - show generic processing state
return {
step: 0,
total: TOTAL_STEPS,
label: status.replace(/_/g, " "),
isBlocking: false,
color: "text-slate-400",
};
}
// Special case: "downloading" status but parallel workers are actually running.
// The backend sets status to "downloading" when audio download completes and workers start,
// but doesn't update to step 4 statuses until screens_worker runs.
// Show step 4 with detailed progress to avoid appearing "stuck" at downloading.
if (status === "downloading" && isParallelProcessingActive(job)) {
const enhancedLabel = getParallelProcessingLabel(job, "Processing");
return {
step: 4,
total: TOTAL_STEPS,
label: enhancedLabel,
isBlocking: false,
color: "text-purple-400", // Same as step 4 processing color
};
}
// During parallel processing (step 4), check state_data for more detail
if (config.step === 4 && job.state_data) {
const enhancedLabel = getParallelProcessingLabel(job, config.label);
return {
step: config.step,
total: TOTAL_STEPS,
label: enhancedLabel,
isBlocking: config.isBlocking,
color: config.color,
};
}
return {
step: config.step,
total: TOTAL_STEPS,
label: config.label,
isBlocking: config.isBlocking,
color: config.color,
};
}
/**
* Get enhanced label for parallel processing stage.
* Combines audio and lyrics progress when both are available.
*/
function getParallelProcessingLabel(job: Job, defaultLabel: string): string {
const audioProgress = job.state_data?.audio_progress as
| { stage?: string; message?: string }
| undefined;
const lyricsProgress = job.state_data?.lyrics_progress as
| { stage?: string; message?: string }
| undefined;
// If we have both progress indicators, show combined status
if (audioProgress && lyricsProgress) {
const audioStage = audioProgress.stage || "";
const lyricsStage = lyricsProgress.stage || "";
const audioDone = audioStage === "audio_complete" || job.state_data?.audio_complete;
const lyricsDone = lyricsStage === "lyrics_complete" || job.state_data?.lyrics_complete;
if (audioDone && lyricsDone) {
return "Processing complete";
}
// Show what's still running
const parts: string[] = [];
if (!audioDone) {
parts.push(getShortAudioStatus(audioStage));
}
if (!lyricsDone) {
parts.push(getShortLyricsStatus(lyricsStage));
}
if (parts.length > 0) {
return parts.join(" + ");
}
}
return defaultLabel;
}
function getShortAudioStatus(stage: string): string {
switch (stage) {
case "separating_stage1":
return "Audio 1/2";
case "separating_stage2":
return "Audio 2/2";
case "audio_complete":
return "Audio done";
default:
return "Audio";
}
}
function getShortLyricsStatus(stage: string): string {
switch (stage) {
case "transcribing":
return "Transcribing";
case "correcting":
return "Correcting";
case "lyrics_complete":
return "Lyrics done";
default:
return "Lyrics";
}
}
/**
* Format a step indicator string like "[4/10] Processing..."
*
* @param step - Current step number (0 for terminal states)
* @param total - Total number of steps
* @param label - Human-readable label
* @returns Formatted string
*/
export function formatStepIndicator(step: number, total: number, label: string): string {
if (step === 0) {
// Terminal states (failed, cancelled) don't show step numbers
return label;
}
return `[${step}/${total}] ${label}`;
}
/**
* Check if a job status requires user action.
*
* @param status - The job status string
* @returns true if the status is a blocking state requiring user action
*/
export function isBlockingStatus(status: string): boolean {
const config = STATUS_CONFIG[status?.toLowerCase()];
return config?.isBlocking ?? false;
}
/**
* Get the progress percentage for a job (0-100).
* This is based on step progression, not the backend progress field.
*
* @param job - The job object
* @returns Progress percentage (0-100)
*/
export function getJobProgressPercent(job: Job): number {
const { step, total } = getJobStep(job);
if (step === 0 || total === 0) return 0;
return Math.round((step / total) * 100);
}
+2 -2
View File
@@ -56,8 +56,8 @@ export default defineConfig({
// Screenshot on failure
screenshot: 'only-on-failure',
// Video on failure (helpful for debugging flows)
video: 'on-first-retry',
// Video recording (set to 'on' to always record, 'on-first-retry' for failures only)
video: 'on',
// Increase timeout for real API calls
actionTimeout: 30000,
+218
View File
@@ -264,6 +264,36 @@ class AudioFetcher(ABC):
"""
pass
@abstractmethod
def download_from_url(
self,
url: str,
output_dir: str,
output_filename: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
) -> AudioFetchResult:
"""
Download audio directly from a URL (e.g., YouTube URL).
This bypasses the search step and downloads directly from the provided URL.
Useful when the user provides a specific YouTube URL rather than artist/title.
Args:
url: The URL to download from (e.g., YouTube video URL)
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
artist: Optional artist name for metadata
title: Optional title for metadata
Returns:
AudioFetchResult with the downloaded file path
Raises:
DownloadError: If download fails
"""
pass
class FlacFetchAudioFetcher(AudioFetcher):
"""
@@ -645,6 +675,94 @@ class FlacFetchAudioFetcher(AudioFetcher):
except Exception as e:
raise DownloadError(f"Failed to download {artist} - {title}: {e}") from e
def download_from_url(
self,
url: str,
output_dir: str,
output_filename: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
) -> AudioFetchResult:
"""
Download audio directly from a URL (e.g., YouTube URL).
Uses flacfetch's download_by_id() method which supports direct YouTube downloads.
Args:
url: The URL to download from (e.g., YouTube video URL)
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
artist: Optional artist name for metadata
title: Optional title for metadata
Returns:
AudioFetchResult with the downloaded file path
Raises:
DownloadError: If download fails
"""
import re
manager = self._get_manager()
# Ensure output directory exists
os.makedirs(output_dir, exist_ok=True)
# Detect source type from URL
source_name = "YouTube" # Default to YouTube for now
source_id = None
# Extract YouTube video ID from URL
youtube_patterns = [
r'(?:youtube\.com/watch\?v=|youtu\.be/)([a-zA-Z0-9_-]{11})',
r'youtube\.com/embed/([a-zA-Z0-9_-]{11})',
r'youtube\.com/v/([a-zA-Z0-9_-]{11})',
]
for pattern in youtube_patterns:
match = re.search(pattern, url)
if match:
source_id = match.group(1)
break
if not source_id:
# For other URLs, use the full URL as the source_id
source_id = url
# Generate filename if not provided
if output_filename is None:
if artist and title:
output_filename = f"{artist} - {title}"
else:
output_filename = source_id
self.logger.info(f"Downloading from URL: {url}")
try:
filepath = manager.download_by_id(
source_name=source_name,
source_id=source_id,
output_path=output_dir,
output_filename=output_filename,
download_url=url, # Pass full URL for direct download
)
if not filepath:
raise DownloadError(f"Download returned no file path for URL: {url}")
self.logger.info(f"Downloaded to: {filepath}")
return AudioFetchResult(
filepath=filepath,
artist=artist or "",
title=title or "",
provider=source_name,
duration=None, # Could extract from yt-dlp info if needed
quality=None,
)
except Exception as e:
raise DownloadError(f"Failed to download from URL {url}: {e}") from e
def _interruptible_search(self, manager, query) -> list:
"""
Run search in a way that can be interrupted by Ctrl+C.
@@ -1548,6 +1666,106 @@ class RemoteFlacFetchAudioFetcher(AudioFetcher):
print("\nCancelled")
raise UserCancelledError("Selection cancelled by user (Ctrl+C)")
def download_from_url(
self,
url: str,
output_dir: str,
output_filename: Optional[str] = None,
artist: Optional[str] = None,
title: Optional[str] = None,
) -> AudioFetchResult:
"""
Download audio directly from a URL (e.g., YouTube URL).
For YouTube URLs, this uses local flacfetch since YouTube downloads
don't require the remote flacfetch infrastructure (no torrents).
Args:
url: The URL to download from (e.g., YouTube video URL)
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
artist: Optional artist name for metadata
title: Optional title for metadata
Returns:
AudioFetchResult with the downloaded file path
Raises:
DownloadError: If download fails
"""
import re
self.logger.info(f"[RemoteFlacFetcher] Downloading from URL: {url}")
self.logger.info(f"[RemoteFlacFetcher] Using local flacfetch for YouTube download (no remote API needed)")
try:
# Use local flacfetch for YouTube downloads - no need for remote API
# This avoids needing yt-dlp directly in karaoke-gen
from flacfetch.core.manager import FetchManager
from flacfetch.providers.youtube import YoutubeProvider
from flacfetch.downloaders.youtube import YoutubeDownloader
# Create a minimal local manager for YouTube downloads
manager = FetchManager()
manager.add_provider(YoutubeProvider())
manager.register_downloader("YouTube", YoutubeDownloader())
# Ensure output directory exists
os.makedirs(output_dir, exist_ok=True)
# Extract video ID from URL
source_id = None
youtube_patterns = [
r'(?:youtube\.com/watch\?v=|youtu\.be/)([a-zA-Z0-9_-]{11})',
r'youtube\.com/embed/([a-zA-Z0-9_-]{11})',
r'youtube\.com/v/([a-zA-Z0-9_-]{11})',
]
for pattern in youtube_patterns:
match = re.search(pattern, url)
if match:
source_id = match.group(1)
break
if not source_id:
source_id = url
# Generate filename if not provided
if output_filename is None:
if artist and title:
output_filename = f"{artist} - {title}"
else:
output_filename = source_id
# Use flacfetch's download_by_id for direct URL download
filepath = manager.download_by_id(
source_name="YouTube",
source_id=source_id,
output_path=output_dir,
output_filename=output_filename,
download_url=url,
)
if not filepath:
raise DownloadError(f"Download returned no file path for URL: {url}")
self.logger.info(f"[RemoteFlacFetcher] Downloaded to: {filepath}")
return AudioFetchResult(
filepath=filepath,
artist=artist or "",
title=title or "",
provider="YouTube",
duration=None,
quality=None,
)
except ImportError as e:
raise DownloadError(
f"flacfetch is required for URL downloads but import failed: {e}"
)
except Exception as e:
raise DownloadError(f"Failed to download from URL {url}: {e}") from e
# Alias for shorter name
RemoteFlacFetcher = RemoteFlacFetchAudioFetcher
+78 -25
View File
@@ -488,41 +488,56 @@ class KaraokePrep:
self.logger.info(f"Found existing media files matching extractor '{self.extractor}', skipping download/conversion.")
elif getattr(self, '_use_audio_fetcher', False):
# Use flacfetch to search and download audio
self.logger.info(f"Using flacfetch to search and download: {self.artist} - {self.title}")
try:
# Search and download audio using the AudioFetcher
fetch_result = self.audio_fetcher.search_and_download(
artist=self.artist,
title=self.title,
output_dir=track_output_dir,
output_filename=f"{artist_title} (flacfetch)",
auto_select=self.auto_download,
)
# Update extractor to reflect the actual provider used
self.extractor = f"flacfetch-{fetch_result.provider}"
# Check if this is a URL download or search+download
if getattr(self, '_use_url_download', False):
# Direct URL download (e.g., YouTube URL)
self.logger.info(f"Using flacfetch to download from URL: {self.url}")
fetch_result = self.audio_fetcher.download_from_url(
url=self.url,
output_dir=track_output_dir,
output_filename=f"{artist_title} (youtube)" if artist_title != "Unknown - Unknown" else None,
artist=self.artist,
title=self.title,
)
# Update extractor to reflect the source
self.extractor = f"youtube"
else:
# Use flacfetch to search and download audio
self.logger.info(f"Using flacfetch to search and download: {self.artist} - {self.title}")
fetch_result = self.audio_fetcher.search_and_download(
artist=self.artist,
title=self.title,
output_dir=track_output_dir,
output_filename=f"{artist_title} (flacfetch)",
auto_select=self.auto_download,
)
# Update extractor to reflect the actual provider used
self.extractor = f"flacfetch-{fetch_result.provider}"
# Set up the output paths
output_filename_no_extension = os.path.join(track_output_dir, f"{artist_title} ({self.extractor})")
# Copy/move the downloaded file to the expected location
processed_track["input_media"] = self.file_handler.download_audio_from_fetcher_result(
fetch_result.filepath, output_filename_no_extension
)
self.logger.info(f"Audio downloaded from {fetch_result.provider}: {processed_track['input_media']}")
# Convert to WAV for audio processing
self.logger.info("Converting downloaded audio to WAV for processing...")
processed_track["input_audio_wav"] = self.file_handler.convert_to_wav(
processed_track["input_media"], output_filename_no_extension
)
# No still image for audio-only downloads
processed_track["input_still_image"] = None
except UserCancelledError:
# User cancelled - propagate up to CLI for graceful exit
raise
@@ -1103,17 +1118,56 @@ class KaraokePrep:
return tracks
def _is_url(self, string: str) -> bool:
"""Check if a string is a URL."""
return string is not None and (string.startswith("http://") or string.startswith("https://"))
async def process(self):
if self.input_media is not None and os.path.isdir(self.input_media):
self.logger.info(f"Input media {self.input_media} is a local folder, processing each file individually...")
return await self.process_folder()
elif self.input_media is not None and os.path.isfile(self.input_media):
self.logger.info(f"Input media {self.input_media} is a local file, audio download will be skipped")
return [await self.prep_single_track()]
elif self.input_media is not None and self._is_url(self.input_media):
# URL provided - download directly via flacfetch
self.logger.info(f"Input media {self.input_media} is a URL, downloading via flacfetch...")
# Extract video ID for metadata if it's a YouTube URL
video_id = None
youtube_patterns = [
r'(?:youtube\.com/watch\?v=|youtu\.be/)([a-zA-Z0-9_-]{11})',
r'youtube\.com/embed/([a-zA-Z0-9_-]{11})',
r'youtube\.com/v/([a-zA-Z0-9_-]{11})',
]
for pattern in youtube_patterns:
match = re.search(pattern, self.input_media)
if match:
video_id = match.group(1)
break
# Set up the extracted_info for metadata consistency
self.extracted_info = {
"title": f"{self.artist} - {self.title}" if self.artist and self.title else video_id or "Unknown",
"artist": self.artist or "",
"track_title": self.title or "",
"extractor_key": "youtube",
"id": video_id or self.input_media,
"url": self.input_media,
"source": "youtube",
}
self.extractor = "youtube"
self.url = self.input_media
# Mark that we need to use audio fetcher for URL download
self._use_audio_fetcher = True
self._use_url_download = True # New flag for URL-based download
return [await self.prep_single_track()]
elif self.artist and self.title:
# No input file provided - use flacfetch to search and download audio
self.logger.info(f"No input file provided, using flacfetch to search for: {self.artist} - {self.title}")
# Set up the extracted_info for metadata consistency
self.extracted_info = {
"title": f"{self.artist} - {self.title}",
@@ -1126,13 +1180,12 @@ class KaraokePrep:
}
self.extractor = "flacfetch"
self.url = None # URL will be determined by flacfetch
# Mark that we need to use audio fetcher for download
self._use_audio_fetcher = True
return [await self.prep_single_track()]
else:
raise ValueError(
"Either a local file path or both artist and title must be provided. "
"URL-based input has been replaced with flacfetch audio fetching."
"Either a local file path, a URL, or both artist and title must be provided."
)
Generated
+12 -12
View File
@@ -1772,14 +1772,14 @@ readthedocs = ["sphinx-rtd-theme"]
[[package]]
name = "fastapi"
version = "0.127.0"
version = "0.128.0"
description = "FastAPI framework, high performance, easy to learn, fast to code, ready for production"
optional = false
python-versions = ">=3.9"
groups = ["main"]
files = [
{file = "fastapi-0.127.0-py3-none-any.whl", hash = "sha256:725aa2bb904e2eff8031557cf4b9b77459bfedd63cae8427634744fd199f6a49"},
{file = "fastapi-0.127.0.tar.gz", hash = "sha256:5a9246e03dcd1fdb19f1396db30894867c1d630f5107dc167dcbc5ed1ea7d259"},
{file = "fastapi-0.128.0-py3-none-any.whl", hash = "sha256:aebd93f9716ee3b4f4fcfe13ffb7cf308d99c9f3ab5622d8877441072561582d"},
{file = "fastapi-0.128.0.tar.gz", hash = "sha256:1cc179e1cef10a6be60ffe429f79b829dce99d8de32d7acb7e6c8dfdf7f2645a"},
]
[package.dependencies]
@@ -1873,14 +1873,14 @@ platformdirs = ">=4.3.6"
[[package]]
name = "flacfetch"
version = "0.11.0"
version = "0.15.0"
description = "Search and download high-quality audio from multiple sources"
optional = false
python-versions = ">=3.8"
groups = ["main"]
files = [
{file = "flacfetch-0.11.0-py3-none-any.whl", hash = "sha256:56657cba85cf6e390b4213d3f44d9fd05603138735d01dd10ebc1f695b5f77f3"},
{file = "flacfetch-0.11.0.tar.gz", hash = "sha256:56bbc3a66f1241c831490a88c51fcbfd5d5026108daac9d41f5a23c66c1f79b3"},
{file = "flacfetch-0.15.0-py3-none-any.whl", hash = "sha256:fa2eb155dd92113d37952fe0df5587ce472c85d5c924574342955c0968e802a1"},
{file = "flacfetch-0.15.0.tar.gz", hash = "sha256:ff7ce633d89fae69c7c221a8244afdf56598bfbe2b1ae71e6584df424b9ca995"},
]
[package.dependencies]
@@ -1889,11 +1889,11 @@ transmission-rpc = ">=7.0.0"
yt-dlp = ">=2023.0.0"
[package.extras]
all = ["fastapi (>=0.115.0)", "google-cloud-secret-manager (>=2.16.0)", "google-cloud-storage (>=2.13.0)", "httpx (>=0.28.0)", "mypy (>=1.0.0)", "pydantic (>=2.0.0)", "pytest (>=7.0.0)", "pytest-asyncio (>=0.23.0)", "pytest-cov (>=4.0.0)", "pytest-mock (>=3.10.0)", "ruff (>=0.1.0)", "spotipy (>=2.25.1)", "types-requests (>=2.28.0)", "uvicorn[standard] (>=0.24.0)", "yt-dlp-ejs (>=0.3.0)"]
all = ["fastapi (>=0.115.0)", "google-cloud-secret-manager (>=2.16.0)", "google-cloud-storage (>=2.13.0)", "httpx (>=0.28.0)", "mypy (>=1.0.0)", "playwright (>=1.40.0)", "pydantic (>=2.0.0)", "pytest (>=7.0.0)", "pytest-asyncio (>=0.23.0)", "pytest-cov (>=4.0.0)", "pytest-mock (>=3.10.0)", "ruff (>=0.1.0)", "spotipy (>=2.25.1)", "types-requests (>=2.28.0)", "uvicorn[standard] (>=0.24.0)", "yt-dlp-ejs (>=0.3.0)"]
api = ["fastapi (>=0.115.0)", "google-cloud-secret-manager (>=2.16.0)", "google-cloud-storage (>=2.13.0)", "httpx (>=0.28.0)", "pydantic (>=2.0.0)", "uvicorn[standard] (>=0.24.0)"]
dev = ["mypy (>=1.0.0)", "pytest (>=7.0.0)", "pytest-asyncio (>=0.23.0)", "pytest-cov (>=4.0.0)", "pytest-mock (>=3.10.0)", "ruff (>=0.1.0)", "types-requests (>=2.28.0)"]
ejs = ["yt-dlp-ejs (>=0.3.0)"]
remote = ["httpx (>=0.28.0)"]
remote = ["httpx (>=0.28.0)", "playwright (>=1.40.0)"]
spotify = ["spotipy (>=2.25.1)"]
[[package]]
@@ -5570,14 +5570,14 @@ win-unicode-console = {version = "*", markers = "platform_system == \"Windows\"
[[package]]
name = "pdfminer-six"
version = "20251107"
version = "20251227"
description = "PDF parser and analyzer"
optional = false
python-versions = ">=3.9"
python-versions = ">=3.10"
groups = ["main"]
files = [
{file = "pdfminer_six-20251107-py3-none-any.whl", hash = "sha256:c09df33e4cbe6b26b2a79248a4ffcccafaa5c5d39c9fff0e6e81567f165b5401"},
{file = "pdfminer_six-20251107.tar.gz", hash = "sha256:5fb0c553799c591777f22c0c72b77fc2522d7d10c70654e25f4c5f1fd996e008"},
{file = "pdfminer_six-20251227-py3-none-any.whl", hash = "sha256:7a41f668a74fdde9e3a6c9265e400992a8312224fb6dbc13ee8861111e330d36"},
{file = "pdfminer_six-20251227.tar.gz", hash = "sha256:1d98781cf884d7274c694e742a94ffe6326ee11ee7ea79e73e1e75c4c74d91eb"},
]
[package.dependencies]
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.76.10"
version = "0.76.17"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""
Fetch and display job data from Firestore.
Usage:
python scripts/get_job.py <job_id>
python scripts/get_job.py ffb0b8fa
Requires:
- google-cloud-firestore
- GOOGLE_APPLICATION_CREDENTIALS or gcloud auth application-default login
"""
import sys
import json
from google.cloud import firestore
def get_job(job_id: str) -> dict | None:
"""Fetch job document from Firestore."""
db = firestore.Client(project='nomadkaraoke')
doc = db.collection('jobs').document(job_id).get()
if doc.exists:
return doc.to_dict()
return None
def main():
if len(sys.argv) < 2:
print("Usage: python get_job.py <job_id> [--field <field_name>]")
print("\nExamples:")
print(" python get_job.py ffb0b8fa # Full job data")
print(" python get_job.py ffb0b8fa --field status # Just status")
print(" python get_job.py ffb0b8fa --field timeline # Just timeline")
sys.exit(1)
job_id = sys.argv[1]
field_filter = None
if len(sys.argv) >= 4 and sys.argv[2] == "--field":
field_filter = sys.argv[3]
data = get_job(job_id)
if data is None:
print(f"Job {job_id} not found", file=sys.stderr)
sys.exit(1)
if field_filter:
if field_filter in data:
output = data[field_filter]
else:
print(f"Field '{field_filter}' not found. Available fields:", file=sys.stderr)
print(", ".join(sorted(data.keys())), file=sys.stderr)
sys.exit(1)
else:
output = data
print(json.dumps(output, indent=2, default=str))
if __name__ == "__main__":
main()
+110 -2
View File
@@ -566,7 +566,115 @@ class TestAsync:
basic_karaoke_gen.input_media = None # Not a file
basic_karaoke_gen.artist = "Test Artist"
basic_karaoke_gen.title = None # Missing title
# Test that ValueError is raised
with pytest.raises(ValueError, match="Either a local file path or both artist and title must be provided"):
with pytest.raises(ValueError, match="Either a local file path.*or both artist and title must be provided"):
await basic_karaoke_gen.process()
@pytest.mark.asyncio
async def test_process_with_youtube_url(self, basic_karaoke_gen):
"""Test processing with a YouTube URL as input."""
# Setup - YouTube URL input
basic_karaoke_gen.input_media = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
basic_karaoke_gen.artist = "Rick Astley"
basic_karaoke_gen.title = "Never Gonna Give You Up"
# Mock dependencies
with patch.object(basic_karaoke_gen, 'prep_single_track', new_callable=AsyncMock) as mock_prep_single_track:
mock_prep_single_track.return_value = {"track": "result"}
result = await basic_karaoke_gen.process()
# Verify prep_single_track was called
mock_prep_single_track.assert_called_once()
# Verify URL mode was enabled
assert basic_karaoke_gen._use_audio_fetcher is True
assert basic_karaoke_gen._use_url_download is True
assert basic_karaoke_gen.extractor == "youtube"
assert basic_karaoke_gen.url == "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
# Verify the result
assert len(result) == 1
assert result[0] == {"track": "result"}
@pytest.mark.asyncio
async def test_process_with_youtube_url_without_artist_title(self, basic_karaoke_gen):
"""Test processing with a YouTube URL but no artist/title."""
# Setup - YouTube URL input without artist/title
basic_karaoke_gen.input_media = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
basic_karaoke_gen.artist = None
basic_karaoke_gen.title = None
# Mock dependencies
with patch.object(basic_karaoke_gen, 'prep_single_track', new_callable=AsyncMock) as mock_prep_single_track:
mock_prep_single_track.return_value = {"track": "result"}
result = await basic_karaoke_gen.process()
# Verify URL mode was enabled
assert basic_karaoke_gen._use_audio_fetcher is True
assert basic_karaoke_gen._use_url_download is True
# Verify extracted info has video ID as fallback
assert basic_karaoke_gen.extracted_info["id"] == "dQw4w9WgXcQ"
@pytest.mark.asyncio
async def test_process_with_youtu_be_short_url(self, basic_karaoke_gen):
"""Test processing with a youtu.be short URL."""
# Setup - Short URL input
basic_karaoke_gen.input_media = "https://youtu.be/dQw4w9WgXcQ"
basic_karaoke_gen.artist = "Rick Astley"
basic_karaoke_gen.title = "Never Gonna Give You Up"
# Mock dependencies
with patch.object(basic_karaoke_gen, 'prep_single_track', new_callable=AsyncMock) as mock_prep_single_track:
mock_prep_single_track.return_value = {"track": "result"}
result = await basic_karaoke_gen.process()
# Verify URL mode was enabled
assert basic_karaoke_gen._use_audio_fetcher is True
assert basic_karaoke_gen._use_url_download is True
assert basic_karaoke_gen.url == "https://youtu.be/dQw4w9WgXcQ"
class TestKaraokePrepIsUrlHelper:
"""Tests for KaraokePrep._is_url() helper method."""
@pytest.fixture
def karaoke_prep(self, tmp_path):
"""Create a KaraokePrep instance for testing."""
from karaoke_gen.karaoke_gen import KaraokePrep
return KaraokePrep(
artist="Test",
title="Test",
output_dir=str(tmp_path),
)
def test_is_url_http(self, karaoke_prep):
"""Test _is_url detects http URLs."""
assert karaoke_prep._is_url("http://example.com") is True
def test_is_url_https(self, karaoke_prep):
"""Test _is_url detects https URLs."""
assert karaoke_prep._is_url("https://www.youtube.com/watch?v=abc") is True
def test_is_url_youtube(self, karaoke_prep):
"""Test _is_url detects YouTube URLs."""
assert karaoke_prep._is_url("https://www.youtube.com/watch?v=dQw4w9WgXcQ") is True
assert karaoke_prep._is_url("https://youtu.be/dQw4w9WgXcQ") is True
def test_is_url_local_path(self, karaoke_prep):
"""Test _is_url returns False for local paths."""
assert karaoke_prep._is_url("/path/to/file.mp3") is False
assert karaoke_prep._is_url("./relative/path.wav") is False
assert karaoke_prep._is_url("file.flac") is False
def test_is_url_none(self, karaoke_prep):
"""Test _is_url returns False for None."""
assert karaoke_prep._is_url(None) is False
def test_is_url_empty_string(self, karaoke_prep):
"""Test _is_url returns False for empty string."""
assert karaoke_prep._is_url("") is False
+250
View File
@@ -2381,3 +2381,253 @@ class TestCreateAudioFetcherRemoteSelection:
assert isinstance(fetcher, RemoteFlacFetchAudioFetcher)
assert fetcher.api_url == "http://arg:9090"
assert fetcher.api_key == "arg_key"
class TestFlacFetchAudioFetcherDownloadFromUrl:
"""Tests for FlacFetchAudioFetcher.download_from_url() method."""
@pytest.fixture
def mock_logger(self):
"""Create a mock logger."""
return MagicMock(spec=logging.Logger)
@pytest.fixture
def fetcher(self, mock_logger):
"""Create a FlacFetchAudioFetcher instance with mocked dependencies."""
return FlacFetchAudioFetcher(logger=mock_logger)
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_youtube_success(self, mock_get_manager, fetcher, tmp_path):
"""Test successful download from YouTube URL."""
mock_manager = MagicMock()
downloaded_path = str(tmp_path / "song.webm")
mock_manager.download_by_id.return_value = downloaded_path
mock_get_manager.return_value = mock_manager
result = fetcher.download_from_url(
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
output_dir=str(tmp_path),
output_filename="Rick Astley - Never Gonna Give You Up",
artist="Rick Astley",
title="Never Gonna Give You Up",
)
assert result.filepath == downloaded_path
assert result.artist == "Rick Astley"
assert result.title == "Never Gonna Give You Up"
assert result.provider == "YouTube"
mock_manager.download_by_id.assert_called_once_with(
source_name="YouTube",
source_id="dQw4w9WgXcQ",
output_path=str(tmp_path),
output_filename="Rick Astley - Never Gonna Give You Up",
download_url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
)
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_youtu_be_short_url(self, mock_get_manager, fetcher, tmp_path):
"""Test download from youtu.be short URL."""
mock_manager = MagicMock()
downloaded_path = str(tmp_path / "song.webm")
mock_manager.download_by_id.return_value = downloaded_path
mock_get_manager.return_value = mock_manager
result = fetcher.download_from_url(
url="https://youtu.be/dQw4w9WgXcQ",
output_dir=str(tmp_path),
)
assert result.filepath == downloaded_path
assert result.provider == "YouTube"
# Should extract video ID from short URL
mock_manager.download_by_id.assert_called_once()
call_args = mock_manager.download_by_id.call_args
assert call_args.kwargs["source_id"] == "dQw4w9WgXcQ"
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_embed_url(self, mock_get_manager, fetcher, tmp_path):
"""Test download from YouTube embed URL."""
mock_manager = MagicMock()
downloaded_path = str(tmp_path / "song.webm")
mock_manager.download_by_id.return_value = downloaded_path
mock_get_manager.return_value = mock_manager
result = fetcher.download_from_url(
url="https://www.youtube.com/embed/dQw4w9WgXcQ",
output_dir=str(tmp_path),
)
call_args = mock_manager.download_by_id.call_args
assert call_args.kwargs["source_id"] == "dQw4w9WgXcQ"
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_generates_filename_from_artist_title(self, mock_get_manager, fetcher, tmp_path):
"""Test that filename is generated from artist and title if not provided."""
mock_manager = MagicMock()
downloaded_path = str(tmp_path / "song.webm")
mock_manager.download_by_id.return_value = downloaded_path
mock_get_manager.return_value = mock_manager
result = fetcher.download_from_url(
url="https://www.youtube.com/watch?v=test123test",
output_dir=str(tmp_path),
artist="Test Artist",
title="Test Song",
)
call_args = mock_manager.download_by_id.call_args
assert call_args.kwargs["output_filename"] == "Test Artist - Test Song"
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_generates_filename_from_video_id(self, mock_get_manager, fetcher, tmp_path):
"""Test that filename falls back to video ID if no artist/title."""
mock_manager = MagicMock()
downloaded_path = str(tmp_path / "song.webm")
mock_manager.download_by_id.return_value = downloaded_path
mock_get_manager.return_value = mock_manager
result = fetcher.download_from_url(
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
output_dir=str(tmp_path),
)
call_args = mock_manager.download_by_id.call_args
assert call_args.kwargs["output_filename"] == "dQw4w9WgXcQ"
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_raises_download_error_on_failure(self, mock_get_manager, fetcher, tmp_path):
"""Test that DownloadError is raised when download fails."""
mock_manager = MagicMock()
mock_manager.download_by_id.side_effect = Exception("Download failed")
mock_get_manager.return_value = mock_manager
with pytest.raises(DownloadError) as exc_info:
fetcher.download_from_url(
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
output_dir=str(tmp_path),
)
assert "Failed to download from URL" in str(exc_info.value)
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_raises_on_empty_filepath(self, mock_get_manager, fetcher, tmp_path):
"""Test that DownloadError is raised when download returns no filepath."""
mock_manager = MagicMock()
mock_manager.download_by_id.return_value = None
mock_get_manager.return_value = mock_manager
with pytest.raises(DownloadError) as exc_info:
fetcher.download_from_url(
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
output_dir=str(tmp_path),
)
assert "Download returned no file path" in str(exc_info.value)
@patch('karaoke_gen.audio_fetcher.FlacFetchAudioFetcher._get_manager')
def test_download_from_url_creates_output_dir(self, mock_get_manager, fetcher, tmp_path):
"""Test that output directory is created if it doesn't exist."""
mock_manager = MagicMock()
mock_manager.download_by_id.return_value = str(tmp_path / "song.webm")
mock_get_manager.return_value = mock_manager
new_dir = tmp_path / "new_subdir"
assert not new_dir.exists()
fetcher.download_from_url(
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
output_dir=str(new_dir),
)
assert new_dir.exists()
class TestRemoteFlacFetchAudioFetcherDownloadFromUrl:
"""Tests for RemoteFlacFetchAudioFetcher.download_from_url() method."""
@pytest.fixture
def mock_logger(self):
"""Create a mock logger."""
return MagicMock(spec=logging.Logger)
@pytest.fixture
def fetcher(self, mock_logger):
"""Create a RemoteFlacFetchAudioFetcher instance."""
return RemoteFlacFetchAudioFetcher(
api_url="http://localhost:8080",
api_key="test_key",
logger=mock_logger,
)
@patch('flacfetch.core.manager.FetchManager')
@patch('flacfetch.providers.youtube.YoutubeProvider')
@patch('flacfetch.downloaders.youtube.YoutubeDownloader')
def test_download_from_url_uses_local_flacfetch(
self, mock_yt_downloader, mock_yt_provider, mock_fetch_manager, fetcher, tmp_path
):
"""Test that download_from_url uses local flacfetch (not remote API)."""
mock_manager_instance = MagicMock()
downloaded_path = str(tmp_path / "song.webm")
mock_manager_instance.download_by_id.return_value = downloaded_path
mock_fetch_manager.return_value = mock_manager_instance
result = fetcher.download_from_url(
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
output_dir=str(tmp_path),
artist="Rick Astley",
title="Never Gonna Give You Up",
)
# Verify local flacfetch was used
mock_fetch_manager.assert_called_once()
mock_manager_instance.add_provider.assert_called_once()
mock_manager_instance.register_downloader.assert_called_once_with("YouTube", mock_yt_downloader.return_value)
mock_manager_instance.download_by_id.assert_called_once()
assert result.filepath == downloaded_path
assert result.artist == "Rick Astley"
assert result.title == "Never Gonna Give You Up"
assert result.provider == "YouTube"
@patch('flacfetch.core.manager.FetchManager')
@patch('flacfetch.providers.youtube.YoutubeProvider')
@patch('flacfetch.downloaders.youtube.YoutubeDownloader')
def test_download_from_url_extracts_video_id(
self, mock_yt_downloader, mock_yt_provider, mock_fetch_manager, fetcher, tmp_path
):
"""Test that video ID is extracted from URL correctly."""
mock_manager_instance = MagicMock()
mock_manager_instance.download_by_id.return_value = str(tmp_path / "song.webm")
mock_fetch_manager.return_value = mock_manager_instance
fetcher.download_from_url(
url="https://www.youtube.com/watch?v=abc123xyz99",
output_dir=str(tmp_path),
)
call_args = mock_manager_instance.download_by_id.call_args
assert call_args.kwargs["source_id"] == "abc123xyz99"
@patch('flacfetch.core.manager.FetchManager')
@patch('flacfetch.providers.youtube.YoutubeProvider')
@patch('flacfetch.downloaders.youtube.YoutubeDownloader')
def test_download_from_url_raises_on_failure(
self, mock_yt_downloader, mock_yt_provider, mock_fetch_manager, fetcher, tmp_path
):
"""Test that DownloadError is raised on failure."""
mock_manager_instance = MagicMock()
mock_manager_instance.download_by_id.side_effect = Exception("Network error")
mock_fetch_manager.return_value = mock_manager_instance
with pytest.raises(DownloadError) as exc_info:
fetcher.download_from_url(
url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
output_dir=str(tmp_path),
)
assert "Failed to download from URL" in str(exc_info.value)
def test_download_from_url_method_exists(self, fetcher):
"""Test that download_from_url method exists on RemoteFlacFetchAudioFetcher."""
assert hasattr(fetcher, 'download_from_url')
assert callable(fetcher.download_from_url)