Compare commits

...
Author SHA1 Message Date
Andrew Beveridge f99d6c1fb4 style: remove extraneous f-string prefixes
Address CodeRabbit nitpick comments - remove f-string prefixes from
strings without placeholders.
2025-12-19 18:42:03 -05:00
Andrew Beveridge f8a45bfbd2 fix: countdown padding synchronization for instrumental audio
When LyricsTranscriber adds 3-second countdown padding to vocals,
the instrumental audio must be padded to match. This was broken when
transcription was skipped due to existing files.

Changes:
- Add _detect_countdown_padding_from_lrc() to detect padding from existing LRC
- Update early returns in transcribe_lyrics() to include padding info
- Add _scan_directory_for_instrumentals() to find existing instrumentals
- Fix counting bug in apply_countdown_padding_to_instrumentals
- Add countdown_padding_seconds safety net to KaraokeFinalise
- Store padding metadata in lyrics_worker job state_data
- Pass padding info to video_worker for instrumental sync
- Add comprehensive unit tests for new functionality
2025-12-19 18:35:37 -05:00
Andrew Beveridge ☄️ f524796f7f Merge pull request #14 from nomadkaraoke/feature/batch-6-two-phase-workflow
feat: Batch 6 - Two-Phase Workflow (prep-only, finalise-only, keep-brand-code)
2025-12-19 00:07:15 -05:00
Andrew Beveridge ☄️ 8372ed211a Merge pull request #34 from nomadkaraoke/fix/youtube-duplicate-checker-channel-filter
Fix YouTube duplicate checker returning videos from other channels
2025-12-19 00:06:11 -05:00
Andrew Beveridge 78068ea1d8 Fix YouTube duplicate checker returning videos from other channels
The YouTube Data API v3 search endpoint's channelId parameter doesn't
always filter correctly and sometimes returns videos from other channels.

This fix verifies each search result's snippet.channelId matches the
user's channel ID before considering it as a potential match, preventing
false positives from unrelated videos with similar titles.

Added tests for the channel filtering behavior.
2025-12-18 23:28:23 -05:00
Andrew Beveridge 84d7c12f13 fix: Address CodeRabbit review comments for Batch 6 PR
- Add PREP_COMPLETE to RENDERING_VIDEO state transitions (job.py)
- Simplify instrumental detection logic for consistency (remote_cli.py)
- Add shell escaping (shlex.quote) for continuation command display (remote_cli.py)
- Update test for --finalise-only now that it's implemented (test_remote_cli.py)
- Add test for PREP_COMPLETE status (test_remote_cli.py)
- Bump version to 0.71.69
2025-12-18 23:22:05 -05:00
Andrew Beveridge 7a4ed9661c Merge origin/main into feature/batch-6-two-phase-workflow 2025-12-18 22:42:49 -05:00
Andrew Beveridge ☄️ dd62c32026 Merge pull request #33 from nomadkaraoke/fix/uploaded-instrumental-selection
Fix: Handle 'uploaded' selection in instrumental review CLI
2025-12-18 21:35:28 -05:00
Andrew Beveridge c130fa8a07 Merge remote-tracking branch 'origin/main' into fix/uploaded-instrumental-selection 2025-12-18 20:24:14 -05:00
Andrew Beveridge ☄️ e0d1fa7536 Merge pull request #32 from nomadkaraoke/replicate-midico-synchronizer
feat: Add MidiCo-style lyrics synchronizer for re-syncing existing lyrics
2025-12-18 20:23:47 -05:00
Andrew Beveridge 29ffbd4ae2 fix: handle 'uploaded' selection in instrumental review CLI
The instrumental review UI supports uploading custom instrumental files,
but gen_cli.py was missing the handler for the 'uploaded' selection type.

When a user uploaded a custom instrumental and selected it, the CLI would
log 'Unknown selection: uploaded' and crash with an unhelpful error message.

This fix adds proper handling for the 'uploaded' case, retrieving the
uploaded file path from the server just like the 'custom' case does.
2025-12-18 20:21:00 -05:00
Andrew Beveridge e440f0bbcd fix: Address CodeRabbit review comments
- Fix word move timing bug: ensure end time is always after start time
  (prevents invalid timings where end < start with very negative deltas)
- Add typeof checks before calling window methods for runtime type safety
2025-12-18 20:13:13 -05:00
Andrew Beveridge e0f5a84e54 feat: Add MidiCo-style lyrics synchronizer for re-syncing existing lyrics
Implements a new synchronizer UI that closely replicates MidiCo's UX:

- Mode selection modal: Choose between "Re-sync Existing Lyrics" or "Replace All Lyrics"
- Canvas-based timeline for high-performance word block rendering
- Two-level alternating segment layout to prevent text overlap
- Spacebar sync mode: tap for short words, hold for longer words
- Current word highlighting (red text + brighter block)
- Word block manipulation:
  - Resize words by dragging handle on right edge
  - Drag to move single or multiple selected words
- Scroll/zoom controls for timeline navigation
- Play/Stop/Pause controls for audio playback
- "Unsync from Cursor" to re-sync only the drifted portion
- Fixed-height layout sections to prevent UI jumping during sync

This addresses the core problem of lyrics that start well-synced but drift
progressively out of sync in the second half of a song.
2025-12-18 19:59:08 -05:00
Andrew Beveridge 8da57da767 Added reference doc explaining the plan to replace our "Edit All" functionality with something much more closely aligned with MidiCo functionality 2025-12-18 16:27:59 -05:00
Andrew Beveridge b840a12669 Added midico synchronizer reference guide 2025-12-18 16:02:19 -05:00
Andrew Beveridge ☄️ f7038ce72c Merge pull request #31 from nomadkaraoke/fix/ctrlc-audio-fetcher
Ensure we can quit during audio fetch
2025-12-18 12:04:35 -05:00
Andrew Beveridge 138ddeecf5 Ensure we can quit during audio fetch 2025-12-18 11:46:33 -05:00
Andrew Beveridge ☄️ a7d9bb40b9 Merge pull request #30 from nomadkaraoke/feature/local-review-ui
Support local review UI for frontend development
2025-12-18 02:16:55 -05:00
Andrew Beveridge ☄️ 20f842fa85 Merge pull request #29 from nomadkaraoke/feature/annotation-toggle
Add toggle to disable annotation prompts when editing lyrics
2025-12-18 02:07:51 -05:00
Andrew Beveridge 1b74199032 Default to bundled local frontend for lyrics review
- Change default from hosted https://lyrics.nomadkaraoke.com to local
- Local karaoke-gen now uses bundled frontend served by FastAPI
- No external dependency for local runs
- Can override with LYRICS_REVIEW_UI_URL env var if needed:
  - 'local' (default): bundled frontend at http://localhost:8000
  - 'http://localhost:5173': Vite dev server for frontend development
  - 'https://lyrics.nomadkaraoke.com': hosted version (legacy)

Also includes rebuilt frontend assets with annotation toggle feature.

Version bump: 0.71.65 -> 0.71.66
2025-12-18 01:58:53 -05:00
Andrew Beveridge e75c5b2723 Support local review UI for development via --review-ui-url
- Add --review-ui-url argument to local gen_cli (was remote-only)
- Set LYRICS_REVIEW_UI_URL env var from CLI argument
- Allows pointing to local Vite dev server (http://localhost:5173)
- Supports both REVIEW_UI_URL and LYRICS_REVIEW_UI_URL env vars

Usage for local frontend development:
  karaoke-gen --review-ui-url http://localhost:5173 ...

Version bump: 0.71.64 -> 0.71.65
2025-12-18 01:53:34 -05:00
Andrew Beveridge 9e53bb7934 Merge remote-tracking branch 'origin/main' into feature/annotation-toggle 2025-12-18 01:53:10 -05:00
Andrew Beveridge ☄️ a85727b147 Merge pull request #28 from nomadkaraoke/fix-flac-fetcher
Fix flacfetch API compatibility and use rich CLI output
2025-12-18 01:52:32 -05:00
Andrew Beveridge 3eb8f707b6 Add toggle to disable annotation prompts when editing lyrics
- Add "Feedback On/Off" chip in header next to title
- Toggle persists to localStorage for session continuity
- Defaults to enabled (annotation prompts shown)
- Hidden in read-only mode
- Tooltip explains functionality

Version bump: 0.71.63 -> 0.71.64
2025-12-18 01:46:41 -05:00
Andrew Beveridge 4b26fe9eac Add integration tests for flacfetch compatibility
Add comprehensive tests that would have caught both recent bugs:

1. TestGetManagerIntegration - Tests that actually execute _get_manager():
   - test_get_manager_imports_work: Would have caught YouTubeProvider rename
   - test_get_manager_registers_*: Would have caught missing downloader registration

2. TestFlacFetchReleaseModelCompatibility - Tests against real flacfetch models:
   - test_release_model_has_expected_attributes: Verifies attribute names match
   - test_search_converts_release_attributes_correctly: Tests with real Release objects

3. TestFlacFetchCLIHandlerCompatibility - Tests CLIHandler integration:
   - test_cli_handler_import_works: Verifies import path
   - test_cli_handler_works_with_real_releases: Tests with real Quality objects

These tests use real flacfetch classes instead of mocks to catch API changes.

Bump version to 0.71.63
2025-12-18 01:37:00 -05:00
Andrew Beveridge 4de3ecc85d Register downloaders for flacfetch sources
- Register YoutubeDownloader for YouTube source
- Register TorrentDownloader for Redacted and OPS sources
- Handle optional TorrentDownloader import (has extra dependencies)

This fixes "No downloader registered for source: Redacted" error when
trying to download from torrent tracker sources.

Bump version to 0.71.62
2025-12-18 01:32:52 -05:00
Andrew Beveridge cbb4cd8f9e Fix flacfetch API compatibility and use rich CLI output
- Update YouTubeProvider import to YoutubeProvider (API change in flacfetch 0.2.0)
- Update attribute names to match flacfetch Release model:
  - provider -> source_name
  - duration -> duration_seconds
  - url -> download_url
  - id -> info_hash
- Use flacfetch's built-in CLIHandler for rich, colorized interactive selection
- Add fallback to basic display if CLIHandler fails (e.g., in tests)
- Update tests to use correct flacfetch attribute names

Bump version to 0.71.61
2025-12-18 01:25:55 -05:00
Andrew Beveridge ☄️ 218011cf2a Merge pull request #27 from nomadkaraoke/refactor/instrumental-review-ui-improvements
Refactor instrumental review UI: extract frontend to static file and improve UX
2025-12-18 01:22:55 -05:00
Andrew Beveridge 22c6e7ed41 Fix audio listeners not attached after DOM rebuild
Track audio element reference instead of boolean flag to detect when the
audio element is replaced during render(). Re-attach listeners when the
element changes, ensuring new audio elements have proper event handlers.

Version bump: 0.71.59 → 0.71.60
2025-12-18 01:13:19 -05:00
Andrew Beveridge e104fb9252 Add original audio option to instrumental review UI
Backend:
- Add original_audio_path parameter to InstrumentalReviewServer
- Add /api/audio/original endpoint for streaming original audio
- Add "original" to valid selection options

Frontend:
- Add "Original" toggle button (shown first when available)
- Add "Original Audio" selection option with description
- Add original to success message labels

Helper script:
- Find original audio files matching "(Original)" pattern
- Log and pass original_audio_path to server

Version bump: 0.71.58 → 0.71.59
2025-12-18 01:05:11 -05:00
Andrew Beveridge 31931cc55d Address CodeRabbit PR review comments
Frontend (index.html):
- Fix duplicate event listeners: add audioListenersInitialized flag and named
  handlers (onAudioPlay, onAudioPause, onAudioEnded) to prevent accumulation
- Remove redundant removeEventListener in loadeddata handler (already using once: true)
- Use data-audio-type attributes on toggle buttons for robust type detection
- Add escapeHtml() helper to prevent XSS in artist/title/filename interpolation
- Guard updatePlayhead() against NaN/Infinity when duration is 0 or invalid
- Add explicit 'uploaded' branch in success message ternary

Backend (server.py):
- Wrap temp file handling in try/finally to ensure cleanup on error
- Track file_moved state to only unlink temp file if not successfully moved

Version bump: 0.71.57 → 0.71.58
2025-12-18 00:45:55 -05:00
Andrew Beveridge d21729bf3d Fix JavaScript syntax error and improve file discovery in helper script
- Fix invalid escape sequences in updateMuteRegionsPanel() - use string
  concatenation instead of nested template literals
- Update helper script to find instrumental files in main job directory
  (not just stems/) and handle "+BV" naming convention

Version bump: 0.71.56 → 0.71.57
2025-12-18 00:39:07 -05:00
Andrew Beveridge b49a09e1c6 Add helper script to run instrumental review UI from existing job folder
New script: scripts/run_instrumental_review.py
- Takes a karaoke job directory path as input
- Auto-discovers backing vocals, clean instrumental, and combined files in stems/
- Runs audio analysis and generates waveform if needed
- Starts the review server for frontend development iteration

Usage:
  python scripts/run_instrumental_review.py "Artist - Title"
  python scripts/run_instrumental_review.py "path/to/job" --port 9000
  python scripts/run_instrumental_review.py "path/to/job" --no-browser

Version bump: 0.71.55 → 0.71.56
2025-12-18 00:34:41 -05:00
Andrew Beveridge e6b2194272 Preserve scroll position when adding/removing mute regions while zoomed
- Replace full render() calls with targeted updates when modifying mute regions
- Add updateMuteRegionsPanel() to update only the mute panel DOM
- Just redraw waveform canvas instead of rebuilding entire UI
- Scroll position is now preserved when adding, removing, or clearing regions

Version bump: 0.71.54 → 0.71.55
2025-12-18 00:31:51 -05:00
Andrew Beveridge a08b83df40 Add custom instrumental upload and waveform zoom features
Upload feature:
- Add POST /api/jobs/local/upload-instrumental endpoint
- Validate uploaded audio duration matches original (±0.5s tolerance)
- Support .flac, .mp3, .wav, .m4a, .ogg formats
- Show upload progress overlay with spinner
- Auto-select uploaded audio after successful upload
- Add "Uploaded" option to audio toggles and final selection

Zoom feature:
- Add 1x, 2x, 4x zoom levels for waveform precision
- Waveform container is horizontally scrollable when zoomed
- Zoom controls in toolbar with active state indication
- Maintain proportional scroll position when changing zoom
- Playhead and selection overlay work correctly at all zoom levels

Version bump: 0.71.53 → 0.71.54
2025-12-18 00:27:38 -05:00
Andrew Beveridge 53d2ed2828 Fix duplicate audio when switching sources during playback
- Remove full render() call from setActiveAudio() to avoid recreating audio element
- Update toggle button states directly via classList instead of DOM rebuild
- Check actual audio.paused state instead of isPlaying variable (which gets stale)
- Prevents multiple audio streams playing simultaneously

Version bump: 0.71.52 → 0.71.53
2025-12-18 00:10:58 -05:00
Andrew Beveridge ff20b5ffb3 Fix audio playback AbortError when DOM rebuilds during render
- Pause audio before rebuilding DOM in render() to prevent AbortError
- Use loadeddata event to safely restore playback position and state
- Handle audio lifecycle properly in setActiveAudio() and createCustomInstrumental()
- Prevents duplicate audio playback and console errors

Version bump: 0.71.51 → 0.71.52
2025-12-18 00:09:08 -05:00
Andrew Beveridge 49290f94d7 Refactor instrumental review UI: extract frontend to static file and improve UX
- Move inline HTML/CSS/JS from server.py to static/index.html for better maintainability
- Add proper syntax highlighting, formatting, and linting support for frontend code
- Server now reads HTML from file using Path(__file__).parent / "static"

UI/UX improvements:
- Click on waveform now seeks AND starts playback automatically
- Switching audio sources (Backing/Clean/Combined) maintains playback state
- Clearer audio mode labels: "Backing Vocals Only", "Pure Instrumental", "Instrumental + Backing"
- Replace M key toggle mode with Shift+drag for intuitive mute region selection
- Update help text and keyboard hints

Package configuration:
- Add karaoke_gen/instrumental_review/static/* to pyproject.toml includes
- Verified static files are correctly packaged in wheel for pip installs

Version bump: 0.71.48 → 0.71.51
2025-12-18 00:07:13 -05:00
Andrew Beveridge ☄️ 9b114fcb3d Merge pull request #26 from nomadkaraoke/fix/waveform-generator-param-name
fix: correct parameter name in WaveformGenerator.generate() call
2025-12-17 23:34:34 -05:00
Andrew Beveridge e507b3f685 fix: prevent duplicate log messages from external package side effects
External packages like lyrics_converter call logging.info() directly on the
root logger, which triggers Python's 'last resort' handler. This causes log
messages to appear twice: once from our custom handler and once from the
root logger's default handler.

Fixed by setting logger.propagate = False on all loggers that have their
own handlers configured. This prevents logs from propagating to the root
logger while still displaying through our custom formatted handlers.

Affected files:
- karaoke_gen/karaoke_finalise/karaoke_finalise.py
- karaoke_gen/karaoke_gen.py
- karaoke_gen/utils/gen_cli.py
- karaoke_gen/utils/remote_cli.py
- karaoke_gen/utils/bulk_cli.py

Also updated tests to use capsys instead of caplog since caplog relies
on log propagation to capture messages.

Bumps version to 0.71.47
2025-12-17 23:17:29 -05:00
Andrew Beveridge 6d0c765f4d test: add integration tests for run_instrumental_review parameter contract
Added tests that verify:
- WaveformGenerator.generate() is called with 'segments' (not 'audible_segments')
- Correct parameter values are passed between components
- Error handling and edge cases

These tests would have caught the parameter name bug that broke
the instrumental review flow. They exercise the actual function
with properly mocked components instead of mocking the entire function.

Bumps version to 0.71.46
2025-12-17 23:04:51 -05:00
Andrew Beveridge 8086242d38 fix: correct parameter name in WaveformGenerator.generate() call
The WaveformGenerator.generate() method expects 'segments' parameter,
not 'audible_segments'. This caused the instrumental review to fail
with 'unexpected keyword argument' error during karaoke-gen CLI execution.

Bumps version to 0.71.45
2025-12-17 22:56:59 -05:00
Andrew Beveridge ☄️ 3c3680632b Merge pull request #25 from nomadkaraoke/fix/instrumental-selection-single-path
Remove legacy instrumental selection fallback, fix path resolution
2025-12-17 22:23:46 -05:00
Andrew Beveridge dd5f05b291 fix: update integration test to use --skip_instrumental_review
The integration test was failing because it didn't provide the
separated_audio data structure expected by run_instrumental_review.
Since the test uses mocked audio data, it makes sense to use the
--skip_instrumental_review flag with auto_select_instrumental instead.

- Added --skip_instrumental_review to test CLI args
- Updated mock_process_audio_separation to return clean_instrumental
  data structure needed by auto_select_instrumental
- Bumped version to 0.71.44
2025-12-14 13:30:47 -05:00
Andrew Beveridge 36a60c09da Merge origin/main into feature/batch-6-two-phase-workflow
Resolved conflicts to merge Batch 6 (Two-Phase Workflow) with:
- Batch 5 (FlacFetch Audio Search) - Added SEARCHING_AUDIO states
- Instrumental review UI fixes
- Worker logging context improvements

Key merge resolutions:
- job.py: Combined state transitions for SEARCHING_AUDIO and AWAITING_INSTRUMENTAL_SELECTION
- remote_cli.py: Added prep_only/keep_brand_code to both URL and file submission paths
- render_video_worker.py: Added job_logging_context while preserving prep_only transition logic
- file_upload.py: Re-added all finalise-only endpoints and models

All 180 tests pass.
2025-12-14 13:15:10 -05:00
Andrew Beveridge 9340ed5d9f Remove legacy instrumental selection fallback, fix path resolution
This change eliminates the silent fallback to the legacy numbered instrumental
selection in KaraokeFinalise and ensures a single code path for instrumental
selection:

1. **Path resolution fix**: Added `_resolve_path_for_cwd()` helper to correctly
   resolve paths in `separated_audio` after `os.chdir(track_dir)`. The paths
   were stored relative to the original directory, causing `os.path.exists()`
   to fail after the directory change.

2. **No silent fallback**: When `run_instrumental_review()` fails, the CLI now
   exits with a clear error message explaining what went wrong and how to fix
   it, rather than silently falling back to the legacy numbered selection.

3. **Auto-select for non-interactive mode**: Added `auto_select_instrumental()`
   function for `--skip_instrumental_review` mode that intelligently selects
   the best instrumental (preferring padded+combined > padded > combined > clean).

4. **Clear error messages**: When instrumental selection fails, users now get
   detailed diagnostics about what to check and how to recover.

The legacy `choose_instrumental_audio_file()` method in KaraokeFinalise is
preserved for direct usage but is no longer called from the gen_cli flow.

Includes comprehensive tests for the new helper functions.

Bump version to 0.71.43.
2025-12-14 13:10:29 -05:00
Andrew Beveridge ☄️ 8959da63a1 Merge pull request #24 from nomadkaraoke/fix/instrumental-review-path-resolution
Fix instrumental review UI path resolution after chdir
2025-12-14 12:56:07 -05:00
Andrew Beveridge ☄️ cb7b4863f5 Merge pull request #13 from nomadkaraoke/feature/batch-5-flacfetch-audio-search
feat: Batch 5 - Flacfetch audio search for remote CLI
2025-12-14 12:53:45 -05:00
Andrew Beveridge e84a511676 Fix instrumental review UI path resolution after chdir
The instrumental review UI was being skipped with "No backing vocals file
found" because file paths stored in separated_audio were relative to the
original working directory, but os.chdir(track_dir) had already been called
before the paths were checked with os.path.exists().

This fix:
- Adds _resolve_path_for_cwd() helper to convert paths that were created
  relative to the original directory to work from the new current directory
- Applies path resolution to backing_vocals, clean_instrumental, and
  combined_instrumentals paths before checking their existence
- Updates waveform and output paths to use current directory since we're
  already inside track_dir after chdir

Includes unit tests for the new helper function.

Bump version to 0.71.42.
2025-12-11 20:27:11 -05:00
Andrew Beveridge aee2b48ffd Merge origin/main into feature/batch-5-flacfetch-audio-search
Resolved conflicts to combine:
- Batch 4 (YouTube URL input) from main
- Batch 5 (Flacfetch audio search) from this branch

Both features are now fully integrated:
- P2 (YouTube URLs): ✅ COMPLETE
- P3 (Flacfetch search): ✅ COMPLETE
- A1 (--auto-download): ✅ COMPLETE

remote_cli.py now supports:
- File upload mode (original)
- YouTube URL mode (from main)
- Audio search mode (from this branch)

Documentation updated to reflect both batches complete.
Version bumped to 0.71.42.
2025-12-11 20:10:03 -05:00
Andrew Beveridge ☄️ a7652ba72d Merge pull request #23 from nomadkaraoke/fix/local-cli-instrumental-review-integration
fix: integrate instrumental review UI into local CLI workflow
2025-12-11 19:42:21 -05:00
Andrew Beveridge 0ce5225bde fix: integrate instrumental review UI into local CLI workflow
The instrumental review UI was built but never integrated into the
local CLI (karaoke-gen) workflow. This fix:

1. Adds run_instrumental_review() function to gen_cli.py that:
   - Analyzes backing vocals using AudioAnalyzer
   - Generates waveform visualization
   - Starts InstrumentalReviewServer and opens browser
   - Returns the user's selected instrumental file path

2. Integrates the review into the finalization loop:
   - Runs before KaraokeFinalise for each track
   - Passes selected_instrumental_file to KaraokeFinalise
   - Skips review if --skip_instrumental_review flag is set

3. Adds --skip_instrumental_review CLI flag for backwards compatibility

4. Adds comprehensive tests in test_cli_feature_parity.py that verify:
   - gen_cli.py has run_instrumental_review function
   - The function uses AudioAnalyzer, WaveformGenerator, and InstrumentalReviewServer
   - The --skip_instrumental_review flag works correctly

These tests would have caught this integration gap and will prevent
similar regressions in the future.

Version bump: 0.71.40 → 0.71.41
2025-12-11 19:15:37 -05:00
Andrew Beveridge ☄️ c494685211 Merge pull request #22 from nomadkaraoke/feature/instrumental-review-ui
feat: Add intelligent instrumental selection with web-based review UI
2025-12-11 19:01:40 -05:00
Andrew Beveridge 599625e21a fix: address CodeRabbit PR review comments and improve test coverage
CodeRabbit fixes:
- Chain exceptions with 'from e' in jobs.py for proper tracebacks
- Use logger.exception() for automatic stack traces
- Add end_seconds validation warning in audio_editing_service.py
- Handle recommended_selection='with_backing' explicitly in frontend
- Sync audio player state with actual audio element via event listeners
- Fix muted duration stats overcounting by clamping to audio duration
- Add num_points validation (1-10000) in server waveform endpoint
- Improve exception handling in server endpoints

Test coverage improvements:
- Add comprehensive tests for InstrumentalReviewServer
- Test request/response models
- Test server initialization and configuration
- Test FastAPI endpoints with TestClient
- Coverage: karaoke_gen 73.05% (above 70% minimum)

Version bump: 0.71.39 → 0.71.40
2025-12-11 18:33:20 -05:00
Andrew Beveridge 5c4404839b chore: remove unused imports from server.py 2025-12-11 18:09:50 -05:00
Andrew Beveridge f699850d30 feat: rewrite local review server with FastAPI and full-featured standalone UI
- Replace http.server with FastAPI for better async/CORS support
- Add complete standalone HTML/CSS/JS frontend with all features:
  - Dark theme matching React frontend
  - Interactive waveform display with Canvas rendering
  - Audio playback with source selection
  - Region selection via click/drag or quick-add from segments
  - Custom instrumental creation
  - Final selection submission with success feedback
- API endpoints now match cloud backend for true feature parity
- Update CLI parity tests for new FastAPI-based server structure
- Add test-instrumental-review.py script for local testing

This enables the same user experience for instrumental review
whether running locally (karaoke-gen) or remotely (karaoke-gen-remote).

Version bump: 0.71.38 → 0.71.39
2025-12-11 18:08:04 -05:00
Andrew Beveridge e8d9871d3e feat: Add intelligent instrumental selection with web-based review UI
Adds a comprehensive instrumental selection workflow with:
- Audio analysis to detect backing vocals (>-40dB threshold)
- Waveform visualization with segment highlighting
- Interactive web UI for audio playback and region selection
- Custom instrumental creation by muting selected regions
- Smart recommendations based on analysis results
- Feature parity between local (karaoke-gen) and remote (karaoke-gen-remote) CLIs

Key components:
- Shared core module (karaoke_gen/instrumental_review/) with AudioAnalyzer,
  AudioEditor, WaveformGenerator, and InstrumentalReviewServer
- Backend services wrapping core module with GCS integration
- New API endpoints for analysis, audio streaming, and custom creation
- React frontend components for waveform display, audio playback, region selection
- Local HTTP server for browser-based review in local CLI workflow
- 1280 tests passing with comprehensive coverage

Closes the need for manual Audacity editing when selecting instrumentals.
2025-12-11 17:12:49 -05:00
Andrew Beveridge ☄️ 473253b5ab Merge pull request #21 from nomadkaraoke/fix/enable-cloud-tasks-deployment
fix: Enable Cloud Tasks in deployment for scalable architecture
2025-12-11 16:20:00 -05:00
Andrew Beveridge 89cc9e9113 fix: Enable Cloud Tasks in deployment for scalable architecture
The scalable architecture (Cloud Tasks queues, IAM, infrastructure) was
deployed via Pulumi but never activated in the Cloud Run deployment.

Changes:
- Enable ENABLE_CLOUD_TASKS=true environment variable
- Set CLOUD_RUN_SERVICE_URL for Cloud Tasks callbacks
- Set GCP_REGION for queue location consistency
- Change concurrency from 4 to 1 for true container isolation
- Increase max-instances from 10 to 50 for higher throughput

This completes Phase 1 of the scalable architecture migration, ensuring
each job gets dedicated container resources with no contention.
2025-12-11 15:45:03 -05:00
Andrew Beveridge ☄️ ec5a816b68 Merge pull request #20 from nomadkaraoke/anchor-sequence-optimization
feat: Optimize AnchorSequenceFinder for cloud compatibility and performance
2025-12-11 14:37:06 -05:00
Andrew Beveridge 7eb082d382 test: Increase unit test coverage from 68% to 72%
Add comprehensive tests for:
- style_loader.py - new test file covering all loading, saving, and helper functions
- file_handler.py - tests for download_video, extract_metadata_from_url, backup_existing_outputs
- remote executor - tests for run_pipeline, _submit_job, _monitor_job

This brings test coverage above the required 70% CI threshold.
2025-12-11 14:15:27 -05:00
Andrew Beveridge 1af7ab0bde fix: Address CodeRabbit review comments
- Fix type mismatch: pass str instead of List[str] for transcribed param
- Move pytest hooks (slow marker, --run-slow option) to conftest.py
- Bump version to 0.71.34
2025-12-11 14:10:54 -05:00
Andrew Beveridge 7ba40af80d feat: Optimize AnchorSequenceFinder for cloud compatibility and performance
Major optimizations to the anchor sequence finding algorithm:

- Remove multiprocessing.Pool usage that caused hangs in Cloud Run
- Replace O(N) n-gram matching with O(1) hash-based index lookup
- Replace O(N^2) overlap removal with O(N) covered positions tracking
- Add simple rule-based scoring to avoid expensive spaCy NLP
- All processing now single-threaded for container compatibility

Performance improvements:
- Test suite execution time reduced from 72s to ~5s
- Golden tests verify functional equivalence with real song data
- All 10 fixture tests complete under 30 seconds each

Tests:
- Add 10 golden test fixtures from real cache data
- Add performance benchmark tests asserting <30s completion
- Update existing tests for new algorithm behavior

Bump version to 0.71.33
2025-12-11 13:56:07 -05:00
Andrew Beveridge f434e3c93d Added plan for anchor sequence optimization, with golden test setup to verify output matches expectations after rewrite 2025-12-11 13:44:57 -05:00
Andrew Beveridge ☄️ 4b6523dbd8 Merge pull request #19 from nomadkaraoke/fix/update-poetry-lock
chore: Update poetry.lock for new GCP dependencies
2025-12-11 11:16:36 -05:00
Andrew Beveridge d67828b78f chore: Update poetry.lock for new GCP dependencies
Regenerate poetry.lock to include:
- google-cloud-tasks (for Cloud Tasks worker coordination)
- google-cloud-run (for Cloud Run Jobs video encoding)

Version bump: 0.71.31 -> 0.71.32
2025-12-11 11:11:22 -05:00
Andrew Beveridge ☄️ db55205127 feat: Add Cloud Tasks infrastructure for scalable job processing (#18)
Complete scalable architecture implementation (Phase 1 & 2)

Phase 1 - Cloud Tasks Integration:
- Add idempotency checks to all worker endpoints
- Add ENABLE_CLOUD_TASKS and GCP_REGION settings
- Update WorkerService with Cloud Tasks support

Phase 2 - Cloud Run Jobs:
- Add Cloud Run Job for long-running video encoding
- Add CLI entry point to video_worker.py
- Add google-cloud-run dependency

All 1043 tests pass. Infrastructure deployed via Pulumi.
2025-12-11 04:05:42 -05:00
Andrew Beveridge ☄️ 01ff0eb042 Rename README.md to WORKFLOWS.md 2025-12-11 03:49:32 -05:00
Andrew Beveridge ☄️ 438ea6a836 Merge pull request #12 from nomadkaraoke/feature/batch-4-youtube-url-input
feat: Batch 4 - YouTube URL input support
2025-12-11 03:46:31 -05:00
Andrew Beveridge 07f48b6213 test: add tests for URL-based jobs, validation, and worker logging
Improves test coverage from ~62% to ~65%:

- Add tests for download_from_url function and URL-based job workflow
- Add tests for /api/jobs/create-from-url endpoint models
- Add tests for _validate_url with all supported domains
- Add tests for backend/validate.py (import, syntax, config checks)
- Add tests for worker_logging.py (JobLogger, JobLogHandler, job_logging_context)
2025-12-11 03:46:01 -05:00
Andrew Beveridge 5630cfd253 fix: address CodeRabbit review comments for Batch 4
1. _validate_url now handles None and non-string inputs safely
   - Added early guard: if url is None or not isinstance(url, str)
   - Updated function signature to accept Optional[str]

2. Fixed cookie tempfile leak in file_handler.py
   - Use context manager pattern to safely write cookies
   - Track cookie_file_path instead of cookie_file object
   - Guaranteed cleanup in finally block

3. Removed unused style_params_path from submit_job_from_url
   - URL-based jobs don't support custom styles (would need signed URL upload)
   - Added warning when user provides --style_params_json with URL input
   - Updated docstring to clarify limitation

Also merged latest main which added job_logging_context for log isolation.
2025-12-11 03:23:49 -05:00
Andrew Beveridge ☄️ f543ea6efa Merge pull request #17 from nomadkaraoke/fix/deploy-only-on-main
fix: deploy only on merge to main, not on PR or feature branches
2025-12-11 03:19:05 -05:00
Andrew Beveridge ☄️ 5a2c77978b Merge pull request #16 from nomadkaraoke/docs/scalable-architecture-plan
docs: Add scalable architecture plan for 100+ concurrent jobs
2025-12-11 03:13:39 -05:00
Andrew Beveridge 0f7ec8f71e refactor: consolidate CI workflows into single file with clear job names
Merged test-and-publish.yml and test.yml into a single ci.yml workflow.

PR checks now show clear job names:
- Package - Unit Tests (70% coverage)
- Package - Integration Tests
- Package - Build & Install
- Backend - Unit Tests
- Backend - Emulator Integration Tests
- Backend - Code Quality

Deploy jobs (main only):
- Deploy - Publish to PyPI
- Deploy - Frontend (GitHub Pages)
- Deploy - Backend (Cloud Run)

Benefits:
- Single workflow run per PR instead of two
- Clear visibility of what's being tested
- Logical grouping of package vs backend tests
- All deployments still gated behind successful tests
2025-12-11 03:10:58 -05:00
Andrew Beveridge dbb6f3b751 fix: deploy only on merge to main, not on PR or feature branches
Updated the GitHub Actions workflow condition so that:
- PRs run tests but don't deploy
- Feature branch pushes run tests but don't deploy
- Only pushes to main (merges) trigger deployment after tests pass
2025-12-11 03:06:01 -05:00
Andrew Beveridge ☄️ dd287f8c92 Merge pull request #15 from nomadkaraoke/fix/worker-log-isolation
fix: Isolate worker logs using contextvars for concurrent job processing
2025-12-11 03:04:24 -05:00
Andrew Beveridge 07c459258a docs: add scalable architecture plan for 100+ concurrent jobs
Create comprehensive SCALABLE-ARCHITECTURE-PLAN.md covering:
- Cloud Tasks for guaranteed delivery and horizontal scaling
- Cloud Run Jobs for long-running video encoding tasks
- Shared pipeline architecture for code reuse (DRY, SOLID)
- Dedicated resources per job (no CPU/memory contention)
- Implementation roadmap with phased approach
- Resource configuration and cost estimates

Update BACKEND-FEATURE-PARITY-PLAN.md to reference new document.
2025-12-11 03:00:21 -05:00
Andrew Beveridge 3522fdd223 fix: isolate worker logs using contextvars for concurrent job processing
When multiple jobs run concurrently on the same Cloud Run instance,
logs from third-party libraries (karaoke_gen, lyrics_transcriber) were
being captured by all job handlers, causing log mixing between jobs.

This fix:
- Adds contextvars-based job ID tracking to worker_logging.py
- Adds job_logging_context() context manager for setting active job
- Filters log records in JobLogHandler.emit() by current context
- Wraps all worker processing in job_logging_context for isolation

Each job's logs now correctly stay isolated even when workers share
global Python loggers.
2025-12-11 02:56:48 -05:00
Andrew Beveridge 651cbbdbca feat: Batch 5 - Flacfetch audio search for remote CLI
Add support for searching and downloading audio by artist+title in remote mode,
enabling the same functionality as the local CLI's flacfetch integration.

New features:
- `karaoke-gen-remote "Artist" "Title"` - search for audio without providing a file
- `--auto-download` flag for automatic best-quality selection
- `-y` flag also triggers auto-download mode
- Interactive mode displays search results for user selection

Backend changes:
- New AudioSearchService integrating flacfetch library
- New API endpoints: POST /api/audio-search/search, GET /api/audio-search/{job_id}/results, POST /api/audio-search/{job_id}/select
- New job statuses: searching_audio, awaiting_audio_selection, downloading_audio
- Job model extended with audio_search_artist, audio_search_title, auto_download fields

Remote CLI changes:
- Detects artist+title input without file path
- Displays search results interactively
- Handles audio source selection flow
- Supports both interactive and auto-download modes

Tests:
- 20+ unit tests for AudioSearchService and Job model fields
- Tests for state transitions and API response handling

Documentation:
- Updated BACKEND-FEATURE-PARITY-PLAN.md with batch 5 status

Version bump: 0.71.26 -> 0.71.28
2025-12-11 02:38:41 -05:00
Andrew Beveridge ba4534fdbd feat: Batch 6 - Two-Phase Workflow (prep-only, finalise-only, keep-brand-code)
Implements the two-phase workflow for remote CLI, allowing users to:
- Run prep-only mode to stop after review phase and download all outputs
- Run finalise-only mode to upload local prep outputs and continue with finalization
- Preserve existing brand codes when re-running finalisation

Backend changes:
- Added PREP_COMPLETE status to JobStatus enum
- Added prep_only, finalise_only, keep_brand_code fields to Job/JobCreate models
- Updated state transitions to allow new workflow paths
- New endpoint /api/jobs/create-finalise-only for uploading prep outputs
- New endpoint /api/jobs/{job_id}/finalise-uploads-complete
- Modified render_video_worker to transition to PREP_COMPLETE for prep-only jobs
- Modified video_worker to use preserved brand code when set

Remote CLI changes:
- Added --prep-only flag support (passes prep_only=true to backend)
- Added --finalise-only flow (detects local prep folder, uploads files)
- Added --keep-brand-code support (extracts from folder name)
- Added PREP_COMPLETE status handling in job monitor

Tests:
- Added comprehensive tests for new Job model fields and PREP_COMPLETE status
- Added tests for finalise-only endpoints and Pydantic models
- All 151 tests pass

Version bumped to 0.71.27
2025-12-11 02:38:22 -05:00
Andrew Beveridge 441f99edeb Merge main: resolve conflicts with Batch 3 (existing instrumental)
Resolves merge conflicts between Batch 4 (YouTube URL input) and Batch 3
(existing instrumental) which was merged while this branch was in progress.

Combined changes:
- Both Batch 3 and Batch 4 are now marked complete in documentation
- Remote CLI now supports both URL input AND existing_instrumental parameter
- Tests include both existing instrumental tests and URL input tests
- Updated poetry.lock with yt-dlp dependency
2025-12-11 02:30:13 -05:00
Andrew Beveridge ☄️ 2e145dd927 Merge pull request #11 from nomadkaraoke/feature/batch-3-existing-instrumental
feat: Batch 3 - Existing Instrumental Support for Remote CLI
2025-12-11 02:23:02 -05:00
Andrew Beveridge 985f5c7718 feat: Batch 4 - YouTube URL input support
Implements YouTube URL input (P2 parameter) for the remote CLI and backend:

- Remote CLI: Detect URL input via is_url(), route to new submit_job_from_url() method
- Backend: New /api/jobs/create-from-url endpoint accepts URL with optional artist/title
- Backend: Audio worker downloads from URL using yt-dlp, auto-detects artist/title from metadata
- FileHandler: Added download_video() and extract_metadata_from_url() methods
- Dependencies: Added yt-dlp to pyproject.toml

Features:
- URL detection in remote CLI positional arguments
- Auto-detection of artist/title from YouTube video metadata
- Support for YouTube, Vimeo, SoundCloud, and other yt-dlp supported platforms
- Metadata cleanup (removes common suffixes like "Official Video")
- Cookie support for authenticated downloads (YOUTUBE_COOKIES env var)

Tests: 20+ unit tests covering URL validation, request/response models, Job URL field
2025-12-11 02:09:20 -05:00
Andrew Beveridge 2b5ef28789 feat: Batch 3 - Existing Instrumental Support for Remote CLI
Add --existing_instrumental parameter to remote CLI to allow users to upload
and use their own pre-made instrumental file instead of AI-separated audio.

Key changes:
- remote_cli.py: Add existing_instrumental parameter to submit_job()
- file_upload.py: Add existing_instrumental file type with duration validation (±0.5s)
- job.py: Add existing_instrumental_gcs_path field to Job and JobCreate models
- video_worker.py: Download and use user-provided instrumental for final video encoding

Backend still runs AI separation for stems (to provide consistent download outputs),
but uses the user-provided instrumental for the final karaoke video.

Includes 20+ unit tests and maintains >70% coverage.
2025-12-11 02:06:18 -05:00
Andrew Beveridge ☄️ 76940691b1 Merge pull request #10 from nomadkaraoke/fix-upload-max-filesize
feat: implement signed URL uploads to bypass 32MB Cloud Run limit
2025-12-11 01:53:34 -05:00
Andrew Beveridge 4b346dd369 fix: address CodeRabbit review comments
- Remove unused `settings` variable in mark_uploads_complete endpoint
- Rename duplicate TestGCSPathGeneration class to TestSignedUrlGCSPathGeneration
  to avoid shadowing the original test class
2025-12-11 01:45:31 -05:00
Andrew Beveridge 7480950a1b feat: implement signed URL uploads to bypass 32MB Cloud Run limit
The remote CLI was hitting Cloud Run's 32MB request body limit when
uploading large FLAC files with 4K PNG style assets. This implements
a two-step signed URL upload flow that bypasses the limit entirely:

1. CLI calls POST /api/jobs/create-with-upload-urls with job metadata
2. Backend returns signed GCS upload URLs for each file
3. CLI uploads files directly to GCS (no size limit)
4. CLI calls POST /api/jobs/{job_id}/uploads-complete to start processing

Benefits:
- No file size limits (GCS supports up to 5TB)
- Faster uploads (direct to storage, no proxy)
- Works with any HTTP client (no HTTP/2 required)
- Future-proof for web UI uploads

Changes:
- backend/services/storage_service.py: Add generate_signed_upload_url()
- backend/api/routes/file_upload.py: Add two new endpoints + models
- karaoke_gen/utils/remote_cli.py: Refactor submit_job() for signed URLs
- Add 21 new tests for the signed URL upload flow
- Bump version to 0.71.25

Also includes: Download spaCy model at Docker build time (not runtime)
2025-12-11 01:34:15 -05:00
Andrew Beveridge ☄️ 64ada25e9c Merge pull request #9 from nomadkaraoke/feature/batch-2-audio-model-selection
feat: Batch 2 - Audio Separation Model Selection
2025-12-11 01:23:12 -05:00
Andrew Beveridge de6322dd87 feat: Batch 2 - Audio Separation Model Selection
Add support for custom audio separation models in remote CLI:
- `--clean_instrumental_model`: Stage 1 separation model
- `--backing_vocals_models`: Backing vocals separation models
- `--other_stems_models`: Multi-stem separation models (bass, drums, etc.)

Implementation:
- Remote CLI: Added 3 form fields to submit_job(), sends models as comma-separated strings
- Backend: Added 3 Form parameters to /api/jobs/upload, parses comma-separated lists
- Job model: Added clean_instrumental_model, backing_vocals_models, other_stems_models fields
- Audio worker: create_audio_processor() accepts model params from job, passes to Modal API

This enables A/B testing different models and using newer models as they release.
Models are sent to the Modal-hosted audio-separator API which can download any model
on demand and caches them.

Resolves: Batch 2 from BACKEND-FEATURE-PARITY-PLAN.md
2025-12-11 01:05:30 -05:00
Andrew Beveridge ☄️ 49b472429d Merge pull request #8 from nomadkaraoke/chase-feature-parity
feat: Batch 1 - Add lyrics configuration for remote CLI
2025-12-11 00:54:17 -05:00
Andrew Beveridge d57282c770 docs: mark Batch 1 complete, add instructions for future agents
Update BACKEND-FEATURE-PARITY-PLAN.md:
- Mark Batch 1 (Lyrics Configuration) as complete with PR #8
- Update Quick Parity Summary (Lyrics: 100%)
- Mark L1-L4 parameters as ✅ implemented
- Add "Instructions for Implementing a Batch" section:
  - Test coverage requirements (>70%)
  - Required documentation updates checklist
  - Version bump requirement
  - PR format guidelines
- Update Next Steps to reflect current progress
- Add Recent Changes entry for v0.71.22
2025-12-11 00:37:41 -05:00
Andrew Beveridge a41948fe37 test: add unit tests for lyrics configuration parameters
Add comprehensive tests for Batch 1 lyrics configuration:
- TestLyricsFileValidation: validate .txt, .docx, .rtf extensions
- TestLyricsConfigurationFields: verify Job model lyrics fields
- TestLyricsGCSPathGeneration: test GCS path format
- TestJobCreateLyricsFields: verify JobCreate has lyrics fields
- TestLyricsWorkerConfiguration: test LyricsProcessor creation
- TestLyricsOverrideParameters: test lyrics_artist/title overrides
- TestRenderVideoWorkerConfiguration: test subtitle_offset_ms

Bump version to 0.71.22
2025-12-11 00:29:02 -05:00
Andrew Beveridge ffd4e9e377 feat: implement Batch 1 - lyrics configuration for remote CLI
Add support for lyrics override parameters in karaoke-gen-remote:
- L1 --lyrics_artist: Override artist name for lyrics search
- L2 --lyrics_title: Override title for lyrics search
- L3 --lyrics_file: Use user-provided lyrics file instead of API fetch
- L4 --subtitle_offset_ms: Adjust subtitle timing offset

Changes:
- backend/models/job.py: Add lyrics config fields to Job and JobCreate
- backend/api/routes/file_upload.py: Accept lyrics params and file upload
- backend/workers/lyrics_worker.py: Pass lyrics overrides to LyricsProcessor
- backend/workers/render_video_worker.py: Apply subtitle_offset_ms
- karaoke_gen/utils/remote_cli.py: Send lyrics params to backend

Bump version to 0.71.21
2025-12-11 00:24:00 -05:00
Andrew Beveridge ef9c4fc243 Added much improved feature parity plan 2025-12-11 00:15:32 -05:00
Andrew Beveridge 1f030333ae fix: reduce anchor sequence logging verbosity and fix duplicate logs
- Change verbose N-GRAM, ANCHOR SEARCH, and FILTERING logs from INFO to DEBUG level
- Keep only high-level summary logs at INFO (start, completion, totals)
- Fix duplicate logging by removing redundant logger handlers from hierarchy
  - job_log_context no longer adds handlers to root logger
  - LYRICS_WORKER_LOGGERS reduced to top-level package loggers only
- Bump version to 0.71.20
2025-12-10 22:49:59 -05:00
Andrew Beveridge ☄️ 707df6af72 Merge pull request #7 from nomadkaraoke/feature/improve-remote-cli-logging
feat: improve progress logging during video generation for remote CLI
2025-12-10 22:35:01 -05:00
Andrew Beveridge c94c553069 fix: allow job retry by adding FAILED state transitions
The retry endpoint was silently failing because FAILED status had no valid
transitions in STATE_TRANSITIONS. When retrying a failed job:

1. Error state was cleared correctly
2. transition_to_state() returned False (FAILED -> INSTRUMENTAL_SELECTED invalid)
3. Worker was triggered anyway, completed successfully
4. Final transition to COMPLETE also failed (still in FAILED state)

Fix:
- Add retry checkpoint states (INSTRUMENTAL_SELECTED, REVIEW_COMPLETE,
  LYRICS_COMPLETE) as valid transitions from FAILED status
- Add explicit error checking for transition failures in retry endpoint
- Add test coverage for FAILED state retry transitions

This fixes the bug where successfully retried jobs remained stuck in 'failed'
status with error_message=None.
2025-12-10 22:29:30 -05:00
Andrew Beveridge d757b0c860 feat: improve progress logging during video generation for remote CLI
This change addresses the long silence users experienced when monitoring
remote jobs, particularly during video generation (encoding) stage.

Changes:
- video_worker.py: Add detailed progress logging during file downloads
  with estimated times for large files
- video_worker.py: Fix VIDEO_WORKER_LOGGERS to capture KaraokeFinalise
  logs (logger name was mismatched)
- karaoke_finalise.py: Add step-by-step logging for each FFmpeg encoding
  operation with time estimates ([Step 1/6], [Step 2/6], etc.)
- remote_cli.py: Add heartbeat messages when polling has no updates for
  ~30 seconds to show the CLI is still running
- remote_cli.py: Add human-readable status descriptions (e.g. "encoding"
  becomes "Encoding final videos (15-20 min, 4 formats)")

Bump version to 0.71.18
2025-12-10 22:16:57 -05:00
Andrew Beveridge c632e68b5b Updated workflow to push on all branches 2025-12-10 21:54:06 -05:00
Andrew Beveridge ☄️ 9376d53fed Merge pull request #6 from nomadkaraoke/fix-discord-webhook-error
fix: handle Discord webhook errors gracefully and add job retry capability
2025-12-10 20:21:01 -05:00
Andrew Beveridge 18ee2df5a2 test: add comprehensive tests for --retry CLI mode to improve coverage 2025-12-10 20:15:11 -05:00
Andrew Beveridge 68b8cd25a4 fix: handle Discord webhook errors gracefully and add job retry capability
- Fix Discord webhook URL validation to strip trailing whitespace/newlines
  that can be introduced from environment variables or secrets (fixes %0A error)
- Make Discord notification failures non-fatal - log error and continue
- Add --retry CLI command to retry failed jobs from last checkpoint
- Add POST /api/jobs/{job_id}/retry endpoint for retrying failed jobs
- Bump version to 0.71.17

The root cause was environment variables or secrets containing trailing
newline characters (\n) which get URL-encoded as %0A, breaking Discord
API authentication.
2025-12-10 19:31:59 -05:00
Andrew Beveridge ☄️ cd91b06eb9 Merge pull request #5 from nomadkaraoke/replace-modal-with-google-cloud
Replace modal with google cloud for web hosted version
2025-12-10 16:47:53 -05:00
132 changed files with 158927 additions and 2792 deletions

No files matched your search

@@ -0,0 +1,368 @@
---
name: Anchor Sequence Optimization
overview: Replace the current multiprocessing-based anchor sequence algorithm with an efficient single-threaded implementation using hash-based n-gram indexing, greedy longest-first matching, and simplified scoring to achieve sub-30-second performance in both local and cloud environments.
todos:
- id: create-golden-fixtures
content: Create script to generate 10 golden test fixtures from representative song data (varying lengths/complexity), run current algorithm to capture expected anchor outputs
status: completed
- id: create-golden-test
content: Create test_anchor_sequence_golden.py with regression tests that verify anchor finder output matches fixtures, ensure tests pass with current implementation before any changes
status: completed
- id: remove-multiprocessing
content: Remove multiprocessing.Pool from find_anchors() and _remove_overlapping_sequences(), replace with single-threaded loops
status: completed
- id: build-ngram-index
content: Add _build_ngram_index() method to create hash-based lookup table for O(1) n-gram matching
status: completed
- id: greedy-skip
content: Add covered_positions set tracking and skip logic to avoid processing overlapping n-grams
status: completed
- id: simple-scoring
content: Add _simple_score_anchor() with rule-based heuristics, make spaCy scoring optional via use_nlp_scoring flag
status: completed
- id: optimize-overlap
content: Replace O(n^2) overlap removal with O(n log n) sorted single-pass algorithm
status: completed
- id: add-perf-tests
content: Add performance benchmark test asserting <30 second completion for all fixture sizes
status: pending
---
# Anchor Sequence Algorithm Optimization
## Problem Analysis
The current `AnchorSequenceFinder` has critical performance issues:
### Root Causes of Hanging
1. **Multiprocessing Pool with fork()**: Python's `Pool()` uses `fork()` on Linux, which causes deadlocks in Cloud Run containers
2. **spaCy model loading in forked processes**: The `PhraseAnalyzer` loads spaCy's `en_core_web_sm` model, which can hang when forked
3. **Pool creation blocking**: The hang occurs before job submission, so timeout checks never trigger
### Algorithmic Inefficiencies
1. **O(n^2) n-gram matching**: Linear search through references for each n-gram
2. **Redundant work**: Processes all n-gram lengths (3 to ~30) even though longer matches are preferred
3. **O(n^2) overlap removal**: Each anchor checked against all previous anchors
4. **Expensive phrase scoring**: spaCy NLP called twice per anchor (phrase + context)
## Proposed Solution: Hash-Based Greedy Algorithm
### Core Algorithm Changes
**1. Hash-Based N-gram Index (O(1) lookup instead of O(n))**
- Pre-build a dictionary mapping n-grams to their positions in each reference
- Use tuple of cleaned words as key for O(1) lookup
- Build index once during preprocessing
**2. Greedy Longest-First Matching**
- Process n-gram lengths from longest to shortest (current behavior)
- BUT: Skip positions already covered by longer anchors immediately
- Use a "covered positions" set to avoid redundant work
**3. Single-Threaded Processing**
- Remove all `multiprocessing.Pool` usage
- Use efficient algorithms instead of parallelism
- Works identically in local and cloud environments
**4. Simplified Phrase Scoring (Optional)**
- Make spaCy-based scoring optional via configuration flag
- Default to rule-based heuristic scoring:
- Prefer sequences starting/ending at word boundaries
- Prefer sequences matching more reference sources
- Prefer longer sequences
- Option to enable full NLP scoring for higher-quality results offline
**5. O(n log n) Overlap Removal**
- Sort anchors by position
- Single pass with interval tracking using sorted list
- Skip anchors that overlap with any already-selected anchor
### New Data Structures
```python
# N-gram index: maps (word1, word2, ...) -> {source: [positions]}
ngram_index: Dict[Tuple[str, ...], Dict[str, List[int]]]
# Covered positions in transcription (for greedy skip)
covered_trans_positions: Set[int]
```
### Performance Targets
- **Target**: Complete in under 30 seconds for any song
- **Typical song**: 200-400 words, 3 reference sources
- **Expected performance**: 1-5 seconds for typical inputs
## Implementation Plan
### Execution Order (Critical)
1. **Phase 0**: Create golden test fixtures and verify tests pass with current code
2. **Phase 1-5**: Implement optimizations incrementally, running golden tests after each phase
3. **Final**: Add performance benchmarks
### Files to Create/Modify
- `lyrics_transcriber_temp/scripts/generate_anchor_fixtures.py` (new)
- `lyrics_transcriber_temp/tests/fixtures/anchor_golden/*.json` (new)
- `lyrics_transcriber_temp/tests/unit/correction/test_anchor_sequence_golden.py` (new)
- `lyrics_transcriber_temp/lyrics_transcriber/correction/anchor_sequence.py` (modify)
### Key Changes
**Phase 0: Golden Test Setup** (MUST complete and pass before any algorithm changes)
- Create fixture generation script with 10 representative test cases
- Run script to generate fixtures using current implementation
- Create golden test file that loads fixtures and verifies outputs
- Verify all golden tests pass with current code
**Phase 1: Remove Multiprocessing**
- Replace `Pool().apply_async()` with direct function calls
- Remove all pool creation, cleanup, and timeout handling
- Single-threaded loop through n-gram lengths
**Phase 2: Add Hash-Based Index**
- New method `_build_ngram_index()` that builds lookup table during preprocessing
- Modify `_find_matching_sources()` to use O(1) index lookup
- Index built once, shared across all n-gram length iterations
**Phase 3: Greedy Position Skipping**
- Track covered transcription positions in a set
- Skip n-grams that start in already-covered positions
- Reduces redundant work significantly for songs with good matches
**Phase 4: Simplify Scoring**
- Add `use_nlp_scoring: bool = False` parameter to constructor
- New method `_simple_score_anchor()` using rule-based heuristics
- Keep spaCy scoring available for offline/high-quality mode
**Phase 5: Optimize Overlap Removal**
- Sort anchors by (start_position, -length)
- Single pass: keep anchor if no overlap with last kept anchor
- O(n log n) total instead of O(n^2)
## API Compatibility
The public interface remains unchanged:
- `find_anchors(transcribed, references, transcription_result) -> List[ScoredAnchor]`
- `find_gaps(transcribed, anchors, references, transcription_result) -> List[GapSequence]`
New optional constructor parameters:
- `use_nlp_scoring: bool = False` - Enable spaCy-based phrase scoring (slower but higher quality)
- Timeout parameter retained but less critical with efficient algorithm
## Fallback Strategy
If performance is still insufficient for edge cases:
1. **Early termination**: Stop after finding sufficient coverage (e.g., 80% of words anchored)
2. **Adaptive n-gram range**: Start with medium lengths (8-15), expand only if needed
3. **Sampling**: For very long songs, sample n-gram positions instead of exhaustive search
## Phase 0: Golden Test Setup (BEFORE any algorithm changes)
### Goal
Create regression tests using REAL song data from local cache to verify the optimized algorithm produces equivalent results.
### Cache File Structure (Understanding)
The local cache at `~/lyrics-transcriber-cache/` contains:
- **Reference lyrics**: `{provider}_{hash}_converted.json` where provider is spotify/genius/lrclib/musixmatch
- Hash is based on (artist, title) lookup - same song shares same hash across providers
- `_raw.json` contains metadata (track name, artist)
- `_converted.json` contains LyricsData structure with segments/words
- **Transcriptions**: `audioshake_{hash}_converted.json`
- Hash is based on audio file content (different from reference hash)
- Contains TranscriptionData structure
- **Anchor results**: `anchors_{hash}.json`
- Hash is computed from combined inputs (transcription + all references)
### Manual Song Selection
Identify 10 songs from cache with good coverage. Search raw files for track names:
### Finding Song Mappings
To link audioshake transcriptions to reference lyrics, search cache files:
```bash
# Find songs by searching raw files for track names
grep -h "trackName" ~/lyrics-transcriber-cache/spotify_*_raw.json | head -20
# Find audioshake files for a known song (e.g., search for filename patterns)
ls ~/lyrics-transcriber-cache/audioshake_*_raw.json | head -20
# Search converted files for specific lyrics text
grep -l "Napoleon did surrender" ~/lyrics-transcriber-cache/*_converted.json
```
**Target song list** (reference_hash to be paired with audioshake_hash):
1. ABBA - Waterloo (refs: 3365f6a1e42388d84bcbfc0ac6f8aa6c) - short, repetitive chorus
2. The Format - Time Bomb (refs: 3a3802ec28c4ab6689909b7ef784bb6f) - medium length
3. Rancid - Time Bomb (refs: d02d20b854cbc6b890f274bd6c278752) - fast punk lyrics
4. Minutemen - Storm in My House (refs: bc625704b26b307d8680ae5235b74994) - short
5. + 6 more songs to be identified during fixture generation by exploring cache
### Fixture Generation Script
Create `lyrics_transcriber_temp/scripts/generate_anchor_fixtures.py`:
```python
# Input: List of (name, audioshake_hash, reference_hash) tuples
# For each:
# 1. Load audioshake_{hash}_converted.json as transcription
# 2. Load {spotify,genius,lrclib}_{hash}_converted.json as references
# 3. Run current AnchorSequenceFinder
# 4. Save complete fixture with inputs + outputs
```
The script will:
1. Accept a config file listing the 10 song mappings
2. Load real LyricsData from cache files
3. Run the CURRENT AnchorSequenceFinder implementation
4. Save fixtures that capture the real behavior
### Fixture Format
```
lyrics_transcriber_temp/tests/fixtures/anchor_golden/
- waterloo_abba.json
- time_bomb_format.json
- time_bomb_rancid.json
- ...
```
Each fixture contains REAL data:
```json
{
"name": "waterloo_abba",
"description": "ABBA - Waterloo, short song with repetitive chorus",
"source_hashes": {
"audioshake": "1654b7c0755fad7f23bc888308f274de",
"references": "3365f6a1e42388d84bcbfc0ac6f8aa6c"
},
"transcription": {
"segments": [...], // Full LyricsData from audioshake cache
"word_count": 150
},
"references": {
"spotify": { "segments": [...] },
"genius": { "segments": [...] },
"lrclib": { "segments": [...] }
},
"expected_anchors": [
{
"text": "My my at Waterloo Napoleon did surrender",
"transcription_position": 0,
"length": 7,
"reference_positions": {"spotify": 0, "genius": 0, "lrclib": 0},
"confidence": 1.0
},
// ... all anchors from current implementation
],
"expected_anchor_count": 25,
"expected_word_coverage": 142,
"expected_coverage_percent": 94.6
}
```
### Golden Test
Create `lyrics_transcriber_temp/tests/unit/correction/test_anchor_sequence_golden.py`:
```python
import pytest
import json
from pathlib import Path
FIXTURES_DIR = Path(__file__).parent.parent.parent / "fixtures" / "anchor_golden"
@pytest.fixture(params=list(FIXTURES_DIR.glob("*.json")))
def golden_fixture(request):
with open(request.param) as f:
return json.load(f)
def test_anchor_count_matches(golden_fixture, anchor_finder):
"""Anchor count should be within 10% of expected."""
anchors = run_anchor_finder(golden_fixture)
expected = golden_fixture["expected_anchor_count"]
assert abs(len(anchors) - expected) <= expected * 0.1
def test_key_anchors_match(golden_fixture, anchor_finder):
"""First, last, and longest anchors should match exactly."""
anchors = run_anchor_finder(golden_fixture)
expected = golden_fixture["expected_anchors"]
# Compare first anchor
assert anchors[0].text == expected[0]["text"]
# Compare positions match
...
def test_word_coverage(golden_fixture, anchor_finder):
"""Total word coverage should be within 5% of expected."""
anchors = run_anchor_finder(golden_fixture)
coverage = calculate_coverage(anchors, golden_fixture)
expected = golden_fixture["expected_coverage_percent"]
assert abs(coverage - expected) <= 5.0
def test_no_invalid_positions(golden_fixture, anchor_finder):
"""All anchor positions should be valid."""
anchors = run_anchor_finder(golden_fixture)
word_count = golden_fixture["transcription"]["word_count"]
for anchor in anchors:
assert 0 <= anchor.transcription_position < word_count
```
### Validation Steps
1. **Generate fixtures from current implementation**:
```bash
cd lyrics_transcriber_temp
python scripts/generate_anchor_fixtures.py
```
2. **Verify fixtures were created**:
```bash
ls tests/fixtures/anchor_golden/
# Should show 10 .json files
```
3. **Run golden tests with CURRENT implementation**:
```bash
pytest tests/unit/correction/test_anchor_sequence_golden.py -v
```
4. **All tests MUST pass before any algorithm changes**
5. **After each optimization phase, re-run golden tests**:
```bash
pytest tests/unit/correction/test_anchor_sequence_golden.py -v
# Must still pass!
```
## Testing (Post-Optimization)
- Golden tests verify correctness (same outputs as before)
- Existing unit tests verify API compatibility
- Performance benchmark test with 30-second timeout assertion
- Test with various song lengths via golden fixtures
@@ -0,0 +1,717 @@
---
name: Instrumental Review UI
overview: Add an intelligent instrumental selection workflow with audio analysis, waveform visualization, playback controls, and the ability to mute sections of backing vocals to create custom instrumentals. This replaces the current text-only CLI prompt with a web UI that works for both local and remote workflows, with shared core logic for DRY/maintainable code.
todos:
- id: shared-core-module
content: Create karaoke_gen/instrumental_review/ shared module with AudioAnalyzer and AudioEditor classes (used by both local and remote)
status: completed
- id: shared-core-tests
content: Add comprehensive unit tests for shared module with >80% coverage (tests/unit/test_instrumental_review.py)
status: completed
- id: backend-services
content: Create backend service wrappers that use shared core module (audio_analysis_service.py, audio_editing_service.py)
status: completed
- id: backend-services-tests
content: Add unit tests for backend services with mocked GCS interactions (backend/tests/test_audio_services.py)
status: completed
- id: api-endpoints
content: Add new API endpoints for analysis, audio streaming, and custom instrumental creation
status: completed
- id: api-endpoints-tests
content: Add API endpoint tests with >80% coverage (backend/tests/test_instrumental_api.py)
status: completed
- id: render-worker-integration
content: Integrate audio analysis into render_video_worker before transitioning to awaiting_instrumental_selection
status: completed
- id: render-worker-tests
content: Add/update render_video_worker tests to cover new analysis integration
status: completed
- id: frontend-components
content: Create React components (WaveformDisplay, AudioPlayer, RegionSelector, InstrumentalReviewPage)
status: completed
- id: local-cli-integration
content: Update local gen_cli.py to use shared core module with local file paths (feature parity with remote)
status: completed
- id: local-cli-review-server
content: Add lightweight review server to local CLI for browser-based review (similar to lyrics review pattern)
status: completed
- id: remote-cli-update
content: Update remote_cli.py to use new analysis API and open browser for review
status: completed
- id: cli-tests
content: Add CLI tests verifying feature parity between local and remote modes
status: completed
- id: integration-tests
content: Add end-to-end integration tests for the full instrumental review workflow
status: completed
---
# Intelligent Instrumental Selection Plan
## Current State
The instrumental selection workflow currently:
- Presents a simple 1/2 choice in the CLI (clean vs. with_backing)
- Requires manual inspection in Audacity to evaluate backing vocals quality
- Cannot handle the common "Scenario 3" where backing vocals need partial muting
- Has no audio preview capability in remote mode
## Target Architecture
```
┌────────────────────────────────────────────────────────────────────────────┐
│ INSTRUMENTAL REVIEW WORKFLOW │
├────────────────────────────────────────────────────────────────────────────┤
│ │
│ 1. AUDIO ANALYSIS (automatic after render_video completes) │
│ │ │
│ ├── Analyze backing vocals stem for amplitude │
│ ├── Detect audible segments (> -40dB threshold) │
│ ├── Generate waveform visualization image │
│ └── Store analysis results in job state_data │
│ │
│ 2. AUTO-DECISION CHECK │
│ │ │
│ └── IF no audible segments detected: │
│ ├── Set recommended_selection = "clean" │
│ └── Skip to step 4 (or show minimal UI with suggestion) │
│ │
│ 3. HUMAN REVIEW (Web UI) │
│ │ │
│ ├── Display waveform with time axis │
│ ├── Show audible segment markers │
│ ├── Audio playback with waveform-click seeking │
│ ├── Toggle between: clean / with_backing / preview-custom │
│ ├── Select regions to mute (click/drag on waveform) │
│ └── Create custom instrumental if regions selected │
│ │
│ 4. SELECTION SUBMISSION │
│ │ │
│ └── POST /api/jobs/{job_id}/select-instrumental │
│ with selection = "clean" | "with_backing" | "custom" │
│ │
└────────────────────────────────────────────────────────────────────────────┘
```
## Architecture: Shared Core for DRY/SOLID
To ensure feature parity between local and remote CLIs while keeping code maintainable, we use a **shared core module** pattern:
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ SHARED CORE ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ karaoke_gen/instrumental_review/ <── SHARED CORE MODULE │
│ ├── __init__.py │
│ ├── analyzer.py # AudioAnalyzer class (pure Python, no GCS) │
│ ├── editor.py # AudioEditor class (pure Python, no GCS) │
│ ├── waveform.py # WaveformGenerator class (matplotlib) │
│ └── models.py # Pydantic models: AnalysisResult, MuteRegion │
│ │
│ ┌─────────────────────┐ ┌──────────────────────┐ │
│ │ LOCAL CLI │ │ REMOTE BACKEND │ │
│ │ (karaoke-gen) │ │ (Cloud Run) │ │
│ ├─────────────────────┤ ├──────────────────────┤ │
│ │ │ │ │ │
│ │ gen_cli.py │ │ audio_analysis_ │ │
│ │ └── uses shared ───┼──────────────┼──► service.py │ │
│ │ core directly │ │ └── wraps shared│ │
│ │ with local │ │ core with │ │
│ │ file paths │ │ GCS I/O │ │
│ │ │ │ │ │
│ │ local_review_ │ │ audio_editing_ │ │
│ │ server.py │ │ service.py │ │
│ │ └── serves same ───┼──────────────┼──► (same pattern) │ │
│ │ React UI │ │ │ │
│ │ │ │ │ │
│ └─────────────────────┘ └──────────────────────┘ │
│ │
│ BENEFITS: │
│ ✓ Single source of truth for analysis logic │
│ ✓ Bug fixes apply to both local and remote │
│ ✓ Easy to test core logic without GCS mocks │
│ ✓ Backend services are thin wrappers (Single Responsibility) │
│ ✓ Same React UI served by both local server and Cloud Run │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Shared Core Module Design
```python
# karaoke_gen/instrumental_review/analyzer.py
class AudioAnalyzer:
"""
Analyzes audio files for backing vocals content.
Pure Python - no cloud dependencies. Works with local file paths.
"""
def __init__(self, silence_threshold_db: float = -40.0,
min_segment_duration_ms: int = 100):
self.silence_threshold_db = silence_threshold_db
self.min_segment_duration_ms = min_segment_duration_ms
def analyze(self, audio_path: str) -> AnalysisResult:
"""Analyze audio file and return structured result."""
...
def get_amplitude_envelope(self, audio_path: str,
window_ms: int = 100) -> List[float]:
"""Get amplitude envelope for waveform rendering."""
...
# karaoke_gen/instrumental_review/editor.py
class AudioEditor:
"""
Creates custom instrumentals by muting regions of backing vocals.
Pure Python - no cloud dependencies.
"""
def create_custom_instrumental(
self,
clean_instrumental_path: str,
backing_vocals_path: str,
mute_regions: List[MuteRegion],
output_path: str
) -> str:
"""Create custom instrumental with muted regions."""
...
# karaoke_gen/instrumental_review/waveform.py
class WaveformGenerator:
"""Generates waveform visualization images."""
def generate(self, audio_path: str, output_path: str,
segments: List[AudibleSegment] = None,
width: int = 1200, height: int = 200) -> str:
"""Generate waveform PNG with optional segment highlighting."""
...
```
## Implementation Components
### 1. Shared Core Module (karaoke_gen/instrumental_review/)
Create the shared module first - this is the foundation:
```python
# karaoke_gen/instrumental_review/models.py
from pydantic import BaseModel
from typing import List, Optional
from enum import Enum
class RecommendedSelection(str, Enum):
CLEAN = "clean"
WITH_BACKING = "with_backing"
REVIEW_NEEDED = "review_needed"
class AudibleSegment(BaseModel):
start_seconds: float
end_seconds: float
duration_seconds: float
avg_amplitude_db: float
class MuteRegion(BaseModel):
start_seconds: float
end_seconds: float
class AnalysisResult(BaseModel):
has_audible_content: bool
total_duration_seconds: float
audible_segments: List[AudibleSegment]
recommended_selection: RecommendedSelection
waveform_data: Optional[List[float]] = None # For JSON API response
```
### 2. Backend Service Wrappers
Create `backend/services/audio_analysis_service.py`:
```python
# Key functionality:
- analyze_backing_vocals(audio_path) -> AnalysisResult
- Uses pydub for audio loading
- Calculates RMS amplitude in dB across time windows
- Detects audible segments (contiguous regions > threshold)
- Returns: {
"has_audible_content": bool,
"total_duration_seconds": float,
"audible_segments": [
{"start_seconds": float, "end_seconds": float,
"duration_seconds": float, "avg_amplitude_db": float}
],
"recommended_selection": "clean" | "with_backing" | "review_needed"
}
- generate_waveform_image(audio_path, output_path, segments=None) -> str
- Uses matplotlib to render amplitude envelope over time
- Highlights audible segments with different colors
- Returns path to generated PNG image
```
### 2. Audio Editing Service
Create `backend/services/audio_editing_service.py`:
```python
# Key functionality:
- create_custom_instrumental(
clean_instrumental_path: str,
backing_vocals_path: str,
mute_regions: List[Tuple[float, float]], # [(start_s, end_s), ...]
output_path: str
) -> str
- Loads backing vocals
- Applies silence to mute_regions
- Mixes with clean instrumental
- Saves as new FLAC file
```
### 3. Backend API Endpoints
Add to `backend/api/routes/jobs.py`:
```python
GET /api/jobs/{job_id}/instrumental-analysis
# Returns analysis data + waveform URL
GET /api/jobs/{job_id}/audio-stream/{stem_type}
# Stream audio for playback (clean, backing_vocals, combined)
POST /api/jobs/{job_id}/create-custom-instrumental
# Body: {"mute_regions": [[0.5, 2.3], [45.0, 48.5]]}
# Creates custom instrumental, updates job state
POST /api/jobs/{job_id}/select-instrumental
# Extended to accept: "clean" | "with_backing" | "custom"
```
### 4. Frontend Web UI
Create `frontend/app/jobs/[id]/instrumental-review/page.tsx`:
```
Components needed:
- WaveformDisplay: SVG/Canvas waveform with time axis
- AudioPlayer: HTML5 audio with controls + waveform sync
- RegionSelector: Click/drag to select mute regions
- InstrumentalOptions: Radio buttons for clean/backing/custom
- SegmentList: Table showing detected audible segments
```
Key interactions:
- Click on waveform to seek playback position
- Drag on waveform to select mute region
- Toggle audio source (clean/backing/preview)
- Submit selection
### 5. Backend Worker Updates
Modify `backend/workers/render_video_worker.py`:
```python
# After video render completes, before transitioning to AWAITING_INSTRUMENTAL_SELECTION:
1. Download backing_vocals stem
2. Run audio analysis
3. Generate waveform image
4. Upload waveform to GCS
5. Store analysis results in job.state_data
6. If no audible content: set recommended_selection = "clean"
7. Transition to AWAITING_INSTRUMENTAL_SELECTION
```
### 6. CLI Updates
Modify `karaoke_gen/utils/remote_cli.py`:
```python
def handle_instrumental_selection(self, job_id: str):
# Get analysis data
analysis = self.client.get_instrumental_analysis(job_id)
# If no backing vocals detected, suggest auto-select
if analysis['recommended_selection'] == 'clean':
if self.config.non_interactive:
# Auto-select clean
self.client.select_instrumental(job_id, 'clean')
return
else:
# Show suggestion, offer to open browser for verification
self.logger.info("No backing vocals detected - recommending clean instrumental")
# Open browser anyway so user can verify if desired
# Open browser to instrumental review UI
self.open_instrumental_review_ui(job_id)
# Poll until selection made
while job.status == 'awaiting_instrumental_selection':
time.sleep(self.config.poll_interval)
```
### 7. Local CLI Integration (Feature Parity)
The local CLI (`karaoke-gen`) uses the **same shared core module** as the backend:
```python
# karaoke_gen/utils/gen_cli.py - Updated instrumental selection flow
def handle_instrumental_selection(self, output_dir: str, base_name: str):
"""Handle instrumental selection using shared core module."""
from karaoke_gen.instrumental_review import AudioAnalyzer, WaveformGenerator
# Paths to local files (already exist from audio separation)
backing_vocals_path = os.path.join(output_dir, "stems",
f"{base_name} (Backing Vocals ...).flac")
clean_instrumental_path = os.path.join(output_dir, "stems",
f"{base_name} (Instrumental ...).flac")
# Use SAME analyzer as backend
analyzer = AudioAnalyzer(silence_threshold_db=-40.0)
analysis = analyzer.analyze(backing_vocals_path)
# Generate waveform locally
waveform_path = os.path.join(output_dir, "backing_vocals_waveform.png")
WaveformGenerator().generate(backing_vocals_path, waveform_path,
analysis.audible_segments)
if not analysis.has_audible_content:
self.logger.info("No backing vocals detected - using clean instrumental")
if not self.non_interactive:
# Offer to open browser for verification
...
return "clean"
# Start local review server (serves same React UI)
from karaoke_gen.instrumental_review.server import InstrumentalReviewServer
server = InstrumentalReviewServer(
output_dir=output_dir,
base_name=base_name,
analysis=analysis,
waveform_path=waveform_path,
)
server.start_and_open_browser() # Blocking until user submits selection
return server.get_selection()
```
### 8. Local Review Server
Create `karaoke_gen/instrumental_review/server.py`:
```python
"""
Lightweight local server for instrumental review.
Serves the same React frontend used by the cloud backend.
Similar pattern to LyricsTranscriber's ReviewServer.
"""
from http.server import HTTPServer, SimpleHTTPRequestHandler
import json
import threading
class InstrumentalReviewServer:
"""
Local HTTP server for instrumental review UI.
Endpoints:
- GET /api/analysis - Returns analysis data
- GET /api/audio/{type} - Streams audio files
- GET /api/waveform - Returns waveform image
- POST /api/create-custom - Creates custom instrumental
- POST /api/select - Submits selection
- GET /* - Serves React frontend static files
"""
def __init__(self, output_dir: str, base_name: str,
analysis: AnalysisResult, waveform_path: str):
...
def start_and_open_browser(self, port: int = 8765):
"""Start server, open browser, block until selection submitted."""
...
def get_selection(self) -> str:
"""Get the user's selection after browser submission."""
...
```
## Testing Strategy
### Test Coverage Requirements
All new code must meet the project's **70% minimum coverage** requirement, with critical paths targeting **>80%**:
| Module | Target Coverage | Test File |
|--------|----------------|-----------|
| `karaoke_gen/instrumental_review/analyzer.py` | >90% | `tests/unit/test_instrumental_analyzer.py` |
| `karaoke_gen/instrumental_review/editor.py` | >90% | `tests/unit/test_instrumental_editor.py` |
| `karaoke_gen/instrumental_review/waveform.py` | >80% | `tests/unit/test_instrumental_waveform.py` |
| `karaoke_gen/instrumental_review/server.py` | >70% | `tests/unit/test_instrumental_server.py` |
| `backend/services/audio_analysis_service.py` | >80% | `backend/tests/test_audio_analysis_service.py` |
| `backend/services/audio_editing_service.py` | >80% | `backend/tests/test_audio_editing_service.py` |
| `backend/api/routes/jobs.py` (new endpoints) | >80% | `backend/tests/test_instrumental_api.py` |
### Test Categories
**1. Unit Tests (Shared Core) - tests/unit/test_instrumental_*.py**
```python
# tests/unit/test_instrumental_analyzer.py
class TestAudioAnalyzer:
def test_analyze_silent_audio_returns_no_audible_content(self):
"""Silent audio should return has_audible_content=False."""
def test_analyze_audio_with_content_detects_segments(self):
"""Audio with content should detect audible segments."""
def test_segment_detection_respects_threshold(self):
"""Segments below threshold should not be detected."""
def test_segment_merging_for_close_segments(self):
"""Close segments should be merged into one."""
def test_recommended_selection_clean_for_silent(self):
"""Silent audio should recommend 'clean'."""
def test_recommended_selection_review_for_content(self):
"""Audio with content should recommend 'review_needed'."""
# tests/unit/test_instrumental_editor.py
class TestAudioEditor:
def test_mute_single_region(self):
"""Single mute region should be applied correctly."""
def test_mute_multiple_regions(self):
"""Multiple mute regions should all be applied."""
def test_mute_region_boundaries(self):
"""Mute region at start/end of audio should work."""
def test_output_format_is_flac(self):
"""Output should be FLAC format."""
def test_combines_with_clean_instrumental(self):
"""Result should be backing vocals mixed with clean instrumental."""
```
**2. Unit Tests (Backend Services) - backend/tests/test_audio_*_service.py**
```python
# backend/tests/test_audio_analysis_service.py
class TestAudioAnalysisService:
@pytest.fixture
def mock_storage(self):
"""Mock GCS storage service."""
def test_analyze_downloads_from_gcs(self, mock_storage):
"""Service should download audio from GCS before analysis."""
def test_analyze_uploads_waveform_to_gcs(self, mock_storage):
"""Service should upload waveform image to GCS."""
def test_analyze_updates_job_state_data(self, mock_storage):
"""Service should store analysis in job.state_data."""
```
**3. API Endpoint Tests - backend/tests/test_instrumental_api.py**
```python
# backend/tests/test_instrumental_api.py
class TestInstrumentalAnalysisEndpoint:
def test_get_analysis_returns_data(self, client, mock_job):
"""GET /instrumental-analysis should return analysis data."""
def test_get_analysis_requires_correct_status(self, client, mock_job):
"""Should return 400 if job not in correct status."""
class TestAudioStreamEndpoint:
def test_stream_backing_vocals(self, client, mock_job):
"""Should stream backing vocals audio."""
def test_stream_clean_instrumental(self, client, mock_job):
"""Should stream clean instrumental audio."""
class TestCreateCustomInstrumental:
def test_create_with_mute_regions(self, client, mock_job):
"""Should create custom instrumental with mute regions."""
def test_validates_mute_regions(self, client, mock_job):
"""Should validate mute region boundaries."""
```
**4. Integration Tests - tests/integration/test_instrumental_review.py**
```python
# tests/integration/test_instrumental_review_flow.py
class TestInstrumentalReviewFlow:
"""End-to-end tests for instrumental review workflow."""
def test_full_flow_clean_selection(self):
"""Test complete flow: analysis -> review -> select clean."""
def test_full_flow_with_backing_selection(self):
"""Test complete flow: analysis -> review -> select with_backing."""
def test_full_flow_custom_creation(self):
"""Test complete flow: analysis -> mute regions -> custom."""
def test_auto_recommend_clean_for_silent_backing(self):
"""When no backing vocals, should recommend clean."""
```
**5. CLI Feature Parity Tests - tests/unit/test_cli_feature_parity.py**
```python
# tests/unit/test_cli_feature_parity.py
class TestCLIFeatureParity:
"""Verify local and remote CLIs have equivalent functionality."""
def test_both_clis_use_same_analyzer(self):
"""Both CLIs should use AudioAnalyzer from shared module."""
def test_both_clis_support_custom_instrumental(self):
"""Both CLIs should support creating custom instrumentals."""
def test_analysis_results_identical(self):
"""Same audio should produce identical analysis from both CLIs."""
```
### Test Data
Create test fixtures in `tests/data/instrumental_review/`:
- `silent_backing_vocals.flac` - Silent/near-silent audio (~10s)
- `backing_vocals_with_content.flac` - Audio with clear backing vocals (~30s)
- `backing_vocals_mixed_quality.flac` - Audio with some good/some bad sections (~30s)
Use pydub to programmatically generate these in `tests/conftest.py` fixtures.
## File Changes Summary
| Category | Files to Create/Modify |
|----------|------------------------|
| **Shared Core** | `karaoke_gen/instrumental_review/__init__.py` (new) |
| Shared Core | `karaoke_gen/instrumental_review/models.py` (new) |
| Shared Core | `karaoke_gen/instrumental_review/analyzer.py` (new) |
| Shared Core | `karaoke_gen/instrumental_review/editor.py` (new) |
| Shared Core | `karaoke_gen/instrumental_review/waveform.py` (new) |
| Shared Core | `karaoke_gen/instrumental_review/server.py` (new) |
| **Backend Services** | `backend/services/audio_analysis_service.py` (new) |
| Backend Services | `backend/services/audio_editing_service.py` (new) |
| **API** | `backend/api/routes/jobs.py` (modify - add endpoints) |
| API | `backend/api/routes/instrumental.py` (new - if endpoints get large) |
| **Workers** | `backend/workers/render_video_worker.py` (modify) |
| **Models** | `backend/models/job.py` (add analysis fields to state_data) |
| **Frontend** | `frontend/app/jobs/[id]/instrumental-review/page.tsx` (new) |
| Frontend | `frontend/components/waveform-display.tsx` (new) |
| Frontend | `frontend/components/audio-player-with-waveform.tsx` (new) |
| Frontend | `frontend/components/region-selector.tsx` (new) |
| **Local CLI** | `karaoke_gen/utils/gen_cli.py` (modify) |
| **Remote CLI** | `karaoke_gen/utils/remote_cli.py` (modify) |
| **Tests - Shared** | `tests/unit/test_instrumental_analyzer.py` (new) |
| Tests - Shared | `tests/unit/test_instrumental_editor.py` (new) |
| Tests - Shared | `tests/unit/test_instrumental_waveform.py` (new) |
| Tests - Shared | `tests/unit/test_instrumental_server.py` (new) |
| **Tests - Backend** | `backend/tests/test_audio_analysis_service.py` (new) |
| Tests - Backend | `backend/tests/test_audio_editing_service.py` (new) |
| Tests - Backend | `backend/tests/test_instrumental_api.py` (new) |
| **Tests - Parity** | `tests/unit/test_cli_feature_parity.py` (new) |
| **Tests - Integration** | `tests/integration/test_instrumental_review.py` (new) |
| **Test Data** | `tests/conftest.py` (modify - add fixtures) |
## Dependencies
Already available in `pyproject.toml`:
- `pydub>=0.25.1` - Audio loading and manipulation
- `matplotlib>=3` - Waveform visualization
- `numpy>=2` - Audio array processing
Frontend:
- React/Next.js (existing)
- HTML5 Audio API (native)
- Canvas or SVG for waveform (native)
## Implementation Order
Execute in this order to enable incremental testing:
### Phase 1: Shared Core Module (Foundation)
1. Create `karaoke_gen/instrumental_review/` package with models, analyzer, editor, waveform
2. Add comprehensive unit tests for shared core (>90% coverage)
3. Create test audio fixtures in `tests/conftest.py`
### Phase 2: Backend Integration
4. Create backend service wrappers with GCS integration
5. Add backend service unit tests with mocked GCS
6. Add new API endpoints to `backend/api/routes/jobs.py`
7. Add API endpoint tests
8. Integrate analysis into `render_video_worker.py`
9. Update render worker tests
### Phase 3: Frontend
10. Create React components (WaveformDisplay, AudioPlayer, RegionSelector)
11. Create instrumental review page
12. Test frontend manually with backend
### Phase 4: CLI Integration
13. Update `remote_cli.py` to use new analysis API and open browser
14. Create local review server `karaoke_gen/instrumental_review/server.py`
15. Update `gen_cli.py` to use shared core and local server
16. Add CLI feature parity tests
### Phase 5: Integration Testing
17. Add end-to-end integration tests
18. Manual testing of full workflow (local and remote)
### Verification Checklist
- [ ] All unit tests pass: `pytest tests/unit/ -v`
- [ ] All backend tests pass: `pytest backend/tests/ -v`
- [ ] Coverage meets 70% minimum: `pytest --cov=karaoke_gen --cov-fail-under=70`
- [ ] Local CLI works: `karaoke-gen "Artist" "Title"` (reaches instrumental selection)
- [ ] Remote CLI works: `karaoke-gen-remote ./audio.flac "Artist" "Title"`
- [ ] Both CLIs produce identical analysis for same audio
- [ ] Custom instrumental creation works in both modes
- [ ] Version bumped in `pyproject.toml`
@@ -0,0 +1,217 @@
---
name: MidiCo-Style Synchronizer
overview: Build a new MidiCo-style Synchronizer component for efficient re-syncing of lyrics timing, particularly for fixing progressive drift in the second half of songs. This involves refactoring the "Edit All" entry point and creating a high-performance timeline-based sync interface.
todos:
- id: mode-selection-modal
content: Create ModeSelectionModal for 'Replace All' vs 'Re-sync Existing' choice
status: completed
- id: timeline-canvas
content: Build canvas-based TimelineCanvas component for high-performance word block rendering
status: completed
- id: upcoming-words-bar
content: Create UpcomingWordsBar component showing next unsynced words in fixed position
status: completed
- id: sync-controls
content: Build SyncControls with Start Sync, Clear Sync, Unsync from Cursor, Delete Selected buttons
status: completed
- id: lyrics-synchronizer
content: Create main LyricsSynchronizer component integrating all sub-components
status: completed
- id: unsync-from-cursor
content: Add 'unsync from cursor' functionality to useManualSync hook
status: completed
- id: integrate-modal
content: Integrate new synchronizer into ReplaceAllLyricsModal with mode selection
status: completed
- id: word-selection
content: Implement single-click and drag-select for word blocks
status: completed
- id: zoom-slider
content: Add smooth zoom slider with 4.5s-24s range
status: completed
- id: performance-test
content: Test performance with long songs and optimize as needed
status: completed
---
# MidiCo-Style Synchronizer Implementation
## Current State Analysis
The existing implementation in [`ReplaceAllLyricsModal.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/ReplaceAllLyricsModal.tsx) has two phases:
1. **Paste Phase** - User pastes new lyrics text
2. **Sync Phase** - Manual sync using spacebar with timeline view
Key existing components:
- [`useManualSync.ts`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/hooks/useManualSync.ts) - Spacebar handling, tap/hold detection
- [`EditTimelineSection.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/EditTimelineSection.tsx) - Timeline controls with zoom/scroll
- [`TimelineEditor.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/TimelineEditor.tsx) - Timeline rendering
**Problems with current implementation:**
1. Forces full lyrics replacement - no option to keep existing sync data
2. Performance issues (laggy, slow rendering)
3. Poor UX for seeing upcoming words while syncing
4. No "unsync from cursor" feature for fixing drift
5. No two-level word block layout for overlapping segments
## Architecture Overview
```mermaid
flowchart TB
subgraph Entry [Entry Point]
EditAll[Edit All Button]
EditAll --> ChoiceModal[Mode Selection Modal]
ChoiceModal -->|Replace All| PastePhase[Paste New Lyrics]
ChoiceModal -->|Re-sync Existing| SyncView[Synchronizer View]
PastePhase --> SyncView
end
subgraph Sync [Synchronizer Component]
SyncView --> Timeline[Timeline Canvas]
SyncView --> Controls[Control Buttons]
SyncView --> UpcomingWords[Upcoming Words Display]
Timeline --> WordBlocks[Word Blocks Renderer]
Timeline --> Playhead[Playhead Marker]
Timeline --> TimeBar[Time Bar]
end
subgraph Audio [Audio Integration]
AudioPlayer[Audio Player]
AudioPlayer <--> SyncView
end
```
## Implementation Plan
### Phase 1: Entry Point Refactor
Modify the "Edit All" button flow to show a choice modal:
```typescript
// New ModeSelectionModal component
interface ModeSelectionModalProps {
open: boolean
onClose: () => void
onSelectReplace: () => void // Goes to paste phase
onSelectResync: () => void // Goes directly to sync view
}
```
- **Replace All Lyrics**: Current flow (paste box -> sync)
- **Re-sync Existing Lyrics**: Load current segments directly into sync view
### Phase 2: New Synchronizer Component
Create a new high-performance `LyricsSynchronizer.tsx` component inspired by MidiCo:
#### Core Features
1. **Timeline Canvas** (HTML5 Canvas for performance)
- Time bar at top (MM:SS format with decisecond ticks)
- Playhead with arrow indicator + vertical line
- Red word blocks with text labels
- Two-level word layout for overlapping segments
- Zoom slider (4.5s to 24s visible range)
- Horizontal scroll with smooth panning
2. **Upcoming Words Display** (dual location)
- Fixed position above timeline: next words as red/white blocks
- On timeline: words appear to right of playhead during sync
3. **Control Buttons**
- **Start Sync** - Begin sync mode from first unsynced word
- **Clear Sync** - Reset all word timings to null
- **Edit Lyrics** - Modal to edit lyrics text (warns about timing loss)
- **Unsync from Cursor** - Critical: reset all words after playhead position
- **Split/Edit Word** - Edit single selected word
- **Delete Selected** - Remove selected word blocks
4. **Word Selection**
- Single click: select one word (white border)
- Click-drag on background: selection rectangle for multi-select
- Click on time bar: set playhead position
5. **Spacebar Sync Logic** (reuse/enhance `useManualSync`)
- Tap: set start time, default 500ms duration
- Hold: set both start and end times
- Handle previous word end time calculation
### Phase 3: Performance Optimization
The current `TimelineEditor` uses DOM elements which causes lag. New approach:
1. **Canvas-based rendering** for word blocks and timeline
- Single canvas element, no DOM nodes per word
- RequestAnimationFrame for smooth playhead updates
- Only re-render changed regions
2. **Virtual scrolling** for very long songs
- Only render visible word blocks
- Cache off-screen block positions
3. **Debounced state updates**
- Batch word timing updates
- Throttle zoom/scroll handlers
### Phase 4: Apply/Cancel Flow
Changes are held in local state until user clicks Apply:
```typescript
interface SynchronizerState {
words: Word[] // Working copy of words
segments: LyricsSegment[] // Working copy of segments
isDirty: boolean // Has unsaved changes
}
```
- **Apply**: Commit changes to parent component
- **Cancel**: Discard changes, close modal
## File Changes
### New Files
- `src/components/LyricsSynchronizer.tsx` - Main synchronizer component
- `src/components/LyricsSynchronizer/TimelineCanvas.tsx` - Canvas-based timeline
- `src/components/LyricsSynchronizer/UpcomingWordsBar.tsx` - Fixed upcoming words display
- `src/components/LyricsSynchronizer/SyncControls.tsx` - Control buttons
- `src/components/ModeSelectionModal.tsx` - Entry point choice modal
### Modified Files
- [`ReplaceAllLyricsModal.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/ReplaceAllLyricsModal.tsx) - Add mode selection, integrate new synchronizer
- [`useManualSync.ts`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/hooks/useManualSync.ts) - Add "unsync from cursor" functionality
## Key UX Details from MidiCo Reference
Per the documentation at [`MIDICO_SYNCHRONIZER_REFERENCE.md`](lyrics_transcriber_temp/docs/MIDICO_SYNCHRONIZER_REFERENCE.md):
1. **Zoom range**: 4.5s (max zoom) to 24s (min zoom) visible
2. **Timeline click behavior**:
- Time bar click = set playhead (no playback change)
- Background click = nothing (prevent accidental deselection)
- Word block click = select
3. **Two-level word blocks**: Adjacent/overlapping words display on alternating vertical levels
4. **Playhead**: Small arrow in time bar + white vertical line spanning timeline height
5. **Selection rectangle**: Click-drag on background creates selection box
## Testing Strategy
1. Manual testing with the provided URL: `http://localhost:8000/?baseApiUrl=...`
2. Test sync accuracy with songs that have known drift issues
3. Performance testing with 400+ word songs
4. Test "unsync from cursor" workflow for fixing drift
+21
View File
@@ -0,0 +1,21 @@
---
description: Git branching and PR workflow
alwaysApply: true
---
# Branching Workflow
**Never commit directly to `main`.** Always work on a feature branch.
Create descriptive branch names like `feature/add-discord-notifications`, `fix/audio-sync-issue`, or `refactor/cleanup-api-routes`.
# Creating a Pull Request
Once all changes are complete, tests pass, and commits are pushed, create a PR with:
1. **Summary** - Brief overview of the changes
2. **Changes Made** - List of specific modifications
3. **Testing** - How changes were tested (unit tests, manual testing)
4. **Related Issues** - Reference any GitHub issues (e.g., "Fixes #123")
Only merge after review and CI checks pass.
File renamed without changes.
@@ -1,36 +1,167 @@
name: Backend Tests
name: CI
on:
push:
branches: [ main, replace-modal-with-google-cloud ]
pull_request:
branches: [ main ]
branches: [main]
workflow_dispatch:
permissions:
contents: write
pages: write
id-token: write
concurrency:
group: "ci-${{ github.ref }}"
cancel-in-progress: true
jobs:
unit-tests:
name: Unit Tests
# ============================================
# PACKAGE TESTS (karaoke-gen Python CLI)
# ============================================
package-unit-tests:
name: "Package - Unit Tests (70% coverage)"
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: false
- uses: actions/checkout@v4
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: '3.13'
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Install dependencies
run: poetry install --with dev
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg
- name: Install spaCy language models
run: poetry run python -m spacy download en_core_web_sm
- name: Run unit tests
run: |
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=xml --cov-report=term-missing --cov-fail-under=70
- name: Upload coverage reports
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.xml
flags: unittests
name: codecov-umbrella
package-integration-tests:
name: "Package - Integration Tests"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Install dependencies
run: poetry install --with dev
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg
- name: Install spaCy language models
run: poetry run python -m spacy download en_core_web_sm
- name: Run integration tests
run: poetry run pytest tests/integration/ -v
package-build:
name: "Package - Build & Install"
runs-on: ubuntu-latest
needs: [package-unit-tests, package-integration-tests]
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Build package
run: poetry build
- name: Free disk space
uses: jlumbroso/free-disk-space@main
with:
tool-cache: false
android: true
haskell: true
dotnet: false
large-packages: false
docker-images: false
swap-storage: false
- name: Test package installation
run: |
pip install dist/*.whl
python -m karaoke_gen.utils.gen_cli --help
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: package-dist
path: dist/
retention-days: 1
# ============================================
# BACKEND TESTS (Cloud Run API service)
# ============================================
backend-unit-tests:
name: "Backend - Unit Tests"
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: false
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Install dependencies
run: poetry install
- name: Run unit tests
env:
GOOGLE_CLOUD_PROJECT: test-project
@@ -44,11 +175,10 @@ jobs:
backend/tests/test_file_upload.py \
backend/tests/test_services.py \
-v --tb=short
emulator-tests:
name: Emulator Integration Tests
backend-emulator-tests:
name: "Backend - Emulator Integration Tests"
runs-on: ubuntu-latest
steps:
- name: Free disk space
run: |
@@ -56,143 +186,250 @@ jobs:
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
df -h
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: false
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: '3.13'
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Set up Java 21 (required by Firestore emulator)
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '21'
distribution: "temurin"
java-version: "21"
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: 'latest'
version: "latest"
- name: Install Firestore emulator
run: |
gcloud components install beta cloud-firestore-emulator --quiet
run: gcloud components install beta cloud-firestore-emulator --quiet
- name: Install dependencies
run: poetry install
- name: Start Docker for GCS emulator
run: |
docker pull fsouza/fake-gcs-server:latest
run: docker pull fsouza/fake-gcs-server:latest
- name: Run emulator integration tests
run: |
chmod +x scripts/*.sh
./scripts/run-emulator-tests.sh
- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results
name: backend-test-results
path: backend/htmlcov/
retention-days: 7
lint:
name: Code Quality
backend-lint:
name: "Backend - Code Quality"
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: false
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: '3.13'
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Install dependencies
run: poetry install
- name: Run validation (syntax check only)
run: |
# Skip full validation as it requires GCP credentials
# Just check Python syntax and imports
poetry run python -m py_compile backend/main.py backend/config.py
echo "✅ Basic syntax validation passed"
deploy:
name: Deploy via Cloud Build
# ============================================
# DEPLOYMENTS (main branch only)
# ============================================
package-publish:
name: "Deploy - Publish to PyPI"
runs-on: ubuntu-latest
needs: [unit-tests, emulator-tests, lint]
# Only deploy on push to the branch (not on PRs)
if: github.event_name == 'push' && github.ref == 'refs/heads/replace-modal-with-google-cloud'
needs: [package-build]
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install Poetry
uses: snok/install-poetry@v1
- name: Download build artifacts
uses: actions/download-artifact@v4
with:
name: package-dist
path: dist/
- name: Extract version
id: extract_version
run: |
VERSION=$(poetry version --short)
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "📋 Extracted version: $VERSION"
- name: Check if tag exists
id: check_tag
run: |
VERSION="${{ steps.extract_version.outputs.version }}"
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
echo "tag_exists=true" >> $GITHUB_OUTPUT
echo "🏷️ Tag v$VERSION already exists, skipping release"
else
echo "tag_exists=false" >> $GITHUB_OUTPUT
echo "✨ Tag v$VERSION does not exist, proceeding with release"
fi
- name: Publish to PyPI
if: steps.check_tag.outputs.tag_exists == 'false'
run: |
echo "📦 Publishing the following files to PyPI:"
ls -la dist/
echo "🚀 Publishing to PyPI..."
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
poetry publish
- name: Create Git Tag
if: steps.check_tag.outputs.tag_exists == 'false'
run: |
git config user.name github-actions
git config user.email github-actions@github.com
git tag -a "v${{ steps.extract_version.outputs.version }}" -m "Release v${{ steps.extract_version.outputs.version }}"
git push https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }} "v${{ steps.extract_version.outputs.version }}"
- name: Create GitHub Release
if: steps.check_tag.outputs.tag_exists == 'false'
uses: softprops/action-gh-release@v1
with:
tag_name: "v${{ steps.extract_version.outputs.version }}"
name: "v${{ steps.extract_version.outputs.version }}"
body: |
## Changes in v${{ steps.extract_version.outputs.version }}
This release was automatically created from the main branch.
### Installation
```bash
pip install karaoke-gen==${{ steps.extract_version.outputs.version }}
```
### Assets
- Python package published to PyPI
- Frontend deployed to GitHub Pages at gen.nomadkaraoke.com
For detailed changes, see the commit history.
draft: false
prerelease: false
files: |
dist/*.whl
dist/*.tar.gz
deploy-frontend:
name: "Deploy - Frontend (GitHub Pages)"
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v4
- name: Prepare frontend files
run: |
echo "📁 Preparing frontend files for deployment..."
mkdir -p _site
cp -r frontend/* _site/
echo "✅ Frontend files copied to _site/"
ls -la _site/
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: "_site"
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
deploy-backend:
name: "Deploy - Backend (Cloud Run)"
runs-on: ubuntu-latest
needs: [backend-unit-tests, backend-emulator-tests, backend-lint]
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
permissions:
contents: read
id-token: write # Required for Workload Identity Federation
id-token: write
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: false
- name: Authenticate to Google Cloud (Workload Identity)
id: auth
uses: google-github-actions/auth@v2
with:
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
with:
version: 'latest'
version: "latest"
- name: Check if base image needs rebuild
id: check-base
run: |
echo "🔍 Checking if base image needs rebuild..."
# Calculate current poetry.lock hash
CURRENT_HASH=$(sha256sum poetry.lock | cut -d' ' -f1)
echo "Current poetry.lock hash: $CURRENT_HASH"
# Try to get the hash from the existing base image
# First, authenticate to Artifact Registry
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
# Try to pull the base image and get its label
BASE_IMAGE="us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest"
EXISTING_HASH=""
if docker pull $BASE_IMAGE 2>/dev/null; then
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
echo "Existing base image hash: $EXISTING_HASH"
else
echo "⚠️ No existing base image found"
fi
# Compare hashes
if [ "$CURRENT_HASH" != "$EXISTING_HASH" ]; then
echo "🔄 Base image needs rebuild (poetry.lock changed)"
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
@@ -200,29 +437,27 @@ jobs:
echo "✅ Base image is up to date"
echo "needs_rebuild=false" >> $GITHUB_OUTPUT
fi
- name: Rebuild base image (if needed)
if: steps.check-base.outputs.needs_rebuild == 'true'
run: |
echo "🏗️ Rebuilding base image with updated dependencies..."
# Submit base image build and wait for completion
BASE_BUILD_ID=$(gcloud builds submit \
--config=cloudbuild-base.yaml \
--project=nomadkaraoke \
--format='value(id)' \
--async \
.)
echo "Base Build ID: $BASE_BUILD_ID"
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BASE_BUILD_ID?project=nomadkaraoke"
# Poll for build completion
echo "Waiting for base image build to complete..."
while true; do
BUILD_STATUS=$(gcloud builds describe $BASE_BUILD_ID --project=nomadkaraoke --format='value(status)')
echo "Base build status: $BUILD_STATUS"
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
echo "✅ Base image build completed successfully"
break
@@ -239,32 +474,30 @@ jobs:
gcloud builds log $BASE_BUILD_ID --project=nomadkaraoke 2>&1 || true
exit 1
fi
sleep 30
done
- name: Trigger Cloud Build
run: |
echo "🚀 Triggering Cloud Build for deployment..."
echo "Commit: ${{ github.sha }}"
# Submit build and wait for completion
BUILD_ID=$(gcloud builds submit \
--config=cloudbuild.yaml \
--project=nomadkaraoke \
--format='value(id)' \
--async \
.)
echo "Build ID: $BUILD_ID"
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BUILD_ID?project=nomadkaraoke"
# Poll for build completion
echo "Waiting for build to complete..."
while true; do
BUILD_STATUS=$(gcloud builds describe $BUILD_ID --project=nomadkaraoke --format='value(status)')
echo "Build status: $BUILD_STATUS"
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
echo "✅ Build completed successfully"
break
@@ -272,7 +505,6 @@ jobs:
echo "❌ Build failed with status: $BUILD_STATUS"
echo ""
echo "==================== BUILD LOGS ===================="
# Use Cloud Logging to get build logs (works with roles/logging.viewer)
gcloud logging read "resource.type=build AND resource.labels.build_id=$BUILD_ID" \
--project=nomadkaraoke \
--format="value(textPayload)" \
@@ -280,30 +512,24 @@ jobs:
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
echo "===================================================="
echo ""
# Also try the direct logs command as fallback
gcloud builds log $BUILD_ID --project=nomadkaraoke 2>&1 || true
exit 1
fi
sleep 30
done
- name: Verify deployment
run: |
SERVICE_URL=$(gcloud run services describe karaoke-backend \
--region us-central1 \
--format 'value(status.url)')
echo "Service URL: $SERVICE_URL"
# Wait a few seconds for the service to be ready
sleep 10
# Test health endpoint
curl -f "$SERVICE_URL/api/health" || exit 1
echo "✅ Deployment verified - service is healthy"
- name: Summary
if: success()
run: |
-187
View File
@@ -1,187 +0,0 @@
name: Test and Publish
on: [push, pull_request, workflow_dispatch]
# Add permissions for GitHub Pages deployment and git operations
permissions:
contents: write # Changed from 'read' to 'write' to allow creating git tags
pages: write
id-token: write
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.13']
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
- name: Install dependencies
run: |
poetry install --with dev
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg
- name: Install spaCy language models
run: |
poetry run python -m spacy download en_core_web_sm
- name: Run unit tests
run: |
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=xml --cov-report=term-missing --cov-fail-under=70
- name: Upload coverage reports
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.xml
flags: unittests
name: codecov-umbrella
- name: Run integration tests
run: |
poetry run pytest tests/integration/ -v
- name: Build package
run: |
poetry build
- name: Free Disk Space (GHA Ubuntu Runner)
uses: jlumbroso/free-disk-space@main
with:
# this might remove tools that are actually needed,
# if set to "true" but frees about 6 GB
tool-cache: false
# all of these default to true, but feel free to set to
# "false" if necessary for your workflow
android: true
haskell: true
dotnet: false
large-packages: false
docker-images: false
swap-storage: false
- name: Test package installation
run: |
pip install dist/*.whl
python -m karaoke_gen.utils.gen_cli --help
- if: github.ref == 'refs/heads/main' && github.event_name == 'push'
name: Extract version
id: extract_version
run: |
VERSION=$(poetry version --short)
echo "version=$VERSION" >> $GITHUB_OUTPUT
echo "📋 Extracted version: $VERSION"
- if: github.ref == 'refs/heads/main' && github.event_name == 'push'
name: Check if tag exists
id: check_tag
run: |
VERSION="${{ steps.extract_version.outputs.version }}"
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
echo "tag_exists=true" >> $GITHUB_OUTPUT
echo "🏷️ Tag v$VERSION already exists, skipping release"
else
echo "tag_exists=false" >> $GITHUB_OUTPUT
echo "✨ Tag v$VERSION does not exist, proceeding with release"
fi
- if: github.ref == 'refs/heads/main' && github.event_name == 'push' && steps.check_tag.outputs.tag_exists == 'false'
name: Publish to PyPI
run: |
echo "📦 Publishing the following files to PyPI:"
ls -la dist/
echo "🚀 Publishing to PyPI..."
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
poetry publish
- if: github.ref == 'refs/heads/main' && github.event_name == 'push' && steps.check_tag.outputs.tag_exists == 'false'
name: Create Git Tag
run: |
git config user.name github-actions
git config user.email github-actions@github.com
git tag -a "v${{ steps.extract_version.outputs.version }}" -m "Release v${{ steps.extract_version.outputs.version }}"
git push https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }} "v${{ steps.extract_version.outputs.version }}"
- if: github.ref == 'refs/heads/main' && github.event_name == 'push' && steps.check_tag.outputs.tag_exists == 'false'
name: Create GitHub Release
uses: softprops/action-gh-release@v1
with:
tag_name: "v${{ steps.extract_version.outputs.version }}"
name: "v${{ steps.extract_version.outputs.version }}"
body: |
## Changes in v${{ steps.extract_version.outputs.version }}
This release was automatically created from the main branch.
### Installation
```bash
pip install karaoke-gen==${{ steps.extract_version.outputs.version }}
```
### Assets
- Python package published to PyPI
- Frontend deployed to GitHub Pages at gen.nomadkaraoke.com
For detailed changes, see the commit history.
draft: false
prerelease: false
files: |
dist/*.whl
dist/*.tar.gz
# Deploy frontend to GitHub Pages
deploy-frontend:
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v4
- name: Prepare frontend files
run: |
echo "📁 Preparing frontend files for deployment..."
mkdir -p _site
cp -r frontend/* _site/
echo "✅ Frontend files copied to _site/"
ls -la _site/
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: '_site'
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
+4
View File
@@ -27,6 +27,10 @@ COPY lyrics_transcriber_temp /app/lyrics_transcriber_temp
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir -e .
# Download spaCy language model for lyrics transcription
# This must be done at build time, not runtime (Cloud Run network restrictions)
RUN python -m spacy download en_core_web_sm
# Label with build info
ARG BUILD_DATE
ARG BUILD_ID
+543
View File
@@ -0,0 +1,543 @@
"""
Audio search API routes for artist+title search mode.
This module provides endpoints for:
1. Creating a job with audio search (artist+title without file)
2. Getting search results for a job
3. Selecting an audio source to download
The flow is:
1. POST /api/audio-search/search - Create job and search for audio
2. GET /api/audio-search/{job_id}/results - Get search results
3. POST /api/audio-search/{job_id}/select - Select audio source
"""
import asyncio
import logging
import os
import tempfile
from typing import Optional, List, Dict, Any
from fastapi import APIRouter, HTTPException, BackgroundTasks, Request
from pydantic import BaseModel, Field
from backend.models.job import JobCreate, JobStatus
from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.services.worker_service import get_worker_service
from backend.services.credential_manager import get_credential_manager, CredentialStatus
from backend.services.audio_search_service import (
get_audio_search_service,
AudioSearchResult,
AudioSearchError,
NoResultsError,
DownloadError,
)
from backend.config import get_settings
from backend.version import VERSION
logger = logging.getLogger(__name__)
router = APIRouter(tags=["audio-search"])
# Initialize services
job_manager = JobManager()
storage_service = StorageService()
worker_service = get_worker_service()
# ============================================================================
# Pydantic models
# ============================================================================
class AudioSearchRequest(BaseModel):
"""Request to search for audio by artist and title."""
artist: str = Field(..., description="Artist name to search for")
title: str = Field(..., description="Song title to search for")
# Auto-download mode
auto_download: bool = Field(False, description="Automatically select best result and download")
# Processing options
enable_cdg: bool = Field(True, description="Generate CDG+MP3 package")
enable_txt: bool = Field(True, description="Generate TXT+MP3 package")
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
# Distribution options
dropbox_path: Optional[str] = Field(None, description="Dropbox folder path for organized output")
gdrive_folder_id: Optional[str] = Field(None, description="Google Drive folder ID for public share uploads")
# Lyrics configuration
lyrics_artist: Optional[str] = Field(None, description="Override artist name for lyrics search")
lyrics_title: Optional[str] = Field(None, description="Override title for lyrics search")
subtitle_offset_ms: int = Field(0, description="Subtitle timing offset in milliseconds")
# Audio separation model configuration
clean_instrumental_model: Optional[str] = Field(None, description="Model for clean instrumental separation")
backing_vocals_models: Optional[List[str]] = Field(None, description="Models for backing vocals separation")
other_stems_models: Optional[List[str]] = Field(None, description="Models for other stems")
class AudioSearchResultResponse(BaseModel):
"""A single audio search result."""
index: int
title: str
artist: str
provider: str
url: str
duration: Optional[int] = None
quality: Optional[str] = None
source_id: Optional[str] = None
class AudioSearchResponse(BaseModel):
"""Response from audio search."""
status: str
job_id: str
message: str
results: Optional[List[AudioSearchResultResponse]] = None
results_count: int = 0
auto_download: bool = False
server_version: str
class AudioSelectRequest(BaseModel):
"""Request to select an audio source."""
selection_index: int = Field(..., description="Index of the selected audio source from search results")
class AudioSelectResponse(BaseModel):
"""Response from audio source selection."""
status: str
job_id: str
message: str
selected_index: int
selected_title: str
selected_artist: str
selected_provider: str
def extract_request_metadata(request: Request, created_from: str = "audio_search") -> Dict[str, Any]:
"""Extract metadata from request for job tracking."""
headers = dict(request.headers)
client_ip = headers.get('x-forwarded-for', '').split(',')[0].strip()
if not client_ip and request.client:
client_ip = request.client.host
user_agent = headers.get('user-agent', '')
environment = headers.get('x-environment', '')
client_id = headers.get('x-client-id', '')
custom_headers = {}
for key, value in headers.items():
if key.lower().startswith('x-') and key.lower() not in ('x-forwarded-for', 'x-forwarded-proto', 'x-forwarded-host'):
custom_headers[key] = value
return {
'client_ip': client_ip,
'user_agent': user_agent,
'environment': environment,
'client_id': client_id,
'server_version': VERSION,
'created_from': created_from,
'custom_headers': custom_headers,
}
async def _trigger_workers_parallel(job_id: str) -> None:
"""Trigger both audio and lyrics workers in parallel."""
await asyncio.gather(
worker_service.trigger_audio_worker(job_id),
worker_service.trigger_lyrics_worker(job_id)
)
async def _download_and_start_processing(
job_id: str,
selection_index: int,
audio_search_service,
background_tasks: BackgroundTasks,
) -> Dict[str, Any]:
"""
Download selected audio and start job processing.
This is called either:
- Immediately after search if auto_download=True
- When user calls the select endpoint
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Get search results from state_data
search_results = job.state_data.get('audio_search_results', [])
if not search_results:
raise HTTPException(status_code=400, detail="No search results available")
if selection_index < 0 or selection_index >= len(search_results):
raise HTTPException(
status_code=400,
detail=f"Invalid selection index {selection_index}. Valid range: 0-{len(search_results)-1}"
)
selected = search_results[selection_index]
# Transition to downloading state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.DOWNLOADING_AUDIO,
progress=10,
message=f"Downloading from {selected['provider']}: {selected['artist']} - {selected['title']}",
state_data_updates={
'selected_audio_index': selection_index,
'selected_audio_provider': selected['provider'],
}
)
try:
# Download to temp directory
temp_dir = tempfile.mkdtemp(prefix=f"audio_download_{job_id}_")
result = audio_search_service.download(
result_index=selection_index,
output_dir=temp_dir,
)
# Upload to GCS
filename = os.path.basename(result.filepath)
audio_gcs_path = f"uploads/{job_id}/audio/{filename}"
with open(result.filepath, 'rb') as f:
storage_service.upload_fileobj(
f,
audio_gcs_path,
content_type='audio/flac' # flacfetch typically returns FLAC
)
logger.info(f"Uploaded audio to GCS: {audio_gcs_path}")
# Clean up temp file
try:
os.remove(result.filepath)
os.rmdir(temp_dir)
except Exception as e:
logger.warning(f"Failed to clean up temp files: {e}")
# Update job with GCS path and transition to DOWNLOADING
job_manager.update_job(job_id, {
'input_media_gcs_path': audio_gcs_path,
'filename': filename,
})
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.DOWNLOADING,
progress=15,
message="Audio downloaded, starting processing"
)
# Trigger workers
background_tasks.add_task(_trigger_workers_parallel, job_id)
return {
'selected_index': selection_index,
'selected_title': selected['title'],
'selected_artist': selected['artist'],
'selected_provider': selected['provider'],
}
except DownloadError as e:
job_manager.fail_job(job_id, f"Audio download failed: {e}")
raise HTTPException(status_code=500, detail=f"Download failed: {e}")
except Exception as e:
job_manager.fail_job(job_id, f"Audio download failed: {e}")
raise HTTPException(status_code=500, detail=f"Download failed: {e}")
@router.post("/audio-search/search", response_model=AudioSearchResponse)
async def search_audio(
request: Request,
background_tasks: BackgroundTasks,
body: AudioSearchRequest,
):
"""
Search for audio by artist and title, creating a new job.
This endpoint:
1. Creates a job in PENDING state
2. Searches for audio using flacfetch
3. Either returns search results for user selection, or
4. If auto_download=True, automatically selects best and starts processing
Use cases:
- Interactive mode (default): Returns results, user calls /select endpoint
- Auto mode (auto_download=True): Automatically selects and downloads best
"""
try:
# Apply default distribution settings
settings = get_settings()
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
# Validate credentials if distribution services are requested
invalid_services = []
credential_manager = get_credential_manager()
if body.enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
if effective_dropbox_path:
result = credential_manager.check_dropbox_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"dropbox ({result.message})")
if effective_gdrive_folder_id:
result = credential_manager.check_gdrive_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"gdrive ({result.message})")
if invalid_services:
raise HTTPException(
status_code=400,
detail={
"error": "credentials_invalid",
"message": f"Distribution services need re-authorization: {', '.join(invalid_services)}",
"invalid_services": invalid_services,
}
)
# Extract request metadata
request_metadata = extract_request_metadata(request, created_from="audio_search")
# Create job
job_create = JobCreate(
artist=body.artist,
title=body.title,
enable_cdg=body.enable_cdg,
enable_txt=body.enable_txt,
brand_prefix=body.brand_prefix,
enable_youtube_upload=body.enable_youtube_upload,
youtube_description=body.youtube_description,
discord_webhook_url=effective_discord_webhook_url,
dropbox_path=effective_dropbox_path,
gdrive_folder_id=effective_gdrive_folder_id,
lyrics_artist=body.lyrics_artist,
lyrics_title=body.lyrics_title,
subtitle_offset_ms=body.subtitle_offset_ms,
clean_instrumental_model=body.clean_instrumental_model,
backing_vocals_models=body.backing_vocals_models,
other_stems_models=body.other_stems_models,
audio_search_artist=body.artist,
audio_search_title=body.title,
auto_download=body.auto_download,
request_metadata=request_metadata,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
logger.info(f"Created job {job_id} for audio search: {body.artist} - {body.title}")
# Update job with audio search fields
job_manager.update_job(job_id, {
'audio_search_artist': body.artist,
'audio_search_title': body.title,
'auto_download': body.auto_download,
})
# Transition to searching state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.SEARCHING_AUDIO,
progress=5,
message=f"Searching for: {body.artist} - {body.title}"
)
# Perform search
audio_search_service = get_audio_search_service()
try:
search_results = audio_search_service.search(body.artist, body.title)
except NoResultsError as e:
job_manager.fail_job(job_id, f"No audio sources found for: {body.artist} - {body.title}")
raise HTTPException(
status_code=404,
detail={
"error": "no_results",
"message": str(e),
"job_id": job_id,
}
)
except AudioSearchError as e:
job_manager.fail_job(job_id, f"Audio search failed: {e}")
raise HTTPException(status_code=500, detail=f"Search failed: {e}")
# Store results in job state_data
results_dicts = [r.to_dict() for r in search_results]
job_manager.update_job(job_id, {
'state_data': {
'audio_search_results': results_dicts,
'audio_search_count': len(results_dicts),
}
})
# If auto_download, select best and start processing
if body.auto_download:
best_index = audio_search_service.select_best(search_results)
logger.info(f"Auto-download enabled, selecting result {best_index}")
selection_info = await _download_and_start_processing(
job_id=job_id,
selection_index=best_index,
audio_search_service=audio_search_service,
background_tasks=background_tasks,
)
return AudioSearchResponse(
status="success",
job_id=job_id,
message=f"Audio found and download started: {selection_info['selected_artist']} - {selection_info['selected_title']} ({selection_info['selected_provider']})",
results=None, # Don't return results in auto mode
results_count=len(search_results),
auto_download=True,
server_version=VERSION,
)
# Interactive mode: return results for user selection
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_AUDIO_SELECTION,
progress=10,
message=f"Found {len(search_results)} audio sources. Waiting for selection."
)
# Convert to response format
result_responses = [
AudioSearchResultResponse(
index=r.index,
title=r.title,
artist=r.artist,
provider=r.provider,
url=r.url,
duration=r.duration,
quality=r.quality,
source_id=r.source_id,
)
for r in search_results
]
return AudioSearchResponse(
status="awaiting_selection",
job_id=job_id,
message=f"Found {len(search_results)} audio sources. Call /api/audio-search/{job_id}/select to choose one.",
results=result_responses,
results_count=len(search_results),
auto_download=False,
server_version=VERSION,
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Error in audio search: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e))
@router.get("/audio-search/{job_id}/results")
async def get_audio_search_results(job_id: str):
"""
Get audio search results for a job.
Returns the cached search results so user can select one.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
search_results = job.state_data.get('audio_search_results', [])
if not search_results:
raise HTTPException(
status_code=400,
detail="No search results available for this job"
)
return {
"status": "success",
"job_id": job_id,
"job_status": job.status,
"results": search_results,
"results_count": len(search_results),
}
@router.post("/audio-search/{job_id}/select", response_model=AudioSelectResponse)
async def select_audio_source(
job_id: str,
background_tasks: BackgroundTasks,
body: AudioSelectRequest,
):
"""
Select an audio source and start job processing.
This endpoint:
1. Validates the job is awaiting selection
2. Downloads the selected audio
3. Uploads to GCS
4. Triggers audio and lyrics workers
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Verify job is awaiting selection
if job.status != JobStatus.AWAITING_AUDIO_SELECTION:
raise HTTPException(
status_code=400,
detail=f"Job is not awaiting audio selection (status: {job.status})"
)
# Re-instantiate the audio search service (it caches raw results from search)
# For production, we'd need to re-search or store raw results differently
audio_search_service = get_audio_search_service()
# Re-run search to populate the cache
# This is necessary because the service caches raw results in memory
search_results = job.state_data.get('audio_search_results', [])
if not search_results:
raise HTTPException(status_code=400, detail="No search results cached for this job")
artist = job.audio_search_artist or job.artist
title = job.audio_search_title or job.title
try:
# Re-search to populate cache
audio_search_service.search(artist, title)
except Exception as e:
logger.warning(f"Re-search failed, trying direct download: {e}")
selection_info = await _download_and_start_processing(
job_id=job_id,
selection_index=body.selection_index,
audio_search_service=audio_search_service,
background_tasks=background_tasks,
)
return AudioSelectResponse(
status="success",
job_id=job_id,
message="Audio selected and download started",
selected_index=selection_info['selected_index'],
selected_title=selection_info['selected_title'],
selected_artist=selection_info['selected_artist'],
selected_provider=selection_info['selected_provider'],
)
File diff suppressed because it is too large. Load diff
+91 -1
View File
@@ -3,10 +3,13 @@ Internal API routes for worker coordination.
These endpoints are for internal use only (backend → workers).
They are protected by admin authentication.
With Cloud Tasks integration, these endpoints may be called multiple times
(retry on failure). Idempotency checks prevent duplicate processing.
"""
import logging
import asyncio
from typing import Tuple
from typing import Tuple, Optional
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends
from pydantic import BaseModel
@@ -17,6 +20,7 @@ from backend.workers.video_worker import generate_video
from backend.workers.render_video_worker import process_render_video
from backend.api.dependencies import require_admin
from backend.services.auth_service import UserType
from backend.services.job_manager import JobManager
logger = logging.getLogger(__name__)
@@ -35,6 +39,57 @@ class WorkerResponse(BaseModel):
message: str
def _check_worker_idempotency(job_id: str, worker_name: str) -> Optional[WorkerResponse]:
"""
Check if a worker is already running or completed for this job.
This provides idempotency for Cloud Tasks retries - if a task is retried
but the worker is already running or has completed, we skip processing.
Args:
job_id: Job ID to check
worker_name: Worker name (audio, lyrics, screens, render, video)
Returns:
WorkerResponse if should skip (already running/complete), None to proceed
"""
job_manager = JobManager()
job = job_manager.get_job(job_id)
if not job:
logger.warning(f"Job {job_id} not found for {worker_name} worker")
return WorkerResponse(
status="not_found",
job_id=job_id,
message=f"Job {job_id} not found"
)
# Check worker-specific progress in state_data
progress_key = f"{worker_name}_progress"
worker_progress = job.state_data.get(progress_key, {})
stage = worker_progress.get('stage')
if stage == 'running':
logger.info(f"{worker_name.capitalize()} worker already running for job {job_id}, skipping")
return WorkerResponse(
status="already_running",
job_id=job_id,
message=f"{worker_name.capitalize()} worker already in progress"
)
if stage == 'complete':
logger.info(f"{worker_name.capitalize()} worker already complete for job {job_id}, skipping")
return WorkerResponse(
status="already_complete",
job_id=job_id,
message=f"{worker_name.capitalize()} worker already completed"
)
# Mark as running before starting (for idempotency on next retry)
job_manager.update_state_data(job_id, progress_key, {'stage': 'running'})
return None
@router.post("/workers/audio", response_model=WorkerResponse)
async def trigger_audio_worker(
request: WorkerRequest,
@@ -47,11 +102,18 @@ async def trigger_audio_worker(
This endpoint is called internally after job creation to start
the audio processing track (parallel with lyrics processing).
Idempotency: If worker is already running or complete, returns early.
The worker runs in the background and updates job state as it progresses.
"""
job_id = request.job_id
logger.info(f"Triggering audio worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "audio")
if skip_response:
return skip_response
# Add task to background tasks
# This allows the HTTP response to return immediately
# while the worker continues processing
@@ -76,11 +138,18 @@ async def trigger_lyrics_worker(
This endpoint is called internally after job creation to start
the lyrics processing track (parallel with audio processing).
Idempotency: If worker is already running or complete, returns early.
The worker runs in the background and updates job state as it progresses.
"""
job_id = request.job_id
logger.info(f"Triggering lyrics worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "lyrics")
if skip_response:
return skip_response
# Add task to background tasks
background_tasks.add_task(process_lyrics_transcription, job_id)
@@ -101,10 +170,17 @@ async def trigger_screens_worker(
Trigger title/end screen generation worker.
This is called automatically when both audio and lyrics are complete.
Idempotency: If worker is already running or complete, returns early.
"""
job_id = request.job_id
logger.info(f"Triggering screens worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "screens")
if skip_response:
return skip_response
# Add task to background tasks
background_tasks.add_task(generate_screens, job_id)
@@ -126,10 +202,17 @@ async def trigger_video_worker(
This is called after user selects their preferred instrumental.
This is the longest-running stage (15-20 minutes).
Idempotency: If worker is already running or complete, returns early.
"""
job_id = request.job_id
logger.info(f"Triggering video worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "video")
if skip_response:
return skip_response
# Add task to background tasks
background_tasks.add_task(generate_video, job_id)
@@ -153,12 +236,19 @@ async def trigger_render_video_worker(
Uses OutputGenerator from LyricsTranscriber to generate the karaoke video
with the corrected lyrics.
Idempotency: If worker is already running or complete, returns early.
Output: with_vocals.mkv in GCS
Next state: AWAITING_INSTRUMENTAL_SELECTION
"""
job_id = request.job_id
logger.info(f"Triggering render-video worker for job {job_id}")
# Idempotency check
skip_response = _check_worker_idempotency(job_id, "render")
if skip_response:
return skip_response
# Add task to background tasks
background_tasks.add_task(process_render_video, job_id)
+460 -1
View File
@@ -19,7 +19,8 @@ from backend.models.requests import (
CorrectionsSubmission,
InstrumentalSelection,
StartReviewRequest,
CancelJobRequest
CancelJobRequest,
CreateCustomInstrumentalRequest,
)
from backend.services.job_manager import JobManager
from backend.services.worker_service import get_worker_service
@@ -523,6 +524,311 @@ async def get_instrumental_options(job_id: str) -> Dict[str, Any]:
}
@router.get("/{job_id}/instrumental-analysis")
async def get_instrumental_analysis(job_id: str) -> Dict[str, Any]:
"""
Get audio analysis data for instrumental selection.
Returns:
- Analysis of backing vocals (audible segments, recommendation)
- Waveform image URL
- Audio stream URLs for playback
This endpoint enables intelligent instrumental selection by providing
detailed analysis of the backing vocals track.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
if job.status != JobStatus.AWAITING_INSTRUMENTAL_SELECTION:
raise HTTPException(
status_code=400,
detail=f"Job not ready for instrumental analysis (current status: {job.status})"
)
storage = StorageService()
# Get analysis from state_data (populated by render_video_worker)
analysis_data = job.state_data.get('backing_vocals_analysis', {})
# Get URLs
stems = job.file_urls.get('stems', {})
analysis_files = job.file_urls.get('analysis', {})
clean_url = stems.get('instrumental_clean')
backing_vocals_url = stems.get('backing_vocals')
with_backing_url = stems.get('instrumental_with_backing')
waveform_url = analysis_files.get('backing_vocals_waveform')
custom_instrumental_url = stems.get('instrumental_custom')
# Build response
response = {
"job_id": job_id,
"artist": job.artist,
"title": job.title,
"status": job.status,
# Analysis results
"analysis": {
"has_audible_content": analysis_data.get('has_audible_content', False),
"total_duration_seconds": analysis_data.get('total_duration_seconds', 0),
"audible_segments": analysis_data.get('audible_segments', []),
"recommended_selection": analysis_data.get('recommended_selection', 'review_needed'),
"total_audible_duration_seconds": analysis_data.get('total_audible_duration_seconds', 0),
"audible_percentage": analysis_data.get('audible_percentage', 0),
"silence_threshold_db": analysis_data.get('silence_threshold_db', -40.0),
},
# Audio URLs for playback
"audio_urls": {
"clean_instrumental": storage.generate_signed_url(clean_url, expiration_minutes=120) if clean_url else None,
"backing_vocals": storage.generate_signed_url(backing_vocals_url, expiration_minutes=120) if backing_vocals_url else None,
"with_backing": storage.generate_signed_url(with_backing_url, expiration_minutes=120) if with_backing_url else None,
"custom_instrumental": storage.generate_signed_url(custom_instrumental_url, expiration_minutes=120) if custom_instrumental_url else None,
},
# Waveform image URL
"waveform_url": storage.generate_signed_url(waveform_url, expiration_minutes=120) if waveform_url else None,
# Whether a custom instrumental has been created
"has_custom_instrumental": custom_instrumental_url is not None,
}
return response
@router.get("/{job_id}/audio-stream/{stem_type}")
async def stream_audio(job_id: str, stem_type: str):
"""
Stream an audio file for playback in the browser.
Supported stem_type values:
- clean_instrumental: Clean instrumental (no backing vocals)
- backing_vocals: Backing vocals only
- with_backing: Instrumental with backing vocals
- custom_instrumental: Custom instrumental (if created)
Returns audio as a streaming response with proper headers for
browser audio playback.
"""
from fastapi.responses import StreamingResponse
import tempfile
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
# Map stem_type to file_urls key
stem_map = {
'clean_instrumental': ('stems', 'instrumental_clean'),
'backing_vocals': ('stems', 'backing_vocals'),
'with_backing': ('stems', 'instrumental_with_backing'),
'custom_instrumental': ('stems', 'instrumental_custom'),
}
if stem_type not in stem_map:
raise HTTPException(
status_code=400,
detail=f"Invalid stem_type. Must be one of: {list(stem_map.keys())}"
)
category, key = stem_map[stem_type]
gcs_path = job.file_urls.get(category, {}).get(key)
if not gcs_path:
raise HTTPException(
status_code=404,
detail=f"Audio file not available: {stem_type}"
)
# Determine content type
ext = gcs_path.split('.')[-1].lower()
content_types = {
'flac': 'audio/flac',
'mp3': 'audio/mpeg',
'wav': 'audio/wav',
'm4a': 'audio/mp4',
}
content_type = content_types.get(ext, 'audio/flac')
try:
storage = StorageService()
with tempfile.NamedTemporaryFile(delete=False, suffix=f'.{ext}') as tmp:
tmp_path = tmp.name
storage.download_file(gcs_path, tmp_path)
def file_iterator():
try:
with open(tmp_path, 'rb') as f:
while chunk := f.read(8192):
yield chunk
finally:
import os
os.unlink(tmp_path)
filename = gcs_path.split('/')[-1]
return StreamingResponse(
file_iterator(),
media_type=content_type,
headers={
'Content-Disposition': f'inline; filename="{filename}"',
'Accept-Ranges': 'bytes',
}
)
except Exception as e:
logger.exception(f"Error streaming audio {gcs_path}: {e}")
raise HTTPException(status_code=500, detail=f"Error streaming audio: {e}") from e
@router.post("/{job_id}/create-custom-instrumental")
async def create_custom_instrumental(
job_id: str,
request: CreateCustomInstrumentalRequest,
) -> Dict[str, Any]:
"""
Create a custom instrumental by muting regions of backing vocals.
This endpoint:
1. Takes a list of time ranges to mute in the backing vocals
2. Creates a custom instrumental (clean + muted backing vocals)
3. Stores the result and makes it available for selection
After calling this endpoint, the user can select "custom" in the
select-instrumental endpoint.
"""
from backend.services.audio_editing_service import AudioEditingService
from karaoke_gen.instrumental_review import MuteRegion
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
if job.status != JobStatus.AWAITING_INSTRUMENTAL_SELECTION:
raise HTTPException(
status_code=400,
detail=f"Job not ready for custom instrumental creation (current status: {job.status})"
)
# Get stem paths
stems = job.file_urls.get('stems', {})
clean_path = stems.get('instrumental_clean')
backing_path = stems.get('backing_vocals')
if not clean_path or not backing_path:
raise HTTPException(
status_code=500,
detail="Required audio files not available"
)
try:
# Convert request mute regions to model objects
mute_regions = [
MuteRegion(
start_seconds=r.start_seconds,
end_seconds=r.end_seconds
)
for r in request.mute_regions
]
# Create custom instrumental
editing_service = AudioEditingService()
# Determine output path
output_path = f"jobs/{job_id}/stems/instrumental_custom.flac"
result = editing_service.create_custom_instrumental(
gcs_clean_instrumental_path=clean_path,
gcs_backing_vocals_path=backing_path,
mute_regions=mute_regions,
gcs_output_path=output_path,
job_id=job_id,
)
# Update job file_urls with custom instrumental
job_manager.update_file_url(job_id, 'stems', 'instrumental_custom', output_path)
# Store mute regions in state_data for reference
job_manager.update_state_data(job_id, 'custom_instrumental_mute_regions', [
{"start_seconds": r.start_seconds, "end_seconds": r.end_seconds}
for r in mute_regions
])
logger.info(f"Job {job_id}: Custom instrumental created with {len(mute_regions)} mute regions")
# Generate signed URL for the new file
storage = StorageService()
return {
"status": "success",
"message": "Custom instrumental created successfully",
"custom_instrumental_url": storage.generate_signed_url(output_path, expiration_minutes=120),
"mute_regions_applied": len(result.mute_regions_applied),
"total_muted_duration_seconds": result.total_muted_duration_seconds,
"output_duration_seconds": result.output_duration_seconds,
}
except Exception as e:
logger.exception(f"Error creating custom instrumental for job {job_id}: {e}")
raise HTTPException(status_code=500, detail=str(e)) from e
@router.get("/{job_id}/waveform-data")
async def get_waveform_data(job_id: str, num_points: int = 500) -> Dict[str, Any]:
"""
Get waveform amplitude data for client-side rendering.
This endpoint returns raw amplitude data that the frontend can use
to render a waveform using Canvas or SVG, enabling interactive
features like click-to-seek.
Args:
num_points: Number of data points to return (default 500)
Returns:
{
"amplitudes": [0.1, 0.2, ...], # Normalized 0-1 values
"duration_seconds": 180.5,
"num_points": 500
}
"""
from backend.services.audio_analysis_service import AudioAnalysisService
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
if job.status != JobStatus.AWAITING_INSTRUMENTAL_SELECTION:
raise HTTPException(
status_code=400,
detail=f"Job not ready for waveform data (current status: {job.status})"
)
backing_vocals_path = job.file_urls.get('stems', {}).get('backing_vocals')
if not backing_vocals_path:
raise HTTPException(status_code=404, detail="Backing vocals file not found")
try:
analysis_service = AudioAnalysisService()
amplitudes, duration = analysis_service.get_waveform_data(
gcs_audio_path=backing_vocals_path,
job_id=job_id,
num_points=num_points,
)
return {
"amplitudes": amplitudes,
"duration_seconds": duration,
"num_points": len(amplitudes),
}
except Exception as e:
logger.exception(f"Error getting waveform data for job {job_id}: {e}")
raise HTTPException(status_code=500, detail=str(e)) from e
@router.post("/{job_id}/select-instrumental")
async def select_instrumental(
job_id: str,
@@ -722,6 +1028,159 @@ async def cancel_job(job_id: str, request: CancelJobRequest) -> dict:
}
@router.post("/{job_id}/retry")
async def retry_job(
job_id: str,
background_tasks: BackgroundTasks
) -> dict:
"""
Retry a failed job from the last successful checkpoint.
This endpoint allows resuming jobs that failed during:
- Video generation (re-runs video worker)
- Encoding (re-runs video worker)
- Packaging (re-runs video worker)
The retry logic determines the appropriate stage to resume from
based on what files/state already exist.
"""
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
if job.status != JobStatus.FAILED:
raise HTTPException(
status_code=400,
detail=f"Only failed jobs can be retried (current status: {job.status})"
)
try:
# Determine retry point based on what's already complete
error_details = job.error_details or {}
error_stage = error_details.get('stage', 'unknown')
logger.info(f"Job {job_id}: Retrying from failed stage '{error_stage}'")
# Check what state exists to determine retry point
file_urls = job.file_urls or {}
state_data = job.state_data or {}
# If we have a video with vocals and instrumental selection, retry video generation
if (file_urls.get('videos', {}).get('with_vocals') and
state_data.get('instrumental_selection')):
logger.info(f"Job {job_id}: Has rendered video and instrumental selection, retrying video generation")
# Clear error state
job_manager.update_job(job_id, {
'error_message': None,
'error_details': None,
})
# Reset to INSTRUMENTAL_SELECTED and trigger video worker
if not job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.INSTRUMENTAL_SELECTED,
progress=65,
message=f"Retrying video generation from failed state"
):
raise HTTPException(
status_code=500,
detail="Failed to transition job status for retry"
)
# Trigger video generation worker
background_tasks.add_task(worker_service.trigger_video_worker, job_id)
return {
"status": "success",
"job_status": "instrumental_selected",
"message": "Job retry started from video generation stage",
"retry_stage": "video_generation"
}
# If we have corrections and screens but no video, retry render
elif (file_urls.get('lyrics', {}).get('corrections') and
file_urls.get('screens', {}).get('title')):
logger.info(f"Job {job_id}: Has corrections and screens, retrying from render stage")
# Clear error state
job_manager.update_job(job_id, {
'error_message': None,
'error_details': None,
})
# Reset to REVIEW_COMPLETE and trigger render worker
if not job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.REVIEW_COMPLETE,
progress=70,
message=f"Retrying video render from failed state"
):
raise HTTPException(
status_code=500,
detail="Failed to transition job status for retry"
)
# Trigger render video worker
background_tasks.add_task(worker_service.trigger_render_video_worker, job_id)
return {
"status": "success",
"job_status": "review_complete",
"message": "Job retry started from render stage",
"retry_stage": "render_video"
}
# If we have stems and corrections, retry from screens generation
elif (file_urls.get('stems', {}).get('instrumental_clean') and
file_urls.get('lyrics', {}).get('corrections')):
logger.info(f"Job {job_id}: Has stems and corrections, retrying from screens stage")
# Clear error state
job_manager.update_job(job_id, {
'error_message': None,
'error_details': None,
})
# Reset to a state before screens and trigger screens worker
if not job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.LYRICS_COMPLETE,
progress=45,
message=f"Retrying from screens generation"
):
raise HTTPException(
status_code=500,
detail="Failed to transition job status for retry"
)
# Trigger screens worker
background_tasks.add_task(worker_service.trigger_screens_worker, job_id)
return {
"status": "success",
"job_status": "lyrics_complete",
"message": "Job retry started from screens generation",
"retry_stage": "screens_generation"
}
else:
# Can't determine a safe retry point, need to restart from beginning
raise HTTPException(
status_code=400,
detail="Cannot determine safe retry point. Job may need to be resubmitted."
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Error retrying job {job_id}: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e))
@router.get("/{job_id}/logs")
async def get_worker_logs(
job_id: str,
+18 -2
View File
@@ -41,13 +41,28 @@ class Settings(BaseSettings):
max_concurrent_jobs: int = int(os.getenv("MAX_CONCURRENT_JOBS", "5"))
job_timeout_seconds: int = int(os.getenv("JOB_TIMEOUT_SECONDS", "3600"))
# Cloud Tasks (for scalable worker coordination)
# When enabled, workers are triggered via Cloud Tasks for guaranteed delivery
# When disabled (default), workers are triggered via direct HTTP (for development)
enable_cloud_tasks: bool = os.getenv("ENABLE_CLOUD_TASKS", "false").lower() in ("true", "1", "yes")
gcp_region: str = os.getenv("GCP_REGION", "us-central1")
# Cloud Run Jobs (for long-running video encoding)
# When enabled AND enable_cloud_tasks is true, video worker uses Cloud Run Jobs
# instead of Cloud Tasks. This supports encoding times >30 minutes (up to 24 hours).
# Default is false - Cloud Tasks is sufficient for most videos (15-20 min).
use_cloud_run_jobs_for_video: bool = os.getenv("USE_CLOUD_RUN_JOBS_FOR_VIDEO", "false").lower() in ("true", "1", "yes")
# Storage paths
temp_dir: str = os.getenv("TEMP_DIR", "/tmp/karaoke-gen")
# Default distribution settings (can be overridden per-request)
default_dropbox_path: Optional[str] = os.getenv("DEFAULT_DROPBOX_PATH")
default_gdrive_folder_id: Optional[str] = os.getenv("DEFAULT_GDRIVE_FOLDER_ID")
default_discord_webhook_url: Optional[str] = os.getenv("DEFAULT_DISCORD_WEBHOOK_URL")
# Strip whitespace/newlines from webhook URL - common issue when env vars are set with trailing newlines
default_discord_webhook_url: Optional[str] = (
os.getenv("DEFAULT_DISCORD_WEBHOOK_URL", "").strip() or None
)
# Secret Manager cache
_secret_cache: Dict[str, str] = {}
@@ -90,7 +105,8 @@ class Settings(BaseSettings):
client = secretmanager.SecretManagerServiceClient()
name = f"projects/{self.google_cloud_project}/secrets/{secret_id}/versions/latest"
response = client.access_secret_version(request={"name": name})
secret_value = response.payload.data.decode('UTF-8')
# Strip whitespace/newlines - common issue when secrets are created with trailing newlines
secret_value = response.payload.data.decode('UTF-8').strip()
# Cache it
self._secret_cache[secret_id] = secret_value
+2 -1
View File
@@ -7,7 +7,7 @@ from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from backend.config import settings
from backend.api.routes import health, jobs, internal, file_upload, review, auth
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
@@ -108,6 +108,7 @@ app.include_router(file_upload.router, prefix="/api") # File upload endpoint
app.include_router(internal.router, prefix="/api") # Internal worker endpoints
app.include_router(review.router, prefix="/api") # Review UI compatibility endpoints
app.include_router(auth.router, prefix="/api") # OAuth credential management
app.include_router(audio_search.router, prefix="/api") # Audio search (artist+title mode)
@app.get("/")
+76 -7
View File
@@ -14,8 +14,8 @@ class JobStatus(str, Enum):
"""
Job status enumeration - Complete state machine.
The workflow has 8 main stages with 2 human interaction points:
1. Input & Setup
The workflow has 8 main stages with 3 human interaction points:
1. Input & Setup (may include audio source selection)
2. Parallel Processing (audio + lyrics)
3. Title/End Screen Generation
4. Countdown Padding Synchronization
@@ -26,6 +26,12 @@ class JobStatus(str, Enum):
"""
# Initial states
PENDING = "pending" # Job created, queued for processing
# Audio search states (for artist+title search mode)
SEARCHING_AUDIO = "searching_audio" # Searching for audio sources via flacfetch
AWAITING_AUDIO_SELECTION = "awaiting_audio_selection" # ⚠️ WAITING FOR USER - select audio source
DOWNLOADING_AUDIO = "downloading_audio" # Downloading selected audio from source
DOWNLOADING = "downloading" # Downloading from URL or processing upload
# Stage 2a: Audio separation (parallel track 1)
@@ -67,6 +73,7 @@ class JobStatus(str, Enum):
# Terminal states
COMPLETE = "complete" # All processing finished successfully
PREP_COMPLETE = "prep_complete" # Prep-only job completed (stops after review)
FAILED = "failed" # Unrecoverable error occurred
CANCELLED = "cancelled" # User cancelled the job
@@ -80,7 +87,14 @@ class JobStatus(str, Enum):
# Valid state transitions
STATE_TRANSITIONS = {
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.FAILED, JobStatus.CANCELLED],
# PENDING can go to DOWNLOADING (file upload), SEARCHING_AUDIO (artist+title search), or AWAITING_INSTRUMENTAL_SELECTION (finalise-only)
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED, JobStatus.CANCELLED],
# Audio search flow (for artist+title search mode)
JobStatus.SEARCHING_AUDIO: [JobStatus.AWAITING_AUDIO_SELECTION, JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED],
JobStatus.AWAITING_AUDIO_SELECTION: [JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
JobStatus.DOWNLOADING_AUDIO: [JobStatus.DOWNLOADING, JobStatus.FAILED],
# DOWNLOADING allows parallel processing (audio + lyrics) and then screens when both complete
JobStatus.DOWNLOADING: [JobStatus.SEPARATING_STAGE1, JobStatus.TRANSCRIBING, JobStatus.GENERATING_SCREENS, JobStatus.FAILED],
@@ -102,10 +116,10 @@ STATE_TRANSITIONS = {
# AWAITING_REVIEW can go directly to REVIEW_COMPLETE (quick review) or to IN_REVIEW (editing)
JobStatus.AWAITING_REVIEW: [JobStatus.IN_REVIEW, JobStatus.REVIEW_COMPLETE, JobStatus.FAILED, JobStatus.CANCELLED],
JobStatus.IN_REVIEW: [JobStatus.REVIEW_COMPLETE, JobStatus.AWAITING_REVIEW, JobStatus.FAILED],
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.FAILED],
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.PREP_COMPLETE, JobStatus.FAILED], # PREP_COMPLETE for prep-only jobs
# Video rendering (post-review)
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED],
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.PREP_COMPLETE, JobStatus.FAILED],
# Instrumental selection flow
JobStatus.AWAITING_INSTRUMENTAL_SELECTION: [JobStatus.INSTRUMENTAL_SELECTED, JobStatus.FAILED, JobStatus.CANCELLED],
@@ -120,9 +134,16 @@ STATE_TRANSITIONS = {
JobStatus.UPLOADING: [JobStatus.NOTIFYING, JobStatus.COMPLETE, JobStatus.FAILED],
JobStatus.NOTIFYING: [JobStatus.COMPLETE, JobStatus.FAILED],
# Terminal states have no transitions
# Terminal states - COMPLETE, PREP_COMPLETE, and CANCELLED have no transitions
# FAILED allows retry transitions to resume from checkpoints
# PREP_COMPLETE allows finalise-only continuation
JobStatus.COMPLETE: [],
JobStatus.FAILED: [],
JobStatus.PREP_COMPLETE: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED], # Finalise-only continues from here
JobStatus.FAILED: [
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
JobStatus.REVIEW_COMPLETE, # Retry from render stage
JobStatus.LYRICS_COMPLETE, # Retry from screens generation
],
JobStatus.CANCELLED: [],
# Legacy states (for backward compatibility)
@@ -206,6 +227,30 @@ class Job(BaseModel):
# Legacy distribution configuration (rclone - for local CLI backward compat)
organised_dir_rclone_root: Optional[str] = None # Deprecated: use dropbox_path instead
# Lyrics configuration (overrides for search/transcription)
lyrics_artist: Optional[str] = None # Override artist name for lyrics search
lyrics_title: Optional[str] = None # Override title for lyrics search
lyrics_file_gcs_path: Optional[str] = None # GCS path to user-provided lyrics file
subtitle_offset_ms: int = 0 # Offset for subtitle timing (positive = delay)
# Audio separation model configuration
clean_instrumental_model: Optional[str] = None # Model for clean instrumental separation
backing_vocals_models: Optional[List[str]] = None # Models for backing vocals separation
other_stems_models: Optional[List[str]] = None # Models for other stems (bass, drums, etc.)
# Existing instrumental configuration (Batch 3)
existing_instrumental_gcs_path: Optional[str] = None # GCS path to user-provided instrumental file
# Audio search configuration (Batch 5 - artist+title search mode)
audio_search_artist: Optional[str] = None # Artist name used for audio search
audio_search_title: Optional[str] = None # Title used for audio search
auto_download: bool = False # Auto-select best audio source (skip selection)
# Two-phase workflow configuration (Batch 6)
prep_only: bool = False # Stop after review, don't run finalisation
finalise_only: bool = False # Skip prep, run only finalisation (requires uploaded prep outputs)
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Processing state
track_output_dir: Optional[str] = None # Local output directory (temp)
audio_hash: Optional[str] = None # Hash for deduplication
@@ -368,6 +413,30 @@ class JobCreate(BaseModel):
# Legacy (rclone - deprecated, use dropbox_path instead)
organised_dir_rclone_root: Optional[str] = None
# Lyrics configuration (overrides for search/transcription)
lyrics_artist: Optional[str] = None # Override artist name for lyrics search
lyrics_title: Optional[str] = None # Override title for lyrics search
lyrics_file_gcs_path: Optional[str] = None # GCS path to user-provided lyrics file
subtitle_offset_ms: int = 0 # Offset for subtitle timing (positive = delay)
# Audio separation model configuration
clean_instrumental_model: Optional[str] = None # Model for clean instrumental separation
backing_vocals_models: Optional[List[str]] = None # Models for backing vocals separation
other_stems_models: Optional[List[str]] = None # Models for other stems (bass, drums, etc.)
# Existing instrumental configuration (Batch 3)
existing_instrumental_gcs_path: Optional[str] = None # GCS path to user-provided instrumental file
# Audio search configuration (Batch 5 - artist+title search mode)
audio_search_artist: Optional[str] = None # Artist name used for audio search
audio_search_title: Optional[str] = None # Title used for audio search
auto_download: bool = False # Auto-select best audio source (skip selection)
# Two-phase workflow configuration (Batch 6)
prep_only: bool = False # Stop after review, don't run finalisation
finalise_only: bool = False # Skip prep, run only finalisation
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Request metadata (set by API endpoint from request headers)
request_metadata: Dict[str, Any] = Field(default_factory=dict)
"""
+38 -3
View File
@@ -59,19 +59,54 @@ class InstrumentalSelection(BaseModel):
Request to select instrumental audio option.
This is the second critical human-in-the-loop interaction point.
User chooses between clean instrumental or instrumental with backing vocals.
User chooses between clean instrumental, instrumental with backing vocals,
or a custom instrumental (created via create-custom-instrumental endpoint).
"""
selection: str # "clean" or "with_backing"
selection: str # "clean", "with_backing", or "custom"
@validator('selection')
def validate_selection(cls, v):
"""Validate selection is a valid option."""
valid_options = ['clean', 'with_backing']
valid_options = ['clean', 'with_backing', 'custom']
if v not in valid_options:
raise ValueError(f"Selection must be one of: {valid_options}")
return v
class MuteRegionRequest(BaseModel):
"""A region to mute in the backing vocals."""
start_seconds: float
end_seconds: float
@validator('start_seconds')
def validate_start(cls, v):
if v < 0:
raise ValueError("start_seconds must be non-negative")
return v
@validator('end_seconds')
def validate_end(cls, v, values):
if 'start_seconds' in values and v <= values['start_seconds']:
raise ValueError("end_seconds must be greater than start_seconds")
return v
class CreateCustomInstrumentalRequest(BaseModel):
"""
Request to create a custom instrumental with muted backing vocal regions.
The mute_regions specify time ranges in the backing vocals track that
should be silenced before mixing with the clean instrumental.
"""
mute_regions: List[MuteRegionRequest]
@validator('mute_regions')
def validate_regions(cls, v):
if not v:
raise ValueError("At least one mute region is required")
return v
class StartReviewRequest(BaseModel):
"""Request to mark job as in-review (user opened interface)."""
pass # No body needed, just triggers state transition
+2
View File
@@ -11,6 +11,8 @@ httpx>=0.25.0,<1.0.0
google-cloud-firestore>=2.14.0,<3.0.0
google-cloud-storage>=2.14.0,<3.0.0
google-cloud-secret-manager>=2.18.0,<3.0.0
google-cloud-tasks>=2.14.0,<3.0.0
google-cloud-run>=0.10.0,<1.0.0
python-dotenv>=1.0.0,<2.0.0
# Audio processing (reuses karaoke_gen library)
+243
View File
@@ -0,0 +1,243 @@
"""
Audio analysis service for analyzing backing vocals.
This service wraps the shared karaoke_gen.instrumental_review module
to provide GCS-integrated audio analysis capabilities.
"""
import logging
import os
import tempfile
from typing import Optional
from backend.services.storage_service import StorageService
from karaoke_gen.instrumental_review import (
AudioAnalyzer,
AnalysisResult,
WaveformGenerator,
)
logger = logging.getLogger(__name__)
class AudioAnalysisService:
"""
Service for analyzing backing vocals audio files stored in GCS.
This service acts as a thin wrapper around the shared AudioAnalyzer
and WaveformGenerator classes, handling GCS download/upload operations.
The actual analysis logic is in the shared karaoke_gen.instrumental_review
module, ensuring feature parity between local and remote workflows.
"""
def __init__(
self,
storage_service: Optional[StorageService] = None,
silence_threshold_db: float = -40.0,
min_segment_duration_ms: int = 100,
):
"""
Initialize the audio analysis service.
Args:
storage_service: GCS storage service. If not provided, a new
instance will be created.
silence_threshold_db: Threshold for considering audio as silent.
Default is -40.0 dB.
min_segment_duration_ms: Minimum duration for audible segments.
Default is 100ms.
"""
self.storage_service = storage_service or StorageService()
self.analyzer = AudioAnalyzer(
silence_threshold_db=silence_threshold_db,
min_segment_duration_ms=min_segment_duration_ms,
)
self.waveform_generator = WaveformGenerator()
def analyze_backing_vocals(
self,
gcs_audio_path: str,
job_id: str,
) -> AnalysisResult:
"""
Analyze a backing vocals audio file from GCS.
This method:
1. Downloads the audio file from GCS to a temp file
2. Runs the analysis using the shared AudioAnalyzer
3. Returns the analysis result
Args:
gcs_audio_path: Path to the audio file in GCS
job_id: Job ID for logging
Returns:
AnalysisResult containing analysis data
"""
logger.info(f"[{job_id}] Analyzing backing vocals: {gcs_audio_path}")
with tempfile.TemporaryDirectory() as temp_dir:
# Download audio file
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
self.storage_service.download_file(gcs_audio_path, local_audio_path)
# Run analysis
result = self.analyzer.analyze(local_audio_path)
logger.info(
f"[{job_id}] Analysis complete: "
f"has_audible={result.has_audible_content}, "
f"segments={result.segment_count}, "
f"recommendation={result.recommended_selection.value}"
)
return result
def analyze_and_generate_waveform(
self,
gcs_audio_path: str,
job_id: str,
gcs_waveform_destination: str,
) -> tuple[AnalysisResult, str]:
"""
Analyze backing vocals and generate a waveform image.
This method:
1. Downloads the audio file from GCS
2. Runs analysis using AudioAnalyzer
3. Generates waveform image using WaveformGenerator
4. Uploads the waveform image to GCS
5. Returns analysis result and waveform GCS path
Args:
gcs_audio_path: Path to the audio file in GCS
job_id: Job ID for logging
gcs_waveform_destination: Where to upload the waveform image in GCS
Returns:
Tuple of (AnalysisResult, waveform_gcs_path)
"""
logger.info(f"[{job_id}] Analyzing and generating waveform: {gcs_audio_path}")
with tempfile.TemporaryDirectory() as temp_dir:
# Download audio file
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
self.storage_service.download_file(gcs_audio_path, local_audio_path)
# Run analysis
result = self.analyzer.analyze(local_audio_path)
# Generate waveform
local_waveform_path = os.path.join(temp_dir, "waveform.png")
self.waveform_generator.generate(
audio_path=local_audio_path,
output_path=local_waveform_path,
segments=result.audible_segments,
show_time_axis=True,
silence_threshold_db=self.analyzer.silence_threshold_db,
)
# Upload waveform to GCS
self.storage_service.upload_file(
local_waveform_path,
gcs_waveform_destination
)
logger.info(
f"[{job_id}] Analysis and waveform generation complete. "
f"Waveform uploaded to: {gcs_waveform_destination}"
)
return result, gcs_waveform_destination
def get_waveform_data(
self,
gcs_audio_path: str,
job_id: str,
num_points: int = 500,
) -> tuple[list[float], float]:
"""
Get waveform data (amplitude envelope) for client-side rendering.
This is useful when the frontend wants to render the waveform
itself using Canvas or SVG, rather than displaying a pre-generated
image.
Args:
gcs_audio_path: Path to the audio file in GCS
job_id: Job ID for logging
num_points: Number of data points to return
Returns:
Tuple of (amplitude_values, duration_seconds)
"""
logger.info(f"[{job_id}] Getting waveform data: {gcs_audio_path}")
with tempfile.TemporaryDirectory() as temp_dir:
# Download audio file
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
self.storage_service.download_file(gcs_audio_path, local_audio_path)
# Generate waveform data
amplitudes, duration = self.waveform_generator.generate_data_only(
audio_path=local_audio_path,
num_points=num_points,
)
return amplitudes, duration
def generate_waveform_with_mutes(
self,
gcs_audio_path: str,
job_id: str,
gcs_waveform_destination: str,
mute_regions: list,
) -> str:
"""
Generate a waveform image with mute regions highlighted.
This is useful for showing the user which regions will be muted
in the custom instrumental.
Args:
gcs_audio_path: Path to the audio file in GCS
job_id: Job ID for logging
gcs_waveform_destination: Where to upload the waveform image
mute_regions: List of MuteRegion objects to highlight
Returns:
GCS path to the uploaded waveform image
"""
from karaoke_gen.instrumental_review import MuteRegion
logger.info(
f"[{job_id}] Generating waveform with {len(mute_regions)} mute regions"
)
with tempfile.TemporaryDirectory() as temp_dir:
# Download audio file
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
self.storage_service.download_file(gcs_audio_path, local_audio_path)
# First run analysis to get segments
result = self.analyzer.analyze(local_audio_path)
# Generate waveform with mute regions
local_waveform_path = os.path.join(temp_dir, "waveform_with_mutes.png")
self.waveform_generator.generate(
audio_path=local_audio_path,
output_path=local_waveform_path,
segments=result.audible_segments,
mute_regions=mute_regions,
show_time_axis=True,
)
# Upload to GCS
self.storage_service.upload_file(
local_waveform_path,
gcs_waveform_destination
)
return gcs_waveform_destination
+278
View File
@@ -0,0 +1,278 @@
"""
Audio editing service for creating custom instrumentals.
This service wraps the shared karaoke_gen.instrumental_review module
to provide GCS-integrated audio editing capabilities.
"""
import logging
import os
import tempfile
from typing import List, Optional
from backend.services.storage_service import StorageService
from karaoke_gen.instrumental_review import (
AudioEditor,
MuteRegion,
)
from karaoke_gen.instrumental_review.models import CustomInstrumentalResult
logger = logging.getLogger(__name__)
class AudioEditingService:
"""
Service for creating custom instrumental tracks with GCS integration.
This service acts as a thin wrapper around the shared AudioEditor
class, handling GCS download/upload operations.
The actual editing logic is in the shared karaoke_gen.instrumental_review
module, ensuring feature parity between local and remote workflows.
"""
def __init__(
self,
storage_service: Optional[StorageService] = None,
output_format: str = "flac",
):
"""
Initialize the audio editing service.
Args:
storage_service: GCS storage service. If not provided, a new
instance will be created.
output_format: Output audio format. Default is "flac".
"""
self.storage_service = storage_service or StorageService()
self.editor = AudioEditor(output_format=output_format)
def create_custom_instrumental(
self,
gcs_clean_instrumental_path: str,
gcs_backing_vocals_path: str,
mute_regions: List[MuteRegion],
gcs_output_path: str,
job_id: str,
) -> CustomInstrumentalResult:
"""
Create a custom instrumental by muting regions of backing vocals.
This method:
1. Downloads clean instrumental and backing vocals from GCS
2. Applies mute regions to backing vocals
3. Combines with clean instrumental
4. Uploads result to GCS
Args:
gcs_clean_instrumental_path: Path to clean instrumental in GCS
gcs_backing_vocals_path: Path to backing vocals in GCS
mute_regions: List of regions to mute in backing vocals
gcs_output_path: Where to upload the custom instrumental in GCS
job_id: Job ID for logging
Returns:
CustomInstrumentalResult with details about the created file
"""
logger.info(
f"[{job_id}] Creating custom instrumental with "
f"{len(mute_regions)} mute regions"
)
with tempfile.TemporaryDirectory() as temp_dir:
# Download input files
local_clean_path = os.path.join(temp_dir, "clean_instrumental.flac")
local_backing_path = os.path.join(temp_dir, "backing_vocals.flac")
local_output_path = os.path.join(temp_dir, "custom_instrumental.flac")
logger.debug(f"[{job_id}] Downloading clean instrumental")
self.storage_service.download_file(
gcs_clean_instrumental_path,
local_clean_path
)
logger.debug(f"[{job_id}] Downloading backing vocals")
self.storage_service.download_file(
gcs_backing_vocals_path,
local_backing_path
)
# Create custom instrumental using shared editor
result = self.editor.create_custom_instrumental(
clean_instrumental_path=local_clean_path,
backing_vocals_path=local_backing_path,
mute_regions=mute_regions,
output_path=local_output_path,
)
# Upload result to GCS
logger.debug(f"[{job_id}] Uploading custom instrumental to GCS")
self.storage_service.upload_file(local_output_path, gcs_output_path)
# Update result with GCS path
result.output_path = gcs_output_path
logger.info(
f"[{job_id}] Custom instrumental created: {gcs_output_path}, "
f"muted {result.total_muted_duration_seconds:.1f}s across "
f"{len(result.mute_regions_applied)} regions"
)
return result
def create_preview(
self,
gcs_clean_instrumental_path: str,
gcs_backing_vocals_path: str,
mute_regions: List[MuteRegion],
gcs_preview_path: str,
job_id: str,
preview_duration_seconds: Optional[float] = None,
) -> str:
"""
Create a preview of the custom instrumental.
This is similar to create_custom_instrumental but optimized for
quick preview generation (optionally truncated duration).
Args:
gcs_clean_instrumental_path: Path to clean instrumental in GCS
gcs_backing_vocals_path: Path to backing vocals in GCS
mute_regions: List of regions to mute
gcs_preview_path: Where to upload the preview in GCS
job_id: Job ID for logging
preview_duration_seconds: Optional max duration for preview
Returns:
GCS path to the uploaded preview
"""
logger.info(f"[{job_id}] Creating preview with {len(mute_regions)} mute regions")
with tempfile.TemporaryDirectory() as temp_dir:
# Download input files
local_clean_path = os.path.join(temp_dir, "clean_instrumental.flac")
local_backing_path = os.path.join(temp_dir, "backing_vocals.flac")
local_preview_path = os.path.join(temp_dir, "preview.flac")
self.storage_service.download_file(
gcs_clean_instrumental_path,
local_clean_path
)
self.storage_service.download_file(
gcs_backing_vocals_path,
local_backing_path
)
# Generate preview
from pydub import AudioSegment
preview = self.editor.preview_with_mutes(
clean_instrumental_path=local_clean_path,
backing_vocals_path=local_backing_path,
mute_regions=mute_regions,
)
# Optionally truncate
if preview_duration_seconds:
max_ms = int(preview_duration_seconds * 1000)
preview = preview[:max_ms]
# Export and upload
preview.export(local_preview_path, format="flac")
self.storage_service.upload_file(local_preview_path, gcs_preview_path)
logger.info(f"[{job_id}] Preview created: {gcs_preview_path}")
return gcs_preview_path
def mute_backing_vocals_only(
self,
gcs_backing_vocals_path: str,
mute_regions: List[MuteRegion],
gcs_output_path: str,
job_id: str,
) -> str:
"""
Apply mute regions to backing vocals without combining with instrumental.
This is useful for creating an edited backing vocals track that
can be combined with the instrumental later.
Args:
gcs_backing_vocals_path: Path to backing vocals in GCS
mute_regions: List of regions to mute
gcs_output_path: Where to upload the edited backing vocals
job_id: Job ID for logging
Returns:
GCS path to the uploaded edited backing vocals
"""
logger.info(
f"[{job_id}] Muting backing vocals with {len(mute_regions)} regions"
)
with tempfile.TemporaryDirectory() as temp_dir:
local_backing_path = os.path.join(temp_dir, "backing_vocals.flac")
local_output_path = os.path.join(temp_dir, "muted_backing.flac")
self.storage_service.download_file(
gcs_backing_vocals_path,
local_backing_path
)
self.editor.apply_mute_to_single_track(
audio_path=local_backing_path,
mute_regions=mute_regions,
output_path=local_output_path,
)
self.storage_service.upload_file(local_output_path, gcs_output_path)
logger.info(f"[{job_id}] Muted backing vocals: {gcs_output_path}")
return gcs_output_path
def validate_mute_regions(
self,
mute_regions: List[MuteRegion],
total_duration_seconds: float,
) -> List[str]:
"""
Validate mute regions for consistency.
Args:
mute_regions: List of mute regions to validate
total_duration_seconds: Total duration of the audio
Returns:
List of validation error messages (empty if valid)
"""
errors = []
for i, region in enumerate(mute_regions):
if region.start_seconds < 0:
errors.append(
f"Region {i}: start_seconds ({region.start_seconds}) cannot be negative"
)
if region.end_seconds <= region.start_seconds:
errors.append(
f"Region {i}: end_seconds ({region.end_seconds}) must be after "
f"start_seconds ({region.start_seconds})"
)
if region.start_seconds > total_duration_seconds:
errors.append(
f"Region {i}: start_seconds ({region.start_seconds}) exceeds "
f"audio duration ({total_duration_seconds})"
)
if region.end_seconds > total_duration_seconds:
# Not an error, but log a warning - the region will be clamped
logger.warning(
f"Region {i}: end_seconds ({region.end_seconds}) exceeds "
f"audio duration ({total_duration_seconds}), will be clamped"
)
return errors
+348
View File
@@ -0,0 +1,348 @@
"""
Audio search service for finding and downloading audio files.
This service integrates with flacfetch to search for audio from various sources
(YouTube, music trackers, etc.) and download the selected audio file.
"""
import logging
import os
import tempfile
from dataclasses import dataclass, asdict
from typing import List, Optional, Dict, Any
logger = logging.getLogger(__name__)
@dataclass
class AudioSearchResult:
"""
Represents a single search result for audio.
This is a serializable representation of a search result that can be
stored in Firestore and sent to clients.
"""
title: str
artist: str
provider: str
url: str
duration: Optional[int] = None # Duration in seconds
quality: Optional[str] = None # e.g., "FLAC", "320kbps", etc.
source_id: Optional[str] = None # Unique ID from the source
index: int = 0 # Index in the results list (for selection)
def to_dict(self) -> Dict[str, Any]:
"""Convert to dict for JSON serialization."""
return asdict(self)
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "AudioSearchResult":
"""Create from dict."""
return cls(**data)
@dataclass
class AudioDownloadResult:
"""Result of downloading an audio file."""
filepath: str
artist: str
title: str
provider: str
duration: Optional[int] = None
quality: Optional[str] = None
class AudioSearchError(Exception):
"""Base exception for audio search errors."""
pass
class NoResultsError(AudioSearchError):
"""Raised when no search results are found."""
pass
class DownloadError(AudioSearchError):
"""Raised when download fails."""
pass
class AudioSearchService:
"""
Service for searching and downloading audio files using flacfetch.
This service is used by the backend to:
1. Search for audio by artist/title
2. Return results for user selection
3. Download the selected audio to a local file
"""
# Sentinel value to indicate "use environment variable"
_USE_ENV = object()
def __init__(
self,
redacted_api_key: Optional[str] = _USE_ENV,
ops_api_key: Optional[str] = _USE_ENV,
):
"""
Initialize the audio search service.
Args:
redacted_api_key: API key for Redacted tracker (optional, uses env if not provided)
ops_api_key: API key for OPS tracker (optional, uses env if not provided)
"""
# Use environment variables if not explicitly provided
if redacted_api_key is self._USE_ENV:
self._redacted_api_key = os.environ.get("REDACTED_API_KEY")
else:
self._redacted_api_key = redacted_api_key
if ops_api_key is self._USE_ENV:
self._ops_api_key = os.environ.get("OPS_API_KEY")
else:
self._ops_api_key = ops_api_key
self._manager = None
self._raw_results = {} # Cache raw results for download
def _get_manager(self):
"""Lazily initialize and return the FetchManager."""
if self._manager is None:
try:
from flacfetch.core.manager import FetchManager
from flacfetch.providers.youtube import YouTubeProvider
except ImportError as e:
logger.error(f"flacfetch not installed: {e}")
raise AudioSearchError(
"flacfetch library is not installed. "
"Install with: pip install flacfetch"
) from e
self._manager = FetchManager()
# Add providers based on available API keys
if self._redacted_api_key:
try:
from flacfetch.providers.redacted import RedactedProvider
self._manager.add_provider(RedactedProvider(api_key=self._redacted_api_key))
logger.debug("Added Redacted provider")
except ImportError:
logger.warning("Redacted provider not available")
if self._ops_api_key:
try:
from flacfetch.providers.ops import OPSProvider
self._manager.add_provider(OPSProvider(api_key=self._ops_api_key))
logger.debug("Added OPS provider")
except ImportError:
logger.warning("OPS provider not available")
# Always add YouTube as a fallback provider
self._manager.add_provider(YouTubeProvider())
logger.debug("Added YouTube provider")
return self._manager
def search(self, artist: str, title: str) -> List[AudioSearchResult]:
"""
Search for audio matching the given artist and title.
Args:
artist: The artist name to search for
title: The track title to search for
Returns:
List of AudioSearchResult objects
Raises:
NoResultsError: If no results are found
AudioSearchError: For other errors
"""
try:
from flacfetch.core.models import TrackQuery
except ImportError as e:
raise AudioSearchError("flacfetch not installed") from e
manager = self._get_manager()
query = TrackQuery(artist=artist, title=title)
logger.info(f"Searching for audio: {artist} - {title}")
try:
results = manager.search(query)
except Exception as e:
logger.error(f"Search failed: {e}")
raise AudioSearchError(f"Search failed: {e}") from e
if not results:
raise NoResultsError(f"No results found for: {artist} - {title}")
# Convert to our AudioSearchResult format and cache raw results
search_results = []
self._raw_results.clear()
for i, result in enumerate(results):
search_result = AudioSearchResult(
title=getattr(result, "title", title),
artist=getattr(result, "artist", artist),
provider=getattr(result, "provider", "Unknown"),
url=getattr(result, "url", ""),
duration=getattr(result, "duration", None),
quality=getattr(result, "quality", None),
source_id=getattr(result, "id", str(i)),
index=i,
)
search_results.append(search_result)
# Cache raw result for later download
self._raw_results[i] = result
logger.info(f"Found {len(search_results)} results for: {artist} - {title}")
return search_results
def select_best(self, results: List[AudioSearchResult]) -> int:
"""
Select the best result from a list of search results.
Uses flacfetch's built-in quality ranking to select the best source.
Args:
results: List of search results
Returns:
Index of the best result
"""
if not results:
return 0
manager = self._get_manager()
# Get raw results for the indices we have
raw_results = [self._raw_results.get(r.index) for r in results if r.index in self._raw_results]
if raw_results:
try:
best = manager.select_best(raw_results)
# Find index of best result
for i, raw in enumerate(raw_results):
if raw == best:
return results[i].index
except Exception as e:
logger.warning(f"select_best failed, using first result: {e}")
# Fallback: return first result
return results[0].index if results else 0
def download(
self,
result_index: int,
output_dir: str,
output_filename: Optional[str] = None,
) -> AudioDownloadResult:
"""
Download audio from a search result.
Args:
result_index: Index of the result to download (from search results)
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
Returns:
AudioDownloadResult with the downloaded file path
Raises:
DownloadError: If download fails
"""
if result_index not in self._raw_results:
raise DownloadError(f"No cached result for index {result_index}. Run search() first.")
raw_result = self._raw_results[result_index]
manager = self._get_manager()
# Ensure output directory exists
os.makedirs(output_dir, exist_ok=True)
# Get metadata from raw result
artist = getattr(raw_result, "artist", "Unknown")
title = getattr(raw_result, "title", "Unknown")
provider = getattr(raw_result, "provider", "Unknown")
# Generate filename if not provided
if output_filename is None:
output_filename = f"{artist} - {title}"
logger.info(f"Downloading: {artist} - {title} from {provider}")
try:
filepath = manager.download(
raw_result,
output_path=output_dir,
output_filename=output_filename,
)
if filepath is None:
raise DownloadError(f"Download returned no file path for: {artist} - {title}")
logger.info(f"Downloaded to: {filepath}")
return AudioDownloadResult(
filepath=filepath,
artist=artist,
title=title,
provider=provider,
duration=getattr(raw_result, "duration", None),
quality=getattr(raw_result, "quality", None),
)
except Exception as e:
raise DownloadError(f"Failed to download {artist} - {title}: {e}") from e
def search_and_download_auto(
self,
artist: str,
title: str,
output_dir: str,
output_filename: Optional[str] = None,
) -> AudioDownloadResult:
"""
Search for audio and automatically download the best result.
This is a convenience method that combines search(), select_best(),
and download() for automated/non-interactive usage.
Args:
artist: The artist name to search for
title: The track title to search for
output_dir: Directory to save the downloaded file
output_filename: Optional filename (without extension)
Returns:
AudioDownloadResult with the downloaded file path
Raises:
NoResultsError: If no results are found
DownloadError: If download fails
"""
# Search
results = self.search(artist, title)
# Select best
best_index = self.select_best(results)
logger.info(f"Auto-selected result {best_index}: {results[best_index].artist} - {results[best_index].title} ({results[best_index].provider})")
# Download
return self.download(best_index, output_dir, output_filename)
# Singleton instance
_audio_search_service: Optional[AudioSearchService] = None
def get_audio_search_service() -> AudioSearchService:
"""Get the singleton AudioSearchService instance."""
global _audio_search_service
if _audio_search_service is None:
_audio_search_service = AudioSearchService()
return _audio_search_service
+4 -5
View File
@@ -138,7 +138,7 @@ def job_log_context(
Args:
job_id: Job ID to associate logs with
worker: Worker name for log categorization
logger_names: List of logger names to capture (None = capture root + common loggers)
logger_names: List of logger names to capture (None = capture common loggers)
min_level: Minimum log level to capture
Usage:
@@ -147,14 +147,13 @@ def job_log_context(
correction_operations.add_lyrics_source(...)
"""
# Default loggers to capture - these cover most of our processing
# Note: We intentionally DON'T include the root logger ('') to avoid
# duplicate log entries since logs propagate up the logger hierarchy.
# Only capture at the top-level package loggers.
if logger_names is None:
logger_names = [
'', # Root logger
'backend.api.routes.review',
'lyrics_transcriber',
'lyrics_transcriber.correction',
'lyrics_transcriber.correction.anchor_sequence',
'lyrics_transcriber.correction.corrector',
'karaoke_gen',
]
+4
View File
@@ -55,6 +55,10 @@ class JobManager:
youtube_description=job_create.youtube_description,
webhook_url=job_create.webhook_url,
user_email=job_create.user_email,
# Two-phase workflow (Batch 6)
prep_only=job_create.prep_only,
finalise_only=job_create.finalise_only,
keep_brand_code=job_create.keep_brand_code,
# Request metadata (for tracking and filtering)
request_metadata=job_create.request_metadata,
)
+37 -16
View File
@@ -65,6 +65,26 @@ class StorageService:
a private key available. Requires the service account to have
roles/iam.serviceAccountTokenCreator on itself.
"""
return self._generate_signed_url_internal(blob_path, "GET", expiration_minutes)
def generate_signed_upload_url(self, blob_path: str, content_type: str = "application/octet-stream", expiration_minutes: int = 60) -> str:
"""Generate a signed URL for uploading a file directly to GCS.
This allows clients to upload files directly to GCS without going through
the backend, bypassing any request body size limits.
Args:
blob_path: The destination path in GCS
content_type: The expected content type of the upload
expiration_minutes: How long the URL is valid for
Returns:
A signed URL that accepts PUT requests with the file content
"""
return self._generate_signed_url_internal(blob_path, "PUT", expiration_minutes, content_type)
def _generate_signed_url_internal(self, blob_path: str, method: str, expiration_minutes: int = 60, content_type: Optional[str] = None) -> str:
"""Internal method to generate signed URLs for GET or PUT operations."""
import google.auth
from google.auth.transport import requests
@@ -74,6 +94,17 @@ class StorageService:
# Get default credentials and refresh to ensure we have a valid token
credentials, project = google.auth.default()
# Common kwargs for signed URL generation
kwargs = {
"version": "v4",
"expiration": timedelta(minutes=expiration_minutes),
"method": method,
}
# For PUT requests, we need to specify the content type in headers
if method == "PUT" and content_type:
kwargs["headers"] = {"Content-Type": content_type}
# Check if we're using compute credentials (Cloud Run/GCE)
# These need to use IAM signBlob via service_account_email + access_token
if hasattr(credentials, 'service_account_email'):
@@ -81,25 +112,15 @@ class StorageService:
auth_request = requests.Request()
credentials.refresh(auth_request)
url = blob.generate_signed_url(
version="v4",
expiration=timedelta(minutes=expiration_minutes),
method="GET",
service_account_email=credentials.service_account_email,
access_token=credentials.token
)
else:
# Local development with service account key or user credentials
url = blob.generate_signed_url(
version="v4",
expiration=timedelta(minutes=expiration_minutes),
method="GET"
)
kwargs["service_account_email"] = credentials.service_account_email
kwargs["access_token"] = credentials.token
logger.info(f"Generated signed URL for {blob_path}")
url = blob.generate_signed_url(**kwargs)
logger.info(f"Generated signed {method} URL for {blob_path}")
return url
except Exception as e:
logger.error(f"Error generating signed URL for {blob_path}: {e}")
logger.error(f"Error generating signed {method} URL for {blob_path}: {e}")
raise
def delete_file(self, blob_path: str) -> None:
+261 -18
View File
@@ -2,7 +2,9 @@
Worker coordination service.
Handles triggering and coordinating background workers.
Implements proper HTTP-based triggers for production use.
Supports two modes:
- Cloud Tasks (production): Guaranteed delivery, automatic retries, horizontal scaling
- Direct HTTP (development): Faster iteration, simpler debugging
SOLID Principles:
- Single Responsibility: Only handles worker coordination
@@ -11,20 +13,44 @@ SOLID Principles:
"""
import logging
import os
import httpx
import json
from typing import Optional
import httpx
from backend.config import get_settings
logger = logging.getLogger(__name__)
# Mapping of worker types to their Cloud Tasks queue names
# These queues are created by Pulumi in infrastructure/__main__.py
WORKER_QUEUES = {
"audio": "audio-worker-queue",
"lyrics": "lyrics-worker-queue",
"screens": "screens-worker-queue",
"render-video": "render-worker-queue",
"video": "video-worker-queue",
}
class WorkerService:
"""
Service for coordinating background workers.
Handles triggering workers via internal HTTP API.
In production, workers run as separate Cloud Run requests or Cloud Tasks.
Supports two execution modes controlled by ENABLE_CLOUD_TASKS env var:
1. Cloud Tasks mode (production, ENABLE_CLOUD_TASKS=true):
- Tasks enqueued to Cloud Tasks for guaranteed delivery
- Automatic retries on failure
- Each task runs in dedicated Cloud Run instance
- Rate limiting to protect external APIs (Modal, AudioShake)
2. Direct HTTP mode (development, ENABLE_CLOUD_TASKS=false):
- Direct HTTP POST to internal worker endpoints
- Worker runs as FastAPI BackgroundTask
- Faster iteration for local development
- No retry guarantees
"""
def __init__(self):
@@ -32,6 +58,44 @@ class WorkerService:
self.settings = get_settings()
self._base_url = self._get_base_url()
self._admin_token = self._get_admin_token()
self._use_cloud_tasks = self._should_use_cloud_tasks()
self._tasks_client = None
if self._use_cloud_tasks:
logger.info("WorkerService initialized with Cloud Tasks mode")
else:
logger.info("WorkerService initialized with direct HTTP mode")
def _should_use_cloud_tasks(self) -> bool:
"""
Check if Cloud Tasks should be used for worker coordination.
Controlled by ENABLE_CLOUD_TASKS setting (from environment variable).
Default is false (direct HTTP mode) for backward compatibility.
Returns:
True if Cloud Tasks should be used, False for direct HTTP
"""
return self.settings.enable_cloud_tasks
@property
def tasks_client(self):
"""
Lazy-initialize Cloud Tasks client.
Only created when needed to avoid import overhead in development mode.
"""
if self._tasks_client is None and self._use_cloud_tasks:
try:
from google.cloud import tasks_v2
self._tasks_client = tasks_v2.CloudTasksClient()
except ImportError:
logger.error(
"google-cloud-tasks not installed. "
"Install with: pip install google-cloud-tasks"
)
raise
return self._tasks_client
def _get_admin_token(self) -> Optional[str]:
"""
@@ -49,9 +113,10 @@ class WorkerService:
"""
Get base URL for internal API calls.
In production: Cloud Run service URL
In test: TEST_SERVER_URL
In development: localhost
Priority:
1. TEST_SERVER_URL env var (for tests)
2. CLOUD_RUN_SERVICE_URL env var (for Cloud Tasks to call back)
3. localhost with PORT env var (for development)
Returns:
Base URL for API calls
@@ -61,14 +126,11 @@ class WorkerService:
if test_url:
return test_url
# Check if running in Cloud Run
cloud_run_url = self.settings.google_cloud_project
if cloud_run_url and self.settings.environment == "production":
# TODO: Get actual Cloud Run service URL
# For now, use environment variable
service_url = os.getenv("CLOUD_RUN_SERVICE_URL")
if service_url:
return service_url
# Production: Cloud Run service URL
# This must be the publicly accessible URL for Cloud Tasks to call
service_url = os.getenv("CLOUD_RUN_SERVICE_URL")
if service_url:
return service_url
# Development mode: use localhost with PORT env var
port = os.getenv("PORT", "8000")
@@ -81,10 +143,120 @@ class WorkerService:
timeout_seconds: int = 30
) -> bool:
"""
Trigger a background worker via internal API.
Trigger a background worker for a job.
In production (ENABLE_CLOUD_TASKS=true):
- Enqueues task to Cloud Tasks queue
- Cloud Tasks delivers HTTP request to internal endpoint
- Automatic retries on failure with exponential backoff
- Rate limiting protects external APIs
In development (ENABLE_CLOUD_TASKS=false):
- Direct HTTP POST to internal endpoint
- Faster iteration, but no retry guarantees
- Worker runs in same container as API
Args:
worker_type: Worker type ("audio", "lyrics", "screens", "video")
worker_type: Worker type ("audio", "lyrics", "screens", "render-video", "video")
job_id: Job ID to process
timeout_seconds: Request timeout (for direct HTTP mode)
Returns:
True if trigger successful (or task enqueued), False otherwise
"""
if self._use_cloud_tasks:
return await self._enqueue_cloud_task(worker_type, job_id)
else:
return await self._trigger_http(worker_type, job_id, timeout_seconds)
async def _enqueue_cloud_task(self, worker_type: str, job_id: str) -> bool:
"""
Enqueue task to Cloud Tasks for guaranteed delivery.
The task will be delivered as an HTTP POST to the internal worker endpoint.
Cloud Tasks handles:
- Retry on failure (with exponential backoff)
- Rate limiting (max dispatches per second)
- Deduplication (via task name if needed)
- OIDC authentication for Cloud Run
Args:
worker_type: Worker type
job_id: Job ID to process
Returns:
True if task enqueued successfully, False otherwise
"""
try:
from google.cloud import tasks_v2
queue_name = WORKER_QUEUES.get(worker_type)
if not queue_name:
logger.error(f"Unknown worker type: {worker_type}")
return False
project = self.settings.google_cloud_project
if not project:
logger.error("GOOGLE_CLOUD_PROJECT not set, cannot enqueue Cloud Task")
return False
location = self.settings.gcp_region # Must match queue location
# Build queue path
parent = self.tasks_client.queue_path(project, location, queue_name)
# Build task payload
task = {
"http_request": {
"http_method": tasks_v2.HttpMethod.POST,
"url": f"{self._base_url}/api/internal/workers/{worker_type}",
"headers": {
"Content-Type": "application/json",
},
"body": json.dumps({"job_id": job_id}).encode(),
# Use OIDC token for Cloud Run authentication
# This allows Cloud Tasks to invoke the Cloud Run service
"oidc_token": {
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
},
},
}
# Add admin auth header if available
# This is needed for the internal endpoint's require_admin dependency
if self._admin_token:
task["http_request"]["headers"]["Authorization"] = f"Bearer {self._admin_token}"
# Create task
# Note: We don't set a task name, allowing Cloud Tasks to generate one
# This prevents duplicate task errors on retries
response = self.tasks_client.create_task(parent=parent, task=task)
logger.info(
f"Created Cloud Task for {worker_type} worker, job {job_id}: {response.name}"
)
return True
except Exception as e:
logger.error(
f"Failed to enqueue Cloud Task for {worker_type}/{job_id}: {e}",
exc_info=True
)
return False
async def _trigger_http(
self,
worker_type: str,
job_id: str,
timeout_seconds: int = 30
) -> bool:
"""
Trigger worker via direct HTTP call (development mode).
This is the original implementation - direct HTTP POST to internal endpoint.
The endpoint adds the worker function to FastAPI BackgroundTasks.
Args:
worker_type: Worker type
job_id: Job ID to process
timeout_seconds: Request timeout
@@ -127,6 +299,9 @@ class WorkerService:
)
return False
# Convenience methods for specific workers
# These provide a cleaner API and better IDE autocomplete
async def trigger_audio_worker(self, job_id: str) -> bool:
"""Trigger audio separation worker."""
return await self.trigger_worker("audio", job_id)
@@ -140,9 +315,68 @@ class WorkerService:
return await self.trigger_worker("screens", job_id)
async def trigger_video_worker(self, job_id: str) -> bool:
"""Trigger video generation worker."""
"""
Trigger video generation worker.
When USE_CLOUD_RUN_JOBS_FOR_VIDEO=true and ENABLE_CLOUD_TASKS=true,
uses Cloud Run Jobs for execution (supports >30 min encoding).
Otherwise, uses Cloud Tasks or direct HTTP.
"""
if self._use_cloud_tasks and self.settings.use_cloud_run_jobs_for_video:
return await self._trigger_cloud_run_job(job_id)
return await self.trigger_worker("video", job_id)
async def _trigger_cloud_run_job(self, job_id: str) -> bool:
"""
Trigger video encoding as a Cloud Run Job.
Cloud Run Jobs support up to 24 hours of execution time,
making them suitable for very long video encoding tasks.
Args:
job_id: Job ID to process
Returns:
True if job was triggered successfully, False otherwise
"""
try:
from google.cloud import run_v2
project = self.settings.google_cloud_project
if not project:
logger.error("GOOGLE_CLOUD_PROJECT not set, cannot trigger Cloud Run Job")
return False
location = self.settings.gcp_region
job_name = f"projects/{project}/locations/{location}/jobs/video-encoding-job"
# Create Cloud Run Jobs client
client = run_v2.JobsClient()
# Run the job with overrides for the specific job_id
request = run_v2.RunJobRequest(
name=job_name,
overrides=run_v2.RunJobRequest.Overrides(
container_overrides=[
run_v2.RunJobRequest.Overrides.ContainerOverride(
args=["python", "-m", "backend.workers.video_worker", "--job-id", job_id],
)
]
)
)
# Run the job (async operation)
operation = client.run_job(request=request)
logger.info(f"Started Cloud Run Job for video encoding, job {job_id}: {operation.metadata}")
return True
except Exception as e:
logger.error(
f"Failed to trigger Cloud Run Job for video/{job_id}: {e}",
exc_info=True
)
return False
async def trigger_render_video_worker(self, job_id: str) -> bool:
"""Trigger render video worker (post-review)."""
return await self.trigger_worker("render-video", job_id)
@@ -166,3 +400,12 @@ def get_worker_service() -> WorkerService:
_worker_service = WorkerService()
return _worker_service
def reset_worker_service() -> None:
"""
Reset the global worker service instance.
Used in tests to ensure clean state between test cases.
"""
global _worker_service
_worker_service = None
+386
View File
@@ -0,0 +1,386 @@
"""
Tests for audio search functionality (Batch 5).
Tests the audio search service, API routes, and job model fields.
"""
import pytest
import os
from unittest.mock import Mock, patch, MagicMock
from datetime import datetime
from backend.models.job import Job, JobCreate, JobStatus, STATE_TRANSITIONS
from backend.services.audio_search_service import (
AudioSearchService,
AudioSearchResult,
AudioDownloadResult,
AudioSearchError,
NoResultsError,
DownloadError,
get_audio_search_service,
)
class TestJobModelAudioSearchFields:
"""Test Job model has audio search fields."""
def test_job_has_audio_search_fields(self):
"""Test Job model has audio search configuration fields."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.utcnow(),
updated_at=datetime.utcnow(),
artist="ABBA",
title="Waterloo",
audio_search_artist="ABBA",
audio_search_title="Waterloo",
auto_download=True,
)
assert job.audio_search_artist == "ABBA"
assert job.audio_search_title == "Waterloo"
assert job.auto_download is True
def test_job_audio_search_fields_default_values(self):
"""Test default values for audio search fields."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.utcnow(),
updated_at=datetime.utcnow(),
)
assert job.audio_search_artist is None
assert job.audio_search_title is None
assert job.auto_download is False
def test_job_create_has_audio_search_fields(self):
"""Test JobCreate model has audio search fields."""
job_create = JobCreate(
artist="ABBA",
title="Waterloo",
audio_search_artist="ABBA",
audio_search_title="Waterloo",
auto_download=True,
)
assert job_create.audio_search_artist == "ABBA"
assert job_create.audio_search_title == "Waterloo"
assert job_create.auto_download is True
class TestJobStatusAudioSearchStates:
"""Test Job status includes audio search states."""
def test_audio_search_statuses_exist(self):
"""Test audio search status values exist."""
assert hasattr(JobStatus, 'SEARCHING_AUDIO')
assert hasattr(JobStatus, 'AWAITING_AUDIO_SELECTION')
assert hasattr(JobStatus, 'DOWNLOADING_AUDIO')
assert JobStatus.SEARCHING_AUDIO == "searching_audio"
assert JobStatus.AWAITING_AUDIO_SELECTION == "awaiting_audio_selection"
assert JobStatus.DOWNLOADING_AUDIO == "downloading_audio"
def test_state_transitions_include_audio_search_flow(self):
"""Test state transitions include audio search flow."""
# PENDING can go to SEARCHING_AUDIO
assert JobStatus.SEARCHING_AUDIO in STATE_TRANSITIONS[JobStatus.PENDING]
# SEARCHING_AUDIO can go to AWAITING_AUDIO_SELECTION or DOWNLOADING_AUDIO
assert JobStatus.AWAITING_AUDIO_SELECTION in STATE_TRANSITIONS[JobStatus.SEARCHING_AUDIO]
assert JobStatus.DOWNLOADING_AUDIO in STATE_TRANSITIONS[JobStatus.SEARCHING_AUDIO]
# AWAITING_AUDIO_SELECTION can go to DOWNLOADING_AUDIO
assert JobStatus.DOWNLOADING_AUDIO in STATE_TRANSITIONS[JobStatus.AWAITING_AUDIO_SELECTION]
# DOWNLOADING_AUDIO can go to DOWNLOADING
assert JobStatus.DOWNLOADING in STATE_TRANSITIONS[JobStatus.DOWNLOADING_AUDIO]
class TestAudioSearchResult:
"""Test AudioSearchResult dataclass."""
def test_create_audio_search_result(self):
"""Test creating an AudioSearchResult."""
result = AudioSearchResult(
title="Waterloo",
artist="ABBA",
provider="YouTube",
url="https://youtube.com/watch?v=abc123",
duration=180,
quality="FLAC",
source_id="abc123",
index=0,
)
assert result.title == "Waterloo"
assert result.artist == "ABBA"
assert result.provider == "YouTube"
assert result.url == "https://youtube.com/watch?v=abc123"
assert result.duration == 180
assert result.quality == "FLAC"
assert result.source_id == "abc123"
assert result.index == 0
def test_to_dict(self):
"""Test converting to dict for serialization."""
result = AudioSearchResult(
title="Waterloo",
artist="ABBA",
provider="YouTube",
url="https://youtube.com/watch?v=abc123",
index=0,
)
data = result.to_dict()
assert data['title'] == "Waterloo"
assert data['artist'] == "ABBA"
assert data['provider'] == "YouTube"
assert data['url'] == "https://youtube.com/watch?v=abc123"
assert data['index'] == 0
def test_from_dict(self):
"""Test creating from dict."""
data = {
'title': "Waterloo",
'artist': "ABBA",
'provider': "YouTube",
'url': "https://youtube.com/watch?v=abc123",
'duration': 180,
'quality': "FLAC",
'source_id': "abc123",
'index': 0,
}
result = AudioSearchResult.from_dict(data)
assert result.title == "Waterloo"
assert result.artist == "ABBA"
assert result.provider == "YouTube"
assert result.index == 0
class TestAudioSearchServiceInit:
"""Test AudioSearchService initialization."""
@patch.dict('os.environ', {'REDACTED_API_KEY': '', 'OPS_API_KEY': ''}, clear=False)
def test_init_with_no_keys(self):
"""Test initialization without API keys."""
# Pass explicit None to override environment
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
# When explicitly passed None, it should be None (not from env)
assert service._redacted_api_key is None
assert service._ops_api_key is None
assert service._manager is None
def test_init_with_keys(self):
"""Test initialization with API keys."""
service = AudioSearchService(
redacted_api_key="test_redacted_key",
ops_api_key="test_ops_key",
)
assert service._redacted_api_key == "test_redacted_key"
assert service._ops_api_key == "test_ops_key"
def test_init_reads_from_environment(self):
"""Test initialization reads keys from environment variables."""
import os
# Just test that the service can be initialized
service = AudioSearchService()
# The service should have API keys from env or None
assert service._manager is None # Not yet initialized
class TestAudioSearchServiceSearch:
"""Test AudioSearchService.search() method."""
@patch('backend.services.audio_search_service.AudioSearchService._get_manager')
def test_search_returns_results(self, mock_get_manager):
"""Test search returns AudioSearchResult list."""
# Mock the flacfetch result
mock_result = Mock()
mock_result.title = "Waterloo"
mock_result.artist = "ABBA"
mock_result.provider = "YouTube"
mock_result.url = "https://youtube.com/watch?v=abc123"
mock_result.duration = 180
mock_result.quality = "FLAC"
mock_result.id = "abc123"
mock_manager = Mock()
mock_manager.search.return_value = [mock_result]
mock_get_manager.return_value = mock_manager
service = AudioSearchService()
results = service.search("ABBA", "Waterloo")
assert len(results) == 1
assert results[0].title == "Waterloo"
assert results[0].artist == "ABBA"
assert results[0].provider == "YouTube"
assert results[0].index == 0
@patch('backend.services.audio_search_service.AudioSearchService._get_manager')
def test_search_no_results_raises_error(self, mock_get_manager):
"""Test search raises NoResultsError when no results."""
mock_manager = Mock()
mock_manager.search.return_value = []
mock_get_manager.return_value = mock_manager
service = AudioSearchService()
with pytest.raises(NoResultsError) as exc_info:
service.search("Unknown Artist", "Unknown Song")
assert "No results found" in str(exc_info.value)
@patch('backend.services.audio_search_service.AudioSearchService._get_manager')
def test_search_multiple_results(self, mock_get_manager):
"""Test search returns multiple results with correct indices."""
mock_results = []
for i in range(3):
mock_result = Mock()
mock_result.title = f"Song {i}"
mock_result.artist = "Artist"
mock_result.provider = ["YouTube", "Redacted", "OPS"][i]
mock_result.url = f"https://example.com/{i}"
mock_result.duration = 180 + i * 10
mock_result.quality = ["320kbps", "FLAC", "FLAC"][i]
mock_result.id = str(i)
mock_results.append(mock_result)
mock_manager = Mock()
mock_manager.search.return_value = mock_results
mock_get_manager.return_value = mock_manager
service = AudioSearchService()
results = service.search("Artist", "Song")
assert len(results) == 3
assert results[0].index == 0
assert results[1].index == 1
assert results[2].index == 2
class TestAudioSearchServiceSelectBest:
"""Test AudioSearchService.select_best() method."""
@patch('backend.services.audio_search_service.AudioSearchService._get_manager')
def test_select_best_uses_manager(self, mock_get_manager):
"""Test select_best uses flacfetch's select_best."""
# First need to do a search to populate cache
mock_result = Mock()
mock_result.title = "Waterloo"
mock_result.artist = "ABBA"
mock_result.provider = "YouTube"
mock_result.url = "https://youtube.com/watch?v=abc123"
mock_result.duration = 180
mock_result.quality = "FLAC"
mock_result.id = "abc123"
mock_manager = Mock()
mock_manager.search.return_value = [mock_result]
mock_manager.select_best.return_value = mock_result
mock_get_manager.return_value = mock_manager
service = AudioSearchService()
results = service.search("ABBA", "Waterloo")
best_index = service.select_best(results)
assert best_index == 0
def test_select_best_empty_list_returns_zero(self):
"""Test select_best returns 0 for empty list."""
service = AudioSearchService()
best_index = service.select_best([])
assert best_index == 0
class TestAudioSearchServiceDownload:
"""Test AudioSearchService.download() method."""
@patch('backend.services.audio_search_service.AudioSearchService._get_manager')
def test_download_without_search_raises_error(self, mock_get_manager):
"""Test download without prior search raises error."""
mock_manager = Mock()
mock_get_manager.return_value = mock_manager
service = AudioSearchService()
with pytest.raises(DownloadError) as exc_info:
service.download(0, "/tmp")
assert "No cached result" in str(exc_info.value)
@patch('backend.services.audio_search_service.AudioSearchService._get_manager')
def test_download_after_search(self, mock_get_manager):
"""Test download after search works."""
import tempfile
import os
# Create a temp file to simulate downloaded file
temp_dir = tempfile.mkdtemp()
temp_file = os.path.join(temp_dir, "test.flac")
with open(temp_file, 'w') as f:
f.write("test")
mock_result = Mock()
mock_result.title = "Waterloo"
mock_result.artist = "ABBA"
mock_result.provider = "YouTube"
mock_result.url = "https://youtube.com/watch?v=abc123"
mock_result.duration = 180
mock_result.quality = "FLAC"
mock_result.id = "abc123"
mock_manager = Mock()
mock_manager.search.return_value = [mock_result]
mock_manager.download.return_value = temp_file
mock_get_manager.return_value = mock_manager
service = AudioSearchService()
service.search("ABBA", "Waterloo")
result = service.download(0, temp_dir)
assert result.filepath == temp_file
assert result.artist == "ABBA"
assert result.title == "Waterloo"
assert result.provider == "YouTube"
# Cleanup
os.remove(temp_file)
os.rmdir(temp_dir)
class TestGetAudioSearchService:
"""Test get_audio_search_service singleton."""
def test_get_audio_search_service_returns_instance(self):
"""Test get_audio_search_service returns an instance."""
# Reset singleton
import backend.services.audio_search_service as module
module._audio_search_service = None
service = get_audio_search_service()
assert service is not None
assert isinstance(service, AudioSearchService)
def test_get_audio_search_service_singleton(self):
"""Test get_audio_search_service returns same instance."""
service1 = get_audio_search_service()
service2 = get_audio_search_service()
assert service1 is service2
+363
View File
@@ -0,0 +1,363 @@
"""
Unit tests for audio analysis and editing services.
These tests verify the backend service wrappers that use the shared
karaoke_gen.instrumental_review module with GCS integration.
"""
import os
import tempfile
import pytest
from unittest.mock import MagicMock, patch, ANY
from pydub import AudioSegment
from pydub.generators import Sine
from karaoke_gen.instrumental_review import MuteRegion
class TestAudioAnalysisService:
"""Tests for AudioAnalysisService."""
@pytest.fixture
def mock_storage_service(self):
"""Create a mock storage service."""
mock = MagicMock()
return mock
@pytest.fixture
def temp_audio_file(self, tmp_path):
"""Create a temporary audio file for testing."""
audio_path = tmp_path / "test_audio.flac"
# Create a simple sine wave audio
tone = Sine(440).to_audio_segment(duration=5000) - 15
tone.export(str(audio_path), format="flac")
return str(audio_path)
@pytest.fixture
def silent_audio_file(self, tmp_path):
"""Create a silent audio file for testing."""
audio_path = tmp_path / "silent_audio.flac"
audio = AudioSegment.silent(duration=5000, frame_rate=44100)
audio.export(str(audio_path), format="flac")
return str(audio_path)
def test_service_initialization(self, mock_storage_service):
"""Test service initializes with default parameters."""
from backend.services.audio_analysis_service import AudioAnalysisService
service = AudioAnalysisService(storage_service=mock_storage_service)
assert service.storage_service == mock_storage_service
assert service.analyzer.silence_threshold_db == -40.0
def test_service_custom_threshold(self, mock_storage_service):
"""Test service accepts custom silence threshold."""
from backend.services.audio_analysis_service import AudioAnalysisService
service = AudioAnalysisService(
storage_service=mock_storage_service,
silence_threshold_db=-35.0,
)
assert service.analyzer.silence_threshold_db == -35.0
def test_analyze_backing_vocals_downloads_file(
self, mock_storage_service, temp_audio_file
):
"""Test that analyze_backing_vocals downloads file from GCS."""
from backend.services.audio_analysis_service import AudioAnalysisService
# Set up mock to copy the temp file to the download location
def mock_download(gcs_path, local_path):
import shutil
shutil.copy(temp_audio_file, local_path)
return local_path
mock_storage_service.download_file.side_effect = mock_download
service = AudioAnalysisService(storage_service=mock_storage_service)
result = service.analyze_backing_vocals(
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
job_id="test-job",
)
# Verify download was called
mock_storage_service.download_file.assert_called_once()
# Verify result is valid
assert result is not None
assert hasattr(result, 'has_audible_content')
def test_analyze_silent_audio_returns_no_audible(
self, mock_storage_service, silent_audio_file
):
"""Test that silent audio returns has_audible_content=False."""
from backend.services.audio_analysis_service import AudioAnalysisService
def mock_download(gcs_path, local_path):
import shutil
shutil.copy(silent_audio_file, local_path)
return local_path
mock_storage_service.download_file.side_effect = mock_download
service = AudioAnalysisService(storage_service=mock_storage_service)
result = service.analyze_backing_vocals(
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
job_id="test-job",
)
assert result.has_audible_content is False
assert len(result.audible_segments) == 0
def test_analyze_and_generate_waveform_uploads_image(
self, mock_storage_service, temp_audio_file
):
"""Test that analyze_and_generate_waveform uploads waveform to GCS."""
from backend.services.audio_analysis_service import AudioAnalysisService
def mock_download(gcs_path, local_path):
import shutil
shutil.copy(temp_audio_file, local_path)
return local_path
mock_storage_service.download_file.side_effect = mock_download
mock_storage_service.upload_file.return_value = "jobs/test/analysis/waveform.png"
service = AudioAnalysisService(storage_service=mock_storage_service)
result, waveform_path = service.analyze_and_generate_waveform(
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
job_id="test-job",
gcs_waveform_destination="jobs/test/analysis/waveform.png",
)
# Verify upload was called
mock_storage_service.upload_file.assert_called_once()
# Verify waveform path is returned
assert waveform_path == "jobs/test/analysis/waveform.png"
def test_get_waveform_data_returns_amplitudes(
self, mock_storage_service, temp_audio_file
):
"""Test that get_waveform_data returns amplitude data."""
from backend.services.audio_analysis_service import AudioAnalysisService
def mock_download(gcs_path, local_path):
import shutil
shutil.copy(temp_audio_file, local_path)
return local_path
mock_storage_service.download_file.side_effect = mock_download
service = AudioAnalysisService(storage_service=mock_storage_service)
amplitudes, duration = service.get_waveform_data(
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
job_id="test-job",
num_points=100,
)
# Verify we get amplitude data
assert isinstance(amplitudes, list)
assert len(amplitudes) > 0
assert all(isinstance(a, float) for a in amplitudes)
# Verify duration is reasonable
assert duration > 0
assert duration <= 10 # Our test file is 5 seconds
class TestAudioEditingService:
"""Tests for AudioEditingService."""
@pytest.fixture
def mock_storage_service(self):
"""Create a mock storage service."""
mock = MagicMock()
return mock
@pytest.fixture
def temp_audio_files(self, tmp_path):
"""Create temporary audio files for testing."""
clean_path = tmp_path / "clean_instrumental.flac"
backing_path = tmp_path / "backing_vocals.flac"
# Create clean instrumental (chord)
tone1 = Sine(220).to_audio_segment(duration=5000) - 15
tone2 = Sine(330).to_audio_segment(duration=5000) - 15
clean = tone1.overlay(tone2)
clean.export(str(clean_path), format="flac")
# Create backing vocals
backing = Sine(440).to_audio_segment(duration=5000) - 20
backing.export(str(backing_path), format="flac")
return str(clean_path), str(backing_path)
def test_service_initialization(self, mock_storage_service):
"""Test service initializes correctly."""
from backend.services.audio_editing_service import AudioEditingService
service = AudioEditingService(storage_service=mock_storage_service)
assert service.storage_service == mock_storage_service
assert service.editor.output_format == "flac"
def test_create_custom_instrumental_downloads_files(
self, mock_storage_service, temp_audio_files
):
"""Test that create_custom_instrumental downloads input files."""
from backend.services.audio_editing_service import AudioEditingService
clean_path, backing_path = temp_audio_files
# Track which files were downloaded
downloaded_files = []
def mock_download(gcs_path, local_path):
downloaded_files.append(gcs_path)
# Copy the appropriate test file
import shutil
if 'clean' in gcs_path:
shutil.copy(clean_path, local_path)
else:
shutil.copy(backing_path, local_path)
return local_path
mock_storage_service.download_file.side_effect = mock_download
mock_storage_service.upload_file.return_value = "jobs/test/stems/custom.flac"
service = AudioEditingService(storage_service=mock_storage_service)
mute_regions = [MuteRegion(start_seconds=1.0, end_seconds=2.0)]
result = service.create_custom_instrumental(
gcs_clean_instrumental_path="jobs/test/stems/clean.flac",
gcs_backing_vocals_path="jobs/test/stems/backing.flac",
mute_regions=mute_regions,
gcs_output_path="jobs/test/stems/custom.flac",
job_id="test-job",
)
# Verify both files were downloaded
assert len(downloaded_files) == 2
assert mock_storage_service.upload_file.called
def test_create_custom_instrumental_uploads_result(
self, mock_storage_service, temp_audio_files
):
"""Test that create_custom_instrumental uploads the result."""
from backend.services.audio_editing_service import AudioEditingService
clean_path, backing_path = temp_audio_files
def mock_download(gcs_path, local_path):
import shutil
if 'clean' in gcs_path:
shutil.copy(clean_path, local_path)
else:
shutil.copy(backing_path, local_path)
return local_path
mock_storage_service.download_file.side_effect = mock_download
mock_storage_service.upload_file.return_value = "jobs/test/stems/custom.flac"
service = AudioEditingService(storage_service=mock_storage_service)
mute_regions = [MuteRegion(start_seconds=1.0, end_seconds=2.0)]
result = service.create_custom_instrumental(
gcs_clean_instrumental_path="jobs/test/stems/clean.flac",
gcs_backing_vocals_path="jobs/test/stems/backing.flac",
mute_regions=mute_regions,
gcs_output_path="jobs/test/stems/custom.flac",
job_id="test-job",
)
# Verify upload was called with correct destination
mock_storage_service.upload_file.assert_called_once()
call_args = mock_storage_service.upload_file.call_args
assert call_args[0][1] == "jobs/test/stems/custom.flac"
# Verify result has correct path
assert result.output_path == "jobs/test/stems/custom.flac"
def test_create_custom_instrumental_returns_statistics(
self, mock_storage_service, temp_audio_files
):
"""Test that create_custom_instrumental returns correct statistics."""
from backend.services.audio_editing_service import AudioEditingService
clean_path, backing_path = temp_audio_files
def mock_download(gcs_path, local_path):
import shutil
if 'clean' in gcs_path:
shutil.copy(clean_path, local_path)
else:
shutil.copy(backing_path, local_path)
return local_path
mock_storage_service.download_file.side_effect = mock_download
mock_storage_service.upload_file.return_value = "jobs/test/stems/custom.flac"
service = AudioEditingService(storage_service=mock_storage_service)
mute_regions = [
MuteRegion(start_seconds=1.0, end_seconds=2.0),
MuteRegion(start_seconds=3.0, end_seconds=4.0),
]
result = service.create_custom_instrumental(
gcs_clean_instrumental_path="jobs/test/stems/clean.flac",
gcs_backing_vocals_path="jobs/test/stems/backing.flac",
mute_regions=mute_regions,
gcs_output_path="jobs/test/stems/custom.flac",
job_id="test-job",
)
# Verify statistics
assert len(result.mute_regions_applied) == 2
assert result.total_muted_duration_seconds == 2.0 # 1s + 1s
assert result.output_duration_seconds > 0
def test_validate_mute_regions_returns_errors_for_invalid(
self, mock_storage_service
):
"""Test that validate_mute_regions catches invalid regions."""
from backend.services.audio_editing_service import AudioEditingService
service = AudioEditingService(storage_service=mock_storage_service)
mute_regions = [
MuteRegion(start_seconds=0.0, end_seconds=1.0), # Valid
MuteRegion(start_seconds=100.0, end_seconds=110.0), # Beyond duration
]
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=60.0)
# Should have error for region exceeding duration
assert len(errors) == 1
assert "exceeds" in errors[0].lower()
def test_validate_mute_regions_returns_empty_for_valid(
self, mock_storage_service
):
"""Test that validate_mute_regions returns empty for valid regions."""
from backend.services.audio_editing_service import AudioEditingService
service = AudioEditingService(storage_service=mock_storage_service)
mute_regions = [
MuteRegion(start_seconds=0.0, end_seconds=1.0),
MuteRegion(start_seconds=10.0, end_seconds=15.0),
]
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=60.0)
assert len(errors) == 0
File diff suppressed because it is too large. Load diff
+457
View File
@@ -0,0 +1,457 @@
"""
API endpoint tests for instrumental review functionality.
Tests for:
- GET /api/jobs/{job_id}/instrumental-analysis
- GET /api/jobs/{job_id}/audio-stream/{stem_type}
- POST /api/jobs/{job_id}/create-custom-instrumental
- GET /api/jobs/{job_id}/waveform-data
"""
import pytest
from datetime import datetime
from unittest.mock import MagicMock, patch, AsyncMock
from fastapi.testclient import TestClient
from backend.models.job import Job, JobStatus
@pytest.fixture
def mock_job_manager():
"""Create a mock job manager."""
with patch('backend.api.routes.jobs.job_manager') as mock:
yield mock
@pytest.fixture
def mock_storage_service():
"""Create a mock storage service."""
with patch('backend.api.routes.jobs.StorageService') as mock:
mock_instance = MagicMock()
mock_instance.generate_signed_url.return_value = "https://storage.googleapis.com/signed-url"
mock.return_value = mock_instance
yield mock_instance
@pytest.fixture
def sample_job():
"""Create a sample job for testing."""
return Job(
job_id="test-job-123",
artist="Test Artist",
title="Test Song",
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
created_at=datetime.now(),
updated_at=datetime.now(),
file_urls={
"stems": {
"instrumental_clean": "jobs/test/stems/instrumental_clean.flac",
"backing_vocals": "jobs/test/stems/backing_vocals.flac",
"instrumental_with_backing": "jobs/test/stems/instrumental_with_backing.flac",
},
"analysis": {
"backing_vocals_waveform": "jobs/test/analysis/waveform.png",
},
},
state_data={
"backing_vocals_analysis": {
"has_audible_content": True,
"total_duration_seconds": 180.0,
"audible_segments": [
{
"start_seconds": 10.0,
"end_seconds": 20.0,
"duration_seconds": 10.0,
"avg_amplitude_db": -25.0,
},
{
"start_seconds": 60.0,
"end_seconds": 80.0,
"duration_seconds": 20.0,
"avg_amplitude_db": -30.0,
},
],
"recommended_selection": "review_needed",
"total_audible_duration_seconds": 30.0,
"audible_percentage": 16.67,
"silence_threshold_db": -40.0,
}
},
)
@pytest.fixture
def silent_job():
"""Create a job with silent backing vocals."""
return Job(
job_id="test-job-silent",
artist="Test Artist",
title="Test Song",
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
created_at=datetime.now(),
updated_at=datetime.now(),
file_urls={
"stems": {
"instrumental_clean": "jobs/test/stems/instrumental_clean.flac",
"backing_vocals": "jobs/test/stems/backing_vocals.flac",
"instrumental_with_backing": "jobs/test/stems/instrumental_with_backing.flac",
},
"analysis": {
"backing_vocals_waveform": "jobs/test/analysis/waveform.png",
},
},
state_data={
"backing_vocals_analysis": {
"has_audible_content": False,
"total_duration_seconds": 180.0,
"audible_segments": [],
"recommended_selection": "clean",
"total_audible_duration_seconds": 0.0,
"audible_percentage": 0.0,
"silence_threshold_db": -40.0,
}
},
)
class TestGetInstrumentalAnalysis:
"""Tests for GET /api/jobs/{job_id}/instrumental-analysis."""
def test_get_analysis_returns_data(self, mock_job_manager, mock_storage_service, sample_job):
"""GET /instrumental-analysis should return analysis data."""
mock_job_manager.get_job.return_value = sample_job
from backend.api.routes.jobs import get_instrumental_analysis
import asyncio
result = asyncio.get_event_loop().run_until_complete(
get_instrumental_analysis("test-job-123")
)
assert result["job_id"] == "test-job-123"
assert result["artist"] == "Test Artist"
assert result["title"] == "Test Song"
assert result["analysis"]["has_audible_content"] is True
assert len(result["analysis"]["audible_segments"]) == 2
def test_get_analysis_includes_audio_urls(self, mock_job_manager, mock_storage_service, sample_job):
"""GET /instrumental-analysis should include audio URLs."""
mock_job_manager.get_job.return_value = sample_job
from backend.api.routes.jobs import get_instrumental_analysis
import asyncio
result = asyncio.get_event_loop().run_until_complete(
get_instrumental_analysis("test-job-123")
)
assert "audio_urls" in result
assert "clean_instrumental" in result["audio_urls"]
assert "backing_vocals" in result["audio_urls"]
assert "with_backing" in result["audio_urls"]
def test_get_analysis_includes_waveform_url(self, mock_job_manager, sample_job):
"""GET /instrumental-analysis should include waveform URL."""
mock_job_manager.get_job.return_value = sample_job
# Need to mock StorageService at module level for the signed URL
with patch('backend.api.routes.jobs.StorageService') as mock_storage_cls:
mock_storage = MagicMock()
mock_storage.generate_signed_url.return_value = "https://storage.googleapis.com/signed-url"
mock_storage_cls.return_value = mock_storage
from backend.api.routes.jobs import get_instrumental_analysis
import asyncio
result = asyncio.get_event_loop().run_until_complete(
get_instrumental_analysis("test-job-123")
)
assert "waveform_url" in result
# Waveform URL should be generated from the file_urls
assert result["waveform_url"] == "https://storage.googleapis.com/signed-url"
def test_get_analysis_silent_audio(self, mock_job_manager, mock_storage_service, silent_job):
"""GET /instrumental-analysis should correctly report silent audio."""
mock_job_manager.get_job.return_value = silent_job
from backend.api.routes.jobs import get_instrumental_analysis
import asyncio
result = asyncio.get_event_loop().run_until_complete(
get_instrumental_analysis("test-job-silent")
)
assert result["analysis"]["has_audible_content"] is False
assert result["analysis"]["recommended_selection"] == "clean"
assert len(result["analysis"]["audible_segments"]) == 0
def test_get_analysis_job_not_found(self, mock_job_manager):
"""GET /instrumental-analysis should return 404 for non-existent job."""
mock_job_manager.get_job.return_value = None
from backend.api.routes.jobs import get_instrumental_analysis
from fastapi import HTTPException
import asyncio
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
get_instrumental_analysis("non-existent")
)
assert exc_info.value.status_code == 404
def test_get_analysis_wrong_status(self, mock_job_manager, sample_job):
"""GET /instrumental-analysis should return 400 for wrong job status."""
sample_job.status = JobStatus.PENDING
mock_job_manager.get_job.return_value = sample_job
from backend.api.routes.jobs import get_instrumental_analysis
from fastapi import HTTPException
import asyncio
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
get_instrumental_analysis("test-job-123")
)
assert exc_info.value.status_code == 400
class TestAudioStream:
"""Tests for GET /api/jobs/{job_id}/audio-stream/{stem_type}."""
def test_stream_backing_vocals_valid(self, mock_job_manager, sample_job):
"""Should return redirect for backing vocals stream."""
mock_job_manager.get_job.return_value = sample_job
from backend.api.routes.jobs import stream_audio
import asyncio
# This will try to stream from GCS which we can't test easily
# Instead, verify the function exists and has correct signature
assert callable(stream_audio)
def test_stream_invalid_stem_type(self, mock_job_manager, sample_job):
"""Should return 400 for invalid stem type."""
mock_job_manager.get_job.return_value = sample_job
from backend.api.routes.jobs import stream_audio
from fastapi import HTTPException
import asyncio
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
stream_audio("test-job-123", "invalid_stem")
)
assert exc_info.value.status_code == 400
def test_stream_job_not_found(self, mock_job_manager):
"""Should return 404 for non-existent job."""
mock_job_manager.get_job.return_value = None
from backend.api.routes.jobs import stream_audio
from fastapi import HTTPException
import asyncio
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
stream_audio("non-existent", "backing_vocals")
)
assert exc_info.value.status_code == 404
class TestCreateCustomInstrumental:
"""Tests for POST /api/jobs/{job_id}/create-custom-instrumental."""
@pytest.fixture
def mock_audio_editing_service(self):
"""Mock the audio editing service."""
with patch('backend.api.routes.jobs.AudioEditingService') as mock:
mock_instance = MagicMock()
mock_instance.create_custom_instrumental.return_value = MagicMock(
output_path="jobs/test/stems/custom_instrumental.flac",
mute_regions_applied=[
MagicMock(start_seconds=10.0, end_seconds=20.0),
],
total_muted_duration_seconds=10.0,
output_duration_seconds=180.0,
)
mock.return_value = mock_instance
yield mock_instance
def test_create_custom_validates_mute_regions(self):
"""Should validate mute regions at model level."""
from backend.models.requests import CreateCustomInstrumentalRequest, MuteRegionRequest
from pydantic import ValidationError
# Empty mute regions should fail at model validation
with pytest.raises(ValidationError) as exc_info:
CreateCustomInstrumentalRequest(mute_regions=[])
assert "At least one mute region is required" in str(exc_info.value)
def test_create_custom_job_not_found(self, mock_job_manager):
"""Should return 404 for non-existent job."""
mock_job_manager.get_job.return_value = None
from backend.api.routes.jobs import create_custom_instrumental
from backend.models.requests import CreateCustomInstrumentalRequest, MuteRegionRequest
from fastapi import HTTPException
import asyncio
request = CreateCustomInstrumentalRequest(
mute_regions=[MuteRegionRequest(start_seconds=10.0, end_seconds=20.0)]
)
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
create_custom_instrumental("non-existent", request)
)
assert exc_info.value.status_code == 404
def test_create_custom_wrong_status(self, mock_job_manager, sample_job):
"""Should return 400 for wrong job status."""
sample_job.status = JobStatus.PENDING
mock_job_manager.get_job.return_value = sample_job
from backend.api.routes.jobs import create_custom_instrumental
from backend.models.requests import CreateCustomInstrumentalRequest, MuteRegionRequest
from fastapi import HTTPException
import asyncio
request = CreateCustomInstrumentalRequest(
mute_regions=[MuteRegionRequest(start_seconds=10.0, end_seconds=20.0)]
)
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
create_custom_instrumental("test-job-123", request)
)
assert exc_info.value.status_code == 400
class TestGetWaveformData:
"""Tests for GET /api/jobs/{job_id}/waveform-data."""
def test_get_waveform_data_job_not_found(self, mock_job_manager):
"""Should return 404 for non-existent job."""
mock_job_manager.get_job.return_value = None
from backend.api.routes.jobs import get_waveform_data
from fastapi import HTTPException
import asyncio
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
get_waveform_data("non-existent")
)
assert exc_info.value.status_code == 404
def test_get_waveform_data_wrong_status(self, mock_job_manager, sample_job):
"""Should return 400 for wrong job status."""
sample_job.status = JobStatus.PENDING
mock_job_manager.get_job.return_value = sample_job
from backend.api.routes.jobs import get_waveform_data
from fastapi import HTTPException
import asyncio
with pytest.raises(HTTPException) as exc_info:
asyncio.get_event_loop().run_until_complete(
get_waveform_data("test-job-123")
)
assert exc_info.value.status_code == 400
def test_waveform_data_endpoint_exists(self):
"""Verify the endpoint function exists and is callable."""
from backend.api.routes.jobs import get_waveform_data
assert callable(get_waveform_data)
# Verify it's an async function
import inspect
assert inspect.iscoroutinefunction(get_waveform_data)
class TestRequestModels:
"""Tests for API request models."""
def test_mute_region_request_valid(self):
"""MuteRegionRequest should accept valid values."""
from backend.models.requests import MuteRegionRequest
region = MuteRegionRequest(start_seconds=10.0, end_seconds=20.0)
assert region.start_seconds == 10.0
assert region.end_seconds == 20.0
def test_mute_region_request_invalid_start(self):
"""MuteRegionRequest should reject negative start."""
from backend.models.requests import MuteRegionRequest
from pydantic import ValidationError
with pytest.raises(ValidationError):
MuteRegionRequest(start_seconds=-1.0, end_seconds=20.0)
def test_mute_region_request_invalid_order(self):
"""MuteRegionRequest should reject end before start."""
from backend.models.requests import MuteRegionRequest
from pydantic import ValidationError
with pytest.raises(ValidationError):
MuteRegionRequest(start_seconds=20.0, end_seconds=10.0)
def test_create_custom_instrumental_request_valid(self):
"""CreateCustomInstrumentalRequest should accept valid mute regions."""
from backend.models.requests import (
CreateCustomInstrumentalRequest,
MuteRegionRequest,
)
request = CreateCustomInstrumentalRequest(
mute_regions=[
MuteRegionRequest(start_seconds=10.0, end_seconds=20.0),
MuteRegionRequest(start_seconds=60.0, end_seconds=80.0),
]
)
assert len(request.mute_regions) == 2
class TestInstrumentalSelectionExtension:
"""Tests for extended instrumental selection (including 'custom')."""
def test_instrumental_selection_accepts_custom(self):
"""InstrumentalSelection should accept 'custom' as valid selection."""
from backend.models.requests import InstrumentalSelection
selection = InstrumentalSelection(selection="custom")
assert selection.selection == "custom"
def test_instrumental_selection_accepts_clean(self):
"""InstrumentalSelection should accept 'clean' as valid selection."""
from backend.models.requests import InstrumentalSelection
selection = InstrumentalSelection(selection="clean")
assert selection.selection == "clean"
def test_instrumental_selection_accepts_with_backing(self):
"""InstrumentalSelection should accept 'with_backing' as valid selection."""
from backend.models.requests import InstrumentalSelection
selection = InstrumentalSelection(selection="with_backing")
assert selection.selection == "with_backing"
def test_instrumental_selection_rejects_invalid(self):
"""InstrumentalSelection should reject invalid values."""
from backend.models.requests import InstrumentalSelection
from pydantic import ValidationError
with pytest.raises(ValidationError):
InstrumentalSelection(selection="invalid_option")
+144 -1
View File
@@ -3,6 +3,9 @@ Unit tests for internal.py API routes using FastAPI TestClient.
Internal routes are used for worker communication within Cloud Run.
These endpoints require admin authentication.
With Cloud Tasks integration, these endpoints include idempotency checks
to prevent duplicate processing on retries.
"""
import os
import pytest
@@ -17,9 +20,20 @@ from backend.models.job import Job, JobStatus
os.environ.setdefault('ADMIN_TOKENS', 'test-admin-token')
def _create_mock_job(job_id: str = "test123") -> Job:
"""Create a mock Job object for testing."""
return Job(
job_id=job_id,
status=JobStatus.PROCESSING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
state_data={}, # Empty state_data - no worker progress yet
)
@pytest.fixture
def client():
"""Create TestClient with mocked workers."""
"""Create TestClient with mocked workers and job manager."""
mock_creds = MagicMock()
mock_creds.universe_domain = 'googleapis.com'
@@ -28,11 +42,19 @@ def client():
mock_lyrics = AsyncMock()
mock_screens = AsyncMock()
mock_video = AsyncMock()
mock_render = AsyncMock()
# Mock JobManager for idempotency checks
mock_job_manager = MagicMock()
mock_job_manager.get_job.return_value = _create_mock_job()
mock_job_manager.update_state_data.return_value = None
with patch('backend.api.routes.internal.process_audio_separation', mock_audio), \
patch('backend.api.routes.internal.process_lyrics_transcription', mock_lyrics), \
patch('backend.api.routes.internal.generate_screens', mock_screens), \
patch('backend.api.routes.internal.generate_video', mock_video), \
patch('backend.api.routes.internal.process_render_video', mock_render), \
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
patch('backend.services.firestore_service.firestore'), \
patch('backend.services.storage_service.storage'), \
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
@@ -175,6 +197,36 @@ class TestVideoWorkerEndpoint:
assert response.status_code == 401
class TestRenderVideoWorkerEndpoint:
"""Tests for POST /api/internal/workers/render-video."""
def test_render_video_worker_returns_200(self, client, auth_headers):
"""Test render-video worker endpoint returns 200 with auth."""
response = client.post(
"/api/internal/workers/render-video",
headers=auth_headers,
json={"job_id": "test123"}
)
assert response.status_code == 200
def test_render_video_worker_requires_job_id(self, client, auth_headers):
"""Test render-video worker requires job_id."""
response = client.post(
"/api/internal/workers/render-video",
headers=auth_headers,
json={}
)
assert response.status_code == 422
def test_render_video_worker_requires_auth(self, client):
"""Test render-video worker requires authentication."""
response = client.post(
"/api/internal/workers/render-video",
json={"job_id": "test123"}
)
assert response.status_code == 401
class TestWorkerResponseFormat:
"""Tests for worker response format."""
@@ -198,3 +250,94 @@ class TestWorkerResponseFormat:
data = response.json()
assert "status" in data or "message" in data
class TestWorkerIdempotency:
"""Tests for worker idempotency checks (Cloud Tasks retry support)."""
def test_audio_worker_returns_already_running_when_in_progress(self, auth_headers):
"""Test audio worker returns already_running if worker is already running."""
mock_creds = MagicMock()
mock_creds.universe_domain = 'googleapis.com'
# Create job with audio already running
mock_job = _create_mock_job()
mock_job.state_data = {'audio_progress': {'stage': 'running'}}
mock_job_manager = MagicMock()
mock_job_manager.get_job.return_value = mock_job
with patch('backend.api.routes.internal.process_audio_separation', AsyncMock()), \
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
patch('backend.services.firestore_service.firestore'), \
patch('backend.services.storage_service.storage'), \
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
from backend.main import app
client = TestClient(app)
response = client.post(
"/api/internal/workers/audio",
headers=auth_headers,
json={"job_id": "test123"}
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "already_running"
def test_audio_worker_returns_already_complete_when_finished(self, auth_headers):
"""Test audio worker returns already_complete if worker already finished."""
mock_creds = MagicMock()
mock_creds.universe_domain = 'googleapis.com'
# Create job with audio already complete
mock_job = _create_mock_job()
mock_job.state_data = {'audio_progress': {'stage': 'complete'}}
mock_job_manager = MagicMock()
mock_job_manager.get_job.return_value = mock_job
with patch('backend.api.routes.internal.process_audio_separation', AsyncMock()), \
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
patch('backend.services.firestore_service.firestore'), \
patch('backend.services.storage_service.storage'), \
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
from backend.main import app
client = TestClient(app)
response = client.post(
"/api/internal/workers/audio",
headers=auth_headers,
json={"job_id": "test123"}
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "already_complete"
def test_audio_worker_returns_not_found_when_job_missing(self, auth_headers):
"""Test audio worker returns not_found if job doesn't exist."""
mock_creds = MagicMock()
mock_creds.universe_domain = 'googleapis.com'
# Return None for job not found
mock_job_manager = MagicMock()
mock_job_manager.get_job.return_value = None
with patch('backend.api.routes.internal.process_audio_separation', AsyncMock()), \
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
patch('backend.services.firestore_service.firestore'), \
patch('backend.services.storage_service.storage'), \
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
from backend.main import app
client = TestClient(app)
response = client.post(
"/api/internal/workers/audio",
headers=auth_headers,
json={"job_id": "nonexistent"}
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "not_found"
+336
View File
@@ -577,6 +577,342 @@ class TestJobModelIntegration:
assert restored_job.state_data["lyrics_metadata"]["line_count"] == 50
class TestExistingInstrumentalModelFields:
"""Test existing instrumental field in Job and JobCreate models (Batch 3)."""
def test_job_has_existing_instrumental_gcs_path(self):
"""Test that Job model has existing_instrumental_gcs_path field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
existing_instrumental_gcs_path="uploads/test123/audio/existing_instrumental.flac"
)
assert hasattr(job, 'existing_instrumental_gcs_path')
assert job.existing_instrumental_gcs_path == "uploads/test123/audio/existing_instrumental.flac"
def test_job_existing_instrumental_defaults_to_none(self):
"""Test that existing_instrumental_gcs_path defaults to None."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.existing_instrumental_gcs_path is None
def test_job_create_has_existing_instrumental_gcs_path(self):
"""Test that JobCreate model has existing_instrumental_gcs_path field."""
from backend.models.job import JobCreate
job_create = JobCreate(
artist="Artist",
title="Title",
existing_instrumental_gcs_path="uploads/test/audio/existing_instrumental.mp3"
)
assert job_create.existing_instrumental_gcs_path == "uploads/test/audio/existing_instrumental.mp3"
def test_job_create_existing_instrumental_optional(self):
"""Test that existing_instrumental_gcs_path is optional in JobCreate."""
from backend.models.job import JobCreate
job_create = JobCreate(
artist="Artist",
title="Title"
)
assert job_create.existing_instrumental_gcs_path is None
def test_existing_instrumental_roundtrip_serialization(self):
"""Test that existing_instrumental_gcs_path survives serialization/deserialization."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
existing_instrumental_gcs_path="uploads/test123/audio/existing_instrumental.wav"
)
# Serialize to dict (simulates writing to Firestore)
job_dict = job.model_dump()
# Convert to JSON and back (simulates Firestore storage)
json_str = json.dumps(job_dict, default=str)
restored_dict = json.loads(json_str)
# Deserialize back to Job (simulates reading from Firestore)
restored_job = Job(**restored_dict)
assert restored_job.existing_instrumental_gcs_path == "uploads/test123/audio/existing_instrumental.wav"
def test_job_with_full_audio_config(self):
"""Test Job with complete audio configuration including existing instrumental."""
job = Job(
job_id="test123",
status=JobStatus.AUDIO_COMPLETE,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
# Audio model configuration
clean_instrumental_model="model_bs_roformer_ep_317_sdr_12.9755.ckpt",
backing_vocals_models=["mel_band_roformer_karaoke.ckpt"],
other_stems_models=["htdemucs_6s.yaml"],
# Existing instrumental
existing_instrumental_gcs_path="uploads/test123/audio/my_instrumental.flac",
)
assert job.clean_instrumental_model == "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
assert job.existing_instrumental_gcs_path == "uploads/test123/audio/my_instrumental.flac"
class TestTwoPhaseWorkflowFields:
"""Test two-phase workflow fields in Job and JobCreate models (Batch 6)."""
def test_job_has_prep_only_field(self):
"""Test that Job model has prep_only field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
prep_only=True
)
assert hasattr(job, 'prep_only')
assert job.prep_only is True
def test_job_prep_only_defaults_to_false(self):
"""Test that prep_only defaults to False."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.prep_only is False
def test_job_has_finalise_only_field(self):
"""Test that Job model has finalise_only field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
finalise_only=True
)
assert hasattr(job, 'finalise_only')
assert job.finalise_only is True
def test_job_finalise_only_defaults_to_false(self):
"""Test that finalise_only defaults to False."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.finalise_only is False
def test_job_has_keep_brand_code_field(self):
"""Test that Job model has keep_brand_code field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
keep_brand_code="NOMAD-1234"
)
assert hasattr(job, 'keep_brand_code')
assert job.keep_brand_code == "NOMAD-1234"
def test_job_keep_brand_code_defaults_to_none(self):
"""Test that keep_brand_code defaults to None."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.keep_brand_code is None
def test_job_create_has_two_phase_workflow_fields(self):
"""Test that JobCreate model has two-phase workflow fields."""
job_create = JobCreate(
artist="Artist",
title="Title",
prep_only=True,
finalise_only=False,
keep_brand_code="BRAND-0001"
)
assert job_create.prep_only is True
assert job_create.finalise_only is False
assert job_create.keep_brand_code == "BRAND-0001"
def test_two_phase_workflow_roundtrip_serialization(self):
"""Test that two-phase workflow fields survive serialization/deserialization."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
prep_only=True,
finalise_only=False,
keep_brand_code="NOMAD-5678"
)
# Serialize to dict (simulates writing to Firestore)
job_dict = job.model_dump()
# Convert to JSON and back (simulates Firestore storage)
json_str = json.dumps(job_dict, default=str)
restored_dict = json.loads(json_str)
# Deserialize back to Job (simulates reading from Firestore)
restored_job = Job(**restored_dict)
assert restored_job.prep_only is True
assert restored_job.finalise_only is False
assert restored_job.keep_brand_code == "NOMAD-5678"
class TestPrepCompleteStatus:
"""Test PREP_COMPLETE status and state transitions (Batch 6)."""
def test_prep_complete_status_exists(self):
"""Test that PREP_COMPLETE status exists in JobStatus enum."""
assert hasattr(JobStatus, 'PREP_COMPLETE')
assert JobStatus.PREP_COMPLETE.value == "prep_complete"
def test_job_can_have_prep_complete_status(self):
"""Test that Job can be created with PREP_COMPLETE status."""
job = Job(
job_id="test123",
status=JobStatus.PREP_COMPLETE,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.status == JobStatus.PREP_COMPLETE
def test_state_transitions_review_complete_to_prep_complete(self):
"""Test that REVIEW_COMPLETE can transition to PREP_COMPLETE."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.REVIEW_COMPLETE, [])
assert JobStatus.PREP_COMPLETE in valid_transitions
def test_state_transitions_prep_complete_to_awaiting_instrumental(self):
"""Test that PREP_COMPLETE can transition to AWAITING_INSTRUMENTAL_SELECTION."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PREP_COMPLETE, [])
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
def test_state_transitions_pending_to_awaiting_instrumental(self):
"""Test that PENDING can transition to AWAITING_INSTRUMENTAL_SELECTION (for finalise-only)."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PENDING, [])
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
def test_critical_statuses_include_prep_complete(self):
"""Test that PREP_COMPLETE is in the list of job statuses."""
all_statuses = [status.value for status in JobStatus]
assert "prep_complete" in all_statuses
class TestJobWithFullTwoPhaseConfig:
"""Integration tests for two-phase workflow job configuration."""
def test_prep_only_job_full_config(self):
"""Test creating a prep-only job with full configuration."""
job = Job(
job_id="prep123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
prep_only=True,
enable_cdg=True,
enable_txt=True,
brand_prefix="NOMAD",
)
assert job.prep_only is True
assert job.finalise_only is False
assert job.brand_prefix == "NOMAD"
def test_finalise_only_job_full_config(self):
"""Test creating a finalise-only job with full configuration."""
job = Job(
job_id="final123",
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
finalise_only=True,
keep_brand_code="NOMAD-1234",
enable_youtube_upload=True,
dropbox_path="/Karaoke/Tracks",
state_data={
"audio_complete": True,
"lyrics_complete": True,
"finalise_only": True,
}
)
assert job.finalise_only is True
assert job.keep_brand_code == "NOMAD-1234"
assert job.state_data["finalise_only"] is True
def test_prep_complete_job_with_file_urls(self):
"""Test a job that has reached PREP_COMPLETE status with all prep outputs."""
job = Job(
job_id="prepcomp123",
status=JobStatus.PREP_COMPLETE,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="ABBA",
title="Waterloo",
prep_only=True,
file_urls={
"stems": {
"instrumental_clean": "gs://bucket/jobs/prepcomp123/stems/instrumental_clean.flac",
"instrumental_with_backing": "gs://bucket/jobs/prepcomp123/stems/instrumental_with_backing.flac",
},
"videos": {
"with_vocals": "gs://bucket/jobs/prepcomp123/videos/with_vocals.mkv",
},
"screens": {
"title": "gs://bucket/jobs/prepcomp123/screens/title.mov",
"end": "gs://bucket/jobs/prepcomp123/screens/end.mov",
},
"lyrics": {
"lrc": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.lrc",
"ass": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.ass",
},
}
)
assert job.status == JobStatus.PREP_COMPLETE
assert job.prep_only is True
assert "with_vocals" in job.file_urls["videos"]
assert "title" in job.file_urls["screens"]
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+10
View File
@@ -62,6 +62,16 @@ class TestJobStatusTransitions:
if status not in excluded:
valid_transitions = STATE_TRANSITIONS.get(status, [])
assert JobStatus.FAILED in valid_transitions, f"{status} should be able to fail"
def test_failed_can_transition_for_retry(self):
"""Test FAILED status can transition to retry checkpoint states."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.FAILED, [])
# FAILED should allow retry transitions
assert JobStatus.INSTRUMENTAL_SELECTED in valid_transitions, "FAILED should allow retry to INSTRUMENTAL_SELECTED"
assert JobStatus.REVIEW_COMPLETE in valid_transitions, "FAILED should allow retry to REVIEW_COMPLETE"
assert JobStatus.LYRICS_COMPLETE in valid_transitions, "FAILED should allow retry to LYRICS_COMPLETE"
class TestJobModelSerialization:
+185
View File
@@ -12,6 +12,7 @@ from datetime import datetime, UTC
import sys
sys.modules['google.cloud.firestore'] = MagicMock()
sys.modules['google.cloud.storage'] = MagicMock()
sys.modules['google.cloud.tasks_v2'] = MagicMock()
from backend.services.auth_service import AuthService, UserType
from backend.services.storage_service import StorageService
@@ -239,6 +240,190 @@ class TestWorkerService:
assert "/api/internal/workers/video" in str(call_args)
class TestWorkerServiceCloudTasks:
"""Test WorkerService Cloud Tasks integration."""
def test_should_use_cloud_tasks_default_false(self):
"""Test that Cloud Tasks is disabled by default."""
from backend.services.worker_service import WorkerService, reset_worker_service
reset_worker_service()
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
mock_getenv.side_effect = lambda k, d=None: {
'PORT': '8000',
}.get(k, d)
with patch('backend.services.worker_service.get_settings') as mock_settings:
mock_settings.return_value.admin_tokens = None
mock_settings.return_value.google_cloud_project = "test-project"
mock_settings.return_value.enable_cloud_tasks = False
mock_settings.return_value.gcp_region = "us-central1"
mock_settings.return_value.use_cloud_run_jobs_for_video = False
service = WorkerService()
assert service._use_cloud_tasks is False
def test_should_use_cloud_tasks_enabled(self):
"""Test that Cloud Tasks can be enabled via settings."""
from backend.services.worker_service import WorkerService, reset_worker_service
reset_worker_service()
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
mock_getenv.side_effect = lambda k, d=None: {
'PORT': '8000',
}.get(k, d)
with patch('backend.services.worker_service.get_settings') as mock_settings:
mock_settings.return_value.admin_tokens = None
mock_settings.return_value.google_cloud_project = "test-project"
mock_settings.return_value.enable_cloud_tasks = True
mock_settings.return_value.gcp_region = "us-central1"
mock_settings.return_value.use_cloud_run_jobs_for_video = False
service = WorkerService()
assert service._use_cloud_tasks is True
def test_worker_queues_mapping(self):
"""Test that all worker types have queue mappings."""
from backend.services.worker_service import WORKER_QUEUES
# Verify all expected workers have queues
assert "audio" in WORKER_QUEUES
assert "lyrics" in WORKER_QUEUES
assert "screens" in WORKER_QUEUES
assert "render-video" in WORKER_QUEUES
assert "video" in WORKER_QUEUES
# Verify queue names are correct
assert WORKER_QUEUES["audio"] == "audio-worker-queue"
assert WORKER_QUEUES["lyrics"] == "lyrics-worker-queue"
assert WORKER_QUEUES["screens"] == "screens-worker-queue"
assert WORKER_QUEUES["render-video"] == "render-worker-queue"
assert WORKER_QUEUES["video"] == "video-worker-queue"
@pytest.mark.asyncio
async def test_trigger_worker_uses_http_when_cloud_tasks_disabled(self):
"""Test that trigger_worker uses HTTP when Cloud Tasks is disabled."""
from backend.services.worker_service import WorkerService, reset_worker_service
reset_worker_service()
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
mock_getenv.side_effect = lambda k, d=None: {
'PORT': '8000',
}.get(k, d)
with patch('backend.services.worker_service.get_settings') as mock_settings:
mock_settings.return_value.admin_tokens = "test-token"
mock_settings.return_value.google_cloud_project = "test-project"
mock_settings.return_value.enable_cloud_tasks = False
mock_settings.return_value.gcp_region = "us-central1"
mock_settings.return_value.use_cloud_run_jobs_for_video = False
with patch('backend.services.worker_service.httpx.AsyncClient') as mock_client_cls:
mock_client = AsyncMock()
mock_client_cls.return_value.__aenter__.return_value = mock_client
mock_response = Mock()
mock_response.status_code = 200
mock_client.post.return_value = mock_response
service = WorkerService()
assert service._use_cloud_tasks is False
result = await service.trigger_worker("audio", "test-job-123")
# Verify HTTP was used
assert result is True
mock_client.post.assert_called_once()
@pytest.mark.asyncio
async def test_trigger_worker_uses_cloud_tasks_when_enabled(self):
"""Test that trigger_worker uses Cloud Tasks when enabled."""
from backend.services.worker_service import WorkerService, reset_worker_service
reset_worker_service()
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
mock_getenv.side_effect = lambda k, d=None: {
'CLOUD_RUN_SERVICE_URL': 'https://api.example.com',
}.get(k, d)
with patch('backend.services.worker_service.get_settings') as mock_settings:
mock_settings.return_value.admin_tokens = "test-token"
mock_settings.return_value.google_cloud_project = "test-project"
mock_settings.return_value.enable_cloud_tasks = True
mock_settings.return_value.gcp_region = "us-central1"
mock_settings.return_value.use_cloud_run_jobs_for_video = False
# Mock Cloud Tasks client
mock_tasks_client = MagicMock()
mock_tasks_client.queue_path.return_value = "projects/test-project/locations/us-central1/queues/audio-worker-queue"
mock_task_response = MagicMock()
mock_task_response.name = "projects/test-project/locations/us-central1/queues/audio-worker-queue/tasks/abc123"
mock_tasks_client.create_task.return_value = mock_task_response
# Mock the google.cloud.tasks_v2 module which is imported inside the method
mock_tasks_module = MagicMock()
mock_tasks_module.CloudTasksClient.return_value = mock_tasks_client
mock_tasks_module.HttpMethod.POST = "POST"
with patch.dict('sys.modules', {'google.cloud.tasks_v2': mock_tasks_module}):
with patch.dict(sys.modules, {'google.cloud': MagicMock()}):
service = WorkerService()
service._tasks_client = mock_tasks_client # Inject mocked client
result = await service.trigger_worker("audio", "test-job-123")
# Verify Cloud Tasks was used
assert result is True
mock_tasks_client.create_task.assert_called_once()
# Verify task payload
call_args = mock_tasks_client.create_task.call_args
assert call_args is not None
@pytest.mark.asyncio
async def test_trigger_worker_returns_false_for_unknown_worker_type(self):
"""Test that trigger_worker returns False for unknown worker types."""
from backend.services.worker_service import WorkerService, reset_worker_service
reset_worker_service()
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
mock_getenv.side_effect = lambda k, d=None: {
'CLOUD_RUN_SERVICE_URL': 'https://api.example.com',
}.get(k, d)
with patch('backend.services.worker_service.get_settings') as mock_settings:
mock_settings.return_value.admin_tokens = "test-token"
mock_settings.return_value.google_cloud_project = "test-project"
mock_settings.return_value.enable_cloud_tasks = True
mock_settings.return_value.gcp_region = "us-central1"
mock_settings.return_value.use_cloud_run_jobs_for_video = False
service = WorkerService()
result = await service.trigger_worker("unknown-worker", "test-job-123")
assert result is False
def test_reset_worker_service(self):
"""Test that reset_worker_service resets the singleton."""
from backend.services.worker_service import get_worker_service, reset_worker_service
with patch('backend.services.worker_service.get_settings') as mock_settings:
mock_settings.return_value.admin_tokens = None
mock_settings.return_value.google_cloud_project = "test-project"
mock_settings.return_value.enable_cloud_tasks = False
mock_settings.return_value.gcp_region = "us-central1"
mock_settings.return_value.use_cloud_run_jobs_for_video = False
service1 = get_worker_service()
reset_worker_service()
service2 = get_worker_service()
# After reset, should be different instances
assert service1 is not service2
class TestFirestoreService:
"""Test FirestoreService database operations."""
+156
View File
@@ -0,0 +1,156 @@
"""
Unit tests for backend validation script (validate.py).
Tests the validation functions that check for import errors, syntax errors,
configuration issues, and FastAPI app creation.
"""
import pytest
from unittest.mock import patch, MagicMock
class TestValidateImports:
"""Tests for the validate_imports function."""
def test_validate_imports_success(self):
"""Test that validate_imports succeeds when all modules import."""
from backend.validate import validate_imports
# This should succeed since we're in a valid test environment
result = validate_imports()
assert result is True
def test_validate_imports_with_failure(self):
"""Test that validate_imports handles import failures."""
from backend.validate import validate_imports
# Mock importlib to simulate a failure
with patch('backend.validate.importlib.import_module') as mock_import:
mock_import.side_effect = ImportError("Module not found")
result = validate_imports()
assert result is False
class TestValidateSyntax:
"""Tests for the validate_syntax function."""
def test_validate_syntax_success(self):
"""Test that validate_syntax succeeds with valid Python files."""
from backend.validate import validate_syntax
# This should succeed since the backend code is valid
result = validate_syntax()
assert result is True
def test_validate_syntax_handles_invalid_file(self):
"""Test that validate_syntax detects syntax errors."""
import tempfile
import os
from pathlib import Path
from backend.validate import validate_syntax
# We can't easily inject invalid files into the backend dir,
# but we can verify the function runs and returns True for valid files
result = validate_syntax()
assert result is True
class TestValidateConfig:
"""Tests for the validate_config function."""
def test_validate_config_success(self):
"""Test that validate_config succeeds with valid configuration."""
from backend.validate import validate_config
# This should succeed in test environment
result = validate_config()
assert result is True
def test_validate_config_failure(self):
"""Test that validate_config handles configuration errors."""
from backend.validate import validate_config
# Patch at the source module
with patch('backend.config.get_settings') as mock_settings:
mock_settings.side_effect = Exception("Config error")
result = validate_config()
assert result is False
class TestValidateFastapiApp:
"""Tests for the validate_fastapi_app function."""
def test_validate_fastapi_app_success(self):
"""Test that validate_fastapi_app succeeds."""
from backend.validate import validate_fastapi_app
# This should succeed since the FastAPI app is valid
result = validate_fastapi_app()
assert result is True
def test_validate_fastapi_app_failure(self):
"""Test that validate_fastapi_app handles app creation errors."""
from backend.validate import validate_fastapi_app
with patch.dict('sys.modules', {'backend.main': MagicMock(app=MagicMock(title="Test", version="1.0", routes=[]))}):
# Even with a mock, the function should work
result = validate_fastapi_app()
# The original module is still accessible, so this should still pass
assert result is True
class TestMain:
"""Tests for the main function."""
def test_main_runs(self):
"""Test that main runs without errors."""
from backend.validate import main
# In a valid test environment, main should return 0 (success)
result = main()
assert result == 0
def test_main_returns_1_on_failure(self):
"""Test that main returns 1 when validations fail."""
from backend.validate import main
# Mock one validation to fail
with patch('backend.validate.validate_imports', return_value=False):
result = main()
assert result == 1
class TestValidateModule:
"""Test the validate module can be imported and has expected functions."""
def test_module_imports(self):
"""Test that the validate module can be imported."""
import backend.validate
assert backend.validate is not None
def test_has_validate_imports(self):
"""Test that validate_imports function exists."""
from backend.validate import validate_imports
assert callable(validate_imports)
def test_has_validate_syntax(self):
"""Test that validate_syntax function exists."""
from backend.validate import validate_syntax
assert callable(validate_syntax)
def test_has_validate_config(self):
"""Test that validate_config function exists."""
from backend.validate import validate_config
assert callable(validate_config)
def test_has_validate_fastapi_app(self):
"""Test that validate_fastapi_app function exists."""
from backend.validate import validate_fastapi_app
assert callable(validate_fastapi_app)
def test_has_main(self):
"""Test that main function exists."""
from backend.validate import main
assert callable(main)
+365
View File
@@ -0,0 +1,365 @@
"""
Unit tests for backend/workers/worker_logging.py.
Tests the job logging utilities including JobLogger, JobLogHandler,
and job_logging_context for concurrent job isolation.
"""
import pytest
import logging
from unittest.mock import MagicMock, patch
class TestJobLoggingContext:
"""Tests for job_logging_context context manager."""
def test_job_logging_context_sets_and_resets_job_id(self):
"""Test that job_logging_context sets and resets the current job ID."""
from backend.workers.worker_logging import job_logging_context, _current_job_id
# Initially no job
assert _current_job_id.get() is None
# Inside context, job is set
with job_logging_context("job123"):
assert _current_job_id.get() == "job123"
# After context, job is reset
assert _current_job_id.get() is None
def test_job_logging_context_nested(self):
"""Test nested job_logging_context calls."""
from backend.workers.worker_logging import job_logging_context, _current_job_id
with job_logging_context("outer_job"):
assert _current_job_id.get() == "outer_job"
with job_logging_context("inner_job"):
assert _current_job_id.get() == "inner_job"
# After inner context, outer job is restored
assert _current_job_id.get() == "outer_job"
assert _current_job_id.get() is None
def test_job_logging_context_handles_exception(self):
"""Test that job_logging_context resets even on exception."""
from backend.workers.worker_logging import job_logging_context, _current_job_id
try:
with job_logging_context("job123"):
assert _current_job_id.get() == "job123"
raise ValueError("Test exception")
except ValueError:
pass
# Job should still be reset after exception
assert _current_job_id.get() is None
class TestJobLogHandler:
"""Tests for JobLogHandler class."""
@pytest.fixture
def mock_job_manager(self):
"""Create a mock JobManager."""
manager = MagicMock()
manager.append_worker_log.return_value = None
return manager
def test_job_log_handler_init(self, mock_job_manager):
"""Test JobLogHandler initialization."""
from backend.workers.worker_logging import JobLogHandler
handler = JobLogHandler(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager
)
assert handler.job_id == "job123"
assert handler.worker_name == "audio"
assert handler.job_manager == mock_job_manager
assert handler.level == logging.INFO
def test_job_log_handler_custom_level(self, mock_job_manager):
"""Test JobLogHandler with custom level."""
from backend.workers.worker_logging import JobLogHandler
handler = JobLogHandler(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager,
level=logging.DEBUG
)
assert handler.level == logging.DEBUG
def test_job_log_handler_emit_logs_to_firestore(self, mock_job_manager):
"""Test that emit() calls job_manager.append_worker_log."""
from backend.workers.worker_logging import JobLogHandler, job_logging_context
handler = JobLogHandler(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager
)
handler.setFormatter(logging.Formatter('%(message)s'))
# Create a log record
record = logging.LogRecord(
name="test",
level=logging.INFO,
pathname="",
lineno=0,
msg="Test message",
args=(),
exc_info=None
)
# Emit within the job context
with job_logging_context("job123"):
handler.emit(record)
# Should have called append_worker_log
mock_job_manager.append_worker_log.assert_called()
def test_job_log_handler_filters_other_job_context(self, mock_job_manager):
"""Test that handler filters logs from other job contexts."""
from backend.workers.worker_logging import JobLogHandler, job_logging_context
handler = JobLogHandler(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager
)
record = logging.LogRecord(
name="test",
level=logging.INFO,
pathname="",
lineno=0,
msg="Test message",
args=(),
exc_info=None
)
# Emit from a different job's context
with job_logging_context("different_job"):
handler.emit(record)
# Should NOT have called append_worker_log
mock_job_manager.append_worker_log.assert_not_called()
def test_job_log_handler_deduplication(self, mock_job_manager):
"""Test that handler deduplicates repeated messages."""
from backend.workers.worker_logging import JobLogHandler, job_logging_context
handler = JobLogHandler(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager
)
handler.setFormatter(logging.Formatter('%(message)s'))
record = logging.LogRecord(
name="test",
level=logging.INFO,
pathname="",
lineno=0,
msg="Duplicate message",
args=(),
exc_info=None
)
with job_logging_context("job123"):
# Emit the same record twice
handler.emit(record)
handler.emit(record)
# Should only be called once due to deduplication
assert mock_job_manager.append_worker_log.call_count == 1
class TestJobLogger:
"""Tests for JobLogger class."""
@pytest.fixture
def mock_job_manager(self):
"""Create a mock JobManager."""
manager = MagicMock()
manager.append_worker_log.return_value = None
return manager
def test_job_logger_init(self, mock_job_manager):
"""Test JobLogger initialization."""
from backend.workers.worker_logging import JobLogger
logger = JobLogger(
job_id="job123",
worker_name="lyrics",
job_manager=mock_job_manager
)
assert logger.job_id == "job123"
assert logger.worker_name == "lyrics"
assert logger.job_manager == mock_job_manager
def test_job_logger_info(self, mock_job_manager):
"""Test JobLogger.info() method."""
from backend.workers.worker_logging import JobLogger
logger = JobLogger(
job_id="job123",
worker_name="lyrics",
job_manager=mock_job_manager
)
logger.info("Processing started")
mock_job_manager.append_worker_log.assert_called_with(
job_id="job123",
worker="lyrics",
level="INFO",
message="Processing started"
)
def test_job_logger_warning(self, mock_job_manager):
"""Test JobLogger.warning() method."""
from backend.workers.worker_logging import JobLogger
logger = JobLogger(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager
)
logger.warning("Low memory")
mock_job_manager.append_worker_log.assert_called_with(
job_id="job123",
worker="audio",
level="WARNING",
message="Low memory"
)
def test_job_logger_error(self, mock_job_manager):
"""Test JobLogger.error() method."""
from backend.workers.worker_logging import JobLogger
logger = JobLogger(
job_id="job123",
worker_name="video",
job_manager=mock_job_manager
)
logger.error("Processing failed")
mock_job_manager.append_worker_log.assert_called_with(
job_id="job123",
worker="video",
level="ERROR",
message="Processing failed"
)
def test_job_logger_debug(self, mock_job_manager):
"""Test JobLogger.debug() method."""
from backend.workers.worker_logging import JobLogger
logger = JobLogger(
job_id="job123",
worker_name="screens",
job_manager=mock_job_manager
)
logger.debug("Debug info")
mock_job_manager.append_worker_log.assert_called_with(
job_id="job123",
worker="screens",
level="DEBUG",
message="Debug info"
)
def test_job_logger_with_format_args(self, mock_job_manager):
"""Test JobLogger with format arguments."""
from backend.workers.worker_logging import JobLogger
logger = JobLogger(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager
)
logger.info("Processing %s of %d", "audio", 10)
mock_job_manager.append_worker_log.assert_called_with(
job_id="job123",
worker="audio",
level="INFO",
message="Processing audio of 10"
)
def test_job_logger_handles_firestore_error(self, mock_job_manager):
"""Test that JobLogger handles Firestore errors gracefully."""
from backend.workers.worker_logging import JobLogger
mock_job_manager.append_worker_log.side_effect = Exception("Firestore error")
logger = JobLogger(
job_id="job123",
worker_name="audio",
job_manager=mock_job_manager
)
# Should not raise exception
logger.info("Test message")
class TestCreateJobLogger:
"""Tests for create_job_logger function."""
def test_create_job_logger(self):
"""Test create_job_logger creates a JobLogger."""
from backend.workers.worker_logging import create_job_logger, JobLogger
# Patch at the source module where JobManager is imported
with patch('backend.services.job_manager.JobManager'):
logger = create_job_logger("job123", "audio")
assert isinstance(logger, JobLogger)
assert logger.job_id == "job123"
assert logger.worker_name == "audio"
class TestSetupJobLogging:
"""Tests for setup_job_logging function."""
def test_setup_job_logging_returns_handler(self):
"""Test setup_job_logging returns a JobLogHandler."""
from backend.workers.worker_logging import setup_job_logging, JobLogHandler
# Patch at the source module
with patch('backend.services.job_manager.JobManager'):
handler = setup_job_logging("job123", "lyrics", "test_logger_wl1")
assert isinstance(handler, JobLogHandler)
assert handler.job_id == "job123"
assert handler.worker_name == "lyrics"
# Clean up - remove the handler
logging.getLogger("test_logger_wl1").removeHandler(handler)
def test_setup_job_logging_adds_handler_to_loggers(self):
"""Test setup_job_logging adds handler to specified loggers."""
from backend.workers.worker_logging import setup_job_logging
with patch('backend.services.job_manager.JobManager'):
handler = setup_job_logging("job123", "audio", "test_logger_wl2", "test_logger_wl3")
# Check handlers are added
assert handler in logging.getLogger("test_logger_wl2").handlers
assert handler in logging.getLogger("test_logger_wl3").handlers
# Clean up
logging.getLogger("test_logger_wl2").removeHandler(handler)
logging.getLogger("test_logger_wl3").removeHandler(handler)
+541
View File
@@ -353,6 +353,108 @@ class TestLyricsWorker:
mock_job_manager.mark_job_failed.assert_called()
class TestLyricsWorkerConfiguration:
"""Tests for lyrics worker configuration parameters."""
def test_create_lyrics_processor_with_defaults(self):
"""Test creating LyricsProcessor with default parameters."""
with patch('backend.workers.lyrics_worker.LyricsProcessor') as mock_processor:
from backend.workers.lyrics_worker import create_lyrics_processor
result = create_lyrics_processor()
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
assert call_kwargs['lyrics_file'] is None
assert call_kwargs['subtitle_offset_ms'] == 0
def test_create_lyrics_processor_with_lyrics_file(self):
"""Test creating LyricsProcessor with custom lyrics file."""
with patch('backend.workers.lyrics_worker.LyricsProcessor') as mock_processor:
from backend.workers.lyrics_worker import create_lyrics_processor
result = create_lyrics_processor(
lyrics_file="/path/to/lyrics.txt",
subtitle_offset_ms=0
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
assert call_kwargs['lyrics_file'] == "/path/to/lyrics.txt"
def test_create_lyrics_processor_with_subtitle_offset(self):
"""Test creating LyricsProcessor with subtitle offset."""
with patch('backend.workers.lyrics_worker.LyricsProcessor') as mock_processor:
from backend.workers.lyrics_worker import create_lyrics_processor
result = create_lyrics_processor(
subtitle_offset_ms=500
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
assert call_kwargs['subtitle_offset_ms'] == 500
class TestLyricsOverrideParameters:
"""Tests for lyrics artist/title override functionality."""
@pytest.fixture
def mock_job_with_overrides(self):
"""Create a mock job with lyrics override fields."""
return Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Beatles, The",
title="Hey Jude - 2009 Remaster",
lyrics_artist="The Beatles",
lyrics_title="Hey Jude",
subtitle_offset_ms=250,
input_media_gcs_path="uploads/test123/song.flac"
)
def test_job_uses_lyrics_artist_override(self, mock_job_with_overrides):
"""Test that job uses lyrics_artist when searching for lyrics."""
job = mock_job_with_overrides
# Use override if present, else fall back to main artist
lyrics_search_artist = job.lyrics_artist or job.artist
assert lyrics_search_artist == "The Beatles"
assert lyrics_search_artist != job.artist # Override is different
def test_job_uses_lyrics_title_override(self, mock_job_with_overrides):
"""Test that job uses lyrics_title when searching for lyrics."""
job = mock_job_with_overrides
# Use override if present, else fall back to main title
lyrics_search_title = job.lyrics_title or job.title
assert lyrics_search_title == "Hey Jude"
assert lyrics_search_title != job.title # Override is different
def test_job_falls_back_when_no_override(self):
"""Test that job falls back to main artist/title when no override."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
input_media_gcs_path="uploads/test123/song.flac"
)
# When override is None, use main values
lyrics_search_artist = job.lyrics_artist or job.artist
lyrics_search_title = job.lyrics_title or job.title
assert lyrics_search_artist == "Test Artist"
assert lyrics_search_title == "Test Song"
class TestScreensWorker:
"""Tests for screens_worker.py functions.
@@ -379,6 +481,71 @@ class TestVideoWorker:
assert hasattr(video_worker, 'logger')
class TestRenderVideoWorkerConfiguration:
"""Tests for render_video_worker subtitle_offset_ms support."""
@pytest.fixture
def mock_job_with_offset(self):
"""Create a mock job with subtitle offset."""
return Job(
job_id="test123",
status=JobStatus.RENDERING_VIDEO,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
subtitle_offset_ms=500,
input_media_gcs_path="uploads/test123/song.flac"
)
def test_job_has_subtitle_offset_ms(self, mock_job_with_offset):
"""Test that job has subtitle_offset_ms field."""
assert mock_job_with_offset.subtitle_offset_ms == 500
def test_subtitle_offset_from_job(self, mock_job_with_offset):
"""Test extracting subtitle offset from job with getattr."""
job = mock_job_with_offset
# This mirrors the logic in render_video_worker.py
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
assert subtitle_offset == 500
def test_subtitle_offset_default_zero(self):
"""Test that subtitle offset defaults to 0 when not set."""
job = Job(
job_id="test123",
status=JobStatus.RENDERING_VIDEO,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
input_media_gcs_path="uploads/test123/song.flac"
)
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
assert subtitle_offset == 0
def test_subtitle_offset_negative_value(self):
"""Test that subtitle offset can be negative (advance subtitles)."""
job = Job(
job_id="test123",
status=JobStatus.RENDERING_VIDEO,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
subtitle_offset_ms=-250, # Negative = advance subtitles
input_media_gcs_path="uploads/test123/song.flac"
)
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
# Negative values should be preserved (not converted to 0)
assert subtitle_offset == -250
class TestDownloadHelpers:
"""Tests for download helper functions in workers."""
@@ -406,3 +573,377 @@ class TestDownloadHelpers:
# Should have downloaded from GCS
mock_storage.download_file.assert_called()
class TestAudioWorkerModelConfiguration:
"""Tests for audio worker model configuration parameters."""
def test_create_audio_processor_with_defaults(self):
"""Test creating AudioProcessor with default models."""
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
from backend.workers.audio_worker import create_audio_processor
with tempfile.TemporaryDirectory() as temp_dir:
result = create_audio_processor(temp_dir)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
# Should use default models
assert call_kwargs['clean_instrumental_model'] == "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
assert call_kwargs['backing_vocals_models'] == ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
assert call_kwargs['other_stems_models'] == ["htdemucs_6s.yaml"]
def test_create_audio_processor_with_custom_clean_model(self):
"""Test creating AudioProcessor with custom clean instrumental model."""
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
from backend.workers.audio_worker import create_audio_processor
with tempfile.TemporaryDirectory() as temp_dir:
result = create_audio_processor(
temp_dir,
clean_instrumental_model="custom_clean_model.ckpt"
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
# Should use custom clean model
assert call_kwargs['clean_instrumental_model'] == "custom_clean_model.ckpt"
# Other models should still be defaults
assert call_kwargs['backing_vocals_models'] == ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
def test_create_audio_processor_with_custom_backing_models(self):
"""Test creating AudioProcessor with custom backing vocals models."""
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
from backend.workers.audio_worker import create_audio_processor
with tempfile.TemporaryDirectory() as temp_dir:
result = create_audio_processor(
temp_dir,
backing_vocals_models=["custom_bv1.ckpt", "custom_bv2.ckpt"]
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
# Should use custom backing vocals models
assert call_kwargs['backing_vocals_models'] == ["custom_bv1.ckpt", "custom_bv2.ckpt"]
def test_create_audio_processor_with_custom_other_stems_models(self):
"""Test creating AudioProcessor with custom other stems models."""
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
from backend.workers.audio_worker import create_audio_processor
with tempfile.TemporaryDirectory() as temp_dir:
result = create_audio_processor(
temp_dir,
other_stems_models=["custom_demucs.yaml"]
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
# Should use custom other stems models
assert call_kwargs['other_stems_models'] == ["custom_demucs.yaml"]
def test_create_audio_processor_with_all_custom_models(self):
"""Test creating AudioProcessor with all custom models."""
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
from backend.workers.audio_worker import create_audio_processor
with tempfile.TemporaryDirectory() as temp_dir:
result = create_audio_processor(
temp_dir,
clean_instrumental_model="custom_clean.ckpt",
backing_vocals_models=["custom_bv.ckpt"],
other_stems_models=["custom_stems.yaml"]
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
# All models should be custom
assert call_kwargs['clean_instrumental_model'] == "custom_clean.ckpt"
assert call_kwargs['backing_vocals_models'] == ["custom_bv.ckpt"]
assert call_kwargs['other_stems_models'] == ["custom_stems.yaml"]
def test_job_model_fields_are_passed_to_processor(self):
"""Test that job model fields can be passed to create_audio_processor."""
mock_job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test",
title="Test",
clean_instrumental_model="job_clean.ckpt",
backing_vocals_models=["job_bv.ckpt"],
other_stems_models=["job_stems.yaml"]
)
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
from backend.workers.audio_worker import create_audio_processor
with tempfile.TemporaryDirectory() as temp_dir:
# Simulate passing job model fields to create_audio_processor
result = create_audio_processor(
temp_dir,
clean_instrumental_model=mock_job.clean_instrumental_model,
backing_vocals_models=mock_job.backing_vocals_models,
other_stems_models=mock_job.other_stems_models
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
# Models from job should be used
assert call_kwargs['clean_instrumental_model'] == "job_clean.ckpt"
assert call_kwargs['backing_vocals_models'] == ["job_bv.ckpt"]
assert call_kwargs['other_stems_models'] == ["job_stems.yaml"]
def test_none_model_values_use_defaults(self):
"""Test that None model values fall back to defaults."""
mock_job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test",
title="Test",
clean_instrumental_model=None, # Not specified
backing_vocals_models=None, # Not specified
other_stems_models=None # Not specified
)
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
from backend.workers.audio_worker import create_audio_processor
with tempfile.TemporaryDirectory() as temp_dir:
result = create_audio_processor(
temp_dir,
clean_instrumental_model=mock_job.clean_instrumental_model,
backing_vocals_models=mock_job.backing_vocals_models,
other_stems_models=mock_job.other_stems_models
)
mock_processor.assert_called_once()
call_kwargs = mock_processor.call_args[1]
# Should fall back to defaults
assert call_kwargs['clean_instrumental_model'] == "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
assert call_kwargs['backing_vocals_models'] == ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
assert call_kwargs['other_stems_models'] == ["htdemucs_6s.yaml"]
class TestDownloadFromUrl:
"""Tests for download_from_url function - URL-based audio download."""
def test_download_from_url_function_exists(self):
"""Test that download_from_url function exists in audio_worker."""
from backend.workers.audio_worker import download_from_url
assert callable(download_from_url)
def test_download_from_url_signature(self):
"""Test that download_from_url has the expected signature."""
import inspect
from backend.workers.audio_worker import download_from_url
sig = inspect.signature(download_from_url)
params = list(sig.parameters.keys())
# Should have these parameters
assert 'url' in params
assert 'temp_dir' in params
assert 'artist' in params
assert 'title' in params
assert 'job_manager' in params
assert 'job_id' in params
def test_download_from_url_is_async(self):
"""Test that download_from_url is an async function."""
import inspect
from backend.workers.audio_worker import download_from_url
assert inspect.iscoroutinefunction(download_from_url)
@pytest.mark.asyncio
async def test_download_from_url_returns_none_for_invalid_url(self):
"""Test that download_from_url handles errors gracefully."""
from backend.workers.audio_worker import download_from_url
with tempfile.TemporaryDirectory() as temp_dir:
# This should fail gracefully (no yt-dlp in test env or invalid URL)
result = await download_from_url(
url='not-a-valid-url',
temp_dir=temp_dir,
artist='Test',
title='Test'
)
# Should return None on error (graceful failure)
assert result is None
class TestDownloadAudioWithUrl:
"""Tests for download_audio function with URL-based jobs."""
@pytest.fixture
def mock_job_with_url(self):
"""Create a mock job with a URL."""
return Job(
job_id="test123",
status=JobStatus.PROCESSING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
url="https://www.youtube.com/watch?v=test123"
)
@pytest.fixture
def mock_job_manager(self, mock_job_with_url):
"""Create a mock JobManager."""
manager = MagicMock()
manager.get_job.return_value = mock_job_with_url
manager.update_job.return_value = None
return manager
@pytest.fixture
def mock_storage(self):
"""Create a mock StorageService."""
storage = MagicMock()
return storage
@pytest.mark.asyncio
async def test_download_audio_from_url(self, mock_job_with_url, mock_job_manager, mock_storage):
"""Test download_audio routes to URL download when job has URL."""
with patch('backend.workers.audio_worker.download_from_url', return_value='/tmp/test.wav') as mock_download:
from backend.workers.audio_worker import download_audio
with tempfile.TemporaryDirectory() as temp_dir:
result = await download_audio(
"test123",
temp_dir,
mock_storage,
mock_job_with_url,
job_manager_instance=mock_job_manager
)
# Should have called download_from_url
mock_download.assert_called_once()
# Check positional args
args = mock_download.call_args[0]
assert args[0] == "https://www.youtube.com/watch?v=test123" # url
assert args[1] == temp_dir # temp_dir
assert args[2] == "Test Artist" # artist
assert args[3] == "Test Song" # title
@pytest.mark.asyncio
async def test_download_audio_from_gcs_when_no_url(self, mock_job_manager, mock_storage):
"""Test download_audio downloads from GCS when job has no URL."""
mock_job_no_url = Job(
job_id="test123",
status=JobStatus.PROCESSING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
input_media_gcs_path="uploads/test123/song.flac"
)
with patch('backend.workers.audio_worker.download_from_url') as mock_url_download:
from backend.workers.audio_worker import download_audio
with tempfile.TemporaryDirectory() as temp_dir:
result = await download_audio(
"test123",
temp_dir,
mock_storage,
mock_job_no_url,
job_manager_instance=mock_job_manager
)
# Should NOT have called download_from_url
mock_url_download.assert_not_called()
# Should have called storage.download_file
mock_storage.download_file.assert_called()
class TestBackingVocalsAnalysis:
"""Tests for backing vocals analysis in render_video_worker."""
def test_analyze_backing_vocals_function_exists(self):
"""Test that _analyze_backing_vocals function exists in render_video_worker."""
from backend.workers.render_video_worker import _analyze_backing_vocals
assert callable(_analyze_backing_vocals)
def test_analyze_backing_vocals_is_async(self):
"""Test that _analyze_backing_vocals is an async function."""
import inspect
from backend.workers.render_video_worker import _analyze_backing_vocals
assert inspect.iscoroutinefunction(_analyze_backing_vocals)
def test_render_video_worker_imports_analysis_service(self):
"""Test that render_video_worker can import AudioAnalysisService."""
# This verifies the import path is correct
from backend.services.audio_analysis_service import AudioAnalysisService
assert AudioAnalysisService is not None
@pytest.mark.asyncio
async def test_analyze_backing_vocals_handles_missing_job(self):
"""Test that analysis returns early when job not found (no error stored)."""
mock_job_manager = MagicMock()
mock_job_manager.get_job.return_value = None # No job found
mock_job_manager.update_state_data = MagicMock()
mock_storage = MagicMock()
mock_logger = MagicMock()
from backend.workers.render_video_worker import _analyze_backing_vocals
# Should not raise when job not found, just log warning and return
await _analyze_backing_vocals(
"nonexistent", mock_job_manager, mock_storage, mock_logger
)
# Should log warning but not update state (early return)
mock_logger.warning.assert_called()
# No state_data update since we return early
mock_job_manager.update_state_data.assert_not_called()
@pytest.mark.asyncio
async def test_analyze_backing_vocals_handles_missing_stems(self):
"""Test that analysis returns early when stems not found (no error stored)."""
mock_job = MagicMock()
mock_job.file_urls = {} # No stems
mock_job_manager = MagicMock()
mock_job_manager.get_job.return_value = mock_job
mock_job_manager.update_state_data = MagicMock()
mock_storage = MagicMock()
mock_logger = MagicMock()
from backend.workers.render_video_worker import _analyze_backing_vocals
# Should not raise when stems not found, just log warning and return
await _analyze_backing_vocals(
"test123", mock_job_manager, mock_storage, mock_logger
)
# Should log warning but not update state (early return)
mock_logger.warning.assert_called()
# No state_data update since we return early
mock_job_manager.update_state_data.assert_not_called()
def test_analysis_service_can_be_instantiated(self):
"""Test that AudioAnalysisService can be instantiated."""
from backend.services.audio_analysis_service import AudioAnalysisService
mock_storage = MagicMock()
service = AudioAnalysisService(storage_service=mock_storage)
assert service is not None
assert service.storage_service == mock_storage
+159 -83
View File
@@ -19,7 +19,7 @@ from backend.models.job import JobStatus
from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.config import get_settings
from backend.workers.worker_logging import create_job_logger, setup_job_logging
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
# Import from karaoke_gen package
from karaoke_gen.audio_processor import AudioProcessor
@@ -34,7 +34,7 @@ AUDIO_WORKER_LOGGERS = [
]
async def download_from_url(url: str, temp_dir: str, artist: str, title: str) -> Optional[str]:
async def download_from_url(url: str, temp_dir: str, artist: str, title: str, job_manager: JobManager = None, job_id: str = None) -> Optional[str]:
"""
Download audio from a URL (YouTube, etc.) using karaoke_gen.FileHandler.
@@ -43,11 +43,16 @@ async def download_from_url(url: str, temp_dir: str, artist: str, title: str) ->
- Cookie support for authenticated downloads
- Retry logic
If artist and/or title are not provided, attempts to extract them from
the URL metadata.
Args:
url: URL to download from
temp_dir: Temporary directory to save to
artist: Artist name for filename
title: Song title for filename
artist: Artist name for filename (can be None for auto-detection)
title: Song title for filename (can be None for auto-detection)
job_manager: Optional JobManager to update job with detected metadata
job_id: Optional job ID to update
Returns:
Path to downloaded audio file, or None if failed
@@ -64,6 +69,34 @@ async def download_from_url(url: str, temp_dir: str, artist: str, title: str) ->
dry_run=False
)
# Try to extract metadata if artist or title not provided
if not artist or not title:
logger.info(f"Extracting metadata from URL: {url}")
metadata = file_handler.extract_metadata_from_url(url)
if metadata:
if not artist:
artist = metadata.get('artist', 'Unknown')
logger.info(f"Auto-detected artist: {artist}")
if not title:
title = metadata.get('title', 'Unknown')
logger.info(f"Auto-detected title: {title}")
# Update job with detected metadata if job_manager provided
if job_manager and job_id:
update_data = {}
if artist:
update_data['artist'] = artist
if title:
update_data['title'] = title
if update_data:
job_manager.update_job(job_id, update_data)
logger.info(f"Updated job {job_id} with detected metadata")
else:
logger.warning("Could not extract metadata from URL, using defaults")
artist = artist or "Unknown"
title = title or "Unknown"
# Create output filename (without extension)
safe_artist = sanitize_filename(artist) if artist else "Unknown"
safe_title = sanitize_filename(title) if title else "Unknown"
@@ -112,7 +145,12 @@ async def download_from_url(url: str, temp_dir: str, artist: str, title: str) ->
return None
def create_audio_processor(temp_dir: str) -> AudioProcessor:
def create_audio_processor(
temp_dir: str,
clean_instrumental_model: Optional[str] = None,
backing_vocals_models: Optional[list] = None,
other_stems_models: Optional[list] = None
) -> AudioProcessor:
"""
Create an AudioProcessor instance configured for remote API processing.
@@ -120,10 +158,13 @@ def create_audio_processor(temp_dir: str) -> AudioProcessor:
- Uses remote Modal API (via AUDIO_SEPARATOR_API_URL env var)
- No local models needed (model_file_dir=None)
- FLAC output format for quality
- Standard model configurations from karaoke-gen CLI
- Model configurations from job or CLI defaults
Args:
temp_dir: Temporary directory for processing
clean_instrumental_model: Model for clean instrumental separation (optional, uses default if not provided)
backing_vocals_models: List of models for backing vocals separation (optional, uses default if not provided)
other_stems_models: List of models for other stems separation (optional, uses default if not provided)
Returns:
Configured AudioProcessor instance
@@ -132,10 +173,10 @@ def create_audio_processor(temp_dir: str) -> AudioProcessor:
audio_logger = logging.getLogger("karaoke_gen.audio_processor")
audio_logger.setLevel(logging.INFO)
# Model configurations (same as CLI defaults)
clean_instrumental_model = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
backing_vocals_models = ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
other_stems_models = ["htdemucs_6s.yaml"] # For 6-stem separation (bass, drums, etc.)
# Model configurations - use provided values or CLI defaults
effective_clean_model = clean_instrumental_model or "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
effective_backing_models = backing_vocals_models or ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
effective_other_models = other_stems_models or ["htdemucs_6s.yaml"] # For 6-stem separation (bass, drums, etc.)
# FFmpeg command for combining audio files (must be a string, not a list)
ffmpeg_base_command = "ffmpeg -hide_banner -loglevel error -nostats -y"
@@ -146,9 +187,9 @@ def create_audio_processor(temp_dir: str) -> AudioProcessor:
log_formatter=None, # Not needed for our use case
model_file_dir=None, # No local models, using remote API
lossless_output_format="FLAC",
clean_instrumental_model=clean_instrumental_model,
backing_vocals_models=backing_vocals_models,
other_stems_models=other_stems_models,
clean_instrumental_model=effective_clean_model,
backing_vocals_models=effective_backing_models,
other_stems_models=effective_other_models,
ffmpeg_base_command=ffmpeg_base_command
)
@@ -200,71 +241,88 @@ async def process_audio_separation(job_id: str) -> bool:
job_log.info(f"Created temp directory: {temp_dir}")
try:
job_log.info(f"Starting audio separation for {job.artist} - {job.title}")
logger.info(f"Starting audio separation for job {job_id}")
# Ensure AUDIO_SEPARATOR_API_URL is set
api_url = os.environ.get("AUDIO_SEPARATOR_API_URL")
if not api_url:
raise Exception("AUDIO_SEPARATOR_API_URL environment variable not set. "
"Cannot perform audio separation without remote API access.")
job_log.info(f"Audio separator API: {api_url}")
# Download audio file from GCS
job_log.info("Downloading audio file from GCS...")
audio_path = await download_audio(job_id, temp_dir, storage, job)
if not audio_path:
raise Exception("Failed to download audio file")
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
# Update progress using state_data (don't change status during parallel processing)
# The status is managed at a higher level - workers just track their progress
job_manager.update_state_data(job_id, 'audio_progress', {
'stage': 'separating_stage1',
'progress': 10,
'message': 'Starting audio separation (Stage 1: Clean instrumental)'
})
# Create AudioProcessor instance (reuses karaoke_gen code)
job_log.info("Creating AudioProcessor instance...")
audio_processor = create_audio_processor(temp_dir)
# Format artist-title for file naming (matches CLI behavior)
artist_title = f"{job.artist} - {job.title}"
# Run audio separation (calls Modal API internally)
# This returns a dict with paths to all separated stems
job_log.info("Starting audio separation (this may take 5-10 minutes)...")
job_log.info(" Stage 1: Clean instrumental separation (MDX models)")
job_log.info(" Stage 2: Backing vocals separation (Demucs model)")
logger.info(f"Job {job_id}: Calling audio_processor.process_audio_separation()")
separation_result = audio_processor.process_audio_separation(
audio_file=audio_path,
artist_title=artist_title,
track_output_dir=temp_dir
)
job_log.info("Audio separation complete!")
job_log.info(f" Generated {len(separation_result)} stem files")
logger.info(f"Job {job_id}: Audio separation complete, organizing results")
# Update progress using state_data (don't change status during parallel processing)
job_manager.update_state_data(job_id, 'audio_progress', {
'stage': 'audio_complete',
'progress': 45,
'message': 'Audio separation complete, uploading stems'
})
# Upload all stems to GCS
await upload_separation_results(job_id, separation_result, storage, job_manager)
logger.info(f"Job {job_id}: All stems uploaded successfully")
# Mark audio processing complete
# This will check if lyrics are also complete and transition to next stage if so
job_manager.mark_audio_complete(job_id)
return True
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
# This ensures logs from third-party libraries (karaoke_gen.audio_processor) are
# only captured by this job's handler, not handlers from other concurrent jobs
with job_logging_context(job_id):
job_log.info(f"Starting audio separation for {job.artist} - {job.title}")
logger.info(f"Starting audio separation for job {job_id}")
# Ensure AUDIO_SEPARATOR_API_URL is set
api_url = os.environ.get("AUDIO_SEPARATOR_API_URL")
if not api_url:
raise Exception("AUDIO_SEPARATOR_API_URL environment variable not set. "
"Cannot perform audio separation without remote API access.")
job_log.info(f"Audio separator API: {api_url}")
# Download audio file from GCS or URL
job_log.info("Downloading audio file...")
audio_path = await download_audio(job_id, temp_dir, storage, job, job_manager_instance=job_manager)
if not audio_path:
raise Exception("Failed to download audio file")
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
# Update progress using state_data (don't change status during parallel processing)
# The status is managed at a higher level - workers just track their progress
job_manager.update_state_data(job_id, 'audio_progress', {
'stage': 'separating_stage1',
'progress': 10,
'message': 'Starting audio separation (Stage 1: Clean instrumental)'
})
# Create AudioProcessor instance (reuses karaoke_gen code)
# Use model configuration from job if provided, otherwise use defaults
job_log.info("Creating AudioProcessor instance...")
if job.clean_instrumental_model:
job_log.info(f" Using clean instrumental model: {job.clean_instrumental_model}")
if job.backing_vocals_models:
job_log.info(f" Using backing vocals models: {job.backing_vocals_models}")
if job.other_stems_models:
job_log.info(f" Using other stems models: {job.other_stems_models}")
audio_processor = create_audio_processor(
temp_dir,
clean_instrumental_model=job.clean_instrumental_model,
backing_vocals_models=job.backing_vocals_models,
other_stems_models=job.other_stems_models
)
# Format artist-title for file naming (matches CLI behavior)
artist_title = f"{job.artist} - {job.title}"
# Run audio separation (calls Modal API internally)
# This returns a dict with paths to all separated stems
job_log.info("Starting audio separation (this may take 5-10 minutes)...")
job_log.info(" Stage 1: Clean instrumental separation (MDX models)")
job_log.info(" Stage 2: Backing vocals separation (Demucs model)")
logger.info(f"Job {job_id}: Calling audio_processor.process_audio_separation()")
separation_result = audio_processor.process_audio_separation(
audio_file=audio_path,
artist_title=artist_title,
track_output_dir=temp_dir
)
job_log.info("Audio separation complete!")
job_log.info(f" Generated {len(separation_result)} stem files")
logger.info(f"Job {job_id}: Audio separation complete, organizing results")
# Update progress using state_data (don't change status during parallel processing)
job_manager.update_state_data(job_id, 'audio_progress', {
'stage': 'audio_complete',
'progress': 45,
'message': 'Audio separation complete, uploading stems'
})
# Upload all stems to GCS
await upload_separation_results(job_id, separation_result, storage, job_manager)
logger.info(f"Job {job_id}: All stems uploaded successfully")
# Mark audio processing complete
# This will check if lyrics are also complete and transition to next stage if so
job_manager.mark_audio_complete(job_id)
return True
except Exception as e:
logger.error(f"Job {job_id}: Audio separation failed: {e}", exc_info=True)
@@ -293,7 +351,8 @@ async def download_audio(
job_id: str,
temp_dir: str,
storage: StorageService,
job
job,
job_manager_instance: JobManager = None
) -> Optional[str]:
"""
Download or fetch audio file to local temp directory.
@@ -302,6 +361,13 @@ async def download_audio(
1. Uploaded file: Download from GCS using input_media_gcs_path
2. URL (YouTube, etc.): Download using yt-dlp or other tools
Args:
job_id: Job ID
temp_dir: Temporary directory to save to
storage: StorageService instance
job: Job object with URL or GCS path
job_manager_instance: Optional JobManager instance for updating job metadata
Returns:
Path to downloaded audio file, or None if failed
"""
@@ -326,7 +392,18 @@ async def download_audio(
# Case 3: Fresh URL that needs downloading
if job.url:
logger.info(f"Job {job_id}: Downloading from URL: {job.url}")
local_path = await download_from_url(job.url, temp_dir, job.artist, job.title)
# Use provided job_manager or create new one
jm = job_manager_instance or JobManager()
local_path = await download_from_url(
job.url,
temp_dir,
job.artist,
job.title,
job_manager=jm,
job_id=job_id
)
if local_path and os.path.exists(local_path):
# Upload to GCS and update job
@@ -334,9 +411,8 @@ async def download_audio(
url = storage.upload_file(local_path, gcs_path)
# Update job with GCS path for lyrics worker
job_manager = JobManager()
job_manager.firestore.update_job(job_id, {'input_media_gcs_path': gcs_path})
job_manager.update_file_url(job_id, 'input', 'audio', url)
jm.update_job(job_id, {'input_media_gcs_path': gcs_path})
jm.update_file_url(job_id, 'input', 'audio', url)
logger.info(f"Job {job_id}: Downloaded and uploaded audio to GCS: {gcs_path}")
return local_path
+186 -141
View File
@@ -26,7 +26,7 @@ from backend.models.job import JobStatus
from karaoke_gen.utils import sanitize_filename
from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.workers.worker_logging import create_job_logger, setup_job_logging
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
from backend.workers.style_helper import load_style_config
# Import from karaoke_gen package
@@ -36,27 +36,21 @@ from karaoke_gen.lyrics_processor import LyricsProcessor
logger = logging.getLogger(__name__)
# Loggers to capture for lyrics worker (includes LyricsTranscriber internals)
# Loggers to capture for lyrics worker (top-level package loggers only)
# Note: We only capture at top-level package loggers to avoid duplicate log entries.
# Logs from sub-modules (e.g., lyrics_transcriber.correction.anchor_sequence) will
# propagate up to their parent logger (lyrics_transcriber) where they get captured.
LYRICS_WORKER_LOGGERS = [
"karaoke_gen.lyrics_processor",
"lyrics_transcriber",
"lyrics_transcriber.core",
"lyrics_transcriber.core.controller",
"lyrics_transcriber.lyrics",
"lyrics_transcriber.lyrics.genius",
"lyrics_transcriber.lyrics.spotify",
"lyrics_transcriber.lyrics.musixmatch",
"lyrics_transcriber.lyrics.lrclib",
"lyrics_transcriber.correction",
"lyrics_transcriber.correction.corrector",
"lyrics_transcriber.correction.anchor_sequence",
"lyrics_transcriber.transcribers",
"lyrics_transcriber.transcribers.audioshake",
"lyrics_transcriber.output",
]
def create_lyrics_processor(style_params_json: Optional[str] = None) -> LyricsProcessor:
def create_lyrics_processor(
style_params_json: Optional[str] = None,
lyrics_file: Optional[str] = None,
subtitle_offset_ms: int = 0
) -> LyricsProcessor:
"""
Create a LyricsProcessor instance configured for Cloud Run processing.
@@ -68,6 +62,8 @@ def create_lyrics_processor(style_params_json: Optional[str] = None) -> LyricsPr
Args:
style_params_json: Optional path to style parameters JSON file
lyrics_file: Optional path to user-provided lyrics file (overrides API fetch)
subtitle_offset_ms: Offset for subtitle timing in milliseconds
Returns:
Configured LyricsProcessor instance
@@ -79,11 +75,11 @@ def create_lyrics_processor(style_params_json: Optional[str] = None) -> LyricsPr
return LyricsProcessor(
logger=lyrics_logger,
style_params_json=style_params_json,
lyrics_file=None, # Will fetch from APIs
lyrics_file=lyrics_file, # Use user-provided lyrics if available
skip_transcription=False, # We want transcription
skip_transcription_review=True, # Skip interactive review (use React UI instead)
render_video=False, # Skip video generation for now (will be done after review)
subtitle_offset_ms=0 # No offset by default
subtitle_offset_ms=subtitle_offset_ms
)
@@ -131,126 +127,162 @@ async def process_lyrics_transcription(job_id: str) -> bool:
temp_dir = None
try:
job = job_manager.get_job(job_id)
if not job:
logger.error(f"Job {job_id} not found")
job_log.error(f"Job {job_id} not found in Firestore!")
return False
# Create temporary working directory
temp_dir = tempfile.mkdtemp(prefix=f"karaoke_lyrics_{job_id}_")
job_log.info(f"Created temp directory: {temp_dir}")
job_log.info(f"Starting lyrics transcription for {job.artist} - {job.title}")
job_log.info(f"Log capture enabled for: {', '.join(LYRICS_WORKER_LOGGERS)}")
logger.info(f"Starting lyrics transcription for job {job_id}")
# Log environment configuration
job_log.info("Checking API configuration...")
if os.environ.get("GENIUS_API_TOKEN"):
job_log.info(" Genius API: configured")
else:
job_log.warning(" Genius API: NOT configured")
if os.environ.get("SPOTIFY_COOKIE_SP_DC"):
job_log.info(" Spotify: configured")
else:
job_log.warning(" Spotify: NOT configured")
if os.environ.get("RAPIDAPI_KEY"):
job_log.info(" Musixmatch (RapidAPI): configured")
else:
job_log.warning(" Musixmatch (RapidAPI): NOT configured")
# Ensure required environment variables are set
if not os.environ.get("AUDIOSHAKE_API_TOKEN"):
job_log.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
logger.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
# Download audio file from GCS (waits for audio worker if URL job)
job_log.info("Downloading audio file from GCS...")
audio_path = await download_audio(job_id, temp_dir, storage, job, job_manager)
if not audio_path:
raise Exception("Failed to download audio file")
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
# Update progress using state_data (don't change status during parallel processing)
# The status is managed at a higher level - workers just track their progress
job_manager.update_state_data(job_id, 'lyrics_progress', {
'stage': 'transcribing',
'progress': 10,
'message': 'Starting lyrics transcription via AudioShake'
})
# Load style configuration (downloads assets from GCS if available)
# This is needed for max_line_length and other style settings in LyricsTranscriber
job_log.info("Loading style configuration for lyrics processing...")
style_config = await load_style_config(job, storage, temp_dir)
style_params_json_path = style_config.get_style_params_path() if style_config.has_custom_styles() else None
if style_params_json_path:
job_log.info(f"Using custom style params: {style_params_json_path}")
# Log the contents of the style params for debugging
try:
with open(style_params_json_path, 'r') as f:
style_content = json.load(f)
job_log.info(f"Style params sections: {list(style_content.keys())}")
if 'karaoke' in style_content:
karaoke_style = style_content['karaoke']
job_log.info(f" karaoke.background_image: {karaoke_style.get('background_image', 'NOT SET')}")
job_log.info(f" karaoke.font_path: {karaoke_style.get('font_path', 'NOT SET')}")
except Exception as e:
job_log.warning(f"Could not read style params for logging: {e}")
else:
job_log.info("No custom style params found, using defaults")
# Create LyricsProcessor instance (reuses karaoke_gen code)
job_log.info("Creating LyricsProcessor instance...")
job_log.info(f" style_params_json: {style_params_json_path}")
lyrics_processor = create_lyrics_processor(style_params_json=style_params_json_path)
# Run transcription + correction
# This will:
# 1. Fetch reference lyrics from Genius/Spotify/Musixmatch/LRCLib
# 2. Transcribe audio via AudioShake API
# 3. Run automatic correction with lyrics_transcriber
# 4. Generate corrections JSON (no interactive review)
# 5. Skip video generation (render_video=False)
job_log.info("Starting LyricsTranscriber processing...")
job_log.info(f" Artist: {job.artist}")
job_log.info(f" Title: {job.title}")
logger.info(f"Job {job_id}: Calling lyrics_processor.transcribe_lyrics()")
# Run transcription in thread pool to avoid blocking the event loop
# (transcribe_lyrics is synchronous and takes 1-2 minutes)
result = await asyncio.to_thread(
lyrics_processor.transcribe_lyrics,
input_audio_wav=audio_path,
artist=job.artist,
title=job.title,
track_output_dir=temp_dir
)
job_log.info("Transcription processing complete")
logger.info(f"Job {job_id}: Transcription complete, uploading results")
# Upload lyrics results to GCS
job_log.info("Uploading lyrics results to GCS...")
await upload_lyrics_results(job_id, temp_dir, result, storage, job_manager, job_log)
job_log.info("All lyrics data uploaded successfully")
logger.info(f"Job {job_id}: All lyrics data uploaded successfully")
# Update progress using state_data (don't change status during parallel processing)
job_manager.update_state_data(job_id, 'lyrics_progress', {
'stage': 'lyrics_complete',
'progress': 45,
'message': 'Lyrics transcription complete'
})
# Mark lyrics processing complete
# This will check if audio is also complete and transition to next stage if so
job_log.info("Lyrics worker complete, checking if audio is also done...")
job_manager.mark_lyrics_complete(job_id)
return True
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
# This ensures logs from third-party libraries (lyrics_transcriber) are only captured
# by this job's handler, not handlers from other concurrent jobs
with job_logging_context(job_id):
job = job_manager.get_job(job_id)
if not job:
logger.error(f"Job {job_id} not found")
job_log.error(f"Job {job_id} not found in Firestore!")
return False
# Create temporary working directory
temp_dir = tempfile.mkdtemp(prefix=f"karaoke_lyrics_{job_id}_")
job_log.info(f"Created temp directory: {temp_dir}")
job_log.info(f"Starting lyrics transcription for {job.artist} - {job.title}")
job_log.info(f"Log capture enabled for: {', '.join(LYRICS_WORKER_LOGGERS)}")
logger.info(f"Starting lyrics transcription for job {job_id}")
# Log environment configuration
job_log.info("Checking API configuration...")
if os.environ.get("GENIUS_API_TOKEN"):
job_log.info(" Genius API: configured")
else:
job_log.warning(" Genius API: NOT configured")
if os.environ.get("SPOTIFY_COOKIE_SP_DC"):
job_log.info(" Spotify: configured")
else:
job_log.warning(" Spotify: NOT configured")
if os.environ.get("RAPIDAPI_KEY"):
job_log.info(" Musixmatch (RapidAPI): configured")
else:
job_log.warning(" Musixmatch (RapidAPI): NOT configured")
# Ensure required environment variables are set
if not os.environ.get("AUDIOSHAKE_API_TOKEN"):
job_log.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
logger.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
# Download audio file from GCS (waits for audio worker if URL job)
job_log.info("Downloading audio file from GCS...")
audio_path = await download_audio(job_id, temp_dir, storage, job, job_manager)
if not audio_path:
raise Exception("Failed to download audio file")
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
# Update progress using state_data (don't change status during parallel processing)
# The status is managed at a higher level - workers just track their progress
job_manager.update_state_data(job_id, 'lyrics_progress', {
'stage': 'transcribing',
'progress': 10,
'message': 'Starting lyrics transcription via AudioShake'
})
# Load style configuration (downloads assets from GCS if available)
# This is needed for max_line_length and other style settings in LyricsTranscriber
job_log.info("Loading style configuration for lyrics processing...")
style_config = await load_style_config(job, storage, temp_dir)
style_params_json_path = style_config.get_style_params_path() if style_config.has_custom_styles() else None
if style_params_json_path:
job_log.info(f"Using custom style params: {style_params_json_path}")
# Log the contents of the style params for debugging
try:
with open(style_params_json_path, 'r') as f:
style_content = json.load(f)
job_log.info(f"Style params sections: {list(style_content.keys())}")
if 'karaoke' in style_content:
karaoke_style = style_content['karaoke']
job_log.info(f" karaoke.background_image: {karaoke_style.get('background_image', 'NOT SET')}")
job_log.info(f" karaoke.font_path: {karaoke_style.get('font_path', 'NOT SET')}")
except Exception as e:
job_log.warning(f"Could not read style params for logging: {e}")
else:
job_log.info("No custom style params found, using defaults")
# Download user-provided lyrics file if available
lyrics_file_path = None
if hasattr(job, 'lyrics_file_gcs_path') and job.lyrics_file_gcs_path:
lyrics_file_path = os.path.join(temp_dir, "user_lyrics.txt")
job_log.info(f"Downloading user-provided lyrics file: {job.lyrics_file_gcs_path}")
storage.download_file(job.lyrics_file_gcs_path, lyrics_file_path)
job_log.info(f"User lyrics file downloaded to: {lyrics_file_path}")
# Get lyrics configuration from job
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
if subtitle_offset != 0:
job_log.info(f"Subtitle offset: {subtitle_offset}ms")
# Create LyricsProcessor instance (reuses karaoke_gen code)
job_log.info("Creating LyricsProcessor instance...")
job_log.info(f" style_params_json: {style_params_json_path}")
job_log.info(f" lyrics_file: {lyrics_file_path}")
job_log.info(f" subtitle_offset_ms: {subtitle_offset}")
lyrics_processor = create_lyrics_processor(
style_params_json=style_params_json_path,
lyrics_file=lyrics_file_path,
subtitle_offset_ms=subtitle_offset
)
# Run transcription + correction
# This will:
# 1. Fetch reference lyrics from Genius/Spotify/Musixmatch/LRCLib
# 2. Transcribe audio via AudioShake API
# 3. Run automatic correction with lyrics_transcriber
# 4. Generate corrections JSON (no interactive review)
# 5. Skip video generation (render_video=False)
# Use lyrics_artist/lyrics_title overrides if provided, else fall back to job artist/title
lyrics_search_artist = getattr(job, 'lyrics_artist', None) or job.artist
lyrics_search_title = getattr(job, 'lyrics_title', None) or job.title
job_log.info("Starting LyricsTranscriber processing...")
job_log.info(f" Artist: {job.artist}")
job_log.info(f" Title: {job.title}")
if lyrics_search_artist != job.artist:
job_log.info(f" Lyrics search artist override: {lyrics_search_artist}")
if lyrics_search_title != job.title:
job_log.info(f" Lyrics search title override: {lyrics_search_title}")
logger.info(f"Job {job_id}: Calling lyrics_processor.transcribe_lyrics()")
# Run transcription in thread pool to avoid blocking the event loop
# (transcribe_lyrics is synchronous and takes 1-2 minutes)
# Note: We pass the original artist/title for file naming, but the LyricsProcessor
# will use lyrics_artist/lyrics_title for the search (set via constructor args)
result = await asyncio.to_thread(
lyrics_processor.transcribe_lyrics,
input_audio_wav=audio_path,
artist=job.artist, # Original artist for file naming
title=job.title, # Original title for file naming
track_output_dir=temp_dir,
lyrics_artist=lyrics_search_artist, # Override for lyrics search
lyrics_title=lyrics_search_title # Override for lyrics search
)
job_log.info("Transcription processing complete")
logger.info(f"Job {job_id}: Transcription complete, uploading results")
# Upload lyrics results to GCS
job_log.info("Uploading lyrics results to GCS...")
await upload_lyrics_results(job_id, temp_dir, result, storage, job_manager, job_log)
job_log.info("All lyrics data uploaded successfully")
logger.info(f"Job {job_id}: All lyrics data uploaded successfully")
# Update progress using state_data (don't change status during parallel processing)
job_manager.update_state_data(job_id, 'lyrics_progress', {
'stage': 'lyrics_complete',
'progress': 45,
'message': 'Lyrics transcription complete'
})
# Mark lyrics processing complete
# This will check if audio is also complete and transition to next stage if so
job_log.info("Lyrics worker complete, checking if audio is also done...")
job_manager.mark_lyrics_complete(job_id)
return True
except Exception as e:
job_log.error(f"Lyrics transcription failed: {str(e)}", exc_info=True)
@@ -415,11 +447,24 @@ async def upload_lyrics_results(
corrections_data = json.load(f)
# Store metadata in state_data
job_manager.update_state_data(job_id, 'lyrics_metadata', {
# Include countdown padding info for instrumental synchronization
lyrics_metadata = {
'segment_count': len(corrections_data.get('corrected_segments', [])),
'has_corrections': True,
'ready_for_review': True
})
'ready_for_review': True,
}
# Add countdown padding info from transcription result
# This is needed by video_worker to pad instrumentals for sync
if transcription_result.get("countdown_padding_added"):
lyrics_metadata['has_countdown_padding'] = True
lyrics_metadata['countdown_padding_seconds'] = transcription_result.get("countdown_padding_seconds", 3.0)
logger.info(f"Job {job_id}: Countdown padding detected: {lyrics_metadata['countdown_padding_seconds']}s")
else:
lyrics_metadata['has_countdown_padding'] = False
lyrics_metadata['countdown_padding_seconds'] = 0.0
job_manager.update_state_data(job_id, 'lyrics_metadata', lyrics_metadata)
except Exception as e:
logger.warning(f"Job {job_id}: Could not parse corrections JSON: {e}")
else:
+308 -180
View File
@@ -26,7 +26,7 @@ from backend.models.job import JobStatus
from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.config import get_settings
from backend.workers.worker_logging import create_job_logger, setup_job_logging
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
# Import from lyrics_transcriber (submodule)
from lyrics_transcriber.output.generator import OutputGenerator
@@ -85,189 +85,213 @@ async def process_render_video(job_id: str) -> bool:
logger.info(f"Job {job_id}: Starting video render (post-review)")
try:
# Transition to RENDERING_VIDEO
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.RENDERING_VIDEO,
progress=75,
message="Rendering karaoke video with corrected lyrics"
)
with tempfile.TemporaryDirectory() as temp_dir:
job_log.info(f"Created temp directory: {temp_dir}")
# 1. Download corrected corrections data
corrections_path = os.path.join(temp_dir, "corrections.json")
# Try updated corrections first (from human review), fall back to original
corrections_gcs_updated = f"jobs/{job_id}/lyrics/corrections_updated.json"
corrections_gcs_original = f"jobs/{job_id}/lyrics/corrections.json"
# Get GCS path from file_urls
corrections_url = job.file_urls.get('lyrics', {}).get('corrections_updated')
if not corrections_url:
corrections_url = job.file_urls.get('lyrics', {}).get('corrections')
if not corrections_url:
# Try direct GCS paths
if storage.file_exists(corrections_gcs_updated):
corrections_gcs = corrections_gcs_updated
elif storage.file_exists(corrections_gcs_original):
corrections_gcs = corrections_gcs_original
else:
raise FileNotFoundError(f"No corrections file found for job {job_id}")
else:
# Extract GCS path from URL
corrections_gcs = _extract_gcs_path(corrections_url)
# 2. Load the ORIGINAL corrections (has full structure)
original_corrections_gcs = f"jobs/{job_id}/lyrics/corrections.json"
original_corrections_path = os.path.join(temp_dir, "corrections_original.json")
job_log.info(f"Downloading original corrections from {original_corrections_gcs}")
logger.info(f"Job {job_id}: Downloading original corrections from {original_corrections_gcs}")
storage.download_file(original_corrections_gcs, original_corrections_path)
with open(original_corrections_path, 'r', encoding='utf-8') as f:
original_data = json.load(f)
# 3. Check if there are updated corrections (from review UI)
# The frontend sends only partial data: {corrections, corrected_segments}
updated_corrections_gcs = f"jobs/{job_id}/lyrics/corrections_updated.json"
if storage.file_exists(updated_corrections_gcs):
job_log.info("Found updated corrections from review, merging")
logger.info(f"Job {job_id}: Found updated corrections, merging")
updated_path = os.path.join(temp_dir, "corrections_updated.json")
storage.download_file(updated_corrections_gcs, updated_path)
with open(updated_path, 'r', encoding='utf-8') as f:
updated_data = json.load(f)
# Merge: update the original with the user's corrections
if 'corrections' in updated_data:
original_data['corrections'] = updated_data['corrections']
if 'corrected_segments' in updated_data:
original_data['corrected_segments'] = updated_data['corrected_segments']
job_log.info("Merged user corrections into original data")
logger.info(f"Job {job_id}: Merged user corrections into original data")
# 4. Convert to CorrectionResult
correction_result = CorrectionResult.from_dict(original_data)
job_log.info(f"Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
logger.info(f"Job {job_id}: Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
# 3. Download audio file
audio_path = os.path.join(temp_dir, "audio.flac")
audio_gcs_path = job.input_media_gcs_path
if not audio_gcs_path:
raise FileNotFoundError(f"No input audio path for job {job_id}")
job_log.info(f"Downloading audio from {audio_gcs_path}")
logger.info(f"Job {job_id}: Downloading audio from {audio_gcs_path}")
storage.download_file(audio_gcs_path, audio_path)
job_log.info(f"Audio downloaded: {os.path.getsize(audio_path)} bytes")
# 4. Get or create styles using the unified style loader
job_log.info("Loading style configuration...")
job_log.info(f" job.style_params_gcs_path: {job.style_params_gcs_path}")
job_log.info(f" job.style_assets: {list(job.style_assets.keys()) if job.style_assets else 'None'}")
styles_path, style_data = load_styles_from_gcs(
style_params_gcs_path=job.style_params_gcs_path,
style_assets=job.style_assets,
temp_dir=temp_dir,
download_func=storage.download_file,
logger=job_log,
)
# 5. Configure OutputGenerator
output_dir = os.path.join(temp_dir, "output")
cache_dir = os.path.join(temp_dir, "cache")
os.makedirs(output_dir, exist_ok=True)
os.makedirs(cache_dir, exist_ok=True)
job_log.info(f"Using styles from: {styles_path}")
config = OutputConfig(
output_dir=output_dir,
cache_dir=cache_dir,
output_styles_json=styles_path,
render_video=True,
generate_cdg=False, # CDG optional, generated separately
generate_plain_text=True,
generate_lrc=True,
video_resolution="4k",
subtitle_offset_ms=0
)
job_log.info(f"OutputConfig: output_styles_json={config.output_styles_json}, render_video={config.render_video}")
output_generator = OutputGenerator(config, logger)
# 6. Generate outputs (video, LRC, ASS, etc.)
output_prefix = f"{job.artist or 'Unknown'} - {job.title or 'Unknown'}"
job_log.info(f"Generating outputs with prefix '{output_prefix}'")
logger.info(f"Job {job_id}: Generating outputs with prefix '{output_prefix}'")
outputs = output_generator.generate_outputs(
transcription_corrected=correction_result,
lyrics_results={}, # Reference lyrics already in correction_result
output_prefix=output_prefix,
audio_filepath=audio_path,
artist=job.artist,
title=job.title
)
# 7. Upload video to GCS
if outputs.video and os.path.exists(outputs.video):
video_size = os.path.getsize(outputs.video)
job_log.info(f"Video generated: {video_size} bytes")
video_gcs_path = f"jobs/{job_id}/videos/with_vocals.mkv"
video_url = storage.upload_file(outputs.video, video_gcs_path)
job_manager.update_file_url(job_id, 'videos', 'with_vocals', video_url)
job_log.info(f"Uploaded with_vocals.mkv to GCS")
logger.info(f"Job {job_id}: Uploaded with_vocals.mkv ({video_size} bytes)")
else:
job_log.error("Video generation failed - no output file produced!")
raise Exception("Video generation failed - no output file produced")
# 8. Upload LRC file
if outputs.lrc and os.path.exists(outputs.lrc):
lrc_gcs_path = f"jobs/{job_id}/lyrics/karaoke.lrc"
lrc_url = storage.upload_file(outputs.lrc, lrc_gcs_path)
job_manager.update_file_url(job_id, 'lyrics', 'lrc', lrc_url)
job_log.info("Uploaded karaoke.lrc")
logger.info(f"Job {job_id}: Uploaded karaoke.lrc")
# 9. Upload ASS subtitle file
if outputs.ass and os.path.exists(outputs.ass):
ass_gcs_path = f"jobs/{job_id}/lyrics/karaoke.ass"
ass_url = storage.upload_file(outputs.ass, ass_gcs_path)
job_manager.update_file_url(job_id, 'lyrics', 'ass', ass_url)
job_log.info("Uploaded karaoke.ass")
logger.info(f"Job {job_id}: Uploaded karaoke.ass")
# 10. Upload corrected text files
if outputs.corrected_txt and os.path.exists(outputs.corrected_txt):
txt_gcs_path = f"jobs/{job_id}/lyrics/corrected.txt"
txt_url = storage.upload_file(outputs.corrected_txt, txt_gcs_path)
job_manager.update_file_url(job_id, 'lyrics', 'corrected_txt', txt_url)
job_log.info("Uploaded corrected.txt")
logger.info(f"Job {job_id}: Uploaded corrected.txt")
# 11. Transition to awaiting instrumental selection
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
# This ensures logs from third-party libraries (lyrics_transcriber.output) are only
# captured by this job's handler, not handlers from other concurrent jobs
with job_logging_context(job_id):
# Transition to RENDERING_VIDEO
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Video rendered - select your instrumental"
new_status=JobStatus.RENDERING_VIDEO,
progress=75,
message="Rendering karaoke video with corrected lyrics"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
return True
with tempfile.TemporaryDirectory() as temp_dir:
job_log.info(f"Created temp directory: {temp_dir}")
# 1. Download corrected corrections data
corrections_path = os.path.join(temp_dir, "corrections.json")
# Try updated corrections first (from human review), fall back to original
corrections_gcs_updated = f"jobs/{job_id}/lyrics/corrections_updated.json"
corrections_gcs_original = f"jobs/{job_id}/lyrics/corrections.json"
# Get GCS path from file_urls
corrections_url = job.file_urls.get('lyrics', {}).get('corrections_updated')
if not corrections_url:
corrections_url = job.file_urls.get('lyrics', {}).get('corrections')
if not corrections_url:
# Try direct GCS paths
if storage.file_exists(corrections_gcs_updated):
corrections_gcs = corrections_gcs_updated
elif storage.file_exists(corrections_gcs_original):
corrections_gcs = corrections_gcs_original
else:
raise FileNotFoundError(f"No corrections file found for job {job_id}")
else:
# Extract GCS path from URL
corrections_gcs = _extract_gcs_path(corrections_url)
# 2. Load the ORIGINAL corrections (has full structure)
original_corrections_gcs = f"jobs/{job_id}/lyrics/corrections.json"
original_corrections_path = os.path.join(temp_dir, "corrections_original.json")
job_log.info(f"Downloading original corrections from {original_corrections_gcs}")
logger.info(f"Job {job_id}: Downloading original corrections from {original_corrections_gcs}")
storage.download_file(original_corrections_gcs, original_corrections_path)
with open(original_corrections_path, 'r', encoding='utf-8') as f:
original_data = json.load(f)
# 3. Check if there are updated corrections (from review UI)
# The frontend sends only partial data: {corrections, corrected_segments}
updated_corrections_gcs = f"jobs/{job_id}/lyrics/corrections_updated.json"
if storage.file_exists(updated_corrections_gcs):
job_log.info("Found updated corrections from review, merging")
logger.info(f"Job {job_id}: Found updated corrections, merging")
updated_path = os.path.join(temp_dir, "corrections_updated.json")
storage.download_file(updated_corrections_gcs, updated_path)
with open(updated_path, 'r', encoding='utf-8') as f:
updated_data = json.load(f)
# Merge: update the original with the user's corrections
if 'corrections' in updated_data:
original_data['corrections'] = updated_data['corrections']
if 'corrected_segments' in updated_data:
original_data['corrected_segments'] = updated_data['corrected_segments']
job_log.info("Merged user corrections into original data")
logger.info(f"Job {job_id}: Merged user corrections into original data")
# 4. Convert to CorrectionResult
correction_result = CorrectionResult.from_dict(original_data)
job_log.info(f"Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
logger.info(f"Job {job_id}: Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
# 3. Download audio file
audio_path = os.path.join(temp_dir, "audio.flac")
audio_gcs_path = job.input_media_gcs_path
if not audio_gcs_path:
raise FileNotFoundError(f"No input audio path for job {job_id}")
job_log.info(f"Downloading audio from {audio_gcs_path}")
logger.info(f"Job {job_id}: Downloading audio from {audio_gcs_path}")
storage.download_file(audio_gcs_path, audio_path)
job_log.info(f"Audio downloaded: {os.path.getsize(audio_path)} bytes")
# 4. Get or create styles using the unified style loader
job_log.info("Loading style configuration...")
job_log.info(f" job.style_params_gcs_path: {job.style_params_gcs_path}")
job_log.info(f" job.style_assets: {list(job.style_assets.keys()) if job.style_assets else 'None'}")
styles_path, style_data = load_styles_from_gcs(
style_params_gcs_path=job.style_params_gcs_path,
style_assets=job.style_assets,
temp_dir=temp_dir,
download_func=storage.download_file,
logger=job_log,
)
# 5. Configure OutputGenerator
output_dir = os.path.join(temp_dir, "output")
cache_dir = os.path.join(temp_dir, "cache")
os.makedirs(output_dir, exist_ok=True)
os.makedirs(cache_dir, exist_ok=True)
job_log.info(f"Using styles from: {styles_path}")
# Get subtitle offset from job (user-specified timing adjustment)
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
if subtitle_offset != 0:
job_log.info(f"Applying subtitle offset: {subtitle_offset}ms")
config = OutputConfig(
output_dir=output_dir,
cache_dir=cache_dir,
output_styles_json=styles_path,
render_video=True,
generate_cdg=False, # CDG optional, generated separately
generate_plain_text=True,
generate_lrc=True,
video_resolution="4k",
subtitle_offset_ms=subtitle_offset
)
job_log.info(f"OutputConfig: output_styles_json={config.output_styles_json}, render_video={config.render_video}")
output_generator = OutputGenerator(config, logger)
# 6. Generate outputs (video, LRC, ASS, etc.)
output_prefix = f"{job.artist or 'Unknown'} - {job.title or 'Unknown'}"
job_log.info(f"Generating outputs with prefix '{output_prefix}'")
logger.info(f"Job {job_id}: Generating outputs with prefix '{output_prefix}'")
outputs = output_generator.generate_outputs(
transcription_corrected=correction_result,
lyrics_results={}, # Reference lyrics already in correction_result
output_prefix=output_prefix,
audio_filepath=audio_path,
artist=job.artist,
title=job.title
)
# 7. Upload video to GCS
if outputs.video and os.path.exists(outputs.video):
video_size = os.path.getsize(outputs.video)
job_log.info(f"Video generated: {video_size} bytes")
video_gcs_path = f"jobs/{job_id}/videos/with_vocals.mkv"
video_url = storage.upload_file(outputs.video, video_gcs_path)
job_manager.update_file_url(job_id, 'videos', 'with_vocals', video_url)
job_log.info(f"Uploaded with_vocals.mkv to GCS")
logger.info(f"Job {job_id}: Uploaded with_vocals.mkv ({video_size} bytes)")
else:
job_log.error("Video generation failed - no output file produced!")
raise Exception("Video generation failed - no output file produced")
# 8. Upload LRC file
if outputs.lrc and os.path.exists(outputs.lrc):
lrc_gcs_path = f"jobs/{job_id}/lyrics/karaoke.lrc"
lrc_url = storage.upload_file(outputs.lrc, lrc_gcs_path)
job_manager.update_file_url(job_id, 'lyrics', 'lrc', lrc_url)
job_log.info("Uploaded karaoke.lrc")
logger.info(f"Job {job_id}: Uploaded karaoke.lrc")
# 9. Upload ASS subtitle file
if outputs.ass and os.path.exists(outputs.ass):
ass_gcs_path = f"jobs/{job_id}/lyrics/karaoke.ass"
ass_url = storage.upload_file(outputs.ass, ass_gcs_path)
job_manager.update_file_url(job_id, 'lyrics', 'ass', ass_url)
job_log.info("Uploaded karaoke.ass")
logger.info(f"Job {job_id}: Uploaded karaoke.ass")
# 10. Upload corrected text files
if outputs.corrected_txt and os.path.exists(outputs.corrected_txt):
txt_gcs_path = f"jobs/{job_id}/lyrics/corrected.txt"
txt_url = storage.upload_file(outputs.corrected_txt, txt_gcs_path)
job_manager.update_file_url(job_id, 'lyrics', 'corrected_txt', txt_url)
job_log.info("Uploaded corrected.txt")
logger.info(f"Job {job_id}: Uploaded corrected.txt")
# 11. Analyze backing vocals for intelligent instrumental selection
job_log.info("Analyzing backing vocals for instrumental selection...")
await _analyze_backing_vocals(job_id, job_manager, storage, job_log)
# 12. Transition based on prep_only flag
if getattr(job, 'prep_only', False):
# Prep-only mode: stop here and mark as prep complete
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.PREP_COMPLETE,
progress=100,
message="Prep phase complete - download outputs to continue locally"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE (PREP ONLY) ===")
logger.info(f"Job {job_id}: Prep-only video render complete")
else:
# Normal mode: proceed to instrumental selection
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Video rendered - select your instrumental"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
return True
except Exception as e:
job_log.error(f"Video render failed: {e}")
@@ -293,5 +317,109 @@ def _extract_gcs_path(url: str) -> str:
return url
async def _analyze_backing_vocals(
job_id: str,
job_manager: JobManager,
storage: StorageService,
job_log: logging.Logger,
) -> None:
"""
Analyze backing vocals to help with intelligent instrumental selection.
This function:
1. Downloads the backing vocals stem from GCS
2. Runs audio analysis to detect audible content
3. Generates a waveform visualization image
4. Stores analysis results and waveform URL in job state
The analysis data is then used by the frontend to provide an
intelligent instrumental selection experience.
"""
from backend.services.audio_analysis_service import AudioAnalysisService
try:
# Get the job to access file URLs
job = job_manager.get_job(job_id)
if not job:
job_log.warning(f"Could not get job {job_id} for backing vocals analysis")
return
# Get backing vocals path
backing_vocals_path = job.file_urls.get('stems', {}).get('backing_vocals')
if not backing_vocals_path:
job_log.warning("No backing vocals file found - skipping analysis")
return
job_log.info(f"Analyzing backing vocals: {backing_vocals_path}")
# Create analysis service and run analysis
analysis_service = AudioAnalysisService()
# Define output path for waveform
waveform_gcs_path = f"jobs/{job_id}/analysis/backing_vocals_waveform.png"
# Run analysis and generate waveform
result, waveform_path = analysis_service.analyze_and_generate_waveform(
gcs_audio_path=backing_vocals_path,
job_id=job_id,
gcs_waveform_destination=waveform_gcs_path,
)
# Store analysis results in job state_data
analysis_data = {
'has_audible_content': result.has_audible_content,
'total_duration_seconds': result.total_duration_seconds,
'audible_segments': [
{
'start_seconds': seg.start_seconds,
'end_seconds': seg.end_seconds,
'duration_seconds': seg.duration_seconds,
'avg_amplitude_db': seg.avg_amplitude_db,
'peak_amplitude_db': seg.peak_amplitude_db,
}
for seg in result.audible_segments
],
'recommended_selection': result.recommended_selection.value,
'total_audible_duration_seconds': result.total_audible_duration_seconds,
'audible_percentage': result.audible_percentage,
'silence_threshold_db': result.silence_threshold_db,
}
job_manager.update_state_data(job_id, 'backing_vocals_analysis', analysis_data)
# Store waveform URL
job_manager.update_file_url(job_id, 'analysis', 'backing_vocals_waveform', waveform_path)
job_log.info(
f"Backing vocals analysis complete: "
f"has_audible={result.has_audible_content}, "
f"segments={result.segment_count}, "
f"recommendation={result.recommended_selection.value}"
)
# Log segment details if there are any
if result.audible_segments:
job_log.info(f"Audible segments ({len(result.audible_segments)}):")
for i, seg in enumerate(result.audible_segments[:5]): # Log first 5
job_log.info(
f" [{i+1}] {seg.start_seconds:.1f}s - {seg.end_seconds:.1f}s "
f"({seg.duration_seconds:.1f}s, avg: {seg.avg_amplitude_db:.1f}dB)"
)
if len(result.audible_segments) > 5:
job_log.info(f" ... and {len(result.audible_segments) - 5} more")
except Exception as e:
# Log the error but don't fail the job - analysis is a nice-to-have
job_log.warning(f"Backing vocals analysis failed (non-fatal): {e}")
logger.warning(f"Job {job_id}: Backing vocals analysis failed: {e}")
# Store empty analysis so the frontend knows analysis was attempted
job_manager.update_state_data(job_id, 'backing_vocals_analysis', {
'has_audible_content': None,
'analysis_error': str(e),
'recommended_selection': 'review_needed',
})
# For compatibility with worker service
render_video_worker = process_render_video
+93 -89
View File
@@ -35,7 +35,7 @@ from backend.services.job_manager import JobManager
from backend.services.storage_service import StorageService
from backend.config import get_settings
from backend.workers.style_helper import load_style_config, StyleConfig
from backend.workers.worker_logging import create_job_logger, setup_job_logging
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
# Import from karaoke_gen package
from karaoke_gen.video_generator import VideoGenerator
@@ -88,94 +88,98 @@ async def generate_screens(job_id: str) -> bool:
temp_dir = tempfile.mkdtemp(prefix=f"karaoke_screens_{job_id}_")
try:
job_log.info(f"Starting screen generation for {job.artist} - {job.title}")
logger.info(f"Starting screen generation for job {job_id}")
# Transition to GENERATING_SCREENS state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.GENERATING_SCREENS,
progress=50,
message="Generating title and end screens"
)
# Log style assets info
style_assets = getattr(job, 'style_assets', {}) or {}
job_log.info(f"Style assets from job: {list(style_assets.keys()) if style_assets else 'None'}")
if style_assets:
for key, path in style_assets.items():
job_log.info(f" {key}: {path}")
# Load style configuration (downloads assets from GCS if available)
job_log.info("Loading style configuration from GCS...")
style_config = await load_style_config(job, storage, temp_dir)
if style_config.has_custom_styles():
job_log.info("Using CUSTOM style configuration")
logger.info(f"Job {job_id}: Using custom style configuration")
else:
job_log.warning("Using DEFAULT style configuration (no custom styles found)")
logger.info(f"Job {job_id}: Using default style configuration")
# Initialize video generator
video_generator = _create_video_generator(temp_dir)
# Generate title screen with style config
job_log.info("Generating title screen...")
title_screen_path = await _generate_title_screen(
job_id=job_id,
job=job,
video_generator=video_generator,
style_config=style_config,
temp_dir=temp_dir,
job_log=job_log
)
if not title_screen_path:
raise Exception("Title screen generation failed")
job_log.info(f"Title screen generated: {title_screen_path}")
# Generate end screen with style config
job_log.info("Generating end screen...")
end_screen_path = await _generate_end_screen(
job_id=job_id,
job=job,
video_generator=video_generator,
style_config=style_config,
temp_dir=temp_dir,
job_log=job_log
)
if not end_screen_path:
raise Exception("End screen generation failed")
job_log.info(f"End screen generated: {end_screen_path}")
# Upload screens to GCS
await _upload_screens(
job_id=job_id,
job_manager=job_manager,
storage=storage,
title_screen_path=title_screen_path,
end_screen_path=end_screen_path
)
# Apply countdown padding if needed
# (This is handled automatically by checking lyrics metadata)
await _apply_countdown_padding_if_needed(job_id, job_manager, job)
# Transition to AWAITING_REVIEW
# Human must review lyrics before video can be rendered
logger.info(f"Job {job_id}: Screens generated, awaiting lyrics review")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_REVIEW,
progress=55,
message="Ready for lyrics review. Please review and correct lyrics."
)
# TODO: Send notification to user that selection is needed
# await notify_user(job_id, "instrumental_selection_ready")
return True
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
# This ensures logs from third-party libraries (karaoke_gen.video_generator) are only
# captured by this job's handler, not handlers from other concurrent jobs
with job_logging_context(job_id):
job_log.info(f"Starting screen generation for {job.artist} - {job.title}")
logger.info(f"Starting screen generation for job {job_id}")
# Transition to GENERATING_SCREENS state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.GENERATING_SCREENS,
progress=50,
message="Generating title and end screens"
)
# Log style assets info
style_assets = getattr(job, 'style_assets', {}) or {}
job_log.info(f"Style assets from job: {list(style_assets.keys()) if style_assets else 'None'}")
if style_assets:
for key, path in style_assets.items():
job_log.info(f" {key}: {path}")
# Load style configuration (downloads assets from GCS if available)
job_log.info("Loading style configuration from GCS...")
style_config = await load_style_config(job, storage, temp_dir)
if style_config.has_custom_styles():
job_log.info("Using CUSTOM style configuration")
logger.info(f"Job {job_id}: Using custom style configuration")
else:
job_log.warning("Using DEFAULT style configuration (no custom styles found)")
logger.info(f"Job {job_id}: Using default style configuration")
# Initialize video generator
video_generator = _create_video_generator(temp_dir)
# Generate title screen with style config
job_log.info("Generating title screen...")
title_screen_path = await _generate_title_screen(
job_id=job_id,
job=job,
video_generator=video_generator,
style_config=style_config,
temp_dir=temp_dir,
job_log=job_log
)
if not title_screen_path:
raise Exception("Title screen generation failed")
job_log.info(f"Title screen generated: {title_screen_path}")
# Generate end screen with style config
job_log.info("Generating end screen...")
end_screen_path = await _generate_end_screen(
job_id=job_id,
job=job,
video_generator=video_generator,
style_config=style_config,
temp_dir=temp_dir,
job_log=job_log
)
if not end_screen_path:
raise Exception("End screen generation failed")
job_log.info(f"End screen generated: {end_screen_path}")
# Upload screens to GCS
await _upload_screens(
job_id=job_id,
job_manager=job_manager,
storage=storage,
title_screen_path=title_screen_path,
end_screen_path=end_screen_path
)
# Apply countdown padding if needed
# (This is handled automatically by checking lyrics metadata)
await _apply_countdown_padding_if_needed(job_id, job_manager, job)
# Transition to AWAITING_REVIEW
# Human must review lyrics before video can be rendered
logger.info(f"Job {job_id}: Screens generated, awaiting lyrics review")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_REVIEW,
progress=55,
message="Ready for lyrics review. Please review and correct lyrics."
)
# TODO: Send notification to user that selection is needed
# await notify_user(job_id, "instrumental_selection_ready")
return True
except Exception as e:
logger.error(f"Job {job_id}: Screen generation failed: {e}", exc_info=True)
+280 -160
View File
@@ -31,7 +31,7 @@ from backend.services.rclone_service import get_rclone_service
from backend.services.youtube_service import get_youtube_service
from backend.config import get_settings
from backend.workers.style_helper import load_style_config
from backend.workers.worker_logging import create_job_logger, setup_job_logging
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
# Import from karaoke_gen package - reuse existing implementation
from karaoke_gen.karaoke_finalise.karaoke_finalise import KaraokeFinalise
@@ -41,8 +41,10 @@ logger = logging.getLogger(__name__)
# Loggers to capture for video worker
# Include the full module path to properly capture KaraokeFinalise logs
VIDEO_WORKER_LOGGERS = [
"karaoke_gen.karaoke_finalise",
"karaoke_gen.karaoke_finalise.karaoke_finalise", # The actual logger name from __name__
]
@@ -103,149 +105,177 @@ async def generate_video(job_id: str) -> bool:
job_log.warning("YouTube credentials not available - upload will be skipped")
try:
job_log.info(f"Starting video finalization for {job.artist} - {job.title}")
logger.info(f"Starting video generation for job {job_id}")
# Transition to GENERATING_VIDEO state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.GENERATING_VIDEO,
progress=70,
message="Preparing files for video generation"
)
# Download and set up files in the format KaraokeFinalise expects
base_name = f"{job.artist} - {job.title}"
job_log.info("Downloading files from GCS...")
await _setup_working_directory(job_id, job, storage, temp_dir, base_name)
job_log.info("Files downloaded successfully")
# Load style config for CDG styles
job_log.info("Loading CDG style configuration...")
style_config = await load_style_config(job, storage, temp_dir)
cdg_styles = style_config.get_cdg_styles()
if cdg_styles:
job_log.info("CDG styles loaded from custom configuration")
else:
job_log.info("Using default CDG styles")
# Change to working directory (KaraokeFinalise works in cwd)
os.chdir(temp_dir)
# Get the selected instrumental file path
instrumental_selection = job.state_data['instrumental_selection']
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
instrumental_file = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
# Transition to encoding state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.ENCODING,
progress=75,
message="Encoding videos (15-20 min)"
)
# Log finalization parameters
job_log.info("Creating KaraokeFinalise with parameters:")
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
job_log.info(f" enable_txt: {getattr(job, 'enable_txt', False)}")
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
job_log.info(f" instrumental: {instrumental_selection}")
# Create KaraokeFinalise with ALL the parameters from the job
# This reuses all existing functionality!
# Set up YouTube description file if template is provided
youtube_desc_path = None
if youtube_credentials and getattr(job, 'youtube_description_template', None):
youtube_desc_path = os.path.join(temp_dir, "youtube_description.txt")
with open(youtube_desc_path, 'w') as f:
f.write(job.youtube_description_template)
job_log.info("YouTube description template written to temp file")
finalise = KaraokeFinalise(
logger=logger,
log_level=logging.INFO,
dry_run=False,
instrumental_format="flac",
# CDG/TXT generation
enable_cdg=getattr(job, 'enable_cdg', False),
enable_txt=getattr(job, 'enable_txt', False),
cdg_styles=cdg_styles,
# Brand code and organization (server-side mode uses rclone)
brand_prefix=getattr(job, 'brand_prefix', None),
organised_dir=None, # Not used in cloud - files stay in GCS
organised_dir_rclone_root=getattr(job, 'organised_dir_rclone_root', None),
public_share_dir=None, # Not used in cloud
# Notifications
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
# YouTube upload (server-side with pre-loaded credentials)
youtube_client_secrets_file=None, # Not used with pre-stored credentials
youtube_description_file=youtube_desc_path,
user_youtube_credentials=youtube_credentials, # Pre-loaded from Secret Manager
rclone_destination=None,
email_template_file=None,
# Server-side optimizations
non_interactive=True,
server_side_mode=True,
selected_instrumental_file=instrumental_file,
)
# Call process() - this does ALL the work:
# - Encodes to 4 video formats
# - Generates CDG/TXT packages if enabled
# - Posts Discord notification if configured
# - Handles brand code generation
job_log.info("Starting KaraokeFinalise.process() - this may take 15-20 minutes...")
logger.info(f"Job {job_id}: Starting KaraokeFinalise.process()")
result = finalise.process()
job_log.info("KaraokeFinalise.process() complete!")
if result.get('brand_code'):
job_log.info(f"Brand code: {result.get('brand_code')}")
logger.info(f"Job {job_id}: KaraokeFinalise.process() complete")
logger.info(f"Job {job_id}: Brand code: {result.get('brand_code')}")
# Native API distribution uploads (used by remote CLI instead of rclone)
await _handle_native_distribution(
job_id=job_id,
job=job,
job_log=job_log,
job_manager=job_manager,
temp_dir=temp_dir,
result=result,
)
# Upload generated files to GCS
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.PACKAGING,
progress=95,
message="Uploading final files"
)
await _upload_results(job_id, job_manager, storage, temp_dir, result)
# Mark job as complete
logger.info(f"Job {job_id}: Video generation complete")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.COMPLETE,
progress=100,
message="Karaoke generation complete!"
)
# Store result metadata in job
job_manager.update_job(job_id, {
'state_data': {
**job.state_data,
'brand_code': result.get('brand_code'),
'youtube_url': result.get('youtube_url'),
}
})
return True
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
# This ensures logs from third-party libraries (karaoke_gen.karaoke_finalise) are only
# captured by this job's handler, not handlers from other concurrent jobs
with job_logging_context(job_id):
job_log.info(f"Starting video finalization for {job.artist} - {job.title}")
logger.info(f"Starting video generation for job {job_id}")
# Transition to GENERATING_VIDEO state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.GENERATING_VIDEO,
progress=70,
message="Preparing files for video generation"
)
# Download and set up files in the format KaraokeFinalise expects
base_name = f"{job.artist} - {job.title}"
job_log.info("Downloading files from GCS (this may take a few minutes for large files)...")
await _setup_working_directory(job_id, job, storage, temp_dir, base_name, job_log)
job_log.info("All files downloaded successfully")
# Load style config for CDG styles
job_log.info("Loading CDG style configuration...")
style_config = await load_style_config(job, storage, temp_dir)
cdg_styles = style_config.get_cdg_styles()
if cdg_styles:
job_log.info("CDG styles loaded from custom configuration")
else:
job_log.info("Using default CDG styles")
# Change to working directory (KaraokeFinalise works in cwd)
os.chdir(temp_dir)
# Get the selected instrumental file path
# Batch 3: If user provided existing instrumental, use that; otherwise use AI-separated
instrumental_selection = job.state_data['instrumental_selection']
existing_instrumental_path = getattr(job, 'existing_instrumental_gcs_path', None)
if existing_instrumental_path:
# User provided existing instrumental
ext = Path(existing_instrumental_path).suffix.lower()
instrumental_file = os.path.join(temp_dir, f"{base_name} (Instrumental User){ext}")
instrumental_source = "user-provided"
else:
# Use AI-separated instrumental
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
instrumental_file = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
instrumental_source = instrumental_selection
# Transition to encoding state
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.ENCODING,
progress=75,
message="Encoding videos (15-20 min)"
)
# Get countdown padding info from lyrics metadata
# This ensures instrumental is padded to match vocals if countdown was added
lyrics_metadata = job.state_data.get('lyrics_metadata', {})
countdown_padding_seconds = None
if lyrics_metadata.get('has_countdown_padding'):
countdown_padding_seconds = lyrics_metadata.get('countdown_padding_seconds', 3.0)
job_log.info(f"Countdown padding detected: {countdown_padding_seconds}s - instrumental will be padded if needed")
# Log finalization parameters
job_log.info("Creating KaraokeFinalise with parameters:")
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
job_log.info(f" enable_txt: {getattr(job, 'enable_txt', False)}")
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
job_log.info(f" instrumental source: {instrumental_source}")
job_log.info(f" countdown_padding_seconds: {countdown_padding_seconds}")
if existing_instrumental_path:
job_log.info(f" using user-provided instrumental (selection was: {instrumental_selection})")
# Create KaraokeFinalise with ALL the parameters from the job
# This reuses all existing functionality!
# Set up YouTube description file if template is provided
youtube_desc_path = None
if youtube_credentials and getattr(job, 'youtube_description_template', None):
youtube_desc_path = os.path.join(temp_dir, "youtube_description.txt")
with open(youtube_desc_path, 'w') as f:
f.write(job.youtube_description_template)
job_log.info("YouTube description template written to temp file")
finalise = KaraokeFinalise(
logger=logger,
log_level=logging.INFO,
dry_run=False,
instrumental_format="flac",
# CDG/TXT generation
enable_cdg=getattr(job, 'enable_cdg', False),
enable_txt=getattr(job, 'enable_txt', False),
cdg_styles=cdg_styles,
# Brand code and organization (server-side mode uses rclone)
brand_prefix=getattr(job, 'brand_prefix', None),
organised_dir=None, # Not used in cloud - files stay in GCS
organised_dir_rclone_root=getattr(job, 'organised_dir_rclone_root', None),
public_share_dir=None, # Not used in cloud
# Notifications
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
# YouTube upload (server-side with pre-loaded credentials)
youtube_client_secrets_file=None, # Not used with pre-stored credentials
youtube_description_file=youtube_desc_path,
user_youtube_credentials=youtube_credentials, # Pre-loaded from Secret Manager
rclone_destination=None,
email_template_file=None,
# Server-side optimizations
non_interactive=True,
server_side_mode=True,
selected_instrumental_file=instrumental_file,
# Audio synchronization - ensure instrumental matches vocal padding
countdown_padding_seconds=countdown_padding_seconds,
)
# Call process() - this does ALL the work:
# - Encodes to 4 video formats
# - Generates CDG/TXT packages if enabled
# - Posts Discord notification if configured
# - Handles brand code generation
job_log.info("Starting KaraokeFinalise.process() - this may take 15-20 minutes...")
logger.info(f"Job {job_id}: Starting KaraokeFinalise.process()")
result = finalise.process()
job_log.info("KaraokeFinalise.process() complete!")
if result.get('brand_code'):
job_log.info(f"Brand code: {result.get('brand_code')}")
logger.info(f"Job {job_id}: KaraokeFinalise.process() complete")
logger.info(f"Job {job_id}: Brand code: {result.get('brand_code')}")
# Native API distribution uploads (used by remote CLI instead of rclone)
await _handle_native_distribution(
job_id=job_id,
job=job,
job_log=job_log,
job_manager=job_manager,
temp_dir=temp_dir,
result=result,
)
# Upload generated files to GCS
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.PACKAGING,
progress=95,
message="Uploading final files"
)
await _upload_results(job_id, job_manager, storage, temp_dir, result)
# Mark job as complete
logger.info(f"Job {job_id}: Video generation complete")
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.COMPLETE,
progress=100,
message="Karaoke generation complete!"
)
# Store result metadata in job
job_manager.update_job(job_id, {
'state_data': {
**job.state_data,
'brand_code': result.get('brand_code'),
'youtube_url': result.get('youtube_url'),
}
})
return True
except Exception as e:
logger.error(f"Job {job_id}: Video generation failed: {e}", exc_info=True)
@@ -305,6 +335,13 @@ async def _handle_native_distribution(
brand_code = result.get('brand_code')
base_name = f"{job.artist} - {job.title}"
# Check if we should preserve existing brand code (Batch 6: --keep-brand-code)
keep_brand_code = getattr(job, 'keep_brand_code', None)
if keep_brand_code:
brand_code = keep_brand_code
result['brand_code'] = brand_code
job_log.info(f"Using preserved brand code: {brand_code}")
# Upload to Dropbox using native SDK
dropbox_path = getattr(job, 'dropbox_path', None)
brand_prefix = getattr(job, 'brand_prefix', None)
@@ -443,7 +480,8 @@ async def _setup_working_directory(
job,
storage: StorageService,
temp_dir: str,
base_name: str
base_name: str,
job_log=None
) -> None:
"""
Download files from GCS and set up the directory structure KaraokeFinalise expects.
@@ -455,34 +493,55 @@ async def _setup_working_directory(
- {base_name} (Karaoke).lrc
- Instrumental audio file
"""
logger.info(f"Job {job_id}: Setting up working directory")
def log_progress(message: str):
"""Log to both module logger and job logger if available."""
logger.info(f"Job {job_id}: {message}")
if job_log:
job_log.info(message)
log_progress("Setting up working directory")
# Download title screen
log_progress("Downloading title screen...")
title_url = job.file_urls['screens']['title']
title_path = os.path.join(temp_dir, f"{base_name} (Title).mov")
storage.download_file(title_url, title_path)
logger.info(f"Job {job_id}: Downloaded title screen")
log_progress("Downloaded title screen")
# Download end screen
log_progress("Downloading end screen...")
end_url = job.file_urls['screens']['end']
end_path = os.path.join(temp_dir, f"{base_name} (End).mov")
storage.download_file(end_url, end_path)
logger.info(f"Job {job_id}: Downloaded end screen")
log_progress("Downloaded end screen")
# Download lyrics video (with vocals)
# Download lyrics video (with vocals) - this is the largest file, ~1-2GB
log_progress("Downloading karaoke video (largest file, may take 1-2 minutes)...")
lyrics_video_url = job.file_urls['videos']['with_vocals']
lyrics_video_path = os.path.join(temp_dir, f"{base_name} (With Vocals).mov")
storage.download_file(lyrics_video_url, lyrics_video_path)
logger.info(f"Job {job_id}: Downloaded lyrics video")
log_progress("Downloaded karaoke video")
# Download selected instrumental
# Download instrumental - either user-provided existing instrumental or AI-separated
existing_instrumental_path = getattr(job, 'existing_instrumental_gcs_path', None)
instrumental_selection = job.state_data['instrumental_selection']
instrumental_key = 'instrumental_clean' if instrumental_selection == 'clean' else 'instrumental_with_backing'
instrumental_url = job.file_urls['stems'][instrumental_key]
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
instrumental_path = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
storage.download_file(instrumental_url, instrumental_path)
logger.info(f"Job {job_id}: Downloaded instrumental ({instrumental_selection})")
if existing_instrumental_path:
# Batch 3: Use user-provided existing instrumental
ext = Path(existing_instrumental_path).suffix.lower()
instrumental_path = os.path.join(temp_dir, f"{base_name} (Instrumental User){ext}")
log_progress(f"Downloading user-provided existing instrumental...")
storage.download_file(existing_instrumental_path, instrumental_path)
log_progress("Downloaded user-provided instrumental")
else:
# Use AI-separated instrumental based on selection
instrumental_key = 'instrumental_clean' if instrumental_selection == 'clean' else 'instrumental_with_backing'
instrumental_url = job.file_urls['stems'][instrumental_key]
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
instrumental_path = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
log_progress(f"Downloading {instrumental_selection} instrumental audio...")
storage.download_file(instrumental_url, instrumental_path)
log_progress(f"Downloaded instrumental ({instrumental_selection})")
# Download LRC file if available (needed for CDG/TXT)
lyrics_urls = job.file_urls.get('lyrics', {})
@@ -490,19 +549,19 @@ async def _setup_working_directory(
lrc_url = lyrics_urls['lrc']
lrc_path = os.path.join(temp_dir, f"{base_name} (Karaoke).lrc")
storage.download_file(lrc_url, lrc_path)
logger.info(f"Job {job_id}: Downloaded LRC file")
log_progress("Downloaded LRC file")
# Download title/end JPG files (used for YouTube thumbnail)
screens = job.file_urls.get('screens', {})
if screens.get('title_jpg'):
title_jpg_path = os.path.join(temp_dir, f"{base_name} (Title).jpg")
storage.download_file(screens['title_jpg'], title_jpg_path)
logger.info(f"Job {job_id}: Downloaded title JPG for thumbnail")
log_progress("Downloaded title JPG for thumbnail")
if screens.get('end_jpg'):
end_jpg_path = os.path.join(temp_dir, f"{base_name} (End).jpg")
storage.download_file(screens['end_jpg'], end_jpg_path)
logger.info(f"Job {job_id}: Downloaded end JPG")
log_progress("Downloaded end JPG")
async def _upload_results(
@@ -536,3 +595,64 @@ async def _upload_results(
logger.info(f"Job {job_id}: Uploaded {file_key}")
except Exception as e:
logger.error(f"Job {job_id}: Failed to upload {file_key}: {e}")
# ==================== CLI Entry Point for Cloud Run Jobs ====================
# This allows the video worker to be run as a standalone Cloud Run Job.
# Usage: python -m backend.workers.video_worker --job-id <job_id>
def main():
"""
CLI entry point for running video worker as a Cloud Run Job.
This is used when video encoding needs more than 30 minutes
(the Cloud Tasks timeout limit). Cloud Run Jobs support up to 24 hours.
Usage:
python -m backend.workers.video_worker --job-id abc123
Environment Variables:
GOOGLE_CLOUD_PROJECT: GCP project ID (required)
GCS_BUCKET_NAME: Storage bucket name (required)
"""
import argparse
import asyncio
import sys
# Set up argument parser
parser = argparse.ArgumentParser(
description="Video encoding worker for karaoke generation"
)
parser.add_argument(
"--job-id",
required=True,
help="Job ID to process"
)
args = parser.parse_args()
job_id = args.job_id
# Configure logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
logger.info(f"Starting video worker CLI for job {job_id}")
# Run the async worker function
try:
success = asyncio.run(generate_video(job_id))
if success:
logger.info(f"Video generation completed successfully for job {job_id}")
sys.exit(0)
else:
logger.error(f"Video generation failed for job {job_id}")
sys.exit(1)
except Exception as e:
logger.error(f"Video worker crashed: {e}", exc_info=True)
sys.exit(1)
if __name__ == "__main__":
main()
+54 -1
View File
@@ -7,11 +7,48 @@ for remote debugging via the CLI.
Two approaches are provided:
1. JobLogger - A custom logger class for explicit logging in workers
2. JobLogHandler - A logging.Handler that captures logs from any logger (including dependencies)
IMPORTANT: Uses contextvars to ensure log isolation between concurrent jobs.
When multiple jobs run in parallel on the same Cloud Run instance, each job's
worker logs are correctly filtered to only include logs from that job.
"""
import contextvars
import logging
from contextlib import contextmanager
from typing import Optional, Set
# Context variable to track the current job being processed
# This is used to filter logs when multiple jobs run concurrently
_current_job_id: contextvars.ContextVar[Optional[str]] = contextvars.ContextVar(
'current_job_id', default=None
)
@contextmanager
def job_logging_context(job_id: str):
"""
Context manager to set the current job ID for log filtering.
When multiple jobs run concurrently on the same Cloud Run instance,
this ensures that each job's handler only captures logs from its own
processing thread/context.
Usage:
with job_logging_context(job_id):
# All logs emitted here will be associated with this job_id
process_job(job_id)
Args:
job_id: The job ID to set as the current context
"""
token = _current_job_id.set(job_id)
try:
yield
finally:
_current_job_id.reset(token)
class JobLogHandler(logging.Handler):
"""
A logging handler that captures log records and stores them in Firestore.
@@ -53,6 +90,15 @@ class JobLogHandler(logging.Handler):
def emit(self, record: logging.LogRecord) -> None:
"""Process a log record."""
try:
# CRITICAL: Filter by current job context to prevent log mixing
# When multiple jobs run concurrently, each attaches handlers to
# the same global loggers. Without this check, logs from Job A
# would be captured by Job B's handler and vice versa.
current_job = _current_job_id.get()
if current_job is not None and current_job != self.job_id:
# This log is from a different job's context, skip it
return
# Format the log message
message = self.format(record)
@@ -182,6 +228,10 @@ def setup_job_logging(job_id: str, worker_name: str, *logger_names: str) -> JobL
This adds a JobLogHandler to capture logs from specified loggers
(including dependency loggers like lyrics_transcriber).
IMPORTANT: When using this function, wrap your job processing code in
`job_logging_context(job_id)` to ensure proper log isolation when multiple
jobs run concurrently on the same Cloud Run instance.
Args:
job_id: Job ID
worker_name: Worker name
@@ -199,7 +249,10 @@ def setup_job_logging(job_id: str, worker_name: str, *logger_names: str) -> JobL
"lyrics_transcriber", # Capture LyricsTranscriber logs
)
# ... do work ...
# IMPORTANT: Use job_logging_context for proper isolation
with job_logging_context(job_id):
# ... do work ...
pass
# Optional: remove handler when done
for name in logger_names:
+4 -3
View File
@@ -54,6 +54,7 @@ steps:
- 'us-central1-docker.pkg.dev/$PROJECT_ID/karaoke-repo/karaoke-backend:latest'
# Deploy to Cloud Run with version from pyproject.toml
# Uses Cloud Tasks for worker coordination (scalable architecture)
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: 'bash'
args:
@@ -71,10 +72,10 @@ steps:
--cpu-boost \
--execution-environment gen2 \
--timeout 1800 \
--concurrency 4 \
--max-instances 10 \
--concurrency 1 \
--max-instances 50 \
--min-instances 0 \
--set-env-vars "GOOGLE_CLOUD_PROJECT=$PROJECT_ID,GCS_BUCKET_NAME=karaoke-gen-storage-$PROJECT_ID,FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX" \
--set-env-vars "GOOGLE_CLOUD_PROJECT=$PROJECT_ID,GCS_BUCKET_NAME=karaoke-gen-storage-$PROJECT_ID,FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1" \
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest"
images:
@@ -1,10 +1,32 @@
# Backend Feature Parity Plan
**Last Updated:** 2024-12-10
**Status:** ✅ Core Feature Parity Achieved (v0.71.0)
**Last Updated:** 2024-12-11
**Status:** ✅ Core Feature Parity Achieved (v0.71.0) | Batch 1 ✅ | Batch 2 ✅ | Batch 3 ✅ | Batch 4 ✅ | Batch 5 ✅
This document tracks the progress toward complete feature parity between the local `karaoke-gen` CLI and the cloud backend, enabling the `karaoke-gen-remote` CLI to have equivalent functionality.
### Quick Parity Summary
| Category | Supported | Needed | Parity | Notes |
|----------|-----------|--------|--------|-------|
| Core Processing | 12 | 12 | **100%** | ✅ Complete |
| Distribution | 5 | 5 | **100%** | ✅ Complete |
| Lyrics Configuration | 5 | 5 | **100%** | ✅ **Batch 1 Complete** (L1-L4) |
| Style Configuration | 1 | 4 | **25%** | S2-S4 are MEDIUM priority |
| Workflow Control | 0 | 3 | **0%** | W1/W2 HIGH, W7 MEDIUM (skip flags deferred) |
| Audio Processing | 4 | 4 | **100%** | ✅ **Batch 2 & 3 Complete** (AP1-AP3, AP5) |
| Input Modes | 3 | 3 | **100%** | ✅ **Batch 4 & 5 Complete** (P2, P3) |
| Audio Fetching | 1 | 1 | **100%** | ✅ **Batch 5 Complete** (A1) |
| Flags to Remove | - | - | - | I3-I6, AP6 (simplify codebase) |
**Overall:** Core workflow complete. See [CLI Parameter Parity Analysis](#-cli-parameter-parity-analysis) for detailed parameter list with implementation plans.
**Priority Summary:**
- **HIGH**: L1-L4 ✅, AP1-AP5 ✅, P3 ✅, A1 ✅ | P2 (YouTube URLs), W1/W2 (prep/finalise phases), F15 (keep-brand-code)
- **MEDIUM**: S2-S4 (style overrides/video bg), W7 (edit-lyrics), F14 (email template)
- **LOW/DEFERRED**: W3-W6 (skip flags), D2-D3 (debugging)
- **REMOVE**: I3, I4, I5, I6, AP6 (unnecessary options)
---
## 🎉 Milestone: First Successful End-to-End Run
@@ -39,6 +61,521 @@ karaoke-gen-remote \
---
## 📊 CLI Parameter Parity Analysis
This section provides a comprehensive comparison of all CLI parameters between `karaoke-gen` (local) and `karaoke-gen-remote`, including detailed explanations of what each parameter does and how it could work remotely.
**Legend:**
- ✅ Fully supported
- ⚠️ Partially supported (limited functionality)
- ❌ Not supported (ignored or errors)
- 🔹 Remote-only (not applicable to local CLI)
- N/A Not applicable to this mode
---
### Positional Arguments
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| P1 | `<file>` | ✅ | ✅ | **Local audio file path.** Copies file to output directory, converts to WAV for processing. Supports MP3, FLAC, WAV, M4A, OGG, AAC. |
| P2 | `<url>` (YouTube) | ✅ | ✅ | **YouTube/online URL.** Uses yt-dlp to download audio and extract metadata (artist/title from video title). |
| P3 | `<artist> <title>` | ✅ | ✅ | **Audio search mode.** When no file is provided, uses flacfetch to search Deezer/Tidal/etc for high-quality audio and downloads it. |
| P4 | `<folder>` | ✅ | ❌ | **Batch folder processing.** Process all audio files in a folder, using `--filename_pattern` to extract track titles from filenames. |
**Remote Implementation Status:** ✅ **BATCH 4 & 5 COMPLETE** (v0.71.42)
- **P2 YouTube URLs**: ✅ IMPLEMENTED. Backend accepts URL via `/api/jobs/create-from-url` endpoint. Uses yt-dlp to download audio and extract metadata (artist/title). Remote CLI detects URLs and routes to URL endpoint.
- **P3 Flacfetch search**: ✅ IMPLEMENTED. Remote CLI submits artist+title, backend searches via flacfetch. Returns results for interactive selection (or auto-selects with `-y`/`--auto-download`). New job statuses: `searching_audio`, `awaiting_audio_selection`, `downloading_audio`.
- **P4 Folder batch**: LOW PRIORITY. Could accept ZIP upload, process each as separate job. Defer for now.
---
### Workflow Control
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| W1 | `--prep-only` | ✅ | ❌ | **Stop after preparation phase.** Downloads audio, separates stems, transcribes lyrics, creates title/end screens, but does NOT run finalisation (encoding, YouTube upload, etc.). Useful for reviewing intermediate outputs before committing to final render. |
| W2 | `--finalise-only` | ✅ | ❌ | **Run only finalisation phase.** Must be run from a directory that was previously prepared with `--prep-only`. Picks up where prep left off: encodes videos, creates CDG/TXT packages, uploads to YouTube/Dropbox, etc. |
| W3 | `--skip-transcription` | ✅ | ❌ | **Skip automatic lyrics transcription.** Skips the AudioShake transcription and auto-correction steps. Use this if you want to manually provide lyrics or if transcription keeps failing. Lyrics review UI will still open if you have existing lyrics files. |
| W4 | `--skip-separation` | ✅ | ❌ | **Skip audio separation.** Skips the AI-powered stem separation (Modal API). Useful when re-processing a track where you already have the stems from a previous run, or when combined with `--existing_instrumental`. |
| W5 | `--skip-lyrics` | ✅ | ❌ | **Skip all lyrics processing.** Skips fetching lyrics from Genius/Spotify, transcription, and review. Output will have no lyrics overlay - just instrumental video with title/end screens. |
| W6 | `--lyrics-only` | ✅ | ❌ | **Process only lyrics.** Sets `--skip-separation` and skips title/end screen generation. Useful for re-doing just the lyrics on an existing track without re-running expensive audio separation. |
| W7 | `--edit-lyrics` | ✅ | ❌ | **Edit lyrics of existing track.** Run from inside an existing track directory (e.g., `NOMAD-1234 - Artist - Title/`). Backs up existing outputs, re-runs lyrics transcription with the existing audio, then re-renders and re-uploads. Uses `--keep-brand-code` implicitly. |
| W8 | `--resume` | N/A | 🔹 | **Resume monitoring a job.** Reconnects to an existing remote job and continues monitoring its progress, handling review/instrumental selection if needed. |
| W9 | `--cancel` | N/A | 🔹 | **Cancel a running job.** Stops processing but keeps the job record in Firestore. Useful for aborting stuck jobs. |
| W10 | `--retry` | N/A | 🔹 | **Retry a failed job.** Restarts processing from the last successful checkpoint. Only works on jobs with `failed` status. |
| W11 | `--delete` | N/A | 🔹 | **Delete a job.** Permanently removes job record from Firestore and all associated files from GCS. |
| W12 | `--list` | N/A | 🔹 | **List all jobs.** Shows all jobs with their status, artist, title. Supports filtering with `--filter-environment` and `--filter-client-id`. |
**Remote Implementation Plan:**
- **W1 `--prep-only`**: HIGH PRIORITY. Remote CLI submits job with `prep_only=true`. Backend runs all prep tasks (audio separation, lyrics transcription with review UI, initial render). After completion, CLI downloads the entire output folder and exits. No finalisation (encoding, YouTube, Dropbox, etc.) runs. Use case: Client needs custom edits to stems or lyrics before final render.
- **W2 `--finalise-only`**: HIGH PRIORITY. Remote CLI checks current directory for expected prep output files. Uploads ALL files from that folder to backend (including any manual customizations user made since prep). Backend runs finalisation and distribution steps. Use case: User edited instrumental manually, now wants cloud to handle encoding/distribution.
- **W3-W6 Skip flags**: LOW PRIORITY. Rarely used; typically all together to regenerate title/end screens with different artist/title. Defer until pipeline refactor provides cleaner stage-targeting.
- **W7 `--edit-lyrics`**: MEDIUM PRIORITY. Re-runs lyrics worker with existing audio/stems. Creates NEW YouTube upload (doesn't delete/replace existing). Backend should handle gracefully if video with identical title already exists (user will have manually deleted old one). Use case: Customer found typo in published video.
---
### Logging & Debugging
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| D1 | `--log_level` | ✅ | ✅ | **Set logging verbosity.** Options: `debug`, `info`, `warning`, `error`. Default: `info`. Affects console output detail level. |
| D2 | `--dry_run` | ✅ | ❌ | **Simulate without changes.** Runs through the workflow logic but doesn't actually download, process, or create files. Useful for testing argument combinations. |
| D3 | `--render_bounding_boxes` | ✅ | ❌ | **Debug text positioning.** Renders red bounding boxes around text regions in title/end screen images. Helps debug custom style configurations. |
**Remote Implementation Plan:**
- **D2 `--dry_run`**: LOW PRIORITY. Backend could accept `dry_run=true`, validate inputs and return what WOULD happen without processing. Nice to have for testing.
- **D3 `--render_bounding_boxes`**: LOW PRIORITY. Backend could enable this, useful for testing styles before full render. Nice to have for debugging style configs.
---
### Input/Output Configuration
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| I1 | `--filename_pattern` | ✅ | N/A | **Regex for batch processing.** Python regex with named group `(?P<title>...)` to extract track title from filenames. Used with folder input. Example: `'(?P<index>\d+) - (?P<title>.+).mp3'` |
| I2 | `--output_dir` | ✅ | ✅ | **Output directory.** Where to write output files. Local: creates subdirectory per track. Remote: where to download completed files. Default: current directory. |
| I3 | `--no_track_subfolders` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Flat output structure. Not needed - always create subfolders. |
| I4 | `--lossless_output_format` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Stem audio format. FLAC is always fine, no need for WAV option. |
| I5 | `--output_png` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Always output both PNG and JPG, no need for flag. |
| I6 | `--output_jpg` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Always output both PNG and JPG, no need for flag. |
**Remote Implementation Plan:**
- **I3, I4, I5, I6**: REMOVE from local CLI entirely. Not worth maintaining. Backend always uses FLAC, outputs both PNG+JPG, creates subfolders.
---
### Audio Fetching Configuration
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| A1 | `--auto-download` | ✅ | ✅ | **Auto-select audio source.** When using artist+title search mode (flacfetch), automatically select the best quality source instead of prompting for manual selection. Useful for scripted/batch processing. |
**Remote Implementation Status:** ✅ **BATCH 5 COMPLETE** (v0.71.28)
- **A1 `--auto-download`**: ✅ IMPLEMENTED. Backend supports BOTH modes:
- **Interactive (default)**: API returns search results to client. Client displays options, user selects via `/api/audio-search/{job_id}/select` endpoint.
- **Auto-download (`--auto-download` or `-y`)**: Backend auto-selects best quality source using flacfetch's ranking.
---
### Audio Processing Configuration
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| AP1 | `--clean_instrumental_model` | ✅ | ✅ | **Stage 1 separation model.** AI model for initial vocal/instrumental separation. Default: `model_bs_roformer_ep_317_sdr_12.9755.ckpt`. Different models have different quality/speed tradeoffs. |
| AP2 | `--backing_vocals_models` | ✅ | ✅ | **Stage 2 separation model(s).** AI model(s) for separating lead vocals from backing vocals. Default: `mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt`. |
| AP3 | `--other_stems_models` | ✅ | ✅ | **Multi-stem separation model.** AI model for separating drums, bass, guitar, piano, other. Default: `htdemucs_6s.yaml`. |
| AP4 | `--model_file_dir` | ✅ | N/A | **Model cache directory.** Where to store/load AI model files. Default: `/tmp/audio-separator-models/`. |
| AP5 | `--existing_instrumental` | ✅ | ✅ | **Use pre-made instrumental.** Path to an existing instrumental file to use instead of running AI separation. Useful when you have a better instrumental from another source (official instrumental release, manual editing, etc.). Backend still runs separation for stems, but uses provided instrumental for final video. |
| AP6 | `--instrumental_format` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** FLAC is always fine, no need for format option. |
**Remote Implementation Status:** ✅ **BATCH 2 & 3 COMPLETE** (PR #9 v0.71.24, PR #10 v0.71.26)
- **AP1 `--clean_instrumental_model`**: ✅ IMPLEMENTED. Sent as form field to backend. Audio worker passes to Modal API.
- **AP2 `--backing_vocals_models`**: ✅ IMPLEMENTED. Sent as comma-separated list, parsed and passed to Modal API.
- **AP3 `--other_stems_models`**: ✅ IMPLEMENTED. Sent as comma-separated list, parsed and passed to Modal API.
- **AP4 `--model_file_dir`**: N/A for remote - Modal API handles model caching internally.
- **AP5 `--existing_instrumental`**: ✅ IMPLEMENTED (Batch 3). User uploads instrumental file with job. Backend validates duration match (±0.5s), still runs separation for stems, but uses provided instrumental for final video encoding.
- **AP6**: REMOVE from local CLI - FLAC is always fine.
---
### Lyrics Configuration
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| L1 | `--lyrics_artist` | ✅ | ✅ | **Override artist for lyrics search.** Use different artist name when searching Genius/Spotify/Musixmatch for lyrics. Useful for covers, remixes, or when artist name differs in lyrics databases. Example: `--lyrics_artist="The Beatles"` when your file says "Beatles, The". |
| L2 | `--lyrics_title` | ✅ | ✅ | **Override title for lyrics search.** Use different song title when searching for lyrics. Useful for songs with alternate titles, subtitle variations, or "(Remastered)" suffixes. Example: `--lyrics_title="Hey Jude"` when file says "Hey Jude - 2009 Remaster". |
| L3 | `--lyrics_file` | ✅ | ✅ | **Use existing lyrics file.** Path to a text file containing lyrics to use instead of fetching from online sources. Supports TXT, DOCX, RTF formats. Useful when online lyrics are wrong or unavailable. Transcription still runs to generate timestamps. |
| L4 | `--subtitle_offset_ms` | ✅ | ✅ | **Adjust subtitle timing.** Shift all subtitle timestamps by N milliseconds. Positive values delay subtitles, negative values advance them. Useful when audio has intro padding or sync drift. Example: `--subtitle_offset_ms=500` delays by 0.5 seconds. |
| L5 | `--skip_transcription_review` | ✅ | ⚠️ | **Skip review UI.** Don't open the browser-based lyrics review interface after transcription. Use existing auto-corrected lyrics as-is. Remote: use `-y` flag instead for non-interactive mode. |
**Remote Implementation Status:** ✅ **BATCH 1 COMPLETE** (PR #8, v0.71.22)
- **L1 `--lyrics_artist`**: ✅ IMPLEMENTED. Sent as form field to backend. Lyrics worker uses for Genius/Spotify/AudioShake queries.
- **L2 `--lyrics_title`**: ✅ IMPLEMENTED. Sent as form field to backend. Lyrics worker uses for search instead of main title.
- **L3 `--lyrics_file`**: ✅ IMPLEMENTED. Uploaded with job, stored in GCS. Lyrics worker reads it instead of fetching online.
- **L4 `--subtitle_offset_ms`**: ✅ IMPLEMENTED. Sent to backend, render worker applies offset when burning subtitles.
- **L5 `--skip_transcription_review`**: Already works via `-y` flag which auto-accepts corrections.
---
### Style Configuration
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| S1 | `--style_params_json` | ✅ | ✅ | **Style configuration file.** Path to JSON file defining visual style: background images, fonts, colors, text positioning for intro/karaoke/end screens and CDG. All referenced image/font files are auto-uploaded in remote mode. |
| S2 | `--style_override` | ✅ | ❌ | **Quick style tweaks.** Override specific style parameters without editing JSON. Can be used multiple times. Format: `section.key=value`. Example: `--style_override 'intro.background_image=/path/to/new_bg.png'`. Creates a temp JSON file with overrides merged. |
| S3 | `--background_video` | ✅ | ❌ | **Video background for karaoke.** Path to video file to use as animated background instead of static image for the karaoke section. Video is looped/trimmed to match audio duration, with lyrics overlaid. |
| S4 | `--background_video_darkness` | ✅ | ❌ | **Dim video background.** Darkness overlay percentage (0-100) applied to video background so lyrics are more readable. 0 = no darkening, 100 = completely black. Default: 0. |
**Remote Implementation Plan:**
- **S2 `--style_override`**: MEDIUM PRIORITY. Send override strings to backend, merge server-side before passing to workers. Works same as local (creates merged config). Use case: Web UI will let users tweak font colors etc., needs to pass adjustments to backend.
- **S3 `--background_video`**: MEDIUM PRIORITY. Upload video file with job. Backend processes with VideoBackgroundProcessor (loop/trim to audio duration). Requires larger file upload limits. Future: Web UI could allow YouTube URL or direct upload for video background.
- **S4 `--background_video_darkness`**: Depends on S3. Send percentage to backend, apply overlay during video processing.
---
### Finalisation Configuration
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| F1 | `--enable_cdg` | ✅ | ✅ | **Generate CDG+MP3 package.** Create CD+Graphics karaoke format files (used by karaoke machines). Produces ZIP containing `.cdg` and `.mp3` files. Requires `cdg` section in style_params_json. |
| F2 | `--enable_txt` | ✅ | ✅ | **Generate TXT+MP3 package.** Create text-based karaoke format with timed lyrics. Produces ZIP containing `.txt` (with timestamps) and `.mp3` files. |
| F3 | `--brand_prefix` | ✅ | ✅ | **Brand code prefix.** Your brand identifier for sequential numbering. When set, output folder is renamed to `PREFIX-XXXX - Artist - Title` where XXXX is the next sequential number found in `organised_dir` or Dropbox. Example: `NOMAD` produces `NOMAD-1234`. |
| F4 | `--organised_dir` | ✅ | N/A | **Local organized output.** Local filesystem path where final track folders are moved after processing. Used with `--brand_prefix` to maintain organized library. Example: `/Volumes/Media/Karaoke/Tracks-Organized/`. |
| F5 | `--organised_dir_rclone_root` | ✅ | ✅ | **Rclone path for Dropbox.** Rclone remote path that maps to your organised_dir. Used to calculate next brand code number and for rclone-based uploads. Example: `dropbox:Media/Karaoke/Tracks-Organized`. |
| F6 | `--public_share_dir` | ✅ | N/A | **Local public share folder.** Local filesystem path where shareable versions (720p video, CDG, TXT) are copied for public distribution. |
| F7 | `--enable_youtube_upload` | ✅ | ✅ | **Upload to YouTube.** After encoding, upload the lossy 4K video to YouTube. Local: uses OAuth flow with client secrets. Remote: uses server-side stored credentials. |
| F8 | `--youtube_client_secrets_file` | ✅ | N/A | **YouTube OAuth credentials.** Local path to Google OAuth client secrets JSON file for YouTube API. Only needed for local mode - remote uses server-stored credentials. |
| F9 | `--youtube_description_file` | ✅ | ✅ | **YouTube video description.** Path to text file containing YouTube video description template. Supports placeholders like `{artist}`, `{title}`, `{brand_code}`. |
| F10 | `--rclone_destination` | ✅ | N/A | **Rclone sync destination.** Rclone remote path to sync public_share_dir to. Used for Google Drive public sharing. Example: `googledrive:KaraokePublic`. |
| F11 | `--dropbox_path` | N/A | 🔹 | **Dropbox folder (native API).** Remote-only: Dropbox folder path for organized output upload using native Dropbox API. Example: `/Karaoke/Tracks-Organized`. |
| F12 | `--gdrive_folder_id` | N/A | 🔹 | **Google Drive folder (native API).** Remote-only: Google Drive folder ID for public share uploads using native API. Example: `1abc123xyz`. |
| F13 | `--discord_webhook_url` | ✅ | ✅ | **Discord notification.** Webhook URL for sending completion notifications to Discord. Posts message with track info, brand code, and links. |
| F14 | `--email_template_file` | ✅ | ❌ | **Email draft template.** Path to email template file for creating Gmail drafts with track info. Supports placeholders. Uses pyperclip for manual email composition. |
| F15 | `--keep-brand-code` | ✅ | ❌ | **Preserve existing brand code.** When run from an existing track directory (with brand code in folder name), use that brand code instead of calculating a new one. Implicitly enabled with `--edit-lyrics`. |
| F16 | `-y` / `--yes` | ✅ | ✅ | **Non-interactive mode.** Auto-accept all prompts: lyrics corrections, instrumental selection, confirmations. Useful for automated/CI pipelines. Remote: auto-completes review, selects clean instrumental. |
| F17 | `--test_email_template` | ✅ | N/A | **Test email template.** Debug mode: test the email template rendering with fake data without processing any track. |
**Remote Implementation Plan:**
- **F14 `--email_template_file`**: MEDIUM PRIORITY. Two behaviors needed:
1. **Finalisation**: Same as local - use template to create Gmail draft (requires Gmail API OAuth). User just changes recipient and sends.
2. **API endpoint**: Expose endpoint to fetch templated message text for a job. Use case: Web frontend button to copy message for pasting into Fiverr, etc.
- **F15 `--keep-brand-code`**: HIGH PRIORITY (for `--finalise-only` workflow). When using `--finalise-only` remotely, user may be re-processing an already-published track. Must preserve existing brand code so updated files overwrite in Dropbox/GDrive rather than creating new sequence number. Implementation: Remote CLI reads brand code from local folder name, sends to backend.
---
### Remote CLI Specific Options
| # | Parameter | Local | Remote | Description |
|---|-----------|-------|--------|-------------|
| R1 | `--service-url` | N/A | 🔹 | **Backend service URL.** URL of the karaoke-gen cloud backend. Can also be set via `KARAOKE_GEN_URL` environment variable. Required for remote mode. |
| R2 | `--review-ui-url` | N/A | 🔹 | **Lyrics review UI URL.** URL of the hosted lyrics review web application. Default: `https://lyrics.nomadkaraoke.com`. |
| R3 | `--poll-interval` | N/A | 🔹 | **Status poll frequency.** Seconds between job status checks while monitoring. Default: 5 seconds. |
| R4 | `--environment` | N/A | 🔹 | **Job environment tag.** Tag jobs with environment label (test/production/development) for filtering and cleanup. Sent as `X-Environment` header. |
| R5 | `--client-id` | N/A | 🔹 | **Client identifier tag.** Tag jobs with client/user identifier for filtering. Sent as `X-Client-ID` header. |
| R6 | `--filter-environment` | N/A | 🔹 | **Filter by environment.** When using `--list` or `--bulk-delete`, only show/delete jobs matching this environment. |
| R7 | `--filter-client-id` | N/A | 🔹 | **Filter by client ID.** When using `--list` or `--bulk-delete`, only show/delete jobs matching this client ID. |
| R8 | `--bulk-delete` | N/A | 🔹 | **Bulk delete jobs.** Delete all jobs matching filter criteria. Requires at least one filter (`--filter-environment` or `--filter-client-id`). |
---
## 🚀 Implementation Batches
The following batches are ordered by priority and grouped by similar functionality. Each batch is designed to be a self-contained unit of work that can be tackled by a single agent session.
### 📋 Instructions for Implementing a Batch
**Before starting a batch, ensure you follow these steps:**
#### 1. Test Coverage Requirements
All code changes must have **>70% test coverage** on modified files. Run:
```bash
python -m pytest backend/tests/ --cov=backend.models.job --cov=backend.api.routes.file_upload --cov=backend.workers.<worker_name> --cov-report=term-missing
```
**Required tests for each batch:**
- **Model changes:** Test new fields exist, have correct defaults, serialize properly
- **API changes:** Test form validation (valid/invalid inputs), file upload handling
- **Worker changes:** Test parameter passing, configuration, error handling
- **CLI changes:** Test submit_job() passes parameters correctly
#### 2. Documentation Updates (REQUIRED)
As part of your PR, you MUST update this document:
- [ ] Update the **Quick Parity Summary** table to reflect new parity percentages
- [ ] Mark parameters as ✅ in the relevant **CLI Parameter Parity Analysis** section
- [ ] Update **Remote Implementation Plan/Status** to show ✅ IMPLEMENTED
- [ ] Mark the batch as **✅ COMPLETE** with PR number in the **Summary Table**
- [ ] Add entry to **Recently Completed** table with version and PR number
- [ ] Remove items from **Not Yet Implemented** list
#### 3. Version Bump
Bump the patch version in `pyproject.toml` before committing.
#### 4. PR Requirements
- Title: `feat: Batch N - <description>`
- Include: Summary of changes, files modified, tests added
- Link to this document in PR description
---
### Batch 1: Lyrics Configuration (HIGH) ✅ COMPLETE
**Parameters:** L1 `--lyrics_artist`, L2 `--lyrics_title`, L3 `--lyrics_file`, L4 `--subtitle_offset_ms`
**Status:** ✅ **IMPLEMENTED** - PR #8 merged (v0.71.22)
**Implementation Summary:**
- Remote CLI: Added 4 form fields to `submit_job()`, uploads lyrics file if provided
- Backend: Added 4 Form parameters to `/api/jobs/upload` endpoint
- Backend: Lyrics worker downloads user lyrics file, passes overrides to LyricsProcessor
- Backend: Render worker applies subtitle offset via OutputConfig
**Files modified:**
- `karaoke_gen/utils/remote_cli.py` - submit_job() with lyrics params
- `backend/api/routes/file_upload.py` - Form fields + lyrics file upload
- `backend/models/job.py` - lyrics_artist, lyrics_title, lyrics_file_gcs_path, subtitle_offset_ms fields
- `backend/workers/lyrics_worker.py` - override values passed to LyricsProcessor
- `backend/workers/render_video_worker.py` - subtitle_offset_ms applied
**Tests added:** 25+ unit tests covering lyrics file validation, Job model fields, worker configuration
**Complexity:** Low-Medium (straightforward form fields + file upload)
---
### Batch 2: Audio Separation Model Selection (HIGH) ✅ COMPLETE
**Parameters:** AP1 `--clean_instrumental_model`, AP2 `--backing_vocals_models`, AP3 `--other_stems_models`
**Status:** ✅ **IMPLEMENTED** - PR #9 merged (v0.71.24)
**Implementation Summary:**
- Remote CLI: Added 3 form fields to `submit_job()`, sends models as comma-separated strings
- Backend: Added 3 Form parameters to `/api/jobs/upload` endpoint, parses comma-separated lists
- Backend: Job model stores model configuration as optional fields
- Backend: Audio worker passes model names to AudioProcessor which uses Modal API
**Files modified:**
- `karaoke_gen/utils/remote_cli.py` - submit_job() with model params
- `backend/api/routes/file_upload.py` - Form fields + comma parsing
- `backend/models/job.py` - clean_instrumental_model, backing_vocals_models, other_stems_models fields
- `backend/workers/audio_worker.py` - create_audio_processor() accepts model params from job
**Tests added:** 15+ unit tests covering Job model fields, form parsing, AudioProcessor configuration
**Complexity:** Low (Modal API already supports dynamic model selection)
---
### Batch 3: Existing Instrumental Support (HIGH) ✅ COMPLETE
**Parameters:** AP5 `--existing_instrumental`
**Status:** ✅ **IMPLEMENTED** - PR #10 merged (v0.71.26)
**Implementation Summary:**
- Remote CLI: Added `--existing_instrumental` parameter, uploads file as `existing_instrumental` type
- Backend: Added `existing_instrumental` to VALID_FILE_TYPES with audio extension validation
- Backend: Duration validation (±0.5s tolerance) at uploads-complete endpoint using pydub
- Backend: Job model has `existing_instrumental_gcs_path` field
- Backend: Video worker downloads user-provided instrumental if present, uses it for final encoding
**Files modified:**
- `karaoke_gen/utils/remote_cli.py` - submit_job() with existing_instrumental param
- `backend/api/routes/file_upload.py` - VALID_FILE_TYPES, GCS path generation, duration validation
- `backend/models/job.py` - existing_instrumental_gcs_path field
- `backend/workers/video_worker.py` - downloads and uses user-provided instrumental
**Tests added:** 20+ unit tests covering model fields, file type validation, GCS path generation, duration validation
**Complexity:** Medium (file upload + validation + conditional logic)
---
### Batch 4: YouTube URL Input (HIGH) ✅ COMPLETE
**Parameters:** P2 `<url>` (YouTube/online URL)
**Status:** ✅ **IMPLEMENTED** (v0.71.26)
**Implementation Summary:**
- Remote CLI: Detects URLs via `is_url()`, routes to `submit_job_from_url()` method
- Backend: New `/api/jobs/create-from-url` endpoint accepts URL and optional artist/title
- Backend: Audio worker downloads from URL using yt-dlp, auto-detects artist/title from metadata
- FileHandler: Added `download_video()` and `extract_metadata_from_url()` methods
- Dependencies: yt-dlp added to pyproject.toml
**Files modified:**
- `karaoke_gen/utils/remote_cli.py` - URL detection, new `submit_job_from_url()` method
- `karaoke_gen/file_handler.py` - `download_video()` and `extract_metadata_from_url()` methods
- `backend/api/routes/file_upload.py` - `CreateJobFromUrlRequest`, `CreateJobFromUrlResponse`, `/api/jobs/create-from-url` endpoint
- `backend/workers/audio_worker.py` - Enhanced `download_from_url()` with metadata extraction
- `pyproject.toml` - Added yt-dlp dependency
**Tests added:** 20+ unit tests covering URL validation, request/response models, Job URL field
**Complexity:** Medium-High (new input mode, yt-dlp integration)
---
### Batch 5: Flacfetch Audio Search (HIGH) ✅ COMPLETE
**Parameters:** P3 `<artist> <title>` (search mode), A1 `--auto-download`
**Status:** ✅ **IMPLEMENTED** - PR #13 merged (v0.71.28)
**Implementation Summary:**
- Remote CLI: Detects artist+title without file, calls `/api/audio-search/search` endpoint
- Backend: New `AudioSearchService` integrates flacfetch for searching
- Backend: New job statuses `searching_audio`, `awaiting_audio_selection`, `downloading_audio`
- Backend: Returns search results for interactive selection, or auto-selects with `auto_download=true`
- Remote CLI: Displays results interactively, user selects, sends to `/api/audio-search/{job_id}/select`
- Backend: Downloads selected audio to GCS, proceeds with normal job flow
- Support `--auto-download` / `-y` for non-interactive auto-selection
**Files modified:**
- `karaoke_gen/utils/remote_cli.py` - search flow, display results, `search_audio()`, `select_audio_source()`, `handle_audio_selection()`
- `backend/api/routes/audio_search.py` - NEW: search and select endpoints
- `backend/services/audio_search_service.py` - NEW: flacfetch integration service
- `backend/models/job.py` - New statuses and fields (`audio_search_artist`, `audio_search_title`, `auto_download`)
- `backend/main.py` - Register audio_search router
**Tests added:** 20+ unit tests covering Job model fields, AudioSearchService, state transitions
**Complexity:** High (new interactive API flow, new service integration)
---
### Batch 6: Two-Phase Workflow (HIGH) ✅ COMPLETE
**Parameters:** W1 `--prep-only`, W2 `--finalise-only`, F15 `--keep-brand-code`
**Status:** ✅ Complete (2025-12-11)
**Implementation Summary:**
- **prep-only:** Added `prep_only` field to Job model. When set, backend transitions to new `PREP_COMPLETE` status after review and video rendering (with_vocals.mkv). CLI downloads all prep outputs.
- **finalise-only:** Added new endpoint `/api/jobs/create-finalise-only` for uploading prep outputs. Remote CLI detects files in prep folder, uploads them via signed URLs, and monitors finalisation.
- **keep-brand-code:** Remote CLI extracts brand code from folder name (e.g., "NOMAD-1234 - Artist - Title"), sends to backend. Video worker uses preserved brand code for distribution.
**Files modified:**
- `backend/models/job.py` - Added `PREP_COMPLETE` status, `prep_only`, `finalise_only`, `keep_brand_code` fields, updated state transitions
- `backend/api/routes/file_upload.py` - Added `prep_only`, `keep_brand_code` to job creation, new finalise-only endpoints
- `backend/workers/render_video_worker.py` - Conditional transition to PREP_COMPLETE for prep-only jobs
- `backend/workers/video_worker.py` - Use preserved brand code when set
- `backend/services/job_manager.py` - Pass new fields through job creation
- `karaoke_gen/utils/remote_cli.py` - Full implementation of prep-only, finalise-only, and keep-brand-code flows
**Tests added:**
- `backend/tests/test_models.py` - Tests for new fields and PREP_COMPLETE status
- `backend/tests/test_file_upload.py` - Tests for finalise-only models and endpoints
**Complexity:** High (significant workflow changes, new upload flow)
---
### Batch 7: Style Overrides (MEDIUM)
**Parameters:** S2 `--style_override`
**Scope:**
- Remote CLI: Parse override strings, send as array to backend
- Backend: Accept style_override array
- Backend: Merge overrides with uploaded style JSON before processing
- Handle asset references in overrides (upload referenced files)
**Files to modify:**
- `karaoke_gen/utils/remote_cli.py` - parse and send overrides
- `backend/api/routes/file_upload.py` - accept overrides
- `backend/services/style_service.py` or similar - merge logic
**Complexity:** Medium (string parsing, merge logic, asset handling)
---
### Batch 8: Video Background (MEDIUM)
**Parameters:** S3 `--background_video`, S4 `--background_video_darkness`
**Scope:**
- Remote CLI: Upload video file (larger file support needed)
- Backend: Accept video upload, store in GCS
- Backend: Render worker uses VideoBackgroundProcessor
- Backend: Apply darkness overlay
**Files to modify:**
- `karaoke_gen/utils/remote_cli.py` - upload video file
- `backend/api/routes/file_upload.py` - accept video upload (larger limits)
- `backend/workers/render_worker.py` - integrate VideoBackgroundProcessor
**Complexity:** Medium (large file upload, video processing)
---
### Batch 9: Edit Lyrics Mode (MEDIUM)
**Parameters:** W7 `--edit-lyrics`
**Scope:**
- Remote CLI: Accept job_id for existing job, trigger edit mode
- Backend: Load existing job's audio/stems from GCS
- Backend: Re-run lyrics worker only
- Backend: Re-render with new lyrics
- Backend: Create NEW YouTube upload (handle duplicate title gracefully)
**Files to modify:**
- `karaoke_gen/utils/remote_cli.py` - edit mode flow
- `backend/api/routes/` - edit endpoint or flag
- `backend/workers/orchestrator.py` - partial re-processing logic
**Complexity:** Medium-High (partial re-processing, state management)
---
### Batch 10: Email Template + Cleanup (LOW)
**Parameters:** F14 `--email_template_file`, D2 `--dry_run`, D3 `--render_bounding_boxes`
**Removal:** I3 `--no_track_subfolders`, I4 `--lossless_output_format`, I5 `--output_png`, I6 `--output_jpg`, AP6 `--instrumental_format`
**Scope:**
- **Email template:** Upload template, backend creates Gmail draft + new API endpoint to fetch formatted text
- **Dry run:** Backend validates inputs, returns plan without executing
- **Bounding boxes:** Enable debug rendering in screens worker
- **Removal:** Remove 5 unnecessary CLI flags from local CLI
**Files to modify:**
- `karaoke_gen/utils/cli_args.py` - remove flags
- `karaoke_gen/utils/remote_cli.py` - email template upload
- `backend/api/routes/` - email template endpoint
- `backend/services/email_service.py` - Gmail API integration
**Complexity:** Mixed (Gmail OAuth is complex, removals are simple)
---
### Summary Table
| Batch | Parameters | Priority | Complexity | Status |
|-------|------------|----------|------------|--------|
| 1 | L1-L4 (lyrics config) | HIGH | Low-Medium | ✅ **COMPLETE** (PR #8) |
| 2 | AP1-AP3 (model selection) | HIGH | Low | ✅ **COMPLETE** (PR #9) |
| 3 | AP5 (existing instrumental) | HIGH | Medium | ✅ **COMPLETE** (PR #10) |
| 4 | P2 (YouTube URLs) | HIGH | Medium-High | ✅ **COMPLETE** |
| 5 | P3, A1 (flacfetch search) | HIGH | High | ✅ **COMPLETE** (PR #13) |
| 6 | W1, W2, F15 (two-phase) | HIGH | High | ⏳ Pending |
| 7 | S2 (style overrides) | MEDIUM | Medium | ⏳ Pending |
| 8 | S3, S4 (video background) | MEDIUM | Medium | ⏳ Pending |
| 9 | W7 (edit-lyrics) | MEDIUM | Medium-High | ⏳ Pending |
| 10 | F14, D2, D3, removals | LOW | Mixed | ⏳ Pending |
**Recommended Order:** ~~1~~ → ~~2~~ → ~~3~~ → ~~4~~ → ~~5~~ → 6 → 7 → 8 → 9 → 10
Batches 1-4 are quick wins that add significant value. Batches 5-6 are larger but high priority. Batches 7-10 can be done as time permits.
---
## ✅ Implementation Decisions (Answered 2024-12-10)
All questions have been answered and incorporated into the "Remote Implementation Plan" sections above. Summary of key decisions:
### Workflow Control
- **W1/W2**: YES, two-phase workflow needed. `--prep-only` downloads all prep outputs after review, `--finalise-only` uploads local folder (with any manual edits) for cloud finalisation.
- **W3-W6 Skip flags**: LOW PRIORITY - rarely used, defer until pipeline refactor provides cleaner stage-targeting.
- **W7 Edit-lyrics**: Creates NEW YouTube upload (user deletes old one manually). Backend handles duplicate title gracefully.
### Input Modes
- **P2 YouTube**: HIGH PRIORITY - needed for niche live versions only on YouTube.
- **P3 Flacfetch**: HIGH PRIORITY - MUST be interactive via API (not auto-select) for future web UI audio source picker.
### Audio Processing
- **AP1-AP3 Models**: HIGH PRIORITY - all model options must work remotely (Modal API can load any model on demand).
- **AP5 Existing Instrumental**: Still run separation (for consistent stem downloads), but use provided instrumental for final video. Validate duration match (±0.5s) at job start before proceeding.
### I/O & Audio Format Options
- **I3, I4, I5, I6, AP6**: REMOVE from local CLI entirely - unnecessary options, simplifies codebase.
### Audio Fetching
- **A1 `--auto-download`**: Backend must support BOTH interactive (default, for web UI) AND auto-select (with `-y` flag) modes.
### Style
- **S2 Style Override**: MEDIUM PRIORITY - needed for web UI style tweaks (font colors, etc.).
- **S3/S4 Background Video**: MEDIUM PRIORITY - implement, will eventually allow YouTube URL or direct upload in web UI.
### Finalisation
- **F14 Email Template**: BOTH - create Gmail draft (like local) AND expose API endpoint to fetch templated text (for Fiverr copy-paste).
- **F15 Keep-brand-code**: HIGH PRIORITY for `--finalise-only` workflow - preserves brand code so updates overwrite existing Dropbox/GDrive files.
---
## Vision
The karaoke-gen system supports multiple interfaces to the same core functionality:
@@ -105,14 +642,27 @@ The karaoke-gen system supports multiple interfaces to the same core functionali
| Google Drive upload | ✅ | Service account credentials |
| Discord notification | ✅ | Webhook URL from job or default |
### ⏳ Not Yet Implemented (Lower Priority)
### ✅ Recently Completed
| Feature | Version | PR | Notes |
|---------|---------|-----|-------|
| Flacfetch audio search | v0.71.42 | #13 | `artist title` (search mode), `--auto-download` - search and download audio by artist+title |
| YouTube URL input | v0.71.26 | - | `<url>` positional argument for YouTube/online URLs |
| Existing instrumental | v0.71.26 | #10 | `--existing_instrumental` - upload and use user-provided instrumental |
| Audio model selection | v0.71.24 | #9 | `--clean_instrumental_model`, `--backing_vocals_models`, `--other_stems_models` |
| Lyrics override params | v0.71.22 | #8 | `--lyrics_artist`, `--lyrics_title`, `--lyrics_file`, `--subtitle_offset_ms` |
### ⏳ Not Yet Implemented
| Feature | Priority | Notes |
|---------|----------|-------|
| Style override | MEDIUM | `--style_override` for quick tweaks |
| Skip separation/transcription | MEDIUM | Workflow control for re-processing |
| Edit lyrics mode | MEDIUM | `--edit-lyrics` for fixing existing tracks |
| Background video | MEDIUM | `--background_video` requires video upload |
| Two-phase workflow | HIGH | `--prep-only`, `--finalise-only` |
| Batch folder processing | LOW | Process multiple files at once |
| Gmail draft creation | LOW | Nice-to-have, not blocking |
| YouTube URL input | LOW | Requires yt-dlp in container |
| Batch processing | LOW | Queue management needed |
| Existing instrumental support | LOW | For re-processing existing tracks |
---
@@ -168,96 +718,17 @@ karaoke-gen-remote \
---
## Future Architecture: Shared Pipeline
## Future Architecture
Once feature parity is achieved, the codebase should be refactored toward a **shared pipeline architecture** where both local and remote execution use the same abstractions.
For the comprehensive architecture plan covering both **horizontal scalability** (Cloud Tasks, Cloud Run Jobs) and **software architecture** (shared pipeline, SOLID, DRY), see:
### Current State (Divergent Paths)
**[SCALABLE-ARCHITECTURE-PLAN.md](./SCALABLE-ARCHITECTURE-PLAN.md)**
```
LOCAL CLI CLOUD BACKEND
───────── ─────────────
KaraokeGen.process() API Routes
│ │
├─► AudioProcessor ├─► audio_worker.py
│ └── Modal API or local │ └── Modal API
│ │
├─► LyricsProcessor ├─► lyrics_worker.py
│ └── Orchestrates everything │ └── Transcription only
│ including video generation │
│ ├─► screens_worker.py
│ │
│ ├─► render_video_worker.py
│ │ └── OutputGenerator directly
│ │
└─► KaraokeFinalise └─► video_worker.py
└── Encoding, distribution └── KaraokeFinalise
```
**Problems:**
- Video generation called differently (via LyricsProcessor vs OutputGenerator directly)
- LyricsProcessor does too many things (fetching, transcription, video, file management)
- Testing requires mocking different things for local vs remote
- Bug fixes may need to be applied in multiple places
### Target State (Shared Pipeline)
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ SHARED PIPELINE ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ AudioInput → Separation → Transcription → Review → Render → Finalize │
│ │ │ │ │ │ │ │
│ ▼ ▼ ▼ ▼ ▼ ▼ │
│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │
│ │ Stage │ │ Stage │ │ Stage │ │ Stage │ │ Stage │ │ Stage │ │
│ │ API │ │ API │ │ API │ │ API │ │ API │ │ API │ │
│ └────┬───┘ └────┬───┘ └────┬───┘ └────┬───┘ └────┬───┘ └────┬───┘ │
│ │ │ │ │ │ │ │
│ ┌────┴───────────┴────────────┴────────────┴──────────┴──────────┴────┐ │
│ │ EXECUTION LAYER │ │
│ │ │ │
│ │ Local Mode: │ Remote Mode: │ │
│ │ - Direct calls │ - HTTP to backend │ │
│ │ - Local GPU/CPU │ - Workers + Modal │ │
│ │ - Blocking │ - Async + polling │ │
│ └──────────────────────┴───────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
**Benefits:**
- Single source of truth for each pipeline stage
- Each stage independently testable
- Same business logic regardless of execution mode
- Easier to add new stages or modify existing ones
- DRY - fix bugs once, works everywhere
### Implementation Plan for Shared Pipeline
This refactor should be done in a separate branch after the v0.71.0 release:
1. **Extract stage interfaces**
```python
class PipelineStage(Protocol):
async def execute(self, context: PipelineContext) -> StageResult:
...
```
2. **Create execution adapters**
```python
class LocalExecutor:
"""Runs stages directly in-process"""
class RemoteExecutor:
"""Runs stages via backend API/workers"""
```
3. **Refactor incrementally**
- Start with one stage (e.g., Separation)
- Prove the pattern works
- Migrate other stages
Key goals:
- **100+ concurrent jobs** with no performance degradation
- **Dedicated resources per job** (no CPU/memory contention)
- **Shared pipeline architecture** - single source of truth for business logic
- **DRY codebase** - bug fixes apply to both local and remote execution
---
@@ -355,6 +826,50 @@ DEFAULT_DISCORD_WEBHOOK_URL=https://discord.com/...
## Recent Changes Log
### 2024-12-11: v0.71.28 - Batch 5 Complete (Flacfetch Audio Search)
**PR #13: Flacfetch Audio Search for Remote CLI**
- ✅ `artist title` (search mode) - Search for audio by artist and title without providing a file
- ✅ `--auto-download` - Auto-select best audio source (skip interactive selection)
- New job statuses: `searching_audio`, `awaiting_audio_selection`, `downloading_audio`
- New API endpoints: `POST /api/audio-search/search`, `GET /api/audio-search/{job_id}/results`, `POST /api/audio-search/{job_id}/select`
- Interactive mode: Returns search results for user selection
- Auto mode: Uses flacfetch's quality ranking to auto-select best source
- Added 20+ unit tests for new functionality
- Files: `remote_cli.py`, `audio_search.py` (new route), `audio_search_service.py` (new service), `job.py`
### 2024-12-11: v0.71.26 - Batch 3 Complete (Existing Instrumental Support)
**PR #10: Existing Instrumental Support for Remote CLI**
- ✅ `--existing_instrumental` - Upload and use user-provided instrumental file
- Duration validation (±0.5s tolerance) before processing starts
- Backend still runs AI separation for stem downloads, but uses provided instrumental for final video
- Added 20+ unit tests for new functionality
- Files: `remote_cli.py`, `file_upload.py`, `job.py`, `video_worker.py`
### 2024-12-11: v0.71.24 - Batch 2 Complete (Audio Model Selection)
**PR #9: Audio Model Selection for Remote CLI**
- ✅ `--clean_instrumental_model` - Custom Stage 1 separation model
- ✅ `--backing_vocals_models` - Custom backing vocals separation models
- ✅ `--other_stems_models` - Custom multi-stem separation models (bass, drums, etc.)
- Added 15+ unit tests for new functionality
- Files: `remote_cli.py`, `file_upload.py`, `job.py`, `audio_worker.py`
### 2024-12-10: v0.71.22 - Batch 1 Complete (Lyrics Configuration)
**PR #8: Lyrics Configuration for Remote CLI**
- ✅ `--lyrics_artist` - Override artist name for lyrics search
- ✅ `--lyrics_title` - Override title for lyrics search
- ✅ `--lyrics_file` - Upload custom lyrics file (TXT, DOCX, RTF)
- ✅ `--subtitle_offset_ms` - Adjust subtitle timing offset
- Added 25+ unit tests for new functionality
- Files: `remote_cli.py`, `file_upload.py`, `job.py`, `lyrics_worker.py`, `render_video_worker.py`
### 2024-12-10: v0.71.0 - Core Feature Parity Complete
**First successful end-to-end remote run!**
@@ -400,14 +915,47 @@ DEFAULT_DISCORD_WEBHOOK_URL=https://discord.com/...
2. ✅ Tag v0.71.0 release
3. Update PyPI package
### Short-Term
1. Monitor production usage
2. Address any edge cases found
3. Improve error messages and logging
### Short-Term: Feature Parity (Priority Order)
### Long-Term (Shared Pipeline Refactor)
1. Create new branch for shared pipeline architecture
2. Extract stage interfaces
3. Implement execution adapters
4. Migrate stages incrementally
5. Achieve true code sharing between local and remote
1. ✅ **Lyrics Configuration** (Batch 1 - COMPLETE)
- `--lyrics_artist`, `--lyrics_title`, `--lyrics_file`, `--subtitle_offset_ms`
- PR #8, v0.71.22
2. ✅ **Audio Model Selection** (Batch 2 - COMPLETE)
- `--clean_instrumental_model`, `--backing_vocals_models`, `--other_stems_models`
- PR #9, v0.71.24
3. ✅ **Existing Instrumental** (Batch 3 - COMPLETE)
- Added `--existing_instrumental` upload support
- PR #10, v0.71.26
4. **YouTube URL Input** (Batch 4 - NEXT)
- Accept YouTube URLs as input
- Requires: yt-dlp in container
- Enables: Processing niche live versions only on YouTube
5. ✅ **Flacfetch Audio Search** (Batch 5 - COMPLETE)
- Interactive audio source selection via API
- v0.71.28
### Medium-Term: Workflow Features
1. **Two-Phase Workflow** (Batch 6) ✅ COMPLETE
- `--prep-only` and `--finalise-only` modes
3. **Style Override** (Batch 7)
- Quick style tweaks without editing JSON
4. **Background Video Support** (Batch 8)
- Upload and use video backgrounds
5. **Edit Lyrics Mode** (Batch 9)
- Allow fixing lyrics on completed jobs
### Long-Term (Scalable Architecture)
See **[SCALABLE-ARCHITECTURE-PLAN.md](./SCALABLE-ARCHITECTURE-PLAN.md)** for the comprehensive plan covering:
1. Cloud Tasks integration for horizontal scalability
2. Cloud Run Jobs for long-running video encoding
3. Shared pipeline architecture for code reuse
4. Implementation roadmap and migration strategy
@@ -0,0 +1,827 @@
# Scalable Architecture Plan
**Last Updated:** 2025-12-11
**Status:** Phase 1 Complete - Cloud Tasks Enabled
This document outlines the plan for evolving karaoke-gen's cloud backend to support **100+ concurrent jobs** with no performance degradation, while also achieving a clean, maintainable codebase with proper separation of concerns.
---
## Executive Summary
After thorough codebase analysis, I recommend a **phased approach** that prioritizes infrastructure scalability first (Cloud Tasks), followed by incremental adoption of the existing pipeline architecture for code sharing.
### Key Findings
1. **The infrastructure problem is well-isolated** - WorkerService already abstracts worker triggering behind HTTP calls
2. **Pipeline architecture exists but is underutilized** - `karaoke_gen/pipeline/` has well-designed stages and executors, but backend workers don't use them
3. **RemoteExecutor is a placeholder** - It submits jobs to the backend but doesn't execute stages remotely
4. **Workers are self-contained** - Each worker only needs `job_id` and fetches everything from Firestore/GCS
### Recommended Priority
1. **Phase 1 (HIGH): Cloud Tasks Integration** - Solves the scalability problem immediately
2. **Phase 2 (MEDIUM): Cloud Run Jobs for Video** - Removes 30-min timeout constraint
3. **Phase 3 (LOW): Pipeline Unification** - Nice-to-have for code maintainability
---
## Goals
### Infrastructure Goals
1. **Horizontal scalability** - Each job gets dedicated resources (no resource contention)
2. **Maximum processing speed** - Same speed for 1 job or 100 concurrent jobs
3. **Reliability** - Automatic retries, no lost jobs from container restarts
4. **Cost efficiency** - Scale to zero when idle, pay only for active processing
### Software Architecture Goals
1. **SOLID principles** - Single responsibility, clean interfaces
2. **DRY** - Share code between local CLI and remote backend
3. **Testability** - Each component independently testable
4. **Maintainability** - Bug fixes apply everywhere, easy to add features
---
## Current Architecture Analysis
### What Works Well
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ CURRENT ARCHITECTURE (v0.71.x) │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ✅ Heavy work already offloaded to external services: │
│ │
│ Audio Separation ──────────────────▶ Modal API (GPU, ~5-8 min) │
│ Lyrics Transcription ──────────────▶ AudioShake API (~2-3 min) │
│ │
│ ✅ State persistence in Firestore (survives restarts) │
│ ✅ Files in GCS (no local storage dependencies) │
│ ✅ Internal HTTP endpoints exist for worker triggers │
│ ✅ Workers are self-contained (only need job_id) │
│ ✅ WorkerService abstracts HTTP calls │
│ ✅ Pipeline stages exist in karaoke_gen/pipeline/ │
│ ✅ LocalExecutor for CLI, RemoteExecutor placeholder ready │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Current Code Structure
```
WorkerService (backend/services/worker_service.py)
│
└── trigger_worker(worker_type, job_id)
│
└── HTTP POST /api/internal/workers/{worker_type}
│
└── BackgroundTasks.add_task(process_xxx, job_id) ⚠️ PROBLEM
```
The **single point of change** is `WorkerService.trigger_worker()`. Currently it:
1. Makes HTTP POST to internal endpoint
2. Internal endpoint adds to FastAPI BackgroundTasks
3. BackgroundTask runs in same container (problem!)
### Current Limitations
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ CURRENT FLOW (PROBLEMATIC) │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ POST /api/jobs/upload │
│ │ │
│ ├──▶ Save job to Firestore ✅ │
│ │ │
│ └──▶ BackgroundTasks.add_task(trigger_workers) │
│ │ │
│ └──▶ HTTP POST /api/internal/workers/audio │
│ │ │
│ └──▶ BackgroundTasks.add_task(process_audio) ⚠️ │
│ │ │
│ └── Runs IN THE SAME CONTAINER │
│ as the API request handler │
│ │
│ PROBLEMS: │
│ ⚠️ Container may be killed before background tasks complete │
│ ⚠️ No guaranteed delivery - if container crashes, job is lost │
│ ⚠️ Multiple concurrent jobs share CPU/memory (resource contention) │
│ ⚠️ Long-running video encoding (15-20 min) vulnerable to timeouts │
│ ⚠️ FFmpeg renders compete for resources when jobs run in parallel │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Existing Pipeline Architecture (Underutilized)
The codebase already has a well-designed pipeline system:
```
karaoke_gen/pipeline/
├── base.py # PipelineStage, StageResult, PipelineExecutor abstracts
├── context.py # PipelineContext for shared state
├── executors/
│ ├── local.py # LocalExecutor - runs stages in-process (for CLI)
│ └── remote.py # RemoteExecutor - placeholder for backend integration
└── stages/
├── separation.py # SeparationStage - wraps AudioProcessor
├── transcription.py # TranscriptionStage - wraps LyricsProcessor
├── screens.py # ScreensStage - title/end screens
├── render.py # RenderStage - lyrics video
└── finalize.py # FinalizeStage - wraps KaraokeFinalise
```
**However**: Backend workers (`backend/workers/*.py`) don't use these stages - they directly call the underlying processors. This creates two parallel implementations.
---
## Recommended Architecture
### Decision: Cloud Tasks First
**Why Cloud Tasks over Pub/Sub or Workflows:**
| Feature | Cloud Tasks | Pub/Sub | Workflows |
|---------|-------------|---------|-----------|
| HTTP delivery | ✅ Native | Requires push subscription | N/A |
| Retry control | ✅ Built-in | Manual | Built-in |
| Rate limiting | ✅ Built-in | Manual | N/A |
| Timeout (max) | 30 min | 10 min ack deadline | 1 year |
| Complexity | Low | Medium | High |
| Cost | Very low | Low | Medium |
| Existing code changes | Minimal | Medium | High |
**Cloud Tasks is the right choice because:**
1. Workers already have HTTP endpoints - Cloud Tasks delivers HTTP requests
2. Need rate limiting to protect Modal/AudioShake APIs
3. Built-in retries with configurable backoff
4. Minimal code changes (update WorkerService only)
### Target Architecture
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ CLOUD TASKS + CLOUD RUN JOBS ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ POST /api/jobs/upload │
│ │ │
│ ├──▶ Save job to Firestore │
│ │ │
│ └──▶ WorkerService.trigger_worker() │
│ │ │
│ │ if ENABLE_CLOUD_TASKS: │
│ │ └──▶ Enqueue Cloud Task │
│ │ │ │
│ │ ├──▶ audio-worker-queue │
│ │ │ {job_id, target: /internal/workers/audio} │
│ │ │ │
│ │ └──▶ lyrics-worker-queue │
│ │ {job_id, target: /internal/workers/lyrics}│
│ │ │
│ │ else: (development/testing) │
│ │ └──▶ HTTP POST directly (existing behavior) │
│ │ │
│ ┌──────────▼────────────────────────────────────────────────────────┐ │
│ │ Cloud Tasks Queue │ │
│ │ │ │
│ │ • Guaranteed delivery (retries on failure) │ │
│ │ • Rate limiting / concurrency control │ │
│ │ • Task deduplication │ │
│ │ • Configurable timeouts (up to 30 min per task) │ │
│ └─────────────────────────────────┬───────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Cloud Run (Auto-Scaled Worker Instances) │ │
│ │ │ │
│ │ Instance 1 Instance 2 Instance 3 ... │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Job A │ │ Job B │ │ Job C │ │ │
│ │ │ Audio │ │ Audio │ │ Lyrics │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
│ │ │ │
│ │ Each task gets a DEDICATED container instance │ │
│ │ • Full CPU/memory for that task alone │ │
│ │ • No resource contention with other jobs │ │
│ │ • Container lifecycle tied to task completion │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
│ For video encoding (>30 min timeout): │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Cloud Run Jobs (Long-Running) │ │
│ │ │ │
│ │ • Up to 24 hour timeout │ │
│ │ • Separate from API service │ │
│ │ • Higher CPU/memory (4 vCPU, 8GB) │ │
│ │ • Triggered via gcloud or API │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Implementation Plan
### Phase 1: Cloud Tasks Integration (HIGH Priority)
**Goal:** Enable 100+ concurrent jobs with no performance degradation
**Effort:** 2-3 days | **Risk:** Low (feature-flagged, easy rollback)
#### Step 1.1: Add Cloud Tasks Infrastructure (Pulumi)
```python
# infrastructure/__main__.py additions
from pulumi_gcp import cloudtasks
# Create Cloud Tasks queues
audio_worker_queue = cloudtasks.Queue(
"audio-worker-queue",
name="audio-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=10, # Protect Modal API
max_concurrent_dispatches=50, # Max parallel audio workers
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=3,
min_backoff="10s",
max_backoff="300s",
max_retry_duration="1800s", # 30 min total retry window
),
)
lyrics_worker_queue = cloudtasks.Queue(
"lyrics-worker-queue",
name="lyrics-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=10,
max_concurrent_dispatches=50,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=3,
min_backoff="10s",
max_backoff="300s",
),
)
screens_worker_queue = cloudtasks.Queue(
"screens-worker-queue",
name="screens-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=50, # Fast, CPU-light
max_concurrent_dispatches=100,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=3,
min_backoff="5s",
max_backoff="60s",
),
)
render_worker_queue = cloudtasks.Queue(
"render-worker-queue",
name="render-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=5, # CPU-intensive
max_concurrent_dispatches=20,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=2,
min_backoff="30s",
max_backoff="300s",
max_retry_duration="3600s", # 60 min for render
),
)
video_worker_queue = cloudtasks.Queue(
"video-worker-queue",
name="video-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=3, # Very CPU-intensive
max_concurrent_dispatches=10,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=2,
min_backoff="60s",
max_backoff="600s",
max_retry_duration="7200s", # 2 hour total (video is long)
),
)
# Grant Cloud Tasks permission to invoke Cloud Run
cloud_tasks_invoker = gcp.projects.IAMMember(
"cloud-tasks-invoker",
project=project_id,
role="roles/run.invoker",
member=f"serviceAccount:service-{project.number}@gcp-sa-cloudtasks.iam.gserviceaccount.com",
)
# Export queue names
pulumi.export("audio_worker_queue", audio_worker_queue.name)
pulumi.export("lyrics_worker_queue", lyrics_worker_queue.name)
pulumi.export("screens_worker_queue", screens_worker_queue.name)
pulumi.export("render_worker_queue", render_worker_queue.name)
pulumi.export("video_worker_queue", video_worker_queue.name)
```
#### Step 1.2: Update WorkerService for Cloud Tasks
```python
# backend/services/worker_service.py - Updated
import logging
import os
import json
from typing import Optional
from google.cloud import tasks_v2
from google.protobuf import timestamp_pb2
import httpx
from backend.config import get_settings
logger = logging.getLogger(__name__)
# Queue mapping
WORKER_QUEUES = {
"audio": "audio-worker-queue",
"lyrics": "lyrics-worker-queue",
"screens": "screens-worker-queue",
"render-video": "render-worker-queue",
"video": "video-worker-queue",
}
class WorkerService:
"""
Service for coordinating background workers.
Supports two modes:
- Cloud Tasks (production): Guaranteed delivery, auto-scaling
- Direct HTTP (development): Faster iteration, simpler debugging
"""
def __init__(self):
self.settings = get_settings()
self._base_url = self._get_base_url()
self._admin_token = self._get_admin_token()
self._use_cloud_tasks = self._should_use_cloud_tasks()
self._tasks_client = None
def _should_use_cloud_tasks(self) -> bool:
"""Check if Cloud Tasks should be used."""
# Feature flag with environment variable
enable_flag = os.getenv("ENABLE_CLOUD_TASKS", "false").lower()
return enable_flag in ("true", "1", "yes")
@property
def tasks_client(self):
"""Lazy-init Cloud Tasks client."""
if self._tasks_client is None and self._use_cloud_tasks:
self._tasks_client = tasks_v2.CloudTasksClient()
return self._tasks_client
def _get_admin_token(self) -> Optional[str]:
admin_tokens_str = self.settings.admin_tokens or ""
tokens = [t.strip() for t in admin_tokens_str.split(",") if t.strip()]
return tokens[0] if tokens else None
def _get_base_url(self) -> str:
# Check for test environment override
test_url = os.getenv("TEST_SERVER_URL")
if test_url:
return test_url
# Production: Cloud Run service URL
service_url = os.getenv("CLOUD_RUN_SERVICE_URL")
if service_url:
return service_url
# Development: localhost
port = os.getenv("PORT", "8000")
return f"http://localhost:{port}"
async def trigger_worker(
self,
worker_type: str,
job_id: str,
timeout_seconds: int = 30
) -> bool:
"""
Trigger a background worker.
In production (ENABLE_CLOUD_TASKS=true):
- Enqueues task to Cloud Tasks queue
- Cloud Tasks calls internal endpoint with retries
In development:
- Direct HTTP call to internal endpoint
- Faster iteration, but no retry guarantees
"""
if self._use_cloud_tasks:
return await self._enqueue_cloud_task(worker_type, job_id)
else:
return await self._trigger_http(worker_type, job_id, timeout_seconds)
async def _enqueue_cloud_task(self, worker_type: str, job_id: str) -> bool:
"""Enqueue task to Cloud Tasks for guaranteed delivery."""
try:
queue_name = WORKER_QUEUES.get(worker_type)
if not queue_name:
logger.error(f"Unknown worker type: {worker_type}")
return False
project = self.settings.google_cloud_project
location = "us-central1"
# Build queue path
parent = self.tasks_client.queue_path(project, location, queue_name)
# Build task
task = {
"http_request": {
"http_method": tasks_v2.HttpMethod.POST,
"url": f"{self._base_url}/api/internal/workers/{worker_type}",
"headers": {
"Content-Type": "application/json",
},
"body": json.dumps({"job_id": job_id}).encode(),
# Use OIDC token for Cloud Run authentication
"oidc_token": {
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
},
},
}
# Add admin auth header if available
if self._admin_token:
task["http_request"]["headers"]["Authorization"] = f"Bearer {self._admin_token}"
# Create task
response = self.tasks_client.create_task(parent=parent, task=task)
logger.info(f"Created Cloud Task for {worker_type} worker, job {job_id}: {response.name}")
return True
except Exception as e:
logger.error(f"Failed to enqueue Cloud Task for {worker_type}/{job_id}: {e}", exc_info=True)
return False
async def _trigger_http(
self,
worker_type: str,
job_id: str,
timeout_seconds: int = 30
) -> bool:
"""Direct HTTP trigger (for development)."""
try:
headers = {}
if self._admin_token:
headers["Authorization"] = f"Bearer {self._admin_token}"
async with httpx.AsyncClient(timeout=timeout_seconds) as client:
url = f"{self._base_url}/api/internal/workers/{worker_type}"
response = await client.post(url, json={"job_id": job_id}, headers=headers)
if response.status_code == 200:
logger.info(f"Triggered {worker_type} worker for job {job_id}")
return True
else:
logger.error(f"Failed to trigger {worker_type}: HTTP {response.status_code}")
return False
except Exception as e:
logger.error(f"Error triggering {worker_type} worker: {e}", exc_info=True)
return False
# Convenience methods (unchanged)
async def trigger_audio_worker(self, job_id: str) -> bool:
return await self.trigger_worker("audio", job_id)
async def trigger_lyrics_worker(self, job_id: str) -> bool:
return await self.trigger_worker("lyrics", job_id)
async def trigger_screens_worker(self, job_id: str) -> bool:
return await self.trigger_worker("screens", job_id)
async def trigger_video_worker(self, job_id: str) -> bool:
return await self.trigger_worker("video", job_id)
async def trigger_render_video_worker(self, job_id: str) -> bool:
return await self.trigger_worker("render-video", job_id)
```
#### Step 1.3: Update Internal Endpoints for Cloud Tasks
The internal endpoints need minor updates to handle Cloud Tasks requests:
```python
# backend/api/routes/internal.py - Updates
# Add idempotency check to prevent duplicate processing
@router.post("/workers/audio", response_model=WorkerResponse)
async def trigger_audio_worker(
request: WorkerRequest,
background_tasks: BackgroundTasks,
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
):
"""Trigger audio separation worker for a job."""
job_id = request.job_id
# Idempotency: Check if already processing
job_manager = JobManager()
job = job_manager.get_job(job_id)
if job and job.state_data.get('audio_progress', {}).get('stage') == 'running':
logger.info(f"Audio worker already running for job {job_id}, skipping")
return WorkerResponse(
status="already_running",
job_id=job_id,
message="Audio worker already in progress"
)
# Mark as running before starting (for idempotency)
job_manager.update_state_data(job_id, 'audio_progress', {'stage': 'running'})
logger.info(f"Triggering audio worker for job {job_id}")
background_tasks.add_task(process_audio_separation, job_id)
return WorkerResponse(
status="started",
job_id=job_id,
message="Audio separation worker started"
)
```
#### Step 1.4: Cloud Run Service Account Permissions
```python
# infrastructure/__main__.py additions
# Grant Cloud Tasks admin permission to the service account
# (needed to create tasks)
cloud_tasks_admin = gcp.projects.IAMMember(
"karaoke-backend-cloudtasks-admin",
project=project_id,
role="roles/cloudtasks.enqueuer",
member=service_account.email.apply(lambda email: f"serviceAccount:{email}"),
)
```
### Phase 2: Cloud Run Jobs for Video Worker (MEDIUM Priority)
**Goal:** Remove 30-minute timeout constraint for video encoding
**Effort:** 1-2 days | **Risk:** Low
The video worker (final encoding) can take 15-20 minutes, sometimes longer. Cloud Tasks has a 30-minute max timeout. For safety, we use Cloud Run Jobs.
#### Step 2.1: Create Cloud Run Job
```yaml
# cloudbuild-video-job.yaml
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args:
- 'run'
- 'jobs'
- 'create'
- 'video-encoding-job'
- '--image=${_IMAGE}'
- '--region=us-central1'
- '--cpu=4'
- '--memory=8Gi'
- '--max-retries=2'
- '--task-timeout=3600' # 1 hour max
- '--service-account=karaoke-backend@${PROJECT_ID}.iam.gserviceaccount.com'
- '--set-env-vars=GOOGLE_CLOUD_PROJECT=${PROJECT_ID}'
```
#### Step 2.2: Update Video Worker Trigger
```python
# backend/services/worker_service.py - Addition
async def trigger_video_worker_job(self, job_id: str) -> bool:
"""Trigger video worker as Cloud Run Job (for long-running encoding)."""
if not self._use_cloud_tasks:
# Development: use HTTP trigger
return await self._trigger_http("video", job_id)
try:
from google.cloud import run_v2
client = run_v2.JobsClient()
project = self.settings.google_cloud_project
location = "us-central1"
job_name = f"projects/{project}/locations/{location}/jobs/video-encoding-job"
# Override container args with job_id
request = run_v2.RunJobRequest(
name=job_name,
overrides=run_v2.RunJobRequest.Overrides(
container_overrides=[
run_v2.RunJobRequest.Overrides.ContainerOverride(
args=["--job-id", job_id],
)
]
)
)
operation = client.run_job(request=request)
logger.info(f"Started Cloud Run Job for video encoding: {job_id}")
return True
except Exception as e:
logger.error(f"Failed to start video encoding job: {e}", exc_info=True)
return False
```
### Phase 3: Pipeline Unification (LOW Priority - Future)
**Goal:** Share code between CLI and backend workers
**Effort:** 1-2 weeks | **Risk:** Medium (requires careful refactoring)
This phase can be deferred - the infrastructure scaling (Phase 1-2) solves the immediate problem.
**When to consider this:**
- When adding new features that need to work in both CLI and backend
- When a bug fix needs to be applied in multiple places
- When the divergence between workers and pipeline stages causes maintenance burden
**Approach:**
1. Make backend workers call pipeline stages instead of processors directly
2. Have RemoteExecutor use Cloud Tasks to trigger stage execution
3. Both CLI and backend use the same pipeline stages
---
## Resource Configuration
### Cloud Tasks Queues
| Queue | Max Concurrent | Rate Limit | Timeout | Retries | Notes |
|-------|---------------|------------|---------|---------|-------|
| audio-worker | 50 | 10/sec | 15 min | 3 | Mostly I/O (Modal API) |
| lyrics-worker | 50 | 10/sec | 10 min | 3 | Mostly I/O (AudioShake) |
| screens-worker | 100 | 50/sec | 2 min | 3 | FFmpeg (quick) |
| render-worker | 20 | 5/sec | 30 min | 2 | LyricsTranscriber + FFmpeg |
| video-worker | 10 | 3/sec | 30 min | 2 | Heavy FFmpeg (or use Jobs) |
### Cloud Run Instance Scaling
| Worker | CPU | Memory | Max Instances | Notes |
|--------|-----|--------|---------------|-------|
| audio | 2 | 2Gi | 50 | Mostly waiting on Modal API |
| lyrics | 2 | 2Gi | 50 | Mostly waiting on AudioShake |
| screens | 2 | 2Gi | 100 | Quick FFmpeg |
| render | 4 | 4Gi | 20 | CPU-intensive |
| video (Job) | 4 | 8Gi | 10 | Very CPU-intensive |
### Cost Estimate (100 concurrent jobs/day)
```
Cloud Tasks: ~$0.40/million tasks ≈ $0.01/day
Cloud Run: ~$0.00002/vCPU-sec ≈ $5-10/day (100 jobs)
Cloud Run Jobs: ~$0.00002/vCPU-sec ≈ $2-5/day (100 jobs)
Modal (GPU): ~$2.50/hour ≈ $20-40/day (100 jobs)
AudioShake: Usage-based ≈ $10-20/day (100 jobs)
Total: ~$40-80/day for 100 concurrent jobs
```
---
## Migration Strategy
### Zero-Downtime Migration
```
Week 1: Deploy Cloud Tasks infrastructure (Pulumi)
└── Queues exist but aren't used yet (ENABLE_CLOUD_TASKS=false)
Week 2: Test in staging
└── Deploy to staging with ENABLE_CLOUD_TASKS=true
└── Run 10-20 test jobs concurrently
└── Monitor for issues
Week 3: Enable in production
└── Set ENABLE_CLOUD_TASKS=true
└── Monitor metrics and logs
└── Have rollback ready
Week 4: Cleanup
└── Remove feature flag after stable week
└── Update documentation
```
### Rollback Plan
If issues occur after enabling Cloud Tasks:
1. Set `ENABLE_CLOUD_TASKS=false` environment variable
2. Redeploy (takes ~2 minutes)
3. New jobs use direct HTTP (old behavior)
4. In-flight Cloud Tasks jobs continue to completion
5. No data loss - Firestore has job state
### Testing Strategy
```bash
# Run unit tests (no Cloud Tasks)
pytest backend/tests/ -v
# Run integration tests with Cloud Tasks (staging)
ENABLE_CLOUD_TASKS=true pytest backend/tests/integration/ -v
# Load test (staging)
# Submit 50 concurrent jobs, verify all complete
for i in {1..50}; do
curl -X POST "https://staging.api.nomadkaraoke.com/api/jobs/upload" \
-F "file=@test.flac" -F "artist=Test" -F "title=Job$i" &
done
wait
```
---
## Success Criteria
### Phase 1 (Cloud Tasks)
- [x] Cloud Tasks queues created and configured via Pulumi
- [x] WorkerService updated to support Cloud Tasks mode
- [x] Deployment configured with ENABLE_CLOUD_TASKS=true
- [x] Container concurrency set to 1 for true isolation
- [ ] 50 concurrent jobs complete without failures (to be validated)
- [ ] Processing time per job unchanged (same as 1 job)
- [ ] No resource contention (CPU/memory per job consistent)
- [ ] Automatic retry on transient failures works
- [ ] Feature flag rollback works
### Phase 2 (Cloud Run Jobs)
- [ ] Video encoding >30 min completes successfully
- [ ] Jobs triggered correctly from WorkerService
- [ ] Progress updates visible in CLI
### Phase 3 (Pipeline Unification)
- [ ] Single stage implementation used by both CLIs
- [ ] Bug fixes apply to both local and remote execution
- [ ] Each stage independently testable
- [ ] 80%+ code coverage on pipeline module
---
## Files to Modify
### Phase 1
| File | Changes |
|------|---------|
| `infrastructure/__main__.py` | Add Cloud Tasks queues, IAM permissions |
| `backend/services/worker_service.py` | Add Cloud Tasks client, feature flag |
| `backend/api/routes/internal.py` | Add idempotency checks |
| `backend/config.py` | Add ENABLE_CLOUD_TASKS setting |
| `backend/requirements.txt` | Add google-cloud-tasks |
### Phase 2
| File | Changes |
|------|---------|
| `cloudbuild-video-job.yaml` | New file for Cloud Run Job |
| `backend/services/worker_service.py` | Add Cloud Run Jobs trigger |
| `backend/workers/video_worker.py` | CLI entry point for Jobs |
---
## Related Documents
- [ARCHITECTURE.md](../01-reference/ARCHITECTURE.md) - Current system architecture
- [BACKEND-FEATURE-PARITY-PLAN.md](./BACKEND-FEATURE-PARITY-PLAN.md) - Feature parity roadmap
- [WORKER-IMPLEMENTATION-PLAN.md](./WORKER-IMPLEMENTATION-PLAN.md) - Worker details
---
## Changelog
### 2025-12-11: Analysis Complete - Updated Plan
- **Analyzed codebase thoroughly** - found existing pipeline architecture underutilized
- **Confirmed Cloud Tasks as best approach** - minimal code changes, single point of change
- **Identified WorkerService as sole modification point** - clean abstraction exists
- **Decided against immediate pipeline unification** - infrastructure scaling is the priority
- **Added detailed implementation code** - ready to start Phase 1
- **Created phased approach** - Phase 1 (Cloud Tasks) → Phase 2 (Jobs) → Phase 3 (Pipeline)
### 2025-12-11: Initial Plan
- Documented current architecture limitations
- Proposed Cloud Tasks + Cloud Run Jobs hybrid
- Outlined shared pipeline architecture
- Created implementation roadmap
@@ -0,0 +1,603 @@
"use client"
import { useParams, useRouter } from "next/navigation"
import { useState, useEffect, useCallback } from "react"
import { AppHeader } from "@/components/app-header"
import { ProtectedRoute } from "@/components/protected-route"
import { Button } from "@/components/ui/button"
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
import { Alert, AlertDescription } from "@/components/ui/alert"
import { Badge } from "@/components/ui/badge"
import { RadioGroup, RadioGroupItem } from "@/components/ui/radio-group"
import { Label } from "@/components/ui/label"
import { Skeleton } from "@/components/ui/skeleton"
import {
WaveformDisplay,
AudioPlayer,
RegionSelector,
} from "@/components/instrumental-review"
import {
ArrowLeft,
AlertCircle,
CheckCircle2,
Loader2,
Music,
VolumeX,
Volume2,
} from "lucide-react"
interface AudibleSegment {
start_seconds: number
end_seconds: number
duration_seconds: number
avg_amplitude_db: number
peak_amplitude_db?: number
}
interface MuteRegion {
start_seconds: number
end_seconds: number
}
interface AnalysisData {
has_audible_content: boolean
total_duration_seconds: number
audible_segments: AudibleSegment[]
recommended_selection: string
total_audible_duration_seconds: number
audible_percentage: number
silence_threshold_db: number
}
interface InstrumentalAnalysisResponse {
job_id: string
artist: string
title: string
status: string
analysis: AnalysisData
audio_urls: {
clean_instrumental: string | null
backing_vocals: string | null
with_backing: string | null
custom_instrumental: string | null
}
waveform_url: string | null
has_custom_instrumental: boolean
}
// Get API base URL from environment or default to relative path
const API_BASE = process.env.NEXT_PUBLIC_API_URL || "/api"
function InstrumentalReviewContent() {
const params = useParams()
const router = useRouter()
const jobId = params.id as string
// State
const [isLoading, setIsLoading] = useState(true)
const [error, setError] = useState<string | null>(null)
const [analysisData, setAnalysisData] = useState<InstrumentalAnalysisResponse | null>(null)
const [waveformData, setWaveformData] = useState<{ amplitudes: number[]; duration: number } | null>(null)
const [selectedOption, setSelectedOption] = useState<string>("clean")
const [muteRegions, setMuteRegions] = useState<MuteRegion[]>([])
const [isSelectionMode, setIsSelectionMode] = useState(false)
const [currentTime, setCurrentTime] = useState(0)
const [seekTo, setSeekTo] = useState<number | undefined>(undefined)
const [isCreatingCustom, setIsCreatingCustom] = useState(false)
const [isSubmitting, setIsSubmitting] = useState(false)
const [activeAudio, setActiveAudio] = useState<"clean" | "backing" | "with_backing" | "custom">("backing")
// Fetch analysis data
const fetchAnalysis = useCallback(async () => {
try {
setIsLoading(true)
setError(null)
const response = await fetch(`${API_BASE}/jobs/${jobId}/instrumental-analysis`)
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
throw new Error(errorData.detail || `Failed to fetch analysis: ${response.status}`)
}
const data: InstrumentalAnalysisResponse = await response.json()
setAnalysisData(data)
// Set initial selection based on recommendation
const recommendation = data.analysis.recommended_selection
if (recommendation === "clean") {
setSelectedOption("clean")
} else if (recommendation === "with_backing") {
setSelectedOption("with_backing")
} else {
// "review_needed" - default to with_backing but user should review
setSelectedOption("with_backing")
}
// Fetch waveform data for interactive display
const waveformResponse = await fetch(`${API_BASE}/jobs/${jobId}/waveform-data?num_points=600`)
if (waveformResponse.ok) {
const waveform = await waveformResponse.json()
setWaveformData(waveform)
}
} catch (err) {
setError(err instanceof Error ? err.message : "Failed to load analysis data")
} finally {
setIsLoading(false)
}
}, [jobId])
useEffect(() => {
fetchAnalysis()
}, [fetchAnalysis])
// Handle seeking from waveform click
const handleSeek = (time: number) => {
setSeekTo(time)
// Reset seekTo after a short delay
setTimeout(() => setSeekTo(undefined), 100)
}
// Handle adding a mute region
const handleAddRegion = (region: MuteRegion) => {
setMuteRegions((prev) => {
// Check for overlaps and merge if needed
const newRegions = [...prev, region].sort((a, b) => a.start_seconds - b.start_seconds)
// Merge overlapping regions
const merged: MuteRegion[] = []
for (const r of newRegions) {
if (merged.length === 0) {
merged.push(r)
} else {
const last = merged[merged.length - 1]
if (r.start_seconds <= last.end_seconds) {
last.end_seconds = Math.max(last.end_seconds, r.end_seconds)
} else {
merged.push(r)
}
}
}
return merged
})
setIsSelectionMode(false)
}
// Handle removing a mute region
const handleRemoveRegion = (index: number) => {
setMuteRegions((prev) => prev.filter((_, i) => i !== index))
}
// Handle clearing all mute regions
const handleClearAll = () => {
setMuteRegions([])
}
// Create custom instrumental
const handleCreateCustom = async () => {
if (muteRegions.length === 0) {
setError("Please select at least one region to mute")
return
}
try {
setIsCreatingCustom(true)
setError(null)
const response = await fetch(`${API_BASE}/jobs/${jobId}/create-custom-instrumental`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
mute_regions: muteRegions.map((r) => ({
start_seconds: r.start_seconds,
end_seconds: r.end_seconds,
})),
}),
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
throw new Error(errorData.detail || "Failed to create custom instrumental")
}
// Refresh analysis to get updated URLs
await fetchAnalysis()
setSelectedOption("custom")
setActiveAudio("custom")
} catch (err) {
setError(err instanceof Error ? err.message : "Failed to create custom instrumental")
} finally {
setIsCreatingCustom(false)
}
}
// Submit selection
const handleSubmit = async () => {
let selection = selectedOption
// If "review" is selected, determine based on mute regions
if (selectedOption === "review") {
if (muteRegions.length > 0 && analysisData?.has_custom_instrumental) {
selection = "custom"
} else {
selection = "with_backing"
}
}
try {
setIsSubmitting(true)
setError(null)
const response = await fetch(`${API_BASE}/jobs/${jobId}/select-instrumental`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ selection }),
})
if (!response.ok) {
const errorData = await response.json().catch(() => ({}))
throw new Error(errorData.detail || "Failed to submit selection")
}
// Navigate back to job page
router.push(`/jobs/${jobId}`)
} catch (err) {
setError(err instanceof Error ? err.message : "Failed to submit selection")
} finally {
setIsSubmitting(false)
}
}
// Get current audio URL based on active selection
const getCurrentAudioUrl = () => {
if (!analysisData) return null
switch (activeAudio) {
case "clean":
return analysisData.audio_urls.clean_instrumental
case "backing":
return analysisData.audio_urls.backing_vocals
case "with_backing":
return analysisData.audio_urls.with_backing
case "custom":
return analysisData.audio_urls.custom_instrumental
default:
return null
}
}
// Loading state
if (isLoading) {
return (
<div className="min-h-screen bg-background">
<AppHeader />
<main className="container mx-auto px-4 py-8 max-w-5xl">
<Skeleton className="h-8 w-48 mb-4" />
<Skeleton className="h-4 w-96 mb-8" />
<Skeleton className="h-[200px] w-full mb-4" />
<Skeleton className="h-32 w-full" />
</main>
</div>
)
}
// Error state
if (error && !analysisData) {
return (
<div className="min-h-screen bg-background">
<AppHeader />
<main className="container mx-auto px-4 py-8 max-w-5xl">
<Alert variant="destructive">
<AlertCircle className="h-4 w-4" />
<AlertDescription>{error}</AlertDescription>
</Alert>
<Button variant="outline" className="mt-4" onClick={() => router.push(`/jobs/${jobId}`)}>
<ArrowLeft className="w-4 h-4 mr-2" />
Back to Job
</Button>
</main>
</div>
)
}
const analysis = analysisData?.analysis
return (
<div className="min-h-screen bg-background">
<AppHeader />
<main className="container mx-auto px-4 py-8 max-w-6xl">
{/* Header */}
<Button variant="ghost" className="mb-6" onClick={() => router.push(`/jobs/${jobId}`)}>
<ArrowLeft className="w-4 h-4 mr-2" />
Back to Job
</Button>
<div className="mb-8">
<h1 className="text-3xl font-bold mb-2">
Select Instrumental Track
</h1>
<p className="text-muted-foreground">
{analysisData?.artist} - {analysisData?.title}
</p>
</div>
{/* Error Alert */}
{error && (
<Alert variant="destructive" className="mb-6">
<AlertCircle className="h-4 w-4" />
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
{/* Analysis Summary */}
{analysis && (
<Card className="mb-6">
<CardHeader>
<CardTitle className="text-lg flex items-center gap-2">
<Music className="h-5 w-5" />
Backing Vocals Analysis
</CardTitle>
<CardDescription>
Automated analysis of the backing vocals stem
</CardDescription>
</CardHeader>
<CardContent>
<div className="flex flex-wrap gap-4">
<div className="flex items-center gap-2">
{analysis.has_audible_content ? (
<Volume2 className="h-4 w-4 text-yellow-500" />
) : (
<VolumeX className="h-4 w-4 text-green-500" />
)}
<span className="text-sm">
{analysis.has_audible_content
? "Backing vocals detected"
: "No backing vocals detected"}
</span>
</div>
{analysis.has_audible_content && (
<>
<Badge variant="outline">
{analysis.audible_segments.length} audible segments
</Badge>
<Badge variant="outline">
{analysis.audible_percentage.toFixed(1)}% of track
</Badge>
<Badge variant="outline">
{analysis.total_audible_duration_seconds.toFixed(1)}s total
</Badge>
</>
)}
<Badge
variant={analysis.recommended_selection === "clean" ? "default" : "secondary"}
>
Recommended: {analysis.recommended_selection === "clean" ? "Clean Instrumental" : "Review Needed"}
</Badge>
</div>
</CardContent>
</Card>
)}
{/* Waveform Display */}
{waveformData && analysis && (
<Card className="mb-6">
<CardHeader>
<CardTitle className="text-lg">Backing Vocals Waveform</CardTitle>
<CardDescription>
Click to seek • {isSelectionMode ? "Drag to select mute region" : "Click 'Add Region' to start selecting"}
</CardDescription>
</CardHeader>
<CardContent>
<WaveformDisplay
amplitudes={waveformData.amplitudes}
duration={waveformData.duration}
currentTime={currentTime}
segments={analysis.audible_segments}
muteRegions={muteRegions}
onSeek={handleSeek}
onRegionSelect={handleAddRegion}
isSelecting={isSelectionMode}
width={900}
height={150}
className="w-full overflow-x-auto"
/>
</CardContent>
</Card>
)}
<div className="grid lg:grid-cols-2 gap-6 mb-6">
{/* Audio Players */}
<Card>
<CardHeader>
<CardTitle className="text-lg">Audio Preview</CardTitle>
<CardDescription>
Listen to different instrumental options
</CardDescription>
</CardHeader>
<CardContent className="space-y-4">
{/* Audio source selector */}
<div className="flex flex-wrap gap-2">
<Button
variant={activeAudio === "backing" ? "default" : "outline"}
size="sm"
onClick={() => setActiveAudio("backing")}
>
Backing Vocals
</Button>
<Button
variant={activeAudio === "clean" ? "default" : "outline"}
size="sm"
onClick={() => setActiveAudio("clean")}
>
Clean Instrumental
</Button>
<Button
variant={activeAudio === "with_backing" ? "default" : "outline"}
size="sm"
onClick={() => setActiveAudio("with_backing")}
>
With Backing
</Button>
{analysisData?.has_custom_instrumental && (
<Button
variant={activeAudio === "custom" ? "default" : "outline"}
size="sm"
onClick={() => setActiveAudio("custom")}
>
Custom
</Button>
)}
</div>
{/* Audio player */}
{getCurrentAudioUrl() && (
<AudioPlayer
src={getCurrentAudioUrl()!}
onTimeUpdate={setCurrentTime}
seekTo={seekTo}
label={
activeAudio === "backing"
? "Backing Vocals Only"
: activeAudio === "clean"
? "Clean Instrumental"
: activeAudio === "custom"
? "Custom Instrumental"
: "Instrumental + Backing Vocals"
}
/>
)}
</CardContent>
</Card>
{/* Region Selector */}
<RegionSelector
muteRegions={muteRegions}
audibleSegments={analysis?.audible_segments}
onAddRegion={handleAddRegion}
onRemoveRegion={handleRemoveRegion}
onClearAll={handleClearAll}
onSeekTo={handleSeek}
isSelectionMode={isSelectionMode}
onToggleSelectionMode={() => setIsSelectionMode(!isSelectionMode)}
/>
</div>
{/* Create Custom Button */}
{muteRegions.length > 0 && !analysisData?.has_custom_instrumental && (
<Card className="mb-6">
<CardContent className="py-4">
<div className="flex items-center justify-between">
<div>
<p className="font-medium">Ready to create custom instrumental</p>
<p className="text-sm text-muted-foreground">
{muteRegions.length} region{muteRegions.length > 1 ? "s" : ""} will be muted
</p>
</div>
<Button onClick={handleCreateCustom} disabled={isCreatingCustom}>
{isCreatingCustom ? (
<>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
Creating...
</>
) : (
"Create Custom Instrumental"
)}
</Button>
</div>
</CardContent>
</Card>
)}
{/* Selection Options */}
<Card className="mb-6">
<CardHeader>
<CardTitle className="text-lg">Final Selection</CardTitle>
<CardDescription>
Choose which instrumental to use for your karaoke video
</CardDescription>
</CardHeader>
<CardContent>
<RadioGroup value={selectedOption} onValueChange={setSelectedOption}>
<div className="space-y-4">
<div className="flex items-center space-x-3">
<RadioGroupItem value="clean" id="clean" />
<Label htmlFor="clean" className="flex-1 cursor-pointer">
<div className="font-medium">Clean Instrumental</div>
<div className="text-sm text-muted-foreground">
Use the instrumental with no backing vocals at all
</div>
</Label>
{analysis?.recommended_selection === "clean" && (
<Badge variant="default" className="bg-green-600">
<CheckCircle2 className="h-3 w-3 mr-1" />
Recommended
</Badge>
)}
</div>
<div className="flex items-center space-x-3">
<RadioGroupItem value="with_backing" id="with_backing" />
<Label htmlFor="with_backing" className="flex-1 cursor-pointer">
<div className="font-medium">Instrumental with Backing Vocals</div>
<div className="text-sm text-muted-foreground">
Use the instrumental with all backing vocals included
</div>
</Label>
</div>
{analysisData?.has_custom_instrumental && (
<div className="flex items-center space-x-3">
<RadioGroupItem value="custom" id="custom" />
<Label htmlFor="custom" className="flex-1 cursor-pointer">
<div className="font-medium">Custom Instrumental</div>
<div className="text-sm text-muted-foreground">
Use your custom instrumental with {muteRegions.length} muted region{muteRegions.length > 1 ? "s" : ""}
</div>
</Label>
<Badge variant="secondary">Custom</Badge>
</div>
)}
</div>
</RadioGroup>
</CardContent>
</Card>
{/* Submit Button */}
<div className="flex gap-4">
<Button
size="lg"
onClick={handleSubmit}
disabled={isSubmitting}
className="flex-1 max-w-md"
>
{isSubmitting ? (
<>
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
Submitting...
</>
) : (
<>
<CheckCircle2 className="h-4 w-4 mr-2" />
Confirm Selection & Continue
</>
)}
</Button>
</div>
</main>
</div>
)
}
export default function InstrumentalReviewPage() {
return (
<ProtectedRoute>
<InstrumentalReviewContent />
</ProtectedRoute>
)
}
+9 -2
View File
@@ -190,8 +190,15 @@ function JobDetailContent() {
{job.status === "awaiting_instrumental" && (
<Alert className="bg-secondary/10 border-secondary/30">
<AlertCircle className="h-4 w-4 text-secondary" />
<AlertDescription className="text-foreground">
Multiple instrumental versions were detected. Please select your preferred version to continue.
<AlertDescription className="text-foreground flex items-center justify-between">
<span>Multiple instrumental versions were detected. Please select your preferred version to continue.</span>
<Button
size="sm"
onClick={() => router.push(`/jobs/${job.id}/instrumental-review`)}
className="ml-4"
>
Select Instrumental
</Button>
</AlertDescription>
</Alert>
)}
@@ -0,0 +1,263 @@
"use client"
import { useRef, useState, useEffect, useCallback } from "react"
import { Button } from "@/components/ui/button"
import { Slider } from "@/components/ui/slider"
import { Play, Pause, SkipBack, SkipForward, Volume2, VolumeX } from "lucide-react"
interface AudioPlayerProps {
src: string
onTimeUpdate?: (currentTime: number) => void
onDurationChange?: (duration: number) => void
seekTo?: number
label?: string
className?: string
}
export function AudioPlayer({
src,
onTimeUpdate,
onDurationChange,
seekTo,
label,
className = "",
}: AudioPlayerProps) {
const audioRef = useRef<HTMLAudioElement>(null)
const [isPlaying, setIsPlaying] = useState(false)
const [currentTime, setCurrentTime] = useState(0)
const [duration, setDuration] = useState(0)
const [volume, setVolume] = useState(1)
const [isMuted, setIsMuted] = useState(false)
const [isLoading, setIsLoading] = useState(true)
// Handle external seek requests
useEffect(() => {
if (seekTo !== undefined && audioRef.current && !isNaN(seekTo)) {
audioRef.current.currentTime = seekTo
}
}, [seekTo])
// Handle audio events
useEffect(() => {
const audio = audioRef.current
if (!audio) return
const handleTimeUpdate = () => {
setCurrentTime(audio.currentTime)
onTimeUpdate?.(audio.currentTime)
}
const handleDurationChange = () => {
setDuration(audio.duration)
onDurationChange?.(audio.duration)
}
const handleLoadedData = () => {
setIsLoading(false)
setDuration(audio.duration)
onDurationChange?.(audio.duration)
}
const handleEnded = () => {
setIsPlaying(false)
}
const handleLoadStart = () => {
setIsLoading(true)
}
const handleCanPlay = () => {
setIsLoading(false)
}
// Sync isPlaying state with actual audio element state
const handlePlay = () => {
setIsPlaying(true)
}
const handlePause = () => {
setIsPlaying(false)
}
audio.addEventListener("timeupdate", handleTimeUpdate)
audio.addEventListener("durationchange", handleDurationChange)
audio.addEventListener("loadeddata", handleLoadedData)
audio.addEventListener("ended", handleEnded)
audio.addEventListener("loadstart", handleLoadStart)
audio.addEventListener("canplay", handleCanPlay)
audio.addEventListener("play", handlePlay)
audio.addEventListener("pause", handlePause)
return () => {
audio.removeEventListener("timeupdate", handleTimeUpdate)
audio.removeEventListener("durationchange", handleDurationChange)
audio.removeEventListener("loadeddata", handleLoadedData)
audio.removeEventListener("ended", handleEnded)
audio.removeEventListener("loadstart", handleLoadStart)
audio.removeEventListener("canplay", handleCanPlay)
audio.removeEventListener("play", handlePlay)
audio.removeEventListener("pause", handlePause)
}
}, [onTimeUpdate, onDurationChange])
// Update volume
useEffect(() => {
if (audioRef.current) {
audioRef.current.volume = isMuted ? 0 : volume
}
}, [volume, isMuted])
const togglePlay = useCallback(async () => {
const audio = audioRef.current
if (!audio) return
try {
if (audio.paused) {
await audio.play()
} else {
audio.pause()
}
// State will be updated by the play/pause event listeners
} catch (error) {
// Play was prevented (e.g., user hasn't interacted with page yet)
console.warn("Audio playback was prevented:", error)
}
}, [])
const handleSeek = useCallback((value: number[]) => {
const audio = audioRef.current
if (!audio) return
const newTime = value[0]
audio.currentTime = newTime
setCurrentTime(newTime)
}, [])
const skipBackward = useCallback(() => {
const audio = audioRef.current
if (!audio) return
audio.currentTime = Math.max(0, audio.currentTime - 5)
}, [])
const skipForward = useCallback(() => {
const audio = audioRef.current
if (!audio) return
audio.currentTime = Math.min(duration, audio.currentTime + 5)
}, [duration])
const toggleMute = useCallback(() => {
setIsMuted(!isMuted)
}, [isMuted])
const handleVolumeChange = useCallback((value: number[]) => {
setVolume(value[0])
if (value[0] > 0 && isMuted) {
setIsMuted(false)
}
}, [isMuted])
const formatTime = (time: number) => {
if (isNaN(time)) return "0:00"
const minutes = Math.floor(time / 60)
const seconds = Math.floor(time % 60)
return `${minutes}:${seconds.toString().padStart(2, "0")}`
}
return (
<div className={`bg-card rounded-lg p-4 ${className}`}>
<audio ref={audioRef} src={src} preload="metadata" />
{label && (
<div className="text-sm font-medium text-muted-foreground mb-3">
{label}
</div>
)}
{/* Progress bar */}
<div className="mb-4">
<Slider
value={[currentTime]}
max={duration || 100}
step={0.1}
onValueChange={handleSeek}
disabled={isLoading}
className="w-full"
/>
<div className="flex justify-between text-xs text-muted-foreground mt-1">
<span>{formatTime(currentTime)}</span>
<span>{formatTime(duration)}</span>
</div>
</div>
{/* Controls */}
<div className="flex items-center justify-between">
<div className="flex items-center gap-2">
<Button
variant="ghost"
size="icon"
onClick={skipBackward}
disabled={isLoading}
className="h-8 w-8"
>
<SkipBack className="h-4 w-4" />
</Button>
<Button
variant="default"
size="icon"
onClick={togglePlay}
disabled={isLoading}
className="h-10 w-10"
>
{isPlaying ? (
<Pause className="h-5 w-5" />
) : (
<Play className="h-5 w-5 ml-0.5" />
)}
</Button>
<Button
variant="ghost"
size="icon"
onClick={skipForward}
disabled={isLoading}
className="h-8 w-8"
>
<SkipForward className="h-4 w-4" />
</Button>
</div>
{/* Volume control */}
<div className="flex items-center gap-2">
<Button
variant="ghost"
size="icon"
onClick={toggleMute}
className="h-8 w-8"
>
{isMuted || volume === 0 ? (
<VolumeX className="h-4 w-4" />
) : (
<Volume2 className="h-4 w-4" />
)}
</Button>
<Slider
value={[isMuted ? 0 : volume]}
max={1}
step={0.1}
onValueChange={handleVolumeChange}
className="w-20"
/>
</div>
</div>
{isLoading && (
<div className="text-xs text-muted-foreground text-center mt-2">
Loading audio...
</div>
)}
</div>
)
}
@@ -0,0 +1,3 @@
export { WaveformDisplay } from "./waveform-display"
export { AudioPlayer } from "./audio-player"
export { RegionSelector } from "./region-selector"
@@ -0,0 +1,222 @@
"use client"
import { Button } from "@/components/ui/button"
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"
import { Badge } from "@/components/ui/badge"
import { Trash2, VolumeX, Plus } from "lucide-react"
interface MuteRegion {
start_seconds: number
end_seconds: number
}
interface AudibleSegment {
start_seconds: number
end_seconds: number
duration_seconds: number
avg_amplitude_db: number
}
interface RegionSelectorProps {
muteRegions: MuteRegion[]
audibleSegments?: AudibleSegment[]
onAddRegion: (region: MuteRegion) => void
onRemoveRegion: (index: number) => void
onClearAll: () => void
onSeekTo?: (time: number) => void
isSelectionMode: boolean
onToggleSelectionMode: () => void
className?: string
}
export function RegionSelector({
muteRegions,
audibleSegments = [],
onAddRegion,
onRemoveRegion,
onClearAll,
onSeekTo,
isSelectionMode,
onToggleSelectionMode,
className = "",
}: RegionSelectorProps) {
const formatTime = (seconds: number) => {
const mins = Math.floor(seconds / 60)
const secs = Math.floor(seconds % 60)
const ms = Math.floor((seconds % 1) * 100)
return `${mins}:${secs.toString().padStart(2, "0")}.${ms.toString().padStart(2, "0")}`
}
const formatDuration = (seconds: number) => {
if (seconds < 1) {
return `${Math.round(seconds * 1000)}ms`
}
return `${seconds.toFixed(1)}s`
}
const totalMutedDuration = muteRegions.reduce(
(sum, r) => sum + (r.end_seconds - r.start_seconds),
0
)
// Quick mute: add mute region for an audible segment
const handleQuickMute = (segment: AudibleSegment) => {
onAddRegion({
start_seconds: segment.start_seconds,
end_seconds: segment.end_seconds,
})
}
// Check if a segment is already muted
const isSegmentMuted = (segment: AudibleSegment) => {
return muteRegions.some(
(r) =>
r.start_seconds <= segment.start_seconds &&
r.end_seconds >= segment.end_seconds
)
}
return (
<div className={`space-y-4 ${className}`}>
{/* Selection Mode Toggle */}
<Card>
<CardHeader className="pb-3">
<div className="flex items-center justify-between">
<CardTitle className="text-lg">Mute Regions</CardTitle>
<Button
variant={isSelectionMode ? "default" : "outline"}
size="sm"
onClick={onToggleSelectionMode}
>
<Plus className="h-4 w-4 mr-2" />
{isSelectionMode ? "Click waveform to select" : "Add Region"}
</Button>
</div>
</CardHeader>
<CardContent>
{muteRegions.length === 0 ? (
<p className="text-sm text-muted-foreground">
No mute regions selected. Click "Add Region" and drag on the waveform to select areas to mute.
</p>
) : (
<div className="space-y-2">
{muteRegions.map((region, index) => (
<div
key={index}
className="flex items-center justify-between p-2 bg-muted/50 rounded-lg group"
>
<div className="flex items-center gap-3">
<VolumeX className="h-4 w-4 text-destructive" />
<button
onClick={() => onSeekTo?.(region.start_seconds)}
className="text-sm font-mono hover:text-primary transition-colors"
>
{formatTime(region.start_seconds)}
</button>
<span className="text-muted-foreground">→</span>
<button
onClick={() => onSeekTo?.(region.end_seconds)}
className="text-sm font-mono hover:text-primary transition-colors"
>
{formatTime(region.end_seconds)}
</button>
<Badge variant="secondary" className="ml-2">
{formatDuration(region.end_seconds - region.start_seconds)}
</Badge>
</div>
<Button
variant="ghost"
size="icon"
onClick={() => onRemoveRegion(index)}
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
>
<Trash2 className="h-4 w-4 text-destructive" />
</Button>
</div>
))}
<div className="flex items-center justify-between pt-2 border-t mt-3">
<span className="text-sm text-muted-foreground">
Total muted: {formatDuration(totalMutedDuration)}
</span>
<Button
variant="ghost"
size="sm"
onClick={onClearAll}
className="text-destructive hover:text-destructive"
>
Clear All
</Button>
</div>
</div>
)}
</CardContent>
</Card>
{/* Detected Audible Segments */}
{audibleSegments.length > 0 && (
<Card>
<CardHeader className="pb-3">
<CardTitle className="text-lg">Detected Backing Vocals</CardTitle>
</CardHeader>
<CardContent>
<p className="text-sm text-muted-foreground mb-3">
Click a segment to jump to that time, or click the mute button to add it as a mute region.
</p>
<div className="space-y-2 max-h-64 overflow-y-auto">
{audibleSegments.map((segment, index) => {
const isMuted = isSegmentMuted(segment)
return (
<div
key={index}
className={`flex items-center justify-between p-2 rounded-lg transition-colors ${
isMuted ? "bg-destructive/10" : "bg-muted/50 hover:bg-muted"
}`}
>
<button
onClick={() => onSeekTo?.(segment.start_seconds)}
className="flex items-center gap-3 text-left"
>
<span className="text-sm font-mono">
{formatTime(segment.start_seconds)} - {formatTime(segment.end_seconds)}
</span>
<Badge variant="outline" className="text-xs">
{formatDuration(segment.duration_seconds)}
</Badge>
<Badge
variant="outline"
className={`text-xs ${
segment.avg_amplitude_db > -20
? "border-yellow-500 text-yellow-500"
: "border-muted-foreground"
}`}
>
{segment.avg_amplitude_db.toFixed(1)}dB
</Badge>
</button>
{!isMuted && (
<Button
variant="ghost"
size="sm"
onClick={() => handleQuickMute(segment)}
className="text-xs"
>
<VolumeX className="h-3 w-3 mr-1" />
Mute
</Button>
)}
{isMuted && (
<Badge variant="destructive" className="text-xs">
Muted
</Badge>
)}
</div>
)
})}
</div>
</CardContent>
</Card>
)}
</div>
)
}
@@ -0,0 +1,259 @@
"use client"
import { useRef, useEffect, useState, useCallback } from "react"
interface AudibleSegment {
start_seconds: number
end_seconds: number
duration_seconds: number
avg_amplitude_db: number
}
interface MuteRegion {
start_seconds: number
end_seconds: number
}
interface WaveformDisplayProps {
amplitudes: number[]
duration: number
currentTime?: number
segments?: AudibleSegment[]
muteRegions?: MuteRegion[]
onSeek?: (time: number) => void
onRegionSelect?: (region: MuteRegion) => void
isSelecting?: boolean
width?: number
height?: number
className?: string
}
export function WaveformDisplay({
amplitudes,
duration,
currentTime = 0,
segments = [],
muteRegions = [],
onSeek,
onRegionSelect,
isSelecting = false,
width = 800,
height = 150,
className = "",
}: WaveformDisplayProps) {
const canvasRef = useRef<HTMLCanvasElement>(null)
const [isDragging, setIsDragging] = useState(false)
const [dragStart, setDragStart] = useState<number | null>(null)
const [dragEnd, setDragEnd] = useState<number | null>(null)
const [hoveredTime, setHoveredTime] = useState<number | null>(null)
// Convert time to x position
const timeToX = useCallback((time: number) => {
return (time / duration) * width
}, [duration, width])
// Convert x position to time
const xToTime = useCallback((x: number) => {
return (x / width) * duration
}, [duration, width])
// Draw the waveform
useEffect(() => {
const canvas = canvasRef.current
if (!canvas) return
const ctx = canvas.getContext("2d")
if (!ctx) return
// Clear canvas
ctx.fillStyle = "#1a1a2e"
ctx.fillRect(0, 0, width, height)
const barWidth = width / amplitudes.length
const centerY = height / 2
// Draw mute regions background
ctx.fillStyle = "rgba(255, 107, 107, 0.2)"
for (const region of muteRegions) {
const startX = timeToX(region.start_seconds)
const endX = timeToX(region.end_seconds)
ctx.fillRect(startX, 0, endX - startX, height)
}
// Draw current drag selection
if (isDragging && dragStart !== null && dragEnd !== null) {
const startX = Math.min(timeToX(dragStart), timeToX(dragEnd))
const endX = Math.max(timeToX(dragStart), timeToX(dragEnd))
ctx.fillStyle = "rgba(255, 107, 107, 0.3)"
ctx.fillRect(startX, 0, endX - startX, height)
}
// Draw segment highlights
for (const segment of segments) {
const startX = timeToX(segment.start_seconds)
const endX = timeToX(segment.end_seconds)
const startIdx = Math.floor((segment.start_seconds / duration) * amplitudes.length)
const endIdx = Math.ceil((segment.end_seconds / duration) * amplitudes.length)
ctx.fillStyle = "rgba(233, 69, 96, 0.4)"
for (let i = startIdx; i < endIdx && i < amplitudes.length; i++) {
const amplitude = amplitudes[i]
const barHeight = amplitude * height * 0.8
const x = i * barWidth
ctx.fillRect(x, centerY - barHeight / 2, barWidth - 1, barHeight)
}
}
// Draw main waveform
ctx.fillStyle = "#4a90d9"
for (let i = 0; i < amplitudes.length; i++) {
const amplitude = amplitudes[i]
const barHeight = amplitude * height * 0.8
const x = i * barWidth
ctx.fillRect(x, centerY - barHeight / 2, barWidth - 1, barHeight)
}
// Draw silence threshold line
ctx.strokeStyle = "rgba(255, 255, 255, 0.2)"
ctx.setLineDash([5, 5])
ctx.beginPath()
const thresholdY = height * 0.5 // Visual representation
ctx.moveTo(0, thresholdY)
ctx.lineTo(width, thresholdY)
ctx.stroke()
ctx.setLineDash([])
// Draw playhead
if (currentTime > 0) {
const playheadX = timeToX(currentTime)
ctx.strokeStyle = "#ffffff"
ctx.lineWidth = 2
ctx.beginPath()
ctx.moveTo(playheadX, 0)
ctx.lineTo(playheadX, height)
ctx.stroke()
}
// Draw hover position
if (hoveredTime !== null && !isDragging) {
const hoverX = timeToX(hoveredTime)
ctx.strokeStyle = "rgba(255, 255, 255, 0.5)"
ctx.lineWidth = 1
ctx.beginPath()
ctx.moveTo(hoverX, 0)
ctx.lineTo(hoverX, height)
ctx.stroke()
}
// Draw time axis
ctx.fillStyle = "#ffffff"
ctx.font = "10px sans-serif"
ctx.textAlign = "center"
const tickInterval = duration > 300 ? 60 : duration > 60 ? 30 : 10
for (let t = 0; t <= duration; t += tickInterval) {
const x = timeToX(t)
const minutes = Math.floor(t / 60)
const seconds = Math.floor(t % 60)
const label = `${minutes}:${seconds.toString().padStart(2, "0")}`
ctx.fillText(label, x, height - 5)
// Tick mark
ctx.strokeStyle = "rgba(255, 255, 255, 0.3)"
ctx.beginPath()
ctx.moveTo(x, height - 20)
ctx.lineTo(x, height - 15)
ctx.stroke()
}
}, [amplitudes, duration, currentTime, segments, muteRegions, width, height, isDragging, dragStart, dragEnd, hoveredTime, timeToX])
// Handle mouse events for seeking and region selection
const handleMouseDown = (e: React.MouseEvent<HTMLCanvasElement>) => {
const canvas = canvasRef.current
if (!canvas) return
const rect = canvas.getBoundingClientRect()
const x = e.clientX - rect.left
const time = xToTime(x)
if (isSelecting) {
setIsDragging(true)
setDragStart(time)
setDragEnd(time)
} else if (onSeek) {
onSeek(time)
}
}
const handleMouseMove = (e: React.MouseEvent<HTMLCanvasElement>) => {
const canvas = canvasRef.current
if (!canvas) return
const rect = canvas.getBoundingClientRect()
const x = e.clientX - rect.left
const time = xToTime(x)
setHoveredTime(time)
if (isDragging && isSelecting) {
setDragEnd(time)
}
}
const handleMouseUp = () => {
if (isDragging && dragStart !== null && dragEnd !== null && onRegionSelect) {
const start = Math.min(dragStart, dragEnd)
const end = Math.max(dragStart, dragEnd)
// Only create region if it's at least 0.1 seconds
if (end - start >= 0.1) {
onRegionSelect({ start_seconds: start, end_seconds: end })
}
}
setIsDragging(false)
setDragStart(null)
setDragEnd(null)
}
const handleMouseLeave = () => {
setHoveredTime(null)
if (isDragging) {
handleMouseUp()
}
}
// Format time for tooltip
const formatTime = (time: number) => {
const minutes = Math.floor(time / 60)
const seconds = Math.floor(time % 60)
const ms = Math.floor((time % 1) * 100)
return `${minutes}:${seconds.toString().padStart(2, "0")}.${ms.toString().padStart(2, "0")}`
}
return (
<div className={`relative ${className}`}>
<canvas
ref={canvasRef}
width={width}
height={height}
className={`rounded-lg ${isSelecting ? "cursor-crosshair" : "cursor-pointer"}`}
onMouseDown={handleMouseDown}
onMouseMove={handleMouseMove}
onMouseUp={handleMouseUp}
onMouseLeave={handleMouseLeave}
/>
{/* Time tooltip */}
{hoveredTime !== null && (
<div
className="absolute top-0 transform -translate-x-1/2 bg-black/80 text-white text-xs px-2 py-1 rounded pointer-events-none"
style={{ left: timeToX(hoveredTime) }}
>
{formatTime(hoveredTime)}
</div>
)}
</div>
)
}
+182 -1
View File
@@ -2,10 +2,20 @@
Pulumi infrastructure for Karaoke Generator backend on Google Cloud Platform.
This replaces the manual setup scripts with proper Infrastructure as Code.
Infrastructure includes:
- Firestore database for job state
- Cloud Storage bucket for files
- Artifact Registry for Docker images
- Service accounts and IAM
- GitHub Actions Workload Identity Federation
- Cloud Tasks queues for worker coordination (Phase 1 scalability)
- Secret Manager secrets
- Cloud Run domain mapping
"""
import pulumi
import pulumi_gcp as gcp
from pulumi_gcp import storage, firestore, secretmanager, artifactregistry, cloudrun, serviceaccount
from pulumi_gcp import storage, firestore, secretmanager, artifactregistry, cloudrun, serviceaccount, cloudtasks
# Get the current GCP project
project = gcp.organizations.get_project()
@@ -227,6 +237,167 @@ github_actions_wif_binding = gcp.serviceaccount.IAMBinding(
)],
)
# ==================== Cloud Tasks Queues (Phase 1 Scalability) ====================
# These queues provide guaranteed delivery and horizontal scaling for worker tasks.
# Workers are triggered via Cloud Tasks instead of BackgroundTasks, ensuring:
# - Tasks survive container restarts
# - Each task gets dedicated resources
# - Automatic retries on failure
# - Rate limiting to protect external APIs
# Audio worker queue - calls Modal API for audio separation
audio_worker_queue = cloudtasks.Queue(
"audio-worker-queue",
name="audio-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=10, # Protect Modal API
max_concurrent_dispatches=50, # Max parallel audio workers
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=3,
min_backoff="10s",
max_backoff="300s",
max_retry_duration="1800s", # 30 min total retry window
),
)
# Lyrics worker queue - calls AudioShake API for transcription
lyrics_worker_queue = cloudtasks.Queue(
"lyrics-worker-queue",
name="lyrics-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=10,
max_concurrent_dispatches=50,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=3,
min_backoff="10s",
max_backoff="300s",
max_retry_duration="1200s", # 20 min total retry window
),
)
# Screens worker queue - generates title/end screens (fast, CPU-light)
screens_worker_queue = cloudtasks.Queue(
"screens-worker-queue",
name="screens-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=50, # Fast operations
max_concurrent_dispatches=100,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=3,
min_backoff="5s",
max_backoff="60s",
max_retry_duration="300s", # 5 min total retry window
),
)
# Render worker queue - LyricsTranscriber + FFmpeg (CPU-intensive)
render_worker_queue = cloudtasks.Queue(
"render-worker-queue",
name="render-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=5, # CPU-intensive
max_concurrent_dispatches=20,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=2,
min_backoff="30s",
max_backoff="300s",
max_retry_duration="3600s", # 60 min for render
),
)
# Video worker queue - final encoding (very CPU-intensive, longest running)
# Note: For tasks >30 min, consider Cloud Run Jobs instead
video_worker_queue = cloudtasks.Queue(
"video-worker-queue",
name="video-worker-queue",
location="us-central1",
rate_limits=cloudtasks.QueueRateLimitsArgs(
max_dispatches_per_second=3, # Very CPU-intensive
max_concurrent_dispatches=10,
),
retry_config=cloudtasks.QueueRetryConfigArgs(
max_attempts=2,
min_backoff="60s",
max_backoff="600s",
max_retry_duration="7200s", # 2 hour total (video is long)
),
)
# Grant Cloud Tasks permission to invoke Cloud Run service
# Cloud Tasks service agent needs to authenticate to Cloud Run
cloud_tasks_invoker = gcp.projects.IAMMember(
"cloud-tasks-invoker",
project=project_id,
role="roles/run.invoker",
member=f"serviceAccount:service-{project.number}@gcp-sa-cloudtasks.iam.gserviceaccount.com",
)
# Grant backend service account permission to enqueue Cloud Tasks
cloud_tasks_enqueuer = gcp.projects.IAMMember(
"karaoke-backend-cloudtasks-enqueuer",
project=project_id,
role="roles/cloudtasks.enqueuer",
member=service_account.email.apply(lambda email: f"serviceAccount:{email}"),
)
# ==================== Cloud Run Job for Video Encoding (Phase 2 Scalability) ====================
# Cloud Run Jobs provide up to 24-hour execution time for long-running video encoding.
# This is used when video encoding might exceed the 30-minute Cloud Tasks timeout.
# Triggered programmatically via WorkerService when ENABLE_CLOUD_TASKS=true and
# USE_CLOUD_RUN_JOBS_FOR_VIDEO=true
video_encoding_job = gcp.cloudrunv2.Job(
"video-encoding-job",
name="video-encoding-job",
location="us-central1",
template=gcp.cloudrunv2.JobTemplateArgs(
template=gcp.cloudrunv2.JobTemplateTemplateArgs(
# Use the same container as the backend service
# The job runs with --job-id argument to process a specific job
containers=[gcp.cloudrunv2.JobTemplateTemplateContainerArgs(
image=f"us-central1-docker.pkg.dev/{project_id}/karaoke-repo/karaoke-backend:latest",
args=["python", "-m", "backend.workers.video_worker"],
resources=gcp.cloudrunv2.JobTemplateTemplateContainerResourcesArgs(
limits={
"cpu": "4", # 4 vCPUs for encoding
"memory": "8Gi", # 8GB RAM
},
),
envs=[
gcp.cloudrunv2.JobTemplateTemplateContainerEnvArgs(
name="GOOGLE_CLOUD_PROJECT",
value=project_id,
),
gcp.cloudrunv2.JobTemplateTemplateContainerEnvArgs(
name="GCS_BUCKET_NAME",
value=bucket.name,
),
],
)],
service_account=service_account.email,
timeout="3600s", # 1 hour max per task
max_retries=2,
),
),
)
# Grant backend service account permission to run Cloud Run Jobs
cloud_run_jobs_developer = gcp.projects.IAMMember(
"karaoke-backend-run-jobs-developer",
project=project_id,
role="roles/run.developer",
member=service_account.email.apply(lambda email: f"serviceAccount:{email}"),
)
# ==================== Secrets ====================
# Create secrets (you'll need to add the actual secret values manually or via pulumi config)
config = pulumi.Config()
@@ -286,6 +457,16 @@ pulumi.export("backend_default_url", "https://karaoke-backend-ipzqd2k4yq-uc.a.ru
pulumi.export("github_actions_service_account", github_actions_sa.email)
pulumi.export("workload_identity_provider", workload_identity_provider.name)
# Cloud Tasks queue exports (Phase 1 scalability)
pulumi.export("audio_worker_queue", audio_worker_queue.name)
pulumi.export("lyrics_worker_queue", lyrics_worker_queue.name)
pulumi.export("screens_worker_queue", screens_worker_queue.name)
pulumi.export("render_worker_queue", render_worker_queue.name)
pulumi.export("video_worker_queue", video_worker_queue.name)
# Cloud Run Job export (Phase 2 scalability)
pulumi.export("video_encoding_job", video_encoding_job.name)
# Export DNS configuration needed for Cloudflare
# These are the records to add to your Cloudflare DNS
domain_mapping.statuses.apply(
+187 -27
View File
@@ -7,11 +7,18 @@ using flacfetch, replacing the previous direct yt-dlp usage.
import logging
import os
import signal
import sys
import tempfile
import threading
from abc import ABC, abstractmethod
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
from dataclasses import dataclass
from typing import List, Optional
# Global flag to track if user requested cancellation via Ctrl+C
_interrupt_requested = False
@dataclass
class AudioSearchResult:
@@ -58,6 +65,19 @@ class DownloadError(AudioFetcherError):
pass
class UserCancelledError(AudioFetcherError):
"""Raised when user explicitly cancels the operation (e.g., enters 0 or Ctrl+C)."""
pass
def _check_interrupt():
"""Check if interrupt was requested and raise UserCancelledError if so."""
global _interrupt_requested
if _interrupt_requested:
raise UserCancelledError("Operation cancelled by user")
class AudioFetcher(ABC):
"""Abstract base class for audio fetching implementations."""
@@ -169,25 +189,38 @@ class FlacFetchAudioFetcher(AudioFetcher):
if self._manager is None:
# Import flacfetch here to avoid import errors if not installed
from flacfetch.core.manager import FetchManager
from flacfetch.providers.youtube import YouTubeProvider
from flacfetch.providers.youtube import YoutubeProvider
from flacfetch.downloaders.youtube import YoutubeDownloader
# Try to import TorrentDownloader (has optional dependencies)
try:
from flacfetch.downloaders.torrent import TorrentDownloader
except ImportError:
TorrentDownloader = None
self.logger.debug("TorrentDownloader not available (missing dependencies)")
self._manager = FetchManager()
# Add providers based on available API keys
# Add providers and downloaders based on available API keys
if self._redacted_api_key:
from flacfetch.providers.redacted import RedactedProvider
self._manager.add_provider(RedactedProvider(api_key=self._redacted_api_key))
if TorrentDownloader:
self._manager.register_downloader("Redacted", TorrentDownloader())
self.logger.debug("Added Redacted provider")
if self._ops_api_key:
from flacfetch.providers.ops import OPSProvider
self._manager.add_provider(OPSProvider(api_key=self._ops_api_key))
if TorrentDownloader:
self._manager.register_downloader("OPS", TorrentDownloader())
self.logger.debug("Added OPS provider")
# Always add YouTube as a fallback provider
self._manager.add_provider(YouTubeProvider())
# Always add YouTube as a fallback provider with its downloader
self._manager.add_provider(YoutubeProvider())
self._manager.register_downloader("YouTube", YoutubeDownloader())
self.logger.debug("Added YouTube provider")
return self._manager
@@ -220,15 +253,19 @@ class FlacFetchAudioFetcher(AudioFetcher):
# Convert to our AudioSearchResult format
search_results = []
for result in results:
# Get quality as string if it's a Quality object
quality = getattr(result, "quality", None)
quality_str = str(quality) if quality else None
search_results.append(
AudioSearchResult(
title=getattr(result, "title", title),
artist=getattr(result, "artist", artist),
url=getattr(result, "url", ""),
provider=getattr(result, "provider", "Unknown"),
duration=getattr(result, "duration", None),
quality=getattr(result, "quality", None),
source_id=getattr(result, "id", None),
url=getattr(result, "download_url", "") or "",
provider=getattr(result, "source_name", "Unknown"),
duration=getattr(result, "duration_seconds", None),
quality=quality_str,
source_id=getattr(result, "info_hash", None),
raw_result=result,
)
)
@@ -265,7 +302,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
if output_filename is None:
output_filename = f"{result.artist} - {result.title}"
self.logger.info(f"Downloading: {result.artist} - {result.title} from {result.provider}")
self.logger.info(f"Downloading: {result.artist} - {result.title} from {result.provider or 'Unknown'}")
try:
# Use flacfetch to download
@@ -319,6 +356,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
Raises:
NoResultsError: If no results are found
DownloadError: If download fails
UserCancelledError: If user cancels (Ctrl+C or enters 0)
"""
from flacfetch.core.models import TrackQuery
@@ -326,7 +364,9 @@ class FlacFetchAudioFetcher(AudioFetcher):
query = TrackQuery(artist=artist, title=title)
self.logger.info(f"Searching for: {artist} - {title}")
results = manager.search(query)
# Run search in a thread so we can handle Ctrl+C
results = self._interruptible_search(manager, query)
if not results:
raise NoResultsError(f"No results found for: {artist} - {title}")
@@ -336,11 +376,13 @@ class FlacFetchAudioFetcher(AudioFetcher):
if auto_select:
# Auto mode: select best result based on flacfetch's ranking
selected = manager.select_best(results)
self.logger.info(f"Auto-selected: {getattr(selected, 'title', title)} from {getattr(selected, 'provider', 'Unknown')}")
self.logger.info(f"Auto-selected: {getattr(selected, 'title', title)} from {getattr(selected, 'source_name', 'Unknown')}")
else:
# Interactive mode: present options to user
selected = self._interactive_select(results, artist, title)
# Note: _interactive_select now raises UserCancelledError instead of returning None
# This check is kept as a safety net
if selected is None:
raise NoResultsError(f"No result selected for: {artist} - {title}")
@@ -351,7 +393,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
if output_filename is None:
output_filename = f"{artist} - {title}"
self.logger.info(f"Downloading from {getattr(selected, 'provider', 'Unknown')}...")
self.logger.info(f"Downloading from {getattr(selected, 'source_name', 'Unknown')}...")
try:
filepath = manager.download(
@@ -365,51 +407,169 @@ class FlacFetchAudioFetcher(AudioFetcher):
self.logger.info(f"Downloaded to: {filepath}")
# Get quality as string if it's a Quality object
quality = getattr(selected, "quality", None)
quality_str = str(quality) if quality else None
return AudioFetchResult(
filepath=filepath,
artist=artist,
title=title,
provider=getattr(selected, "provider", "Unknown"),
duration=getattr(selected, "duration", None),
quality=getattr(selected, "quality", None),
provider=getattr(selected, "source_name", "Unknown"),
duration=getattr(selected, "duration_seconds", None),
quality=quality_str,
)
except (UserCancelledError, KeyboardInterrupt):
# Let cancellation exceptions propagate without wrapping
raise
except Exception as e:
raise DownloadError(f"Failed to download {artist} - {title}: {e}") from e
def _interruptible_search(self, manager, query) -> list:
"""
Run search in a way that can be interrupted by Ctrl+C.
The flacfetch search is a blocking network operation that doesn't
respond to SIGINT while running. This method runs it in a background
thread and periodically checks for interrupts.
Args:
manager: The FetchManager instance
query: The TrackQuery to search for
Returns:
List of search results
Raises:
UserCancelledError: If user presses Ctrl+C during search
"""
global _interrupt_requested
_interrupt_requested = False
result_container = {"results": None, "error": None}
def do_search():
try:
result_container["results"] = manager.search(query)
except Exception as e:
result_container["error"] = e
# Set up signal handler for immediate response to Ctrl+C
original_handler = signal.getsignal(signal.SIGINT)
def interrupt_handler(signum, frame):
global _interrupt_requested
_interrupt_requested = True
# Print immediately so user knows it was received
print("\nCancelling... please wait", file=sys.stderr)
signal.signal(signal.SIGINT, interrupt_handler)
try:
# Start search in background thread
search_thread = threading.Thread(target=do_search, daemon=True)
search_thread.start()
# Wait for completion with periodic interrupt checks
while search_thread.is_alive():
search_thread.join(timeout=0.1) # Check every 100ms
if _interrupt_requested:
# Don't wait for thread - it's a daemon and will be killed
raise UserCancelledError("Search cancelled by user (Ctrl+C)")
# Check for errors from the search
if result_container["error"] is not None:
raise result_container["error"]
return result_container["results"]
finally:
# Restore original signal handler
signal.signal(signal.SIGINT, original_handler)
_interrupt_requested = False
def _interactive_select(self, results: list, artist: str, title: str) -> object:
"""
Present search results to the user for interactive selection.
Uses flacfetch's built-in CLIHandler for rich, colorized output.
Args:
results: List of search results from flacfetch
results: List of Release objects from flacfetch
artist: The artist name being searched
title: The track title being searched
Returns:
The selected result object
The selected Release object
Raises:
UserCancelledError: If user cancels selection
"""
try:
# Use flacfetch's built-in CLIHandler for rich display
from flacfetch.interface.cli import CLIHandler
handler = CLIHandler(target_artist=artist)
result = handler.select_release(results)
if result is None:
# User selected 0 to cancel
raise UserCancelledError("Selection cancelled by user")
return result
except ImportError:
# Fallback to basic display if CLIHandler not available
return self._basic_interactive_select(results, artist, title)
except (KeyboardInterrupt, EOFError):
raise UserCancelledError("Selection cancelled by user (Ctrl+C)")
except (AttributeError, TypeError):
# Fallback if results aren't proper Release objects (e.g., in tests)
return self._basic_interactive_select(results, artist, title)
def _basic_interactive_select(self, results: list, artist: str, title: str) -> object:
"""
Basic fallback for interactive selection without rich formatting.
Args:
results: List of Release objects from flacfetch
artist: The artist name being searched
title: The track title being searched
Returns:
The selected Release object
Raises:
UserCancelledError: If user cancels selection
"""
print(f"\n{'=' * 60}")
print(f"Search Results for: {artist} - {title}")
print(f"{'=' * 60}\n")
for i, result in enumerate(results, 1):
provider = getattr(result, "provider", "Unknown")
# Use correct flacfetch attribute names
source_name = getattr(result, "source_name", "Unknown")
result_title = getattr(result, "title", "Unknown")
result_artist = getattr(result, "artist", "Unknown")
quality = getattr(result, "quality", "")
duration = getattr(result, "duration", None)
quality = getattr(result, "quality", None)
duration_seconds = getattr(result, "duration_seconds", None)
seeders = getattr(result, "seeders", None)
target_file = getattr(result, "target_file", None)
# Format duration if available
duration_str = ""
if duration:
minutes = duration // 60
seconds = duration % 60
if duration_seconds:
minutes = duration_seconds // 60
seconds = duration_seconds % 60
duration_str = f" [{minutes}:{seconds:02d}]"
# Format quality - it's a Quality object with __str__
quality_str = f" ({quality})" if quality else ""
print(f" {i}. [{provider}] {result_artist} - {result_title}{quality_str}{duration_str}")
# Format seeders for torrent sources
seeders_str = f" Seeders: {seeders}" if seeders is not None else ""
# Format target file
file_str = f' "{target_file}"' if target_file else ""
print(f" {i}. [{source_name}] {result_artist} - {result_title}{quality_str}{duration_str}{seeders_str}{file_str}")
print()
print(" 0. Cancel")
@@ -421,7 +581,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
if choice == "0":
self.logger.info("User cancelled selection")
return None
raise UserCancelledError("Selection cancelled by user")
choice_num = int(choice)
if 1 <= choice_num <= len(results):
@@ -435,7 +595,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
print("Please enter a valid number")
except KeyboardInterrupt:
print("\nCancelled")
return None
raise UserCancelledError("Selection cancelled by user (Ctrl+C)")
def create_audio_fetcher(
+11 -3
View File
@@ -771,8 +771,16 @@ class AudioProcessor:
padded_result["other_stems"] = separation_result.get("other_stems", {})
padded_result["backing_vocals"] = separation_result.get("backing_vocals", {})
self.logger.info(
f"✓ Countdown padding applied to {len(padded_result['combined_instrumentals']) + 1} instrumental file(s)"
)
# Count actual padded files (don't assume clean instrumental was padded)
padded_count = (1 if padded_result["clean_instrumental"].get("instrumental") else 0) + len(padded_result["combined_instrumentals"])
if padded_count > 0:
self.logger.info(
f"✓ Countdown padding applied to {padded_count} instrumental file(s)"
)
else:
self.logger.warning(
"No instrumental files found to pad. Check that audio separation completed successfully."
)
return padded_result
+192
View File
@@ -5,6 +5,12 @@ import shutil
import tempfile
from .utils import sanitize_filename
try:
import yt_dlp
YT_DLP_AVAILABLE = True
except ImportError:
YT_DLP_AVAILABLE = False
# Placeholder class or functions for file handling
class FileHandler:
@@ -71,6 +77,192 @@ class FileHandler:
return target_path
def download_video(self, url, output_filename_no_extension, cookies_str=None):
"""
Download audio from a URL (YouTube, etc.) using yt-dlp.
This method downloads the best quality audio from a URL and saves it
to the specified output path. It handles YouTube and other video platforms
supported by yt-dlp.
Args:
url: URL to download from (YouTube, Vimeo, etc.)
output_filename_no_extension: Output filename without extension
cookies_str: Optional cookies string for authenticated downloads
Returns:
Path to downloaded audio file, or None if failed
"""
if not YT_DLP_AVAILABLE:
self.logger.error("yt-dlp is not installed. Install with: pip install yt-dlp")
return None
self.logger.info(f"Downloading audio from URL: {url}")
# Configure yt-dlp options
ydl_opts = {
'format': 'bestaudio/best',
'outtmpl': output_filename_no_extension + '.%(ext)s',
'postprocessors': [{
'key': 'FFmpegExtractAudio',
'preferredcodec': 'best',
'preferredquality': '0', # Best quality
}],
'quiet': True,
'no_warnings': True,
'extract_flat': False,
# Anti-detection options
'user_agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
'retries': 3,
'fragment_retries': 3,
'http_headers': {
'Accept-Language': 'en-US,en;q=0.9',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
},
}
# Handle cookies if provided - use safe tempfile pattern to avoid leaks
cookie_file_path = None
if cookies_str:
try:
# Use context manager to safely write cookies file
with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as cookie_file:
cookie_file.write(cookies_str)
cookie_file_path = cookie_file.name
ydl_opts['cookiefile'] = cookie_file_path
except Exception as e:
self.logger.warning(f"Failed to write cookies file: {e}")
cookie_file_path = None
try:
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
# Extract info first to get actual filename
info = ydl.extract_info(url, download=True)
if info is None:
self.logger.error("Failed to extract info from URL")
return None
# Find the downloaded file
# The actual filename might differ from template due to post-processing
downloaded_file = None
# Check common extensions
for ext in ['m4a', 'opus', 'webm', 'mp3', 'flac', 'wav', 'ogg', 'aac']:
candidate = f"{output_filename_no_extension}.{ext}"
if os.path.exists(candidate):
downloaded_file = candidate
break
if downloaded_file is None:
# Try to find any audio file with matching prefix
import glob
matches = glob.glob(f"{output_filename_no_extension}.*")
audio_extensions = ['.m4a', '.opus', '.webm', '.mp3', '.flac', '.wav', '.ogg', '.aac']
for match in matches:
if any(match.endswith(ext) for ext in audio_extensions):
downloaded_file = match
break
if downloaded_file and os.path.exists(downloaded_file):
self.logger.info(f"Successfully downloaded: {downloaded_file}")
return downloaded_file
else:
self.logger.error("Downloaded file not found after yt-dlp completed")
return None
except yt_dlp.DownloadError as e:
self.logger.error(f"yt-dlp download error: {e}")
return None
except Exception as e:
self.logger.error(f"Failed to download from URL: {e}")
return None
finally:
# Clean up cookie file if we created one
if cookie_file_path is not None:
try:
os.unlink(cookie_file_path)
except Exception:
pass
def extract_metadata_from_url(self, url):
"""
Extract metadata (artist, title) from a URL without downloading.
Uses yt-dlp to fetch video metadata including title, uploader/artist,
and other information that can be used for the karaoke generation.
Args:
url: URL to extract metadata from
Returns:
Dict with 'artist', 'title', 'duration', and 'raw_info', or None if failed
"""
if not YT_DLP_AVAILABLE:
self.logger.error("yt-dlp is not installed. Install with: pip install yt-dlp")
return None
self.logger.info(f"Extracting metadata from URL: {url}")
ydl_opts = {
'quiet': True,
'no_warnings': True,
'extract_flat': False,
'skip_download': True,
}
try:
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
info = ydl.extract_info(url, download=False)
if info is None:
self.logger.error("Failed to extract metadata from URL")
return None
# Try to extract artist and title from various fields
raw_title = info.get('title', '')
uploader = info.get('uploader', '') or info.get('channel', '') or info.get('artist', '')
duration = info.get('duration', 0)
# Attempt to parse "Artist - Title" format from title
artist = None
title = raw_title
if ' - ' in raw_title:
parts = raw_title.split(' - ', 1)
if len(parts) == 2:
artist = parts[0].strip()
title = parts[1].strip()
# Fall back to uploader as artist if not found in title
if not artist:
artist = uploader
# Clean up title (remove common suffixes like "(Official Video)")
title_cleanup_patterns = [
'(official video)', '(official music video)', '(official audio)',
'(lyric video)', '(lyrics)', '(visualizer)', '(music video)',
'[official video]', '[official music video]', '[official audio]',
'(hd)', '(4k)', '(remastered)', '| official video', '| official audio',
]
title_lower = title.lower()
for pattern in title_cleanup_patterns:
if pattern in title_lower:
idx = title_lower.find(pattern)
title = title[:idx].strip()
title_lower = title.lower()
return {
'artist': artist,
'title': title,
'duration': duration,
'raw_info': info,
}
except Exception as e:
self.logger.error(f"Failed to extract metadata from URL: {e}")
return None
def extract_still_image_from_video(self, input_filename, output_filename_no_extension):
output_filename = output_filename_no_extension + ".png"
self.logger.info(f"Extracting still image from position 30s input media")
@@ -0,0 +1,45 @@
"""
Instrumental Review Module - Shared core for both local and remote CLI.
This module provides audio analysis and editing functionality for instrumental
selection in karaoke generation. It's designed to be:
- Pure Python with no cloud dependencies (GCS, etc.)
- Reusable by both local CLI (karaoke-gen) and remote backend (Cloud Run)
- Easy to test without mocking cloud services
Classes:
AudioAnalyzer: Analyzes backing vocals audio for audible content
AudioEditor: Creates custom instrumentals by muting regions
WaveformGenerator: Generates waveform visualization images
InstrumentalReviewServer: Local HTTP server for browser-based review
Models:
AnalysisResult: Result of audio analysis
AudibleSegment: A detected segment of audible content
MuteRegion: A region to mute in the backing vocals
RecommendedSelection: Enum of selection recommendations
"""
from .models import (
AnalysisResult,
AudibleSegment,
MuteRegion,
RecommendedSelection,
)
from .analyzer import AudioAnalyzer
from .editor import AudioEditor
from .waveform import WaveformGenerator
from .server import InstrumentalReviewServer
__all__ = [
# Models
"AnalysisResult",
"AudibleSegment",
"MuteRegion",
"RecommendedSelection",
# Classes
"AudioAnalyzer",
"AudioEditor",
"WaveformGenerator",
"InstrumentalReviewServer",
]
+408
View File
@@ -0,0 +1,408 @@
"""
Audio analyzer for detecting audible content in backing vocals.
This module provides the AudioAnalyzer class which analyzes audio files
to detect segments of audible content above a silence threshold. It's used
to help determine whether backing vocals should be included in the final
karaoke instrumental.
"""
import logging
import math
from pathlib import Path
from typing import List, Optional, Tuple
from pydub import AudioSegment
from .models import AnalysisResult, AudibleSegment, RecommendedSelection
logger = logging.getLogger(__name__)
class AudioAnalyzer:
"""
Analyzes audio files for backing vocals content.
This class is pure Python with no cloud dependencies. It works with
local file paths and uses pydub for audio loading and analysis.
The analyzer detects segments of audible content (above a silence threshold)
and provides recommendations for instrumental selection based on the
analysis results.
Attributes:
silence_threshold_db: Amplitude threshold below which audio is
considered silent (default: -40.0 dB)
min_segment_duration_ms: Minimum duration for a segment to be
considered audible (default: 100ms)
merge_gap_ms: Maximum gap between segments to merge them
(default: 500ms)
window_ms: Analysis window size in milliseconds (default: 50ms)
Example:
>>> analyzer = AudioAnalyzer(silence_threshold_db=-40.0)
>>> result = analyzer.analyze("/path/to/backing_vocals.flac")
>>> if result.has_audible_content:
... print(f"Found {result.segment_count} audible segments")
... for seg in result.audible_segments:
... print(f" {seg.start_seconds:.1f}s - {seg.end_seconds:.1f}s")
"""
def __init__(
self,
silence_threshold_db: float = -40.0,
min_segment_duration_ms: int = 100,
merge_gap_ms: int = 500,
window_ms: int = 50,
):
"""
Initialize the audio analyzer.
Args:
silence_threshold_db: Amplitude threshold below which audio is
considered silent. Default is -40.0 dB.
min_segment_duration_ms: Minimum duration for a segment to be
reported as audible. Segments shorter than this are ignored.
Default is 100ms.
merge_gap_ms: If two audible segments are separated by a gap
shorter than this, they are merged into one segment.
Default is 500ms.
window_ms: Size of the analysis window in milliseconds.
Smaller windows give more precise timing but slower analysis.
Default is 50ms.
"""
self.silence_threshold_db = silence_threshold_db
self.min_segment_duration_ms = min_segment_duration_ms
self.merge_gap_ms = merge_gap_ms
self.window_ms = window_ms
def analyze(self, audio_path: str) -> AnalysisResult:
"""
Analyze an audio file for audible content.
This method loads the audio file, calculates amplitude levels across
the duration, and identifies segments where the amplitude exceeds
the silence threshold.
Args:
audio_path: Path to the audio file to analyze. Supports formats
that pydub/ffmpeg can read (FLAC, WAV, MP3, etc.)
Returns:
AnalysisResult containing:
- has_audible_content: Whether any audible content was found
- total_duration_seconds: Total duration of the audio
- audible_segments: List of detected audible segments
- recommended_selection: Recommendation for which instrumental
- Various statistics about the audible content
Raises:
FileNotFoundError: If the audio file doesn't exist
Exception: If the audio file cannot be loaded
"""
path = Path(audio_path)
if not path.exists():
raise FileNotFoundError(f"Audio file not found: {audio_path}")
logger.info(f"Analyzing audio file: {audio_path}")
# Load audio file
audio = AudioSegment.from_file(audio_path)
total_duration_ms = len(audio)
total_duration_seconds = total_duration_ms / 1000.0
logger.debug(f"Audio duration: {total_duration_seconds:.2f}s, "
f"channels: {audio.channels}, "
f"sample_rate: {audio.frame_rate}")
# Convert to mono for consistent analysis
if audio.channels > 1:
audio = audio.set_channels(1)
# Analyze amplitude in windows
audible_windows = self._find_audible_windows(audio)
# Merge adjacent windows into segments
raw_segments = self._windows_to_segments(audible_windows, audio)
# Merge close segments and filter short ones
segments = self._merge_and_filter_segments(raw_segments)
# Calculate statistics
total_audible_ms = sum(
seg.duration_seconds * 1000 for seg in segments
)
total_audible_seconds = total_audible_ms / 1000.0
audible_percentage = (
(total_audible_seconds / total_duration_seconds * 100)
if total_duration_seconds > 0 else 0.0
)
has_audible_content = len(segments) > 0
# Determine recommendation
recommended_selection = self._get_recommendation(
has_audible_content,
segments,
audible_percentage
)
logger.info(
f"Analysis complete: {len(segments)} segments, "
f"{audible_percentage:.1f}% audible, "
f"recommendation: {recommended_selection.value}"
)
return AnalysisResult(
has_audible_content=has_audible_content,
total_duration_seconds=total_duration_seconds,
audible_segments=segments,
recommended_selection=recommended_selection,
silence_threshold_db=self.silence_threshold_db,
total_audible_duration_seconds=total_audible_seconds,
audible_percentage=audible_percentage,
)
def get_amplitude_envelope(
self,
audio_path: str,
window_ms: int = 100,
normalize: bool = True,
) -> List[float]:
"""
Get the amplitude envelope for waveform visualization.
This method returns a list of amplitude values suitable for
rendering a waveform display. Each value represents the RMS
amplitude of a window of audio.
Args:
audio_path: Path to the audio file
window_ms: Size of each window in milliseconds. Smaller values
give more detail but larger data. Default is 100ms.
normalize: If True, normalize amplitudes to 0.0-1.0 range.
Default is True.
Returns:
List of amplitude values (floats). If normalize=True, values
are in the range [0.0, 1.0]. Otherwise, values are in dBFS.
"""
path = Path(audio_path)
if not path.exists():
raise FileNotFoundError(f"Audio file not found: {audio_path}")
audio = AudioSegment.from_file(audio_path)
# Convert to mono
if audio.channels > 1:
audio = audio.set_channels(1)
amplitudes = []
duration_ms = len(audio)
for start_ms in range(0, duration_ms, window_ms):
end_ms = min(start_ms + window_ms, duration_ms)
window = audio[start_ms:end_ms]
# Get RMS amplitude in dBFS
if window.rms > 0:
db = 20 * math.log10(window.rms / window.max_possible_amplitude)
else:
db = -100.0 # Effectively silent
amplitudes.append(db)
if normalize and amplitudes:
# Normalize to 0.0 - 1.0 range
# Map from [silence_threshold, 0] to [0, 1]
min_db = self.silence_threshold_db
max_db = 0.0
amplitudes = [
max(0.0, min(1.0, (db - min_db) / (max_db - min_db)))
for db in amplitudes
]
return amplitudes
def _find_audible_windows(
self,
audio: AudioSegment
) -> List[Tuple[int, float, float]]:
"""
Find windows with amplitude above the silence threshold.
Returns a list of tuples: (start_ms, avg_db, peak_db)
"""
audible_windows = []
duration_ms = len(audio)
for start_ms in range(0, duration_ms, self.window_ms):
end_ms = min(start_ms + self.window_ms, duration_ms)
window = audio[start_ms:end_ms]
# Calculate RMS amplitude in dB
if window.rms > 0:
avg_db = 20 * math.log10(window.rms / window.max_possible_amplitude)
# Peak is approximated as max sample value
peak_db = window.dBFS if hasattr(window, 'dBFS') else avg_db
else:
avg_db = -100.0
peak_db = -100.0
if avg_db > self.silence_threshold_db:
audible_windows.append((start_ms, avg_db, peak_db))
return audible_windows
def _windows_to_segments(
self,
audible_windows: List[Tuple[int, float, float]],
audio: AudioSegment
) -> List[AudibleSegment]:
"""
Convert list of audible windows into contiguous segments.
"""
if not audible_windows:
return []
segments = []
segment_start_ms = audible_windows[0][0]
segment_dbs = [audible_windows[0][1]]
segment_peaks = [audible_windows[0][2]]
last_end_ms = audible_windows[0][0] + self.window_ms
for i in range(1, len(audible_windows)):
start_ms, avg_db, peak_db = audible_windows[i]
# Check if this window is contiguous with the previous
gap_ms = start_ms - last_end_ms
if gap_ms <= self.window_ms:
# Extend current segment
segment_dbs.append(avg_db)
segment_peaks.append(peak_db)
last_end_ms = start_ms + self.window_ms
else:
# Save current segment and start a new one
segments.append(self._create_segment(
segment_start_ms, last_end_ms, segment_dbs, segment_peaks
))
segment_start_ms = start_ms
segment_dbs = [avg_db]
segment_peaks = [peak_db]
last_end_ms = start_ms + self.window_ms
# Don't forget the last segment
segments.append(self._create_segment(
segment_start_ms, last_end_ms, segment_dbs, segment_peaks
))
return segments
def _create_segment(
self,
start_ms: int,
end_ms: int,
dbs: List[float],
peaks: List[float]
) -> AudibleSegment:
"""Create an AudibleSegment from window data."""
return AudibleSegment(
start_seconds=start_ms / 1000.0,
end_seconds=end_ms / 1000.0,
duration_seconds=(end_ms - start_ms) / 1000.0,
avg_amplitude_db=sum(dbs) / len(dbs) if dbs else -100.0,
peak_amplitude_db=max(peaks) if peaks else -100.0,
)
def _merge_and_filter_segments(
self,
segments: List[AudibleSegment]
) -> List[AudibleSegment]:
"""
Merge segments that are close together and filter out short ones.
"""
if not segments:
return []
# Sort by start time
segments = sorted(segments, key=lambda s: s.start_seconds)
# Merge segments with small gaps
merged = []
current = segments[0]
for next_seg in segments[1:]:
gap_ms = (next_seg.start_seconds - current.end_seconds) * 1000
if gap_ms <= self.merge_gap_ms:
# Merge segments
combined_duration = (
next_seg.end_seconds - current.start_seconds
)
# Weight average amplitude by duration
total_duration = (
current.duration_seconds + next_seg.duration_seconds
)
weighted_avg_db = (
(current.avg_amplitude_db * current.duration_seconds +
next_seg.avg_amplitude_db * next_seg.duration_seconds)
/ total_duration
) if total_duration > 0 else -100.0
current = AudibleSegment(
start_seconds=current.start_seconds,
end_seconds=next_seg.end_seconds,
duration_seconds=combined_duration,
avg_amplitude_db=weighted_avg_db,
peak_amplitude_db=max(
current.peak_amplitude_db,
next_seg.peak_amplitude_db
),
)
else:
merged.append(current)
current = next_seg
merged.append(current)
# Filter out segments shorter than minimum duration
min_duration_seconds = self.min_segment_duration_ms / 1000.0
filtered = [
seg for seg in merged
if seg.duration_seconds >= min_duration_seconds
]
return filtered
def _get_recommendation(
self,
has_audible_content: bool,
segments: List[AudibleSegment],
audible_percentage: float
) -> RecommendedSelection:
"""
Determine the recommended instrumental selection.
Logic:
- If no audible content: recommend clean instrumental
- If audible content covers > 20% of the audio: likely has
meaningful backing vocals, recommend review
- Otherwise: minimal content, recommend clean
"""
if not has_audible_content:
return RecommendedSelection.CLEAN
# If there's significant audible content, recommend review
if audible_percentage > 20.0:
return RecommendedSelection.REVIEW_NEEDED
# If there are loud segments, recommend review
loud_segments = [seg for seg in segments if seg.is_loud]
if loud_segments:
return RecommendedSelection.REVIEW_NEEDED
# Minimal content - recommend clean
return RecommendedSelection.CLEAN
+322
View File
@@ -0,0 +1,322 @@
"""
Audio editor for creating custom instrumentals with muted regions.
This module provides the AudioEditor class which creates custom instrumental
tracks by muting specified regions of backing vocals audio and combining
it with the clean instrumental.
"""
import logging
import os
from pathlib import Path
from typing import List, Optional
from pydub import AudioSegment
from .models import CustomInstrumentalResult, MuteRegion
logger = logging.getLogger(__name__)
class AudioEditor:
"""
Creates custom instrumentals by muting regions of backing vocals.
This class is pure Python with no cloud dependencies. It works with
local file paths and uses pydub for audio manipulation.
The editor takes a clean instrumental and backing vocals track,
applies silence to specified regions of the backing vocals, and
combines them to create a custom instrumental track.
Example:
>>> editor = AudioEditor()
>>> mute_regions = [
... MuteRegion(start_seconds=45.0, end_seconds=48.5),
... MuteRegion(start_seconds=120.0, end_seconds=125.0),
... ]
>>> result = editor.create_custom_instrumental(
... clean_instrumental_path="/path/to/clean.flac",
... backing_vocals_path="/path/to/backing.flac",
... mute_regions=mute_regions,
... output_path="/path/to/custom.flac"
... )
>>> print(f"Created: {result.output_path}")
"""
def __init__(self, output_format: str = "flac"):
"""
Initialize the audio editor.
Args:
output_format: Output audio format. Default is "flac".
Supported formats depend on ffmpeg installation.
"""
self.output_format = output_format
def create_custom_instrumental(
self,
clean_instrumental_path: str,
backing_vocals_path: str,
mute_regions: List[MuteRegion],
output_path: str,
) -> CustomInstrumentalResult:
"""
Create a custom instrumental by muting regions of backing vocals.
This method:
1. Loads the clean instrumental and backing vocals tracks
2. Applies silence to the specified regions of the backing vocals
3. Combines the clean instrumental with the edited backing vocals
4. Exports the result to the specified output path
Args:
clean_instrumental_path: Path to the clean instrumental audio file
backing_vocals_path: Path to the backing vocals audio file
mute_regions: List of regions to mute in the backing vocals
output_path: Path where the output file should be saved
Returns:
CustomInstrumentalResult containing the output path and statistics
Raises:
FileNotFoundError: If input files don't exist
ValueError: If mute regions are invalid
"""
# Validate inputs
if not Path(clean_instrumental_path).exists():
raise FileNotFoundError(
f"Clean instrumental not found: {clean_instrumental_path}"
)
if not Path(backing_vocals_path).exists():
raise FileNotFoundError(
f"Backing vocals not found: {backing_vocals_path}"
)
# Normalize and validate mute regions
normalized_regions = self._normalize_mute_regions(mute_regions)
logger.info(
f"Creating custom instrumental with {len(normalized_regions)} "
f"mute regions"
)
# Load audio files
logger.debug(f"Loading clean instrumental: {clean_instrumental_path}")
clean_instrumental = AudioSegment.from_file(clean_instrumental_path)
logger.debug(f"Loading backing vocals: {backing_vocals_path}")
backing_vocals = AudioSegment.from_file(backing_vocals_path)
# Ensure same duration (use shorter one)
clean_duration_ms = len(clean_instrumental)
backing_duration_ms = len(backing_vocals)
if abs(clean_duration_ms - backing_duration_ms) > 1000:
logger.warning(
f"Duration mismatch: clean={clean_duration_ms}ms, "
f"backing={backing_duration_ms}ms. Using shorter duration."
)
target_duration_ms = min(clean_duration_ms, backing_duration_ms)
clean_instrumental = clean_instrumental[:target_duration_ms]
backing_vocals = backing_vocals[:target_duration_ms]
# Apply mute regions to backing vocals
edited_backing = self._apply_mute_regions(
backing_vocals, normalized_regions
)
# Combine: clean instrumental + edited backing vocals
# The backing vocals are mixed on top of the clean instrumental
combined = clean_instrumental.overlay(edited_backing)
# Ensure output directory exists
output_dir = Path(output_path).parent
output_dir.mkdir(parents=True, exist_ok=True)
# Export
logger.info(f"Exporting custom instrumental to: {output_path}")
combined.export(output_path, format=self.output_format)
# Calculate statistics with clamping to actual audio duration
output_duration_seconds = len(combined) / 1000.0
total_muted_ms = sum(
(min(r.end_seconds, output_duration_seconds) - max(r.start_seconds, 0)) * 1000
for r in normalized_regions
if r.start_seconds < output_duration_seconds # Skip regions entirely outside audio
)
return CustomInstrumentalResult(
output_path=output_path,
mute_regions_applied=normalized_regions,
total_muted_duration_seconds=max(0, total_muted_ms / 1000.0),
output_duration_seconds=output_duration_seconds,
)
def apply_mute_to_single_track(
self,
audio_path: str,
mute_regions: List[MuteRegion],
output_path: str,
) -> str:
"""
Apply mute regions to a single audio track.
This is useful for muting sections of backing vocals without
combining with the clean instrumental.
Args:
audio_path: Path to the input audio file
mute_regions: List of regions to mute
output_path: Path where the output file should be saved
Returns:
Path to the output file
"""
if not Path(audio_path).exists():
raise FileNotFoundError(f"Audio file not found: {audio_path}")
normalized_regions = self._normalize_mute_regions(mute_regions)
logger.info(f"Applying {len(normalized_regions)} mute regions to audio")
audio = AudioSegment.from_file(audio_path)
edited = self._apply_mute_regions(audio, normalized_regions)
output_dir = Path(output_path).parent
output_dir.mkdir(parents=True, exist_ok=True)
edited.export(output_path, format=self.output_format)
return output_path
def _normalize_mute_regions(
self,
regions: List[MuteRegion]
) -> List[MuteRegion]:
"""
Normalize mute regions: sort, validate, and merge overlapping ones.
"""
if not regions:
return []
# Validate regions
for region in regions:
if region.start_seconds < 0:
raise ValueError(
f"Mute region start cannot be negative: {region.start_seconds}"
)
if region.end_seconds <= region.start_seconds:
raise ValueError(
f"Mute region end ({region.end_seconds}) must be after "
f"start ({region.start_seconds})"
)
# Sort by start time
sorted_regions = sorted(regions, key=lambda r: r.start_seconds)
# Merge overlapping regions
merged = [sorted_regions[0]]
for region in sorted_regions[1:]:
last = merged[-1]
# Check if overlapping or adjacent
if region.start_seconds <= last.end_seconds:
# Merge
merged[-1] = MuteRegion(
start_seconds=last.start_seconds,
end_seconds=max(last.end_seconds, region.end_seconds)
)
else:
merged.append(region)
return merged
def _apply_mute_regions(
self,
audio: AudioSegment,
regions: List[MuteRegion]
) -> AudioSegment:
"""
Apply silence to the specified regions of an audio segment.
"""
if not regions:
return audio
duration_ms = len(audio)
result = audio
for region in regions:
start_ms = int(region.start_seconds * 1000)
end_ms = int(region.end_seconds * 1000)
# Clamp to audio boundaries
start_ms = max(0, start_ms)
end_ms = min(duration_ms, end_ms)
if start_ms >= end_ms:
continue
# Create silence segment
silence_duration = end_ms - start_ms
silence = AudioSegment.silent(
duration=silence_duration,
frame_rate=audio.frame_rate
)
# Replace the region with silence
before = result[:start_ms]
after = result[end_ms:]
result = before + silence + after
logger.debug(
f"Muted region: {start_ms/1000:.2f}s - {end_ms/1000:.2f}s"
)
return result
def preview_with_mutes(
self,
clean_instrumental_path: str,
backing_vocals_path: str,
mute_regions: List[MuteRegion],
output_path: Optional[str] = None,
) -> AudioSegment:
"""
Create a preview of the custom instrumental (in memory).
This is useful for creating temporary previews without saving
to disk. If output_path is provided, the preview is also saved.
Args:
clean_instrumental_path: Path to the clean instrumental
backing_vocals_path: Path to the backing vocals
mute_regions: Regions to mute
output_path: Optional path to save the preview
Returns:
AudioSegment of the preview
"""
clean_instrumental = AudioSegment.from_file(clean_instrumental_path)
backing_vocals = AudioSegment.from_file(backing_vocals_path)
# Match durations
target_duration = min(len(clean_instrumental), len(backing_vocals))
clean_instrumental = clean_instrumental[:target_duration]
backing_vocals = backing_vocals[:target_duration]
# Apply mutes
normalized = self._normalize_mute_regions(mute_regions)
edited_backing = self._apply_mute_regions(backing_vocals, normalized)
# Combine
combined = clean_instrumental.overlay(edited_backing)
if output_path:
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
combined.export(output_path, format=self.output_format)
return combined
+171
View File
@@ -0,0 +1,171 @@
"""
Pydantic models for the instrumental review module.
These models define the data structures used for audio analysis results,
mute regions, and recommendations. They're designed to be serializable
to JSON for API responses.
"""
from enum import Enum
from typing import List, Optional
from pydantic import BaseModel, Field
class RecommendedSelection(str, Enum):
"""Recommendation for which instrumental to use."""
CLEAN = "clean"
"""Use clean instrumental - no backing vocals detected or recommended."""
WITH_BACKING = "with_backing"
"""Use instrumental with backing vocals - backing vocals sound good."""
REVIEW_NEEDED = "review_needed"
"""Human review needed - backing vocals detected but quality uncertain."""
class AudibleSegment(BaseModel):
"""A detected segment of audible content in the backing vocals."""
start_seconds: float = Field(
...,
ge=0,
description="Start time of the segment in seconds"
)
end_seconds: float = Field(
...,
ge=0,
description="End time of the segment in seconds"
)
duration_seconds: float = Field(
...,
ge=0,
description="Duration of the segment in seconds"
)
avg_amplitude_db: float = Field(
...,
description="Average amplitude of the segment in dB"
)
peak_amplitude_db: float = Field(
default=0.0,
description="Peak amplitude of the segment in dB"
)
@property
def is_loud(self) -> bool:
"""Check if this segment is relatively loud (> -20dB average)."""
return self.avg_amplitude_db > -20.0
class MuteRegion(BaseModel):
"""A region to mute in the backing vocals audio."""
start_seconds: float = Field(
...,
ge=0,
description="Start time of the region to mute in seconds"
)
end_seconds: float = Field(
...,
ge=0,
description="End time of the region to mute in seconds"
)
@property
def duration_seconds(self) -> float:
"""Duration of the mute region in seconds."""
return self.end_seconds - self.start_seconds
def overlaps(self, other: "MuteRegion") -> bool:
"""Check if this region overlaps with another."""
return not (self.end_seconds <= other.start_seconds or
self.start_seconds >= other.end_seconds)
def merge(self, other: "MuteRegion") -> "MuteRegion":
"""Merge this region with another overlapping region."""
return MuteRegion(
start_seconds=min(self.start_seconds, other.start_seconds),
end_seconds=max(self.end_seconds, other.end_seconds)
)
class AnalysisResult(BaseModel):
"""Result of analyzing backing vocals audio for audible content."""
has_audible_content: bool = Field(
...,
description="Whether any audible content was detected"
)
total_duration_seconds: float = Field(
...,
ge=0,
description="Total duration of the audio file in seconds"
)
audible_segments: List[AudibleSegment] = Field(
default_factory=list,
description="List of detected audible segments"
)
recommended_selection: RecommendedSelection = Field(
...,
description="Recommended instrumental selection based on analysis"
)
silence_threshold_db: float = Field(
default=-40.0,
description="Threshold used to detect silence (in dB)"
)
total_audible_duration_seconds: float = Field(
default=0.0,
ge=0,
description="Total duration of all audible segments combined"
)
audible_percentage: float = Field(
default=0.0,
ge=0,
le=100,
description="Percentage of audio that is audible"
)
waveform_data: Optional[List[float]] = Field(
default=None,
description="Amplitude envelope data for waveform rendering (optional)"
)
@property
def segment_count(self) -> int:
"""Number of audible segments detected."""
return len(self.audible_segments)
def get_segments_in_range(
self,
start_seconds: float,
end_seconds: float
) -> List[AudibleSegment]:
"""Get segments that overlap with the given time range."""
return [
seg for seg in self.audible_segments
if not (seg.end_seconds <= start_seconds or
seg.start_seconds >= end_seconds)
]
class CustomInstrumentalResult(BaseModel):
"""Result of creating a custom instrumental with muted regions."""
output_path: str = Field(
...,
description="Path to the created custom instrumental file"
)
mute_regions_applied: List[MuteRegion] = Field(
default_factory=list,
description="List of mute regions that were applied"
)
total_muted_duration_seconds: float = Field(
default=0.0,
ge=0,
description="Total duration of muted audio in seconds"
)
output_duration_seconds: float = Field(
default=0.0,
ge=0,
description="Duration of the output file in seconds"
)
+423
View File
@@ -0,0 +1,423 @@
"""
Local FastAPI server for instrumental review.
This module provides a local HTTP server that serves the instrumental review
UI for local CLI usage. It provides the same API endpoints as the cloud backend
to enable UI reuse.
Similar pattern to LyricsTranscriber's ReviewServer.
"""
import logging
import os
from pathlib import Path
import threading
import webbrowser
from typing import List, Optional
from fastapi import FastAPI, HTTPException, UploadFile, File
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, HTMLResponse
from pydantic import BaseModel
import shutil
import tempfile
import uvicorn
from pydub import AudioSegment
from karaoke_gen.instrumental_review import (
AnalysisResult,
AudioAnalyzer,
AudioEditor,
MuteRegion,
WaveformGenerator,
)
logger = logging.getLogger(__name__)
# Request/Response Models
class MuteRegionRequest(BaseModel):
start_seconds: float
end_seconds: float
class CreateCustomRequest(BaseModel):
mute_regions: List[MuteRegionRequest]
class SelectionRequest(BaseModel):
selection: str
class InstrumentalReviewServer:
"""
Local FastAPI server for instrumental review UI.
This server provides a web interface for reviewing and selecting
instrumental tracks in the local CLI workflow. It serves the same
API endpoints as the cloud backend to enable UI reuse.
"""
def __init__(
self,
output_dir: str,
base_name: str,
analysis: AnalysisResult,
waveform_path: str,
backing_vocals_path: str,
clean_instrumental_path: str,
with_backing_path: Optional[str] = None,
original_audio_path: Optional[str] = None,
):
"""
Initialize the review server.
Args:
output_dir: Directory containing the audio files
base_name: Base name for output files (e.g., "Artist - Title")
analysis: Analysis result from AudioAnalyzer
waveform_path: Path to the waveform image
backing_vocals_path: Path to the backing vocals audio file
clean_instrumental_path: Path to the clean instrumental audio file
with_backing_path: Path to the instrumental with backing vocals
original_audio_path: Path to the original audio file (with vocals)
"""
self.output_dir = output_dir
self.base_name = base_name
self.analysis = analysis
self.waveform_path = waveform_path
self.backing_vocals_path = backing_vocals_path
self.clean_instrumental_path = clean_instrumental_path
self.with_backing_path = with_backing_path
self.original_audio_path = original_audio_path
self.custom_instrumental_path: Optional[str] = None
self.uploaded_instrumental_path: Optional[str] = None
self.selection: Optional[str] = None
self._app: Optional[FastAPI] = None
self._server_thread: Optional[threading.Thread] = None
self._selection_event = threading.Event()
self._shutdown_event = threading.Event()
def _create_app(self) -> FastAPI:
"""Create and configure the FastAPI application."""
app = FastAPI(title="Instrumental Review", docs_url=None, redoc_url=None)
# Configure CORS
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Register routes
self._register_routes(app)
return app
def _register_routes(self, app: FastAPI) -> None:
"""Register API routes."""
@app.get("/")
async def serve_frontend():
"""Serve the frontend HTML."""
return HTMLResponse(content=self._get_frontend_html())
@app.get("/api/jobs/local/instrumental-analysis")
async def get_analysis():
"""Get analysis data for the instrumental review."""
return {
"job_id": "local",
"artist": self.base_name.split(" - ")[0] if " - " in self.base_name else "",
"title": self.base_name.split(" - ")[1] if " - " in self.base_name else self.base_name,
"status": "awaiting_instrumental_selection",
"analysis": {
"has_audible_content": self.analysis.has_audible_content,
"total_duration_seconds": self.analysis.total_duration_seconds,
"audible_segments": [
{
"start_seconds": seg.start_seconds,
"end_seconds": seg.end_seconds,
"duration_seconds": seg.duration_seconds,
"avg_amplitude_db": seg.avg_amplitude_db,
"peak_amplitude_db": seg.peak_amplitude_db,
}
for seg in self.analysis.audible_segments
],
"recommended_selection": self.analysis.recommended_selection.value,
"total_audible_duration_seconds": self.analysis.total_audible_duration_seconds,
"audible_percentage": self.analysis.audible_percentage,
"silence_threshold_db": self.analysis.silence_threshold_db,
},
"audio_urls": {
"clean_instrumental": "/api/audio/clean_instrumental" if self.clean_instrumental_path else None,
"backing_vocals": "/api/audio/backing_vocals" if self.backing_vocals_path else None,
"with_backing": "/api/audio/with_backing" if self.with_backing_path else None,
"custom_instrumental": "/api/audio/custom_instrumental" if self.custom_instrumental_path else None,
"uploaded_instrumental": "/api/audio/uploaded_instrumental" if self.uploaded_instrumental_path else None,
"original": "/api/audio/original" if self.original_audio_path else None,
},
"waveform_url": "/api/waveform" if self.waveform_path else None,
"has_custom_instrumental": self.custom_instrumental_path is not None,
"has_uploaded_instrumental": self.uploaded_instrumental_path is not None,
"has_original": self.original_audio_path is not None,
}
@app.get("/api/jobs/local/waveform-data")
async def get_waveform_data(num_points: int = 600):
"""Get waveform amplitude data for client-side rendering."""
# Validate num_points parameter
if num_points <= 0 or num_points > 10000:
raise HTTPException(
status_code=400,
detail="num_points must be between 1 and 10000"
)
if not self.backing_vocals_path or not os.path.exists(self.backing_vocals_path):
raise HTTPException(status_code=404, detail="Backing vocals file not found")
try:
generator = WaveformGenerator()
amplitudes, duration = generator.generate_data_only(self.backing_vocals_path, num_points)
return {"amplitudes": amplitudes, "duration": duration}
except Exception as e:
logger.exception(f"Error generating waveform data: {e}")
raise HTTPException(status_code=500, detail=str(e)) from e
@app.get("/api/audio/{stem_type}")
async def stream_audio(stem_type: str):
"""Stream audio file."""
path_map = {
"clean_instrumental": self.clean_instrumental_path,
"backing_vocals": self.backing_vocals_path,
"with_backing": self.with_backing_path,
"custom_instrumental": self.custom_instrumental_path,
"uploaded_instrumental": self.uploaded_instrumental_path,
"original": self.original_audio_path,
}
audio_path = path_map.get(stem_type)
if not audio_path or not os.path.exists(audio_path):
raise HTTPException(status_code=404, detail=f"Audio file not found: {stem_type}")
# Determine content type
ext = os.path.splitext(audio_path)[1].lower()
content_types = {
".flac": "audio/flac",
".mp3": "audio/mpeg",
".wav": "audio/wav",
}
content_type = content_types.get(ext, "application/octet-stream")
return FileResponse(audio_path, media_type=content_type)
@app.get("/api/waveform")
async def get_waveform_image():
"""Serve waveform image."""
if not self.waveform_path or not os.path.exists(self.waveform_path):
raise HTTPException(status_code=404, detail="Waveform image not found")
return FileResponse(self.waveform_path, media_type="image/png")
@app.post("/api/jobs/local/create-custom-instrumental")
async def create_custom_instrumental(request: CreateCustomRequest):
"""Create a custom instrumental with muted regions."""
if not request.mute_regions:
raise HTTPException(status_code=400, detail="No mute regions provided")
try:
mute_regions = [
MuteRegion(
start_seconds=r.start_seconds,
end_seconds=r.end_seconds,
)
for r in request.mute_regions
]
editor = AudioEditor()
output_path = os.path.join(
self.output_dir,
f"{self.base_name} (Instrumental Custom).flac"
)
result = editor.create_custom_instrumental(
clean_instrumental_path=self.clean_instrumental_path,
backing_vocals_path=self.backing_vocals_path,
mute_regions=mute_regions,
output_path=output_path,
)
self.custom_instrumental_path = result.output_path
return {
"status": "success",
"custom_instrumental_url": "/api/audio/custom_instrumental",
"statistics": {
"mute_regions_applied": len(result.mute_regions_applied),
"total_muted_duration_seconds": result.total_muted_duration_seconds,
"output_duration_seconds": result.output_duration_seconds,
},
}
except Exception as e:
logger.exception(f"Error creating custom instrumental: {e}")
raise HTTPException(status_code=500, detail=str(e)) from e
@app.post("/api/jobs/local/upload-instrumental")
async def upload_instrumental(file: UploadFile = File(...)):
"""Upload a custom instrumental audio file."""
# Validate file type
allowed_extensions = {".flac", ".mp3", ".wav", ".m4a", ".ogg"}
ext = os.path.splitext(file.filename or "")[1].lower()
if ext not in allowed_extensions:
raise HTTPException(
status_code=400,
detail=f"Invalid file type. Allowed: {', '.join(allowed_extensions)}"
)
tmp_path = None
file_moved = False
try:
# Save to temp file first to validate
with tempfile.NamedTemporaryFile(delete=False, suffix=ext) as tmp:
shutil.copyfileobj(file.file, tmp)
tmp_path = tmp.name
# Load and check duration
uploaded_audio = AudioSegment.from_file(tmp_path)
uploaded_duration = len(uploaded_audio) / 1000.0 # ms to seconds
expected_duration = self.analysis.total_duration_seconds
duration_diff = abs(uploaded_duration - expected_duration)
if duration_diff > 0.5:
raise HTTPException(
status_code=400,
detail=f"Duration mismatch: uploaded file is {uploaded_duration:.2f}s, "
f"expected {expected_duration:.2f}s (±0.5s allowed)"
)
# Move to final location
output_path = os.path.join(
self.output_dir,
f"{self.base_name} (Instrumental Uploaded){ext}"
)
shutil.move(tmp_path, output_path)
file_moved = True
self.uploaded_instrumental_path = output_path
return {
"status": "success",
"uploaded_instrumental_url": "/api/audio/uploaded_instrumental",
"duration_seconds": uploaded_duration,
"filename": file.filename,
}
except HTTPException:
raise
except Exception as e:
logger.exception(f"Error uploading instrumental: {e}")
raise HTTPException(status_code=500, detail=str(e)) from e
finally:
# Clean up temp file if it wasn't moved
if tmp_path and not file_moved and os.path.exists(tmp_path):
try:
os.unlink(tmp_path)
except OSError:
pass # Best effort cleanup
@app.post("/api/jobs/local/select-instrumental")
async def select_instrumental(request: SelectionRequest):
"""Submit instrumental selection."""
if request.selection not in ("clean", "with_backing", "custom", "uploaded", "original"):
raise HTTPException(status_code=400, detail=f"Invalid selection: {request.selection}")
self.selection = request.selection
self._selection_event.set()
return {"status": "success", "selection": request.selection}
@staticmethod
def _get_static_dir() -> Path:
"""Get the path to the static assets directory."""
return Path(__file__).parent / "static"
def _get_frontend_html(self) -> str:
"""Return the frontend HTML by reading from the static file."""
static_file = self._get_static_dir() / "index.html"
if static_file.exists():
return static_file.read_text(encoding="utf-8")
else:
# Fallback error message if file is missing
return """<!DOCTYPE html>
<html>
<head><title>Error</title></head>
<body style="background:#1a1a1a;color:#fff;font-family:sans-serif;padding:2rem;">
<h1>Frontend assets not found</h1>
<p>The static/index.html file is missing from the instrumental_review module.</p>
</body>
</html>"""
def start_and_open_browser(self, port: int = 8765) -> str:
"""
Start server, open browser, and block until selection is submitted.
Args:
port: Port to run the server on
Returns:
The user's selection ("clean", "with_backing", or "custom")
"""
self._app = self._create_app()
# Run uvicorn in a separate thread
config = uvicorn.Config(
self._app,
host="127.0.0.1",
port=port,
log_level="warning",
)
server = uvicorn.Server(config)
def run_server():
server.run()
self._server_thread = threading.Thread(target=run_server, daemon=True)
self._server_thread.start()
# Wait a moment for server to start
import time
time.sleep(0.5)
url = f"http://localhost:{port}/"
logger.info(f"Instrumental review server started at {url}")
# Open browser
webbrowser.open(url)
# Wait for selection
logger.info("Waiting for instrumental selection...")
self._selection_event.wait()
# Give a moment for response to be sent
time.sleep(0.5)
return self.get_selection()
def stop(self) -> None:
"""Stop the server."""
self._shutdown_event.set()
def get_selection(self) -> str:
"""Get the user's selection."""
if self.selection is None:
raise ValueError("No selection has been made")
return self.selection
def get_custom_instrumental_path(self) -> Optional[str]:
"""Get the path to the custom instrumental if one was created."""
return self.custom_instrumental_path
def get_uploaded_instrumental_path(self) -> Optional[str]:
"""Get the path to the uploaded instrumental if one was uploaded."""
return self.uploaded_instrumental_path
File diff suppressed because it is too large. Load diff
+409
View File
@@ -0,0 +1,409 @@
"""
Waveform visualization generator for audio files.
This module provides the WaveformGenerator class which creates waveform
images suitable for display in the instrumental review UI.
"""
import logging
import math
from pathlib import Path
from typing import List, Optional, Tuple
import matplotlib
matplotlib.use('Agg') # Use non-interactive backend
import matplotlib.pyplot as plt
import numpy as np
from pydub import AudioSegment
from .models import AudibleSegment, MuteRegion
logger = logging.getLogger(__name__)
class WaveformGenerator:
"""
Generates waveform visualization images from audio files.
This class creates PNG images showing the amplitude envelope of an
audio file over time. It can highlight detected audible segments
and mute regions with different colors.
The generated images are suitable for display in web UIs and can
be used for interactive seeking (click-to-seek) functionality.
Attributes:
width: Width of the output image in pixels (default: 1200)
height: Height of the output image in pixels (default: 200)
background_color: Background color (default: "#1a1a2e")
waveform_color: Main waveform color (default: "#4a90d9")
segment_color: Color for audible segments (default: "#e94560")
mute_color: Color for mute regions (default: "#ff6b6b")
time_axis_color: Color for time axis (default: "#ffffff")
Example:
>>> generator = WaveformGenerator(width=1200, height=200)
>>> generator.generate(
... audio_path="/path/to/backing_vocals.flac",
... output_path="/path/to/waveform.png",
... segments=analysis_result.audible_segments
... )
"""
def __init__(
self,
width: int = 1200,
height: int = 200,
background_color: str = "#1a1a2e",
waveform_color: str = "#4a90d9",
segment_color: str = "#e94560",
mute_color: str = "#ff6b6b",
time_axis_color: str = "#ffffff",
dpi: int = 100,
):
"""
Initialize the waveform generator.
Args:
width: Width of the output image in pixels
height: Height of the output image in pixels
background_color: Background color (hex or named color)
waveform_color: Main waveform color
segment_color: Color for highlighting audible segments
mute_color: Color for highlighting mute regions
time_axis_color: Color for time axis labels
dpi: DPI for the output image
"""
self.width = width
self.height = height
self.background_color = background_color
self.waveform_color = waveform_color
self.segment_color = segment_color
self.mute_color = mute_color
self.time_axis_color = time_axis_color
self.dpi = dpi
def generate(
self,
audio_path: str,
output_path: str,
segments: Optional[List[AudibleSegment]] = None,
mute_regions: Optional[List[MuteRegion]] = None,
show_time_axis: bool = True,
silence_threshold_db: float = -40.0,
) -> str:
"""
Generate a waveform image from an audio file.
Args:
audio_path: Path to the audio file
output_path: Path where the PNG image will be saved
segments: Optional list of audible segments to highlight
mute_regions: Optional list of mute regions to highlight
show_time_axis: Whether to show time axis labels
silence_threshold_db: Threshold for visual reference line
Returns:
Path to the generated image file
Raises:
FileNotFoundError: If the audio file doesn't exist
"""
path = Path(audio_path)
if not path.exists():
raise FileNotFoundError(f"Audio file not found: {audio_path}")
logger.info(f"Generating waveform for: {audio_path}")
# Load audio
audio = AudioSegment.from_file(audio_path)
duration_seconds = len(audio) / 1000.0
# Convert to mono if needed
if audio.channels > 1:
audio = audio.set_channels(1)
# Get amplitude envelope
envelope = self._get_envelope(audio)
# Create the figure
fig, ax = self._create_figure(duration_seconds, show_time_axis)
# Draw waveform
self._draw_waveform(ax, envelope, duration_seconds)
# Highlight mute regions (if any) - draw first so waveform is on top
if mute_regions:
self._draw_mute_regions(ax, mute_regions, duration_seconds)
# Highlight audible segments (if any)
if segments:
self._draw_segments(ax, segments, envelope, duration_seconds)
# Draw silence threshold reference line
self._draw_threshold_line(ax, silence_threshold_db, duration_seconds)
# Save the figure
output_dir = Path(output_path).parent
output_dir.mkdir(parents=True, exist_ok=True)
fig.savefig(
output_path,
facecolor=self.background_color,
edgecolor='none',
bbox_inches='tight',
pad_inches=0.1,
)
plt.close(fig)
logger.info(f"Waveform saved to: {output_path}")
return output_path
def generate_data_only(
self,
audio_path: str,
num_points: int = 500,
) -> Tuple[List[float], float]:
"""
Generate waveform data without creating an image.
This is useful for sending data to a frontend that will
render the waveform itself (e.g., using Canvas or SVG).
Args:
audio_path: Path to the audio file
num_points: Number of data points to return
Returns:
Tuple of (amplitude_values, duration_seconds)
Amplitude values are normalized to 0.0-1.0 range.
"""
path = Path(audio_path)
if not path.exists():
raise FileNotFoundError(f"Audio file not found: {audio_path}")
audio = AudioSegment.from_file(audio_path)
duration_seconds = len(audio) / 1000.0
if audio.channels > 1:
audio = audio.set_channels(1)
# Calculate window size to get desired number of points
duration_ms = len(audio)
window_ms = max(1, duration_ms // num_points)
amplitudes = []
for start_ms in range(0, duration_ms, window_ms):
end_ms = min(start_ms + window_ms, duration_ms)
window = audio[start_ms:end_ms]
if window.rms > 0:
db = 20 * math.log10(window.rms / window.max_possible_amplitude)
else:
db = -100.0
# Normalize to 0-1 range (mapping -60dB to 0dB -> 0 to 1)
normalized = max(0.0, min(1.0, (db + 60) / 60))
amplitudes.append(normalized)
return amplitudes, duration_seconds
def _get_envelope(
self,
audio: AudioSegment,
window_ms: int = 50,
) -> np.ndarray:
"""
Extract amplitude envelope from audio.
Returns array of amplitude values in dB.
"""
duration_ms = len(audio)
amplitudes = []
for start_ms in range(0, duration_ms, window_ms):
end_ms = min(start_ms + window_ms, duration_ms)
window = audio[start_ms:end_ms]
if window.rms > 0:
db = 20 * math.log10(window.rms / window.max_possible_amplitude)
else:
db = -100.0
amplitudes.append(db)
return np.array(amplitudes)
def _create_figure(
self,
duration_seconds: float,
show_time_axis: bool,
) -> Tuple[plt.Figure, plt.Axes]:
"""
Create matplotlib figure and axes.
"""
fig_width = self.width / self.dpi
fig_height = self.height / self.dpi
fig, ax = plt.subplots(figsize=(fig_width, fig_height), dpi=self.dpi)
# Set background
fig.patch.set_facecolor(self.background_color)
ax.set_facecolor(self.background_color)
# Configure axes
ax.set_xlim(0, duration_seconds)
ax.set_ylim(-60, 0) # dB range
# Remove spines
for spine in ax.spines.values():
spine.set_visible(False)
# Configure ticks
if show_time_axis:
ax.tick_params(
axis='x',
colors=self.time_axis_color,
labelsize=8,
)
ax.tick_params(axis='y', left=False, labelleft=False)
# Set time axis ticks
self._set_time_ticks(ax, duration_seconds)
else:
ax.tick_params(
axis='both',
left=False,
bottom=False,
labelleft=False,
labelbottom=False,
)
return fig, ax
def _set_time_ticks(self, ax: plt.Axes, duration_seconds: float):
"""
Set appropriate time axis tick marks.
"""
if duration_seconds <= 60:
# Under 1 minute: tick every 10 seconds
tick_interval = 10
elif duration_seconds <= 300:
# Under 5 minutes: tick every 30 seconds
tick_interval = 30
else:
# Over 5 minutes: tick every minute
tick_interval = 60
ticks = np.arange(0, duration_seconds + 1, tick_interval)
ax.set_xticks(ticks)
# Format tick labels as MM:SS
labels = []
for t in ticks:
minutes = int(t // 60)
seconds = int(t % 60)
labels.append(f"{minutes}:{seconds:02d}")
ax.set_xticklabels(labels)
def _draw_waveform(
self,
ax: plt.Axes,
envelope: np.ndarray,
duration_seconds: float,
):
"""
Draw the main waveform.
"""
num_points = len(envelope)
time_points = np.linspace(0, duration_seconds, num_points)
# Draw as filled area
ax.fill_between(
time_points,
envelope,
-60, # Bottom of range
color=self.waveform_color,
alpha=0.7,
)
# Draw outline
ax.plot(
time_points,
envelope,
color=self.waveform_color,
linewidth=0.5,
alpha=0.9,
)
def _draw_segments(
self,
ax: plt.Axes,
segments: List[AudibleSegment],
envelope: np.ndarray,
duration_seconds: float,
):
"""
Highlight audible segments on the waveform.
"""
num_points = len(envelope)
time_points = np.linspace(0, duration_seconds, num_points)
for segment in segments:
# Find indices corresponding to this segment
start_idx = int(segment.start_seconds / duration_seconds * num_points)
end_idx = int(segment.end_seconds / duration_seconds * num_points)
start_idx = max(0, min(start_idx, num_points - 1))
end_idx = max(0, min(end_idx, num_points))
if start_idx >= end_idx:
continue
segment_time = time_points[start_idx:end_idx]
segment_envelope = envelope[start_idx:end_idx]
# Highlight this segment with a different color
ax.fill_between(
segment_time,
segment_envelope,
-60,
color=self.segment_color,
alpha=0.6,
)
def _draw_mute_regions(
self,
ax: plt.Axes,
mute_regions: List[MuteRegion],
duration_seconds: float,
):
"""
Draw mute region overlays.
"""
for region in mute_regions:
ax.axvspan(
region.start_seconds,
region.end_seconds,
color=self.mute_color,
alpha=0.3,
zorder=0,
)
def _draw_threshold_line(
self,
ax: plt.Axes,
threshold_db: float,
duration_seconds: float,
):
"""
Draw a reference line at the silence threshold.
"""
ax.axhline(
y=threshold_db,
color=self.time_axis_color,
linestyle='--',
linewidth=0.5,
alpha=0.3,
)
@@ -47,6 +47,7 @@ class KaraokeFinalise:
user_youtube_credentials=None, # Add support for pre-stored credentials
server_side_mode=False, # New parameter for server-side deployment
selected_instrumental_file=None, # Add support for pre-selected instrumental file
countdown_padding_seconds=None, # Padding applied to vocals; instrumental must match
):
self.log_level = log_level
self.log_formatter = log_formatter
@@ -54,6 +55,9 @@ class KaraokeFinalise:
if logger is None:
self.logger = logging.getLogger(__name__)
self.logger.setLevel(log_level)
# Prevent log propagation to root logger to avoid duplicate logs
# when external packages (like lyrics_converter) configure root logger handlers
self.logger.propagate = False
self.log_handler = logging.StreamHandler()
@@ -105,6 +109,7 @@ class KaraokeFinalise:
self.user_youtube_credentials = user_youtube_credentials
self.server_side_mode = server_side_mode
self.selected_instrumental_file = selected_instrumental_file
self.countdown_padding_seconds = countdown_padding_seconds
self.suffixes = {
"title_mov": " (Title).mov",
@@ -268,6 +273,8 @@ class KaraokeFinalise:
# Enable discord notifications if webhook URL is provided and is valid URL
if self.discord_webhook_url is not None:
# Strip whitespace/newlines that may have been introduced from environment variables or secrets
self.discord_webhook_url = self.discord_webhook_url.strip()
if not self.discord_webhook_url.startswith("https://discord.com/api/webhooks/"):
raise Exception(f"Discord webhook URL is not valid: {self.discord_webhook_url}")
@@ -419,6 +426,15 @@ class KaraokeFinalise:
# Check if any videos were found
if "items" in response and len(response["items"]) > 0:
for item in response["items"]:
# YouTube search API sometimes returns results from other channels even with channelId filter
# Verify the video actually belongs to our channel
result_channel_id = item["snippet"]["channelId"]
if result_channel_id != channel_id:
self.logger.debug(
f"Skipping video from different channel: {item['snippet']['title']} (channel: {result_channel_id})"
)
continue
found_title = item["snippet"]["title"]
# In server-side mode, require an exact match to avoid false positives.
@@ -718,6 +734,32 @@ class KaraokeFinalise:
artist, title = base_name.split(" - ", 1)
return base_name, artist, title
def _pad_audio_file(self, input_audio, output_audio, padding_seconds):
"""
Pad an audio file by prepending silence at the beginning.
Uses the same ffmpeg approach as LyricsTranscriber's CountdownProcessor
to ensure consistent padding behavior.
Args:
input_audio: Path to input audio file
output_audio: Path for the padded output file
padding_seconds: Amount of silence to prepend (in seconds)
"""
self.logger.info(f"Padding audio file with {padding_seconds}s of silence")
# Use ffmpeg to prepend silence - this matches the approach in audio_processor.py
# adelay filter adds delay in milliseconds
delay_ms = int(padding_seconds * 1000)
ffmpeg_command = (
f'{self.ffmpeg_base_command} -i "{input_audio}" '
f'-af "adelay={delay_ms}|{delay_ms}" '
f'"{output_audio}"'
)
self.execute_command(ffmpeg_command, f"Padding audio with {padding_seconds}s silence")
def execute_command(self, command, description):
"""Execute a shell command and log the output. For general commands (rclone, etc.)"""
self.logger.info(f"{description}")
@@ -762,11 +804,32 @@ class KaraokeFinalise:
def remux_with_instrumental(self, with_vocals_file, instrumental_audio, output_file):
"""Remux the video with instrumental audio to create karaoke version"""
# Safety net: If countdown padding was applied to vocals, ensure instrumental is padded too
actual_instrumental = instrumental_audio
if self.countdown_padding_seconds and self.countdown_padding_seconds > 0:
# Check if the instrumental file is already padded (has "(Padded)" in name)
if "(Padded)" not in instrumental_audio:
self.logger.warning(
f"Countdown padding ({self.countdown_padding_seconds}s) was applied to vocals, "
f"but instrumental doesn't appear to be padded. Creating padded version..."
)
# Create a padded version of the instrumental
base, ext = os.path.splitext(instrumental_audio)
padded_instrumental = f"{base} (Padded){ext}"
if not os.path.exists(padded_instrumental):
self._pad_audio_file(instrumental_audio, padded_instrumental, self.countdown_padding_seconds)
self.logger.info(f"Created padded instrumental: {padded_instrumental}")
actual_instrumental = padded_instrumental
else:
self.logger.info(f"Using already-padded instrumental: {instrumental_audio}")
# This operation is primarily I/O bound (remuxing), so hardware acceleration doesn't provide significant benefit
# Keep the existing approach but use the new execute method
ffmpeg_command = (
f'{self.ffmpeg_base_command} -an -i "{with_vocals_file}" '
f'-vn -i "{instrumental_audio}" -c:v copy -c:a pcm_s16le "{output_file}"'
f'-vn -i "{actual_instrumental}" -c:v copy -c:a pcm_s16le "{output_file}"'
)
self.execute_command(ffmpeg_command, "Remuxing video with instrumental audio")
@@ -858,7 +921,7 @@ class KaraokeFinalise:
return env_mov_input, ffmpeg_filter
def remux_and_encode_output_video_files(self, with_vocals_file, input_files, output_files):
self.logger.info(f"Remuxing and encoding output video files...")
self.logger.info(f"Remuxing and encoding output video files (4 formats, ~15-20 minutes total)...")
# Check if output files already exist
if os.path.isfile(output_files["final_karaoke_lossless_mp4"]) and os.path.isfile(output_files["final_karaoke_lossless_mkv"]):
@@ -869,16 +932,20 @@ class KaraokeFinalise:
return
# Create karaoke version with instrumental audio
self.logger.info(f"[Step 1/6] Remuxing video with instrumental audio...")
self.remux_with_instrumental(with_vocals_file, input_files["instrumental_audio"], output_files["karaoke_mp4"])
# Convert the with vocals video to MP4 if needed
if not with_vocals_file.endswith(".mp4"):
self.logger.info(f"[Step 2/6] Converting karaoke video to MP4...")
self.convert_mov_to_mp4(with_vocals_file, output_files["with_vocals_mp4"])
# Delete the with vocals mov after successfully converting it to mp4
if not self.dry_run and os.path.isfile(with_vocals_file):
self.logger.info(f"Deleting with vocals MOV file: {with_vocals_file}")
os.remove(with_vocals_file)
else:
self.logger.info(f"[Step 2/6] Skipped - video already in MP4 format")
# Quote file paths to handle special characters
title_mov_file = shlex.quote(os.path.abspath(input_files["title_mov"]))
@@ -887,10 +954,17 @@ class KaraokeFinalise:
# Prepare concat filter for combining videos
env_mov_input, ffmpeg_filter = self.prepare_concat_filter(input_files)
# Create all output versions
# Create all output versions with progress logging
self.logger.info(f"[Step 3/6] Encoding lossless 4K MP4 (title + karaoke + end, ~5 minutes)...")
self.encode_lossless_mp4(title_mov_file, karaoke_mp4_file, env_mov_input, ffmpeg_filter, output_files["final_karaoke_lossless_mp4"])
self.logger.info(f"[Step 4/6] Encoding lossy 4K MP4 with AAC audio (~1 minute)...")
self.encode_lossy_mp4(output_files["final_karaoke_lossless_mp4"], output_files["final_karaoke_lossy_mp4"])
self.logger.info(f"[Step 5/6] Creating MKV with FLAC audio for YouTube (~1 minute)...")
self.encode_lossless_mkv(output_files["final_karaoke_lossless_mp4"], output_files["final_karaoke_lossless_mkv"])
self.logger.info(f"[Step 6/6] Encoding 720p version (~3 minutes)...")
self.encode_720p_version(output_files["final_karaoke_lossless_mp4"], output_files["final_karaoke_lossy_720p_mp4"])
# Skip user confirmation in non-interactive mode for Modal deployment
@@ -1298,8 +1372,13 @@ class KaraokeFinalise:
self.logger.error("YouTube upload failed in non-interactive mode, skipping")
# Discord notification - runs independently of YouTube upload
# Wrapped in try/except so failures don't crash the entire job
if self.discord_notication_enabled:
self.post_discord_notification()
try:
self.post_discord_notification()
except Exception as e:
self.logger.error(f"Failed to send Discord notification: {e}")
self.logger.warning("Continuing without Discord notification - this is non-fatal")
# Handle folder organization - different logic for server-side vs local mode
if self.server_side_mode and self.brand_prefix and self.organised_dir_rclone_root:
@@ -1661,11 +1740,17 @@ class KaraokeFinalise:
if self.dry_run:
self.logger.warning("Dry run enabled. No actions will be performed.")
self.logger.info("=" * 60)
self.logger.info("Starting KaraokeFinalise processing pipeline")
self.logger.info("=" * 60)
# Check required input files and parameters exist, get user to confirm features before proceeding
self.validate_input_parameters_for_features()
with_vocals_file = self.find_with_vocals_file()
base_name, artist, title = self.get_names_from_withvocals(with_vocals_file)
self.logger.info(f"Processing: {artist} - {title}")
# Use the selected instrumental file if provided, otherwise search for one
if self.selected_instrumental_file:
@@ -1681,13 +1766,20 @@ class KaraokeFinalise:
output_files = self.prepare_output_filenames(base_name)
if self.enable_cdg:
self.logger.info("Creating CDG package...")
self.create_cdg_zip_file(input_files, output_files, artist, title)
self.logger.info("CDG package created successfully")
if self.enable_txt:
self.logger.info("Creating TXT package...")
self.create_txt_zip_file(input_files, output_files)
self.logger.info("TXT package created successfully")
self.logger.info("Starting video encoding (this is the longest step, ~15-20 minutes)...")
self.remux_and_encode_output_video_files(with_vocals_file, input_files, output_files)
self.logger.info("Video encoding completed successfully")
self.logger.info("Executing distribution features (YouTube, Dropbox, Discord)...")
self.execute_optional_features(artist, title, base_name, input_files, output_files, replace_existing)
result = {
+114 -1
View File
@@ -29,7 +29,7 @@ from .audio_processor import AudioProcessor
from .lyrics_processor import LyricsProcessor
from .video_generator import VideoGenerator
from .video_background_processor import VideoBackgroundProcessor
from .audio_fetcher import create_audio_fetcher, AudioFetcherError, NoResultsError
from .audio_fetcher import create_audio_fetcher, AudioFetcherError, NoResultsError, UserCancelledError
class KaraokePrep:
@@ -84,6 +84,9 @@ class KaraokePrep:
if logger is None:
self.logger = logging.getLogger(__name__)
self.logger.setLevel(log_level)
# Prevent log propagation to root logger to avoid duplicate logs
# when external packages (like lyrics_converter) configure root logger handlers
self.logger.propagate = False
self.log_handler = logging.StreamHandler()
@@ -256,6 +259,101 @@ class KaraokePrep:
self.artist = metadata_result["artist"]
self.title = metadata_result["title"]
def _scan_directory_for_instrumentals(self, track_output_dir, artist_title):
"""
Scan the directory for existing instrumental files and build a separated_audio structure.
This is used when transcription was skipped (existing files found) but we need to
pad instrumentals due to countdown padding.
Args:
track_output_dir: The track output directory to scan
artist_title: The "{artist} - {title}" string for matching files
Returns:
Dictionary with separated_audio structure containing found instrumental paths
"""
self.logger.info(f"Scanning directory for existing instrumentals: {track_output_dir}")
separated_audio = {
"clean_instrumental": {},
"backing_vocals": {},
"other_stems": {},
"combined_instrumentals": {},
}
# Search patterns for instrumental files
# Files are named like: "{artist} - {title} (Instrumental {model}).flac"
# Or with backing vocals: "{artist} - {title} (Instrumental +BV {model}).flac"
# Look for files in the track output directory
search_dir = track_output_dir
# Find all instrumental files (not padded ones - we want the originals)
instrumental_pattern = os.path.join(search_dir, f"{artist_title} (Instrumental*.flac")
instrumental_files = glob.glob(instrumental_pattern)
# Also check for wav files
instrumental_pattern_wav = os.path.join(search_dir, f"{artist_title} (Instrumental*.wav")
instrumental_files.extend(glob.glob(instrumental_pattern_wav))
self.logger.debug(f"Found {len(instrumental_files)} instrumental files")
for filepath in instrumental_files:
filename = os.path.basename(filepath)
# Skip already padded files
if "(Padded)" in filename:
self.logger.debug(f"Skipping already padded file: {filename}")
continue
# Determine if it's a combined instrumental (+BV) or clean instrumental
if "+BV" in filename or "+bv" in filename.lower():
# Combined instrumental with backing vocals
# Extract model name from filename
# Pattern: "(Instrumental +BV {model}).flac"
model_match = re.search(r'\(Instrumental \+BV ([^)]+)\)', filename)
if model_match:
model_name = model_match.group(1).strip()
separated_audio["combined_instrumentals"][model_name] = filepath
self.logger.info(f"Found combined instrumental: {filename}")
else:
# Clean instrumental (no backing vocals)
# Pattern: "(Instrumental {model}).flac"
model_match = re.search(r'\(Instrumental ([^)]+)\)', filename)
if model_match:
# Use as clean instrumental if we don't have one yet
if not separated_audio["clean_instrumental"].get("instrumental"):
separated_audio["clean_instrumental"]["instrumental"] = filepath
self.logger.info(f"Found clean instrumental: {filename}")
else:
# Additional clean instrumentals go to combined_instrumentals for padding
model_name = model_match.group(1).strip()
separated_audio["combined_instrumentals"][model_name] = filepath
self.logger.info(f"Found additional instrumental: {filename}")
# Also look for backing vocals files
backing_vocals_pattern = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.flac")
backing_vocals_files = glob.glob(backing_vocals_pattern)
backing_vocals_pattern_wav = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.wav")
backing_vocals_files.extend(glob.glob(backing_vocals_pattern_wav))
for filepath in backing_vocals_files:
filename = os.path.basename(filepath)
model_match = re.search(r'\(Backing Vocals ([^)]+)\)', filename)
if model_match:
model_name = model_match.group(1).strip()
if model_name not in separated_audio["backing_vocals"]:
separated_audio["backing_vocals"][model_name] = {"backing_vocals": filepath}
self.logger.info(f"Found backing vocals: {filename}")
# Log summary
clean_count = 1 if separated_audio["clean_instrumental"].get("instrumental") else 0
combined_count = len(separated_audio["combined_instrumentals"])
self.logger.info(f"Directory scan complete: {clean_count} clean instrumental, {combined_count} combined instrumentals")
return separated_audio
async def prep_single_track(self):
# Add signal handler at the start
loop = asyncio.get_running_loop()
@@ -419,6 +517,9 @@ class KaraokePrep:
# No still image for audio-only downloads
processed_track["input_still_image"] = None
except UserCancelledError:
# User cancelled - propagate up to CLI for graceful exit
raise
except NoResultsError as e:
self.logger.error(f"No audio found: {e}")
return None
@@ -761,6 +862,18 @@ class KaraokePrep:
f"Applying {padding_seconds}s padding to all instrumental files to sync with vocal countdown"
)
# If separated_audio is empty (e.g., transcription was skipped but existing files have countdown),
# scan the directory for existing instrumental files
has_instrumentals = (
processed_track["separated_audio"].get("clean_instrumental", {}).get("instrumental") or
processed_track["separated_audio"].get("combined_instrumentals")
)
if not has_instrumentals:
self.logger.info("No instrumentals in separated_audio, scanning directory for existing files...")
processed_track["separated_audio"] = self._scan_directory_for_instrumentals(
track_output_dir, artist_title
)
# Apply padding using AudioProcessor
padded_separation_result = self.audio_processor.apply_countdown_padding_to_instrumentals(
separation_result=processed_track["separated_audio"],
+81 -4
View File
@@ -11,6 +11,9 @@ from .utils import sanitize_filename
# Placeholder class or functions for lyrics processing
class LyricsProcessor:
# Standard countdown padding duration used by LyricsTranscriber
COUNTDOWN_PADDING_SECONDS = 3.0
def __init__(
self, logger, style_params_json, lyrics_file, skip_transcription, skip_transcription_review, render_video, subtitle_offset_ms
):
@@ -22,6 +25,60 @@ class LyricsProcessor:
self.render_video = render_video
self.subtitle_offset_ms = subtitle_offset_ms
def _detect_countdown_padding_from_lrc(self, lrc_filepath):
"""
Detect if countdown padding was applied by checking the first lyric timestamp in the LRC file.
LRC format timestamps look like: [mm:ss.xx] or [mm:ss.xxx]
If the first lyric timestamp is >= 3.0 seconds, countdown padding was likely applied.
Args:
lrc_filepath: Path to the LRC file
Returns:
Tuple of (countdown_padding_added: bool, countdown_padding_seconds: float)
"""
try:
with open(lrc_filepath, 'r', encoding='utf-8') as f:
content = f.read()
# Find all timestamp patterns in the LRC file
# LRC timestamps: [mm:ss.xx] or [mm:ss.xxx]
timestamp_pattern = r'\[(\d{1,2}):(\d{2})\.(\d{2,3})\]'
matches = re.findall(timestamp_pattern, content)
if not matches:
self.logger.debug("No timestamps found in LRC file")
return (False, 0.0)
# Find the first non-metadata timestamp (metadata like [ar:Artist] doesn't have decimal)
# We already filtered for decimal timestamps in our pattern
first_timestamp = matches[0]
minutes = int(first_timestamp[0])
seconds = int(first_timestamp[1])
# Handle both .xx and .xxx formats
centiseconds = first_timestamp[2]
if len(centiseconds) == 2:
milliseconds = int(centiseconds) * 10
else:
milliseconds = int(centiseconds)
first_lyric_time = minutes * 60 + seconds + milliseconds / 1000.0
self.logger.debug(f"First lyric timestamp in LRC: {first_lyric_time:.3f}s")
# If first lyric is at or after 3 seconds, countdown padding was applied
# Use a small buffer (2.5s) to account for songs that naturally start a bit late
if first_lyric_time >= 2.5:
self.logger.info(f"Detected countdown padding from LRC: first lyric at {first_lyric_time:.2f}s")
return (True, self.COUNTDOWN_PADDING_SECONDS)
return (False, 0.0)
except Exception as e:
self.logger.warning(f"Failed to detect countdown padding from LRC file: {e}")
return (False, 0.0)
def find_best_split_point(self, line):
"""
Find the best split point in a line based on the specified criteria.
@@ -138,23 +195,43 @@ class LyricsProcessor:
lyrics_video_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (With Vocals).mkv")
lyrics_lrc_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (Karaoke).lrc")
# If files exist in parent directory, return early
# If files exist in parent directory, return early (but detect countdown padding first)
if os.path.exists(parent_video_path) and os.path.exists(parent_lrc_path):
self.logger.info(f"Found existing video and LRC files in parent directory, skipping transcription")
self.logger.info("Found existing video and LRC files in parent directory, skipping transcription")
# Detect countdown padding from existing LRC file
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
if countdown_padding_added:
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
return {
"lrc_filepath": parent_lrc_path,
"ass_filepath": parent_video_path,
"countdown_padding_added": countdown_padding_added,
"countdown_padding_seconds": countdown_padding_seconds,
"padded_audio_filepath": None, # Original padded audio may not exist
}
# If files exist in lyrics directory, copy to parent and return
# If files exist in lyrics directory, copy to parent and return (but detect countdown padding first)
if os.path.exists(lyrics_video_path) and os.path.exists(lyrics_lrc_path):
self.logger.info(f"Found existing video and LRC files in lyrics directory, copying to parent")
self.logger.info("Found existing video and LRC files in lyrics directory, copying to parent")
os.makedirs(track_output_dir, exist_ok=True)
shutil.copy2(lyrics_video_path, parent_video_path)
shutil.copy2(lyrics_lrc_path, parent_lrc_path)
# Detect countdown padding from existing LRC file
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
if countdown_padding_added:
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
return {
"lrc_filepath": parent_lrc_path,
"ass_filepath": parent_video_path,
"countdown_padding_added": countdown_padding_added,
"countdown_padding_seconds": countdown_padding_seconds,
"padded_audio_filepath": None, # Original padded audio may not exist
}
# Create lyrics directory if it doesn't exist
+3
View File
@@ -19,6 +19,9 @@ from karaoke_gen.karaoke_finalise import KaraokeFinalise
# Global logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO) # Set initial log level
# Prevent log propagation to root logger to avoid duplicate logs
# when external packages (like lyrics_converter) configure root logger handlers
logger.propagate = False
async def process_track_prep(row, args, logger, log_formatter):
+14 -2
View File
@@ -90,6 +90,11 @@ def create_parser(prog: str = "karaoke-gen") -> argparse.ArgumentParser:
metavar="JOB_ID",
help="Cancel a running job (remote mode only). Stops processing but keeps the job record. Example: --cancel abc12345",
)
workflow_group.add_argument(
"--retry",
metavar="JOB_ID",
help="Retry a failed job from the last successful checkpoint (remote mode only). Example: --retry abc12345",
)
workflow_group.add_argument(
"--delete",
metavar="JOB_ID",
@@ -203,6 +208,11 @@ def create_parser(prog: str = "karaoke-gen") -> argparse.ArgumentParser:
default="flac",
help="Optional: format / file extension for instrumental track to use for remux (default: %(default)s). Example: --instrumental_format=mp3",
)
audio_group.add_argument(
"--skip_instrumental_review",
action="store_true",
help="Optional: Skip the interactive instrumental review UI and use the old numeric selection. Example: --skip_instrumental_review",
)
# Lyrics Configuration
lyrics_group = parser.add_argument_group("Lyrics Configuration")
@@ -342,8 +352,10 @@ def create_parser(prog: str = "karaoke-gen") -> argparse.ArgumentParser:
)
remote_group.add_argument(
"--review-ui-url",
default=os.environ.get('REVIEW_UI_URL', 'https://lyrics.nomadkaraoke.com'),
help="Lyrics review UI URL (default: https://lyrics.nomadkaraoke.com)",
default=os.environ.get('REVIEW_UI_URL', os.environ.get('LYRICS_REVIEW_UI_URL', 'local')),
help="Lyrics review UI URL. Use 'local' for bundled frontend (default), "
"'http://localhost:5173' for Vite dev server, or 'https://lyrics.nomadkaraoke.com' for hosted. "
"(env: REVIEW_UI_URL or LYRICS_REVIEW_UI_URL)",
)
remote_group.add_argument(
"--poll-interval",
+369 -2
View File
@@ -14,14 +14,285 @@ import sys
import json
import asyncio
import time
import glob
import pyperclip
from karaoke_gen import KaraokePrep
from karaoke_gen.karaoke_finalise import KaraokeFinalise
from karaoke_gen.audio_fetcher import UserCancelledError
from karaoke_gen.instrumental_review import (
AudioAnalyzer,
WaveformGenerator,
InstrumentalReviewServer,
)
from .cli_args import create_parser, process_style_overrides, is_url, is_file
def _resolve_path_for_cwd(path: str, track_dir: str) -> str:
"""
Resolve a path that may have been created relative to the original working directory.
After os.chdir(track_dir), paths like './TrackDir/stems/file.flac' become invalid.
This function converts such paths to work from the new current directory.
Args:
path: The path to resolve (may be relative or absolute)
track_dir: The track directory we've chdir'd into
Returns:
A path that's valid from the current working directory
"""
if os.path.isabs(path):
return path
# Normalize both paths for comparison
norm_path = os.path.normpath(path)
norm_track_dir = os.path.normpath(track_dir)
# If path starts with track_dir, strip it to get the relative path from within track_dir
# e.g., './Four Lanes Male Choir - The White Rose/stems/file.flac' -> 'stems/file.flac'
if norm_path.startswith(norm_track_dir + os.sep):
return norm_path[len(norm_track_dir) + 1:]
elif norm_path.startswith(norm_track_dir):
return norm_path[len(norm_track_dir):].lstrip(os.sep) or '.'
# If path doesn't start with track_dir, it might already be relative to track_dir
# or it's a path that doesn't need transformation
return path
def auto_select_instrumental(track: dict, track_dir: str, logger: logging.Logger) -> str:
"""
Auto-select the best instrumental file when --skip_instrumental_review is used.
Selection priority:
1. Padded combined instrumental (+BV) - synchronized with vocals + backing vocals
2. Non-padded combined instrumental (+BV) - has backing vocals
3. Padded clean instrumental - synchronized with vocals
4. Non-padded clean instrumental - basic instrumental
Args:
track: The track dictionary from KaraokePrep containing separated audio info
track_dir: The track output directory (we're already chdir'd into it)
logger: Logger instance
Returns:
Path to the selected instrumental file
Raises:
FileNotFoundError: If no suitable instrumental file can be found
"""
separated = track.get("separated_audio", {})
# Look for combined instrumentals first (they include backing vocals)
combined = separated.get("combined_instrumentals", {})
for model, path in combined.items():
if path:
resolved = _resolve_path_for_cwd(path, track_dir)
# Prefer padded version if it exists
base, ext = os.path.splitext(resolved)
padded = f"{base} (Padded){ext}"
if os.path.exists(padded):
logger.info(f"Auto-selected padded combined instrumental: {padded}")
return padded
if os.path.exists(resolved):
logger.info(f"Auto-selected combined instrumental: {resolved}")
return resolved
# Fall back to clean instrumental
clean = separated.get("clean_instrumental", {})
if clean.get("instrumental"):
resolved = _resolve_path_for_cwd(clean["instrumental"], track_dir)
# Prefer padded version if it exists
base, ext = os.path.splitext(resolved)
padded = f"{base} (Padded){ext}"
if os.path.exists(padded):
logger.info(f"Auto-selected padded clean instrumental: {padded}")
return padded
if os.path.exists(resolved):
logger.info(f"Auto-selected clean instrumental: {resolved}")
return resolved
# If separated_audio doesn't have what we need, search the directory
# This handles edge cases and custom instrumentals
logger.info("No instrumental found in separated_audio, searching directory...")
instrumental_files = glob.glob("*(Instrumental*.flac") + glob.glob("*(Instrumental*.wav")
# Sort to prefer padded versions and combined instrumentals
padded_combined = [f for f in instrumental_files if "(Padded)" in f and "+BV" in f]
if padded_combined:
logger.info(f"Auto-selected from directory: {padded_combined[0]}")
return padded_combined[0]
padded_files = [f for f in instrumental_files if "(Padded)" in f]
if padded_files:
logger.info(f"Auto-selected from directory: {padded_files[0]}")
return padded_files[0]
combined_files = [f for f in instrumental_files if "+BV" in f]
if combined_files:
logger.info(f"Auto-selected from directory: {combined_files[0]}")
return combined_files[0]
if instrumental_files:
logger.info(f"Auto-selected from directory: {instrumental_files[0]}")
return instrumental_files[0]
raise FileNotFoundError(
"No instrumental file found. Audio separation may have failed. "
"Check the stems/ directory for separated audio files."
)
def run_instrumental_review(track: dict, logger: logging.Logger) -> str | None:
"""
Run the instrumental review UI to let user select the best instrumental track.
This analyzes the backing vocals, generates a waveform, and opens a browser
with an interactive UI for reviewing and selecting the instrumental.
Args:
track: The track dictionary from KaraokePrep containing separated audio info
logger: Logger instance
Returns:
Path to the selected instrumental file, or None to use the old numeric selection
"""
track_dir = track.get("track_output_dir", ".")
artist = track.get("artist", "")
title = track.get("title", "")
base_name = f"{artist} - {title}"
# Get separation results
separated = track.get("separated_audio", {})
if not separated:
logger.info("No separated audio found, skipping instrumental review UI")
return None
# Find the backing vocals file
# Note: Paths in separated_audio may be relative to the original working directory,
# but we've already chdir'd into track_dir. Use _resolve_path_for_cwd to fix paths.
backing_vocals_path = None
backing_vocals_result = separated.get("backing_vocals", {})
for model, paths in backing_vocals_result.items():
if paths.get("backing_vocals"):
backing_vocals_path = _resolve_path_for_cwd(paths["backing_vocals"], track_dir)
break
if not backing_vocals_path or not os.path.exists(backing_vocals_path):
logger.info("No backing vocals file found, skipping instrumental review UI")
return None
# Find the clean instrumental file
clean_result = separated.get("clean_instrumental", {})
raw_clean_path = clean_result.get("instrumental")
clean_instrumental_path = _resolve_path_for_cwd(raw_clean_path, track_dir) if raw_clean_path else None
if not clean_instrumental_path or not os.path.exists(clean_instrumental_path):
logger.info("No clean instrumental file found, skipping instrumental review UI")
return None
# Find the combined instrumental (with backing vocals) file - these have "(Padded)" suffix if padded
combined_result = separated.get("combined_instrumentals", {})
with_backing_path = None
for model, path in combined_result.items():
resolved_path = _resolve_path_for_cwd(path, track_dir) if path else None
if resolved_path and os.path.exists(resolved_path):
with_backing_path = resolved_path
break
try:
logger.info("=== Starting Instrumental Review ===")
logger.info(f"Analyzing backing vocals: {backing_vocals_path}")
# Analyze backing vocals
analyzer = AudioAnalyzer()
analysis = analyzer.analyze(backing_vocals_path)
logger.info(f"Analysis complete:")
logger.info(f" Has audible content: {analysis.has_audible_content}")
logger.info(f" Total duration: {analysis.total_duration_seconds:.1f}s")
logger.info(f" Audible segments: {len(analysis.audible_segments)}")
logger.info(f" Recommendation: {analysis.recommended_selection.value}")
# Generate waveform
# Note: We're already in track_dir after chdir, so use current directory
logger.info("Generating waveform visualization...")
waveform_generator = WaveformGenerator()
waveform_path = f"{base_name} (Backing Vocals Waveform).png"
waveform_generator.generate(
audio_path=backing_vocals_path,
output_path=waveform_path,
segments=analysis.audible_segments,
)
# Start the review server
# Note: We're already in track_dir after chdir, so output_dir is "."
logger.info("Starting instrumental review UI...")
server = InstrumentalReviewServer(
output_dir=".",
base_name=base_name,
analysis=analysis,
waveform_path=waveform_path,
backing_vocals_path=backing_vocals_path,
clean_instrumental_path=clean_instrumental_path,
with_backing_path=with_backing_path,
)
# Start server and open browser, wait for selection
server.start_and_open_browser()
logger.info("Waiting for instrumental selection in browser...")
logger.info("(Close the browser tab or press Ctrl+C to cancel)")
try:
# Wait for user selection (blocking)
server._selection_event.wait()
selection = server.get_selection()
logger.info(f"User selected: {selection}")
# Stop the server
server.stop()
# Return the selected instrumental path
if selection == "clean":
return clean_instrumental_path
elif selection == "with_backing":
return with_backing_path
elif selection == "custom":
custom_path = server.get_custom_instrumental_path()
if custom_path and os.path.exists(custom_path):
return custom_path
else:
logger.warning("Custom instrumental not found, falling back to clean")
return clean_instrumental_path
elif selection == "uploaded":
uploaded_path = server.get_uploaded_instrumental_path()
if uploaded_path and os.path.exists(uploaded_path):
return uploaded_path
else:
logger.warning("Uploaded instrumental not found, falling back to clean")
return clean_instrumental_path
else:
logger.warning(f"Unknown selection: {selection}, falling back to numeric selection")
return None
except KeyboardInterrupt:
logger.info("Instrumental review cancelled by user")
server.stop()
return None
except Exception as e:
logger.error(f"Error during instrumental review: {e}")
logger.info("Falling back to numeric selection")
return None
async def async_main():
logger = logging.getLogger(__name__)
# Prevent log propagation to root logger to avoid duplicate logs
# when external packages (like lyrics_converter) configure root logger handlers
logger.propagate = False
log_handler = logging.StreamHandler()
log_formatter = logging.Formatter(fmt="%(asctime)s.%(msecs)03d - %(levelname)s - %(module)s - %(message)s", datefmt="%Y-%m-%d %H:%M:%S")
log_handler.setFormatter(log_formatter)
@@ -31,6 +302,11 @@ async def async_main():
parser = create_parser(prog="karaoke-gen")
args = parser.parse_args()
# Set review UI URL environment variable for the lyrics transcriber review server
# This allows development against a local frontend dev server (e.g., http://localhost:5173)
if hasattr(args, 'review_ui_url') and args.review_ui_url:
os.environ['LYRICS_REVIEW_UI_URL'] = args.review_ui_url
# Process style overrides
try:
style_overrides = process_style_overrides(args.style_override, logger)
@@ -122,7 +398,21 @@ async def async_main():
kprep.input_media = input_audio_wav
# Run KaraokePrep
tracks = await kprep.process()
try:
tracks = await kprep.process()
except UserCancelledError:
logger.info("Operation cancelled by user")
return
except KeyboardInterrupt:
logger.info("Operation cancelled by user (Ctrl+C)")
return
# Filter out None tracks (can happen if prep failed for some tracks)
tracks = [t for t in tracks if t is not None] if tracks else []
if not tracks:
logger.warning("No tracks to process")
return
# Load CDG styles if CDG generation is enabled
cdg_styles = None
@@ -441,7 +731,21 @@ async def async_main():
kprep = kprep_coroutine
# Create final tracks data structure
tracks = await kprep.process()
try:
tracks = await kprep.process()
except UserCancelledError:
logger.info("Operation cancelled by user")
return
except KeyboardInterrupt:
logger.info("Operation cancelled by user (Ctrl+C)")
return
# Filter out None tracks (can happen if prep failed for some tracks)
tracks = [t for t in tracks if t is not None] if tracks else []
if not tracks:
logger.warning("No tracks to process")
return
# If prep-only mode, we're done
if args.prep_only:
@@ -461,6 +765,67 @@ async def async_main():
logger.info(f"Changing to directory: {track_dir}")
os.chdir(track_dir)
# Select instrumental file - either via web UI or auto-selection
# This ALWAYS produces a selected file - no silent fallback to legacy code
selected_instrumental_file = None
skip_review = getattr(args, 'skip_instrumental_review', False)
if skip_review:
# Auto-select instrumental when review is skipped (non-interactive mode)
logger.info("Instrumental review skipped (--skip_instrumental_review), auto-selecting instrumental file...")
try:
selected_instrumental_file = auto_select_instrumental(
track=track,
track_dir=track_dir,
logger=logger,
)
except FileNotFoundError as e:
logger.error(f"Failed to auto-select instrumental: {e}")
logger.error("Check that audio separation completed successfully.")
sys.exit(1)
return # Explicit return for testing
else:
# Run instrumental review web UI
selected_instrumental_file = run_instrumental_review(
track=track,
logger=logger,
)
# If instrumental review failed/returned None, show error and exit
# NO SILENT FALLBACK - we want to know if the new flow has issues
if selected_instrumental_file is None:
logger.error("")
logger.error("=" * 70)
logger.error("INSTRUMENTAL SELECTION FAILED")
logger.error("=" * 70)
logger.error("")
logger.error("The instrumental review UI could not find the required files.")
logger.error("")
logger.error("Common causes:")
logger.error(" - No backing vocals file was found (check stems/ directory)")
logger.error(" - No clean instrumental was found (audio separation may have failed)")
logger.error(" - Path resolution failed after directory change")
logger.error("")
logger.error("To investigate:")
logger.error(" - Check the stems/ directory for: *Backing Vocals*.flac and *Instrumental*.flac")
logger.error(" - Look for separation errors earlier in the log")
logger.error(" - Verify audio separation completed without errors")
logger.error("")
logger.error("Workarounds:")
logger.error(" - Re-run with --skip_instrumental_review to auto-select an instrumental")
logger.error(" - Re-run the full pipeline to regenerate stems")
logger.error("")
sys.exit(1)
return # Explicit return for testing
logger.info(f"Selected instrumental file: {selected_instrumental_file}")
# Get countdown padding info from track (if vocals were padded, instrumental must match)
countdown_padding_seconds = None
if track.get("countdown_padding_added", False):
countdown_padding_seconds = track.get("countdown_padding_seconds", 3.0)
logger.info(f"Countdown padding detected: {countdown_padding_seconds}s (will be applied to instrumental if needed)")
# Load CDG styles if CDG generation is enabled
cdg_styles = None
if args.enable_cdg:
@@ -504,6 +869,8 @@ async def async_main():
cdg_styles=cdg_styles,
keep_brand_code=getattr(args, 'keep_brand_code', False),
non_interactive=args.yes,
selected_instrumental_file=selected_instrumental_file,
countdown_padding_seconds=countdown_padding_seconds,
)
try:
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,336 @@
# MidiCo Synchronizer Reference
This document provides a detailed reference for replicating the MidiCo Synchronizer user experience in our Nomad Karaoke lyrics review UI. The MidiCo Synchronizer is a native macOS application that provides a polished, intuitive interface for manually syncing lyrics to audio.
## Problem Statement
Sometimes transcribed lyrics are correct and start well-synced, but progressively drift out of sync towards the second half of the song. We need an efficient way to fix timing for multiple segments without having to re-sync everything from scratch.
## Overview of MidiCo Interface
MidiCo consists of two windows:
1. **Main Audio Player Window** - Controls audio playback
2. **Synchronizer Window** - Timeline-based lyrics sync interface
![Empty Timeline Before Sync](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Empty-Timeline-Before-Any-Words-Synced.png)
---
## 1. Audio Player Integration
The main MidiCo window provides an audio player that stays synchronized with the Synchronizer:
- **Play/Pause/Stop buttons** - Standard playback controls
- **Rewind/Fast-forward buttons** - Skip through the track
- **Progress slider** - Shows current position (00:00 format) and remaining time (-03:45 format)
- **Volume slider** - Control audio volume
- **Tempo/Pitch sliders** - Adjust playback speed and pitch (useful for difficult sections)
**Key behavior:** The playhead position is always synced between the audio player and the Synchronizer timeline.
---
## 2. Synchronizer Primary View
The Synchronizer window contains the main timeline interface for syncing lyrics.
### 2.1 Top Control Buttons
Four main control buttons at the top:
| Button | Function |
|--------|----------|
| **Voice no.1** | Select which vocal track to sync (for multi-voice karaoke) |
| **Start Sync** | Begin sync mode - starts audio, shows upcoming words, listens for spacebar |
| **Clear Sync** | Delete all sync data, reset to unsynced state |
| **Edit Lyrics** | Open modal to edit the lyrics text directly |
| **Import Lrc** | Import timing data from an LRC file |
### 2.2 Timeline View
The main rectangular timeline area is the core of the interface.
![During Sync Showing Synced And Upcoming Words](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Synced-And-Upcoming-Words.png)
#### Time Bar (Top Strip)
- Thin bar at the top showing time in `MM:SS` format
- Small tick marks for decisecond precision
- Clicking anywhere on the time bar sets the playhead position (without changing playback state)
#### Playhead Marker
- Small arrow/triangle indicator in the time bar
- White vertical line spanning the full height of the timeline
- Shows current playback position
- Synced with the audio player position
#### Timeline Background
- Light gray gradient background
- **Clicking on the background deselects any selected words**
- **Click-drag creates a selection rectangle** for selecting multiple word blocks
#### Zoom Control
- Horizontal slider below the timeline labeled "Zoom"
- Smooth continuous zoom (approximately 50 notches)
- **Most zoomed in:** ~4.5 seconds visible edge-to-edge
- **Most zoomed out:** ~24 seconds visible edge-to-edge
- Smooth, responsive zooming is critical for fast songs
![Max Zoom Level](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Max-Zoom-Level.png)
#### Horizontal Scrolling
- Timeline scrolls horizontally to show different parts of the song
- Small horizontal scrollbar at the bottom of the timeline area
- During sync mode, the timeline auto-scrolls to follow the playhead
### 2.3 Word Blocks
Synced words appear as **red rectangular blocks** on the timeline:
- Each block shows the word text above the red bar
- If word text is too long to fit the width of the red bar, the alignment with the red blocks becomes "best effort". Words don't overlap, they're all readable above the red bars for that segment
- Block width corresponds to the word's duration
- Block horizontal position corresponds to the word's start time
#### Two-Level Word Layout (Alternating Segments)
Segments alternate between two vertical positions on the timeline. Each consecutive segment (in time order) is placed on the opposite level from the previous one:
- **Level 0 (top)**: 1st, 3rd, 5th, 7th... segments
- **Level 1 (bottom)**: 2nd, 4th, 6th, 8th... segments
![Two Level Word Blocks](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Two-Levels-Word-Blocks.png)
This alternating layout is critical because:
1. **Text never overlaps between consecutive segments** - Since each segment is on a different level, the text labels above the red blocks from one segment won't collide with text from the next segment, even if the text extends past its block width.
2. **All words remain readable** - Text is allowed to extend horizontally beyond the red block boundaries. Because consecutive segments are always on different vertical levels, this extended text doesn't overlap with adjacent segments.
![Non-Overlapping Segments](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Timeline-With-Non-Overlapping-Segments.png)
In the example above, notice how:
- "Monica Mountains" (level 0) has text that extends beyond its block
- "Where she then proceeded to pump her vehicle in and out of turns" (level 1) is on a different vertical level
- "Sometimes dropping down to fifty miles per hour" (level 0) returns to the top level
- "Only to immediately gun it back up to 90 again" (level 1) is back on the bottom level
This simple alternating pattern ensures that even long segment text remains fully readable without truncation.
**Edge case:** Overlap can still occur in rare situations - if one segment's text is extremely long AND the segment two positions later (on the same level) is very short/early. But this is uncommon in practice.
#### Word Block Selection
- **Single click** on a word block: Selects that block (shown with white 1px border)
- **Click-drag on background**: Creates selection rectangle to select multiple blocks
![Click Drag Word Selection](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Click-Drag-Word-Selection.png)
#### Word Block Manipulation (Resize & Drag)
Selected word blocks can be manipulated directly on the timeline:
##### Resizing Word Blocks (Change Duration)
When a word block is selected and you hover over it, a **small white dot/handle** appears just inside the right edge of the block. This resize handle allows you to change the word's duration:
1. Select a word block (single click)
2. Hover over the selected block to reveal the resize handle (white dot on right edge)
3. Click and drag the handle left/right to shorten/lengthen the word's duration
4. The word's end time changes; start time remains fixed
This is essential for fine-tuning word durations after initial sync, especially for:
- Words that were synced too long or too short
- Adjusting timing to better match the audio
- Creating smoother transitions between words
##### Dragging Word Blocks (Change Position)
Selected word blocks can be dragged horizontally to change their position in time:
**Single word:**
1. Select a word block
2. Click and drag anywhere on the block (not on the resize handle)
3. The entire word moves earlier/later in time (both start and end times shift)
**Multiple words:**
1. Drag-select multiple word blocks (or Ctrl/Cmd+click to add to selection)
2. Click and drag any of the selected blocks
3. **All selected blocks move together**, maintaining their relative timing
This is particularly useful for:
- Shifting an entire segment earlier or later when it's consistently off
- Adjusting groups of words that were synced correctly relative to each other but offset from the audio
- Fine-tuning timing without having to re-sync
**Example workflow:** If a segment of 10 words all come in 0.2 seconds too late:
1. Drag-select all 10 word blocks
2. Drag them 0.2 seconds earlier
3. All words maintain their relative timing but are now properly aligned with the audio
### 2.4 Upcoming Words Display
During sync mode, unsynced words appear in **two locations**:
![Start Sync Showing Upcoming Words](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Start-Sync-Showing-Upcoming-Words.png)
1. **Fixed position above time bar** (left side): Shows upcoming words as red/white blocks in a horizontal row. The next word to sync is highlighted in red, subsequent words in white.
2. **On the timeline, right of playhead**: Same words appear starting from the playhead position and extending to the right, with a pale yellow background and black text, moving along with the playhead during playback.
This dual display makes it easy to:
- See what's coming next (fixed position - easy to read while focused on timing)
- See where words will land on the timeline relative to already-synced words
---
## 3. Action Buttons (Icon Toolbar)
A row of small icon-only buttons appears above the timeline. Key buttons we need:
### Essential Buttons
| Icon | Tooltip | Function |
|------|---------|----------|
| **Rectangle with Pencil** | Split, spell or edit selected word | Opens popup to edit a single word. Useful for typos. Entering multiple space-separated words splits the selected word while preserving timing. |
| **No entry symbol** | Unsynchronize from cursor position | All word blocks **after** the current playhead position are reset to unsynced state. Critical for fixing drift in later parts of a song. |
| **Trash** | Delete selected | Deletes selected word(s) entirely. Useful for removing backing vocals or nonsense words. |
### Typical Workflow for Deleting Unwanted Words
1. Tap spacebar to sync the unwanted words (so they appear on timeline)
2. Stop playback
3. Click-drag to select the unwanted word blocks
4. Click trash icon to delete
5. Click Start Sync to resume syncing from where you left off
---
## 4. Sync Mode Operation
### Starting Sync
1. Click **Start Sync** button
2. Audio begins playing from current playhead position
3. Upcoming unsynced words appear in both display locations
4. System begins listening for spacebar input
### Spacebar Timing
Two input modes:
**Tap (press and release quickly):**
- Sets the **start time** of the next unsynced word to the current playhead position
- The word's **end time** is set when the following word is synced
- If gap between words > 1 second, previous word gets default 1 second duration
**Press-hold:**
- Hold spacebar down when word starts
- Release when word ends
- Sets both start and end times explicitly
### Stopping Sync
- Click the **Stop** button (replaces Play when in sync mode)
- Or click **Start Sync** again to toggle off
- Sync progress is preserved
---
## 5. Edit Lyrics Modal
![Lyrics Editor Modal](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Lyrics-Editor-Modal.png)
Accessed via the **Edit Lyrics** button. Provides:
- **Left panel**: Plain text editor with all lyrics, newlines separate segments
- **Right panel**: Preview of how lyrics will appear (karaoke style rendering)
- **Language selector**: For hyphenation support
- **Hyphenate button**: Auto-hyphenate words for syllable-level sync
- **Reset button**: Revert to original lyrics
- **Cancel/OK buttons**: Discard or apply changes
### Important Behavior
Editing lyrics when there are already synced timestamps **usually messes up the syncs**. Users typically need to Clear Sync and start over after editing lyrics.
---
## 6. Apply Button
The **Apply** button in the bottom-right corner is critical:
- Changes made in the Synchronizer are **not saved** until Apply is clicked
- This allows experimenting without fear of losing work
- If sync goes badly wrong, user can close the Synchronizer without applying changes
- Provides a safe "sandbox" for sync work
---
## 7. Performance Requirements
The entire interface must be:
- **Smooth and responsive** - No lag or stutter during playback
- **Low latency** - Spacebar input must register with sub-100ms precision
- **Efficient** - No unnecessary computation during sync (user is focused on timing)
- **Native-app feel** - As expected from a macOS application
Since we're dealing with sub-second time precision for synced lyrics, performance is critical.
---
## 8. Implementation Notes for Our App
### Entry Point: "Edit All" Button
The current "Edit All" button should show a choice:
1. **Replace all lyrics** - Shows the existing paste box flow (for completely new lyrics)
2. **Re-sync existing lyrics** - Goes directly to the new Synchronizer view with all existing words/segments plotted on the timeline
This preserves existing sync data while allowing re-sync of later portions.
### Key Features to Implement
1. **Timeline with word blocks showing existing sync data**
2. **Playhead synced with audio player**
3. **Zoom slider (4.5s to 24s range)**
4. **Horizontal scrolling**
5. **Two-level word block layout**
6. **Start Sync mode with spacebar capture**
7. **Upcoming words display (fixed + timeline)**
8. **"Unsynchronize from cursor" - critical for fixing drift**
9. **Word selection (single click + drag-select)**
10. **Delete selected words**
11. **Edit single word popup**
12. **Apply/Cancel for safe experimentation**
### Fixing Timing Drift (Primary Use Case)
The main workflow for fixing drift:
1. Open Synchronizer (with existing sync data loaded)
2. Play audio to find where sync starts drifting
3. Position playhead at last correctly-synced word
4. Click "Unsynchronize from cursor position"
5. Click "Start Sync" to resume syncing from that point
6. Tap spacebar to re-sync the remaining words
7. Click "Apply" when done
This workflow allows keeping good sync data while fixing only the problematic portion.
---
## Screenshots Reference
All screenshots are located in: `docs/midico-sync-screenshots/`
| Filename | Description |
|----------|-------------|
| `MidiCo-Synchronizer-2025-12-18-Empty-Timeline-Before-Any-Words-Synced.png` | Initial state with empty timeline |
| `MidiCo-Synchronizer-2025-12-18-Start-Sync-Showing-Upcoming-Words.png` | Sync mode started, showing upcoming words in both locations |
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Synced-And-Upcoming-Words.png` | Mid-sync showing both synced blocks and upcoming words |
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Two-Levels-Word-Blocks.png` | Demonstration of two-level word block layout |
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Max-Zoom-Level.png` | Maximum zoom level (~4.5 seconds visible) |
| `MidiCo-Synchronizer-2025-12-18-Click-Drag-Word-Selection.png` | Selection rectangle for multi-word selection |
| `MidiCo-Synchronizer-2025-12-18-Lyrics-Editor-Modal.png` | Edit Lyrics modal dialog |
| `MidiCo-Synchronizer-2025-12-18-Timeline-With-Non-Overlapping-Segments.png` | Shows alternating segment levels with long text that doesn't overlap |
Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

@@ -1,9 +1,8 @@
import threading
import time
from typing import Any, Dict, List, Optional, Tuple, Union
from typing import Any, Dict, List, Optional, Set, Tuple, Union
import logging
from tqdm import tqdm
from multiprocessing import Pool, cpu_count
from functools import partial
from pathlib import Path
import json
@@ -64,8 +63,70 @@ class AnchorSequenceFinder:
# self.logger.debug(f"_find_ngrams called with {len(words)} words, n={n}")
return [(words[i : i + n], i) for i in range(len(words) - n + 1)]
def _build_ngram_index(
self,
references: Dict[str, List[str]],
n: int
) -> Dict[Tuple[str, ...], Dict[str, List[int]]]:
"""
Build a hash-based index mapping n-grams to their positions in each reference.
Args:
references: Dict mapping source names to lists of cleaned words
n: The n-gram length to index
Returns:
Dict mapping n-gram tuples to {source: [positions]} dicts
"""
index: Dict[Tuple[str, ...], Dict[str, List[int]]] = {}
for source, words in references.items():
for i in range(len(words) - n + 1):
ngram_tuple = tuple(words[i:i + n])
if ngram_tuple not in index:
index[ngram_tuple] = {}
if source not in index[ngram_tuple]:
index[ngram_tuple][source] = []
index[ngram_tuple][source].append(i)
return index
def _find_matching_sources_indexed(
self,
ngram: List[str],
ngram_index: Dict[Tuple[str, ...], Dict[str, List[int]]]
) -> Dict[str, int]:
"""
Find which sources contain the given n-gram using pre-built index (O(1) lookup).
Args:
ngram: List of words to find
ngram_index: Pre-built index from _build_ngram_index()
Returns:
Dict mapping source names to first unused position
"""
matches = {}
ngram_tuple = tuple(ngram)
# O(1) lookup in the index
if ngram_tuple not in ngram_index:
return matches
source_positions = ngram_index[ngram_tuple]
# For each source that contains this n-gram, find first unused position
for source, positions in source_positions.items():
used = self.used_positions.get(source, set())
for pos in positions:
if pos not in used:
matches[source] = pos
break
return matches
def _find_matching_sources(self, ngram: List[str], references: Dict[str, List[str]], n: int) -> Dict[str, int]:
"""Find which sources contain the given n-gram and at what positions."""
"""Find which sources contain the given n-gram and at what positions (legacy O(n) method)."""
# self.logger.debug(f"_find_matching_sources called for ngram: '{' '.join(ngram)}'")
matches = {}
all_positions = {source: [] for source in references}
@@ -193,121 +254,59 @@ class AnchorSequenceFinder:
ref_words: Dict[str, List[Word]],
min_sources: int,
) -> List[AnchorSequence]:
"""Process a single n-gram length to find matching sequences with timeout and early termination."""
self.logger.info(f"🔍 N-GRAM {n}: Starting processing with {len(trans_words)} transcription words")
self.logger.info(f"🔍 N-GRAM {n}: Reference sources: {list(ref_texts_clean.keys())}")
self.logger.info(f"🔍 N-GRAM {n}: Max iterations limit: {self.max_iterations_per_ngram}")
"""Process a single n-gram length to find matching sequences using hash-based index."""
self.logger.debug(f"🔍 N-GRAM {n}: Starting processing with {len(trans_words)} transcription words")
candidate_anchors = []
used_positions = {source: set() for source in ref_texts_clean.keys()}
used_trans_positions = set()
used_trans_positions: Set[int] = set()
iteration_count = 0
last_progress_check = 0
last_anchor_count = 0
stagnation_count = 0
self.logger.debug(f"🔍 N-GRAM {n}: Processing n-gram length {n} with max {self.max_iterations_per_ngram} iterations")
# Build hash-based index for O(1) lookups
ngram_index = self._build_ngram_index(ref_texts_clean, n)
self.logger.debug(f"🔍 N-GRAM {n}: Built index with {len(ngram_index)} unique n-grams")
# Generate n-grams from transcribed text once
# Generate n-grams from transcribed text
trans_ngrams = self._find_ngrams(trans_words, n)
self.logger.info(f"🔍 N-GRAM {n}: Generated {len(trans_ngrams)} n-grams for processing")
self.logger.debug(f"🔍 N-GRAM {n}: Processing {len(trans_ngrams)} transcription n-grams")
# Process all n-grams efficiently in multiple passes
found_new_match = True
while found_new_match and iteration_count < self.max_iterations_per_ngram:
found_new_match = False
iteration_count += 1
anchors_found_this_iteration = 0
# Single pass through all transcription n-grams
for ngram, trans_pos in trans_ngrams:
# Skip if we've already used this transcription position
if trans_pos in used_trans_positions:
continue
# Log every 10th iteration to track progress
if iteration_count % 10 == 0:
self.logger.debug(f"🔍 N-GRAM {n}: Iteration {iteration_count}, anchors found: {len(candidate_anchors)}")
# Check for progress stagnation every N iterations
if iteration_count - last_progress_check >= self.progress_check_interval:
current_anchor_count = len(candidate_anchors)
if current_anchor_count == last_anchor_count:
stagnation_count += 1
self.logger.debug(f"🔍 N-GRAM {n}: Stagnation check {stagnation_count}/3 at iteration {iteration_count}")
if stagnation_count >= 3: # No progress for 3 consecutive checks
self.logger.info(f"🔍 N-GRAM {n}: ⏹️ Early termination due to stagnation after {iteration_count} iterations")
break
else:
stagnation_count = 0 # Reset stagnation counter
last_anchor_count = current_anchor_count
last_progress_check = iteration_count
self.logger.debug(f"🔍 N-GRAM {n}: iteration {iteration_count}, anchors: {current_anchor_count}, stagnation: {stagnation_count}")
# Process all n-grams in this iteration
for ngram, trans_pos in trans_ngrams:
# Skip if we've already used this transcription position
if trans_pos in used_trans_positions:
continue
# Get the actual words from the transcription at this position
actual_words = [w.text.lower().strip('.,?!"\n') for w in all_words[trans_pos : trans_pos + n]]
ngram_words = [w.lower() for w in ngram]
if actual_words != ngram_words:
self.logger.error(f"🔍 N-GRAM {n}: ❌ Mismatch between ngram and actual words at position {trans_pos}:")
self.logger.error(f"🔍 N-GRAM {n}: Ngram words: {ngram_words}")
self.logger.error(f"🔍 N-GRAM {n}: Actual words: {actual_words}")
self.logger.error(f"🔍 N-GRAM {n}: Full trans_words: {trans_words}")
self.logger.error(f"🔍 N-GRAM {n}: Full all_words: {[w.text for w in all_words]}")
raise AssertionError(
f"Ngram words don't match actual words at position {trans_pos}. "
f"This should never happen as trans_words should be derived from all_words."
)
matches = self._find_matching_sources(ngram, ref_texts_clean, n)
if len(matches) >= min_sources:
# Log successful match
if len(candidate_anchors) < 5: # Only log first few matches to avoid spam
self.logger.debug(f"🔍 N-GRAM {n}: ✅ Found match: '{' '.join(ngram)}' at pos {trans_pos} with {len(matches)} sources")
# Get Word IDs for transcribed words
transcribed_word_ids = [w.id for w in all_words[trans_pos : trans_pos + n]]
# Get Word IDs for reference words
reference_word_ids = {source: [w.id for w in ref_words[source][pos : pos + n]] for source, pos in matches.items()}
# Mark positions as used
for source, pos in matches.items():
used_positions[source].add(pos)
used_trans_positions.add(trans_pos)
anchor = AnchorSequence(
id=WordUtils.generate_id(),
transcribed_word_ids=transcribed_word_ids,
transcription_position=trans_pos,
reference_positions=matches,
reference_word_ids=reference_word_ids,
confidence=len(matches) / len(ref_texts_clean),
)
candidate_anchors.append(anchor)
anchors_found_this_iteration += 1
found_new_match = True
# For efficiency, if we have very low iteration limits, find one match per iteration
if self.max_iterations_per_ngram <= 10:
break
# Use indexed lookup (O(1) instead of O(n))
matches = self._find_matching_sources_indexed(ngram, ngram_index)
# Log progress for this iteration
if anchors_found_this_iteration > 0:
self.logger.debug(f"🔍 N-GRAM {n}: Found {anchors_found_this_iteration} anchors in iteration {iteration_count}")
# Early termination if we've found enough anchors or processed all positions
if len(used_trans_positions) >= len(trans_ngrams) or len(candidate_anchors) >= len(trans_ngrams):
self.logger.info(f"🔍 N-GRAM {n}: ⏹️ Early termination - processed all positions after {iteration_count} iterations")
break
if len(matches) >= min_sources:
# Get Word IDs for transcribed words
transcribed_word_ids = [w.id for w in all_words[trans_pos : trans_pos + n]]
if iteration_count >= self.max_iterations_per_ngram:
self.logger.warning(f"🔍 N-GRAM {n}: ⏰ Processing terminated after reaching max iterations ({self.max_iterations_per_ngram})")
# Get Word IDs for reference words
reference_word_ids = {
source: [w.id for w in ref_words[source][pos : pos + n]]
for source, pos in matches.items()
}
# Mark transcription position as used
used_trans_positions.add(trans_pos)
# Mark reference positions as used
for source, pos in matches.items():
if source not in self.used_positions:
self.used_positions[source] = set()
self.used_positions[source].add(pos)
anchor = AnchorSequence(
id=WordUtils.generate_id(),
transcribed_word_ids=transcribed_word_ids,
transcription_position=trans_pos,
reference_positions=matches,
reference_word_ids=reference_word_ids,
confidence=len(matches) / len(ref_texts_clean),
)
candidate_anchors.append(anchor)
self.logger.info(f"🔍 N-GRAM {n}: ✅ Completed processing after {iteration_count} iterations, found {len(candidate_anchors)} anchors")
self.logger.debug(f"🔍 N-GRAM {n}: Found {len(candidate_anchors)} anchors")
return candidate_anchors
def find_anchors(
@@ -347,8 +346,10 @@ class AnchorSequenceFinder:
self.logger.error("Could not serialize first cached anchor for logging")
# If not in cache or cache format invalid, perform the computation
self.logger.info(f"🔍 ANCHOR SEARCH: ❌ Cache miss - computing anchors with timeout {self.timeout_seconds}s")
self.logger.info(f"🔍 ANCHOR SEARCH: Finding anchor sequences for transcription with length {len(transcribed)}")
self.logger.info(f"🔍 ANCHOR SEARCH: Cache miss - computing anchors")
# Reset used positions for fresh computation
self.used_positions = {}
# Check timeout before starting computation
self._check_timeout(start_time, "anchor computation initialization")
@@ -407,122 +408,45 @@ class AnchorSequenceFinder:
min_sources=self.min_sources,
)
# Process n-gram lengths in parallel with timeout
# Process n-gram lengths sequentially (single-threaded for cloud compatibility)
candidate_anchors = []
pool_timeout = max(60, self.timeout_seconds // 2) if self.timeout_seconds > 0 else 300 # Use half the total timeout for pool operations
# Check timeout before parallel processing
self._check_timeout(start_time, "parallel processing start")
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Timeout check passed - about to start parallel processing")
# Check timeout before processing
self._check_timeout(start_time, "n-gram processing start")
self.logger.info(f"🔍 ANCHOR SEARCH: Starting sequential n-gram processing ({len(n_gram_lengths)} lengths)")
pool = None
try:
self.logger.info(f"🔍 ANCHOR SEARCH: 🚀 Starting parallel processing with {max(cpu_count() - 1, 1)} processes, pool timeout: {pool_timeout}s")
pool = Pool(processes=max(cpu_count() - 1, 1))
self.logger.debug(f"🔍 ANCHOR SEARCH: Pool created successfully")
results = []
# Submit all jobs first
self.logger.info(f"🔍 ANCHOR SEARCH: Submitting {len(n_gram_lengths)} n-gram processing jobs...")
async_results = []
for i, n in enumerate(n_gram_lengths):
self.logger.debug(f"🔍 ANCHOR SEARCH: Submitting job {i+1}/{len(n_gram_lengths)} for n-gram length {n}")
async_result = pool.apply_async(process_length_partial, (n,))
async_results.append(async_result)
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ All {len(async_results)} jobs submitted")
# Collect results with individual timeouts
batch_results = []
batch_size = 10
for i, async_result in enumerate(async_results):
n_gram_length = n_gram_lengths[i]
try:
# Check remaining time for pool timeout (more lenient than overall timeout)
batch_size = 10
batch_results = []
for i, n in enumerate(n_gram_lengths):
try:
# Check timeout periodically
if self.timeout_seconds > 0:
elapsed_time = time.time() - start_time
remaining_time = max(10, self.timeout_seconds - elapsed_time) if self.timeout_seconds > 0 else pool_timeout
self.logger.debug(f"🔍 ANCHOR SEARCH: Remaining time for n-gram {n_gram_length}: {remaining_time}s")
# Use a more lenient timeout for individual results to allow fallback
individual_timeout = min(pool_timeout, remaining_time) if self.timeout_seconds > 0 else pool_timeout
result = async_result.get(timeout=individual_timeout)
results.append(result)
# Batch logging - collect info for batched logging
batch_results.append((n_gram_length, len(result)))
# Log progress every batch_size results or on the last result
if (i + 1) % batch_size == 0 or (i + 1) == len(async_results):
total_anchors_in_batch = sum(anchor_count for _, anchor_count in batch_results)
n_gram_ranges = [str(ng) for ng, _ in batch_results]
range_str = f"{n_gram_ranges[0]}-{n_gram_ranges[-1]}" if len(n_gram_ranges) > 1 else n_gram_ranges[0]
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Completed n-gram lengths {range_str} ({i+1-len(batch_results)+1}-{i+1}/{len(async_results)}) - found {total_anchors_in_batch} anchors total")
batch_results = [] # Reset batch
except Exception as e:
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ n-gram length {n_gram_length} failed or timed out: {str(e)}")
results.append([]) # Add empty result to maintain order
# Add failed result to batch for logging
batch_results.append((n_gram_length, 0))
# If we're running short on time, trigger fallback early
if self.timeout_seconds > 0 and (time.time() - start_time) > (self.timeout_seconds * 0.8):
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ Approaching timeout limit, triggering early fallback")
# Raise exception to trigger fallback to sequential processing
raise Exception("Parallel processing timeout, triggering fallback")
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Parallel processing completed, combining results...")
for anchors in results:
if elapsed_time > self.timeout_seconds:
self.logger.warning(f"🔍 ANCHOR SEARCH: ⏰ Timeout reached at n-gram {n}, stopping")
break
anchors = self._process_ngram_length(
n, trans_words, all_words, ref_texts_clean, ref_words, self.min_sources
)
candidate_anchors.extend(anchors)
# Explicitly cleanup pool to avoid hangs in containerized environments
self.logger.info(f"🔍 ANCHOR SEARCH: 🧹 Cleaning up pool...")
pool.close()
pool.terminate()
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Pool cleanup completed")
# Batch logging
batch_results.append((n, len(anchors)))
# Log progress every batch_size results or on the last result
if (i + 1) % batch_size == 0 or (i + 1) == len(n_gram_lengths):
total_anchors_in_batch = sum(anchor_count for _, anchor_count in batch_results)
n_gram_ranges = [str(ng) for ng, _ in batch_results]
range_str = f"{n_gram_ranges[0]}-{n_gram_ranges[-1]}" if len(n_gram_ranges) > 1 else n_gram_ranges[0]
self.logger.debug(f"🔍 ANCHOR SEARCH: Completed n-gram lengths {range_str} - found {total_anchors_in_batch} anchors")
batch_results = []
except AnchorSequenceTimeoutError:
self.logger.error(f"🔍 ANCHOR SEARCH: ❌ Parallel processing timed out")
# Re-raise timeout errors
raise
except Exception as e:
self.logger.error(f"🔍 ANCHOR SEARCH: ❌ Parallel processing failed: {str(e)}")
# Fall back to sequential processing with timeout checks
self.logger.info("🔍 ANCHOR SEARCH: 🔄 Falling back to sequential processing")
for n in n_gram_lengths:
try:
# Check timeout more leniently during sequential processing
if self.timeout_seconds > 0:
elapsed_time = time.time() - start_time
# Allow more time for sequential processing (up to 2x the original timeout)
if elapsed_time > (self.timeout_seconds * 2.0):
self.logger.warning(f"🔍 ANCHOR SEARCH: ⏰ Sequential processing timeout for n-gram {n}")
break
self.logger.info(f"🔍 ANCHOR SEARCH: 🔄 Sequential processing n-gram length {n}")
anchors = self._process_ngram_length(
n, trans_words, all_words, ref_texts_clean, ref_words, self.min_sources
)
candidate_anchors.extend(anchors)
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Sequential n-gram {n} completed - found {len(anchors)} anchors")
except Exception as e:
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ Sequential processing failed for n-gram length {n}: {str(e)}")
continue
finally:
# Always ensure pool is cleaned up to avoid hangs in containerized environments
if pool is not None:
try:
self.logger.info(f"🔍 ANCHOR SEARCH: 🧹 Final pool cleanup...")
pool.terminate()
pool.join(timeout=5) # Wait max 5 seconds for workers to terminate
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Final pool cleanup completed")
except Exception as cleanup_error:
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ Pool cleanup error (ignored): {cleanup_error}")
except Exception as e:
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ n-gram length {n} failed: {str(e)}")
batch_results.append((n, 0))
continue
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Found {len(candidate_anchors)} candidate anchors in {time.time() - start_time:.1f}s")
@@ -669,123 +593,75 @@ class AnchorSequenceFinder:
start_time = time.time()
# Try different pool sizes with timeout
num_processes = max(cpu_count() - 1, 1) # Leave one CPU free
self.logger.info(f"🔍 FILTERING: Using {num_processes} processes for scoring")
# Create a partial function with the context parameter fixed
score_anchor_partial = partial(self._score_anchor_static, context=context)
# Use multiprocessing to score anchors in parallel with timeout
# Score anchors sequentially using simple rule-based scoring
# (Avoids expensive spaCy NLP and works in cloud environments)
scored_anchors = []
pool_timeout = 300 # 5 minutes for scoring phase
scoring_pool = None
try:
self.logger.info(f"🔍 FILTERING: 🚀 Starting parallel scoring with timeout {pool_timeout}s")
scoring_pool = Pool(processes=num_processes)
# Submit scoring jobs with timeout
async_results = []
batch_size = 50
self.logger.info(f"🔍 FILTERING: Splitting {len(anchors)} anchors into batches of {batch_size}")
for i in range(0, len(anchors), batch_size):
batch = anchors[i:i + batch_size]
async_result = scoring_pool.apply_async(self._score_batch_static, (batch, context))
async_results.append(async_result)
self.logger.info(f"🔍 FILTERING: Submitted {len(async_results)} scoring batches")
# Collect results with timeout
for i, async_result in enumerate(async_results):
try:
self.logger.debug(f"🔍 FILTERING: ⏳ Collecting batch {i+1}/{len(async_results)}")
batch_results = async_result.get(timeout=pool_timeout)
scored_anchors.extend(batch_results)
self.logger.debug(f"🔍 FILTERING: ✅ Completed scoring batch {i+1}/{len(async_results)}")
except Exception as e:
self.logger.warning(f"🔍 FILTERING: ⚠️ Scoring batch {i+1} failed or timed out: {str(e)}")
# Add basic scores for failed batch
start_idx = i * batch_size
end_idx = min((i + 1) * batch_size, len(anchors))
for j in range(start_idx, end_idx):
if j < len(anchors):
try:
phrase_score = PhraseScore(
total_score=1.0,
natural_break_score=1.0,
phrase_type=PhraseType.COMPLETE
)
scored_anchors.append(ScoredAnchor(anchor=anchors[j], phrase_score=phrase_score))
except:
continue
# Explicitly cleanup pool to avoid hangs in containerized environments
self.logger.info(f"🔍 FILTERING: 🧹 Cleaning up scoring pool...")
scoring_pool.close()
scoring_pool.terminate()
self.logger.info(f"🔍 FILTERING: ✅ Scoring pool cleanup completed")
except Exception as e:
self.logger.warning(f"🔍 FILTERING: ❌ Parallel scoring failed: {str(e)}, falling back to basic scoring")
# Fall back to basic scoring
for anchor in anchors:
try:
phrase_score = PhraseScore(
total_score=1.0,
natural_break_score=1.0,
phrase_type=PhraseType.COMPLETE
)
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
except:
continue
finally:
# Always ensure scoring pool is cleaned up to avoid hangs
if scoring_pool is not None:
try:
self.logger.info(f"🔍 FILTERING: 🧹 Final scoring pool cleanup...")
scoring_pool.terminate()
scoring_pool.join(timeout=5) # Wait max 5 seconds for workers to terminate
self.logger.info(f"🔍 FILTERING: ✅ Final scoring pool cleanup completed")
except Exception as cleanup_error:
self.logger.warning(f"🔍 FILTERING: ⚠️ Scoring pool cleanup error (ignored): {cleanup_error}")
self.logger.debug(f"🔍 FILTERING: Scoring {len(anchors)} anchors sequentially")
parallel_time = time.time() - start_time
self.logger.info(f"🔍 FILTERING: ✅ Parallel scoring completed in {parallel_time:.2f}s, scored {len(scored_anchors)} anchors")
for i, anchor in enumerate(anchors):
try:
# Simple rule-based scoring based on anchor properties
phrase_score = self._simple_score_anchor(anchor)
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
except Exception as e:
# Fallback to default score on error
self.logger.debug(f"🔍 FILTERING: Scoring failed for anchor {i}: {e}")
phrase_score = PhraseScore(
phrase_type=PhraseType.COMPLETE,
natural_break_score=1.0,
length_score=1.0
)
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
# Sort and filter as before
self.logger.info(f"🔍 FILTERING: 🔄 Sorting anchors by priority...")
scoring_time = time.time() - start_time
self.logger.debug(f"🔍 FILTERING: Scoring completed in {scoring_time:.2f}s, scored {len(scored_anchors)} anchors")
# Sort anchors by priority (highest first)
self.logger.debug(f"🔍 FILTERING: Sorting anchors by priority...")
scored_anchors.sort(key=self._get_sequence_priority, reverse=True)
self.logger.info(f"🔍 FILTERING: ✅ Sorting completed")
self.logger.info(f"🔍 FILTERING: 🔄 Filtering {len(scored_anchors)} overlapping sequences")
# O(N) overlap filtering using covered positions set
self.logger.debug(f"🔍 FILTERING: Filtering {len(scored_anchors)} overlapping sequences")
filtered_scored = []
for i, scored_anchor in enumerate(scored_anchors):
# Check timeout every 100 anchors using our timeout mechanism (more lenient)
if i % 100 == 0 and i > 0:
# Only check timeout if we're significantly over the limit
if self.timeout_seconds > 0:
elapsed_time = time.time() - start_time
# Use a more lenient timeout for filtering (allow 50% more time)
if elapsed_time > (self.timeout_seconds * 1.5):
self.logger.warning(f"🔍 FILTERING: ⏰ Filtering timed out, returning {len(filtered_scored)} anchors out of {len(scored_anchors)}")
break
self.logger.debug(f"🔍 FILTERING: Progress: {i}/{len(scored_anchors)} processed, {len(filtered_scored)} kept")
covered_positions: Set[int] = set()
for scored_anchor in scored_anchors:
anchor = scored_anchor.anchor
start_pos = anchor.transcription_position
end_pos = start_pos + anchor.length
overlaps = False
for existing in filtered_scored:
if self._sequences_overlap(scored_anchor.anchor, existing.anchor):
overlaps = True
break
if not overlaps:
# Check if any position in this anchor's range is already covered
anchor_positions = set(range(start_pos, end_pos))
if not anchor_positions & covered_positions: # No overlap with covered
filtered_scored.append(scored_anchor)
covered_positions.update(anchor_positions)
self.logger.info(f"🔍 FILTERING: ✅ Filtering completed - kept {len(filtered_scored)} non-overlapping anchors out of {len(scored_anchors)}")
self.logger.debug(f"🔍 FILTERING: Kept {len(filtered_scored)} non-overlapping anchors out of {len(scored_anchors)}")
return filtered_scored
def _simple_score_anchor(self, anchor: AnchorSequence) -> PhraseScore:
"""
Simple rule-based scoring for anchors without expensive NLP.
Scoring criteria:
- Longer sequences are preferred (length_score)
- Sequences matching more reference sources are preferred (natural_break_score)
- All sequences treated as COMPLETE type for simplicity
"""
# Length score: normalize to 0-1 range (3-15 words typical)
length = anchor.length
length_score = min(1.0, (length - 2) / 10.0) # 3 words = 0.1, 12 words = 1.0
# Source match score: more sources = higher score
num_sources = len(anchor.reference_positions)
natural_break_score = min(1.0, num_sources / 3.0) # 1 source = 0.33, 3+ sources = 1.0
return PhraseScore(
phrase_type=PhraseType.COMPLETE,
natural_break_score=natural_break_score,
length_score=length_score
)
@staticmethod
def _score_anchor_static(anchor: AnchorSequence, context: str) -> ScoredAnchor:
"""Static version of _score_anchor for multiprocessing compatibility."""
@@ -816,9 +692,9 @@ class AnchorSequenceFinder:
except Exception:
# Add basic score for failed anchor
phrase_score = PhraseScore(
total_score=1.0,
phrase_type=PhraseType.COMPLETE,
natural_break_score=1.0,
phrase_type=PhraseType.COMPLETE
length_score=1.0
)
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
@@ -2,7 +2,7 @@
"name": "lyrics-transcriber-frontend",
"private": true,
"homepage": "https://nomadkaraoke.github.io/lyrics-transcriber-frontend",
"version": "0.80.0",
"version": "0.82.0",
"type": "module",
"scripts": {
"dev": "vite",
@@ -1,4 +1,4 @@
import { Box, Button, Typography, useMediaQuery, useTheme, Switch, FormControlLabel, Tooltip, Paper, IconButton } from '@mui/material'
import { Box, Button, Typography, useMediaQuery, useTheme, Switch, FormControlLabel, Tooltip, Paper, IconButton, Chip } from '@mui/material'
import LockIcon from '@mui/icons-material/Lock'
import UploadFileIcon from '@mui/icons-material/UploadFile'
import FindReplaceIcon from '@mui/icons-material/FindReplace'
@@ -7,6 +7,7 @@ import UndoIcon from '@mui/icons-material/Undo'
import RedoIcon from '@mui/icons-material/Redo'
import TimerIcon from '@mui/icons-material/Timer'
import RestoreIcon from '@mui/icons-material/Restore'
import RateReviewIcon from '@mui/icons-material/RateReview'
import { CorrectionData, InteractionMode } from '../types'
import CorrectionMetrics from './CorrectionMetrics'
import AgenticCorrectionMetrics from './AgenticCorrectionMetrics'
@@ -41,6 +42,8 @@ interface HeaderProps {
onRedo: () => void
canUndo: boolean
canRedo: boolean
annotationsEnabled?: boolean
onAnnotationsToggle?: (enabled: boolean) => void
}
export default function Header({
@@ -65,6 +68,8 @@ export default function Header({
onRedo,
canUndo,
canRedo,
annotationsEnabled = true,
onAnnotationsToggle,
}: HeaderProps) {
const theme = useTheme()
const isMobile = useMediaQuery(theme.breakpoints.down('md'))
@@ -135,17 +140,38 @@ export default function Header({
<Typography variant="h4" sx={{ fontSize: isMobile ? '1.3rem' : '1.5rem' }}>
Nomad Karaoke: Lyrics Transcription Review
</Typography>
{isReadOnly && (
<Button
variant="outlined"
size="small"
startIcon={<UploadFileIcon />}
onClick={onFileLoad}
fullWidth={isMobile}
>
Load File
</Button>
)}
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
{!isReadOnly && onAnnotationsToggle && (
<Tooltip title={annotationsEnabled
? "Click to disable annotation prompts when editing"
: "Click to enable annotation prompts when editing"
}>
<Chip
icon={<RateReviewIcon />}
label={annotationsEnabled ? "Feedback On" : "Feedback Off"}
onClick={() => onAnnotationsToggle(!annotationsEnabled)}
color={annotationsEnabled ? "primary" : "default"}
variant={annotationsEnabled ? "filled" : "outlined"}
size="small"
sx={{
cursor: 'pointer',
'& .MuiChip-icon': { fontSize: '1rem' }
}}
/>
</Tooltip>
)}
{isReadOnly && (
<Button
variant="outlined"
size="small"
startIcon={<UploadFileIcon />}
onClick={onFileLoad}
fullWidth={isMobile}
>
Load File
</Button>
)}
</Box>
</Box>
<Box sx={{
@@ -200,6 +200,8 @@ interface MemoizedHeaderProps {
canUndo: boolean
canRedo: boolean
onUnCorrectAll: () => void
annotationsEnabled: boolean
onAnnotationsToggle: (enabled: boolean) => void
}
// Create a memoized Header component
@@ -224,7 +226,9 @@ const MemoizedHeader = memo(function MemoizedHeader({
onRedo,
canUndo,
canRedo,
onUnCorrectAll
onUnCorrectAll,
annotationsEnabled,
onAnnotationsToggle
}: MemoizedHeaderProps) {
return (
<Header
@@ -249,6 +253,8 @@ const MemoizedHeader = memo(function MemoizedHeader({
canUndo={canUndo}
canRedo={canRedo}
onUnCorrectAll={onUnCorrectAll}
annotationsEnabled={annotationsEnabled}
onAnnotationsToggle={onAnnotationsToggle}
/>
);
});
@@ -295,13 +301,18 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
wordIdsAffected: string[]
gapId?: string
} | null>(null)
const [annotationsEnabled] = useState(() => {
const [annotationsEnabled, setAnnotationsEnabled] = useState(() => {
// Check localStorage for user preference
const saved = localStorage.getItem('annotationsEnabled')
return saved !== null ? saved === 'true' : true // Default: enabled
// TODO: Add UI toggle to enable/disable via setAnnotationsEnabled
})
// Persist annotation preference to localStorage
const handleAnnotationsToggle = useCallback((enabled: boolean) => {
setAnnotationsEnabled(enabled)
localStorage.setItem('annotationsEnabled', String(enabled))
}, [])
// Correction detail card state
const [correctionDetailOpen, setCorrectionDetailOpen] = useState(false)
const [selectedCorrection, setSelectedCorrection] = useState<{
@@ -1164,6 +1175,8 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
canUndo={canUndo}
canRedo={canRedo}
onUnCorrectAll={handleUnCorrectAll}
annotationsEnabled={annotationsEnabled}
onAnnotationsToggle={handleAnnotationsToggle}
/>
<Grid container direction={isMobile ? 'column' : 'row'}>
@@ -1313,6 +1326,7 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
onPlaySegment={handlePlaySegment}
currentTime={currentAudioTime}
setModalSpacebarHandler={handleSetModalSpacebarHandler}
existingSegments={data.corrected_segments}
/>
{pendingAnnotation && (
@@ -0,0 +1,185 @@
import { memo } from 'react'
import {
Box,
Button,
Divider,
Stack
} from '@mui/material'
import PlayCircleOutlineIcon from '@mui/icons-material/PlayCircleOutline'
import StopIcon from '@mui/icons-material/Stop'
import PauseIcon from '@mui/icons-material/Pause'
import PlayArrowIcon from '@mui/icons-material/PlayArrow'
import ClearAllIcon from '@mui/icons-material/ClearAll'
import EditNoteIcon from '@mui/icons-material/EditNote'
import BlockIcon from '@mui/icons-material/Block'
import EditIcon from '@mui/icons-material/Edit'
import DeleteIcon from '@mui/icons-material/Delete'
interface SyncControlsProps {
// Main buttons
isManualSyncing: boolean
isPaused: boolean
onStartSync: () => void
onPauseSync: () => void
onResumeSync: () => void
onClearSync: () => void
onEditLyrics: () => void
// Playback controls
onPlay: () => void
onStop: () => void
isPlaying: boolean
// Word action buttons
hasSelectedWords: boolean
selectedWordCount: number
onUnsyncFromCursor: () => void
onEditSelectedWord: () => void
onDeleteSelected: () => void
// Additional state
canUnsyncFromCursor: boolean
}
const SyncControls = memo(function SyncControls({
isManualSyncing,
isPaused,
onStartSync,
onPauseSync,
onResumeSync,
onClearSync,
onEditLyrics,
onPlay,
onStop,
isPlaying,
hasSelectedWords,
selectedWordCount,
onUnsyncFromCursor,
onEditSelectedWord,
onDeleteSelected,
canUnsyncFromCursor
}: SyncControlsProps) {
return (
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 1.5 }}>
{/* Row 1: Playback & Sync Mode Controls */}
<Stack direction="row" spacing={1} alignItems="center" flexWrap="wrap">
{/* Playback controls */}
<Button
variant="outlined"
color="primary"
onClick={onPlay}
startIcon={<PlayArrowIcon />}
size="small"
disabled={isPlaying}
>
Play
</Button>
<Button
variant="outlined"
color="error"
onClick={onStop}
startIcon={<StopIcon />}
size="small"
disabled={!isPlaying}
>
Stop
</Button>
<Divider orientation="vertical" flexItem sx={{ mx: 0.5 }} />
{/* Sync Mode controls */}
{isManualSyncing ? (
<>
<Button
variant="contained"
color="error"
onClick={onStartSync}
startIcon={<StopIcon />}
size="small"
>
Stop Sync
</Button>
<Button
variant="outlined"
color={isPaused ? 'success' : 'warning'}
onClick={isPaused ? onResumeSync : onPauseSync}
startIcon={isPaused ? <PlayArrowIcon /> : <PauseIcon />}
size="small"
>
{isPaused ? 'Resume' : 'Pause'}
</Button>
</>
) : (
<Button
variant="contained"
color="primary"
onClick={onStartSync}
startIcon={<PlayCircleOutlineIcon />}
size="small"
>
Start Sync
</Button>
)}
</Stack>
{/* Row 2: Editing & Word Actions */}
<Stack direction="row" spacing={1} alignItems="center" flexWrap="wrap">
<Button
variant="outlined"
color="warning"
onClick={onClearSync}
startIcon={<ClearAllIcon />}
size="small"
disabled={isManualSyncing && !isPaused}
>
Clear Sync
</Button>
<Button
variant="outlined"
onClick={onEditLyrics}
startIcon={<EditNoteIcon />}
size="small"
disabled={isManualSyncing && !isPaused}
>
Edit Lyrics
</Button>
<Divider orientation="vertical" flexItem sx={{ mx: 0.5 }} />
<Button
variant="outlined"
onClick={onUnsyncFromCursor}
startIcon={<BlockIcon />}
size="small"
disabled={!canUnsyncFromCursor || (isManualSyncing && !isPaused)}
>
Unsync from Cursor
</Button>
<Button
variant="outlined"
onClick={onEditSelectedWord}
startIcon={<EditIcon />}
size="small"
disabled={!hasSelectedWords || selectedWordCount !== 1}
>
Edit Word
</Button>
<Button
variant="outlined"
color="error"
onClick={onDeleteSelected}
startIcon={<DeleteIcon />}
size="small"
disabled={!hasSelectedWords || (isManualSyncing && !isPaused)}
>
Delete{hasSelectedWords && selectedWordCount > 0 ? ` (${selectedWordCount})` : ''}
</Button>
</Stack>
</Box>
)
})
export default SyncControls
@@ -0,0 +1,704 @@
import { useRef, useEffect, useCallback, useState, memo } from 'react'
import { Box, IconButton, Tooltip } from '@mui/material'
import ArrowBackIcon from '@mui/icons-material/ArrowBack'
import ArrowForwardIcon from '@mui/icons-material/ArrowForward'
import { Word, LyricsSegment } from '../../types'
interface TimelineCanvasProps {
words: Word[]
segments: LyricsSegment[]
visibleStartTime: number
visibleEndTime: number
currentTime: number
selectedWordIds: Set<string>
onWordClick: (wordId: string, event: React.MouseEvent) => void
onBackgroundClick: () => void
onTimeBarClick: (time: number) => void
onSelectionComplete: (wordIds: string[]) => void
onWordTimingChange: (wordId: string, newStartTime: number, newEndTime: number) => void
onWordsMove: (updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }>) => void
syncWordIndex: number
isManualSyncing: boolean
onScrollChange: (newStartTime: number) => void
audioDuration: number
zoomSeconds: number
height?: number
}
// Constants for rendering
const TIME_BAR_HEIGHT = 28
const WORD_BLOCK_HEIGHT = 24
const WORD_LEVEL_SPACING = 50
const CANVAS_PADDING = 8
const TEXT_ABOVE_BLOCK = 14
const RESIZE_HANDLE_SIZE = 8
const RESIZE_HANDLE_HITAREA = 12
const PLAYHEAD_COLOR = '#ffffff'
const WORD_BLOCK_COLOR = '#d32f2f'
const WORD_BLOCK_SELECTED_COLOR = '#b71c1c'
const WORD_BLOCK_CURRENT_COLOR = '#f44336'
const WORD_TEXT_CURRENT_COLOR = '#d32f2f'
const UPCOMING_WORD_BG = '#fff9c4'
const UPCOMING_WORD_TEXT = '#000000'
const TIME_BAR_BG = '#f5f5f5'
const TIME_BAR_TEXT = '#666666'
const TIMELINE_BG = '#e0e0e0'
// Drag modes
type DragMode = 'none' | 'selection' | 'resize' | 'move'
// Build a map of word ID to segment index
function buildWordToSegmentMap(segments: LyricsSegment[]): Map<string, number> {
const map = new Map<string, number>()
segments.forEach((segment, idx) => {
segment.words.forEach(word => {
map.set(word.id, idx)
})
})
return map
}
// Calculate which vertical level a word should be on
function calculateWordLevels(words: Word[], segments: LyricsSegment[]): Map<string, number> {
const levels = new Map<string, number>()
const wordToSegment = buildWordToSegmentMap(segments)
const segmentsWithTiming = segments
.map((segment, idx) => {
const timedWords = segment.words.filter(w => w.start_time !== null)
const minStart = timedWords.length > 0
? Math.min(...timedWords.map(w => w.start_time!))
: Infinity
return { idx, minStart }
})
.filter(s => s.minStart !== Infinity)
.sort((a, b) => a.minStart - b.minStart)
const segmentLevels = new Map<number, number>()
segmentsWithTiming.forEach(({ idx }, orderIndex) => {
segmentLevels.set(idx, orderIndex % 2)
})
for (const word of words) {
const segmentIdx = wordToSegment.get(word.id)
if (segmentIdx !== undefined && segmentLevels.has(segmentIdx)) {
levels.set(word.id, segmentLevels.get(segmentIdx)!)
} else {
levels.set(word.id, 0)
}
}
return levels
}
function formatTime(seconds: number): string {
const mins = Math.floor(seconds / 60)
const secs = Math.floor(seconds % 60)
return `${mins.toString().padStart(2, '0')}:${secs.toString().padStart(2, '0')}`
}
const TimelineCanvas = memo(function TimelineCanvas({
words,
segments,
visibleStartTime,
visibleEndTime,
currentTime,
selectedWordIds,
onWordClick,
onBackgroundClick,
onTimeBarClick,
onSelectionComplete,
onWordTimingChange,
onWordsMove,
syncWordIndex,
isManualSyncing,
onScrollChange,
audioDuration,
zoomSeconds,
height = 200
}: TimelineCanvasProps) {
const canvasRef = useRef<HTMLCanvasElement>(null)
const containerRef = useRef<HTMLDivElement>(null)
const [canvasWidth, setCanvasWidth] = useState(800)
const animationFrameRef = useRef<number>()
const wordLevelsRef = useRef<Map<string, number>>(new Map())
// Drag state
const [dragMode, setDragMode] = useState<DragMode>('none')
const dragStartRef = useRef<{ x: number; y: number; time: number } | null>(null)
const dragWordIdRef = useRef<string | null>(null)
const dragOriginalTimesRef = useRef<Map<string, { start: number; end: number }>>(new Map())
// Selection rectangle
const [selectionRect, setSelectionRect] = useState<{
startX: number; startY: number; endX: number; endY: number
} | null>(null)
// Hover state for showing resize handle
const [hoveredWordId, setHoveredWordId] = useState<string | null>(null)
const [cursorStyle, setCursorStyle] = useState<string>('default')
// Update canvas width on resize
useEffect(() => {
const updateWidth = () => {
if (containerRef.current) {
setCanvasWidth(containerRef.current.clientWidth)
}
}
updateWidth()
const resizeObserver = new ResizeObserver(updateWidth)
if (containerRef.current) {
resizeObserver.observe(containerRef.current)
}
return () => resizeObserver.disconnect()
}, [])
// Calculate word levels when words or segments change
useEffect(() => {
wordLevelsRef.current = calculateWordLevels(words, segments)
}, [words, segments])
// Convert time to x position
const timeToX = useCallback((time: number): number => {
const duration = visibleEndTime - visibleStartTime
if (duration <= 0) return 0
return CANVAS_PADDING + ((time - visibleStartTime) / duration) * (canvasWidth - CANVAS_PADDING * 2)
}, [visibleStartTime, visibleEndTime, canvasWidth])
// Convert x position to time
const xToTime = useCallback((x: number): number => {
const duration = visibleEndTime - visibleStartTime
return visibleStartTime + ((x - CANVAS_PADDING) / (canvasWidth - CANVAS_PADDING * 2)) * duration
}, [visibleStartTime, visibleEndTime, canvasWidth])
// Get word bounds
const getWordBounds = useCallback((word: Word) => {
if (word.start_time === null || word.end_time === null) return null
const level = wordLevelsRef.current.get(word.id) || 0
const startX = timeToX(word.start_time)
const endX = timeToX(word.end_time)
const blockWidth = Math.max(endX - startX, 4)
const y = TIME_BAR_HEIGHT + CANVAS_PADDING + TEXT_ABOVE_BLOCK + level * WORD_LEVEL_SPACING
return { startX, endX, blockWidth, y, level }
}, [timeToX])
// Check if position is near resize handle
const isNearResizeHandlePos = useCallback((word: Word, x: number, y: number): boolean => {
const bounds = getWordBounds(word)
if (!bounds) return false
const handleX = bounds.startX + bounds.blockWidth - RESIZE_HANDLE_SIZE / 2
const handleY = bounds.y + WORD_BLOCK_HEIGHT / 2
return Math.abs(x - handleX) < RESIZE_HANDLE_HITAREA / 2 &&
Math.abs(y - handleY) < RESIZE_HANDLE_HITAREA / 2
}, [getWordBounds])
// Find word at position
const findWordAtPosition = useCallback((x: number, y: number): Word | null => {
for (const word of words) {
const bounds = getWordBounds(word)
if (!bounds) continue
if (x >= bounds.startX && x <= bounds.startX + bounds.blockWidth &&
y >= bounds.y && y <= bounds.y + WORD_BLOCK_HEIGHT) {
return word
}
}
return null
}, [words, getWordBounds])
// Find words in selection rectangle
const findWordsInRect = useCallback((rect: { startX: number; startY: number; endX: number; endY: number }): string[] => {
const rectLeft = Math.min(rect.startX, rect.endX)
const rectRight = Math.max(rect.startX, rect.endX)
const rectTop = Math.min(rect.startY, rect.endY)
const rectBottom = Math.max(rect.startY, rect.endY)
const selectedIds: string[] = []
for (const word of words) {
const bounds = getWordBounds(word)
if (!bounds) continue
if (bounds.startX + bounds.blockWidth >= rectLeft && bounds.startX <= rectRight &&
bounds.y + WORD_BLOCK_HEIGHT >= rectTop && bounds.y <= rectBottom) {
selectedIds.push(word.id)
}
}
return selectedIds
}, [words, getWordBounds])
// Draw the timeline
const draw = useCallback(() => {
const canvas = canvasRef.current
if (!canvas) return
const ctx = canvas.getContext('2d')
if (!ctx) return
const dpr = window.devicePixelRatio || 1
canvas.width = canvasWidth * dpr
canvas.height = height * dpr
ctx.scale(dpr, dpr)
// Clear canvas
ctx.fillStyle = TIMELINE_BG
ctx.fillRect(0, 0, canvasWidth, height)
// Draw time bar background
ctx.fillStyle = TIME_BAR_BG
ctx.fillRect(0, 0, canvasWidth, TIME_BAR_HEIGHT)
// Draw time markers
const duration = visibleEndTime - visibleStartTime
const secondsPerTick = duration > 15 ? 2 : duration > 8 ? 1 : 0.5
const startSecond = Math.ceil(visibleStartTime / secondsPerTick) * secondsPerTick
ctx.fillStyle = TIME_BAR_TEXT
ctx.font = '11px system-ui, -apple-system, sans-serif'
ctx.textAlign = 'center'
for (let t = startSecond; t <= visibleEndTime; t += secondsPerTick) {
const x = timeToX(t)
ctx.beginPath()
ctx.strokeStyle = '#999999'
ctx.lineWidth = 1
ctx.moveTo(x, TIME_BAR_HEIGHT - 6)
ctx.lineTo(x, TIME_BAR_HEIGHT)
ctx.stroke()
if (t % 1 === 0) {
ctx.fillText(formatTime(t), x, TIME_BAR_HEIGHT - 10)
}
}
ctx.beginPath()
ctx.strokeStyle = '#cccccc'
ctx.lineWidth = 1
ctx.moveTo(0, TIME_BAR_HEIGHT)
ctx.lineTo(canvasWidth, TIME_BAR_HEIGHT)
ctx.stroke()
const wordToSegment = buildWordToSegmentMap(segments)
const syncedWords = words.filter(w => w.start_time !== null && w.end_time !== null)
const currentWordId = syncedWords.find(w =>
currentTime >= w.start_time! && currentTime <= w.end_time!
)?.id || null
// First pass: draw all blocks
for (const word of syncedWords) {
const bounds = getWordBounds(word)
if (!bounds) continue
const isSelected = selectedWordIds.has(word.id)
const isCurrent = word.id === currentWordId
const isHovered = word.id === hoveredWordId
// Draw word block background
if (isSelected) {
ctx.fillStyle = WORD_BLOCK_SELECTED_COLOR
} else if (isCurrent) {
ctx.fillStyle = WORD_BLOCK_CURRENT_COLOR
} else {
ctx.fillStyle = WORD_BLOCK_COLOR
}
ctx.fillRect(bounds.startX, bounds.y, bounds.blockWidth, WORD_BLOCK_HEIGHT)
// Draw selection border
if (isSelected) {
ctx.strokeStyle = '#ffffff'
ctx.lineWidth = 2
ctx.strokeRect(bounds.startX, bounds.y, bounds.blockWidth, WORD_BLOCK_HEIGHT)
// Draw resize handle (white dot on right edge) for selected words when hovered
if (isHovered || selectedWordIds.size === 1) {
const handleX = bounds.startX + bounds.blockWidth - RESIZE_HANDLE_SIZE / 2
const handleY = bounds.y + WORD_BLOCK_HEIGHT / 2
ctx.beginPath()
ctx.fillStyle = '#ffffff'
ctx.arc(handleX, handleY, RESIZE_HANDLE_SIZE / 2, 0, Math.PI * 2)
ctx.fill()
ctx.strokeStyle = '#666666'
ctx.lineWidth = 1
ctx.stroke()
}
}
}
// Second pass: draw text
const wordsBySegment = new Map<number, Word[]>()
for (const word of syncedWords) {
const segIdx = wordToSegment.get(word.id)
if (segIdx !== undefined) {
if (!wordsBySegment.has(segIdx)) {
wordsBySegment.set(segIdx, [])
}
wordsBySegment.get(segIdx)!.push(word)
}
}
ctx.font = '11px system-ui, -apple-system, sans-serif'
ctx.textAlign = 'left'
for (const [, segmentWords] of wordsBySegment) {
const sortedWords = [...segmentWords].sort((a, b) =>
(a.start_time || 0) - (b.start_time || 0)
)
if (sortedWords.length === 0) continue
const level = wordLevelsRef.current.get(sortedWords[0].id) || 0
const textY = TIME_BAR_HEIGHT + CANVAS_PADDING + TEXT_ABOVE_BLOCK + level * WORD_LEVEL_SPACING - 3
let rightmostTextEnd = -Infinity
for (const word of sortedWords) {
const blockStartX = timeToX(word.start_time!)
const textWidth = ctx.measureText(word.text).width
const textStartX = Math.max(blockStartX, rightmostTextEnd + 3)
if (textStartX < canvasWidth - 10) {
const isCurrent = word.id === currentWordId
ctx.fillStyle = isCurrent ? WORD_TEXT_CURRENT_COLOR : '#333333'
ctx.fillText(word.text, textStartX, textY)
rightmostTextEnd = textStartX + textWidth
}
}
}
// Draw upcoming words during sync
if (isManualSyncing && syncWordIndex >= 0) {
const upcomingWords = words.slice(syncWordIndex).filter(w => w.start_time === null)
const playheadX = timeToX(currentTime)
let offsetX = playheadX + 10
ctx.font = '11px system-ui, -apple-system, sans-serif'
for (let i = 0; i < Math.min(upcomingWords.length, 12); i++) {
const word = upcomingWords[i]
const textWidth = ctx.measureText(word.text).width + 10
ctx.fillStyle = UPCOMING_WORD_BG
ctx.fillRect(offsetX, TIME_BAR_HEIGHT + CANVAS_PADDING + WORD_LEVEL_SPACING + 60, textWidth, 20)
ctx.fillStyle = UPCOMING_WORD_TEXT
ctx.textAlign = 'left'
ctx.fillText(word.text, offsetX + 5, TIME_BAR_HEIGHT + CANVAS_PADDING + WORD_LEVEL_SPACING + 74)
offsetX += textWidth + 3
if (offsetX > canvasWidth - 20) break
}
}
// Draw playhead
if (currentTime >= visibleStartTime && currentTime <= visibleEndTime) {
const playheadX = timeToX(currentTime)
ctx.beginPath()
ctx.fillStyle = PLAYHEAD_COLOR
ctx.strokeStyle = '#333333'
ctx.lineWidth = 1
ctx.moveTo(playheadX - 6, 2)
ctx.lineTo(playheadX + 6, 2)
ctx.lineTo(playheadX, TIME_BAR_HEIGHT - 4)
ctx.closePath()
ctx.fill()
ctx.stroke()
ctx.beginPath()
ctx.strokeStyle = PLAYHEAD_COLOR
ctx.lineWidth = 2
ctx.moveTo(playheadX, TIME_BAR_HEIGHT)
ctx.lineTo(playheadX, height)
ctx.stroke()
ctx.beginPath()
ctx.strokeStyle = 'rgba(0,0,0,0.4)'
ctx.lineWidth = 1
ctx.moveTo(playheadX + 1, TIME_BAR_HEIGHT)
ctx.lineTo(playheadX + 1, height)
ctx.stroke()
}
// Draw selection rectangle
if (selectionRect) {
ctx.fillStyle = 'rgba(25, 118, 210, 0.2)'
ctx.strokeStyle = 'rgba(25, 118, 210, 0.8)'
ctx.lineWidth = 1
const rectX = Math.min(selectionRect.startX, selectionRect.endX)
const rectY = Math.min(selectionRect.startY, selectionRect.endY)
const rectW = Math.abs(selectionRect.endX - selectionRect.startX)
const rectH = Math.abs(selectionRect.endY - selectionRect.startY)
ctx.fillRect(rectX, rectY, rectW, rectH)
ctx.strokeRect(rectX, rectY, rectW, rectH)
}
}, [
canvasWidth, height, visibleStartTime, visibleEndTime, currentTime,
words, segments, selectedWordIds, selectionRect, hoveredWordId,
syncWordIndex, isManualSyncing, timeToX, getWordBounds
])
// Animation frame
useEffect(() => {
const animate = () => {
draw()
animationFrameRef.current = requestAnimationFrame(animate)
}
animate()
return () => {
if (animationFrameRef.current) {
cancelAnimationFrame(animationFrameRef.current)
}
}
}, [draw])
// Mouse handlers
const handleMouseDown = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const rect = canvasRef.current?.getBoundingClientRect()
if (!rect) return
const x = e.clientX - rect.left
const y = e.clientY - rect.top
const time = xToTime(x)
// Time bar click
if (y < TIME_BAR_HEIGHT) {
onTimeBarClick(Math.max(0, time))
return
}
const clickedWord = findWordAtPosition(x, y)
if (clickedWord && selectedWordIds.has(clickedWord.id)) {
// Check if clicking on resize handle
if (isNearResizeHandlePos(clickedWord, x, y)) {
// Start resize
setDragMode('resize')
dragStartRef.current = { x, y, time }
dragWordIdRef.current = clickedWord.id
dragOriginalTimesRef.current = new Map([[clickedWord.id, {
start: clickedWord.start_time!,
end: clickedWord.end_time!
}]])
return
}
// Start move (for all selected words)
setDragMode('move')
dragStartRef.current = { x, y, time }
dragWordIdRef.current = clickedWord.id
// Store original times for all selected words
const originalTimes = new Map<string, { start: number; end: number }>()
for (const wordId of selectedWordIds) {
const word = words.find(w => w.id === wordId)
if (word && word.start_time !== null && word.end_time !== null) {
originalTimes.set(wordId, { start: word.start_time, end: word.end_time })
}
}
dragOriginalTimesRef.current = originalTimes
return
}
if (clickedWord) {
// Click on unselected word - select it
onWordClick(clickedWord.id, e)
return
}
// Background - start selection
setDragMode('selection')
dragStartRef.current = { x, y, time }
setSelectionRect({ startX: x, startY: y, endX: x, endY: y })
}, [xToTime, onTimeBarClick, findWordAtPosition, selectedWordIds, isNearResizeHandlePos, onWordClick, words])
const handleMouseMove = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const rect = canvasRef.current?.getBoundingClientRect()
if (!rect) return
const x = e.clientX - rect.left
const y = e.clientY - rect.top
const time = xToTime(x)
// Update hover state and cursor
if (dragMode === 'none') {
const hoveredWord = findWordAtPosition(x, y)
setHoveredWordId(hoveredWord?.id || null)
if (hoveredWord && selectedWordIds.has(hoveredWord.id)) {
const nearHandle = isNearResizeHandlePos(hoveredWord, x, y)
setCursorStyle(nearHandle ? 'ew-resize' : 'grab')
} else if (hoveredWord) {
setCursorStyle('pointer')
} else if (y < TIME_BAR_HEIGHT) {
setCursorStyle('pointer')
} else {
setCursorStyle('default')
}
}
if (!dragStartRef.current) return
if (dragMode === 'selection') {
setSelectionRect({
startX: dragStartRef.current.x,
startY: dragStartRef.current.y,
endX: x,
endY: y
})
} else if (dragMode === 'resize' && dragWordIdRef.current) {
// Resize the word
const originalTimes = dragOriginalTimesRef.current.get(dragWordIdRef.current)
if (originalTimes) {
const deltaTime = time - dragStartRef.current.time
const newEndTime = Math.max(originalTimes.start + 0.05, originalTimes.end + deltaTime)
onWordTimingChange(dragWordIdRef.current, originalTimes.start, newEndTime)
}
setCursorStyle('ew-resize')
} else if (dragMode === 'move') {
// Move all selected words
const deltaTime = time - dragStartRef.current.time
const updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }> = []
for (const [wordId, originalTimes] of dragOriginalTimesRef.current) {
// Ensure end time is always after start time (at least 0.05s duration)
const newStartTime = Math.max(0, originalTimes.start + deltaTime)
const newEndTime = Math.max(newStartTime + 0.05, originalTimes.end + deltaTime)
updates.push({
wordId,
newStartTime,
newEndTime
})
}
if (updates.length > 0) {
onWordsMove(updates)
}
setCursorStyle('grabbing')
}
}, [dragMode, xToTime, findWordAtPosition, selectedWordIds, isNearResizeHandlePos, onWordTimingChange, onWordsMove])
const handleMouseUp = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const rect = canvasRef.current?.getBoundingClientRect()
if (dragMode === 'selection' && dragStartRef.current && rect) {
const endX = e.clientX - rect.left
const endY = e.clientY - rect.top
const dragDistance = Math.sqrt(
Math.pow(endX - dragStartRef.current.x, 2) +
Math.pow(endY - dragStartRef.current.y, 2)
)
if (dragDistance < 5) {
onBackgroundClick()
} else {
const finalRect = {
startX: dragStartRef.current.x,
startY: dragStartRef.current.y,
endX,
endY
}
const selectedIds = findWordsInRect(finalRect)
if (selectedIds.length > 0) {
onSelectionComplete(selectedIds)
}
}
}
// Reset drag state
setDragMode('none')
dragStartRef.current = null
dragWordIdRef.current = null
dragOriginalTimesRef.current = new Map()
setSelectionRect(null)
setCursorStyle('default')
}, [dragMode, onBackgroundClick, findWordsInRect, onSelectionComplete])
// Wheel handler
const handleWheel = useCallback((e: React.WheelEvent<HTMLCanvasElement>) => {
const delta = e.deltaX !== 0 ? e.deltaX : e.deltaY
const scrollAmount = (delta / 100) * (zoomSeconds / 4)
let newStart = Math.max(0, Math.min(audioDuration - zoomSeconds, visibleStartTime + scrollAmount))
if (newStart !== visibleStartTime) {
onScrollChange(newStart)
}
}, [visibleStartTime, zoomSeconds, audioDuration, onScrollChange])
const handleScrollLeft = useCallback(() => {
const newStart = Math.max(0, visibleStartTime - zoomSeconds * 0.25)
onScrollChange(newStart)
}, [visibleStartTime, zoomSeconds, onScrollChange])
const handleScrollRight = useCallback(() => {
const newStart = Math.min(audioDuration - zoomSeconds, visibleStartTime + zoomSeconds * 0.25)
onScrollChange(Math.max(0, newStart))
}, [visibleStartTime, zoomSeconds, audioDuration, onScrollChange])
return (
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 0.5 }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Tooltip title="Scroll Left">
<IconButton
size="small"
onClick={handleScrollLeft}
disabled={visibleStartTime <= 0}
>
<ArrowBackIcon fontSize="small" />
</IconButton>
</Tooltip>
<Box
ref={containerRef}
sx={{
flexGrow: 1,
height,
cursor: cursorStyle,
borderRadius: 1,
overflow: 'hidden'
}}
>
<canvas
ref={canvasRef}
style={{
width: '100%',
height: '100%',
display: 'block',
cursor: cursorStyle
}}
onMouseDown={handleMouseDown}
onMouseMove={handleMouseMove}
onMouseUp={handleMouseUp}
onMouseLeave={handleMouseUp}
onWheel={handleWheel}
/>
</Box>
<Tooltip title="Scroll Right">
<IconButton
size="small"
onClick={handleScrollRight}
disabled={visibleStartTime >= audioDuration - zoomSeconds}
>
<ArrowForwardIcon fontSize="small" />
</IconButton>
</Tooltip>
</Box>
</Box>
)
})
export default TimelineCanvas
@@ -0,0 +1,80 @@
import { memo, useMemo } from 'react'
import { Box, Typography } from '@mui/material'
import { Word } from '../../types'
interface UpcomingWordsBarProps {
words: Word[]
syncWordIndex: number
isManualSyncing: boolean
maxWordsToShow?: number
}
const UpcomingWordsBar = memo(function UpcomingWordsBar({
words,
syncWordIndex,
isManualSyncing,
maxWordsToShow = 20
}: UpcomingWordsBarProps) {
// Get upcoming unsynced words
const upcomingWords = useMemo(() => {
if (!isManualSyncing || syncWordIndex < 0) return []
return words
.slice(syncWordIndex)
.filter(w => w.start_time === null)
.slice(0, maxWordsToShow)
}, [words, syncWordIndex, isManualSyncing, maxWordsToShow])
const totalRemaining = useMemo(() => {
if (!isManualSyncing || syncWordIndex < 0) return 0
return words.slice(syncWordIndex).filter(w => w.start_time === null).length
}, [words, syncWordIndex, isManualSyncing])
// Show empty placeholder when no upcoming words
if (upcomingWords.length === 0) {
return null
}
return (
<Box sx={{
height: 44,
bgcolor: 'grey.100',
borderRadius: 1,
display: 'flex',
alignItems: 'center',
px: 1,
gap: 0.5,
overflow: 'hidden',
boxSizing: 'border-box'
}}>
{upcomingWords.map((word, index) => (
<Box
key={word.id}
sx={{
px: 1,
py: 0.5,
borderRadius: 0.5,
bgcolor: index === 0 ? 'error.main' : 'grey.300',
color: index === 0 ? 'white' : 'text.primary',
fontWeight: index === 0 ? 'bold' : 'normal',
fontSize: '13px',
fontFamily: 'system-ui, -apple-system, sans-serif',
whiteSpace: 'nowrap',
border: index === 0 ? '2px solid' : 'none',
borderColor: 'error.dark'
}}
>
{word.text}
</Box>
))}
{totalRemaining > maxWordsToShow && (
<Typography variant="caption" color="text.secondary" sx={{ ml: 1 }}>
+{totalRemaining - maxWordsToShow} more
</Typography>
)}
</Box>
)
})
export default UpcomingWordsBar
@@ -0,0 +1,905 @@
import { useState, useCallback, useEffect, useRef, useMemo, memo } from 'react'
import {
Box,
Typography,
Slider,
Dialog,
DialogTitle,
DialogContent,
DialogActions,
TextField,
Button,
Paper,
Alert
} from '@mui/material'
import ZoomInIcon from '@mui/icons-material/ZoomIn'
import ZoomOutIcon from '@mui/icons-material/ZoomOut'
import { Word, LyricsSegment } from '../../types'
import TimelineCanvas from './TimelineCanvas'
import UpcomingWordsBar from './UpcomingWordsBar'
import SyncControls from './SyncControls'
// Augment window type for audio functions
declare global {
interface Window {
getAudioDuration?: () => number
toggleAudioPlayback?: () => void
isAudioPlaying?: boolean
}
}
interface LyricsSynchronizerProps {
segments: LyricsSegment[]
currentTime: number
onPlaySegment?: (startTime: number) => void
onSave: (segments: LyricsSegment[]) => void
onCancel: () => void
setModalSpacebarHandler: (handler: (() => (e: KeyboardEvent) => void) | undefined) => void
}
// Constants for zoom
const MIN_ZOOM_SECONDS = 4.5 // Most zoomed in - 4.5 seconds visible
const MAX_ZOOM_SECONDS = 24 // Most zoomed out - 24 seconds visible
const ZOOM_STEPS = 50 // Number of zoom levels
// Get all words from segments
function getAllWords(segments: LyricsSegment[]): Word[] {
return segments.flatMap(s => s.words)
}
// Deep clone segments
function cloneSegments(segments: LyricsSegment[]): LyricsSegment[] {
return JSON.parse(JSON.stringify(segments))
}
const LyricsSynchronizer = memo(function LyricsSynchronizer({
segments: initialSegments,
currentTime,
onPlaySegment,
onSave,
onCancel,
setModalSpacebarHandler
}: LyricsSynchronizerProps) {
// Working copy of segments
const [workingSegments, setWorkingSegments] = useState<LyricsSegment[]>(() =>
cloneSegments(initialSegments)
)
// Get all words flattened
const allWords = useMemo(() => getAllWords(workingSegments), [workingSegments])
// Audio duration
const audioDuration = useMemo(() => {
if (typeof window.getAudioDuration === 'function') {
const duration = window.getAudioDuration()
return duration > 0 ? duration : 300
}
return 300 // 5 minute fallback
}, [])
// Zoom state (value is the visible time window in seconds)
const [zoomSeconds, setZoomSeconds] = useState(12) // Default ~12 seconds visible
// Visible time range
const [visibleStartTime, setVisibleStartTime] = useState(0)
const visibleEndTime = useMemo(() =>
Math.min(visibleStartTime + zoomSeconds, audioDuration),
[visibleStartTime, zoomSeconds, audioDuration]
)
// Manual sync state
const [isManualSyncing, setIsManualSyncing] = useState(false)
const [isPaused, setIsPaused] = useState(false)
const [syncWordIndex, setSyncWordIndex] = useState(-1)
const [isSpacebarPressed, setIsSpacebarPressed] = useState(false)
const wordStartTimeRef = useRef<number | null>(null)
const spacebarPressTimeRef = useRef<number | null>(null)
const currentTimeRef = useRef(currentTime)
// Selection state
const [selectedWordIds, setSelectedWordIds] = useState<Set<string>>(new Set())
// Edit lyrics modal state
const [showEditLyricsModal, setShowEditLyricsModal] = useState(false)
const [editLyricsText, setEditLyricsText] = useState('')
// Edit word modal state
const [showEditWordModal, setShowEditWordModal] = useState(false)
const [editWordText, setEditWordText] = useState('')
const [editWordId, setEditWordId] = useState<string | null>(null)
// Keep currentTimeRef up to date
useEffect(() => {
currentTimeRef.current = currentTime
}, [currentTime])
// Auto-scroll to follow playhead during sync
useEffect(() => {
if (isManualSyncing && !isPaused && currentTime > 0) {
// If playhead is near the end of visible area, scroll forward
if (currentTime > visibleEndTime - (zoomSeconds * 0.1)) {
const newStart = Math.max(0, currentTime - zoomSeconds * 0.1)
setVisibleStartTime(newStart)
}
// If playhead is before visible area, scroll back
else if (currentTime < visibleStartTime) {
setVisibleStartTime(Math.max(0, currentTime - 1))
}
}
}, [currentTime, isManualSyncing, isPaused, visibleStartTime, visibleEndTime, zoomSeconds])
// Handle zoom slider change
const handleZoomChange = useCallback((_: Event, value: number | number[]) => {
const zoomValue = value as number
// Map slider value (0-50) to zoom range (4.5-24 seconds)
const newZoomSeconds = MIN_ZOOM_SECONDS + (zoomValue / ZOOM_STEPS) * (MAX_ZOOM_SECONDS - MIN_ZOOM_SECONDS)
setZoomSeconds(newZoomSeconds)
}, [])
// Get slider value from zoom seconds
const sliderValue = useMemo(() => {
return ((zoomSeconds - MIN_ZOOM_SECONDS) / (MAX_ZOOM_SECONDS - MIN_ZOOM_SECONDS)) * ZOOM_STEPS
}, [zoomSeconds])
// Handle scroll change from timeline
const handleScrollChange = useCallback((newStartTime: number) => {
setVisibleStartTime(newStartTime)
}, [])
// Update words in segments
const updateWords = useCallback((newWords: Word[]) => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
// Create a map of word id to word for quick lookup
const wordMap = new Map(newWords.map(w => [w.id, w]))
// Update each segment's words
for (const segment of newSegments) {
segment.words = segment.words.map(w => wordMap.get(w.id) || w)
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
} else {
segment.start_time = null
segment.end_time = null
}
}
return newSegments
})
}, [])
// Check if audio is playing
const [isPlaying, setIsPlaying] = useState(false)
// Update isPlaying state periodically
useEffect(() => {
const checkPlaying = () => {
setIsPlaying(typeof window.isAudioPlaying === 'boolean' ? window.isAudioPlaying : false)
}
checkPlaying()
const interval = setInterval(checkPlaying, 100)
return () => clearInterval(interval)
}, [])
// Play audio from current position
const handlePlayAudio = useCallback(() => {
if (onPlaySegment) {
onPlaySegment(currentTimeRef.current)
}
}, [onPlaySegment])
// Stop audio playback - also exits sync mode
const handleStopAudio = useCallback(() => {
if (typeof window.toggleAudioPlayback === 'function' && window.isAudioPlaying) {
window.toggleAudioPlayback()
}
// Also exit sync mode when stopping
if (isManualSyncing) {
setIsManualSyncing(false)
setIsPaused(false)
setIsSpacebarPressed(false)
}
}, [isManualSyncing])
// Start manual sync
const handleStartSync = useCallback(() => {
if (isManualSyncing) {
// Stop sync
setIsManualSyncing(false)
setIsPaused(false)
setSyncWordIndex(-1)
setIsSpacebarPressed(false)
// Stop audio
handleStopAudio()
return
}
// Find first unsynced word
const firstUnsyncedIndex = allWords.findIndex(w =>
w.start_time === null || w.end_time === null
)
const startIndex = firstUnsyncedIndex !== -1 ? firstUnsyncedIndex : 0
setIsManualSyncing(true)
setIsPaused(false)
setSyncWordIndex(startIndex)
setIsSpacebarPressed(false)
// Start playback
if (onPlaySegment) {
onPlaySegment(Math.max(0, currentTimeRef.current - 1))
}
}, [isManualSyncing, allWords, onPlaySegment, handleStopAudio])
// Pause sync
const handlePauseSync = useCallback(() => {
setIsPaused(true)
handleStopAudio()
}, [handleStopAudio])
// Resume sync
const handleResumeSync = useCallback(() => {
setIsPaused(false)
// Find first unsynced word from current position
const firstUnsyncedIndex = allWords.findIndex(w =>
w.start_time === null || w.end_time === null
)
if (firstUnsyncedIndex !== -1 && firstUnsyncedIndex !== syncWordIndex) {
setSyncWordIndex(firstUnsyncedIndex)
}
// Resume playback
if (onPlaySegment) {
onPlaySegment(currentTimeRef.current)
}
}, [allWords, syncWordIndex, onPlaySegment])
// Clear all sync data
const handleClearSync = useCallback(() => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
for (const word of segment.words) {
word.start_time = null
word.end_time = null
}
segment.start_time = null
segment.end_time = null
}
return newSegments
})
setSyncWordIndex(-1)
}, [])
// Unsync from cursor position
const handleUnsyncFromCursor = useCallback(() => {
const cursorTime = currentTimeRef.current
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
for (const word of segment.words) {
// Reset words that start after cursor position
if (word.start_time !== null && word.start_time > cursorTime) {
word.start_time = null
word.end_time = null
}
}
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
} else {
segment.start_time = null
segment.end_time = null
}
}
return newSegments
})
}, [])
// Check if there are words after cursor that can be unsynced
const canUnsyncFromCursor = useMemo(() => {
const cursorTime = currentTimeRef.current
return allWords.some(w =>
w.start_time !== null && w.start_time > cursorTime
)
}, [allWords, currentTime])
// Open edit lyrics modal
const handleEditLyrics = useCallback(() => {
const text = workingSegments.map(s => s.text).join('\n')
setEditLyricsText(text)
setShowEditLyricsModal(true)
}, [workingSegments])
// Save edited lyrics (warning: resets timing)
const handleSaveEditedLyrics = useCallback(() => {
const lines = editLyricsText.split('\n').filter(l => l.trim())
const newSegments: LyricsSegment[] = lines.map((line, idx) => {
const words = line.trim().split(/\s+/).map((text, wIdx) => ({
id: `word-${idx}-${wIdx}-${Date.now()}`,
text,
start_time: null,
end_time: null,
confidence: 1.0
}))
return {
id: `segment-${idx}-${Date.now()}`,
text: line.trim(),
words,
start_time: null,
end_time: null
}
})
setWorkingSegments(newSegments)
setShowEditLyricsModal(false)
setSyncWordIndex(-1)
}, [editLyricsText])
// Open edit word modal
const handleEditSelectedWord = useCallback(() => {
if (selectedWordIds.size !== 1) return
const wordId = Array.from(selectedWordIds)[0]
const word = allWords.find(w => w.id === wordId)
if (word) {
setEditWordId(wordId)
setEditWordText(word.text)
setShowEditWordModal(true)
}
}, [selectedWordIds, allWords])
// Save edited word
const handleSaveEditedWord = useCallback(() => {
if (!editWordId) return
const newText = editWordText.trim()
if (!newText) return
// Check if we're splitting into multiple words
const newWords = newText.split(/\s+/)
if (newWords.length === 1) {
// Simple rename
const updatedWords = allWords.map(w =>
w.id === editWordId ? { ...w, text: newWords[0] } : w
)
updateWords(updatedWords)
} else {
// Split word - preserve timing for first word, null for rest
const originalWord = allWords.find(w => w.id === editWordId)
if (!originalWord) return
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
const wordIndex = segment.words.findIndex(w => w.id === editWordId)
if (wordIndex !== -1) {
const newWordObjects = newWords.map((text, idx) => ({
id: idx === 0 ? editWordId : `${editWordId}-split-${idx}`,
text,
start_time: idx === 0 ? originalWord.start_time : null,
end_time: idx === 0 ? originalWord.end_time : null,
confidence: 1.0
}))
segment.words.splice(wordIndex, 1, ...newWordObjects)
segment.text = segment.words.map(w => w.text).join(' ')
break
}
}
return newSegments
})
}
setShowEditWordModal(false)
setEditWordId(null)
setEditWordText('')
setSelectedWordIds(new Set())
}, [editWordId, editWordText, allWords, updateWords])
// Delete selected words
const handleDeleteSelected = useCallback(() => {
if (selectedWordIds.size === 0) return
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
segment.words = segment.words.filter(w => !selectedWordIds.has(w.id))
segment.text = segment.words.map(w => w.text).join(' ')
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
} else {
segment.start_time = null
segment.end_time = null
}
}
// Remove empty segments
return newSegments.filter(s => s.words.length > 0)
})
setSelectedWordIds(new Set())
}, [selectedWordIds])
// Handle word click (selection)
const handleWordClick = useCallback((wordId: string, event: React.MouseEvent) => {
if (event.shiftKey || event.ctrlKey || event.metaKey) {
// Add to selection
setSelectedWordIds(prev => {
const newSet = new Set(prev)
if (newSet.has(wordId)) {
newSet.delete(wordId)
} else {
newSet.add(wordId)
}
return newSet
})
} else {
// Single selection
setSelectedWordIds(new Set([wordId]))
}
}, [])
// Handle background click (deselect)
const handleBackgroundClick = useCallback(() => {
setSelectedWordIds(new Set())
}, [])
// Handle single word timing change (from resize)
const handleWordTimingChange = useCallback((wordId: string, newStartTime: number, newEndTime: number) => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
const word = segment.words.find(w => w.id === wordId)
if (word) {
word.start_time = Math.max(0, newStartTime)
word.end_time = Math.max(word.start_time + 0.05, newEndTime)
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
}
break
}
}
return newSegments
})
}, [])
// Handle moving multiple words (from drag)
const handleWordsMove = useCallback((updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }>) => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
// Create a map for quick lookup
const updateMap = new Map(updates.map(u => [u.wordId, u]))
for (const segment of newSegments) {
for (const word of segment.words) {
const update = updateMap.get(word.id)
if (update) {
word.start_time = update.newStartTime
word.end_time = update.newEndTime
}
}
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
}
}
return newSegments
})
}, [])
// Handle time bar click (seek to position without playing)
const handleTimeBarClick = useCallback((time: number) => {
// Scroll the timeline to show this time centered
const newStart = Math.max(0, time - zoomSeconds / 2)
setVisibleStartTime(Math.min(newStart, Math.max(0, audioDuration - zoomSeconds)))
// Seek to the position (this will briefly start playback)
if (onPlaySegment) {
onPlaySegment(time)
// Immediately stop playback after seeking
setTimeout(() => {
if (typeof window.toggleAudioPlayback === 'function' && window.isAudioPlaying) {
window.toggleAudioPlayback()
}
}, 50)
}
}, [zoomSeconds, audioDuration, onPlaySegment])
// Handle selection complete from drag
const handleSelectionComplete = useCallback((wordIds: string[]) => {
setSelectedWordIds(new Set(wordIds))
}, [])
// Handle spacebar for manual sync
const handleKeyDown = useCallback((e: KeyboardEvent) => {
if (e.code !== 'Space') return
if (!isManualSyncing || isPaused) return
if (syncWordIndex < 0 || syncWordIndex >= allWords.length) return
e.preventDefault()
e.stopPropagation()
if (isSpacebarPressed) return
setIsSpacebarPressed(true)
wordStartTimeRef.current = currentTimeRef.current
spacebarPressTimeRef.current = Date.now()
// Set start time for current word
const newWords = [...allWords]
const currentWord = newWords[syncWordIndex]
currentWord.start_time = currentTimeRef.current
// Handle previous word's end time
if (syncWordIndex > 0) {
const prevWord = newWords[syncWordIndex - 1]
if (prevWord.start_time !== null && prevWord.end_time === null) {
const gap = currentTimeRef.current - prevWord.start_time
if (gap > 1.0) {
prevWord.end_time = prevWord.start_time + 0.5
} else {
prevWord.end_time = currentTimeRef.current - 0.005
}
}
}
updateWords(newWords)
}, [isManualSyncing, isPaused, syncWordIndex, allWords, isSpacebarPressed, updateWords])
const handleKeyUp = useCallback((e: KeyboardEvent) => {
if (e.code !== 'Space') return
if (!isManualSyncing || isPaused) return
if (!isSpacebarPressed) return
e.preventDefault()
e.stopPropagation()
setIsSpacebarPressed(false)
const pressDuration = spacebarPressTimeRef.current
? Date.now() - spacebarPressTimeRef.current
: 0
const isTap = pressDuration < 200 // 200ms threshold
const newWords = [...allWords]
const currentWord = newWords[syncWordIndex]
if (isTap) {
// Short tap: default 500ms duration
currentWord.end_time = (wordStartTimeRef.current || currentTimeRef.current) + 0.5
} else {
// Hold: use actual timing
currentWord.end_time = currentTimeRef.current
}
updateWords(newWords)
// Move to next word
if (syncWordIndex < allWords.length - 1) {
setSyncWordIndex(syncWordIndex + 1)
} else {
// All words synced
setIsManualSyncing(false)
setSyncWordIndex(-1)
handleStopAudio()
}
wordStartTimeRef.current = null
spacebarPressTimeRef.current = null
}, [isManualSyncing, isPaused, isSpacebarPressed, syncWordIndex, allWords, updateWords, handleStopAudio])
// Combined spacebar handler
const handleSpacebar = useCallback((e: KeyboardEvent) => {
if (e.type === 'keydown') {
handleKeyDown(e)
} else if (e.type === 'keyup') {
handleKeyUp(e)
}
}, [handleKeyDown, handleKeyUp])
// Keep ref for handler
const spacebarHandlerRef = useRef(handleSpacebar)
spacebarHandlerRef.current = handleSpacebar
// Set up spacebar handler
useEffect(() => {
const handler = (e: KeyboardEvent) => {
if (e.code === 'Space') {
e.preventDefault()
e.stopPropagation()
spacebarHandlerRef.current(e)
}
}
setModalSpacebarHandler(() => handler)
return () => {
setModalSpacebarHandler(undefined)
}
}, [setModalSpacebarHandler])
// Handle save
const handleSave = useCallback(() => {
onSave(workingSegments)
}, [workingSegments, onSave])
// Progress stats
const stats = useMemo(() => {
const total = allWords.length
const synced = allWords.filter(w =>
w.start_time !== null && w.end_time !== null
).length
return { total, synced, remaining: total - synced }
}, [allWords])
// Get instruction text based on state
const getInstructionText = useCallback(() => {
if (isManualSyncing) {
if (isSpacebarPressed) {
// Always include secondary text to prevent layout shift
return { primary: '⏱️ Holding... release when word ends', secondary: 'Release spacebar when the word finishes' }
}
if (stats.remaining === 0) {
return { primary: '✅ All words synced!', secondary: 'Click "Stop Sync" then "Apply" to save' }
}
return { primary: '👆 Press SPACEBAR when you hear each word', secondary: 'Tap for short words, hold for longer words' }
}
if (stats.synced === 0) {
return { primary: 'Click "Start Sync" to begin timing words', secondary: 'Audio will play and you\'ll tap spacebar for each word' }
}
if (stats.remaining > 0) {
return { primary: `${stats.remaining} words remaining to sync`, secondary: 'Click "Start Sync" to continue, or "Unsync from Cursor" to re-sync from a point' }
}
return { primary: '✅ All words synced!', secondary: 'Click "Apply" to save changes, or make adjustments first' }
}, [isManualSyncing, isSpacebarPressed, stats.synced, stats.remaining])
const instruction = getInstructionText()
return (
<Box sx={{ display: 'flex', flexDirection: 'column', height: '100%', gap: 1 }}>
{/* Stats bar - fixed height */}
<Box sx={{ display: 'flex', justifyContent: 'flex-end', alignItems: 'center', height: 24 }}>
<Typography variant="body2" color="text.secondary">
{stats.synced} / {stats.total} words synced
{stats.remaining > 0 && ` (${stats.remaining} remaining)`}
</Typography>
</Box>
{/* Instruction banner - fixed height to prevent layout shifts */}
<Box
sx={{
height: 56,
flexShrink: 0
}}
>
<Paper
sx={{
p: 1.5,
height: '100%',
bgcolor: isManualSyncing ? 'info.main' : 'grey.100',
color: isManualSyncing ? 'info.contrastText' : 'text.primary',
display: 'flex',
flexDirection: 'column',
justifyContent: 'center',
overflow: 'hidden',
boxSizing: 'border-box'
}}
>
<Typography variant="body2" sx={{ fontWeight: 500, lineHeight: 1.3 }}>
{instruction.primary}
</Typography>
<Typography variant="caption" sx={{ opacity: 0.85, display: 'block', lineHeight: 1.3 }}>
{instruction.secondary}
</Typography>
</Paper>
</Box>
{/* Controls - fixed height section */}
<Box sx={{ height: 88, flexShrink: 0 }}>
<SyncControls
isManualSyncing={isManualSyncing}
isPaused={isPaused}
onStartSync={handleStartSync}
onPauseSync={handlePauseSync}
onResumeSync={handleResumeSync}
onClearSync={handleClearSync}
onEditLyrics={handleEditLyrics}
onPlay={handlePlayAudio}
onStop={handleStopAudio}
isPlaying={isPlaying}
hasSelectedWords={selectedWordIds.size > 0}
selectedWordCount={selectedWordIds.size}
onUnsyncFromCursor={handleUnsyncFromCursor}
onEditSelectedWord={handleEditSelectedWord}
onDeleteSelected={handleDeleteSelected}
canUnsyncFromCursor={canUnsyncFromCursor}
/>
</Box>
{/* Upcoming words bar - fixed height, immediately above timeline */}
<Box sx={{ height: 44, flexShrink: 0 }}>
{isManualSyncing && (
<UpcomingWordsBar
words={allWords}
syncWordIndex={syncWordIndex}
isManualSyncing={isManualSyncing}
/>
)}
</Box>
{/* Timeline canvas */}
<Box sx={{ flexGrow: 1, minHeight: 200 }}>
<TimelineCanvas
words={allWords}
segments={workingSegments}
visibleStartTime={visibleStartTime}
visibleEndTime={visibleEndTime}
currentTime={currentTime}
selectedWordIds={selectedWordIds}
onWordClick={handleWordClick}
onBackgroundClick={handleBackgroundClick}
onTimeBarClick={handleTimeBarClick}
onSelectionComplete={handleSelectionComplete}
onWordTimingChange={handleWordTimingChange}
onWordsMove={handleWordsMove}
syncWordIndex={syncWordIndex}
isManualSyncing={isManualSyncing}
onScrollChange={handleScrollChange}
audioDuration={audioDuration}
zoomSeconds={zoomSeconds}
height={200}
/>
</Box>
{/* Zoom slider */}
<Box sx={{ display: 'flex', alignItems: 'center', gap: 2, px: 2 }}>
<ZoomInIcon color="action" fontSize="small" />
<Slider
value={sliderValue}
onChange={handleZoomChange}
min={0}
max={ZOOM_STEPS}
step={1}
sx={{ flexGrow: 1 }}
disabled={isManualSyncing && !isPaused}
/>
<ZoomOutIcon color="action" fontSize="small" />
<Typography variant="caption" color="text.secondary" sx={{ minWidth: 60 }}>
{zoomSeconds.toFixed(1)}s view
</Typography>
</Box>
{/* Action buttons */}
<Box sx={{ display: 'flex', justifyContent: 'flex-end', gap: 2, pt: 2, borderTop: 1, borderColor: 'divider' }}>
<Button onClick={onCancel} color="inherit">
Cancel
</Button>
<Button
onClick={handleSave}
variant="contained"
color="primary"
disabled={isManualSyncing && !isPaused}
>
Apply
</Button>
</Box>
{/* Edit Lyrics Modal */}
<Dialog
open={showEditLyricsModal}
onClose={() => setShowEditLyricsModal(false)}
maxWidth="md"
fullWidth
>
<DialogTitle>Edit Lyrics</DialogTitle>
<DialogContent>
<Alert severity="warning" sx={{ mb: 2 }}>
Editing lyrics will reset all timing data. You will need to re-sync the entire song.
</Alert>
<TextField
multiline
rows={15}
fullWidth
value={editLyricsText}
onChange={(e) => setEditLyricsText(e.target.value)}
placeholder="Enter lyrics, one line per segment..."
/>
</DialogContent>
<DialogActions>
<Button onClick={() => setShowEditLyricsModal(false)}>Cancel</Button>
<Button onClick={handleSaveEditedLyrics} variant="contained" color="warning">
Save & Reset Timing
</Button>
</DialogActions>
</Dialog>
{/* Edit Word Modal */}
<Dialog
open={showEditWordModal}
onClose={() => setShowEditWordModal(false)}
maxWidth="xs"
fullWidth
>
<DialogTitle>Edit Word</DialogTitle>
<DialogContent>
<Typography variant="body2" color="text.secondary" sx={{ mb: 2 }}>
Edit the word text. Enter multiple words separated by spaces to split.
</Typography>
<TextField
fullWidth
value={editWordText}
onChange={(e) => setEditWordText(e.target.value)}
autoFocus
onKeyDown={(e) => {
if (e.key === 'Enter') {
handleSaveEditedWord()
}
}}
/>
</DialogContent>
<DialogActions>
<Button onClick={() => setShowEditWordModal(false)}>Cancel</Button>
<Button onClick={handleSaveEditedWord} variant="contained">
Save
</Button>
</DialogActions>
</Dialog>
</Box>
)
})
export default LyricsSynchronizer
@@ -0,0 +1,127 @@
import {
Dialog,
DialogTitle,
DialogContent,
DialogActions,
IconButton,
Box,
Button,
Typography,
Paper
} from '@mui/material'
import CloseIcon from '@mui/icons-material/Close'
import ContentPasteIcon from '@mui/icons-material/ContentPaste'
import SyncIcon from '@mui/icons-material/Sync'
interface ModeSelectionModalProps {
open: boolean
onClose: () => void
onSelectReplace: () => void // Goes to paste phase for replacing all lyrics
onSelectResync: () => void // Goes directly to sync view with existing lyrics
hasExistingLyrics: boolean // Whether there are existing lyrics to re-sync
}
export default function ModeSelectionModal({
open,
onClose,
onSelectReplace,
onSelectResync,
hasExistingLyrics
}: ModeSelectionModalProps) {
return (
<Dialog
open={open}
onClose={onClose}
maxWidth="sm"
fullWidth
>
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Box sx={{ flex: 1 }}>
Edit All Lyrics
</Box>
<IconButton onClick={onClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
<DialogContent dividers>
<Typography variant="body1" sx={{ mb: 3 }}>
Choose how you want to edit the lyrics:
</Typography>
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 2 }}>
{/* Re-sync Existing option - only show if there are existing lyrics */}
{hasExistingLyrics && (
<Paper
sx={{
p: 2,
cursor: 'pointer',
border: 2,
borderColor: 'primary.main',
'&:hover': {
bgcolor: 'action.hover',
borderColor: 'primary.dark'
}
}}
onClick={onSelectResync}
>
<Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}>
<SyncIcon color="primary" sx={{ fontSize: 40, mt: 0.5 }} />
<Box>
<Typography variant="h6" color="primary">
Re-sync Existing Lyrics
</Typography>
<Typography variant="body2" color="text.secondary">
Keep the current lyrics text and fix timing issues.
Use this when lyrics are correct but timing has drifted,
especially in the second half of the song.
</Typography>
<Typography variant="caption" color="success.main" sx={{ mt: 1, display: 'block' }}>
Recommended for fixing timing drift
</Typography>
</Box>
</Box>
</Paper>
)}
{/* Replace All option */}
<Paper
sx={{
p: 2,
cursor: 'pointer',
border: 1,
borderColor: 'divider',
'&:hover': {
bgcolor: 'action.hover',
borderColor: 'text.secondary'
}
}}
onClick={onSelectReplace}
>
<Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}>
<ContentPasteIcon sx={{ fontSize: 40, mt: 0.5, color: 'text.secondary' }} />
<Box>
<Typography variant="h6">
Replace All Lyrics
</Typography>
<Typography variant="body2" color="text.secondary">
Paste completely new lyrics from clipboard and manually
sync timing for all words from scratch.
</Typography>
<Typography variant="caption" color="warning.main" sx={{ mt: 1, display: 'block' }}>
All existing timing data will be lost
</Typography>
</Box>
</Box>
</Paper>
</Box>
</DialogContent>
<DialogActions>
<Button onClick={onClose} color="inherit">
Cancel
</Button>
</DialogActions>
</Dialog>
)
}
@@ -7,19 +7,17 @@ import {
Box,
Button,
Typography,
TextField,
Paper,
Divider
TextField
} from '@mui/material'
import CloseIcon from '@mui/icons-material/Close'
import ContentPasteIcon from '@mui/icons-material/ContentPaste'
import AutoFixHighIcon from '@mui/icons-material/AutoFixHigh'
import ArrowBackIcon from '@mui/icons-material/ArrowBack'
import { LyricsSegment, Word } from '../types'
import { useState, useEffect, useCallback, useMemo, useRef, memo } from 'react'
import { useState, useEffect, useCallback, useMemo } from 'react'
import { nanoid } from 'nanoid'
import useManualSync from '../hooks/useManualSync'
import EditTimelineSection from './EditTimelineSection'
import EditActionBar from './EditActionBar'
import ModeSelectionModal from './ModeSelectionModal'
import LyricsSynchronizer from './LyricsSynchronizer'
// Augment window type for audio functions
declare global {
@@ -30,6 +28,8 @@ declare global {
}
}
type ModalMode = 'selection' | 'replace' | 'resync'
interface ReplaceAllLyricsModalProps {
open: boolean
onClose: () => void
@@ -37,6 +37,7 @@ interface ReplaceAllLyricsModalProps {
onPlaySegment?: (startTime: number) => void
currentTime?: number
setModalSpacebarHandler: (handler: (() => (e: KeyboardEvent) => void) | undefined) => void
existingSegments?: LyricsSegment[] // Current segments for re-sync mode
}
export default function ReplaceAllLyricsModal({
@@ -45,22 +46,21 @@ export default function ReplaceAllLyricsModal({
onSave,
onPlaySegment,
currentTime = 0,
setModalSpacebarHandler
setModalSpacebarHandler,
existingSegments = []
}: ReplaceAllLyricsModalProps) {
const [mode, setMode] = useState<ModalMode>('selection')
const [inputText, setInputText] = useState('')
const [isReplaced, setIsReplaced] = useState(false)
const [globalSegment, setGlobalSegment] = useState<LyricsSegment | null>(null)
const [originalSegments, setOriginalSegments] = useState<LyricsSegment[]>([])
const [currentSegments, setCurrentSegments] = useState<LyricsSegment[]>([])
const [newSegments, setNewSegments] = useState<LyricsSegment[]>([])
// Get the real audio duration, with fallback
const getAudioDuration = useCallback(() => {
if (window.getAudioDuration) {
const duration = window.getAudioDuration()
return duration > 0 ? duration : 600 // Use real duration or 10 min fallback
// Reset state when modal opens
useEffect(() => {
if (open) {
setMode('selection')
setInputText('')
setNewSegments([])
}
return 600 // 10 minute fallback if audio not loaded
}, [])
}, [open])
// Parse the input text to get line and word counts
const parseInfo = useMemo(() => {
@@ -79,61 +79,31 @@ export default function ReplaceAllLyricsModal({
if (!inputText.trim()) return
const lines = inputText.trim().split('\n').filter(line => line.trim().length > 0)
const newSegments: LyricsSegment[] = []
const allWords: Word[] = []
const segments: LyricsSegment[] = []
lines.forEach((line) => {
const words = line.trim().split(/\s+/).filter(word => word.length > 0)
const segmentWords: Word[] = []
const segmentWords: Word[] = words.map((wordText) => ({
id: nanoid(),
text: wordText,
start_time: null,
end_time: null,
confidence: 1.0,
created_during_correction: true
}))
words.forEach((wordText) => {
const word: Word = {
id: nanoid(),
text: wordText,
start_time: null,
end_time: null,
confidence: 1.0,
created_during_correction: true
}
segmentWords.push(word)
allWords.push(word)
})
const segment: LyricsSegment = {
segments.push({
id: nanoid(),
text: line.trim(),
words: segmentWords,
start_time: null,
end_time: null
}
newSegments.push(segment)
})
})
// Create a global segment with all words for manual sync
// Set a very large end time to ensure manual sync doesn't stop prematurely
const audioDuration = getAudioDuration()
const endTime = Math.max(audioDuration, 3600) // At least 1 hour to prevent auto-stop
console.log('ReplaceAllLyricsModal - Creating global segment', {
audioDuration,
endTime,
wordCount: allWords.length
})
const globalSegment: LyricsSegment = {
id: 'global-replacement',
text: allWords.map(w => w.text).join(' '),
words: allWords,
start_time: 0,
end_time: endTime
}
setCurrentSegments(newSegments)
setOriginalSegments(JSON.parse(JSON.stringify(newSegments)))
setGlobalSegment(globalSegment)
setIsReplaced(true)
}, [inputText, getAudioDuration])
setNewSegments(segments)
setMode('resync') // Go to synchronizer with the new segments
}, [inputText])
// Handle paste from clipboard
const handlePasteFromClipboard = useCallback(async () => {
@@ -146,311 +116,90 @@ export default function ReplaceAllLyricsModal({
}
}, [])
// Update segment when words change during manual sync
const updateSegment = useCallback((newWords: Word[]) => {
if (!globalSegment) return
const validStartTimes = newWords.map(w => w.start_time).filter((t): t is number => t !== null)
const validEndTimes = newWords.map(w => w.end_time).filter((t): t is number => t !== null)
const segmentStartTime = validStartTimes.length > 0 ? Math.min(...validStartTimes) : null
const segmentEndTime = validEndTimes.length > 0 ? Math.max(...validEndTimes) : null
const updatedGlobalSegment = {
...globalSegment,
words: newWords,
text: newWords.map(w => w.text).join(' '),
start_time: segmentStartTime,
end_time: segmentEndTime
}
// Batch state updates to prevent multiple re-renders
setGlobalSegment(updatedGlobalSegment)
// Update individual segment timing as words get synced - but only calculate when needed
const updatedSegments = currentSegments.map(segment => {
// Find words that belong to this segment and have been timed
const segmentWordsWithTiming = segment.words.map(segmentWord => {
const globalWord = newWords.find(w => w.id === segmentWord.id)
return globalWord || segmentWord
})
// Calculate segment timing if all words have timing
const wordsWithTiming = segmentWordsWithTiming.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (wordsWithTiming.length === segmentWordsWithTiming.length && wordsWithTiming.length > 0) {
// All words in this segment have timing - update segment timing
const segmentStart = Math.min(...wordsWithTiming.map(w => w.start_time!))
const segmentEnd = Math.max(...wordsWithTiming.map(w => w.end_time!))
return {
...segment,
words: segmentWordsWithTiming,
start_time: segmentStart,
end_time: segmentEnd
}
} else {
// Some words still don't have timing - just update the words
return {
...segment,
words: segmentWordsWithTiming
}
}
})
setCurrentSegments(updatedSegments)
}, [globalSegment, currentSegments])
// Use the manual sync hook
const {
isManualSyncing,
isPaused,
syncWordIndex,
startManualSync,
pauseManualSync,
resumeManualSync,
cleanupManualSync,
handleSpacebar,
isSpacebarPressed
} = useManualSync({
editedSegment: globalSegment,
currentTime,
onPlaySegment,
updateSegment
})
// Handle manual word updates (drag/resize in timeline)
const handleWordUpdate = useCallback((wordIndex: number, updates: Partial<Word>) => {
if (!globalSegment) return
// Only allow manual adjustments when manual sync is paused or not active
if (isManualSyncing && !isPaused) {
console.log('ReplaceAllLyricsModal - Ignoring word update during active manual sync')
return
}
console.log('ReplaceAllLyricsModal - Manual word update', {
wordIndex,
wordText: globalSegment.words[wordIndex]?.text,
updates,
isManualSyncing,
isPaused
})
// Update the word in the global segment
const newWords = [...globalSegment.words]
newWords[wordIndex] = {
...newWords[wordIndex],
...updates
}
// Update the global segment through the existing updateSegment function
updateSegment(newWords)
}, [globalSegment, updateSegment, isManualSyncing, isPaused])
// Handle un-syncing a word (right-click context menu)
const handleUnsyncWord = useCallback((wordIndex: number) => {
if (!globalSegment) return
console.log('ReplaceAllLyricsModal - Un-syncing word', {
wordIndex,
wordText: globalSegment.words[wordIndex]?.text
})
// Update the word to remove timing
const newWords = [...globalSegment.words]
newWords[wordIndex] = {
...newWords[wordIndex],
start_time: null,
end_time: null
}
// Update the global segment through the existing updateSegment function
updateSegment(newWords)
}, [globalSegment, updateSegment])
// Handle modal close
const handleClose = useCallback(() => {
cleanupManualSync()
setMode('selection')
setInputText('')
setIsReplaced(false)
setGlobalSegment(null)
setOriginalSegments([])
setCurrentSegments([])
setNewSegments([])
onClose()
}, [onClose, cleanupManualSync])
}, [onClose])
// Handle save
const handleSave = useCallback(() => {
if (!globalSegment || !currentSegments.length) return
// Distribute the timed words back to their original segments
const finalSegments: LyricsSegment[] = []
let wordIndex = 0
currentSegments.forEach((segment) => {
const originalWordCount = segment.words.length
const segmentWords = globalSegment.words.slice(wordIndex, wordIndex + originalWordCount)
wordIndex += originalWordCount
if (segmentWords.length > 0) {
// Recalculate segment start and end times
const validStartTimes = segmentWords.map(w => w.start_time).filter((t): t is number => t !== null)
const validEndTimes = segmentWords.map(w => w.end_time).filter((t): t is number => t !== null)
const segmentStartTime = validStartTimes.length > 0 ? Math.min(...validStartTimes) : null
const segmentEndTime = validEndTimes.length > 0 ? Math.max(...validEndTimes) : null
finalSegments.push({
...segment,
words: segmentWords,
text: segmentWords.map(w => w.text).join(' '),
start_time: segmentStartTime,
end_time: segmentEndTime
})
}
})
console.log('ReplaceAllLyricsModal - Saving new segments:', {
originalSegmentCount: currentSegments.length,
finalSegmentCount: finalSegments.length,
totalWords: finalSegments.reduce((count, seg) => count + seg.words.length, 0)
})
onSave(finalSegments)
// Handle save from synchronizer
const handleSave = useCallback((segments: LyricsSegment[]) => {
onSave(segments)
handleClose()
}, [globalSegment, currentSegments, onSave, handleClose])
}, [onSave, handleClose])
// Handle reset
const handleReset = useCallback(() => {
if (!originalSegments.length) return
// Handle mode selection
const handleSelectReplace = useCallback(() => {
setMode('replace')
}, [])
console.log('ReplaceAllLyricsModal - Resetting to original state')
// Reset all words to have null timing (ready for fresh manual sync)
const resetWords = originalSegments.flatMap(segment =>
segment.words.map(word => ({
...word,
start_time: null,
end_time: null
}))
)
const audioDuration = getAudioDuration()
const resetGlobalSegment: LyricsSegment = {
id: 'global-replacement',
text: resetWords.map(w => w.text).join(' '),
words: resetWords,
start_time: 0,
end_time: Math.max(audioDuration, 3600) // At least 1 hour to prevent auto-stop
}
const handleSelectResync = useCallback(() => {
setMode('resync')
}, [])
// Also reset the current segments to have null timing
const resetCurrentSegments = originalSegments.map(segment => ({
...segment,
words: segment.words.map(word => ({
...word,
start_time: null,
end_time: null
})),
start_time: null,
end_time: null
}))
// Handle back to selection
const handleBackToSelection = useCallback(() => {
setMode('selection')
setInputText('')
setNewSegments([])
}, [])
setGlobalSegment(resetGlobalSegment)
setCurrentSegments(resetCurrentSegments)
}, [originalSegments, getAudioDuration])
// Determine which segments to use for synchronizer
const segmentsForSync = mode === 'resync' && newSegments.length > 0
? newSegments
: existingSegments
// Keep a ref to the current spacebar handler to avoid closure issues
const spacebarHandlerRef = useRef(handleSpacebar)
spacebarHandlerRef.current = handleSpacebar
// Update the spacebar handler when modal state changes
useEffect(() => {
if (open && isReplaced) {
console.log('ReplaceAllLyricsModal - Setting up spacebar handler')
const handleKeyEvent = (e: KeyboardEvent) => {
if (e.code === 'Space') {
console.log('ReplaceAllLyricsModal - Spacebar captured in modal')
e.preventDefault()
e.stopPropagation()
// Use the ref to get the current handler
spacebarHandlerRef.current(e)
}
}
setModalSpacebarHandler(() => handleKeyEvent)
return () => {
if (!open) {
console.log('ReplaceAllLyricsModal - Clearing spacebar handler')
setModalSpacebarHandler(undefined)
}
}
} else if (open) {
// Clear handler when not in replaced state
setModalSpacebarHandler(undefined)
}
}, [open, isReplaced, setModalSpacebarHandler])
// Memoize timeline range to prevent recalculation
const timeRange = useMemo(() => {
const audioDuration = getAudioDuration()
// Always use full song duration for replace-all mode
return { start: 0, end: audioDuration }
}, [getAudioDuration])
// Memoize the segment progress props to prevent unnecessary re-renders
const segmentProgressProps = useMemo(() => ({
currentSegments,
globalSegment,
syncWordIndex
}), [currentSegments, globalSegment, syncWordIndex])
// Check if we have existing lyrics
const hasExistingLyrics = existingSegments.length > 0 &&
existingSegments.some(s => s.words.length > 0)
return (
<Dialog
open={open}
onClose={handleClose}
maxWidth={false}
fullWidth={true}
onKeyDown={(e) => {
if (e.key === 'Enter' && !e.shiftKey && isReplaced) {
e.preventDefault()
handleSave()
}
}}
PaperProps={{
sx: {
height: '90vh',
margin: '5vh 2vh',
maxWidth: 'calc(100vw - 4vh)',
width: 'calc(100vw - 4vh)'
}
}}
>
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Box sx={{ flex: 1 }}>
Replace All Lyrics
</Box>
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
<>
{/* Mode Selection Modal */}
<ModeSelectionModal
open={open && mode === 'selection'}
onClose={handleClose}
onSelectReplace={handleSelectReplace}
onSelectResync={handleSelectResync}
hasExistingLyrics={hasExistingLyrics}
/>
<DialogContent
dividers
sx={{
display: 'flex',
flexDirection: 'column',
flexGrow: 1,
overflow: 'hidden'
{/* Replace All Lyrics Modal (Paste Phase) */}
<Dialog
open={open && mode === 'replace'}
onClose={handleClose}
maxWidth="md"
fullWidth
PaperProps={{
sx: {
height: '80vh',
maxHeight: '80vh'
}
}}
>
{!isReplaced ? (
// Step 1: Input new lyrics
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<IconButton onClick={handleBackToSelection} size="small">
<ArrowBackIcon />
</IconButton>
<Box sx={{ flex: 1 }}>
Replace All Lyrics
</Box>
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
<DialogContent
dividers
sx={{
display: 'flex',
flexDirection: 'column',
flexGrow: 1,
overflow: 'hidden'
}}
>
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 2, height: '100%' }}>
<Typography variant="h6" gutterBottom>
Paste your new lyrics below:
@@ -492,197 +241,96 @@ export default function ReplaceAllLyricsModal({
}
}}
/>
<Box sx={{ display: 'flex', justifyContent: 'flex-end', gap: 2 }}>
<Button
variant="contained"
onClick={processLyrics}
disabled={!inputText.trim()}
startIcon={<AutoFixHighIcon />}
>
Replace All Lyrics
</Button>
</Box>
</Box>
) : (
// Step 2: Manual sync interface
<Box sx={{ display: 'flex', flexDirection: 'column', height: '100%', gap: 2 }}>
<Paper sx={{ p: 2, bgcolor: 'background.paper' }}>
<Typography variant="h6" gutterBottom>
Lyrics Replaced Successfully
</Typography>
<Typography variant="body2" color="text.secondary">
Created {currentSegments.length} segments with {globalSegment?.words.length} words total.
Use Manual Sync to set timing for all words.
</Typography>
</Paper>
</DialogContent>
<Divider />
<DialogActions>
<Button onClick={handleClose} color="inherit">
Cancel
</Button>
<Button
variant="contained"
onClick={processLyrics}
disabled={!inputText.trim()}
startIcon={<AutoFixHighIcon />}
>
Continue to Sync
</Button>
</DialogActions>
</Dialog>
{globalSegment && (
<Box sx={{ display: 'flex', gap: 2, flexGrow: 1, minHeight: 0 }}>
{/* Timeline Section */}
<Box sx={{ flex: 2, display: 'flex', flexDirection: 'column', minHeight: 0 }}>
<EditTimelineSection
words={globalSegment.words}
startTime={timeRange.start}
endTime={timeRange.end}
originalStartTime={0}
originalEndTime={getAudioDuration()}
currentStartTime={globalSegment.start_time}
currentEndTime={globalSegment.end_time}
currentTime={currentTime}
isManualSyncing={isManualSyncing}
syncWordIndex={syncWordIndex}
isSpacebarPressed={isSpacebarPressed}
onWordUpdate={handleWordUpdate}
onUnsyncWord={handleUnsyncWord}
onPlaySegment={onPlaySegment}
onStopAudio={() => {
// Stop audio playback using global function
if (window.toggleAudioPlayback && window.isAudioPlaying) {
window.toggleAudioPlayback()
}
}}
startManualSync={startManualSync}
pauseManualSync={pauseManualSync}
resumeManualSync={resumeManualSync}
isPaused={isPaused}
isGlobal={true}
defaultZoomLevel={10} // Show 10 seconds by default
isReplaceAllMode={true} // Prevent zoom changes during sync
/>
</Box>
{/* Segment Progress Section */}
<SegmentProgressPanel
currentSegments={segmentProgressProps.currentSegments}
globalSegment={segmentProgressProps.globalSegment}
syncWordIndex={segmentProgressProps.syncWordIndex}
/>
</Box>
)}
</Box>
)}
</DialogContent>
<DialogActions>
{isReplaced && (
<EditActionBar
onReset={handleReset}
onClose={handleClose}
onSave={handleSave}
editedSegment={globalSegment}
isGlobal={true}
/>
)}
</DialogActions>
</Dialog>
)
}
// Memoized Segment Progress Item to prevent unnecessary re-renders
const SegmentProgressItem = memo(({
segment,
index,
isActive
}: {
segment: LyricsSegment
index: number
isActive: boolean
}) => {
const wordsWithTiming = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
).length
const totalWords = segment.words.length
const isComplete = wordsWithTiming === totalWords
return (
<Paper
key={segment.id}
ref={isActive ? (el) => {
// Auto-scroll to active segment
if (el) {
el.scrollIntoView({
behavior: 'smooth',
block: 'center'
})
}
} : undefined}
sx={{
p: 1,
mb: 1,
bgcolor: isActive ? 'primary.light' :
isComplete ? 'success.light' : 'background.paper',
border: isActive ? 2 : 1,
borderColor: isActive ? 'primary.main' : 'divider'
}}
>
<Typography
variant="body2"
sx={{
fontWeight: isActive ? 'bold' : 'normal',
mb: 0.5
{/* Synchronizer Modal */}
<Dialog
open={open && mode === 'resync'}
onClose={handleClose}
maxWidth={false}
fullWidth
PaperProps={{
sx: {
height: '90vh',
margin: '5vh 2vw',
maxWidth: 'calc(100vw - 4vw)',
width: 'calc(100vw - 4vw)'
}
}}
>
Segment {index + 1}: {segment.text.slice(0, 50)}
{segment.text.length > 50 ? '...' : ''}
</Typography>
<Typography variant="caption" color="text.secondary">
{wordsWithTiming}/{totalWords} words synced
{isComplete && segment.start_time !== null && segment.end_time !== null && (
<>
<br />
{segment.start_time.toFixed(2)}s - {segment.end_time.toFixed(2)}s
</>
)}
</Typography>
</Paper>
)
})
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<IconButton onClick={handleBackToSelection} size="small">
<ArrowBackIcon />
</IconButton>
<Box sx={{ flex: 1 }}>
{newSegments.length > 0 ? 'Sync New Lyrics' : 'Re-sync Existing Lyrics'}
</Box>
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
// Memoized Segment Progress Panel
const SegmentProgressPanel = memo(({
currentSegments,
globalSegment,
syncWordIndex
}: {
currentSegments: LyricsSegment[]
globalSegment: LyricsSegment | null
syncWordIndex: number
}) => {
return (
<Box sx={{ flex: 1, display: 'flex', flexDirection: 'column', minHeight: 0 }}>
<Typography variant="h6" gutterBottom>
Segment Progress
</Typography>
<Box sx={{
overflow: 'auto',
flexGrow: 1,
border: 1,
borderColor: 'divider',
borderRadius: 1,
p: 1
}}>
{currentSegments.map((segment, index) => {
const isActive = Boolean(
globalSegment &&
syncWordIndex >= 0 &&
syncWordIndex < globalSegment.words.length &&
globalSegment.words[syncWordIndex] &&
segment.words.some(w => w.id === globalSegment.words[syncWordIndex].id)
)
return (
<SegmentProgressItem
key={segment.id}
segment={segment}
index={index}
isActive={isActive}
<DialogContent
dividers
sx={{
display: 'flex',
flexDirection: 'column',
flexGrow: 1,
overflow: 'hidden',
p: 2
}}
>
{segmentsForSync.length > 0 ? (
<LyricsSynchronizer
segments={segmentsForSync}
currentTime={currentTime}
onPlaySegment={onPlaySegment}
onSave={handleSave}
onCancel={handleClose}
setModalSpacebarHandler={setModalSpacebarHandler}
/>
)
})}
</Box>
</Box>
) : (
<Box sx={{
display: 'flex',
flexDirection: 'column',
alignItems: 'center',
justifyContent: 'center',
height: '100%',
gap: 2
}}>
<Typography variant="h6" color="text.secondary">
No lyrics to sync
</Typography>
<Typography variant="body2" color="text.secondary">
Go back and paste new lyrics, or close this modal.
</Typography>
<Button
variant="outlined"
onClick={handleBackToSelection}
startIcon={<ArrowBackIcon />}
>
Back to Selection
</Button>
</Box>
)}
</DialogContent>
</Dialog>
</>
)
})
}
@@ -1 +1 @@
{"root":["./src/app.tsx","./src/api.ts","./src/main.tsx","./src/theme.ts","./src/types.ts","./src/validation.ts","./src/vite-env.d.ts","./src/components/aifeedbackmodal.tsx","./src/components/addlyricsmodal.tsx","./src/components/agenticcorrectionmetrics.tsx","./src/components/audioplayer.tsx","./src/components/correctedwordwithactions.tsx","./src/components/correctionannotationmodal.tsx","./src/components/correctiondetailcard.tsx","./src/components/correctionmetrics.tsx","./src/components/durationtimelineview.tsx","./src/components/editactionbar.tsx","./src/components/editmodal.tsx","./src/components/edittimelinesection.tsx","./src/components/editwordlist.tsx","./src/components/fileupload.tsx","./src/components/findreplacemodal.tsx","./src/components/header.tsx","./src/components/lyricsanalyzer.tsx","./src/components/metricsdashboard.tsx","./src/components/modeselector.tsx","./src/components/modelselector.tsx","./src/components/previewvideosection.tsx","./src/components/referenceview.tsx","./src/components/replacealllyricsmodal.tsx","./src/components/reviewchangesmodal.tsx","./src/components/segmentdetailsmodal.tsx","./src/components/timelineeditor.tsx","./src/components/timingoffsetmodal.tsx","./src/components/transcriptionview.tsx","./src/components/worddivider.tsx","./src/components/shared/constants.ts","./src/components/shared/styles.ts","./src/components/shared/types.ts","./src/components/shared/components/highlightedtext.tsx","./src/components/shared/components/sourceselector.tsx","./src/components/shared/components/word.tsx","./src/components/shared/hooks/usewordclick.ts","./src/components/shared/utils/keyboardhandlers.ts","./src/components/shared/utils/localstorage.ts","./src/components/shared/utils/referencelinecalculator.ts","./src/components/shared/utils/segmentoperations.ts","./src/components/shared/utils/timingutils.ts","./src/components/shared/utils/wordutils.ts","./src/hooks/usemanualsync.ts","./src/types/global.d.ts"],"version":"5.6.3"}
{"root":["./src/app.tsx","./src/api.ts","./src/main.tsx","./src/theme.ts","./src/types.ts","./src/validation.ts","./src/vite-env.d.ts","./src/components/aifeedbackmodal.tsx","./src/components/addlyricsmodal.tsx","./src/components/agenticcorrectionmetrics.tsx","./src/components/audioplayer.tsx","./src/components/correctedwordwithactions.tsx","./src/components/correctionannotationmodal.tsx","./src/components/correctiondetailcard.tsx","./src/components/correctionmetrics.tsx","./src/components/durationtimelineview.tsx","./src/components/editactionbar.tsx","./src/components/editmodal.tsx","./src/components/edittimelinesection.tsx","./src/components/editwordlist.tsx","./src/components/fileupload.tsx","./src/components/findreplacemodal.tsx","./src/components/header.tsx","./src/components/lyricsanalyzer.tsx","./src/components/metricsdashboard.tsx","./src/components/modeselectionmodal.tsx","./src/components/modeselector.tsx","./src/components/modelselector.tsx","./src/components/previewvideosection.tsx","./src/components/referenceview.tsx","./src/components/replacealllyricsmodal.tsx","./src/components/reviewchangesmodal.tsx","./src/components/segmentdetailsmodal.tsx","./src/components/timelineeditor.tsx","./src/components/timingoffsetmodal.tsx","./src/components/transcriptionview.tsx","./src/components/worddivider.tsx","./src/components/lyricssynchronizer/synccontrols.tsx","./src/components/lyricssynchronizer/timelinecanvas.tsx","./src/components/lyricssynchronizer/upcomingwordsbar.tsx","./src/components/lyricssynchronizer/index.tsx","./src/components/shared/constants.ts","./src/components/shared/styles.ts","./src/components/shared/types.ts","./src/components/shared/components/highlightedtext.tsx","./src/components/shared/components/sourceselector.tsx","./src/components/shared/components/word.tsx","./src/components/shared/hooks/usewordclick.ts","./src/components/shared/utils/keyboardhandlers.ts","./src/components/shared/utils/localstorage.ts","./src/components/shared/utils/referencelinecalculator.ts","./src/components/shared/utils/segmentoperations.ts","./src/components/shared/utils/timingutils.ts","./src/components/shared/utils/wordutils.ts","./src/hooks/usemanualsync.ts","./src/types/global.d.ts"],"version":"5.6.3"}
@@ -0,0 +1 @@
{"version":3,"file":"index-COYImAcx.js","sources":["../../node_modules/react/cjs/react.production.min.js","../../node_modules/react/index.js","../../node_modules/react/cjs/react-jsx-runtime.production.min.js","../../node_modules/react/jsx-runtime.js","../../node_modules/scheduler/cjs/scheduler.production.min.js","../../node_modules/scheduler/index.js","../../node_modules/react-dom/cjs/react-dom.production.min.js","../../node_modules/react-dom/index.js","../../node_modules/react-dom/client.js","../../node_modules/@mui/utils/esm/formatMuiErrorMessage/formatMuiErrorMessage.js","../../node_modules/@mui/material/styles/identifier.js","../../node_modules/@babel/runtime/helpers/esm/extends.js","../../node_modules/@emotion/sheet/dist/emotion-sheet.esm.js","../../node_modules/stylis/src/Enum.js","../../node_modules/stylis/src/Utility.js","../../node_modules/stylis/src/Tokenizer.js","../../node_modules/stylis/src/Parser.js","../../node_modules/stylis/src/Serializer.js","../../node_modules/stylis/src/Middleware.js","../../node_modules/@emotion/memoize/dist/emotion-memoize.esm.js","../../node_modules/@emotion/cache/dist/emotion-cache.browser.esm.js","../../node_modules/react-is/cjs/react-is.production.min.js","../../node_modules/react-is/index.js","../../node_modules/hoist-non-react-statics/dist/hoist-non-react-statics.cjs.js","../../node_modules/@emotion/utils/dist/emotion-utils.browser.esm.js","../../node_modules/@emotion/hash/dist/emotion-hash.esm.js","../../node_modules/@emotion/unitless/dist/emotion-unitless.esm.js","../../node_modules/@emotion/serialize/dist/emotion-serialize.esm.js","../../node_modules/@emotion/use-insertion-effect-with-fallbacks/dist/emotion-use-insertion-effect-with-fallbacks.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-element-f0de968e.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-react.browser.esm.js","../../node_modules/@emotion/is-prop-valid/dist/emotion-is-prop-valid.esm.js","../../node_modules/@emotion/styled/base/dist/emotion-styled-base.browser.esm.js","../../node_modules/@emotion/styled/dist/emotion-styled.browser.esm.js","../../node_modules/@mui/styled-engine/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/styled-engine/index.js","../../node_modules/@mui/utils/node_modules/react-is/cjs/react-is.production.js","../../node_modules/@mui/utils/node_modules/react-is/index.js","../../node_modules/@mui/utils/esm/deepmerge/deepmerge.js","../../node_modules/@mui/system/esm/createBreakpoints/createBreakpoints.js","../../node_modules/@mui/system/esm/cssContainerQueries/cssContainerQueries.js","../../node_modules/@mui/system/esm/createTheme/shape.js","../../node_modules/@mui/system/esm/merge/merge.js","../../node_modules/@mui/system/esm/breakpoints/breakpoints.js","../../node_modules/@mui/utils/esm/capitalize/capitalize.js","../../node_modules/@mui/system/esm/style/style.js","../../node_modules/@mui/system/esm/memoize/memoize.js","../../node_modules/@mui/system/esm/spacing/spacing.js","../../node_modules/@mui/system/esm/createTheme/createSpacing.js","../../node_modules/@mui/system/esm/compose/compose.js","../../node_modules/@mui/system/esm/borders/borders.js","../../node_modules/@mui/system/esm/cssGrid/cssGrid.js","../../node_modules/@mui/system/esm/palette/palette.js","../../node_modules/@mui/system/esm/sizing/sizing.js","../../node_modules/@mui/system/esm/styleFunctionSx/defaultSxConfig.js","../../node_modules/@mui/system/esm/styleFunctionSx/styleFunctionSx.js","../../node_modules/@mui/system/esm/createTheme/applyStyles.js","../../node_modules/@mui/system/esm/createTheme/createTheme.js","../../node_modules/@mui/system/esm/useThemeWithoutDefault/useThemeWithoutDefault.js","../../node_modules/@mui/system/esm/useTheme/useTheme.js","../../node_modules/@mui/system/esm/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/system/esm/styleFunctionSx/extendSxProp.js","../../node_modules/@mui/utils/esm/ClassNameGenerator/ClassNameGenerator.js","../../node_modules/clsx/dist/clsx.mjs","../../node_modules/@mui/system/esm/createBox/createBox.js","../../node_modules/@mui/utils/esm/generateUtilityClass/generateUtilityClass.js","../../node_modules/@mui/utils/esm/generateUtilityClasses/generateUtilityClasses.js","../../node_modules/@mui/system/esm/preprocessStyles.js","../../node_modules/@mui/system/esm/createStyled/createStyled.js","../../node_modules/@mui/system/esm/styled/styled.js","../../node_modules/@mui/utils/esm/resolveProps/resolveProps.js","../../node_modules/@mui/system/esm/useThemeProps/getThemeProps.js","../../node_modules/@mui/system/esm/useThemeProps/useThemeProps.js","../../node_modules/@mui/utils/esm/useEnhancedEffect/useEnhancedEffect.js","../../node_modules/@mui/system/esm/useMediaQuery/useMediaQuery.js","../../node_modules/@mui/utils/esm/clamp/clamp.js","../../node_modules/@mui/system/esm/colorManipulator/colorManipulator.js","../../node_modules/@mui/utils/esm/createChainedFunction/createChainedFunction.js","../../node_modules/@mui/utils/esm/debLine truncated
@@ -1 +0,0 @@
{"version":3,"file":"index-DdJTDWH3.js","sources":["../../node_modules/react/cjs/react.production.min.js","../../node_modules/react/index.js","../../node_modules/react/cjs/react-jsx-runtime.production.min.js","../../node_modules/react/jsx-runtime.js","../../node_modules/scheduler/cjs/scheduler.production.min.js","../../node_modules/scheduler/index.js","../../node_modules/react-dom/cjs/react-dom.production.min.js","../../node_modules/react-dom/index.js","../../node_modules/react-dom/client.js","../../node_modules/@mui/utils/esm/formatMuiErrorMessage/formatMuiErrorMessage.js","../../node_modules/@mui/material/styles/identifier.js","../../node_modules/@babel/runtime/helpers/esm/extends.js","../../node_modules/@emotion/sheet/dist/emotion-sheet.esm.js","../../node_modules/stylis/src/Enum.js","../../node_modules/stylis/src/Utility.js","../../node_modules/stylis/src/Tokenizer.js","../../node_modules/stylis/src/Parser.js","../../node_modules/stylis/src/Serializer.js","../../node_modules/stylis/src/Middleware.js","../../node_modules/@emotion/memoize/dist/emotion-memoize.esm.js","../../node_modules/@emotion/cache/dist/emotion-cache.browser.esm.js","../../node_modules/react-is/cjs/react-is.production.min.js","../../node_modules/react-is/index.js","../../node_modules/hoist-non-react-statics/dist/hoist-non-react-statics.cjs.js","../../node_modules/@emotion/utils/dist/emotion-utils.browser.esm.js","../../node_modules/@emotion/hash/dist/emotion-hash.esm.js","../../node_modules/@emotion/unitless/dist/emotion-unitless.esm.js","../../node_modules/@emotion/serialize/dist/emotion-serialize.esm.js","../../node_modules/@emotion/use-insertion-effect-with-fallbacks/dist/emotion-use-insertion-effect-with-fallbacks.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-element-f0de968e.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-react.browser.esm.js","../../node_modules/@emotion/is-prop-valid/dist/emotion-is-prop-valid.esm.js","../../node_modules/@emotion/styled/base/dist/emotion-styled-base.browser.esm.js","../../node_modules/@emotion/styled/dist/emotion-styled.browser.esm.js","../../node_modules/@mui/styled-engine/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/styled-engine/index.js","../../node_modules/@mui/utils/node_modules/react-is/cjs/react-is.production.js","../../node_modules/@mui/utils/node_modules/react-is/index.js","../../node_modules/@mui/utils/esm/deepmerge/deepmerge.js","../../node_modules/@mui/system/esm/createBreakpoints/createBreakpoints.js","../../node_modules/@mui/system/esm/cssContainerQueries/cssContainerQueries.js","../../node_modules/@mui/system/esm/createTheme/shape.js","../../node_modules/@mui/system/esm/merge/merge.js","../../node_modules/@mui/system/esm/breakpoints/breakpoints.js","../../node_modules/@mui/utils/esm/capitalize/capitalize.js","../../node_modules/@mui/system/esm/style/style.js","../../node_modules/@mui/system/esm/memoize/memoize.js","../../node_modules/@mui/system/esm/spacing/spacing.js","../../node_modules/@mui/system/esm/createTheme/createSpacing.js","../../node_modules/@mui/system/esm/compose/compose.js","../../node_modules/@mui/system/esm/borders/borders.js","../../node_modules/@mui/system/esm/cssGrid/cssGrid.js","../../node_modules/@mui/system/esm/palette/palette.js","../../node_modules/@mui/system/esm/sizing/sizing.js","../../node_modules/@mui/system/esm/styleFunctionSx/defaultSxConfig.js","../../node_modules/@mui/system/esm/styleFunctionSx/styleFunctionSx.js","../../node_modules/@mui/system/esm/createTheme/applyStyles.js","../../node_modules/@mui/system/esm/createTheme/createTheme.js","../../node_modules/@mui/system/esm/useThemeWithoutDefault/useThemeWithoutDefault.js","../../node_modules/@mui/system/esm/useTheme/useTheme.js","../../node_modules/@mui/system/esm/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/system/esm/styleFunctionSx/extendSxProp.js","../../node_modules/@mui/utils/esm/ClassNameGenerator/ClassNameGenerator.js","../../node_modules/clsx/dist/clsx.mjs","../../node_modules/@mui/system/esm/createBox/createBox.js","../../node_modules/@mui/utils/esm/generateUtilityClass/generateUtilityClass.js","../../node_modules/@mui/utils/esm/generateUtilityClasses/generateUtilityClasses.js","../../node_modules/@mui/system/esm/preprocessStyles.js","../../node_modules/@mui/system/esm/createStyled/createStyled.js","../../node_modules/@mui/system/esm/styled/styled.js","../../node_modules/@mui/utils/esm/resolveProps/resolveProps.js","../../node_modules/@mui/system/esm/useThemeProps/getThemeProps.js","../../node_modules/@mui/system/esm/useThemeProps/useThemeProps.js","../../node_modules/@mui/utils/esm/useEnhancedEffect/useEnhancedEffect.js","../../node_modules/@mui/system/esm/useMediaQuery/useMediaQuery.js","../../node_modules/@mui/utils/esm/clamp/clamp.js","../../node_modules/@mui/system/esm/colorManipulator/colorManipulator.js","../../node_modules/@mui/utils/esm/createChainedFunction/createChainedFunction.js","../../node_modules/@mui/utils/esm/debLine truncated
@@ -10,7 +10,7 @@
<link rel="icon" type="image/png" sizes="512x512" href="/android-chrome-512x512.png" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Nomad Karaoke: Lyrics Review</title>
<script type="module" crossorigin src="/assets/index-DdJTDWH3.js"></script>
<script type="module" crossorigin src="/assets/index-COYImAcx.js"></script>
</head>
<body>
<div id="root"></div>
@@ -629,14 +629,14 @@ class ReviewServer:
else ""
)
# Use hosted review UI by default, can be overridden with LYRICS_REVIEW_UI_URL env var
# Set to "local" to use the bundled local frontend instead
review_ui_url = os.environ.get("LYRICS_REVIEW_UI_URL", "https://lyrics.nomadkaraoke.com")
# Use bundled local frontend by default for local karaoke-gen runs
# Can override with LYRICS_REVIEW_UI_URL env var (e.g., http://localhost:5173 for Vite dev)
review_ui_url = os.environ.get("LYRICS_REVIEW_UI_URL", "local")
if review_ui_url.lower() == "local":
# Use the bundled local frontend
# Use the bundled local frontend served by this FastAPI server
browser_url = f"http://localhost:8000?baseApiUrl={encoded_api_url}{audio_hash_param}"
else:
# Use the hosted/external review UI
# Use an external review UI (dev server or hosted)
browser_url = f"{review_ui_url}?baseApiUrl={encoded_api_url}{audio_hash_param}"
self.logger.info(f"Opening review UI: {browser_url}")
@@ -0,0 +1,363 @@
#!/usr/bin/env python3
"""
Generate golden test fixtures for AnchorSequenceFinder using real cache data.
This script loads real transcription and reference data from the local cache,
runs the current AnchorSequenceFinder implementation, and saves the results
as fixtures for regression testing.
Usage:
python scripts/generate_anchor_fixtures.py
"""
import json
import os
import sys
from pathlib import Path
from dataclasses import dataclass
from typing import Dict, List, Optional, Any
# Add the parent directory to the path so we can import lyrics_transcriber
sys.path.insert(0, str(Path(__file__).parent.parent))
from lyrics_transcriber.types import (
LyricsData,
LyricsSegment,
LyricsMetadata,
TranscriptionData,
TranscriptionResult,
Word,
)
from lyrics_transcriber.correction.anchor_sequence import AnchorSequenceFinder
@dataclass
class SongMapping:
"""Maps an audioshake transcription to its reference lyrics."""
name: str
description: str
audioshake_hash: str
reference_hash: str
# 10 diverse songs with varying lengths and characteristics
SONG_MAPPINGS = [
SongMapping(
name="waterloo_abba",
description="ABBA - Waterloo, short song snippet (30 words)",
audioshake_hash="1654b7c0755fad7f23bc888308f274de",
reference_hash="3365f6a1e42388d84bcbfc0ac6f8aa6c",
),
SongMapping(
name="let_your_hair_down",
description="Let Your Hair Down, short song (86 words)",
audioshake_hash="b19b4fca60015bbec16e4e0d68c2e777",
reference_hash="a2e1d435193bf261db0246b535a87de9",
),
SongMapping(
name="daddy_fell_into_sun",
description="Daddy Fell Into The Sun, short (97 words)",
audioshake_hash="2f7e53617228901f67c2256e5648fbd6",
reference_hash="b1aac2e85a3800fddae4a77c74b37280",
),
SongMapping(
name="pull_the_curtain",
description="Pull the Curtain, medium-short (140 words)",
audioshake_hash="87a40e919d0ba2066fa22c2b74cacc19",
reference_hash="3448335d15fc3b842eef858e59681300",
),
SongMapping(
name="bright_eyes",
description="Bright Eyes, medium (164 words)",
audioshake_hash="6cfe3e59cb5c0a10d935396ff8b96752",
reference_hash="748e5a24b339ab66e8bbb55e42e67770",
),
SongMapping(
name="texas_sun",
description="Texas Sun, medium (183 words)",
audioshake_hash="d0f851c8b3e7d56784c185f02416485b",
reference_hash="2f0173340cb1169caf003cbfe535315b",
),
SongMapping(
name="shut_up_and_let_me_go",
description="Ting Tings - Shut Up and Let Me Go, medium-long (231 words)",
audioshake_hash="489c65695fca5bbe9cd11325af8e34ca",
reference_hash="e96729ca29274f64ddc9ad461058949d",
),
SongMapping(
name="for_your_pleasure",
description="For Your Pleasure, medium-long (251 words)",
audioshake_hash="89fec5f0c8520e98bbeb17fe349e3182",
reference_hash="7b2afa24bfb3f9611b45fb70698a8263",
),
SongMapping(
name="mayday_situation",
description="Mayday Situation Overload, long (292 words)",
audioshake_hash="0655f3ddeac47008cee8f484f4a1ed55",
reference_hash="35d368e93554c99d18d666eafe37f16c",
),
SongMapping(
name="devil_went_down_to_georgia",
description="The Devil Went Down to Georgia, long (391 words)",
audioshake_hash="6f76f504e6a3c6dff7ee99191973579a",
reference_hash="a98b3484d65c132218575f9cc118994d",
),
]
def get_cache_dir() -> Path:
"""Get the lyrics transcriber cache directory."""
return Path(os.path.expanduser("~/lyrics-transcriber-cache"))
def load_audioshake_as_transcription(cache_dir: Path, audioshake_hash: str) -> Optional[TranscriptionResult]:
"""Load an audioshake converted file and convert it to TranscriptionResult."""
file_path = cache_dir / f"audioshake_{audioshake_hash}_converted.json"
if not file_path.exists():
print(f" WARNING: Audioshake file not found: {file_path}")
return None
with open(file_path) as f:
data = json.load(f)
# Convert the raw cache format to LyricsSegment objects
segments = []
all_words = []
for seg_data in data.get("segments", []):
words = []
for w_data in seg_data.get("words", []):
word = Word(
id=w_data.get("id", f"w_{len(all_words)}"),
text=w_data.get("text", ""),
start_time=w_data.get("start_time") or 0.0,
end_time=w_data.get("end_time") or 0.0,
confidence=w_data.get("confidence"),
created_during_correction=w_data.get("created_during_correction", False),
)
words.append(word)
all_words.append(word)
segment = LyricsSegment(
id=seg_data.get("id", f"s_{len(segments)}"),
text=seg_data.get("text", ""),
words=words,
start_time=seg_data.get("start_time") or 0.0,
end_time=seg_data.get("end_time") or 0.0,
)
segments.append(segment)
# Create TranscriptionData
full_text = " ".join(seg.text for seg in segments)
transcription_data = TranscriptionData(
segments=segments,
words=all_words,
text=full_text,
source="audioshake",
)
return TranscriptionResult(
name="audioshake",
priority=1,
result=transcription_data,
)
def load_reference_lyrics(cache_dir: Path, reference_hash: str) -> Dict[str, LyricsData]:
"""Load all available reference lyrics for a given hash."""
references = {}
providers = ["spotify", "genius", "lrclib", "musixmatch"]
for provider in providers:
file_path = cache_dir / f"{provider}_{reference_hash}_converted.json"
if not file_path.exists():
continue
with open(file_path) as f:
data = json.load(f)
# Convert to LyricsSegment objects
segments = []
for seg_data in data.get("segments", []):
words = []
for w_data in seg_data.get("words", []):
word = Word(
id=w_data.get("id", f"w_{len(words)}"),
text=w_data.get("text", ""),
start_time=w_data.get("start_time") or 0.0,
end_time=w_data.get("end_time") or 0.0,
confidence=w_data.get("confidence"),
created_during_correction=w_data.get("created_during_correction", False),
)
words.append(word)
segment = LyricsSegment(
id=seg_data.get("id", f"s_{len(segments)}"),
text=seg_data.get("text", ""),
words=words,
start_time=seg_data.get("start_time") or 0.0,
end_time=seg_data.get("end_time") or 0.0,
)
segments.append(segment)
# Try to get metadata from raw file
raw_path = cache_dir / f"{provider}_{reference_hash}_raw.json"
track_name = "Unknown"
artist_names = "Unknown"
if raw_path.exists():
try:
with open(raw_path) as f:
raw_data = json.load(f)
track_name = raw_data.get("trackName") or raw_data.get("name") or raw_data.get("track_data", {}).get("name", "Unknown")
artist_names = raw_data.get("artistName") or raw_data.get("track_data", {}).get("artists", [{}])[0].get("name", "Unknown")
except:
pass
metadata = LyricsMetadata(
source=provider,
track_name=track_name,
artist_names=artist_names,
is_synced=True,
)
lyrics_data = LyricsData(
segments=segments,
metadata=metadata,
source=provider,
)
references[provider] = lyrics_data
return references
def extract_words_from_segments(segments: List[LyricsSegment]) -> List[str]:
"""Extract all word texts from segments."""
words = []
for seg in segments:
for word in seg.words:
words.append(word.text)
return words
def generate_fixture(mapping: SongMapping, cache_dir: Path, output_dir: Path) -> bool:
"""Generate a fixture for a single song mapping."""
print(f"Generating fixture for: {mapping.name}")
print(f" Description: {mapping.description}")
# Load transcription
transcription_result = load_audioshake_as_transcription(cache_dir, mapping.audioshake_hash)
if transcription_result is None:
print(f" SKIPPED: Could not load transcription")
return False
# Load references
references = load_reference_lyrics(cache_dir, mapping.reference_hash)
if not references:
print(f" SKIPPED: No reference lyrics found")
return False
print(f" Loaded {len(references)} reference sources: {list(references.keys())}")
# Get transcribed words and text
transcribed_words = extract_words_from_segments(transcription_result.result.segments)
transcribed_text = " ".join(transcribed_words)
print(f" Transcription word count: {len(transcribed_words)}")
# Run AnchorSequenceFinder
print(f" Running AnchorSequenceFinder...")
try:
finder = AnchorSequenceFinder(
cache_dir=str(cache_dir),
timeout_seconds=300, # 5 minute timeout for fixture generation
)
anchors = finder.find_anchors(
transcribed=transcribed_text,
references=references,
transcription_result=transcription_result,
)
print(f" Found {len(anchors)} anchors")
except Exception as e:
print(f" ERROR running AnchorSequenceFinder: {e}")
import traceback
traceback.print_exc()
return False
# Calculate coverage
covered_positions = set()
for anchor in anchors:
pos = anchor.anchor.transcription_position
length = anchor.anchor.length
for i in range(pos, pos + length):
covered_positions.add(i)
coverage_percent = (len(covered_positions) / len(transcribed_words) * 100) if transcribed_words else 0
# Build fixture
fixture = {
"name": mapping.name,
"description": mapping.description,
"source_hashes": {
"audioshake": mapping.audioshake_hash,
"references": mapping.reference_hash,
},
"transcription": {
"segments": [seg.to_dict() for seg in transcription_result.result.segments],
"word_count": len(transcribed_words),
"text": transcription_result.result.text,
},
"references": {
source: {
"segments": [seg.to_dict() for seg in lyrics.segments],
"word_count": sum(len(seg.words) for seg in lyrics.segments),
}
for source, lyrics in references.items()
},
"expected_anchors": [anchor.to_dict() for anchor in anchors],
"expected_anchor_count": len(anchors),
"expected_word_coverage": len(covered_positions),
"expected_coverage_percent": round(coverage_percent, 2),
}
# Save fixture
output_path = output_dir / f"{mapping.name}.json"
with open(output_path, 'w') as f:
json.dump(fixture, f, indent=2)
print(f" Saved fixture to: {output_path}")
print(f" Coverage: {coverage_percent:.1f}% ({len(covered_positions)}/{len(transcribed_words)} words)")
return True
def main():
"""Generate all fixtures."""
cache_dir = get_cache_dir()
output_dir = Path(__file__).parent.parent / "tests" / "fixtures" / "anchor_golden"
print(f"Cache directory: {cache_dir}")
print(f"Output directory: {output_dir}")
print()
# Ensure output directory exists
output_dir.mkdir(parents=True, exist_ok=True)
# Generate fixtures
success_count = 0
for mapping in SONG_MAPPINGS:
if generate_fixture(mapping, cache_dir, output_dir):
success_count += 1
print()
print(f"Generated {success_count}/{len(SONG_MAPPINGS)} fixtures successfully")
if success_count < len(SONG_MAPPINGS):
print("\nWARNING: Some fixtures could not be generated. Check the errors above.")
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+27 -1
View File
@@ -11,9 +11,35 @@ sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')
# All tests are now working! No more skip patterns needed.
SKIP_PATTERNS = []
def pytest_configure(config):
"""Register custom markers."""
config.addinivalue_line(
"markers", "slow: marks tests as slow (run with --run-slow)"
)
def pytest_addoption(parser):
"""Add custom command line options."""
parser.addoption(
"--run-slow",
action="store_true",
default=False,
help="run slow tests that execute expensive algorithms"
)
def pytest_collection_modifyitems(config, items):
"""Modify test collection to skip slow tests unless --run-slow is specified."""
# Skip slow tests unless --run-slow is specified
if not config.getoption("--run-slow", default=False):
skip_slow = pytest.mark.skip(reason="need --run-slow option to run")
for item in items:
if "slow" in item.keywords:
item.add_marker(skip_slow)
# Legacy skip patterns
for item in items:
# Check if any test path pattern matches the current test
for pattern in SKIP_PATTERNS:
if pattern in item.nodeid:
item.add_marker(pytest.mark.skip(reason="Skipped due to codebase changes"))
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
Loaded 100 of 132 files, more files were not shown because too many files have changed in this diff. Show more