Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ddc4460065 | ||
|
|
b0f3d15112 | ||
|
|
753499db01 | ||
|
|
1b6e682860 | ||
|
|
9baba12280 | ||
|
|
24b7b75ab2 | ||
|
|
0b59b7abfc | ||
|
|
942cbb4f66 | ||
|
|
96383a60d4 | ||
|
|
0cf14be531 | ||
|
|
95edd82555 | ||
|
|
6e02ca3056 | ||
|
|
966e8cc963 | ||
|
|
04b1c8fd3e | ||
|
|
f47b126c08 | ||
|
|
4438293b6d | ||
|
|
de4e92d9bf | ||
|
|
e7f88b60d0 | ||
|
|
fbb1cd7299 | ||
|
|
77820a44c9 | ||
|
|
e92ddd8609 | ||
|
|
a2bf2230ab | ||
|
|
75c6879b7e | ||
|
|
f51ef68179 | ||
|
|
8a5420f5f8 | ||
|
|
c9b7f1bb1a | ||
|
|
d9da1dd360 | ||
|
|
a8fa6b5f46 | ||
|
|
ebe0121938 | ||
|
|
01db9a764a | ||
|
|
f524796f7f | ||
|
|
8372ed211a | ||
|
|
78068ea1d8 | ||
|
|
84d7c12f13 | ||
|
|
7a4ed9661c | ||
|
|
dd62c32026 | ||
|
|
c130fa8a07 | ||
|
|
e0d1fa7536 | ||
|
|
29ffbd4ae2 | ||
|
|
e440f0bbcd | ||
|
|
e0f5a84e54 | ||
|
|
8da57da767 | ||
|
|
b840a12669 | ||
|
|
f7038ce72c | ||
|
|
138ddeecf5 | ||
|
|
a7d9bb40b9 | ||
|
|
20f842fa85 | ||
|
|
1b74199032 | ||
|
|
e75c5b2723 | ||
|
|
9e53bb7934 | ||
|
|
a85727b147 | ||
|
|
3eb8f707b6 | ||
|
|
4b26fe9eac | ||
|
|
4de3ecc85d | ||
|
|
cbb4cd8f9e | ||
|
|
218011cf2a | ||
|
|
22c6e7ed41 | ||
|
|
e104fb9252 | ||
|
|
31931cc55d | ||
|
|
d21729bf3d | ||
|
|
b49a09e1c6 | ||
|
|
e6b2194272 | ||
|
|
a08b83df40 | ||
|
|
53d2ed2828 | ||
|
|
ff20b5ffb3 | ||
|
|
49290f94d7 | ||
|
|
9b114fcb3d | ||
|
|
e507b3f685 | ||
|
|
6d0c765f4d | ||
|
|
8086242d38 | ||
|
|
3c3680632b | ||
|
|
dd5f05b291 | ||
|
|
36a60c09da | ||
|
|
9340ed5d9f | ||
|
|
8959da63a1 | ||
|
|
cb7b4863f5 | ||
|
|
e84a511676 | ||
|
|
aee2b48ffd | ||
|
|
a7652ba72d | ||
|
|
0ce5225bde | ||
|
|
c494685211 | ||
|
|
599625e21a | ||
|
|
5c4404839b | ||
|
|
f699850d30 | ||
|
|
e8d9871d3e | ||
|
|
473253b5ab | ||
|
|
89cc9e9113 | ||
|
|
ec5a816b68 | ||
|
|
7eb082d382 | ||
|
|
1af7ab0bde | ||
|
|
7ba40af80d | ||
|
|
f434e3c93d | ||
|
|
4b6523dbd8 | ||
|
|
d67828b78f | ||
|
|
db55205127 | ||
|
|
01ff0eb042 | ||
|
|
438ea6a836 | ||
|
|
07f48b6213 | ||
|
|
5630cfd253 | ||
|
|
f543ea6efa | ||
|
|
5a2c77978b | ||
|
|
0f7ec8f71e | ||
|
|
dbb6f3b751 | ||
|
|
dd287f8c92 | ||
|
|
07c459258a | ||
|
|
3522fdd223 | ||
|
|
651cbbdbca | ||
|
|
ba4534fdbd | ||
|
|
441f99edeb | ||
|
|
2e145dd927 | ||
|
|
985f5c7718 | ||
|
|
2b5ef28789 | ||
|
|
76940691b1 | ||
|
|
4b346dd369 | ||
|
|
7480950a1b | ||
|
|
64ada25e9c | ||
|
|
de6322dd87 | ||
|
|
49b472429d | ||
|
|
d57282c770 | ||
|
|
a41948fe37 | ||
|
|
ffd4e9e377 | ||
|
|
ef9c4fc243 | ||
|
|
1f030333ae | ||
|
|
707df6af72 |
No files matched your search
@@ -0,0 +1,368 @@
|
||||
---
|
||||
name: Anchor Sequence Optimization
|
||||
overview: Replace the current multiprocessing-based anchor sequence algorithm with an efficient single-threaded implementation using hash-based n-gram indexing, greedy longest-first matching, and simplified scoring to achieve sub-30-second performance in both local and cloud environments.
|
||||
todos:
|
||||
- id: create-golden-fixtures
|
||||
content: Create script to generate 10 golden test fixtures from representative song data (varying lengths/complexity), run current algorithm to capture expected anchor outputs
|
||||
status: completed
|
||||
- id: create-golden-test
|
||||
content: Create test_anchor_sequence_golden.py with regression tests that verify anchor finder output matches fixtures, ensure tests pass with current implementation before any changes
|
||||
status: completed
|
||||
- id: remove-multiprocessing
|
||||
content: Remove multiprocessing.Pool from find_anchors() and _remove_overlapping_sequences(), replace with single-threaded loops
|
||||
status: completed
|
||||
- id: build-ngram-index
|
||||
content: Add _build_ngram_index() method to create hash-based lookup table for O(1) n-gram matching
|
||||
status: completed
|
||||
- id: greedy-skip
|
||||
content: Add covered_positions set tracking and skip logic to avoid processing overlapping n-grams
|
||||
status: completed
|
||||
- id: simple-scoring
|
||||
content: Add _simple_score_anchor() with rule-based heuristics, make spaCy scoring optional via use_nlp_scoring flag
|
||||
status: completed
|
||||
- id: optimize-overlap
|
||||
content: Replace O(n^2) overlap removal with O(n log n) sorted single-pass algorithm
|
||||
status: completed
|
||||
- id: add-perf-tests
|
||||
content: Add performance benchmark test asserting <30 second completion for all fixture sizes
|
||||
status: pending
|
||||
---
|
||||
|
||||
# Anchor Sequence Algorithm Optimization
|
||||
|
||||
## Problem Analysis
|
||||
|
||||
The current `AnchorSequenceFinder` has critical performance issues:
|
||||
|
||||
### Root Causes of Hanging
|
||||
|
||||
1. **Multiprocessing Pool with fork()**: Python's `Pool()` uses `fork()` on Linux, which causes deadlocks in Cloud Run containers
|
||||
2. **spaCy model loading in forked processes**: The `PhraseAnalyzer` loads spaCy's `en_core_web_sm` model, which can hang when forked
|
||||
3. **Pool creation blocking**: The hang occurs before job submission, so timeout checks never trigger
|
||||
|
||||
### Algorithmic Inefficiencies
|
||||
|
||||
1. **O(n^2) n-gram matching**: Linear search through references for each n-gram
|
||||
2. **Redundant work**: Processes all n-gram lengths (3 to ~30) even though longer matches are preferred
|
||||
3. **O(n^2) overlap removal**: Each anchor checked against all previous anchors
|
||||
4. **Expensive phrase scoring**: spaCy NLP called twice per anchor (phrase + context)
|
||||
|
||||
## Proposed Solution: Hash-Based Greedy Algorithm
|
||||
|
||||
### Core Algorithm Changes
|
||||
|
||||
**1. Hash-Based N-gram Index (O(1) lookup instead of O(n))**
|
||||
|
||||
- Pre-build a dictionary mapping n-grams to their positions in each reference
|
||||
- Use tuple of cleaned words as key for O(1) lookup
|
||||
- Build index once during preprocessing
|
||||
|
||||
**2. Greedy Longest-First Matching**
|
||||
|
||||
- Process n-gram lengths from longest to shortest (current behavior)
|
||||
- BUT: Skip positions already covered by longer anchors immediately
|
||||
- Use a "covered positions" set to avoid redundant work
|
||||
|
||||
**3. Single-Threaded Processing**
|
||||
|
||||
- Remove all `multiprocessing.Pool` usage
|
||||
- Use efficient algorithms instead of parallelism
|
||||
- Works identically in local and cloud environments
|
||||
|
||||
**4. Simplified Phrase Scoring (Optional)**
|
||||
|
||||
- Make spaCy-based scoring optional via configuration flag
|
||||
- Default to rule-based heuristic scoring:
|
||||
- Prefer sequences starting/ending at word boundaries
|
||||
- Prefer sequences matching more reference sources
|
||||
- Prefer longer sequences
|
||||
- Option to enable full NLP scoring for higher-quality results offline
|
||||
|
||||
**5. O(n log n) Overlap Removal**
|
||||
|
||||
- Sort anchors by position
|
||||
- Single pass with interval tracking using sorted list
|
||||
- Skip anchors that overlap with any already-selected anchor
|
||||
|
||||
### New Data Structures
|
||||
|
||||
```python
|
||||
# N-gram index: maps (word1, word2, ...) -> {source: [positions]}
|
||||
ngram_index: Dict[Tuple[str, ...], Dict[str, List[int]]]
|
||||
|
||||
# Covered positions in transcription (for greedy skip)
|
||||
covered_trans_positions: Set[int]
|
||||
```
|
||||
|
||||
### Performance Targets
|
||||
|
||||
- **Target**: Complete in under 30 seconds for any song
|
||||
- **Typical song**: 200-400 words, 3 reference sources
|
||||
- **Expected performance**: 1-5 seconds for typical inputs
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Execution Order (Critical)
|
||||
|
||||
1. **Phase 0**: Create golden test fixtures and verify tests pass with current code
|
||||
2. **Phase 1-5**: Implement optimizations incrementally, running golden tests after each phase
|
||||
3. **Final**: Add performance benchmarks
|
||||
|
||||
### Files to Create/Modify
|
||||
|
||||
- `lyrics_transcriber_temp/scripts/generate_anchor_fixtures.py` (new)
|
||||
- `lyrics_transcriber_temp/tests/fixtures/anchor_golden/*.json` (new)
|
||||
- `lyrics_transcriber_temp/tests/unit/correction/test_anchor_sequence_golden.py` (new)
|
||||
- `lyrics_transcriber_temp/lyrics_transcriber/correction/anchor_sequence.py` (modify)
|
||||
|
||||
### Key Changes
|
||||
|
||||
**Phase 0: Golden Test Setup** (MUST complete and pass before any algorithm changes)
|
||||
|
||||
- Create fixture generation script with 10 representative test cases
|
||||
- Run script to generate fixtures using current implementation
|
||||
- Create golden test file that loads fixtures and verifies outputs
|
||||
- Verify all golden tests pass with current code
|
||||
|
||||
**Phase 1: Remove Multiprocessing**
|
||||
|
||||
- Replace `Pool().apply_async()` with direct function calls
|
||||
- Remove all pool creation, cleanup, and timeout handling
|
||||
- Single-threaded loop through n-gram lengths
|
||||
|
||||
**Phase 2: Add Hash-Based Index**
|
||||
|
||||
- New method `_build_ngram_index()` that builds lookup table during preprocessing
|
||||
- Modify `_find_matching_sources()` to use O(1) index lookup
|
||||
- Index built once, shared across all n-gram length iterations
|
||||
|
||||
**Phase 3: Greedy Position Skipping**
|
||||
|
||||
- Track covered transcription positions in a set
|
||||
- Skip n-grams that start in already-covered positions
|
||||
- Reduces redundant work significantly for songs with good matches
|
||||
|
||||
**Phase 4: Simplify Scoring**
|
||||
|
||||
- Add `use_nlp_scoring: bool = False` parameter to constructor
|
||||
- New method `_simple_score_anchor()` using rule-based heuristics
|
||||
- Keep spaCy scoring available for offline/high-quality mode
|
||||
|
||||
**Phase 5: Optimize Overlap Removal**
|
||||
|
||||
- Sort anchors by (start_position, -length)
|
||||
- Single pass: keep anchor if no overlap with last kept anchor
|
||||
- O(n log n) total instead of O(n^2)
|
||||
|
||||
## API Compatibility
|
||||
|
||||
The public interface remains unchanged:
|
||||
|
||||
- `find_anchors(transcribed, references, transcription_result) -> List[ScoredAnchor]`
|
||||
- `find_gaps(transcribed, anchors, references, transcription_result) -> List[GapSequence]`
|
||||
|
||||
New optional constructor parameters:
|
||||
|
||||
- `use_nlp_scoring: bool = False` - Enable spaCy-based phrase scoring (slower but higher quality)
|
||||
- Timeout parameter retained but less critical with efficient algorithm
|
||||
|
||||
## Fallback Strategy
|
||||
|
||||
If performance is still insufficient for edge cases:
|
||||
|
||||
1. **Early termination**: Stop after finding sufficient coverage (e.g., 80% of words anchored)
|
||||
2. **Adaptive n-gram range**: Start with medium lengths (8-15), expand only if needed
|
||||
3. **Sampling**: For very long songs, sample n-gram positions instead of exhaustive search
|
||||
|
||||
## Phase 0: Golden Test Setup (BEFORE any algorithm changes)
|
||||
|
||||
### Goal
|
||||
|
||||
Create regression tests using REAL song data from local cache to verify the optimized algorithm produces equivalent results.
|
||||
|
||||
### Cache File Structure (Understanding)
|
||||
|
||||
The local cache at `~/lyrics-transcriber-cache/` contains:
|
||||
|
||||
- **Reference lyrics**: `{provider}_{hash}_converted.json` where provider is spotify/genius/lrclib/musixmatch
|
||||
- Hash is based on (artist, title) lookup - same song shares same hash across providers
|
||||
- `_raw.json` contains metadata (track name, artist)
|
||||
- `_converted.json` contains LyricsData structure with segments/words
|
||||
- **Transcriptions**: `audioshake_{hash}_converted.json`
|
||||
- Hash is based on audio file content (different from reference hash)
|
||||
- Contains TranscriptionData structure
|
||||
- **Anchor results**: `anchors_{hash}.json`
|
||||
- Hash is computed from combined inputs (transcription + all references)
|
||||
|
||||
### Manual Song Selection
|
||||
|
||||
Identify 10 songs from cache with good coverage. Search raw files for track names:
|
||||
|
||||
### Finding Song Mappings
|
||||
|
||||
To link audioshake transcriptions to reference lyrics, search cache files:
|
||||
|
||||
```bash
|
||||
# Find songs by searching raw files for track names
|
||||
grep -h "trackName" ~/lyrics-transcriber-cache/spotify_*_raw.json | head -20
|
||||
|
||||
# Find audioshake files for a known song (e.g., search for filename patterns)
|
||||
ls ~/lyrics-transcriber-cache/audioshake_*_raw.json | head -20
|
||||
|
||||
# Search converted files for specific lyrics text
|
||||
grep -l "Napoleon did surrender" ~/lyrics-transcriber-cache/*_converted.json
|
||||
```
|
||||
|
||||
**Target song list** (reference_hash to be paired with audioshake_hash):
|
||||
|
||||
1. ABBA - Waterloo (refs: 3365f6a1e42388d84bcbfc0ac6f8aa6c) - short, repetitive chorus
|
||||
2. The Format - Time Bomb (refs: 3a3802ec28c4ab6689909b7ef784bb6f) - medium length
|
||||
3. Rancid - Time Bomb (refs: d02d20b854cbc6b890f274bd6c278752) - fast punk lyrics
|
||||
4. Minutemen - Storm in My House (refs: bc625704b26b307d8680ae5235b74994) - short
|
||||
5. + 6 more songs to be identified during fixture generation by exploring cache
|
||||
|
||||
### Fixture Generation Script
|
||||
|
||||
Create `lyrics_transcriber_temp/scripts/generate_anchor_fixtures.py`:
|
||||
|
||||
```python
|
||||
# Input: List of (name, audioshake_hash, reference_hash) tuples
|
||||
# For each:
|
||||
# 1. Load audioshake_{hash}_converted.json as transcription
|
||||
# 2. Load {spotify,genius,lrclib}_{hash}_converted.json as references
|
||||
# 3. Run current AnchorSequenceFinder
|
||||
# 4. Save complete fixture with inputs + outputs
|
||||
```
|
||||
|
||||
The script will:
|
||||
|
||||
1. Accept a config file listing the 10 song mappings
|
||||
2. Load real LyricsData from cache files
|
||||
3. Run the CURRENT AnchorSequenceFinder implementation
|
||||
4. Save fixtures that capture the real behavior
|
||||
|
||||
### Fixture Format
|
||||
|
||||
```
|
||||
lyrics_transcriber_temp/tests/fixtures/anchor_golden/
|
||||
- waterloo_abba.json
|
||||
- time_bomb_format.json
|
||||
- time_bomb_rancid.json
|
||||
- ...
|
||||
```
|
||||
|
||||
Each fixture contains REAL data:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "waterloo_abba",
|
||||
"description": "ABBA - Waterloo, short song with repetitive chorus",
|
||||
"source_hashes": {
|
||||
"audioshake": "1654b7c0755fad7f23bc888308f274de",
|
||||
"references": "3365f6a1e42388d84bcbfc0ac6f8aa6c"
|
||||
},
|
||||
"transcription": {
|
||||
"segments": [...], // Full LyricsData from audioshake cache
|
||||
"word_count": 150
|
||||
},
|
||||
"references": {
|
||||
"spotify": { "segments": [...] },
|
||||
"genius": { "segments": [...] },
|
||||
"lrclib": { "segments": [...] }
|
||||
},
|
||||
"expected_anchors": [
|
||||
{
|
||||
"text": "My my at Waterloo Napoleon did surrender",
|
||||
"transcription_position": 0,
|
||||
"length": 7,
|
||||
"reference_positions": {"spotify": 0, "genius": 0, "lrclib": 0},
|
||||
"confidence": 1.0
|
||||
},
|
||||
// ... all anchors from current implementation
|
||||
],
|
||||
"expected_anchor_count": 25,
|
||||
"expected_word_coverage": 142,
|
||||
"expected_coverage_percent": 94.6
|
||||
}
|
||||
```
|
||||
|
||||
### Golden Test
|
||||
|
||||
Create `lyrics_transcriber_temp/tests/unit/correction/test_anchor_sequence_golden.py`:
|
||||
|
||||
```python
|
||||
import pytest
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
FIXTURES_DIR = Path(__file__).parent.parent.parent / "fixtures" / "anchor_golden"
|
||||
|
||||
@pytest.fixture(params=list(FIXTURES_DIR.glob("*.json")))
|
||||
def golden_fixture(request):
|
||||
with open(request.param) as f:
|
||||
return json.load(f)
|
||||
|
||||
def test_anchor_count_matches(golden_fixture, anchor_finder):
|
||||
"""Anchor count should be within 10% of expected."""
|
||||
anchors = run_anchor_finder(golden_fixture)
|
||||
expected = golden_fixture["expected_anchor_count"]
|
||||
assert abs(len(anchors) - expected) <= expected * 0.1
|
||||
|
||||
def test_key_anchors_match(golden_fixture, anchor_finder):
|
||||
"""First, last, and longest anchors should match exactly."""
|
||||
anchors = run_anchor_finder(golden_fixture)
|
||||
expected = golden_fixture["expected_anchors"]
|
||||
# Compare first anchor
|
||||
assert anchors[0].text == expected[0]["text"]
|
||||
# Compare positions match
|
||||
...
|
||||
|
||||
def test_word_coverage(golden_fixture, anchor_finder):
|
||||
"""Total word coverage should be within 5% of expected."""
|
||||
anchors = run_anchor_finder(golden_fixture)
|
||||
coverage = calculate_coverage(anchors, golden_fixture)
|
||||
expected = golden_fixture["expected_coverage_percent"]
|
||||
assert abs(coverage - expected) <= 5.0
|
||||
|
||||
def test_no_invalid_positions(golden_fixture, anchor_finder):
|
||||
"""All anchor positions should be valid."""
|
||||
anchors = run_anchor_finder(golden_fixture)
|
||||
word_count = golden_fixture["transcription"]["word_count"]
|
||||
for anchor in anchors:
|
||||
assert 0 <= anchor.transcription_position < word_count
|
||||
```
|
||||
|
||||
### Validation Steps
|
||||
|
||||
1. **Generate fixtures from current implementation**:
|
||||
```bash
|
||||
cd lyrics_transcriber_temp
|
||||
python scripts/generate_anchor_fixtures.py
|
||||
```
|
||||
|
||||
2. **Verify fixtures were created**:
|
||||
```bash
|
||||
ls tests/fixtures/anchor_golden/
|
||||
# Should show 10 .json files
|
||||
```
|
||||
|
||||
3. **Run golden tests with CURRENT implementation**:
|
||||
```bash
|
||||
pytest tests/unit/correction/test_anchor_sequence_golden.py -v
|
||||
```
|
||||
|
||||
4. **All tests MUST pass before any algorithm changes**
|
||||
|
||||
5. **After each optimization phase, re-run golden tests**:
|
||||
```bash
|
||||
pytest tests/unit/correction/test_anchor_sequence_golden.py -v
|
||||
# Must still pass!
|
||||
```
|
||||
|
||||
|
||||
## Testing (Post-Optimization)
|
||||
|
||||
- Golden tests verify correctness (same outputs as before)
|
||||
- Existing unit tests verify API compatibility
|
||||
- Performance benchmark test with 30-second timeout assertion
|
||||
- Test with various song lengths via golden fixtures
|
||||
@@ -0,0 +1,717 @@
|
||||
---
|
||||
name: Instrumental Review UI
|
||||
overview: Add an intelligent instrumental selection workflow with audio analysis, waveform visualization, playback controls, and the ability to mute sections of backing vocals to create custom instrumentals. This replaces the current text-only CLI prompt with a web UI that works for both local and remote workflows, with shared core logic for DRY/maintainable code.
|
||||
todos:
|
||||
- id: shared-core-module
|
||||
content: Create karaoke_gen/instrumental_review/ shared module with AudioAnalyzer and AudioEditor classes (used by both local and remote)
|
||||
status: completed
|
||||
- id: shared-core-tests
|
||||
content: Add comprehensive unit tests for shared module with >80% coverage (tests/unit/test_instrumental_review.py)
|
||||
status: completed
|
||||
- id: backend-services
|
||||
content: Create backend service wrappers that use shared core module (audio_analysis_service.py, audio_editing_service.py)
|
||||
status: completed
|
||||
- id: backend-services-tests
|
||||
content: Add unit tests for backend services with mocked GCS interactions (backend/tests/test_audio_services.py)
|
||||
status: completed
|
||||
- id: api-endpoints
|
||||
content: Add new API endpoints for analysis, audio streaming, and custom instrumental creation
|
||||
status: completed
|
||||
- id: api-endpoints-tests
|
||||
content: Add API endpoint tests with >80% coverage (backend/tests/test_instrumental_api.py)
|
||||
status: completed
|
||||
- id: render-worker-integration
|
||||
content: Integrate audio analysis into render_video_worker before transitioning to awaiting_instrumental_selection
|
||||
status: completed
|
||||
- id: render-worker-tests
|
||||
content: Add/update render_video_worker tests to cover new analysis integration
|
||||
status: completed
|
||||
- id: frontend-components
|
||||
content: Create React components (WaveformDisplay, AudioPlayer, RegionSelector, InstrumentalReviewPage)
|
||||
status: completed
|
||||
- id: local-cli-integration
|
||||
content: Update local gen_cli.py to use shared core module with local file paths (feature parity with remote)
|
||||
status: completed
|
||||
- id: local-cli-review-server
|
||||
content: Add lightweight review server to local CLI for browser-based review (similar to lyrics review pattern)
|
||||
status: completed
|
||||
- id: remote-cli-update
|
||||
content: Update remote_cli.py to use new analysis API and open browser for review
|
||||
status: completed
|
||||
- id: cli-tests
|
||||
content: Add CLI tests verifying feature parity between local and remote modes
|
||||
status: completed
|
||||
- id: integration-tests
|
||||
content: Add end-to-end integration tests for the full instrumental review workflow
|
||||
status: completed
|
||||
---
|
||||
|
||||
# Intelligent Instrumental Selection Plan
|
||||
|
||||
## Current State
|
||||
|
||||
The instrumental selection workflow currently:
|
||||
|
||||
- Presents a simple 1/2 choice in the CLI (clean vs. with_backing)
|
||||
- Requires manual inspection in Audacity to evaluate backing vocals quality
|
||||
- Cannot handle the common "Scenario 3" where backing vocals need partial muting
|
||||
- Has no audio preview capability in remote mode
|
||||
|
||||
## Target Architecture
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────────────────────┐
|
||||
│ INSTRUMENTAL REVIEW WORKFLOW │
|
||||
├────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 1. AUDIO ANALYSIS (automatic after render_video completes) │
|
||||
│ │ │
|
||||
│ ├── Analyze backing vocals stem for amplitude │
|
||||
│ ├── Detect audible segments (> -40dB threshold) │
|
||||
│ ├── Generate waveform visualization image │
|
||||
│ └── Store analysis results in job state_data │
|
||||
│ │
|
||||
│ 2. AUTO-DECISION CHECK │
|
||||
│ │ │
|
||||
│ └── IF no audible segments detected: │
|
||||
│ ├── Set recommended_selection = "clean" │
|
||||
│ └── Skip to step 4 (or show minimal UI with suggestion) │
|
||||
│ │
|
||||
│ 3. HUMAN REVIEW (Web UI) │
|
||||
│ │ │
|
||||
│ ├── Display waveform with time axis │
|
||||
│ ├── Show audible segment markers │
|
||||
│ ├── Audio playback with waveform-click seeking │
|
||||
│ ├── Toggle between: clean / with_backing / preview-custom │
|
||||
│ ├── Select regions to mute (click/drag on waveform) │
|
||||
│ └── Create custom instrumental if regions selected │
|
||||
│ │
|
||||
│ 4. SELECTION SUBMISSION │
|
||||
│ │ │
|
||||
│ └── POST /api/jobs/{job_id}/select-instrumental │
|
||||
│ with selection = "clean" | "with_backing" | "custom" │
|
||||
│ │
|
||||
└────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Architecture: Shared Core for DRY/SOLID
|
||||
|
||||
To ensure feature parity between local and remote CLIs while keeping code maintainable, we use a **shared core module** pattern:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ SHARED CORE ARCHITECTURE │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ karaoke_gen/instrumental_review/ <── SHARED CORE MODULE │
|
||||
│ ├── __init__.py │
|
||||
│ ├── analyzer.py # AudioAnalyzer class (pure Python, no GCS) │
|
||||
│ ├── editor.py # AudioEditor class (pure Python, no GCS) │
|
||||
│ ├── waveform.py # WaveformGenerator class (matplotlib) │
|
||||
│ └── models.py # Pydantic models: AnalysisResult, MuteRegion │
|
||||
│ │
|
||||
│ ┌─────────────────────┐ ┌──────────────────────┐ │
|
||||
│ │ LOCAL CLI │ │ REMOTE BACKEND │ │
|
||||
│ │ (karaoke-gen) │ │ (Cloud Run) │ │
|
||||
│ ├─────────────────────┤ ├──────────────────────┤ │
|
||||
│ │ │ │ │ │
|
||||
│ │ gen_cli.py │ │ audio_analysis_ │ │
|
||||
│ │ └── uses shared ───┼──────────────┼──► service.py │ │
|
||||
│ │ core directly │ │ └── wraps shared│ │
|
||||
│ │ with local │ │ core with │ │
|
||||
│ │ file paths │ │ GCS I/O │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ local_review_ │ │ audio_editing_ │ │
|
||||
│ │ server.py │ │ service.py │ │
|
||||
│ │ └── serves same ───┼──────────────┼──► (same pattern) │ │
|
||||
│ │ React UI │ │ │ │
|
||||
│ │ │ │ │ │
|
||||
│ └─────────────────────┘ └──────────────────────┘ │
|
||||
│ │
|
||||
│ BENEFITS: │
|
||||
│ ✓ Single source of truth for analysis logic │
|
||||
│ ✓ Bug fixes apply to both local and remote │
|
||||
│ ✓ Easy to test core logic without GCS mocks │
|
||||
│ ✓ Backend services are thin wrappers (Single Responsibility) │
|
||||
│ ✓ Same React UI served by both local server and Cloud Run │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Shared Core Module Design
|
||||
|
||||
```python
|
||||
# karaoke_gen/instrumental_review/analyzer.py
|
||||
class AudioAnalyzer:
|
||||
"""
|
||||
Analyzes audio files for backing vocals content.
|
||||
Pure Python - no cloud dependencies. Works with local file paths.
|
||||
"""
|
||||
def __init__(self, silence_threshold_db: float = -40.0,
|
||||
min_segment_duration_ms: int = 100):
|
||||
self.silence_threshold_db = silence_threshold_db
|
||||
self.min_segment_duration_ms = min_segment_duration_ms
|
||||
|
||||
def analyze(self, audio_path: str) -> AnalysisResult:
|
||||
"""Analyze audio file and return structured result."""
|
||||
...
|
||||
|
||||
def get_amplitude_envelope(self, audio_path: str,
|
||||
window_ms: int = 100) -> List[float]:
|
||||
"""Get amplitude envelope for waveform rendering."""
|
||||
...
|
||||
|
||||
# karaoke_gen/instrumental_review/editor.py
|
||||
class AudioEditor:
|
||||
"""
|
||||
Creates custom instrumentals by muting regions of backing vocals.
|
||||
Pure Python - no cloud dependencies.
|
||||
"""
|
||||
def create_custom_instrumental(
|
||||
self,
|
||||
clean_instrumental_path: str,
|
||||
backing_vocals_path: str,
|
||||
mute_regions: List[MuteRegion],
|
||||
output_path: str
|
||||
) -> str:
|
||||
"""Create custom instrumental with muted regions."""
|
||||
...
|
||||
|
||||
# karaoke_gen/instrumental_review/waveform.py
|
||||
class WaveformGenerator:
|
||||
"""Generates waveform visualization images."""
|
||||
def generate(self, audio_path: str, output_path: str,
|
||||
segments: List[AudibleSegment] = None,
|
||||
width: int = 1200, height: int = 200) -> str:
|
||||
"""Generate waveform PNG with optional segment highlighting."""
|
||||
...
|
||||
```
|
||||
|
||||
## Implementation Components
|
||||
|
||||
### 1. Shared Core Module (karaoke_gen/instrumental_review/)
|
||||
|
||||
Create the shared module first - this is the foundation:
|
||||
|
||||
```python
|
||||
# karaoke_gen/instrumental_review/models.py
|
||||
from pydantic import BaseModel
|
||||
from typing import List, Optional
|
||||
from enum import Enum
|
||||
|
||||
class RecommendedSelection(str, Enum):
|
||||
CLEAN = "clean"
|
||||
WITH_BACKING = "with_backing"
|
||||
REVIEW_NEEDED = "review_needed"
|
||||
|
||||
class AudibleSegment(BaseModel):
|
||||
start_seconds: float
|
||||
end_seconds: float
|
||||
duration_seconds: float
|
||||
avg_amplitude_db: float
|
||||
|
||||
class MuteRegion(BaseModel):
|
||||
start_seconds: float
|
||||
end_seconds: float
|
||||
|
||||
class AnalysisResult(BaseModel):
|
||||
has_audible_content: bool
|
||||
total_duration_seconds: float
|
||||
audible_segments: List[AudibleSegment]
|
||||
recommended_selection: RecommendedSelection
|
||||
waveform_data: Optional[List[float]] = None # For JSON API response
|
||||
```
|
||||
|
||||
### 2. Backend Service Wrappers
|
||||
|
||||
Create `backend/services/audio_analysis_service.py`:
|
||||
|
||||
```python
|
||||
# Key functionality:
|
||||
- analyze_backing_vocals(audio_path) -> AnalysisResult
|
||||
- Uses pydub for audio loading
|
||||
- Calculates RMS amplitude in dB across time windows
|
||||
- Detects audible segments (contiguous regions > threshold)
|
||||
- Returns: {
|
||||
"has_audible_content": bool,
|
||||
"total_duration_seconds": float,
|
||||
"audible_segments": [
|
||||
{"start_seconds": float, "end_seconds": float,
|
||||
"duration_seconds": float, "avg_amplitude_db": float}
|
||||
],
|
||||
"recommended_selection": "clean" | "with_backing" | "review_needed"
|
||||
}
|
||||
|
||||
- generate_waveform_image(audio_path, output_path, segments=None) -> str
|
||||
- Uses matplotlib to render amplitude envelope over time
|
||||
- Highlights audible segments with different colors
|
||||
- Returns path to generated PNG image
|
||||
```
|
||||
|
||||
### 2. Audio Editing Service
|
||||
|
||||
Create `backend/services/audio_editing_service.py`:
|
||||
|
||||
```python
|
||||
# Key functionality:
|
||||
- create_custom_instrumental(
|
||||
clean_instrumental_path: str,
|
||||
backing_vocals_path: str,
|
||||
mute_regions: List[Tuple[float, float]], # [(start_s, end_s), ...]
|
||||
output_path: str
|
||||
) -> str
|
||||
- Loads backing vocals
|
||||
- Applies silence to mute_regions
|
||||
- Mixes with clean instrumental
|
||||
- Saves as new FLAC file
|
||||
```
|
||||
|
||||
### 3. Backend API Endpoints
|
||||
|
||||
Add to `backend/api/routes/jobs.py`:
|
||||
|
||||
```python
|
||||
GET /api/jobs/{job_id}/instrumental-analysis
|
||||
# Returns analysis data + waveform URL
|
||||
|
||||
GET /api/jobs/{job_id}/audio-stream/{stem_type}
|
||||
# Stream audio for playback (clean, backing_vocals, combined)
|
||||
|
||||
POST /api/jobs/{job_id}/create-custom-instrumental
|
||||
# Body: {"mute_regions": [[0.5, 2.3], [45.0, 48.5]]}
|
||||
# Creates custom instrumental, updates job state
|
||||
|
||||
POST /api/jobs/{job_id}/select-instrumental
|
||||
# Extended to accept: "clean" | "with_backing" | "custom"
|
||||
```
|
||||
|
||||
### 4. Frontend Web UI
|
||||
|
||||
Create `frontend/app/jobs/[id]/instrumental-review/page.tsx`:
|
||||
|
||||
```
|
||||
Components needed:
|
||||
- WaveformDisplay: SVG/Canvas waveform with time axis
|
||||
- AudioPlayer: HTML5 audio with controls + waveform sync
|
||||
- RegionSelector: Click/drag to select mute regions
|
||||
- InstrumentalOptions: Radio buttons for clean/backing/custom
|
||||
- SegmentList: Table showing detected audible segments
|
||||
```
|
||||
|
||||
Key interactions:
|
||||
|
||||
- Click on waveform to seek playback position
|
||||
- Drag on waveform to select mute region
|
||||
- Toggle audio source (clean/backing/preview)
|
||||
- Submit selection
|
||||
|
||||
### 5. Backend Worker Updates
|
||||
|
||||
Modify `backend/workers/render_video_worker.py`:
|
||||
|
||||
```python
|
||||
# After video render completes, before transitioning to AWAITING_INSTRUMENTAL_SELECTION:
|
||||
1. Download backing_vocals stem
|
||||
2. Run audio analysis
|
||||
3. Generate waveform image
|
||||
4. Upload waveform to GCS
|
||||
5. Store analysis results in job.state_data
|
||||
6. If no audible content: set recommended_selection = "clean"
|
||||
7. Transition to AWAITING_INSTRUMENTAL_SELECTION
|
||||
```
|
||||
|
||||
### 6. CLI Updates
|
||||
|
||||
Modify `karaoke_gen/utils/remote_cli.py`:
|
||||
|
||||
```python
|
||||
def handle_instrumental_selection(self, job_id: str):
|
||||
# Get analysis data
|
||||
analysis = self.client.get_instrumental_analysis(job_id)
|
||||
|
||||
# If no backing vocals detected, suggest auto-select
|
||||
if analysis['recommended_selection'] == 'clean':
|
||||
if self.config.non_interactive:
|
||||
# Auto-select clean
|
||||
self.client.select_instrumental(job_id, 'clean')
|
||||
return
|
||||
else:
|
||||
# Show suggestion, offer to open browser for verification
|
||||
self.logger.info("No backing vocals detected - recommending clean instrumental")
|
||||
# Open browser anyway so user can verify if desired
|
||||
|
||||
# Open browser to instrumental review UI
|
||||
self.open_instrumental_review_ui(job_id)
|
||||
|
||||
# Poll until selection made
|
||||
while job.status == 'awaiting_instrumental_selection':
|
||||
time.sleep(self.config.poll_interval)
|
||||
```
|
||||
|
||||
### 7. Local CLI Integration (Feature Parity)
|
||||
|
||||
The local CLI (`karaoke-gen`) uses the **same shared core module** as the backend:
|
||||
|
||||
```python
|
||||
# karaoke_gen/utils/gen_cli.py - Updated instrumental selection flow
|
||||
|
||||
def handle_instrumental_selection(self, output_dir: str, base_name: str):
|
||||
"""Handle instrumental selection using shared core module."""
|
||||
from karaoke_gen.instrumental_review import AudioAnalyzer, WaveformGenerator
|
||||
|
||||
# Paths to local files (already exist from audio separation)
|
||||
backing_vocals_path = os.path.join(output_dir, "stems",
|
||||
f"{base_name} (Backing Vocals ...).flac")
|
||||
clean_instrumental_path = os.path.join(output_dir, "stems",
|
||||
f"{base_name} (Instrumental ...).flac")
|
||||
|
||||
# Use SAME analyzer as backend
|
||||
analyzer = AudioAnalyzer(silence_threshold_db=-40.0)
|
||||
analysis = analyzer.analyze(backing_vocals_path)
|
||||
|
||||
# Generate waveform locally
|
||||
waveform_path = os.path.join(output_dir, "backing_vocals_waveform.png")
|
||||
WaveformGenerator().generate(backing_vocals_path, waveform_path,
|
||||
analysis.audible_segments)
|
||||
|
||||
if not analysis.has_audible_content:
|
||||
self.logger.info("No backing vocals detected - using clean instrumental")
|
||||
if not self.non_interactive:
|
||||
# Offer to open browser for verification
|
||||
...
|
||||
return "clean"
|
||||
|
||||
# Start local review server (serves same React UI)
|
||||
from karaoke_gen.instrumental_review.server import InstrumentalReviewServer
|
||||
server = InstrumentalReviewServer(
|
||||
output_dir=output_dir,
|
||||
base_name=base_name,
|
||||
analysis=analysis,
|
||||
waveform_path=waveform_path,
|
||||
)
|
||||
server.start_and_open_browser() # Blocking until user submits selection
|
||||
|
||||
return server.get_selection()
|
||||
```
|
||||
|
||||
### 8. Local Review Server
|
||||
|
||||
Create `karaoke_gen/instrumental_review/server.py`:
|
||||
|
||||
```python
|
||||
"""
|
||||
Lightweight local server for instrumental review.
|
||||
Serves the same React frontend used by the cloud backend.
|
||||
Similar pattern to LyricsTranscriber's ReviewServer.
|
||||
"""
|
||||
from http.server import HTTPServer, SimpleHTTPRequestHandler
|
||||
import json
|
||||
import threading
|
||||
|
||||
class InstrumentalReviewServer:
|
||||
"""
|
||||
Local HTTP server for instrumental review UI.
|
||||
|
||||
Endpoints:
|
||||
- GET /api/analysis - Returns analysis data
|
||||
- GET /api/audio/{type} - Streams audio files
|
||||
- GET /api/waveform - Returns waveform image
|
||||
- POST /api/create-custom - Creates custom instrumental
|
||||
- POST /api/select - Submits selection
|
||||
- GET /* - Serves React frontend static files
|
||||
"""
|
||||
def __init__(self, output_dir: str, base_name: str,
|
||||
analysis: AnalysisResult, waveform_path: str):
|
||||
...
|
||||
|
||||
def start_and_open_browser(self, port: int = 8765):
|
||||
"""Start server, open browser, block until selection submitted."""
|
||||
...
|
||||
|
||||
def get_selection(self) -> str:
|
||||
"""Get the user's selection after browser submission."""
|
||||
...
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Test Coverage Requirements
|
||||
|
||||
All new code must meet the project's **70% minimum coverage** requirement, with critical paths targeting **>80%**:
|
||||
|
||||
| Module | Target Coverage | Test File |
|
||||
|
||||
|--------|----------------|-----------|
|
||||
|
||||
| `karaoke_gen/instrumental_review/analyzer.py` | >90% | `tests/unit/test_instrumental_analyzer.py` |
|
||||
|
||||
| `karaoke_gen/instrumental_review/editor.py` | >90% | `tests/unit/test_instrumental_editor.py` |
|
||||
|
||||
| `karaoke_gen/instrumental_review/waveform.py` | >80% | `tests/unit/test_instrumental_waveform.py` |
|
||||
|
||||
| `karaoke_gen/instrumental_review/server.py` | >70% | `tests/unit/test_instrumental_server.py` |
|
||||
|
||||
| `backend/services/audio_analysis_service.py` | >80% | `backend/tests/test_audio_analysis_service.py` |
|
||||
|
||||
| `backend/services/audio_editing_service.py` | >80% | `backend/tests/test_audio_editing_service.py` |
|
||||
|
||||
| `backend/api/routes/jobs.py` (new endpoints) | >80% | `backend/tests/test_instrumental_api.py` |
|
||||
|
||||
### Test Categories
|
||||
|
||||
**1. Unit Tests (Shared Core) - tests/unit/test_instrumental_*.py**
|
||||
|
||||
```python
|
||||
# tests/unit/test_instrumental_analyzer.py
|
||||
class TestAudioAnalyzer:
|
||||
def test_analyze_silent_audio_returns_no_audible_content(self):
|
||||
"""Silent audio should return has_audible_content=False."""
|
||||
|
||||
def test_analyze_audio_with_content_detects_segments(self):
|
||||
"""Audio with content should detect audible segments."""
|
||||
|
||||
def test_segment_detection_respects_threshold(self):
|
||||
"""Segments below threshold should not be detected."""
|
||||
|
||||
def test_segment_merging_for_close_segments(self):
|
||||
"""Close segments should be merged into one."""
|
||||
|
||||
def test_recommended_selection_clean_for_silent(self):
|
||||
"""Silent audio should recommend 'clean'."""
|
||||
|
||||
def test_recommended_selection_review_for_content(self):
|
||||
"""Audio with content should recommend 'review_needed'."""
|
||||
|
||||
# tests/unit/test_instrumental_editor.py
|
||||
class TestAudioEditor:
|
||||
def test_mute_single_region(self):
|
||||
"""Single mute region should be applied correctly."""
|
||||
|
||||
def test_mute_multiple_regions(self):
|
||||
"""Multiple mute regions should all be applied."""
|
||||
|
||||
def test_mute_region_boundaries(self):
|
||||
"""Mute region at start/end of audio should work."""
|
||||
|
||||
def test_output_format_is_flac(self):
|
||||
"""Output should be FLAC format."""
|
||||
|
||||
def test_combines_with_clean_instrumental(self):
|
||||
"""Result should be backing vocals mixed with clean instrumental."""
|
||||
```
|
||||
|
||||
**2. Unit Tests (Backend Services) - backend/tests/test_audio_*_service.py**
|
||||
|
||||
```python
|
||||
# backend/tests/test_audio_analysis_service.py
|
||||
class TestAudioAnalysisService:
|
||||
@pytest.fixture
|
||||
def mock_storage(self):
|
||||
"""Mock GCS storage service."""
|
||||
|
||||
def test_analyze_downloads_from_gcs(self, mock_storage):
|
||||
"""Service should download audio from GCS before analysis."""
|
||||
|
||||
def test_analyze_uploads_waveform_to_gcs(self, mock_storage):
|
||||
"""Service should upload waveform image to GCS."""
|
||||
|
||||
def test_analyze_updates_job_state_data(self, mock_storage):
|
||||
"""Service should store analysis in job.state_data."""
|
||||
```
|
||||
|
||||
**3. API Endpoint Tests - backend/tests/test_instrumental_api.py**
|
||||
|
||||
```python
|
||||
# backend/tests/test_instrumental_api.py
|
||||
class TestInstrumentalAnalysisEndpoint:
|
||||
def test_get_analysis_returns_data(self, client, mock_job):
|
||||
"""GET /instrumental-analysis should return analysis data."""
|
||||
|
||||
def test_get_analysis_requires_correct_status(self, client, mock_job):
|
||||
"""Should return 400 if job not in correct status."""
|
||||
|
||||
class TestAudioStreamEndpoint:
|
||||
def test_stream_backing_vocals(self, client, mock_job):
|
||||
"""Should stream backing vocals audio."""
|
||||
|
||||
def test_stream_clean_instrumental(self, client, mock_job):
|
||||
"""Should stream clean instrumental audio."""
|
||||
|
||||
class TestCreateCustomInstrumental:
|
||||
def test_create_with_mute_regions(self, client, mock_job):
|
||||
"""Should create custom instrumental with mute regions."""
|
||||
|
||||
def test_validates_mute_regions(self, client, mock_job):
|
||||
"""Should validate mute region boundaries."""
|
||||
```
|
||||
|
||||
**4. Integration Tests - tests/integration/test_instrumental_review.py**
|
||||
|
||||
```python
|
||||
# tests/integration/test_instrumental_review_flow.py
|
||||
class TestInstrumentalReviewFlow:
|
||||
"""End-to-end tests for instrumental review workflow."""
|
||||
|
||||
def test_full_flow_clean_selection(self):
|
||||
"""Test complete flow: analysis -> review -> select clean."""
|
||||
|
||||
def test_full_flow_with_backing_selection(self):
|
||||
"""Test complete flow: analysis -> review -> select with_backing."""
|
||||
|
||||
def test_full_flow_custom_creation(self):
|
||||
"""Test complete flow: analysis -> mute regions -> custom."""
|
||||
|
||||
def test_auto_recommend_clean_for_silent_backing(self):
|
||||
"""When no backing vocals, should recommend clean."""
|
||||
```
|
||||
|
||||
**5. CLI Feature Parity Tests - tests/unit/test_cli_feature_parity.py**
|
||||
|
||||
```python
|
||||
# tests/unit/test_cli_feature_parity.py
|
||||
class TestCLIFeatureParity:
|
||||
"""Verify local and remote CLIs have equivalent functionality."""
|
||||
|
||||
def test_both_clis_use_same_analyzer(self):
|
||||
"""Both CLIs should use AudioAnalyzer from shared module."""
|
||||
|
||||
def test_both_clis_support_custom_instrumental(self):
|
||||
"""Both CLIs should support creating custom instrumentals."""
|
||||
|
||||
def test_analysis_results_identical(self):
|
||||
"""Same audio should produce identical analysis from both CLIs."""
|
||||
```
|
||||
|
||||
### Test Data
|
||||
|
||||
Create test fixtures in `tests/data/instrumental_review/`:
|
||||
|
||||
- `silent_backing_vocals.flac` - Silent/near-silent audio (~10s)
|
||||
- `backing_vocals_with_content.flac` - Audio with clear backing vocals (~30s)
|
||||
- `backing_vocals_mixed_quality.flac` - Audio with some good/some bad sections (~30s)
|
||||
|
||||
Use pydub to programmatically generate these in `tests/conftest.py` fixtures.
|
||||
|
||||
## File Changes Summary
|
||||
|
||||
| Category | Files to Create/Modify |
|
||||
|
||||
|----------|------------------------|
|
||||
|
||||
| **Shared Core** | `karaoke_gen/instrumental_review/__init__.py` (new) |
|
||||
|
||||
| Shared Core | `karaoke_gen/instrumental_review/models.py` (new) |
|
||||
|
||||
| Shared Core | `karaoke_gen/instrumental_review/analyzer.py` (new) |
|
||||
|
||||
| Shared Core | `karaoke_gen/instrumental_review/editor.py` (new) |
|
||||
|
||||
| Shared Core | `karaoke_gen/instrumental_review/waveform.py` (new) |
|
||||
|
||||
| Shared Core | `karaoke_gen/instrumental_review/server.py` (new) |
|
||||
|
||||
| **Backend Services** | `backend/services/audio_analysis_service.py` (new) |
|
||||
|
||||
| Backend Services | `backend/services/audio_editing_service.py` (new) |
|
||||
|
||||
| **API** | `backend/api/routes/jobs.py` (modify - add endpoints) |
|
||||
|
||||
| API | `backend/api/routes/instrumental.py` (new - if endpoints get large) |
|
||||
|
||||
| **Workers** | `backend/workers/render_video_worker.py` (modify) |
|
||||
|
||||
| **Models** | `backend/models/job.py` (add analysis fields to state_data) |
|
||||
|
||||
| **Frontend** | `frontend/app/jobs/[id]/instrumental-review/page.tsx` (new) |
|
||||
|
||||
| Frontend | `frontend/components/waveform-display.tsx` (new) |
|
||||
|
||||
| Frontend | `frontend/components/audio-player-with-waveform.tsx` (new) |
|
||||
|
||||
| Frontend | `frontend/components/region-selector.tsx` (new) |
|
||||
|
||||
| **Local CLI** | `karaoke_gen/utils/gen_cli.py` (modify) |
|
||||
|
||||
| **Remote CLI** | `karaoke_gen/utils/remote_cli.py` (modify) |
|
||||
|
||||
| **Tests - Shared** | `tests/unit/test_instrumental_analyzer.py` (new) |
|
||||
|
||||
| Tests - Shared | `tests/unit/test_instrumental_editor.py` (new) |
|
||||
|
||||
| Tests - Shared | `tests/unit/test_instrumental_waveform.py` (new) |
|
||||
|
||||
| Tests - Shared | `tests/unit/test_instrumental_server.py` (new) |
|
||||
|
||||
| **Tests - Backend** | `backend/tests/test_audio_analysis_service.py` (new) |
|
||||
|
||||
| Tests - Backend | `backend/tests/test_audio_editing_service.py` (new) |
|
||||
|
||||
| Tests - Backend | `backend/tests/test_instrumental_api.py` (new) |
|
||||
|
||||
| **Tests - Parity** | `tests/unit/test_cli_feature_parity.py` (new) |
|
||||
|
||||
| **Tests - Integration** | `tests/integration/test_instrumental_review.py` (new) |
|
||||
|
||||
| **Test Data** | `tests/conftest.py` (modify - add fixtures) |
|
||||
|
||||
## Dependencies
|
||||
|
||||
Already available in `pyproject.toml`:
|
||||
|
||||
- `pydub>=0.25.1` - Audio loading and manipulation
|
||||
- `matplotlib>=3` - Waveform visualization
|
||||
- `numpy>=2` - Audio array processing
|
||||
|
||||
Frontend:
|
||||
|
||||
- React/Next.js (existing)
|
||||
- HTML5 Audio API (native)
|
||||
- Canvas or SVG for waveform (native)
|
||||
|
||||
## Implementation Order
|
||||
|
||||
Execute in this order to enable incremental testing:
|
||||
|
||||
### Phase 1: Shared Core Module (Foundation)
|
||||
|
||||
1. Create `karaoke_gen/instrumental_review/` package with models, analyzer, editor, waveform
|
||||
2. Add comprehensive unit tests for shared core (>90% coverage)
|
||||
3. Create test audio fixtures in `tests/conftest.py`
|
||||
|
||||
### Phase 2: Backend Integration
|
||||
|
||||
4. Create backend service wrappers with GCS integration
|
||||
5. Add backend service unit tests with mocked GCS
|
||||
6. Add new API endpoints to `backend/api/routes/jobs.py`
|
||||
7. Add API endpoint tests
|
||||
8. Integrate analysis into `render_video_worker.py`
|
||||
9. Update render worker tests
|
||||
|
||||
### Phase 3: Frontend
|
||||
|
||||
10. Create React components (WaveformDisplay, AudioPlayer, RegionSelector)
|
||||
11. Create instrumental review page
|
||||
12. Test frontend manually with backend
|
||||
|
||||
### Phase 4: CLI Integration
|
||||
|
||||
13. Update `remote_cli.py` to use new analysis API and open browser
|
||||
14. Create local review server `karaoke_gen/instrumental_review/server.py`
|
||||
15. Update `gen_cli.py` to use shared core and local server
|
||||
16. Add CLI feature parity tests
|
||||
|
||||
### Phase 5: Integration Testing
|
||||
|
||||
17. Add end-to-end integration tests
|
||||
18. Manual testing of full workflow (local and remote)
|
||||
|
||||
### Verification Checklist
|
||||
|
||||
- [ ] All unit tests pass: `pytest tests/unit/ -v`
|
||||
- [ ] All backend tests pass: `pytest backend/tests/ -v`
|
||||
- [ ] Coverage meets 70% minimum: `pytest --cov=karaoke_gen --cov-fail-under=70`
|
||||
- [ ] Local CLI works: `karaoke-gen "Artist" "Title"` (reaches instrumental selection)
|
||||
- [ ] Remote CLI works: `karaoke-gen-remote ./audio.flac "Artist" "Title"`
|
||||
- [ ] Both CLIs produce identical analysis for same audio
|
||||
- [ ] Custom instrumental creation works in both modes
|
||||
- [ ] Version bumped in `pyproject.toml`
|
||||
@@ -0,0 +1,217 @@
|
||||
---
|
||||
name: MidiCo-Style Synchronizer
|
||||
overview: Build a new MidiCo-style Synchronizer component for efficient re-syncing of lyrics timing, particularly for fixing progressive drift in the second half of songs. This involves refactoring the "Edit All" entry point and creating a high-performance timeline-based sync interface.
|
||||
todos:
|
||||
- id: mode-selection-modal
|
||||
content: Create ModeSelectionModal for 'Replace All' vs 'Re-sync Existing' choice
|
||||
status: completed
|
||||
- id: timeline-canvas
|
||||
content: Build canvas-based TimelineCanvas component for high-performance word block rendering
|
||||
status: completed
|
||||
- id: upcoming-words-bar
|
||||
content: Create UpcomingWordsBar component showing next unsynced words in fixed position
|
||||
status: completed
|
||||
- id: sync-controls
|
||||
content: Build SyncControls with Start Sync, Clear Sync, Unsync from Cursor, Delete Selected buttons
|
||||
status: completed
|
||||
- id: lyrics-synchronizer
|
||||
content: Create main LyricsSynchronizer component integrating all sub-components
|
||||
status: completed
|
||||
- id: unsync-from-cursor
|
||||
content: Add 'unsync from cursor' functionality to useManualSync hook
|
||||
status: completed
|
||||
- id: integrate-modal
|
||||
content: Integrate new synchronizer into ReplaceAllLyricsModal with mode selection
|
||||
status: completed
|
||||
- id: word-selection
|
||||
content: Implement single-click and drag-select for word blocks
|
||||
status: completed
|
||||
- id: zoom-slider
|
||||
content: Add smooth zoom slider with 4.5s-24s range
|
||||
status: completed
|
||||
- id: performance-test
|
||||
content: Test performance with long songs and optimize as needed
|
||||
status: completed
|
||||
---
|
||||
|
||||
# MidiCo-Style Synchronizer Implementation
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
The existing implementation in [`ReplaceAllLyricsModal.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/ReplaceAllLyricsModal.tsx) has two phases:
|
||||
|
||||
1. **Paste Phase** - User pastes new lyrics text
|
||||
2. **Sync Phase** - Manual sync using spacebar with timeline view
|
||||
|
||||
Key existing components:
|
||||
|
||||
- [`useManualSync.ts`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/hooks/useManualSync.ts) - Spacebar handling, tap/hold detection
|
||||
- [`EditTimelineSection.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/EditTimelineSection.tsx) - Timeline controls with zoom/scroll
|
||||
- [`TimelineEditor.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/TimelineEditor.tsx) - Timeline rendering
|
||||
|
||||
**Problems with current implementation:**
|
||||
|
||||
1. Forces full lyrics replacement - no option to keep existing sync data
|
||||
2. Performance issues (laggy, slow rendering)
|
||||
3. Poor UX for seeing upcoming words while syncing
|
||||
4. No "unsync from cursor" feature for fixing drift
|
||||
5. No two-level word block layout for overlapping segments
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Entry [Entry Point]
|
||||
EditAll[Edit All Button]
|
||||
EditAll --> ChoiceModal[Mode Selection Modal]
|
||||
ChoiceModal -->|Replace All| PastePhase[Paste New Lyrics]
|
||||
ChoiceModal -->|Re-sync Existing| SyncView[Synchronizer View]
|
||||
PastePhase --> SyncView
|
||||
end
|
||||
|
||||
subgraph Sync [Synchronizer Component]
|
||||
SyncView --> Timeline[Timeline Canvas]
|
||||
SyncView --> Controls[Control Buttons]
|
||||
SyncView --> UpcomingWords[Upcoming Words Display]
|
||||
Timeline --> WordBlocks[Word Blocks Renderer]
|
||||
Timeline --> Playhead[Playhead Marker]
|
||||
Timeline --> TimeBar[Time Bar]
|
||||
end
|
||||
|
||||
subgraph Audio [Audio Integration]
|
||||
AudioPlayer[Audio Player]
|
||||
AudioPlayer <--> SyncView
|
||||
end
|
||||
```
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Entry Point Refactor
|
||||
|
||||
Modify the "Edit All" button flow to show a choice modal:
|
||||
|
||||
```typescript
|
||||
// New ModeSelectionModal component
|
||||
interface ModeSelectionModalProps {
|
||||
open: boolean
|
||||
onClose: () => void
|
||||
onSelectReplace: () => void // Goes to paste phase
|
||||
onSelectResync: () => void // Goes directly to sync view
|
||||
}
|
||||
```
|
||||
|
||||
- **Replace All Lyrics**: Current flow (paste box -> sync)
|
||||
- **Re-sync Existing Lyrics**: Load current segments directly into sync view
|
||||
|
||||
### Phase 2: New Synchronizer Component
|
||||
|
||||
Create a new high-performance `LyricsSynchronizer.tsx` component inspired by MidiCo:
|
||||
|
||||
#### Core Features
|
||||
|
||||
1. **Timeline Canvas** (HTML5 Canvas for performance)
|
||||
|
||||
- Time bar at top (MM:SS format with decisecond ticks)
|
||||
- Playhead with arrow indicator + vertical line
|
||||
- Red word blocks with text labels
|
||||
- Two-level word layout for overlapping segments
|
||||
- Zoom slider (4.5s to 24s visible range)
|
||||
- Horizontal scroll with smooth panning
|
||||
|
||||
2. **Upcoming Words Display** (dual location)
|
||||
|
||||
- Fixed position above timeline: next words as red/white blocks
|
||||
- On timeline: words appear to right of playhead during sync
|
||||
|
||||
3. **Control Buttons**
|
||||
|
||||
- **Start Sync** - Begin sync mode from first unsynced word
|
||||
- **Clear Sync** - Reset all word timings to null
|
||||
- **Edit Lyrics** - Modal to edit lyrics text (warns about timing loss)
|
||||
- **Unsync from Cursor** - Critical: reset all words after playhead position
|
||||
- **Split/Edit Word** - Edit single selected word
|
||||
- **Delete Selected** - Remove selected word blocks
|
||||
|
||||
4. **Word Selection**
|
||||
|
||||
- Single click: select one word (white border)
|
||||
- Click-drag on background: selection rectangle for multi-select
|
||||
- Click on time bar: set playhead position
|
||||
|
||||
5. **Spacebar Sync Logic** (reuse/enhance `useManualSync`)
|
||||
|
||||
- Tap: set start time, default 500ms duration
|
||||
- Hold: set both start and end times
|
||||
- Handle previous word end time calculation
|
||||
|
||||
### Phase 3: Performance Optimization
|
||||
|
||||
The current `TimelineEditor` uses DOM elements which causes lag. New approach:
|
||||
|
||||
1. **Canvas-based rendering** for word blocks and timeline
|
||||
|
||||
- Single canvas element, no DOM nodes per word
|
||||
- RequestAnimationFrame for smooth playhead updates
|
||||
- Only re-render changed regions
|
||||
|
||||
2. **Virtual scrolling** for very long songs
|
||||
|
||||
- Only render visible word blocks
|
||||
- Cache off-screen block positions
|
||||
|
||||
3. **Debounced state updates**
|
||||
|
||||
- Batch word timing updates
|
||||
- Throttle zoom/scroll handlers
|
||||
|
||||
### Phase 4: Apply/Cancel Flow
|
||||
|
||||
Changes are held in local state until user clicks Apply:
|
||||
|
||||
```typescript
|
||||
interface SynchronizerState {
|
||||
words: Word[] // Working copy of words
|
||||
segments: LyricsSegment[] // Working copy of segments
|
||||
isDirty: boolean // Has unsaved changes
|
||||
}
|
||||
```
|
||||
|
||||
- **Apply**: Commit changes to parent component
|
||||
- **Cancel**: Discard changes, close modal
|
||||
|
||||
## File Changes
|
||||
|
||||
### New Files
|
||||
|
||||
- `src/components/LyricsSynchronizer.tsx` - Main synchronizer component
|
||||
- `src/components/LyricsSynchronizer/TimelineCanvas.tsx` - Canvas-based timeline
|
||||
- `src/components/LyricsSynchronizer/UpcomingWordsBar.tsx` - Fixed upcoming words display
|
||||
- `src/components/LyricsSynchronizer/SyncControls.tsx` - Control buttons
|
||||
- `src/components/ModeSelectionModal.tsx` - Entry point choice modal
|
||||
|
||||
### Modified Files
|
||||
|
||||
- [`ReplaceAllLyricsModal.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/ReplaceAllLyricsModal.tsx) - Add mode selection, integrate new synchronizer
|
||||
- [`useManualSync.ts`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/hooks/useManualSync.ts) - Add "unsync from cursor" functionality
|
||||
|
||||
## Key UX Details from MidiCo Reference
|
||||
|
||||
Per the documentation at [`MIDICO_SYNCHRONIZER_REFERENCE.md`](lyrics_transcriber_temp/docs/MIDICO_SYNCHRONIZER_REFERENCE.md):
|
||||
|
||||
1. **Zoom range**: 4.5s (max zoom) to 24s (min zoom) visible
|
||||
2. **Timeline click behavior**:
|
||||
|
||||
- Time bar click = set playhead (no playback change)
|
||||
- Background click = nothing (prevent accidental deselection)
|
||||
- Word block click = select
|
||||
|
||||
3. **Two-level word blocks**: Adjacent/overlapping words display on alternating vertical levels
|
||||
4. **Playhead**: Small arrow in time bar + white vertical line spanning timeline height
|
||||
5. **Selection rectangle**: Click-drag on background creates selection box
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
1. Manual testing with the provided URL: `http://localhost:8000/?baseApiUrl=...`
|
||||
2. Test sync accuracy with songs that have known drift issues
|
||||
3. Performance testing with 400+ word songs
|
||||
4. Test "unsync from cursor" workflow for fixing drift
|
||||
@@ -0,0 +1,544 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: "ci-${{ github.ref }}"
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
# ============================================
|
||||
# PACKAGE TESTS (karaoke-gen Python CLI)
|
||||
# ============================================
|
||||
|
||||
package-unit-tests:
|
||||
name: "Package - Unit Tests (70% coverage)"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install --with dev
|
||||
|
||||
- name: Install system dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y ffmpeg
|
||||
|
||||
- name: Install spaCy language models
|
||||
run: poetry run python -m spacy download en_core_web_sm
|
||||
|
||||
- name: Run unit tests
|
||||
run: |
|
||||
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=xml --cov-report=term-missing --cov-fail-under=70
|
||||
|
||||
- name: Upload coverage reports
|
||||
uses: codecov/codecov-action@v5
|
||||
with:
|
||||
token: ${{ secrets.CODECOV_TOKEN }}
|
||||
files: ./coverage.xml
|
||||
flags: unittests
|
||||
name: codecov-umbrella
|
||||
|
||||
package-integration-tests:
|
||||
name: "Package - Integration Tests"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install --with dev
|
||||
|
||||
- name: Install system dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y ffmpeg
|
||||
|
||||
- name: Install spaCy language models
|
||||
run: poetry run python -m spacy download en_core_web_sm
|
||||
|
||||
- name: Run integration tests
|
||||
run: poetry run pytest tests/integration/ -v
|
||||
|
||||
package-build:
|
||||
name: "Package - Build & Install"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [package-unit-tests, package-integration-tests]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Build package
|
||||
run: poetry build
|
||||
|
||||
- name: Free disk space
|
||||
uses: jlumbroso/free-disk-space@main
|
||||
with:
|
||||
tool-cache: false
|
||||
android: true
|
||||
haskell: true
|
||||
dotnet: false
|
||||
large-packages: false
|
||||
docker-images: false
|
||||
swap-storage: false
|
||||
|
||||
- name: Test package installation
|
||||
run: |
|
||||
pip install dist/*.whl
|
||||
python -m karaoke_gen.utils.gen_cli --help
|
||||
|
||||
- name: Upload build artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: package-dist
|
||||
path: dist/
|
||||
retention-days: 1
|
||||
|
||||
# ============================================
|
||||
# BACKEND TESTS (Cloud Run API service)
|
||||
# ============================================
|
||||
|
||||
backend-unit-tests:
|
||||
name: "Backend - Unit Tests"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
- name: Run unit tests
|
||||
env:
|
||||
GOOGLE_CLOUD_PROJECT: test-project
|
||||
GCS_BUCKET_NAME: test-bucket
|
||||
FIRESTORE_COLLECTION: jobs
|
||||
ENVIRONMENT: test
|
||||
run: |
|
||||
poetry run pytest \
|
||||
backend/tests/test_models.py \
|
||||
backend/tests/test_job_manager.py \
|
||||
backend/tests/test_file_upload.py \
|
||||
backend/tests/test_services.py \
|
||||
-v --tb=short
|
||||
|
||||
backend-emulator-tests:
|
||||
name: "Backend - Emulator Integration Tests"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /opt/ghc
|
||||
sudo rm -rf /usr/local/share/boost
|
||||
df -h
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Set up Java 21 (required by Firestore emulator)
|
||||
uses: actions/setup-java@v4
|
||||
with:
|
||||
distribution: "temurin"
|
||||
java-version: "21"
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Install Firestore emulator
|
||||
run: gcloud components install beta cloud-firestore-emulator --quiet
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
- name: Start Docker for GCS emulator
|
||||
run: docker pull fsouza/fake-gcs-server:latest
|
||||
|
||||
- name: Run emulator integration tests
|
||||
run: |
|
||||
chmod +x scripts/*.sh
|
||||
./scripts/run-emulator-tests.sh
|
||||
|
||||
- name: Upload test results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: backend-test-results
|
||||
path: backend/htmlcov/
|
||||
retention-days: 7
|
||||
|
||||
backend-lint:
|
||||
name: "Backend - Code Quality"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
- name: Run validation (syntax check only)
|
||||
run: |
|
||||
poetry run python -m py_compile backend/main.py backend/config.py
|
||||
echo "✅ Basic syntax validation passed"
|
||||
|
||||
# ============================================
|
||||
# DEPLOYMENTS (main branch only)
|
||||
# ============================================
|
||||
|
||||
package-publish:
|
||||
name: "Deploy - Publish to PyPI"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [package-build]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.13"
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
|
||||
- name: Download build artifacts
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: package-dist
|
||||
path: dist/
|
||||
|
||||
- name: Extract version
|
||||
id: extract_version
|
||||
run: |
|
||||
VERSION=$(poetry version --short)
|
||||
echo "version=$VERSION" >> $GITHUB_OUTPUT
|
||||
echo "📋 Extracted version: $VERSION"
|
||||
|
||||
- name: Check if tag exists
|
||||
id: check_tag
|
||||
run: |
|
||||
VERSION="${{ steps.extract_version.outputs.version }}"
|
||||
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
|
||||
echo "tag_exists=true" >> $GITHUB_OUTPUT
|
||||
echo "🏷️ Tag v$VERSION already exists, skipping release"
|
||||
else
|
||||
echo "tag_exists=false" >> $GITHUB_OUTPUT
|
||||
echo "✨ Tag v$VERSION does not exist, proceeding with release"
|
||||
fi
|
||||
|
||||
- name: Publish to PyPI
|
||||
if: steps.check_tag.outputs.tag_exists == 'false'
|
||||
run: |
|
||||
echo "📦 Publishing the following files to PyPI:"
|
||||
ls -la dist/
|
||||
echo "🚀 Publishing to PyPI..."
|
||||
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
|
||||
poetry publish
|
||||
|
||||
- name: Create Git Tag
|
||||
if: steps.check_tag.outputs.tag_exists == 'false'
|
||||
run: |
|
||||
git config user.name github-actions
|
||||
git config user.email github-actions@github.com
|
||||
git tag -a "v${{ steps.extract_version.outputs.version }}" -m "Release v${{ steps.extract_version.outputs.version }}"
|
||||
git push https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }} "v${{ steps.extract_version.outputs.version }}"
|
||||
|
||||
- name: Create GitHub Release
|
||||
if: steps.check_tag.outputs.tag_exists == 'false'
|
||||
uses: softprops/action-gh-release@v1
|
||||
with:
|
||||
tag_name: "v${{ steps.extract_version.outputs.version }}"
|
||||
name: "v${{ steps.extract_version.outputs.version }}"
|
||||
body: |
|
||||
## Changes in v${{ steps.extract_version.outputs.version }}
|
||||
|
||||
This release was automatically created from the main branch.
|
||||
|
||||
### Installation
|
||||
```bash
|
||||
pip install karaoke-gen==${{ steps.extract_version.outputs.version }}
|
||||
```
|
||||
|
||||
### Assets
|
||||
- Python package published to PyPI
|
||||
- Frontend deployed to GitHub Pages at gen.nomadkaraoke.com
|
||||
|
||||
For detailed changes, see the commit history.
|
||||
draft: false
|
||||
prerelease: false
|
||||
files: |
|
||||
dist/*.whl
|
||||
dist/*.tar.gz
|
||||
|
||||
deploy-frontend:
|
||||
name: "Deploy - Frontend (GitHub Pages)"
|
||||
runs-on: ubuntu-latest
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Pages
|
||||
uses: actions/configure-pages@v4
|
||||
|
||||
- name: Prepare frontend files
|
||||
run: |
|
||||
echo "📁 Preparing frontend files for deployment..."
|
||||
mkdir -p _site
|
||||
cp -r frontend/* _site/
|
||||
echo "✅ Frontend files copied to _site/"
|
||||
ls -la _site/
|
||||
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: "_site"
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
|
||||
deploy-backend:
|
||||
name: "Deploy - Backend (Cloud Run)"
|
||||
runs-on: ubuntu-latest
|
||||
needs: [backend-unit-tests, backend-emulator-tests, backend-lint]
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
id: auth
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: "latest"
|
||||
|
||||
- name: Check if base image needs rebuild
|
||||
id: check-base
|
||||
run: |
|
||||
echo "🔍 Checking if base image needs rebuild..."
|
||||
|
||||
# Hash both poetry.lock AND Dockerfile.base - rebuild if either changes
|
||||
CURRENT_HASH=$(cat poetry.lock backend/Dockerfile.base | sha256sum | cut -d' ' -f1)
|
||||
echo "Current combined hash (poetry.lock + Dockerfile.base): $CURRENT_HASH"
|
||||
|
||||
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
|
||||
|
||||
BASE_IMAGE="us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest"
|
||||
EXISTING_HASH=""
|
||||
|
||||
if docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "base.content.hash"}}' 2>/dev/null || echo "")
|
||||
# Fallback to old label name for backwards compatibility
|
||||
if [ -z "$EXISTING_HASH" ]; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
|
||||
fi
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
else
|
||||
echo "⚠️ No existing base image found"
|
||||
fi
|
||||
|
||||
if [ "$CURRENT_HASH" != "$EXISTING_HASH" ]; then
|
||||
echo "🔄 Base image needs rebuild (poetry.lock or Dockerfile.base changed)"
|
||||
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "✅ Base image is up to date"
|
||||
echo "needs_rebuild=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Rebuild base image (if needed)
|
||||
if: steps.check-base.outputs.needs_rebuild == 'true'
|
||||
run: |
|
||||
echo "🏗️ Rebuilding base image with updated dependencies..."
|
||||
|
||||
BASE_BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild-base.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
|
||||
echo "Base Build ID: $BASE_BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BASE_BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
echo "Waiting for base image build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BASE_BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Base build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Base image build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Base image build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BASE_BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
gcloud builds log $BASE_BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Trigger Cloud Build
|
||||
run: |
|
||||
echo "🚀 Triggering Cloud Build for deployment..."
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
|
||||
BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
|
||||
echo "Build ID: $BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
echo "Waiting for build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
echo ""
|
||||
gcloud builds log $BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Verify deployment
|
||||
run: |
|
||||
SERVICE_URL=$(gcloud run services describe karaoke-backend \
|
||||
--region us-central1 \
|
||||
--format 'value(status.url)')
|
||||
|
||||
echo "Service URL: $SERVICE_URL"
|
||||
sleep 10
|
||||
curl -f "$SERVICE_URL/api/health" || exit 1
|
||||
echo "✅ Deployment verified - service is healthy"
|
||||
|
||||
- name: Summary
|
||||
if: success()
|
||||
run: |
|
||||
echo "🎉 Deployment successful!"
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
echo "Branch: ${{ github.ref_name }}"
|
||||
echo "Run: ${{ github.run_number }}"
|
||||
@@ -1,187 +0,0 @@
|
||||
name: Test and Publish
|
||||
|
||||
on: [push, pull_request, workflow_dispatch]
|
||||
|
||||
# Add permissions for GitHub Pages deployment and git operations
|
||||
permissions:
|
||||
contents: write # Changed from 'read' to 'write' to allow creating git tags
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
|
||||
concurrency:
|
||||
group: "pages"
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ['3.13']
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
poetry install --with dev
|
||||
|
||||
- name: Install system dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y ffmpeg
|
||||
|
||||
- name: Install spaCy language models
|
||||
run: |
|
||||
poetry run python -m spacy download en_core_web_sm
|
||||
|
||||
- name: Run unit tests
|
||||
run: |
|
||||
poetry run pytest tests/unit/ -v --cov=karaoke_gen --cov-report=xml --cov-report=term-missing --cov-fail-under=70
|
||||
|
||||
- name: Upload coverage reports
|
||||
uses: codecov/codecov-action@v5
|
||||
with:
|
||||
token: ${{ secrets.CODECOV_TOKEN }}
|
||||
files: ./coverage.xml
|
||||
flags: unittests
|
||||
name: codecov-umbrella
|
||||
|
||||
- name: Run integration tests
|
||||
run: |
|
||||
poetry run pytest tests/integration/ -v
|
||||
|
||||
- name: Build package
|
||||
run: |
|
||||
poetry build
|
||||
|
||||
- name: Free Disk Space (GHA Ubuntu Runner)
|
||||
uses: jlumbroso/free-disk-space@main
|
||||
with:
|
||||
# this might remove tools that are actually needed,
|
||||
# if set to "true" but frees about 6 GB
|
||||
tool-cache: false
|
||||
|
||||
# all of these default to true, but feel free to set to
|
||||
# "false" if necessary for your workflow
|
||||
android: true
|
||||
haskell: true
|
||||
dotnet: false
|
||||
large-packages: false
|
||||
docker-images: false
|
||||
swap-storage: false
|
||||
|
||||
- name: Test package installation
|
||||
run: |
|
||||
pip install dist/*.whl
|
||||
python -m karaoke_gen.utils.gen_cli --help
|
||||
|
||||
- if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
name: Extract version
|
||||
id: extract_version
|
||||
run: |
|
||||
VERSION=$(poetry version --short)
|
||||
echo "version=$VERSION" >> $GITHUB_OUTPUT
|
||||
echo "📋 Extracted version: $VERSION"
|
||||
|
||||
- if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
name: Check if tag exists
|
||||
id: check_tag
|
||||
run: |
|
||||
VERSION="${{ steps.extract_version.outputs.version }}"
|
||||
if git ls-remote --tags origin | grep -q "refs/tags/v$VERSION$"; then
|
||||
echo "tag_exists=true" >> $GITHUB_OUTPUT
|
||||
echo "🏷️ Tag v$VERSION already exists, skipping release"
|
||||
else
|
||||
echo "tag_exists=false" >> $GITHUB_OUTPUT
|
||||
echo "✨ Tag v$VERSION does not exist, proceeding with release"
|
||||
fi
|
||||
|
||||
- if: github.ref == 'refs/heads/main' && github.event_name == 'push' && steps.check_tag.outputs.tag_exists == 'false'
|
||||
name: Publish to PyPI
|
||||
run: |
|
||||
echo "📦 Publishing the following files to PyPI:"
|
||||
ls -la dist/
|
||||
echo "🚀 Publishing to PyPI..."
|
||||
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
|
||||
poetry publish
|
||||
|
||||
- if: github.ref == 'refs/heads/main' && github.event_name == 'push' && steps.check_tag.outputs.tag_exists == 'false'
|
||||
name: Create Git Tag
|
||||
run: |
|
||||
git config user.name github-actions
|
||||
git config user.email github-actions@github.com
|
||||
git tag -a "v${{ steps.extract_version.outputs.version }}" -m "Release v${{ steps.extract_version.outputs.version }}"
|
||||
git push https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }} "v${{ steps.extract_version.outputs.version }}"
|
||||
|
||||
- if: github.ref == 'refs/heads/main' && github.event_name == 'push' && steps.check_tag.outputs.tag_exists == 'false'
|
||||
name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v1
|
||||
with:
|
||||
tag_name: "v${{ steps.extract_version.outputs.version }}"
|
||||
name: "v${{ steps.extract_version.outputs.version }}"
|
||||
body: |
|
||||
## Changes in v${{ steps.extract_version.outputs.version }}
|
||||
|
||||
This release was automatically created from the main branch.
|
||||
|
||||
### Installation
|
||||
```bash
|
||||
pip install karaoke-gen==${{ steps.extract_version.outputs.version }}
|
||||
```
|
||||
|
||||
### Assets
|
||||
- Python package published to PyPI
|
||||
- Frontend deployed to GitHub Pages at gen.nomadkaraoke.com
|
||||
|
||||
For detailed changes, see the commit history.
|
||||
draft: false
|
||||
prerelease: false
|
||||
files: |
|
||||
dist/*.whl
|
||||
dist/*.tar.gz
|
||||
|
||||
# Deploy frontend to GitHub Pages
|
||||
deploy-frontend:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Pages
|
||||
uses: actions/configure-pages@v4
|
||||
|
||||
- name: Prepare frontend files
|
||||
run: |
|
||||
echo "📁 Preparing frontend files for deployment..."
|
||||
mkdir -p _site
|
||||
cp -r frontend/* _site/
|
||||
echo "✅ Frontend files copied to _site/"
|
||||
ls -la _site/
|
||||
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: '_site'
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
|
||||
@@ -1,313 +0,0 @@
|
||||
name: Backend Tests
|
||||
|
||||
on:
|
||||
push: # Run on all branch pushes
|
||||
pull_request:
|
||||
branches: [ main ]
|
||||
|
||||
jobs:
|
||||
unit-tests:
|
||||
name: Unit Tests
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.13'
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
- name: Run unit tests
|
||||
env:
|
||||
GOOGLE_CLOUD_PROJECT: test-project
|
||||
GCS_BUCKET_NAME: test-bucket
|
||||
FIRESTORE_COLLECTION: jobs
|
||||
ENVIRONMENT: test
|
||||
run: |
|
||||
poetry run pytest \
|
||||
backend/tests/test_models.py \
|
||||
backend/tests/test_job_manager.py \
|
||||
backend/tests/test_file_upload.py \
|
||||
backend/tests/test_services.py \
|
||||
-v --tb=short
|
||||
|
||||
emulator-tests:
|
||||
name: Emulator Integration Tests
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Free disk space
|
||||
run: |
|
||||
sudo rm -rf /usr/share/dotnet
|
||||
sudo rm -rf /opt/ghc
|
||||
sudo rm -rf /usr/local/share/boost
|
||||
df -h
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.13'
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Set up Java 21 (required by Firestore emulator)
|
||||
uses: actions/setup-java@v4
|
||||
with:
|
||||
distribution: 'temurin'
|
||||
java-version: '21'
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: 'latest'
|
||||
|
||||
- name: Install Firestore emulator
|
||||
run: |
|
||||
gcloud components install beta cloud-firestore-emulator --quiet
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
- name: Start Docker for GCS emulator
|
||||
run: |
|
||||
docker pull fsouza/fake-gcs-server:latest
|
||||
|
||||
- name: Run emulator integration tests
|
||||
run: |
|
||||
chmod +x scripts/*.sh
|
||||
./scripts/run-emulator-tests.sh
|
||||
|
||||
- name: Upload test results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: test-results
|
||||
path: backend/htmlcov/
|
||||
retention-days: 7
|
||||
|
||||
lint:
|
||||
name: Code Quality
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Set up Python 3.13
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.13'
|
||||
|
||||
- name: Install Poetry
|
||||
uses: snok/install-poetry@v1
|
||||
with:
|
||||
virtualenvs-create: true
|
||||
virtualenvs-in-project: true
|
||||
|
||||
- name: Install dependencies
|
||||
run: poetry install
|
||||
|
||||
- name: Run validation (syntax check only)
|
||||
run: |
|
||||
# Skip full validation as it requires GCP credentials
|
||||
# Just check Python syntax and imports
|
||||
poetry run python -m py_compile backend/main.py backend/config.py
|
||||
echo "✅ Basic syntax validation passed"
|
||||
|
||||
deploy:
|
||||
name: Deploy via Cloud Build
|
||||
runs-on: ubuntu-latest
|
||||
needs: [unit-tests, emulator-tests, lint]
|
||||
# Deploy on any push where tests pass (pre-alpha: rapid iteration mode)
|
||||
# TODO: Change back to main-only when ready for production: github.ref == 'refs/heads/main'
|
||||
if: github.event_name == 'push'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # Required for Workload Identity Federation
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: false
|
||||
|
||||
- name: Authenticate to Google Cloud (Workload Identity)
|
||||
id: auth
|
||||
uses: google-github-actions/auth@v2
|
||||
with:
|
||||
workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
|
||||
service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
|
||||
|
||||
- name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v2
|
||||
with:
|
||||
version: 'latest'
|
||||
|
||||
- name: Check if base image needs rebuild
|
||||
id: check-base
|
||||
run: |
|
||||
echo "🔍 Checking if base image needs rebuild..."
|
||||
|
||||
# Calculate current poetry.lock hash
|
||||
CURRENT_HASH=$(sha256sum poetry.lock | cut -d' ' -f1)
|
||||
echo "Current poetry.lock hash: $CURRENT_HASH"
|
||||
|
||||
# Try to get the hash from the existing base image
|
||||
# First, authenticate to Artifact Registry
|
||||
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
|
||||
|
||||
# Try to pull the base image and get its label
|
||||
BASE_IMAGE="us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest"
|
||||
EXISTING_HASH=""
|
||||
|
||||
if docker pull $BASE_IMAGE 2>/dev/null; then
|
||||
EXISTING_HASH=$(docker inspect $BASE_IMAGE --format='{{index .Config.Labels "poetry.lock.hash"}}' 2>/dev/null || echo "")
|
||||
echo "Existing base image hash: $EXISTING_HASH"
|
||||
else
|
||||
echo "⚠️ No existing base image found"
|
||||
fi
|
||||
|
||||
# Compare hashes
|
||||
if [ "$CURRENT_HASH" != "$EXISTING_HASH" ]; then
|
||||
echo "🔄 Base image needs rebuild (poetry.lock changed)"
|
||||
echo "needs_rebuild=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "✅ Base image is up to date"
|
||||
echo "needs_rebuild=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Rebuild base image (if needed)
|
||||
if: steps.check-base.outputs.needs_rebuild == 'true'
|
||||
run: |
|
||||
echo "🏗️ Rebuilding base image with updated dependencies..."
|
||||
|
||||
# Submit base image build and wait for completion
|
||||
BASE_BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild-base.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
|
||||
echo "Base Build ID: $BASE_BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BASE_BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
# Poll for build completion
|
||||
echo "Waiting for base image build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BASE_BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Base build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Base image build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Base image build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BASE_BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
gcloud builds log $BASE_BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Trigger Cloud Build
|
||||
run: |
|
||||
echo "🚀 Triggering Cloud Build for deployment..."
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
|
||||
# Submit build and wait for completion
|
||||
BUILD_ID=$(gcloud builds submit \
|
||||
--config=cloudbuild.yaml \
|
||||
--project=nomadkaraoke \
|
||||
--format='value(id)' \
|
||||
--async \
|
||||
.)
|
||||
|
||||
echo "Build ID: $BUILD_ID"
|
||||
echo "Build URL: https://console.cloud.google.com/cloud-build/builds/$BUILD_ID?project=nomadkaraoke"
|
||||
|
||||
# Poll for build completion
|
||||
echo "Waiting for build to complete..."
|
||||
while true; do
|
||||
BUILD_STATUS=$(gcloud builds describe $BUILD_ID --project=nomadkaraoke --format='value(status)')
|
||||
echo "Build status: $BUILD_STATUS"
|
||||
|
||||
if [ "$BUILD_STATUS" = "SUCCESS" ]; then
|
||||
echo "✅ Build completed successfully"
|
||||
break
|
||||
elif [ "$BUILD_STATUS" = "FAILURE" ] || [ "$BUILD_STATUS" = "CANCELLED" ] || [ "$BUILD_STATUS" = "TIMEOUT" ]; then
|
||||
echo "❌ Build failed with status: $BUILD_STATUS"
|
||||
echo ""
|
||||
echo "==================== BUILD LOGS ===================="
|
||||
# Use Cloud Logging to get build logs (works with roles/logging.viewer)
|
||||
gcloud logging read "resource.type=build AND resource.labels.build_id=$BUILD_ID" \
|
||||
--project=nomadkaraoke \
|
||||
--format="value(textPayload)" \
|
||||
--limit=200 \
|
||||
--order=asc || echo "Failed to retrieve logs from Cloud Logging"
|
||||
echo "===================================================="
|
||||
echo ""
|
||||
# Also try the direct logs command as fallback
|
||||
gcloud builds log $BUILD_ID --project=nomadkaraoke 2>&1 || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
|
||||
- name: Verify deployment
|
||||
run: |
|
||||
SERVICE_URL=$(gcloud run services describe karaoke-backend \
|
||||
--region us-central1 \
|
||||
--format 'value(status.url)')
|
||||
|
||||
echo "Service URL: $SERVICE_URL"
|
||||
|
||||
# Wait a few seconds for the service to be ready
|
||||
sleep 10
|
||||
|
||||
# Test health endpoint
|
||||
curl -f "$SERVICE_URL/api/health" || exit 1
|
||||
|
||||
echo "✅ Deployment verified - service is healthy"
|
||||
|
||||
- name: Summary
|
||||
if: success()
|
||||
run: |
|
||||
echo "🎉 Deployment successful!"
|
||||
echo "Commit: ${{ github.sha }}"
|
||||
echo "Branch: ${{ github.ref_name }}"
|
||||
echo "Run: ${{ github.run_number }}"
|
||||
@@ -11,7 +11,9 @@
|
||||
# gcloud builds submit --config=cloudbuild-base.yaml --project=nomadkaraoke
|
||||
|
||||
# Use pre-built base image with all dependencies
|
||||
FROM us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest
|
||||
# Override with --build-arg BASE_IMAGE=karaoke-backend-base:local for local testing
|
||||
ARG BASE_IMAGE=us-central1-docker.pkg.dev/nomadkaraoke/karaoke-repo/karaoke-backend-base:latest
|
||||
FROM ${BASE_IMAGE}
|
||||
|
||||
# Set working directory (should already exist from base)
|
||||
WORKDIR /app
|
||||
@@ -34,5 +36,7 @@ ENV PORT=8080
|
||||
# Expose port
|
||||
EXPOSE 8080
|
||||
|
||||
# Run the application
|
||||
# Run uvicorn directly
|
||||
# Note: Transmission daemon removed - torrent downloads now handled by
|
||||
# dedicated flacfetch VM service (see docs/00-current-plan/TORRENT-SERVICE-PLAN.md)
|
||||
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8080"]
|
||||
@@ -7,12 +7,15 @@
|
||||
FROM python:3.11-slim
|
||||
|
||||
# Install system dependencies required by karaoke-gen
|
||||
# Note: Transmission daemon removed - torrent downloads now handled by
|
||||
# dedicated flacfetch VM service (see docs/00-current-plan/TORRENT-SERVICE-PLAN.md)
|
||||
RUN apt-get update && apt-get install -y \
|
||||
ffmpeg \
|
||||
libsndfile1 \
|
||||
libsox-dev \
|
||||
sox \
|
||||
build-essential \
|
||||
curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Set working directory
|
||||
@@ -27,12 +30,16 @@ COPY lyrics_transcriber_temp /app/lyrics_transcriber_temp
|
||||
RUN pip install --no-cache-dir --upgrade pip && \
|
||||
pip install --no-cache-dir -e .
|
||||
|
||||
# Download spaCy language model for lyrics transcription
|
||||
# This must be done at build time, not runtime (Cloud Run network restrictions)
|
||||
RUN python -m spacy download en_core_web_sm
|
||||
|
||||
# Label with build info
|
||||
ARG BUILD_DATE
|
||||
ARG BUILD_ID
|
||||
ARG POETRY_LOCK_HASH
|
||||
ARG BASE_CONTENT_HASH
|
||||
LABEL org.opencontainers.image.created="${BUILD_DATE}"
|
||||
LABEL org.opencontainers.image.revision="${BUILD_ID}"
|
||||
LABEL org.opencontainers.image.title="karaoke-backend-base"
|
||||
LABEL org.opencontainers.image.description="Base image with Python dependencies for karaoke-backend"
|
||||
LABEL poetry.lock.hash="${POETRY_LOCK_HASH}"
|
||||
LABEL org.opencontainers.image.description="Base image with Python dependencies and system tools for karaoke-backend"
|
||||
LABEL base.content.hash="${BASE_CONTENT_HASH}"
|
||||
@@ -0,0 +1,600 @@
|
||||
"""
|
||||
Audio search API routes for artist+title search mode.
|
||||
|
||||
This module provides endpoints for:
|
||||
1. Creating a job with audio search (artist+title without file)
|
||||
2. Getting search results for a job
|
||||
3. Selecting an audio source to download
|
||||
|
||||
The flow is:
|
||||
1. POST /api/audio-search/search - Create job and search for audio
|
||||
2. GET /api/audio-search/{job_id}/results - Get search results
|
||||
3. POST /api/audio-search/{job_id}/select - Select audio source
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
from typing import Optional, List, Dict, Any
|
||||
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Request
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from backend.models.job import JobCreate, JobStatus
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.services.worker_service import get_worker_service
|
||||
from backend.services.credential_manager import get_credential_manager, CredentialStatus
|
||||
from backend.services.audio_search_service import (
|
||||
get_audio_search_service,
|
||||
AudioSearchResult,
|
||||
AudioSearchError,
|
||||
NoResultsError,
|
||||
DownloadError,
|
||||
)
|
||||
from backend.config import get_settings
|
||||
from backend.version import VERSION
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(tags=["audio-search"])
|
||||
|
||||
# Initialize services
|
||||
job_manager = JobManager()
|
||||
storage_service = StorageService()
|
||||
worker_service = get_worker_service()
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Pydantic models
|
||||
# ============================================================================
|
||||
|
||||
class AudioSearchRequest(BaseModel):
|
||||
"""Request to search for audio by artist and title."""
|
||||
artist: str = Field(..., description="Artist name to search for")
|
||||
title: str = Field(..., description="Song title to search for")
|
||||
|
||||
# Auto-download mode
|
||||
auto_download: bool = Field(False, description="Automatically select best result and download")
|
||||
|
||||
# Processing options
|
||||
enable_cdg: bool = Field(True, description="Generate CDG+MP3 package")
|
||||
enable_txt: bool = Field(True, description="Generate TXT+MP3 package")
|
||||
|
||||
# Finalisation options
|
||||
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
|
||||
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
|
||||
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
|
||||
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
|
||||
|
||||
# Distribution options
|
||||
dropbox_path: Optional[str] = Field(None, description="Dropbox folder path for organized output")
|
||||
gdrive_folder_id: Optional[str] = Field(None, description="Google Drive folder ID for public share uploads")
|
||||
|
||||
# Lyrics configuration
|
||||
lyrics_artist: Optional[str] = Field(None, description="Override artist name for lyrics search")
|
||||
lyrics_title: Optional[str] = Field(None, description="Override title for lyrics search")
|
||||
subtitle_offset_ms: int = Field(0, description="Subtitle timing offset in milliseconds")
|
||||
|
||||
# Audio separation model configuration
|
||||
clean_instrumental_model: Optional[str] = Field(None, description="Model for clean instrumental separation")
|
||||
backing_vocals_models: Optional[List[str]] = Field(None, description="Models for backing vocals separation")
|
||||
other_stems_models: Optional[List[str]] = Field(None, description="Models for other stems")
|
||||
|
||||
|
||||
class AudioSearchResultResponse(BaseModel):
|
||||
"""A single audio search result.
|
||||
|
||||
Contains all fields needed for rich display using flacfetch's
|
||||
shared formatting functions.
|
||||
"""
|
||||
index: int
|
||||
title: str
|
||||
artist: str
|
||||
provider: str # Maps to source_name in Release
|
||||
url: str # Maps to download_url in Release
|
||||
duration: Optional[int] = None # Maps to duration_seconds in Release
|
||||
quality: Optional[str] = None # Stringified quality
|
||||
source_id: Optional[str] = None # Maps to info_hash in Release
|
||||
seeders: Optional[int] = None
|
||||
target_file: Optional[str] = None
|
||||
# Additional fields for rich display (from Release.to_dict())
|
||||
year: Optional[int] = None
|
||||
label: Optional[str] = None
|
||||
edition_info: Optional[str] = None
|
||||
release_type: Optional[str] = None
|
||||
channel: Optional[str] = None # For YouTube
|
||||
view_count: Optional[int] = None # For YouTube
|
||||
size_bytes: Optional[int] = None
|
||||
target_file_size: Optional[int] = None
|
||||
track_pattern: Optional[str] = None
|
||||
match_score: Optional[float] = None
|
||||
# Pre-computed display fields
|
||||
formatted_size: Optional[str] = None
|
||||
formatted_duration: Optional[str] = None
|
||||
formatted_views: Optional[str] = None
|
||||
is_lossless: Optional[bool] = None
|
||||
quality_str: Optional[str] = None
|
||||
# Full quality object for Release.from_dict()
|
||||
quality_data: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
class AudioSearchResponse(BaseModel):
|
||||
"""Response from audio search."""
|
||||
status: str
|
||||
job_id: str
|
||||
message: str
|
||||
results: Optional[List[AudioSearchResultResponse]] = None
|
||||
results_count: int = 0
|
||||
auto_download: bool = False
|
||||
server_version: str
|
||||
|
||||
|
||||
class AudioSelectRequest(BaseModel):
|
||||
"""Request to select an audio source."""
|
||||
selection_index: int = Field(..., description="Index of the selected audio source from search results")
|
||||
|
||||
|
||||
class AudioSelectResponse(BaseModel):
|
||||
"""Response from audio source selection."""
|
||||
status: str
|
||||
job_id: str
|
||||
message: str
|
||||
selected_index: int
|
||||
selected_title: str
|
||||
selected_artist: str
|
||||
selected_provider: str
|
||||
|
||||
|
||||
def extract_request_metadata(request: Request, created_from: str = "audio_search") -> Dict[str, Any]:
|
||||
"""Extract metadata from request for job tracking."""
|
||||
headers = dict(request.headers)
|
||||
|
||||
client_ip = headers.get('x-forwarded-for', '').split(',')[0].strip()
|
||||
if not client_ip and request.client:
|
||||
client_ip = request.client.host
|
||||
|
||||
user_agent = headers.get('user-agent', '')
|
||||
environment = headers.get('x-environment', '')
|
||||
client_id = headers.get('x-client-id', '')
|
||||
|
||||
custom_headers = {}
|
||||
for key, value in headers.items():
|
||||
if key.lower().startswith('x-') and key.lower() not in ('x-forwarded-for', 'x-forwarded-proto', 'x-forwarded-host'):
|
||||
custom_headers[key] = value
|
||||
|
||||
return {
|
||||
'client_ip': client_ip,
|
||||
'user_agent': user_agent,
|
||||
'environment': environment,
|
||||
'client_id': client_id,
|
||||
'server_version': VERSION,
|
||||
'created_from': created_from,
|
||||
'custom_headers': custom_headers,
|
||||
}
|
||||
|
||||
|
||||
async def _trigger_workers_parallel(job_id: str) -> None:
|
||||
"""Trigger both audio and lyrics workers in parallel."""
|
||||
await asyncio.gather(
|
||||
worker_service.trigger_audio_worker(job_id),
|
||||
worker_service.trigger_lyrics_worker(job_id)
|
||||
)
|
||||
|
||||
|
||||
async def _download_and_start_processing(
|
||||
job_id: str,
|
||||
selection_index: int,
|
||||
audio_search_service,
|
||||
background_tasks: BackgroundTasks,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Download selected audio and start job processing.
|
||||
|
||||
This is called either:
|
||||
- Immediately after search if auto_download=True
|
||||
- When user calls the select endpoint
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Get search results from state_data
|
||||
search_results = job.state_data.get('audio_search_results', [])
|
||||
if not search_results:
|
||||
raise HTTPException(status_code=400, detail="No search results available")
|
||||
|
||||
if selection_index < 0 or selection_index >= len(search_results):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid selection index {selection_index}. Valid range: 0-{len(search_results)-1}"
|
||||
)
|
||||
|
||||
selected = search_results[selection_index]
|
||||
|
||||
# Transition to downloading state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.DOWNLOADING_AUDIO,
|
||||
progress=10,
|
||||
message=f"Downloading from {selected['provider']}: {selected['artist']} - {selected['title']}",
|
||||
state_data_updates={
|
||||
'selected_audio_index': selection_index,
|
||||
'selected_audio_provider': selected['provider'],
|
||||
}
|
||||
)
|
||||
|
||||
try:
|
||||
# Download to temp directory
|
||||
temp_dir = tempfile.mkdtemp(prefix=f"audio_download_{job_id}_")
|
||||
|
||||
result = audio_search_service.download(
|
||||
result_index=selection_index,
|
||||
output_dir=temp_dir,
|
||||
)
|
||||
|
||||
# Upload to GCS
|
||||
filename = os.path.basename(result.filepath)
|
||||
audio_gcs_path = f"uploads/{job_id}/audio/{filename}"
|
||||
|
||||
with open(result.filepath, 'rb') as f:
|
||||
storage_service.upload_fileobj(
|
||||
f,
|
||||
audio_gcs_path,
|
||||
content_type='audio/flac' # flacfetch typically returns FLAC
|
||||
)
|
||||
|
||||
logger.info(f"Uploaded audio to GCS: {audio_gcs_path}")
|
||||
|
||||
# Clean up temp file
|
||||
try:
|
||||
os.remove(result.filepath)
|
||||
os.rmdir(temp_dir)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to clean up temp files: {e}")
|
||||
|
||||
# Update job with GCS path and transition to DOWNLOADING
|
||||
job_manager.update_job(job_id, {
|
||||
'input_media_gcs_path': audio_gcs_path,
|
||||
'filename': filename,
|
||||
})
|
||||
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.DOWNLOADING,
|
||||
progress=15,
|
||||
message="Audio downloaded, starting processing"
|
||||
)
|
||||
|
||||
# Trigger workers
|
||||
background_tasks.add_task(_trigger_workers_parallel, job_id)
|
||||
|
||||
return {
|
||||
'selected_index': selection_index,
|
||||
'selected_title': selected['title'],
|
||||
'selected_artist': selected['artist'],
|
||||
'selected_provider': selected['provider'],
|
||||
}
|
||||
|
||||
except DownloadError as e:
|
||||
job_manager.fail_job(job_id, f"Audio download failed: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Download failed: {e}")
|
||||
except Exception as e:
|
||||
job_manager.fail_job(job_id, f"Audio download failed: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Download failed: {e}")
|
||||
|
||||
|
||||
@router.post("/audio-search/search", response_model=AudioSearchResponse)
|
||||
async def search_audio(
|
||||
request: Request,
|
||||
background_tasks: BackgroundTasks,
|
||||
body: AudioSearchRequest,
|
||||
):
|
||||
"""
|
||||
Search for audio by artist and title, creating a new job.
|
||||
|
||||
This endpoint:
|
||||
1. Creates a job in PENDING state
|
||||
2. Searches for audio using flacfetch
|
||||
3. Either returns search results for user selection, or
|
||||
4. If auto_download=True, automatically selects best and starts processing
|
||||
|
||||
Use cases:
|
||||
- Interactive mode (default): Returns results, user calls /select endpoint
|
||||
- Auto mode (auto_download=True): Automatically selects and downloads best
|
||||
"""
|
||||
try:
|
||||
# Apply default distribution settings
|
||||
settings = get_settings()
|
||||
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
|
||||
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
|
||||
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
|
||||
|
||||
# Validate credentials if distribution services are requested
|
||||
invalid_services = []
|
||||
credential_manager = get_credential_manager()
|
||||
|
||||
if body.enable_youtube_upload:
|
||||
result = credential_manager.check_youtube_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"youtube ({result.message})")
|
||||
|
||||
if effective_dropbox_path:
|
||||
result = credential_manager.check_dropbox_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"dropbox ({result.message})")
|
||||
|
||||
if effective_gdrive_folder_id:
|
||||
result = credential_manager.check_gdrive_credentials()
|
||||
if result.status != CredentialStatus.VALID:
|
||||
invalid_services.append(f"gdrive ({result.message})")
|
||||
|
||||
if invalid_services:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail={
|
||||
"error": "credentials_invalid",
|
||||
"message": f"Distribution services need re-authorization: {', '.join(invalid_services)}",
|
||||
"invalid_services": invalid_services,
|
||||
}
|
||||
)
|
||||
|
||||
# Extract request metadata
|
||||
request_metadata = extract_request_metadata(request, created_from="audio_search")
|
||||
|
||||
# Create job
|
||||
job_create = JobCreate(
|
||||
artist=body.artist,
|
||||
title=body.title,
|
||||
enable_cdg=body.enable_cdg,
|
||||
enable_txt=body.enable_txt,
|
||||
brand_prefix=body.brand_prefix,
|
||||
enable_youtube_upload=body.enable_youtube_upload,
|
||||
youtube_description=body.youtube_description,
|
||||
discord_webhook_url=effective_discord_webhook_url,
|
||||
dropbox_path=effective_dropbox_path,
|
||||
gdrive_folder_id=effective_gdrive_folder_id,
|
||||
lyrics_artist=body.lyrics_artist,
|
||||
lyrics_title=body.lyrics_title,
|
||||
subtitle_offset_ms=body.subtitle_offset_ms,
|
||||
clean_instrumental_model=body.clean_instrumental_model,
|
||||
backing_vocals_models=body.backing_vocals_models,
|
||||
other_stems_models=body.other_stems_models,
|
||||
audio_search_artist=body.artist,
|
||||
audio_search_title=body.title,
|
||||
auto_download=body.auto_download,
|
||||
request_metadata=request_metadata,
|
||||
)
|
||||
job = job_manager.create_job(job_create)
|
||||
job_id = job.job_id
|
||||
|
||||
logger.info(f"Created job {job_id} for audio search: {body.artist} - {body.title}")
|
||||
|
||||
# Update job with audio search fields
|
||||
job_manager.update_job(job_id, {
|
||||
'audio_search_artist': body.artist,
|
||||
'audio_search_title': body.title,
|
||||
'auto_download': body.auto_download,
|
||||
})
|
||||
|
||||
# Transition to searching state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.SEARCHING_AUDIO,
|
||||
progress=5,
|
||||
message=f"Searching for: {body.artist} - {body.title}"
|
||||
)
|
||||
|
||||
# Perform search
|
||||
audio_search_service = get_audio_search_service()
|
||||
|
||||
try:
|
||||
search_results = audio_search_service.search(body.artist, body.title)
|
||||
except NoResultsError as e:
|
||||
job_manager.fail_job(job_id, f"No audio sources found for: {body.artist} - {body.title}")
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail={
|
||||
"error": "no_results",
|
||||
"message": str(e),
|
||||
"job_id": job_id,
|
||||
}
|
||||
)
|
||||
except AudioSearchError as e:
|
||||
job_manager.fail_job(job_id, f"Audio search failed: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Search failed: {e}")
|
||||
|
||||
# Store results in job state_data
|
||||
results_dicts = [r.to_dict() for r in search_results]
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
'audio_search_results': results_dicts,
|
||||
'audio_search_count': len(results_dicts),
|
||||
}
|
||||
})
|
||||
|
||||
# If auto_download, select best and start processing
|
||||
if body.auto_download:
|
||||
best_index = audio_search_service.select_best(search_results)
|
||||
|
||||
logger.info(f"Auto-download enabled, selecting result {best_index}")
|
||||
|
||||
selection_info = await _download_and_start_processing(
|
||||
job_id=job_id,
|
||||
selection_index=best_index,
|
||||
audio_search_service=audio_search_service,
|
||||
background_tasks=background_tasks,
|
||||
)
|
||||
|
||||
return AudioSearchResponse(
|
||||
status="success",
|
||||
job_id=job_id,
|
||||
message=f"Audio found and download started: {selection_info['selected_artist']} - {selection_info['selected_title']} ({selection_info['selected_provider']})",
|
||||
results=None, # Don't return results in auto mode
|
||||
results_count=len(search_results),
|
||||
auto_download=True,
|
||||
server_version=VERSION,
|
||||
)
|
||||
|
||||
# Interactive mode: return results for user selection
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.AWAITING_AUDIO_SELECTION,
|
||||
progress=10,
|
||||
message=f"Found {len(search_results)} audio sources. Waiting for selection."
|
||||
)
|
||||
|
||||
# Convert to response format with full Release data for rich display
|
||||
result_responses = []
|
||||
for r in search_results:
|
||||
# Get full serialized data from raw_result if available
|
||||
raw_dict = {}
|
||||
if r.raw_result:
|
||||
try:
|
||||
raw_dict = r.raw_result.to_dict()
|
||||
except AttributeError:
|
||||
pass # Not a Release object
|
||||
|
||||
result_responses.append(
|
||||
AudioSearchResultResponse(
|
||||
index=r.index,
|
||||
title=r.title,
|
||||
artist=r.artist,
|
||||
provider=r.provider,
|
||||
url=r.url,
|
||||
duration=r.duration,
|
||||
quality=r.quality,
|
||||
source_id=r.source_id,
|
||||
seeders=raw_dict.get('seeders') or r.seeders,
|
||||
target_file=raw_dict.get('target_file') or r.target_file,
|
||||
# Additional fields for rich display
|
||||
year=raw_dict.get('year'),
|
||||
label=raw_dict.get('label'),
|
||||
edition_info=raw_dict.get('edition_info'),
|
||||
release_type=raw_dict.get('release_type'),
|
||||
channel=raw_dict.get('channel'),
|
||||
view_count=raw_dict.get('view_count'),
|
||||
size_bytes=raw_dict.get('size_bytes'),
|
||||
target_file_size=raw_dict.get('target_file_size'),
|
||||
track_pattern=raw_dict.get('track_pattern'),
|
||||
match_score=raw_dict.get('match_score'),
|
||||
# Pre-computed display fields
|
||||
formatted_size=raw_dict.get('formatted_size'),
|
||||
formatted_duration=raw_dict.get('formatted_duration'),
|
||||
formatted_views=raw_dict.get('formatted_views'),
|
||||
is_lossless=raw_dict.get('is_lossless'),
|
||||
quality_str=raw_dict.get('quality_str'),
|
||||
# Full quality object for Release.from_dict()
|
||||
quality_data=raw_dict.get('quality'),
|
||||
)
|
||||
)
|
||||
|
||||
return AudioSearchResponse(
|
||||
status="awaiting_selection",
|
||||
job_id=job_id,
|
||||
message=f"Found {len(search_results)} audio sources. Call /api/audio-search/{job_id}/select to choose one.",
|
||||
results=result_responses,
|
||||
results_count=len(search_results),
|
||||
auto_download=False,
|
||||
server_version=VERSION,
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Error in audio search: {e}", exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
|
||||
@router.get("/audio-search/{job_id}/results")
|
||||
async def get_audio_search_results(job_id: str):
|
||||
"""
|
||||
Get audio search results for a job.
|
||||
|
||||
Returns the cached search results so user can select one.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
search_results = job.state_data.get('audio_search_results', [])
|
||||
|
||||
if not search_results:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="No search results available for this job"
|
||||
)
|
||||
|
||||
return {
|
||||
"status": "success",
|
||||
"job_id": job_id,
|
||||
"job_status": job.status,
|
||||
"artist": job.audio_search_artist or job.artist,
|
||||
"title": job.audio_search_title or job.title,
|
||||
"results": search_results,
|
||||
"results_count": len(search_results),
|
||||
}
|
||||
|
||||
|
||||
@router.post("/audio-search/{job_id}/select", response_model=AudioSelectResponse)
|
||||
async def select_audio_source(
|
||||
job_id: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
body: AudioSelectRequest,
|
||||
):
|
||||
"""
|
||||
Select an audio source and start job processing.
|
||||
|
||||
This endpoint:
|
||||
1. Validates the job is awaiting selection
|
||||
2. Downloads the selected audio
|
||||
3. Uploads to GCS
|
||||
4. Triggers audio and lyrics workers
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
# Verify job is awaiting selection
|
||||
if job.status != JobStatus.AWAITING_AUDIO_SELECTION:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Job is not awaiting audio selection (status: {job.status})"
|
||||
)
|
||||
|
||||
# Re-instantiate the audio search service (it caches raw results from search)
|
||||
# For production, we'd need to re-search or store raw results differently
|
||||
audio_search_service = get_audio_search_service()
|
||||
|
||||
# Re-run search to populate the cache
|
||||
# This is necessary because the service caches raw results in memory
|
||||
search_results = job.state_data.get('audio_search_results', [])
|
||||
if not search_results:
|
||||
raise HTTPException(status_code=400, detail="No search results cached for this job")
|
||||
|
||||
artist = job.audio_search_artist or job.artist
|
||||
title = job.audio_search_title or job.title
|
||||
|
||||
try:
|
||||
# Re-search to populate cache
|
||||
audio_search_service.search(artist, title)
|
||||
except Exception as e:
|
||||
logger.warning(f"Re-search failed, trying direct download: {e}")
|
||||
|
||||
selection_info = await _download_and_start_processing(
|
||||
job_id=job_id,
|
||||
selection_index=body.selection_index,
|
||||
audio_search_service=audio_search_service,
|
||||
background_tasks=background_tasks,
|
||||
)
|
||||
|
||||
return AudioSelectResponse(
|
||||
status="success",
|
||||
job_id=job_id,
|
||||
message="Audio selected and download started",
|
||||
selected_index=selection_info['selected_index'],
|
||||
selected_title=selection_info['selected_title'],
|
||||
selected_artist=selection_info['selected_artist'],
|
||||
selected_provider=selection_info['selected_provider'],
|
||||
)
|
||||
|
||||
|
||||
@@ -1,10 +1,64 @@
|
||||
"""
|
||||
Health check routes.
|
||||
"""
|
||||
import os
|
||||
import logging
|
||||
from fastapi import APIRouter
|
||||
from typing import Dict
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
from backend.version import VERSION
|
||||
from backend.services.flacfetch_client import get_flacfetch_client
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def check_transmission_status() -> Dict[str, Any]:
|
||||
"""Check if Transmission daemon is available and responsive."""
|
||||
try:
|
||||
import transmission_rpc
|
||||
host = os.environ.get("TRANSMISSION_HOST", "localhost")
|
||||
port = int(os.environ.get("TRANSMISSION_PORT", "9091"))
|
||||
|
||||
client = transmission_rpc.Client(host=host, port=port, timeout=5)
|
||||
stats = client.session_stats()
|
||||
|
||||
# Get detailed torrent info
|
||||
torrents = client.get_torrents()
|
||||
torrent_details = []
|
||||
for t in torrents:
|
||||
torrent_info = {
|
||||
"name": t.name,
|
||||
"progress": round(t.progress, 1),
|
||||
"status": str(t.status),
|
||||
"peers": t.peers_connected if hasattr(t, 'peers_connected') else 0,
|
||||
"download_speed": round(t.rate_download / 1024, 1) if hasattr(t, 'rate_download') else 0, # KB/s
|
||||
}
|
||||
# Check if stalled (downloading but no progress and no peers)
|
||||
if t.status.downloading and t.progress < 100 and torrent_info['peers'] == 0:
|
||||
torrent_info['stalled'] = True
|
||||
torrent_details.append(torrent_info)
|
||||
|
||||
return {
|
||||
"available": True,
|
||||
"host": host,
|
||||
"port": port,
|
||||
"download_dir": stats.download_dir if hasattr(stats, 'download_dir') else None,
|
||||
"active_torrent_count": stats.active_torrent_count if hasattr(stats, 'active_torrent_count') else 0,
|
||||
"torrents": torrent_details,
|
||||
}
|
||||
except ImportError as e:
|
||||
return {
|
||||
"available": False,
|
||||
"error": f"transmission_rpc not installed: {e}",
|
||||
}
|
||||
except Exception as e:
|
||||
return {
|
||||
"available": False,
|
||||
"host": os.environ.get("TRANSMISSION_HOST", "localhost"),
|
||||
"port": int(os.environ.get("TRANSMISSION_PORT", "9091")),
|
||||
"error": str(e),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health")
|
||||
@@ -16,6 +70,67 @@ async def health_check() -> Dict[str, str]:
|
||||
}
|
||||
|
||||
|
||||
async def check_flacfetch_service_status() -> Dict[str, Any]:
|
||||
"""Check if remote flacfetch service is available and healthy."""
|
||||
client = get_flacfetch_client()
|
||||
|
||||
if not client:
|
||||
return {
|
||||
"configured": False,
|
||||
"message": "Remote flacfetch service not configured (FLACFETCH_API_URL not set)",
|
||||
}
|
||||
|
||||
try:
|
||||
health = await client.health_check()
|
||||
return {
|
||||
"configured": True,
|
||||
"available": True,
|
||||
"status": health.get("status"),
|
||||
"version": health.get("version"),
|
||||
"transmission": health.get("transmission", {}),
|
||||
"disk": health.get("disk", {}),
|
||||
"providers": health.get("providers", {}),
|
||||
}
|
||||
except Exception as e:
|
||||
return {
|
||||
"configured": True,
|
||||
"available": False,
|
||||
"error": str(e),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/health/detailed")
|
||||
async def detailed_health_check() -> Dict[str, Any]:
|
||||
"""
|
||||
Detailed health check including dependencies.
|
||||
|
||||
Use this to debug issues with Transmission, flacfetch service, etc.
|
||||
"""
|
||||
transmission_status = check_transmission_status()
|
||||
flacfetch_status = await check_flacfetch_service_status()
|
||||
|
||||
# Log for debugging
|
||||
if not transmission_status.get("available"):
|
||||
logger.warning(f"Local Transmission not available: {transmission_status.get('error')}")
|
||||
else:
|
||||
logger.info(f"Local Transmission available at {transmission_status.get('host')}:{transmission_status.get('port')}")
|
||||
|
||||
if flacfetch_status.get("configured") and not flacfetch_status.get("available"):
|
||||
logger.warning(f"Remote flacfetch service not available: {flacfetch_status.get('error')}")
|
||||
elif flacfetch_status.get("available"):
|
||||
logger.info(f"Remote flacfetch service healthy: {flacfetch_status.get('status')}")
|
||||
|
||||
return {
|
||||
"status": "healthy",
|
||||
"service": "karaoke-gen-backend",
|
||||
"version": VERSION,
|
||||
"dependencies": {
|
||||
"transmission_local": transmission_status,
|
||||
"flacfetch_remote": flacfetch_status,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@router.get("/readiness")
|
||||
async def readiness_check() -> Dict[str, str]:
|
||||
"""Readiness check endpoint for Cloud Run."""
|
||||
|
||||
@@ -3,10 +3,13 @@ Internal API routes for worker coordination.
|
||||
|
||||
These endpoints are for internal use only (backend → workers).
|
||||
They are protected by admin authentication.
|
||||
|
||||
With Cloud Tasks integration, these endpoints may be called multiple times
|
||||
(retry on failure). Idempotency checks prevent duplicate processing.
|
||||
"""
|
||||
import logging
|
||||
import asyncio
|
||||
from typing import Tuple
|
||||
from typing import Tuple, Optional
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -17,6 +20,7 @@ from backend.workers.video_worker import generate_video
|
||||
from backend.workers.render_video_worker import process_render_video
|
||||
from backend.api.dependencies import require_admin
|
||||
from backend.services.auth_service import UserType
|
||||
from backend.services.job_manager import JobManager
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -35,6 +39,57 @@ class WorkerResponse(BaseModel):
|
||||
message: str
|
||||
|
||||
|
||||
def _check_worker_idempotency(job_id: str, worker_name: str) -> Optional[WorkerResponse]:
|
||||
"""
|
||||
Check if a worker is already running or completed for this job.
|
||||
|
||||
This provides idempotency for Cloud Tasks retries - if a task is retried
|
||||
but the worker is already running or has completed, we skip processing.
|
||||
|
||||
Args:
|
||||
job_id: Job ID to check
|
||||
worker_name: Worker name (audio, lyrics, screens, render, video)
|
||||
|
||||
Returns:
|
||||
WorkerResponse if should skip (already running/complete), None to proceed
|
||||
"""
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
if not job:
|
||||
logger.warning(f"Job {job_id} not found for {worker_name} worker")
|
||||
return WorkerResponse(
|
||||
status="not_found",
|
||||
job_id=job_id,
|
||||
message=f"Job {job_id} not found"
|
||||
)
|
||||
|
||||
# Check worker-specific progress in state_data
|
||||
progress_key = f"{worker_name}_progress"
|
||||
worker_progress = job.state_data.get(progress_key, {})
|
||||
stage = worker_progress.get('stage')
|
||||
|
||||
if stage == 'running':
|
||||
logger.info(f"{worker_name.capitalize()} worker already running for job {job_id}, skipping")
|
||||
return WorkerResponse(
|
||||
status="already_running",
|
||||
job_id=job_id,
|
||||
message=f"{worker_name.capitalize()} worker already in progress"
|
||||
)
|
||||
|
||||
if stage == 'complete':
|
||||
logger.info(f"{worker_name.capitalize()} worker already complete for job {job_id}, skipping")
|
||||
return WorkerResponse(
|
||||
status="already_complete",
|
||||
job_id=job_id,
|
||||
message=f"{worker_name.capitalize()} worker already completed"
|
||||
)
|
||||
|
||||
# Mark as running before starting (for idempotency on next retry)
|
||||
job_manager.update_state_data(job_id, progress_key, {'stage': 'running'})
|
||||
return None
|
||||
|
||||
|
||||
@router.post("/workers/audio", response_model=WorkerResponse)
|
||||
async def trigger_audio_worker(
|
||||
request: WorkerRequest,
|
||||
@@ -47,11 +102,18 @@ async def trigger_audio_worker(
|
||||
This endpoint is called internally after job creation to start
|
||||
the audio processing track (parallel with lyrics processing).
|
||||
|
||||
Idempotency: If worker is already running or complete, returns early.
|
||||
|
||||
The worker runs in the background and updates job state as it progresses.
|
||||
"""
|
||||
job_id = request.job_id
|
||||
logger.info(f"Triggering audio worker for job {job_id}")
|
||||
|
||||
# Idempotency check
|
||||
skip_response = _check_worker_idempotency(job_id, "audio")
|
||||
if skip_response:
|
||||
return skip_response
|
||||
|
||||
# Add task to background tasks
|
||||
# This allows the HTTP response to return immediately
|
||||
# while the worker continues processing
|
||||
@@ -76,11 +138,18 @@ async def trigger_lyrics_worker(
|
||||
This endpoint is called internally after job creation to start
|
||||
the lyrics processing track (parallel with audio processing).
|
||||
|
||||
Idempotency: If worker is already running or complete, returns early.
|
||||
|
||||
The worker runs in the background and updates job state as it progresses.
|
||||
"""
|
||||
job_id = request.job_id
|
||||
logger.info(f"Triggering lyrics worker for job {job_id}")
|
||||
|
||||
# Idempotency check
|
||||
skip_response = _check_worker_idempotency(job_id, "lyrics")
|
||||
if skip_response:
|
||||
return skip_response
|
||||
|
||||
# Add task to background tasks
|
||||
background_tasks.add_task(process_lyrics_transcription, job_id)
|
||||
|
||||
@@ -101,10 +170,17 @@ async def trigger_screens_worker(
|
||||
Trigger title/end screen generation worker.
|
||||
|
||||
This is called automatically when both audio and lyrics are complete.
|
||||
|
||||
Idempotency: If worker is already running or complete, returns early.
|
||||
"""
|
||||
job_id = request.job_id
|
||||
logger.info(f"Triggering screens worker for job {job_id}")
|
||||
|
||||
# Idempotency check
|
||||
skip_response = _check_worker_idempotency(job_id, "screens")
|
||||
if skip_response:
|
||||
return skip_response
|
||||
|
||||
# Add task to background tasks
|
||||
background_tasks.add_task(generate_screens, job_id)
|
||||
|
||||
@@ -126,10 +202,17 @@ async def trigger_video_worker(
|
||||
|
||||
This is called after user selects their preferred instrumental.
|
||||
This is the longest-running stage (15-20 minutes).
|
||||
|
||||
Idempotency: If worker is already running or complete, returns early.
|
||||
"""
|
||||
job_id = request.job_id
|
||||
logger.info(f"Triggering video worker for job {job_id}")
|
||||
|
||||
# Idempotency check
|
||||
skip_response = _check_worker_idempotency(job_id, "video")
|
||||
if skip_response:
|
||||
return skip_response
|
||||
|
||||
# Add task to background tasks
|
||||
background_tasks.add_task(generate_video, job_id)
|
||||
|
||||
@@ -153,12 +236,19 @@ async def trigger_render_video_worker(
|
||||
Uses OutputGenerator from LyricsTranscriber to generate the karaoke video
|
||||
with the corrected lyrics.
|
||||
|
||||
Idempotency: If worker is already running or complete, returns early.
|
||||
|
||||
Output: with_vocals.mkv in GCS
|
||||
Next state: AWAITING_INSTRUMENTAL_SELECTION
|
||||
"""
|
||||
job_id = request.job_id
|
||||
logger.info(f"Triggering render-video worker for job {job_id}")
|
||||
|
||||
# Idempotency check
|
||||
skip_response = _check_worker_idempotency(job_id, "render")
|
||||
if skip_response:
|
||||
return skip_response
|
||||
|
||||
# Add task to background tasks
|
||||
background_tasks.add_task(process_render_video, job_id)
|
||||
|
||||
|
||||
@@ -19,7 +19,8 @@ from backend.models.requests import (
|
||||
CorrectionsSubmission,
|
||||
InstrumentalSelection,
|
||||
StartReviewRequest,
|
||||
CancelJobRequest
|
||||
CancelJobRequest,
|
||||
CreateCustomInstrumentalRequest,
|
||||
)
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.worker_service import get_worker_service
|
||||
@@ -523,6 +524,311 @@ async def get_instrumental_options(job_id: str) -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
@router.get("/{job_id}/instrumental-analysis")
|
||||
async def get_instrumental_analysis(job_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Get audio analysis data for instrumental selection.
|
||||
|
||||
Returns:
|
||||
- Analysis of backing vocals (audible segments, recommendation)
|
||||
- Waveform image URL
|
||||
- Audio stream URLs for playback
|
||||
|
||||
This endpoint enables intelligent instrumental selection by providing
|
||||
detailed analysis of the backing vocals track.
|
||||
"""
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
if job.status != JobStatus.AWAITING_INSTRUMENTAL_SELECTION:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Job not ready for instrumental analysis (current status: {job.status})"
|
||||
)
|
||||
|
||||
storage = StorageService()
|
||||
|
||||
# Get analysis from state_data (populated by render_video_worker)
|
||||
analysis_data = job.state_data.get('backing_vocals_analysis', {})
|
||||
|
||||
# Get URLs
|
||||
stems = job.file_urls.get('stems', {})
|
||||
analysis_files = job.file_urls.get('analysis', {})
|
||||
|
||||
clean_url = stems.get('instrumental_clean')
|
||||
backing_vocals_url = stems.get('backing_vocals')
|
||||
with_backing_url = stems.get('instrumental_with_backing')
|
||||
waveform_url = analysis_files.get('backing_vocals_waveform')
|
||||
custom_instrumental_url = stems.get('instrumental_custom')
|
||||
|
||||
# Build response
|
||||
response = {
|
||||
"job_id": job_id,
|
||||
"artist": job.artist,
|
||||
"title": job.title,
|
||||
"status": job.status,
|
||||
|
||||
# Analysis results
|
||||
"analysis": {
|
||||
"has_audible_content": analysis_data.get('has_audible_content', False),
|
||||
"total_duration_seconds": analysis_data.get('total_duration_seconds', 0),
|
||||
"audible_segments": analysis_data.get('audible_segments', []),
|
||||
"recommended_selection": analysis_data.get('recommended_selection', 'review_needed'),
|
||||
"total_audible_duration_seconds": analysis_data.get('total_audible_duration_seconds', 0),
|
||||
"audible_percentage": analysis_data.get('audible_percentage', 0),
|
||||
"silence_threshold_db": analysis_data.get('silence_threshold_db', -40.0),
|
||||
},
|
||||
|
||||
# Audio URLs for playback
|
||||
"audio_urls": {
|
||||
"clean_instrumental": storage.generate_signed_url(clean_url, expiration_minutes=120) if clean_url else None,
|
||||
"backing_vocals": storage.generate_signed_url(backing_vocals_url, expiration_minutes=120) if backing_vocals_url else None,
|
||||
"with_backing": storage.generate_signed_url(with_backing_url, expiration_minutes=120) if with_backing_url else None,
|
||||
"custom_instrumental": storage.generate_signed_url(custom_instrumental_url, expiration_minutes=120) if custom_instrumental_url else None,
|
||||
},
|
||||
|
||||
# Waveform image URL
|
||||
"waveform_url": storage.generate_signed_url(waveform_url, expiration_minutes=120) if waveform_url else None,
|
||||
|
||||
# Whether a custom instrumental has been created
|
||||
"has_custom_instrumental": custom_instrumental_url is not None,
|
||||
}
|
||||
|
||||
return response
|
||||
|
||||
|
||||
@router.get("/{job_id}/audio-stream/{stem_type}")
|
||||
async def stream_audio(job_id: str, stem_type: str):
|
||||
"""
|
||||
Stream an audio file for playback in the browser.
|
||||
|
||||
Supported stem_type values:
|
||||
- clean_instrumental: Clean instrumental (no backing vocals)
|
||||
- backing_vocals: Backing vocals only
|
||||
- with_backing: Instrumental with backing vocals
|
||||
- custom_instrumental: Custom instrumental (if created)
|
||||
|
||||
Returns audio as a streaming response with proper headers for
|
||||
browser audio playback.
|
||||
"""
|
||||
from fastapi.responses import StreamingResponse
|
||||
import tempfile
|
||||
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
# Map stem_type to file_urls key
|
||||
stem_map = {
|
||||
'clean_instrumental': ('stems', 'instrumental_clean'),
|
||||
'backing_vocals': ('stems', 'backing_vocals'),
|
||||
'with_backing': ('stems', 'instrumental_with_backing'),
|
||||
'custom_instrumental': ('stems', 'instrumental_custom'),
|
||||
}
|
||||
|
||||
if stem_type not in stem_map:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid stem_type. Must be one of: {list(stem_map.keys())}"
|
||||
)
|
||||
|
||||
category, key = stem_map[stem_type]
|
||||
gcs_path = job.file_urls.get(category, {}).get(key)
|
||||
|
||||
if not gcs_path:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"Audio file not available: {stem_type}"
|
||||
)
|
||||
|
||||
# Determine content type
|
||||
ext = gcs_path.split('.')[-1].lower()
|
||||
content_types = {
|
||||
'flac': 'audio/flac',
|
||||
'mp3': 'audio/mpeg',
|
||||
'wav': 'audio/wav',
|
||||
'm4a': 'audio/mp4',
|
||||
}
|
||||
content_type = content_types.get(ext, 'audio/flac')
|
||||
|
||||
try:
|
||||
storage = StorageService()
|
||||
with tempfile.NamedTemporaryFile(delete=False, suffix=f'.{ext}') as tmp:
|
||||
tmp_path = tmp.name
|
||||
|
||||
storage.download_file(gcs_path, tmp_path)
|
||||
|
||||
def file_iterator():
|
||||
try:
|
||||
with open(tmp_path, 'rb') as f:
|
||||
while chunk := f.read(8192):
|
||||
yield chunk
|
||||
finally:
|
||||
import os
|
||||
os.unlink(tmp_path)
|
||||
|
||||
filename = gcs_path.split('/')[-1]
|
||||
|
||||
return StreamingResponse(
|
||||
file_iterator(),
|
||||
media_type=content_type,
|
||||
headers={
|
||||
'Content-Disposition': f'inline; filename="{filename}"',
|
||||
'Accept-Ranges': 'bytes',
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
logger.exception(f"Error streaming audio {gcs_path}: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Error streaming audio: {e}") from e
|
||||
|
||||
|
||||
@router.post("/{job_id}/create-custom-instrumental")
|
||||
async def create_custom_instrumental(
|
||||
job_id: str,
|
||||
request: CreateCustomInstrumentalRequest,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Create a custom instrumental by muting regions of backing vocals.
|
||||
|
||||
This endpoint:
|
||||
1. Takes a list of time ranges to mute in the backing vocals
|
||||
2. Creates a custom instrumental (clean + muted backing vocals)
|
||||
3. Stores the result and makes it available for selection
|
||||
|
||||
After calling this endpoint, the user can select "custom" in the
|
||||
select-instrumental endpoint.
|
||||
"""
|
||||
from backend.services.audio_editing_service import AudioEditingService
|
||||
from karaoke_gen.instrumental_review import MuteRegion
|
||||
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
if job.status != JobStatus.AWAITING_INSTRUMENTAL_SELECTION:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Job not ready for custom instrumental creation (current status: {job.status})"
|
||||
)
|
||||
|
||||
# Get stem paths
|
||||
stems = job.file_urls.get('stems', {})
|
||||
clean_path = stems.get('instrumental_clean')
|
||||
backing_path = stems.get('backing_vocals')
|
||||
|
||||
if not clean_path or not backing_path:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail="Required audio files not available"
|
||||
)
|
||||
|
||||
try:
|
||||
# Convert request mute regions to model objects
|
||||
mute_regions = [
|
||||
MuteRegion(
|
||||
start_seconds=r.start_seconds,
|
||||
end_seconds=r.end_seconds
|
||||
)
|
||||
for r in request.mute_regions
|
||||
]
|
||||
|
||||
# Create custom instrumental
|
||||
editing_service = AudioEditingService()
|
||||
|
||||
# Determine output path
|
||||
output_path = f"jobs/{job_id}/stems/instrumental_custom.flac"
|
||||
|
||||
result = editing_service.create_custom_instrumental(
|
||||
gcs_clean_instrumental_path=clean_path,
|
||||
gcs_backing_vocals_path=backing_path,
|
||||
mute_regions=mute_regions,
|
||||
gcs_output_path=output_path,
|
||||
job_id=job_id,
|
||||
)
|
||||
|
||||
# Update job file_urls with custom instrumental
|
||||
job_manager.update_file_url(job_id, 'stems', 'instrumental_custom', output_path)
|
||||
|
||||
# Store mute regions in state_data for reference
|
||||
job_manager.update_state_data(job_id, 'custom_instrumental_mute_regions', [
|
||||
{"start_seconds": r.start_seconds, "end_seconds": r.end_seconds}
|
||||
for r in mute_regions
|
||||
])
|
||||
|
||||
logger.info(f"Job {job_id}: Custom instrumental created with {len(mute_regions)} mute regions")
|
||||
|
||||
# Generate signed URL for the new file
|
||||
storage = StorageService()
|
||||
|
||||
return {
|
||||
"status": "success",
|
||||
"message": "Custom instrumental created successfully",
|
||||
"custom_instrumental_url": storage.generate_signed_url(output_path, expiration_minutes=120),
|
||||
"mute_regions_applied": len(result.mute_regions_applied),
|
||||
"total_muted_duration_seconds": result.total_muted_duration_seconds,
|
||||
"output_duration_seconds": result.output_duration_seconds,
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error creating custom instrumental for job {job_id}: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e)) from e
|
||||
|
||||
|
||||
@router.get("/{job_id}/waveform-data")
|
||||
async def get_waveform_data(job_id: str, num_points: int = 500) -> Dict[str, Any]:
|
||||
"""
|
||||
Get waveform amplitude data for client-side rendering.
|
||||
|
||||
This endpoint returns raw amplitude data that the frontend can use
|
||||
to render a waveform using Canvas or SVG, enabling interactive
|
||||
features like click-to-seek.
|
||||
|
||||
Args:
|
||||
num_points: Number of data points to return (default 500)
|
||||
|
||||
Returns:
|
||||
{
|
||||
"amplitudes": [0.1, 0.2, ...], # Normalized 0-1 values
|
||||
"duration_seconds": 180.5,
|
||||
"num_points": 500
|
||||
}
|
||||
"""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
|
||||
if job.status != JobStatus.AWAITING_INSTRUMENTAL_SELECTION:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Job not ready for waveform data (current status: {job.status})"
|
||||
)
|
||||
|
||||
backing_vocals_path = job.file_urls.get('stems', {}).get('backing_vocals')
|
||||
if not backing_vocals_path:
|
||||
raise HTTPException(status_code=404, detail="Backing vocals file not found")
|
||||
|
||||
try:
|
||||
analysis_service = AudioAnalysisService()
|
||||
amplitudes, duration = analysis_service.get_waveform_data(
|
||||
gcs_audio_path=backing_vocals_path,
|
||||
job_id=job_id,
|
||||
num_points=num_points,
|
||||
)
|
||||
|
||||
return {
|
||||
"amplitudes": amplitudes,
|
||||
"duration_seconds": duration,
|
||||
"num_points": len(amplitudes),
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.exception(f"Error getting waveform data for job {job_id}: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e)) from e
|
||||
|
||||
|
||||
@router.post("/{job_id}/select-instrumental")
|
||||
async def select_instrumental(
|
||||
job_id: str,
|
||||
|
||||
@@ -41,9 +41,27 @@ class Settings(BaseSettings):
|
||||
max_concurrent_jobs: int = int(os.getenv("MAX_CONCURRENT_JOBS", "5"))
|
||||
job_timeout_seconds: int = int(os.getenv("JOB_TIMEOUT_SECONDS", "3600"))
|
||||
|
||||
# Cloud Tasks (for scalable worker coordination)
|
||||
# When enabled, workers are triggered via Cloud Tasks for guaranteed delivery
|
||||
# When disabled (default), workers are triggered via direct HTTP (for development)
|
||||
enable_cloud_tasks: bool = os.getenv("ENABLE_CLOUD_TASKS", "false").lower() in ("true", "1", "yes")
|
||||
gcp_region: str = os.getenv("GCP_REGION", "us-central1")
|
||||
|
||||
# Cloud Run Jobs (for long-running video encoding)
|
||||
# When enabled AND enable_cloud_tasks is true, video worker uses Cloud Run Jobs
|
||||
# instead of Cloud Tasks. This supports encoding times >30 minutes (up to 24 hours).
|
||||
# Default is false - Cloud Tasks is sufficient for most videos (15-20 min).
|
||||
use_cloud_run_jobs_for_video: bool = os.getenv("USE_CLOUD_RUN_JOBS_FOR_VIDEO", "false").lower() in ("true", "1", "yes")
|
||||
|
||||
# Storage paths
|
||||
temp_dir: str = os.getenv("TEMP_DIR", "/tmp/karaoke-gen")
|
||||
|
||||
# Flacfetch remote service (for torrent downloads)
|
||||
# When configured, audio search uses the remote flacfetch HTTP API instead of local flacfetch.
|
||||
# This is required for torrent downloads since Cloud Run doesn't support BitTorrent.
|
||||
flacfetch_api_url: Optional[str] = os.getenv("FLACFETCH_API_URL") # e.g., http://10.0.0.5:8080
|
||||
flacfetch_api_key: Optional[str] = os.getenv("FLACFETCH_API_KEY")
|
||||
|
||||
# Default distribution settings (can be overridden per-request)
|
||||
default_dropbox_path: Optional[str] = os.getenv("DEFAULT_DROPBOX_PATH")
|
||||
default_gdrive_folder_id: Optional[str] = os.getenv("DEFAULT_GDRIVE_FOLDER_ID")
|
||||
|
||||
@@ -7,7 +7,7 @@ from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from backend.config import settings
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth
|
||||
from backend.api.routes import health, jobs, internal, file_upload, review, auth, audio_search
|
||||
from backend.services.tracing import setup_tracing, instrument_app, get_current_trace_id
|
||||
|
||||
|
||||
@@ -108,6 +108,7 @@ app.include_router(file_upload.router, prefix="/api") # File upload endpoint
|
||||
app.include_router(internal.router, prefix="/api") # Internal worker endpoints
|
||||
app.include_router(review.router, prefix="/api") # Review UI compatibility endpoints
|
||||
app.include_router(auth.router, prefix="/api") # OAuth credential management
|
||||
app.include_router(audio_search.router, prefix="/api") # Audio search (artist+title mode)
|
||||
|
||||
|
||||
@app.get("/")
|
||||
|
||||
@@ -14,8 +14,8 @@ class JobStatus(str, Enum):
|
||||
"""
|
||||
Job status enumeration - Complete state machine.
|
||||
|
||||
The workflow has 8 main stages with 2 human interaction points:
|
||||
1. Input & Setup
|
||||
The workflow has 8 main stages with 3 human interaction points:
|
||||
1. Input & Setup (may include audio source selection)
|
||||
2. Parallel Processing (audio + lyrics)
|
||||
3. Title/End Screen Generation
|
||||
4. Countdown Padding Synchronization
|
||||
@@ -26,6 +26,12 @@ class JobStatus(str, Enum):
|
||||
"""
|
||||
# Initial states
|
||||
PENDING = "pending" # Job created, queued for processing
|
||||
|
||||
# Audio search states (for artist+title search mode)
|
||||
SEARCHING_AUDIO = "searching_audio" # Searching for audio sources via flacfetch
|
||||
AWAITING_AUDIO_SELECTION = "awaiting_audio_selection" # ⚠️ WAITING FOR USER - select audio source
|
||||
DOWNLOADING_AUDIO = "downloading_audio" # Downloading selected audio from source
|
||||
|
||||
DOWNLOADING = "downloading" # Downloading from URL or processing upload
|
||||
|
||||
# Stage 2a: Audio separation (parallel track 1)
|
||||
@@ -67,6 +73,7 @@ class JobStatus(str, Enum):
|
||||
|
||||
# Terminal states
|
||||
COMPLETE = "complete" # All processing finished successfully
|
||||
PREP_COMPLETE = "prep_complete" # Prep-only job completed (stops after review)
|
||||
FAILED = "failed" # Unrecoverable error occurred
|
||||
CANCELLED = "cancelled" # User cancelled the job
|
||||
|
||||
@@ -80,7 +87,14 @@ class JobStatus(str, Enum):
|
||||
|
||||
# Valid state transitions
|
||||
STATE_TRANSITIONS = {
|
||||
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
# PENDING can go to DOWNLOADING (file upload), SEARCHING_AUDIO (artist+title search), or AWAITING_INSTRUMENTAL_SELECTION (finalise-only)
|
||||
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
|
||||
# Audio search flow (for artist+title search mode)
|
||||
JobStatus.SEARCHING_AUDIO: [JobStatus.AWAITING_AUDIO_SELECTION, JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED],
|
||||
JobStatus.AWAITING_AUDIO_SELECTION: [JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
JobStatus.DOWNLOADING_AUDIO: [JobStatus.DOWNLOADING, JobStatus.FAILED],
|
||||
|
||||
# DOWNLOADING allows parallel processing (audio + lyrics) and then screens when both complete
|
||||
JobStatus.DOWNLOADING: [JobStatus.SEPARATING_STAGE1, JobStatus.TRANSCRIBING, JobStatus.GENERATING_SCREENS, JobStatus.FAILED],
|
||||
|
||||
@@ -102,10 +116,10 @@ STATE_TRANSITIONS = {
|
||||
# AWAITING_REVIEW can go directly to REVIEW_COMPLETE (quick review) or to IN_REVIEW (editing)
|
||||
JobStatus.AWAITING_REVIEW: [JobStatus.IN_REVIEW, JobStatus.REVIEW_COMPLETE, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
JobStatus.IN_REVIEW: [JobStatus.REVIEW_COMPLETE, JobStatus.AWAITING_REVIEW, JobStatus.FAILED],
|
||||
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.FAILED],
|
||||
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.PREP_COMPLETE, JobStatus.FAILED], # PREP_COMPLETE for prep-only jobs
|
||||
|
||||
# Video rendering (post-review)
|
||||
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED],
|
||||
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.PREP_COMPLETE, JobStatus.FAILED],
|
||||
|
||||
# Instrumental selection flow
|
||||
JobStatus.AWAITING_INSTRUMENTAL_SELECTION: [JobStatus.INSTRUMENTAL_SELECTED, JobStatus.FAILED, JobStatus.CANCELLED],
|
||||
@@ -120,9 +134,11 @@ STATE_TRANSITIONS = {
|
||||
JobStatus.UPLOADING: [JobStatus.NOTIFYING, JobStatus.COMPLETE, JobStatus.FAILED],
|
||||
JobStatus.NOTIFYING: [JobStatus.COMPLETE, JobStatus.FAILED],
|
||||
|
||||
# Terminal states - COMPLETE and CANCELLED have no transitions
|
||||
# Terminal states - COMPLETE, PREP_COMPLETE, and CANCELLED have no transitions
|
||||
# FAILED allows retry transitions to resume from checkpoints
|
||||
# PREP_COMPLETE allows finalise-only continuation
|
||||
JobStatus.COMPLETE: [],
|
||||
JobStatus.PREP_COMPLETE: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED], # Finalise-only continues from here
|
||||
JobStatus.FAILED: [
|
||||
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
|
||||
JobStatus.REVIEW_COMPLETE, # Retry from render stage
|
||||
@@ -211,6 +227,30 @@ class Job(BaseModel):
|
||||
# Legacy distribution configuration (rclone - for local CLI backward compat)
|
||||
organised_dir_rclone_root: Optional[str] = None # Deprecated: use dropbox_path instead
|
||||
|
||||
# Lyrics configuration (overrides for search/transcription)
|
||||
lyrics_artist: Optional[str] = None # Override artist name for lyrics search
|
||||
lyrics_title: Optional[str] = None # Override title for lyrics search
|
||||
lyrics_file_gcs_path: Optional[str] = None # GCS path to user-provided lyrics file
|
||||
subtitle_offset_ms: int = 0 # Offset for subtitle timing (positive = delay)
|
||||
|
||||
# Audio separation model configuration
|
||||
clean_instrumental_model: Optional[str] = None # Model for clean instrumental separation
|
||||
backing_vocals_models: Optional[List[str]] = None # Models for backing vocals separation
|
||||
other_stems_models: Optional[List[str]] = None # Models for other stems (bass, drums, etc.)
|
||||
|
||||
# Existing instrumental configuration (Batch 3)
|
||||
existing_instrumental_gcs_path: Optional[str] = None # GCS path to user-provided instrumental file
|
||||
|
||||
# Audio search configuration (Batch 5 - artist+title search mode)
|
||||
audio_search_artist: Optional[str] = None # Artist name used for audio search
|
||||
audio_search_title: Optional[str] = None # Title used for audio search
|
||||
auto_download: bool = False # Auto-select best audio source (skip selection)
|
||||
|
||||
# Two-phase workflow configuration (Batch 6)
|
||||
prep_only: bool = False # Stop after review, don't run finalisation
|
||||
finalise_only: bool = False # Skip prep, run only finalisation (requires uploaded prep outputs)
|
||||
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
|
||||
|
||||
# Processing state
|
||||
track_output_dir: Optional[str] = None # Local output directory (temp)
|
||||
audio_hash: Optional[str] = None # Hash for deduplication
|
||||
@@ -373,6 +413,30 @@ class JobCreate(BaseModel):
|
||||
# Legacy (rclone - deprecated, use dropbox_path instead)
|
||||
organised_dir_rclone_root: Optional[str] = None
|
||||
|
||||
# Lyrics configuration (overrides for search/transcription)
|
||||
lyrics_artist: Optional[str] = None # Override artist name for lyrics search
|
||||
lyrics_title: Optional[str] = None # Override title for lyrics search
|
||||
lyrics_file_gcs_path: Optional[str] = None # GCS path to user-provided lyrics file
|
||||
subtitle_offset_ms: int = 0 # Offset for subtitle timing (positive = delay)
|
||||
|
||||
# Audio separation model configuration
|
||||
clean_instrumental_model: Optional[str] = None # Model for clean instrumental separation
|
||||
backing_vocals_models: Optional[List[str]] = None # Models for backing vocals separation
|
||||
other_stems_models: Optional[List[str]] = None # Models for other stems (bass, drums, etc.)
|
||||
|
||||
# Existing instrumental configuration (Batch 3)
|
||||
existing_instrumental_gcs_path: Optional[str] = None # GCS path to user-provided instrumental file
|
||||
|
||||
# Audio search configuration (Batch 5 - artist+title search mode)
|
||||
audio_search_artist: Optional[str] = None # Artist name used for audio search
|
||||
audio_search_title: Optional[str] = None # Title used for audio search
|
||||
auto_download: bool = False # Auto-select best audio source (skip selection)
|
||||
|
||||
# Two-phase workflow configuration (Batch 6)
|
||||
prep_only: bool = False # Stop after review, don't run finalisation
|
||||
finalise_only: bool = False # Skip prep, run only finalisation
|
||||
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
|
||||
|
||||
# Request metadata (set by API endpoint from request headers)
|
||||
request_metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
"""
|
||||
|
||||
@@ -59,19 +59,54 @@ class InstrumentalSelection(BaseModel):
|
||||
Request to select instrumental audio option.
|
||||
|
||||
This is the second critical human-in-the-loop interaction point.
|
||||
User chooses between clean instrumental or instrumental with backing vocals.
|
||||
User chooses between clean instrumental, instrumental with backing vocals,
|
||||
or a custom instrumental (created via create-custom-instrumental endpoint).
|
||||
"""
|
||||
selection: str # "clean" or "with_backing"
|
||||
selection: str # "clean", "with_backing", or "custom"
|
||||
|
||||
@validator('selection')
|
||||
def validate_selection(cls, v):
|
||||
"""Validate selection is a valid option."""
|
||||
valid_options = ['clean', 'with_backing']
|
||||
valid_options = ['clean', 'with_backing', 'custom']
|
||||
if v not in valid_options:
|
||||
raise ValueError(f"Selection must be one of: {valid_options}")
|
||||
return v
|
||||
|
||||
|
||||
class MuteRegionRequest(BaseModel):
|
||||
"""A region to mute in the backing vocals."""
|
||||
start_seconds: float
|
||||
end_seconds: float
|
||||
|
||||
@validator('start_seconds')
|
||||
def validate_start(cls, v):
|
||||
if v < 0:
|
||||
raise ValueError("start_seconds must be non-negative")
|
||||
return v
|
||||
|
||||
@validator('end_seconds')
|
||||
def validate_end(cls, v, values):
|
||||
if 'start_seconds' in values and v <= values['start_seconds']:
|
||||
raise ValueError("end_seconds must be greater than start_seconds")
|
||||
return v
|
||||
|
||||
|
||||
class CreateCustomInstrumentalRequest(BaseModel):
|
||||
"""
|
||||
Request to create a custom instrumental with muted backing vocal regions.
|
||||
|
||||
The mute_regions specify time ranges in the backing vocals track that
|
||||
should be silenced before mixing with the clean instrumental.
|
||||
"""
|
||||
mute_regions: List[MuteRegionRequest]
|
||||
|
||||
@validator('mute_regions')
|
||||
def validate_regions(cls, v):
|
||||
if not v:
|
||||
raise ValueError("At least one mute region is required")
|
||||
return v
|
||||
|
||||
|
||||
class StartReviewRequest(BaseModel):
|
||||
"""Request to mark job as in-review (user opened interface)."""
|
||||
pass # No body needed, just triggers state transition
|
||||
|
||||
@@ -11,6 +11,8 @@ httpx>=0.25.0,<1.0.0
|
||||
google-cloud-firestore>=2.14.0,<3.0.0
|
||||
google-cloud-storage>=2.14.0,<3.0.0
|
||||
google-cloud-secret-manager>=2.18.0,<3.0.0
|
||||
google-cloud-tasks>=2.14.0,<3.0.0
|
||||
google-cloud-run>=0.10.0,<1.0.0
|
||||
python-dotenv>=1.0.0,<2.0.0
|
||||
|
||||
# Audio processing (reuses karaoke_gen library)
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
"""
|
||||
Audio analysis service for analyzing backing vocals.
|
||||
|
||||
This service wraps the shared karaoke_gen.instrumental_review module
|
||||
to provide GCS-integrated audio analysis capabilities.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
from typing import Optional
|
||||
|
||||
from backend.services.storage_service import StorageService
|
||||
from karaoke_gen.instrumental_review import (
|
||||
AudioAnalyzer,
|
||||
AnalysisResult,
|
||||
WaveformGenerator,
|
||||
)
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AudioAnalysisService:
|
||||
"""
|
||||
Service for analyzing backing vocals audio files stored in GCS.
|
||||
|
||||
This service acts as a thin wrapper around the shared AudioAnalyzer
|
||||
and WaveformGenerator classes, handling GCS download/upload operations.
|
||||
|
||||
The actual analysis logic is in the shared karaoke_gen.instrumental_review
|
||||
module, ensuring feature parity between local and remote workflows.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
storage_service: Optional[StorageService] = None,
|
||||
silence_threshold_db: float = -40.0,
|
||||
min_segment_duration_ms: int = 100,
|
||||
):
|
||||
"""
|
||||
Initialize the audio analysis service.
|
||||
|
||||
Args:
|
||||
storage_service: GCS storage service. If not provided, a new
|
||||
instance will be created.
|
||||
silence_threshold_db: Threshold for considering audio as silent.
|
||||
Default is -40.0 dB.
|
||||
min_segment_duration_ms: Minimum duration for audible segments.
|
||||
Default is 100ms.
|
||||
"""
|
||||
self.storage_service = storage_service or StorageService()
|
||||
self.analyzer = AudioAnalyzer(
|
||||
silence_threshold_db=silence_threshold_db,
|
||||
min_segment_duration_ms=min_segment_duration_ms,
|
||||
)
|
||||
self.waveform_generator = WaveformGenerator()
|
||||
|
||||
def analyze_backing_vocals(
|
||||
self,
|
||||
gcs_audio_path: str,
|
||||
job_id: str,
|
||||
) -> AnalysisResult:
|
||||
"""
|
||||
Analyze a backing vocals audio file from GCS.
|
||||
|
||||
This method:
|
||||
1. Downloads the audio file from GCS to a temp file
|
||||
2. Runs the analysis using the shared AudioAnalyzer
|
||||
3. Returns the analysis result
|
||||
|
||||
Args:
|
||||
gcs_audio_path: Path to the audio file in GCS
|
||||
job_id: Job ID for logging
|
||||
|
||||
Returns:
|
||||
AnalysisResult containing analysis data
|
||||
"""
|
||||
logger.info(f"[{job_id}] Analyzing backing vocals: {gcs_audio_path}")
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Download audio file
|
||||
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
|
||||
self.storage_service.download_file(gcs_audio_path, local_audio_path)
|
||||
|
||||
# Run analysis
|
||||
result = self.analyzer.analyze(local_audio_path)
|
||||
|
||||
logger.info(
|
||||
f"[{job_id}] Analysis complete: "
|
||||
f"has_audible={result.has_audible_content}, "
|
||||
f"segments={result.segment_count}, "
|
||||
f"recommendation={result.recommended_selection.value}"
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
def analyze_and_generate_waveform(
|
||||
self,
|
||||
gcs_audio_path: str,
|
||||
job_id: str,
|
||||
gcs_waveform_destination: str,
|
||||
) -> tuple[AnalysisResult, str]:
|
||||
"""
|
||||
Analyze backing vocals and generate a waveform image.
|
||||
|
||||
This method:
|
||||
1. Downloads the audio file from GCS
|
||||
2. Runs analysis using AudioAnalyzer
|
||||
3. Generates waveform image using WaveformGenerator
|
||||
4. Uploads the waveform image to GCS
|
||||
5. Returns analysis result and waveform GCS path
|
||||
|
||||
Args:
|
||||
gcs_audio_path: Path to the audio file in GCS
|
||||
job_id: Job ID for logging
|
||||
gcs_waveform_destination: Where to upload the waveform image in GCS
|
||||
|
||||
Returns:
|
||||
Tuple of (AnalysisResult, waveform_gcs_path)
|
||||
"""
|
||||
logger.info(f"[{job_id}] Analyzing and generating waveform: {gcs_audio_path}")
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Download audio file
|
||||
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
|
||||
self.storage_service.download_file(gcs_audio_path, local_audio_path)
|
||||
|
||||
# Run analysis
|
||||
result = self.analyzer.analyze(local_audio_path)
|
||||
|
||||
# Generate waveform
|
||||
local_waveform_path = os.path.join(temp_dir, "waveform.png")
|
||||
self.waveform_generator.generate(
|
||||
audio_path=local_audio_path,
|
||||
output_path=local_waveform_path,
|
||||
segments=result.audible_segments,
|
||||
show_time_axis=True,
|
||||
silence_threshold_db=self.analyzer.silence_threshold_db,
|
||||
)
|
||||
|
||||
# Upload waveform to GCS
|
||||
self.storage_service.upload_file(
|
||||
local_waveform_path,
|
||||
gcs_waveform_destination
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"[{job_id}] Analysis and waveform generation complete. "
|
||||
f"Waveform uploaded to: {gcs_waveform_destination}"
|
||||
)
|
||||
|
||||
return result, gcs_waveform_destination
|
||||
|
||||
def get_waveform_data(
|
||||
self,
|
||||
gcs_audio_path: str,
|
||||
job_id: str,
|
||||
num_points: int = 500,
|
||||
) -> tuple[list[float], float]:
|
||||
"""
|
||||
Get waveform data (amplitude envelope) for client-side rendering.
|
||||
|
||||
This is useful when the frontend wants to render the waveform
|
||||
itself using Canvas or SVG, rather than displaying a pre-generated
|
||||
image.
|
||||
|
||||
Args:
|
||||
gcs_audio_path: Path to the audio file in GCS
|
||||
job_id: Job ID for logging
|
||||
num_points: Number of data points to return
|
||||
|
||||
Returns:
|
||||
Tuple of (amplitude_values, duration_seconds)
|
||||
"""
|
||||
logger.info(f"[{job_id}] Getting waveform data: {gcs_audio_path}")
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Download audio file
|
||||
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
|
||||
self.storage_service.download_file(gcs_audio_path, local_audio_path)
|
||||
|
||||
# Generate waveform data
|
||||
amplitudes, duration = self.waveform_generator.generate_data_only(
|
||||
audio_path=local_audio_path,
|
||||
num_points=num_points,
|
||||
)
|
||||
|
||||
return amplitudes, duration
|
||||
|
||||
def generate_waveform_with_mutes(
|
||||
self,
|
||||
gcs_audio_path: str,
|
||||
job_id: str,
|
||||
gcs_waveform_destination: str,
|
||||
mute_regions: list,
|
||||
) -> str:
|
||||
"""
|
||||
Generate a waveform image with mute regions highlighted.
|
||||
|
||||
This is useful for showing the user which regions will be muted
|
||||
in the custom instrumental.
|
||||
|
||||
Args:
|
||||
gcs_audio_path: Path to the audio file in GCS
|
||||
job_id: Job ID for logging
|
||||
gcs_waveform_destination: Where to upload the waveform image
|
||||
mute_regions: List of MuteRegion objects to highlight
|
||||
|
||||
Returns:
|
||||
GCS path to the uploaded waveform image
|
||||
"""
|
||||
from karaoke_gen.instrumental_review import MuteRegion
|
||||
|
||||
logger.info(
|
||||
f"[{job_id}] Generating waveform with {len(mute_regions)} mute regions"
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Download audio file
|
||||
local_audio_path = os.path.join(temp_dir, "backing_vocals.flac")
|
||||
self.storage_service.download_file(gcs_audio_path, local_audio_path)
|
||||
|
||||
# First run analysis to get segments
|
||||
result = self.analyzer.analyze(local_audio_path)
|
||||
|
||||
# Generate waveform with mute regions
|
||||
local_waveform_path = os.path.join(temp_dir, "waveform_with_mutes.png")
|
||||
self.waveform_generator.generate(
|
||||
audio_path=local_audio_path,
|
||||
output_path=local_waveform_path,
|
||||
segments=result.audible_segments,
|
||||
mute_regions=mute_regions,
|
||||
show_time_axis=True,
|
||||
)
|
||||
|
||||
# Upload to GCS
|
||||
self.storage_service.upload_file(
|
||||
local_waveform_path,
|
||||
gcs_waveform_destination
|
||||
)
|
||||
|
||||
return gcs_waveform_destination
|
||||
@@ -0,0 +1,278 @@
|
||||
"""
|
||||
Audio editing service for creating custom instrumentals.
|
||||
|
||||
This service wraps the shared karaoke_gen.instrumental_review module
|
||||
to provide GCS-integrated audio editing capabilities.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import tempfile
|
||||
from typing import List, Optional
|
||||
|
||||
from backend.services.storage_service import StorageService
|
||||
from karaoke_gen.instrumental_review import (
|
||||
AudioEditor,
|
||||
MuteRegion,
|
||||
)
|
||||
from karaoke_gen.instrumental_review.models import CustomInstrumentalResult
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AudioEditingService:
|
||||
"""
|
||||
Service for creating custom instrumental tracks with GCS integration.
|
||||
|
||||
This service acts as a thin wrapper around the shared AudioEditor
|
||||
class, handling GCS download/upload operations.
|
||||
|
||||
The actual editing logic is in the shared karaoke_gen.instrumental_review
|
||||
module, ensuring feature parity between local and remote workflows.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
storage_service: Optional[StorageService] = None,
|
||||
output_format: str = "flac",
|
||||
):
|
||||
"""
|
||||
Initialize the audio editing service.
|
||||
|
||||
Args:
|
||||
storage_service: GCS storage service. If not provided, a new
|
||||
instance will be created.
|
||||
output_format: Output audio format. Default is "flac".
|
||||
"""
|
||||
self.storage_service = storage_service or StorageService()
|
||||
self.editor = AudioEditor(output_format=output_format)
|
||||
|
||||
def create_custom_instrumental(
|
||||
self,
|
||||
gcs_clean_instrumental_path: str,
|
||||
gcs_backing_vocals_path: str,
|
||||
mute_regions: List[MuteRegion],
|
||||
gcs_output_path: str,
|
||||
job_id: str,
|
||||
) -> CustomInstrumentalResult:
|
||||
"""
|
||||
Create a custom instrumental by muting regions of backing vocals.
|
||||
|
||||
This method:
|
||||
1. Downloads clean instrumental and backing vocals from GCS
|
||||
2. Applies mute regions to backing vocals
|
||||
3. Combines with clean instrumental
|
||||
4. Uploads result to GCS
|
||||
|
||||
Args:
|
||||
gcs_clean_instrumental_path: Path to clean instrumental in GCS
|
||||
gcs_backing_vocals_path: Path to backing vocals in GCS
|
||||
mute_regions: List of regions to mute in backing vocals
|
||||
gcs_output_path: Where to upload the custom instrumental in GCS
|
||||
job_id: Job ID for logging
|
||||
|
||||
Returns:
|
||||
CustomInstrumentalResult with details about the created file
|
||||
"""
|
||||
logger.info(
|
||||
f"[{job_id}] Creating custom instrumental with "
|
||||
f"{len(mute_regions)} mute regions"
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Download input files
|
||||
local_clean_path = os.path.join(temp_dir, "clean_instrumental.flac")
|
||||
local_backing_path = os.path.join(temp_dir, "backing_vocals.flac")
|
||||
local_output_path = os.path.join(temp_dir, "custom_instrumental.flac")
|
||||
|
||||
logger.debug(f"[{job_id}] Downloading clean instrumental")
|
||||
self.storage_service.download_file(
|
||||
gcs_clean_instrumental_path,
|
||||
local_clean_path
|
||||
)
|
||||
|
||||
logger.debug(f"[{job_id}] Downloading backing vocals")
|
||||
self.storage_service.download_file(
|
||||
gcs_backing_vocals_path,
|
||||
local_backing_path
|
||||
)
|
||||
|
||||
# Create custom instrumental using shared editor
|
||||
result = self.editor.create_custom_instrumental(
|
||||
clean_instrumental_path=local_clean_path,
|
||||
backing_vocals_path=local_backing_path,
|
||||
mute_regions=mute_regions,
|
||||
output_path=local_output_path,
|
||||
)
|
||||
|
||||
# Upload result to GCS
|
||||
logger.debug(f"[{job_id}] Uploading custom instrumental to GCS")
|
||||
self.storage_service.upload_file(local_output_path, gcs_output_path)
|
||||
|
||||
# Update result with GCS path
|
||||
result.output_path = gcs_output_path
|
||||
|
||||
logger.info(
|
||||
f"[{job_id}] Custom instrumental created: {gcs_output_path}, "
|
||||
f"muted {result.total_muted_duration_seconds:.1f}s across "
|
||||
f"{len(result.mute_regions_applied)} regions"
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
def create_preview(
|
||||
self,
|
||||
gcs_clean_instrumental_path: str,
|
||||
gcs_backing_vocals_path: str,
|
||||
mute_regions: List[MuteRegion],
|
||||
gcs_preview_path: str,
|
||||
job_id: str,
|
||||
preview_duration_seconds: Optional[float] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Create a preview of the custom instrumental.
|
||||
|
||||
This is similar to create_custom_instrumental but optimized for
|
||||
quick preview generation (optionally truncated duration).
|
||||
|
||||
Args:
|
||||
gcs_clean_instrumental_path: Path to clean instrumental in GCS
|
||||
gcs_backing_vocals_path: Path to backing vocals in GCS
|
||||
mute_regions: List of regions to mute
|
||||
gcs_preview_path: Where to upload the preview in GCS
|
||||
job_id: Job ID for logging
|
||||
preview_duration_seconds: Optional max duration for preview
|
||||
|
||||
Returns:
|
||||
GCS path to the uploaded preview
|
||||
"""
|
||||
logger.info(f"[{job_id}] Creating preview with {len(mute_regions)} mute regions")
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Download input files
|
||||
local_clean_path = os.path.join(temp_dir, "clean_instrumental.flac")
|
||||
local_backing_path = os.path.join(temp_dir, "backing_vocals.flac")
|
||||
local_preview_path = os.path.join(temp_dir, "preview.flac")
|
||||
|
||||
self.storage_service.download_file(
|
||||
gcs_clean_instrumental_path,
|
||||
local_clean_path
|
||||
)
|
||||
self.storage_service.download_file(
|
||||
gcs_backing_vocals_path,
|
||||
local_backing_path
|
||||
)
|
||||
|
||||
# Generate preview
|
||||
from pydub import AudioSegment
|
||||
|
||||
preview = self.editor.preview_with_mutes(
|
||||
clean_instrumental_path=local_clean_path,
|
||||
backing_vocals_path=local_backing_path,
|
||||
mute_regions=mute_regions,
|
||||
)
|
||||
|
||||
# Optionally truncate
|
||||
if preview_duration_seconds:
|
||||
max_ms = int(preview_duration_seconds * 1000)
|
||||
preview = preview[:max_ms]
|
||||
|
||||
# Export and upload
|
||||
preview.export(local_preview_path, format="flac")
|
||||
self.storage_service.upload_file(local_preview_path, gcs_preview_path)
|
||||
|
||||
logger.info(f"[{job_id}] Preview created: {gcs_preview_path}")
|
||||
|
||||
return gcs_preview_path
|
||||
|
||||
def mute_backing_vocals_only(
|
||||
self,
|
||||
gcs_backing_vocals_path: str,
|
||||
mute_regions: List[MuteRegion],
|
||||
gcs_output_path: str,
|
||||
job_id: str,
|
||||
) -> str:
|
||||
"""
|
||||
Apply mute regions to backing vocals without combining with instrumental.
|
||||
|
||||
This is useful for creating an edited backing vocals track that
|
||||
can be combined with the instrumental later.
|
||||
|
||||
Args:
|
||||
gcs_backing_vocals_path: Path to backing vocals in GCS
|
||||
mute_regions: List of regions to mute
|
||||
gcs_output_path: Where to upload the edited backing vocals
|
||||
job_id: Job ID for logging
|
||||
|
||||
Returns:
|
||||
GCS path to the uploaded edited backing vocals
|
||||
"""
|
||||
logger.info(
|
||||
f"[{job_id}] Muting backing vocals with {len(mute_regions)} regions"
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
local_backing_path = os.path.join(temp_dir, "backing_vocals.flac")
|
||||
local_output_path = os.path.join(temp_dir, "muted_backing.flac")
|
||||
|
||||
self.storage_service.download_file(
|
||||
gcs_backing_vocals_path,
|
||||
local_backing_path
|
||||
)
|
||||
|
||||
self.editor.apply_mute_to_single_track(
|
||||
audio_path=local_backing_path,
|
||||
mute_regions=mute_regions,
|
||||
output_path=local_output_path,
|
||||
)
|
||||
|
||||
self.storage_service.upload_file(local_output_path, gcs_output_path)
|
||||
|
||||
logger.info(f"[{job_id}] Muted backing vocals: {gcs_output_path}")
|
||||
|
||||
return gcs_output_path
|
||||
|
||||
def validate_mute_regions(
|
||||
self,
|
||||
mute_regions: List[MuteRegion],
|
||||
total_duration_seconds: float,
|
||||
) -> List[str]:
|
||||
"""
|
||||
Validate mute regions for consistency.
|
||||
|
||||
Args:
|
||||
mute_regions: List of mute regions to validate
|
||||
total_duration_seconds: Total duration of the audio
|
||||
|
||||
Returns:
|
||||
List of validation error messages (empty if valid)
|
||||
"""
|
||||
errors = []
|
||||
|
||||
for i, region in enumerate(mute_regions):
|
||||
if region.start_seconds < 0:
|
||||
errors.append(
|
||||
f"Region {i}: start_seconds ({region.start_seconds}) cannot be negative"
|
||||
)
|
||||
|
||||
if region.end_seconds <= region.start_seconds:
|
||||
errors.append(
|
||||
f"Region {i}: end_seconds ({region.end_seconds}) must be after "
|
||||
f"start_seconds ({region.start_seconds})"
|
||||
)
|
||||
|
||||
if region.start_seconds > total_duration_seconds:
|
||||
errors.append(
|
||||
f"Region {i}: start_seconds ({region.start_seconds}) exceeds "
|
||||
f"audio duration ({total_duration_seconds})"
|
||||
)
|
||||
|
||||
if region.end_seconds > total_duration_seconds:
|
||||
# Not an error, but log a warning - the region will be clamped
|
||||
logger.warning(
|
||||
f"Region {i}: end_seconds ({region.end_seconds}) exceeds "
|
||||
f"audio duration ({total_duration_seconds}), will be clamped"
|
||||
)
|
||||
|
||||
return errors
|
||||
@@ -0,0 +1,459 @@
|
||||
"""
|
||||
Audio search service for finding and downloading audio files.
|
||||
|
||||
This service integrates with karaoke_gen.audio_fetcher (which wraps flacfetch)
|
||||
to search for audio from various sources (YouTube, music trackers, etc.)
|
||||
and download the selected audio file.
|
||||
|
||||
This is a thin wrapper that adds backend-specific functionality:
|
||||
- Caching raw results for API-based selection flow
|
||||
- Firestore-compatible serialization
|
||||
- Singleton pattern for service lifecycle
|
||||
- Remote flacfetch service integration for torrent downloads
|
||||
|
||||
When FLACFETCH_API_URL is configured, this service uses the remote flacfetch
|
||||
HTTP API for search and download operations. This is required for torrent
|
||||
downloads since Cloud Run doesn't support BitTorrent peer connections.
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from typing import List, Optional
|
||||
|
||||
# Import shared classes from karaoke_gen.audio_fetcher - single source of truth
|
||||
# Note: Import directly from audio_fetcher module, not karaoke_gen.__init__
|
||||
# to avoid pulling in heavier dependencies like lyrics_transcriber
|
||||
from karaoke_gen.audio_fetcher import (
|
||||
FlacFetcher,
|
||||
AudioSearchResult,
|
||||
AudioFetchResult,
|
||||
AudioFetcherError,
|
||||
NoResultsError,
|
||||
DownloadError,
|
||||
)
|
||||
|
||||
from .flacfetch_client import get_flacfetch_client, FlacfetchServiceError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Re-export exception under backend's naming convention for compatibility
|
||||
AudioSearchError = AudioFetcherError
|
||||
|
||||
|
||||
# Also alias AudioFetchResult as AudioDownloadResult for backwards compatibility
|
||||
AudioDownloadResult = AudioFetchResult
|
||||
|
||||
|
||||
class AudioSearchService:
|
||||
"""
|
||||
Service for searching and downloading audio files.
|
||||
|
||||
This is a thin wrapper around karaoke_gen.audio_fetcher.FlacFetcher that adds
|
||||
backend-specific functionality:
|
||||
- Caches raw results for API-based selection (client picks by index)
|
||||
- Provides Firestore-compatible serialization
|
||||
- Manages service lifecycle as singleton
|
||||
- Remote flacfetch service integration for torrent downloads
|
||||
|
||||
When FLACFETCH_API_URL is configured, this service uses the remote flacfetch
|
||||
HTTP API for search and download operations. This enables torrent downloads
|
||||
which are not possible in Cloud Run due to network restrictions.
|
||||
|
||||
The actual flacfetch integration is in karaoke_gen.audio_fetcher.FlacFetcher,
|
||||
which is shared between local CLI and cloud backend.
|
||||
"""
|
||||
|
||||
# Sentinel value to indicate "use environment variable"
|
||||
_USE_ENV = object()
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
redacted_api_key: Optional[str] = _USE_ENV,
|
||||
ops_api_key: Optional[str] = _USE_ENV,
|
||||
):
|
||||
"""
|
||||
Initialize the audio search service.
|
||||
|
||||
Args:
|
||||
redacted_api_key: API key for Redacted tracker (optional, uses env if not provided)
|
||||
ops_api_key: API key for OPS tracker (optional, uses env if not provided)
|
||||
"""
|
||||
# Use environment variables if not explicitly provided
|
||||
if redacted_api_key is self._USE_ENV:
|
||||
redacted_api_key = os.environ.get("REDACTED_API_KEY")
|
||||
|
||||
if ops_api_key is self._USE_ENV:
|
||||
ops_api_key = os.environ.get("OPS_API_KEY")
|
||||
|
||||
# Check for remote flacfetch client
|
||||
self._remote_client = get_flacfetch_client()
|
||||
|
||||
# Log which mode we're using
|
||||
if self._remote_client:
|
||||
logger.info("AudioSearchService using REMOTE flacfetch service (torrent downloads enabled)")
|
||||
else:
|
||||
logger.info("AudioSearchService using LOCAL flacfetch (YouTube only in Cloud Run)")
|
||||
|
||||
# Delegate to shared FlacFetcher implementation (for local mode or fallback)
|
||||
self._fetcher = FlacFetcher(
|
||||
redacted_api_key=redacted_api_key,
|
||||
ops_api_key=ops_api_key,
|
||||
)
|
||||
|
||||
# Cache search results for API-based selection flow
|
||||
# Key: index, Value: AudioSearchResult (with raw_result)
|
||||
self._cached_results: List[AudioSearchResult] = []
|
||||
|
||||
# Cache for remote search results (search_id -> results)
|
||||
self._remote_search_id: Optional[str] = None
|
||||
|
||||
def search(self, artist: str, title: str) -> List[AudioSearchResult]:
|
||||
"""
|
||||
Search for audio matching the given artist and title.
|
||||
|
||||
Results are cached internally for later download via download().
|
||||
|
||||
If a remote flacfetch service is configured (FLACFETCH_API_URL), uses that
|
||||
for better torrent download support. Otherwise falls back to local flacfetch.
|
||||
|
||||
Args:
|
||||
artist: The artist name to search for
|
||||
title: The track title to search for
|
||||
|
||||
Returns:
|
||||
List of AudioSearchResult objects
|
||||
|
||||
Raises:
|
||||
NoResultsError: If no results are found
|
||||
AudioSearchError: For other errors (e.g., flacfetch not installed)
|
||||
"""
|
||||
try:
|
||||
# Try remote flacfetch service first if configured
|
||||
if self._remote_client:
|
||||
return self._search_remote(artist, title)
|
||||
|
||||
# Fallback to local flacfetch
|
||||
results = self._fetcher.search(artist, title)
|
||||
|
||||
# Cache results for later download
|
||||
self._cached_results = results
|
||||
self._remote_search_id = None
|
||||
|
||||
logger.info(f"Found {len(results)} results for: {artist} - {title}")
|
||||
return results
|
||||
|
||||
except (NoResultsError, AudioFetcherError):
|
||||
# Re-raise these as-is
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Search failed: {e}")
|
||||
raise AudioSearchError(f"Search failed: {e}") from e
|
||||
|
||||
def _search_remote(self, artist: str, title: str) -> List[AudioSearchResult]:
|
||||
"""
|
||||
Search using the remote flacfetch service.
|
||||
|
||||
Runs async code in a sync context for compatibility with existing API.
|
||||
"""
|
||||
try:
|
||||
# Run async search in sync context
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
try:
|
||||
response = loop.run_until_complete(
|
||||
self._remote_client.search(artist, title)
|
||||
)
|
||||
finally:
|
||||
loop.close()
|
||||
|
||||
# Check for empty results
|
||||
if not response.get("results"):
|
||||
raise NoResultsError(f"No results found for: {artist} - {title}")
|
||||
|
||||
# Store search_id for download
|
||||
self._remote_search_id = response.get("search_id")
|
||||
|
||||
# Convert remote results to AudioSearchResult objects
|
||||
results = []
|
||||
for item in response.get("results", []):
|
||||
result = AudioSearchResult(
|
||||
title=item.get("title", "Unknown"),
|
||||
artist=item.get("artist", "Unknown"),
|
||||
provider=item.get("provider", "Unknown"),
|
||||
quality=item.get("quality", ""),
|
||||
size_bytes=item.get("size_bytes"),
|
||||
duration_seconds=item.get("duration_seconds"),
|
||||
is_lossless=item.get("is_lossless", False),
|
||||
seeders=item.get("seeders"),
|
||||
year=item.get("year"),
|
||||
extra_info={
|
||||
"index": item.get("index"),
|
||||
"target_file": item.get("target_file"),
|
||||
"view_count": item.get("view_count"),
|
||||
"channel": item.get("channel"),
|
||||
"remote": True, # Flag for download logic
|
||||
},
|
||||
)
|
||||
results.append(result)
|
||||
|
||||
# Cache results
|
||||
self._cached_results = results
|
||||
|
||||
logger.info(f"Found {len(results)} results from remote flacfetch for: {artist} - {title}")
|
||||
return results
|
||||
|
||||
except FlacfetchServiceError as e:
|
||||
logger.warning(f"Remote flacfetch search failed, falling back to local: {e}")
|
||||
# Fallback to local search
|
||||
self._remote_search_id = None
|
||||
results = self._fetcher.search(artist, title)
|
||||
self._cached_results = results
|
||||
return results
|
||||
|
||||
def select_best(self, results: List[AudioSearchResult]) -> int:
|
||||
"""
|
||||
Select the best result from a list of search results.
|
||||
|
||||
Uses flacfetch's built-in quality ranking to select the best source.
|
||||
|
||||
Args:
|
||||
results: List of search results
|
||||
|
||||
Returns:
|
||||
Index of the best result
|
||||
"""
|
||||
return self._fetcher.select_best(results)
|
||||
|
||||
def download(
|
||||
self,
|
||||
result_index: int,
|
||||
output_dir: str,
|
||||
output_filename: Optional[str] = None,
|
||||
gcs_path: Optional[str] = None,
|
||||
) -> AudioDownloadResult:
|
||||
"""
|
||||
Download audio from a cached search result.
|
||||
|
||||
This method uses the cached results from the last search() call.
|
||||
The API flow is:
|
||||
1. Client calls search() -> gets list of results
|
||||
2. Client picks an index
|
||||
3. Client calls download(index) -> gets downloaded file
|
||||
|
||||
If a remote flacfetch service is configured and the search was performed
|
||||
remotely (for torrent sources), uses the remote service for download.
|
||||
|
||||
Args:
|
||||
result_index: Index of the result to download (from search results)
|
||||
output_dir: Directory to save the downloaded file
|
||||
output_filename: Optional filename (without extension)
|
||||
gcs_path: Optional GCS path for remote uploads (e.g., "uploads/job123/audio/")
|
||||
|
||||
Returns:
|
||||
AudioDownloadResult with the downloaded file path
|
||||
|
||||
Raises:
|
||||
DownloadError: If download fails or no cached result for index
|
||||
"""
|
||||
if result_index < 0 or result_index >= len(self._cached_results):
|
||||
raise DownloadError(
|
||||
f"No cached result for index {result_index}. "
|
||||
f"Available indices: 0-{len(self._cached_results) - 1}. "
|
||||
"Run search() first."
|
||||
)
|
||||
|
||||
result = self._cached_results[result_index]
|
||||
|
||||
logger.info(f"Downloading: {result.artist} - {result.title} from {result.provider}")
|
||||
|
||||
# Check if this was a remote search (torrent sources need remote download)
|
||||
if self._remote_search_id and self._remote_client:
|
||||
# Check if this is a torrent source that needs remote handling
|
||||
is_remote = result.extra_info.get("remote", False) if result.extra_info else False
|
||||
is_torrent = result.provider in ["Redacted", "OPS"]
|
||||
|
||||
if is_remote or is_torrent:
|
||||
return self._download_remote(result_index, output_dir, output_filename, gcs_path)
|
||||
|
||||
# Delegate to shared FlacFetcher (local download)
|
||||
fetch_result = self._fetcher.download(result, output_dir, output_filename)
|
||||
|
||||
logger.info(f"Downloaded to: {fetch_result.filepath}")
|
||||
|
||||
return fetch_result
|
||||
|
||||
def _download_remote(
|
||||
self,
|
||||
result_index: int,
|
||||
output_dir: str,
|
||||
output_filename: Optional[str] = None,
|
||||
gcs_path: Optional[str] = None,
|
||||
) -> AudioDownloadResult:
|
||||
"""
|
||||
Download using the remote flacfetch service.
|
||||
|
||||
The remote service downloads via torrent and optionally uploads to GCS.
|
||||
"""
|
||||
if not self._remote_search_id:
|
||||
raise DownloadError("No remote search ID - run search() first")
|
||||
|
||||
result = self._cached_results[result_index]
|
||||
|
||||
try:
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
try:
|
||||
# Start download
|
||||
download_id = loop.run_until_complete(
|
||||
self._remote_client.download(
|
||||
search_id=self._remote_search_id,
|
||||
result_index=result_index,
|
||||
output_filename=output_filename,
|
||||
gcs_path=gcs_path,
|
||||
)
|
||||
)
|
||||
|
||||
logger.info(f"Remote download started: {download_id}")
|
||||
|
||||
# Wait for completion with progress logging
|
||||
def log_progress(status):
|
||||
progress = status.get("progress", 0)
|
||||
speed = status.get("download_speed_kbps", 0)
|
||||
logger.debug(f"Download progress: {progress:.1f}% ({speed:.1f} KB/s)")
|
||||
|
||||
final_status = loop.run_until_complete(
|
||||
self._remote_client.wait_for_download(
|
||||
download_id,
|
||||
timeout=600, # 10 minute timeout
|
||||
progress_callback=log_progress,
|
||||
)
|
||||
)
|
||||
finally:
|
||||
loop.close()
|
||||
|
||||
# Determine file path
|
||||
filepath = final_status.get("gcs_path") or final_status.get("output_path")
|
||||
|
||||
if not filepath:
|
||||
raise DownloadError("Remote download completed but no file path returned")
|
||||
|
||||
logger.info(f"Remote download complete: {filepath}")
|
||||
|
||||
return AudioDownloadResult(
|
||||
filepath=filepath,
|
||||
artist=result.artist,
|
||||
title=result.title,
|
||||
provider=result.provider,
|
||||
quality=result.quality,
|
||||
)
|
||||
|
||||
except FlacfetchServiceError as e:
|
||||
raise DownloadError(f"Remote download failed: {e}") from e
|
||||
except Exception as e:
|
||||
logger.error(f"Remote download error: {e}")
|
||||
raise DownloadError(f"Remote download failed: {e}") from e
|
||||
|
||||
def search_and_download_auto(
|
||||
self,
|
||||
artist: str,
|
||||
title: str,
|
||||
output_dir: str,
|
||||
output_filename: Optional[str] = None,
|
||||
gcs_path: Optional[str] = None,
|
||||
) -> AudioDownloadResult:
|
||||
"""
|
||||
Search for audio and automatically download the best result.
|
||||
|
||||
This is a convenience method that combines search(), select_best(),
|
||||
and download() for automated/non-interactive usage.
|
||||
|
||||
Args:
|
||||
artist: The artist name to search for
|
||||
title: The track title to search for
|
||||
output_dir: Directory to save the downloaded file
|
||||
output_filename: Optional filename (without extension)
|
||||
gcs_path: Optional GCS path for remote uploads
|
||||
|
||||
Returns:
|
||||
AudioDownloadResult with the downloaded file path
|
||||
|
||||
Raises:
|
||||
NoResultsError: If no results are found
|
||||
DownloadError: If download fails
|
||||
"""
|
||||
# Search
|
||||
results = self.search(artist, title)
|
||||
|
||||
# Select best
|
||||
best_index = self.select_best(results)
|
||||
best_result = results[best_index]
|
||||
logger.info(
|
||||
f"Auto-selected result {best_index}: "
|
||||
f"{best_result.artist} - {best_result.title} ({best_result.provider})"
|
||||
)
|
||||
|
||||
# Download
|
||||
return self.download(best_index, output_dir, output_filename, gcs_path)
|
||||
|
||||
async def search_async(self, artist: str, title: str) -> List[AudioSearchResult]:
|
||||
"""
|
||||
Async version of search for use in async routes.
|
||||
|
||||
Note: Currently wraps sync search in executor. Future optimization
|
||||
could make this fully async.
|
||||
"""
|
||||
loop = asyncio.get_event_loop()
|
||||
return await loop.run_in_executor(None, self.search, artist, title)
|
||||
|
||||
async def download_async(
|
||||
self,
|
||||
result_index: int,
|
||||
output_dir: str,
|
||||
output_filename: Optional[str] = None,
|
||||
gcs_path: Optional[str] = None,
|
||||
) -> AudioDownloadResult:
|
||||
"""
|
||||
Async version of download for use in async routes.
|
||||
|
||||
Note: Currently wraps sync download in executor. Future optimization
|
||||
could make this fully async.
|
||||
"""
|
||||
loop = asyncio.get_event_loop()
|
||||
return await loop.run_in_executor(
|
||||
None,
|
||||
lambda: self.download(result_index, output_dir, output_filename, gcs_path)
|
||||
)
|
||||
|
||||
def is_remote_enabled(self) -> bool:
|
||||
"""Check if remote flacfetch service is configured."""
|
||||
return self._remote_client is not None
|
||||
|
||||
async def check_remote_health(self) -> Optional[dict]:
|
||||
"""
|
||||
Check health of remote flacfetch service.
|
||||
|
||||
Returns:
|
||||
Health status dict if remote service is configured and healthy,
|
||||
None if not configured or unhealthy.
|
||||
"""
|
||||
if not self._remote_client:
|
||||
return None
|
||||
|
||||
try:
|
||||
return await self._remote_client.health_check()
|
||||
except Exception as e:
|
||||
logger.warning(f"Remote flacfetch health check failed: {e}")
|
||||
return None
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_audio_search_service: Optional[AudioSearchService] = None
|
||||
|
||||
|
||||
def get_audio_search_service() -> AudioSearchService:
|
||||
"""Get the singleton AudioSearchService instance."""
|
||||
global _audio_search_service
|
||||
if _audio_search_service is None:
|
||||
_audio_search_service = AudioSearchService()
|
||||
return _audio_search_service
|
||||
@@ -0,0 +1,412 @@
|
||||
"""
|
||||
Client for the remote flacfetch HTTP API service.
|
||||
|
||||
This client communicates with a dedicated flacfetch VM that handles:
|
||||
- BitTorrent downloads from private trackers (Redacted, OPS)
|
||||
- YouTube downloads
|
||||
- GCS uploads of downloaded files
|
||||
|
||||
The flacfetch service provides:
|
||||
- Full peer connectivity for torrents (not possible in Cloud Run)
|
||||
- Indefinite seeding of completed torrents
|
||||
- Automatic disk cleanup
|
||||
|
||||
Usage:
|
||||
client = get_flacfetch_client()
|
||||
if client:
|
||||
# Search for audio
|
||||
search_result = await client.search("ABBA", "Waterloo")
|
||||
|
||||
# Download the best result
|
||||
download_id = await client.download(
|
||||
search_id=search_result["search_id"],
|
||||
result_index=0,
|
||||
gcs_path="uploads/job123/audio/",
|
||||
)
|
||||
|
||||
# Wait for completion
|
||||
result = await client.wait_for_download(download_id)
|
||||
print(f"Downloaded to: {result['gcs_path']}")
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class FlacfetchServiceError(Exception):
|
||||
"""Error communicating with or returned from flacfetch service."""
|
||||
pass
|
||||
|
||||
|
||||
class FlacfetchClient:
|
||||
"""
|
||||
Client for remote flacfetch HTTP API.
|
||||
|
||||
All endpoints require authentication via X-API-Key header.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
base_url: str,
|
||||
api_key: str,
|
||||
timeout: int = 60,
|
||||
):
|
||||
"""
|
||||
Initialize flacfetch client.
|
||||
|
||||
Args:
|
||||
base_url: Base URL of flacfetch service (e.g., http://10.0.0.5:8080)
|
||||
api_key: API key for authentication
|
||||
timeout: Default request timeout in seconds
|
||||
"""
|
||||
self.base_url = base_url.rstrip('/')
|
||||
self.api_key = api_key
|
||||
self.timeout = timeout
|
||||
logger.info(f"FlacfetchClient initialized with base_url={self.base_url}")
|
||||
|
||||
def _headers(self) -> Dict[str, str]:
|
||||
"""Get request headers with authentication."""
|
||||
return {
|
||||
"X-API-Key": self.api_key,
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
async def health_check(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Check if flacfetch service is healthy.
|
||||
|
||||
Returns:
|
||||
Health status dict with transmission, disk, and provider info
|
||||
|
||||
Raises:
|
||||
FlacfetchServiceError: If service is unhealthy or unreachable
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.get(
|
||||
f"{self.base_url}/health",
|
||||
headers=self._headers(),
|
||||
timeout=10,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
if data.get("status") not in ["healthy", "degraded"]:
|
||||
raise FlacfetchServiceError(f"Service unhealthy: {data}")
|
||||
|
||||
return data
|
||||
except httpx.RequestError as e:
|
||||
raise FlacfetchServiceError(f"Cannot reach flacfetch service: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
raise FlacfetchServiceError(f"Health check failed: {e.response.status_code}")
|
||||
|
||||
async def search(
|
||||
self,
|
||||
artist: str,
|
||||
title: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Search for audio matching artist and title.
|
||||
|
||||
Args:
|
||||
artist: Artist name
|
||||
title: Track title
|
||||
|
||||
Returns:
|
||||
Search response dict with search_id and results list
|
||||
|
||||
Raises:
|
||||
FlacfetchServiceError: On search failure
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.post(
|
||||
f"{self.base_url}/search",
|
||||
headers=self._headers(),
|
||||
json={"artist": artist, "title": title},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
|
||||
if resp.status_code == 404:
|
||||
# No results found - return empty results
|
||||
return {
|
||||
"search_id": None,
|
||||
"artist": artist,
|
||||
"title": title,
|
||||
"results": [],
|
||||
"results_count": 0,
|
||||
}
|
||||
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
except httpx.RequestError as e:
|
||||
raise FlacfetchServiceError(f"Search request failed: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
raise FlacfetchServiceError(f"Search failed: {e.response.status_code} - {e.response.text}")
|
||||
|
||||
async def download(
|
||||
self,
|
||||
search_id: str,
|
||||
result_index: int,
|
||||
output_filename: Optional[str] = None,
|
||||
gcs_path: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Start downloading an audio file.
|
||||
|
||||
Args:
|
||||
search_id: Search ID from previous search
|
||||
result_index: Index of result to download
|
||||
output_filename: Optional custom filename (without extension)
|
||||
gcs_path: GCS path for upload (e.g., "uploads/job123/audio/")
|
||||
|
||||
Returns:
|
||||
Download ID for tracking progress
|
||||
|
||||
Raises:
|
||||
FlacfetchServiceError: On download start failure
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
payload = {
|
||||
"search_id": search_id,
|
||||
"result_index": result_index,
|
||||
}
|
||||
if output_filename:
|
||||
payload["output_filename"] = output_filename
|
||||
if gcs_path:
|
||||
payload["upload_to_gcs"] = True
|
||||
payload["gcs_path"] = gcs_path
|
||||
|
||||
resp = await client.post(
|
||||
f"{self.base_url}/download",
|
||||
headers=self._headers(),
|
||||
json=payload,
|
||||
timeout=self.timeout,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
|
||||
data = resp.json()
|
||||
return data["download_id"]
|
||||
|
||||
except httpx.RequestError as e:
|
||||
raise FlacfetchServiceError(f"Download request failed: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
raise FlacfetchServiceError(f"Download start failed: {e.response.status_code} - {e.response.text}")
|
||||
|
||||
async def get_download_status(self, download_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Get the current status of a download.
|
||||
|
||||
Args:
|
||||
download_id: Download ID from start_download
|
||||
|
||||
Returns:
|
||||
Status dict with progress, speed, path info
|
||||
|
||||
Raises:
|
||||
FlacfetchServiceError: On status check failure
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.get(
|
||||
f"{self.base_url}/download/{download_id}/status",
|
||||
headers=self._headers(),
|
||||
timeout=10,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
except httpx.RequestError as e:
|
||||
raise FlacfetchServiceError(f"Status request failed: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
if e.response.status_code == 404:
|
||||
raise FlacfetchServiceError(f"Download not found: {download_id}")
|
||||
raise FlacfetchServiceError(f"Status check failed: {e.response.status_code}")
|
||||
|
||||
async def wait_for_download(
|
||||
self,
|
||||
download_id: str,
|
||||
timeout: int = 600,
|
||||
poll_interval: float = 2.0,
|
||||
progress_callback: Optional[callable] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Wait for a download to complete.
|
||||
|
||||
Args:
|
||||
download_id: Download ID to wait for
|
||||
timeout: Maximum wait time in seconds
|
||||
poll_interval: Time between status checks
|
||||
progress_callback: Optional callback(status_dict) for progress updates
|
||||
|
||||
Returns:
|
||||
Final status dict with output_path or gcs_path
|
||||
|
||||
Raises:
|
||||
FlacfetchServiceError: On download failure or timeout
|
||||
"""
|
||||
elapsed = 0.0
|
||||
|
||||
while elapsed < timeout:
|
||||
status = await self.get_download_status(download_id)
|
||||
|
||||
if progress_callback:
|
||||
try:
|
||||
progress_callback(status)
|
||||
except Exception as e:
|
||||
logger.warning(f"Progress callback error: {e}")
|
||||
|
||||
download_status = status.get("status")
|
||||
|
||||
if download_status in ["complete", "seeding"]:
|
||||
logger.info(f"Download {download_id} complete: {status.get('gcs_path') or status.get('output_path')}")
|
||||
return status
|
||||
elif download_status == "failed":
|
||||
error = status.get("error", "Unknown error")
|
||||
raise FlacfetchServiceError(f"Download failed: {error}")
|
||||
elif download_status == "cancelled":
|
||||
raise FlacfetchServiceError("Download was cancelled")
|
||||
|
||||
# Log progress
|
||||
progress = status.get("progress", 0)
|
||||
speed = status.get("download_speed_kbps", 0)
|
||||
peers = status.get("peers", 0)
|
||||
logger.debug(
|
||||
f"Download {download_id}: {progress:.1f}% "
|
||||
f"({speed:.1f} KB/s, {peers} peers)"
|
||||
)
|
||||
|
||||
await asyncio.sleep(poll_interval)
|
||||
elapsed += poll_interval
|
||||
|
||||
raise FlacfetchServiceError(f"Download timed out after {timeout}s")
|
||||
|
||||
async def list_torrents(self) -> Dict[str, Any]:
|
||||
"""
|
||||
List all torrents in Transmission.
|
||||
|
||||
Returns:
|
||||
Torrent list response with torrent info and totals
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.get(
|
||||
f"{self.base_url}/torrents",
|
||||
headers=self._headers(),
|
||||
timeout=30,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
except httpx.RequestError as e:
|
||||
raise FlacfetchServiceError(f"List torrents failed: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
raise FlacfetchServiceError(f"List torrents failed: {e.response.status_code}")
|
||||
|
||||
async def delete_torrent(
|
||||
self,
|
||||
torrent_id: int,
|
||||
delete_data: bool = True,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Delete a torrent from Transmission.
|
||||
|
||||
Args:
|
||||
torrent_id: Transmission torrent ID
|
||||
delete_data: Also delete downloaded files
|
||||
|
||||
Returns:
|
||||
Delete response
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.delete(
|
||||
f"{self.base_url}/torrents/{torrent_id}",
|
||||
headers=self._headers(),
|
||||
params={"delete_data": str(delete_data).lower()},
|
||||
timeout=30,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
except httpx.RequestError as e:
|
||||
raise FlacfetchServiceError(f"Delete torrent failed: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
raise FlacfetchServiceError(f"Delete torrent failed: {e.response.status_code}")
|
||||
|
||||
async def cleanup_torrents(
|
||||
self,
|
||||
strategy: str = "oldest",
|
||||
target_free_gb: float = 10.0,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Trigger disk cleanup.
|
||||
|
||||
Args:
|
||||
strategy: Cleanup strategy (oldest, largest, lowest_ratio)
|
||||
target_free_gb: Target free space to achieve
|
||||
|
||||
Returns:
|
||||
Cleanup response with removed count and freed space
|
||||
"""
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.post(
|
||||
f"{self.base_url}/torrents/cleanup",
|
||||
headers=self._headers(),
|
||||
json={
|
||||
"strategy": strategy,
|
||||
"target_free_gb": target_free_gb,
|
||||
},
|
||||
timeout=60,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
except httpx.RequestError as e:
|
||||
raise FlacfetchServiceError(f"Cleanup failed: {e}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
raise FlacfetchServiceError(f"Cleanup failed: {e.response.status_code}")
|
||||
|
||||
|
||||
# Singleton client instance
|
||||
_client: Optional[FlacfetchClient] = None
|
||||
|
||||
|
||||
def get_flacfetch_client() -> Optional[FlacfetchClient]:
|
||||
"""
|
||||
Get the flacfetch client if configured.
|
||||
|
||||
Returns:
|
||||
FlacfetchClient instance if FLACFETCH_API_URL and FLACFETCH_API_KEY are set,
|
||||
None otherwise (indicating local-only mode).
|
||||
"""
|
||||
global _client
|
||||
|
||||
if _client is None:
|
||||
from backend.config import get_settings
|
||||
settings = get_settings()
|
||||
|
||||
if settings.flacfetch_api_url and settings.flacfetch_api_key:
|
||||
_client = FlacfetchClient(
|
||||
base_url=settings.flacfetch_api_url,
|
||||
api_key=settings.flacfetch_api_key,
|
||||
)
|
||||
logger.info("FlacfetchClient initialized for remote service")
|
||||
else:
|
||||
logger.debug("FlacfetchClient not configured (missing URL or API key)")
|
||||
|
||||
return _client
|
||||
|
||||
|
||||
def reset_flacfetch_client():
|
||||
"""Reset the singleton client (for testing)."""
|
||||
global _client
|
||||
_client = None
|
||||
|
||||
@@ -138,7 +138,7 @@ def job_log_context(
|
||||
Args:
|
||||
job_id: Job ID to associate logs with
|
||||
worker: Worker name for log categorization
|
||||
logger_names: List of logger names to capture (None = capture root + common loggers)
|
||||
logger_names: List of logger names to capture (None = capture common loggers)
|
||||
min_level: Minimum log level to capture
|
||||
|
||||
Usage:
|
||||
@@ -147,14 +147,13 @@ def job_log_context(
|
||||
correction_operations.add_lyrics_source(...)
|
||||
"""
|
||||
# Default loggers to capture - these cover most of our processing
|
||||
# Note: We intentionally DON'T include the root logger ('') to avoid
|
||||
# duplicate log entries since logs propagate up the logger hierarchy.
|
||||
# Only capture at the top-level package loggers.
|
||||
if logger_names is None:
|
||||
logger_names = [
|
||||
'', # Root logger
|
||||
'backend.api.routes.review',
|
||||
'lyrics_transcriber',
|
||||
'lyrics_transcriber.correction',
|
||||
'lyrics_transcriber.correction.anchor_sequence',
|
||||
'lyrics_transcriber.correction.corrector',
|
||||
'karaoke_gen',
|
||||
]
|
||||
|
||||
|
||||
@@ -55,6 +55,10 @@ class JobManager:
|
||||
youtube_description=job_create.youtube_description,
|
||||
webhook_url=job_create.webhook_url,
|
||||
user_email=job_create.user_email,
|
||||
# Two-phase workflow (Batch 6)
|
||||
prep_only=job_create.prep_only,
|
||||
finalise_only=job_create.finalise_only,
|
||||
keep_brand_code=job_create.keep_brand_code,
|
||||
# Request metadata (for tracking and filtering)
|
||||
request_metadata=job_create.request_metadata,
|
||||
)
|
||||
|
||||
@@ -65,6 +65,26 @@ class StorageService:
|
||||
a private key available. Requires the service account to have
|
||||
roles/iam.serviceAccountTokenCreator on itself.
|
||||
"""
|
||||
return self._generate_signed_url_internal(blob_path, "GET", expiration_minutes)
|
||||
|
||||
def generate_signed_upload_url(self, blob_path: str, content_type: str = "application/octet-stream", expiration_minutes: int = 60) -> str:
|
||||
"""Generate a signed URL for uploading a file directly to GCS.
|
||||
|
||||
This allows clients to upload files directly to GCS without going through
|
||||
the backend, bypassing any request body size limits.
|
||||
|
||||
Args:
|
||||
blob_path: The destination path in GCS
|
||||
content_type: The expected content type of the upload
|
||||
expiration_minutes: How long the URL is valid for
|
||||
|
||||
Returns:
|
||||
A signed URL that accepts PUT requests with the file content
|
||||
"""
|
||||
return self._generate_signed_url_internal(blob_path, "PUT", expiration_minutes, content_type)
|
||||
|
||||
def _generate_signed_url_internal(self, blob_path: str, method: str, expiration_minutes: int = 60, content_type: Optional[str] = None) -> str:
|
||||
"""Internal method to generate signed URLs for GET or PUT operations."""
|
||||
import google.auth
|
||||
from google.auth.transport import requests
|
||||
|
||||
@@ -74,6 +94,17 @@ class StorageService:
|
||||
# Get default credentials and refresh to ensure we have a valid token
|
||||
credentials, project = google.auth.default()
|
||||
|
||||
# Common kwargs for signed URL generation
|
||||
kwargs = {
|
||||
"version": "v4",
|
||||
"expiration": timedelta(minutes=expiration_minutes),
|
||||
"method": method,
|
||||
}
|
||||
|
||||
# For PUT requests, we need to specify the content type in headers
|
||||
if method == "PUT" and content_type:
|
||||
kwargs["headers"] = {"Content-Type": content_type}
|
||||
|
||||
# Check if we're using compute credentials (Cloud Run/GCE)
|
||||
# These need to use IAM signBlob via service_account_email + access_token
|
||||
if hasattr(credentials, 'service_account_email'):
|
||||
@@ -81,25 +112,15 @@ class StorageService:
|
||||
auth_request = requests.Request()
|
||||
credentials.refresh(auth_request)
|
||||
|
||||
url = blob.generate_signed_url(
|
||||
version="v4",
|
||||
expiration=timedelta(minutes=expiration_minutes),
|
||||
method="GET",
|
||||
service_account_email=credentials.service_account_email,
|
||||
access_token=credentials.token
|
||||
)
|
||||
else:
|
||||
# Local development with service account key or user credentials
|
||||
url = blob.generate_signed_url(
|
||||
version="v4",
|
||||
expiration=timedelta(minutes=expiration_minutes),
|
||||
method="GET"
|
||||
)
|
||||
kwargs["service_account_email"] = credentials.service_account_email
|
||||
kwargs["access_token"] = credentials.token
|
||||
|
||||
logger.info(f"Generated signed URL for {blob_path}")
|
||||
url = blob.generate_signed_url(**kwargs)
|
||||
|
||||
logger.info(f"Generated signed {method} URL for {blob_path}")
|
||||
return url
|
||||
except Exception as e:
|
||||
logger.error(f"Error generating signed URL for {blob_path}: {e}")
|
||||
logger.error(f"Error generating signed {method} URL for {blob_path}: {e}")
|
||||
raise
|
||||
|
||||
def delete_file(self, blob_path: str) -> None:
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
Worker coordination service.
|
||||
|
||||
Handles triggering and coordinating background workers.
|
||||
Implements proper HTTP-based triggers for production use.
|
||||
Supports two modes:
|
||||
- Cloud Tasks (production): Guaranteed delivery, automatic retries, horizontal scaling
|
||||
- Direct HTTP (development): Faster iteration, simpler debugging
|
||||
|
||||
SOLID Principles:
|
||||
- Single Responsibility: Only handles worker coordination
|
||||
@@ -11,20 +13,44 @@ SOLID Principles:
|
||||
"""
|
||||
import logging
|
||||
import os
|
||||
import httpx
|
||||
import json
|
||||
from typing import Optional
|
||||
import httpx
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Mapping of worker types to their Cloud Tasks queue names
|
||||
# These queues are created by Pulumi in infrastructure/__main__.py
|
||||
WORKER_QUEUES = {
|
||||
"audio": "audio-worker-queue",
|
||||
"lyrics": "lyrics-worker-queue",
|
||||
"screens": "screens-worker-queue",
|
||||
"render-video": "render-worker-queue",
|
||||
"video": "video-worker-queue",
|
||||
}
|
||||
|
||||
|
||||
class WorkerService:
|
||||
"""
|
||||
Service for coordinating background workers.
|
||||
|
||||
Handles triggering workers via internal HTTP API.
|
||||
In production, workers run as separate Cloud Run requests or Cloud Tasks.
|
||||
Supports two execution modes controlled by ENABLE_CLOUD_TASKS env var:
|
||||
|
||||
1. Cloud Tasks mode (production, ENABLE_CLOUD_TASKS=true):
|
||||
- Tasks enqueued to Cloud Tasks for guaranteed delivery
|
||||
- Automatic retries on failure
|
||||
- Each task runs in dedicated Cloud Run instance
|
||||
- Rate limiting to protect external APIs (Modal, AudioShake)
|
||||
|
||||
2. Direct HTTP mode (development, ENABLE_CLOUD_TASKS=false):
|
||||
- Direct HTTP POST to internal worker endpoints
|
||||
- Worker runs as FastAPI BackgroundTask
|
||||
- Faster iteration for local development
|
||||
- No retry guarantees
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
@@ -32,6 +58,44 @@ class WorkerService:
|
||||
self.settings = get_settings()
|
||||
self._base_url = self._get_base_url()
|
||||
self._admin_token = self._get_admin_token()
|
||||
self._use_cloud_tasks = self._should_use_cloud_tasks()
|
||||
self._tasks_client = None
|
||||
|
||||
if self._use_cloud_tasks:
|
||||
logger.info("WorkerService initialized with Cloud Tasks mode")
|
||||
else:
|
||||
logger.info("WorkerService initialized with direct HTTP mode")
|
||||
|
||||
def _should_use_cloud_tasks(self) -> bool:
|
||||
"""
|
||||
Check if Cloud Tasks should be used for worker coordination.
|
||||
|
||||
Controlled by ENABLE_CLOUD_TASKS setting (from environment variable).
|
||||
Default is false (direct HTTP mode) for backward compatibility.
|
||||
|
||||
Returns:
|
||||
True if Cloud Tasks should be used, False for direct HTTP
|
||||
"""
|
||||
return self.settings.enable_cloud_tasks
|
||||
|
||||
@property
|
||||
def tasks_client(self):
|
||||
"""
|
||||
Lazy-initialize Cloud Tasks client.
|
||||
|
||||
Only created when needed to avoid import overhead in development mode.
|
||||
"""
|
||||
if self._tasks_client is None and self._use_cloud_tasks:
|
||||
try:
|
||||
from google.cloud import tasks_v2
|
||||
self._tasks_client = tasks_v2.CloudTasksClient()
|
||||
except ImportError:
|
||||
logger.error(
|
||||
"google-cloud-tasks not installed. "
|
||||
"Install with: pip install google-cloud-tasks"
|
||||
)
|
||||
raise
|
||||
return self._tasks_client
|
||||
|
||||
def _get_admin_token(self) -> Optional[str]:
|
||||
"""
|
||||
@@ -49,9 +113,10 @@ class WorkerService:
|
||||
"""
|
||||
Get base URL for internal API calls.
|
||||
|
||||
In production: Cloud Run service URL
|
||||
In test: TEST_SERVER_URL
|
||||
In development: localhost
|
||||
Priority:
|
||||
1. TEST_SERVER_URL env var (for tests)
|
||||
2. CLOUD_RUN_SERVICE_URL env var (for Cloud Tasks to call back)
|
||||
3. localhost with PORT env var (for development)
|
||||
|
||||
Returns:
|
||||
Base URL for API calls
|
||||
@@ -61,14 +126,11 @@ class WorkerService:
|
||||
if test_url:
|
||||
return test_url
|
||||
|
||||
# Check if running in Cloud Run
|
||||
cloud_run_url = self.settings.google_cloud_project
|
||||
if cloud_run_url and self.settings.environment == "production":
|
||||
# TODO: Get actual Cloud Run service URL
|
||||
# For now, use environment variable
|
||||
service_url = os.getenv("CLOUD_RUN_SERVICE_URL")
|
||||
if service_url:
|
||||
return service_url
|
||||
# Production: Cloud Run service URL
|
||||
# This must be the publicly accessible URL for Cloud Tasks to call
|
||||
service_url = os.getenv("CLOUD_RUN_SERVICE_URL")
|
||||
if service_url:
|
||||
return service_url
|
||||
|
||||
# Development mode: use localhost with PORT env var
|
||||
port = os.getenv("PORT", "8000")
|
||||
@@ -81,10 +143,120 @@ class WorkerService:
|
||||
timeout_seconds: int = 30
|
||||
) -> bool:
|
||||
"""
|
||||
Trigger a background worker via internal API.
|
||||
Trigger a background worker for a job.
|
||||
|
||||
In production (ENABLE_CLOUD_TASKS=true):
|
||||
- Enqueues task to Cloud Tasks queue
|
||||
- Cloud Tasks delivers HTTP request to internal endpoint
|
||||
- Automatic retries on failure with exponential backoff
|
||||
- Rate limiting protects external APIs
|
||||
|
||||
In development (ENABLE_CLOUD_TASKS=false):
|
||||
- Direct HTTP POST to internal endpoint
|
||||
- Faster iteration, but no retry guarantees
|
||||
- Worker runs in same container as API
|
||||
|
||||
Args:
|
||||
worker_type: Worker type ("audio", "lyrics", "screens", "video")
|
||||
worker_type: Worker type ("audio", "lyrics", "screens", "render-video", "video")
|
||||
job_id: Job ID to process
|
||||
timeout_seconds: Request timeout (for direct HTTP mode)
|
||||
|
||||
Returns:
|
||||
True if trigger successful (or task enqueued), False otherwise
|
||||
"""
|
||||
if self._use_cloud_tasks:
|
||||
return await self._enqueue_cloud_task(worker_type, job_id)
|
||||
else:
|
||||
return await self._trigger_http(worker_type, job_id, timeout_seconds)
|
||||
|
||||
async def _enqueue_cloud_task(self, worker_type: str, job_id: str) -> bool:
|
||||
"""
|
||||
Enqueue task to Cloud Tasks for guaranteed delivery.
|
||||
|
||||
The task will be delivered as an HTTP POST to the internal worker endpoint.
|
||||
Cloud Tasks handles:
|
||||
- Retry on failure (with exponential backoff)
|
||||
- Rate limiting (max dispatches per second)
|
||||
- Deduplication (via task name if needed)
|
||||
- OIDC authentication for Cloud Run
|
||||
|
||||
Args:
|
||||
worker_type: Worker type
|
||||
job_id: Job ID to process
|
||||
|
||||
Returns:
|
||||
True if task enqueued successfully, False otherwise
|
||||
"""
|
||||
try:
|
||||
from google.cloud import tasks_v2
|
||||
|
||||
queue_name = WORKER_QUEUES.get(worker_type)
|
||||
if not queue_name:
|
||||
logger.error(f"Unknown worker type: {worker_type}")
|
||||
return False
|
||||
|
||||
project = self.settings.google_cloud_project
|
||||
if not project:
|
||||
logger.error("GOOGLE_CLOUD_PROJECT not set, cannot enqueue Cloud Task")
|
||||
return False
|
||||
|
||||
location = self.settings.gcp_region # Must match queue location
|
||||
|
||||
# Build queue path
|
||||
parent = self.tasks_client.queue_path(project, location, queue_name)
|
||||
|
||||
# Build task payload
|
||||
task = {
|
||||
"http_request": {
|
||||
"http_method": tasks_v2.HttpMethod.POST,
|
||||
"url": f"{self._base_url}/api/internal/workers/{worker_type}",
|
||||
"headers": {
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
"body": json.dumps({"job_id": job_id}).encode(),
|
||||
# Use OIDC token for Cloud Run authentication
|
||||
# This allows Cloud Tasks to invoke the Cloud Run service
|
||||
"oidc_token": {
|
||||
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
# Add admin auth header if available
|
||||
# This is needed for the internal endpoint's require_admin dependency
|
||||
if self._admin_token:
|
||||
task["http_request"]["headers"]["Authorization"] = f"Bearer {self._admin_token}"
|
||||
|
||||
# Create task
|
||||
# Note: We don't set a task name, allowing Cloud Tasks to generate one
|
||||
# This prevents duplicate task errors on retries
|
||||
response = self.tasks_client.create_task(parent=parent, task=task)
|
||||
logger.info(
|
||||
f"Created Cloud Task for {worker_type} worker, job {job_id}: {response.name}"
|
||||
)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to enqueue Cloud Task for {worker_type}/{job_id}: {e}",
|
||||
exc_info=True
|
||||
)
|
||||
return False
|
||||
|
||||
async def _trigger_http(
|
||||
self,
|
||||
worker_type: str,
|
||||
job_id: str,
|
||||
timeout_seconds: int = 30
|
||||
) -> bool:
|
||||
"""
|
||||
Trigger worker via direct HTTP call (development mode).
|
||||
|
||||
This is the original implementation - direct HTTP POST to internal endpoint.
|
||||
The endpoint adds the worker function to FastAPI BackgroundTasks.
|
||||
|
||||
Args:
|
||||
worker_type: Worker type
|
||||
job_id: Job ID to process
|
||||
timeout_seconds: Request timeout
|
||||
|
||||
@@ -127,6 +299,9 @@ class WorkerService:
|
||||
)
|
||||
return False
|
||||
|
||||
# Convenience methods for specific workers
|
||||
# These provide a cleaner API and better IDE autocomplete
|
||||
|
||||
async def trigger_audio_worker(self, job_id: str) -> bool:
|
||||
"""Trigger audio separation worker."""
|
||||
return await self.trigger_worker("audio", job_id)
|
||||
@@ -140,9 +315,68 @@ class WorkerService:
|
||||
return await self.trigger_worker("screens", job_id)
|
||||
|
||||
async def trigger_video_worker(self, job_id: str) -> bool:
|
||||
"""Trigger video generation worker."""
|
||||
"""
|
||||
Trigger video generation worker.
|
||||
|
||||
When USE_CLOUD_RUN_JOBS_FOR_VIDEO=true and ENABLE_CLOUD_TASKS=true,
|
||||
uses Cloud Run Jobs for execution (supports >30 min encoding).
|
||||
Otherwise, uses Cloud Tasks or direct HTTP.
|
||||
"""
|
||||
if self._use_cloud_tasks and self.settings.use_cloud_run_jobs_for_video:
|
||||
return await self._trigger_cloud_run_job(job_id)
|
||||
return await self.trigger_worker("video", job_id)
|
||||
|
||||
async def _trigger_cloud_run_job(self, job_id: str) -> bool:
|
||||
"""
|
||||
Trigger video encoding as a Cloud Run Job.
|
||||
|
||||
Cloud Run Jobs support up to 24 hours of execution time,
|
||||
making them suitable for very long video encoding tasks.
|
||||
|
||||
Args:
|
||||
job_id: Job ID to process
|
||||
|
||||
Returns:
|
||||
True if job was triggered successfully, False otherwise
|
||||
"""
|
||||
try:
|
||||
from google.cloud import run_v2
|
||||
|
||||
project = self.settings.google_cloud_project
|
||||
if not project:
|
||||
logger.error("GOOGLE_CLOUD_PROJECT not set, cannot trigger Cloud Run Job")
|
||||
return False
|
||||
|
||||
location = self.settings.gcp_region
|
||||
job_name = f"projects/{project}/locations/{location}/jobs/video-encoding-job"
|
||||
|
||||
# Create Cloud Run Jobs client
|
||||
client = run_v2.JobsClient()
|
||||
|
||||
# Run the job with overrides for the specific job_id
|
||||
request = run_v2.RunJobRequest(
|
||||
name=job_name,
|
||||
overrides=run_v2.RunJobRequest.Overrides(
|
||||
container_overrides=[
|
||||
run_v2.RunJobRequest.Overrides.ContainerOverride(
|
||||
args=["python", "-m", "backend.workers.video_worker", "--job-id", job_id],
|
||||
)
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
# Run the job (async operation)
|
||||
operation = client.run_job(request=request)
|
||||
logger.info(f"Started Cloud Run Job for video encoding, job {job_id}: {operation.metadata}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to trigger Cloud Run Job for video/{job_id}: {e}",
|
||||
exc_info=True
|
||||
)
|
||||
return False
|
||||
|
||||
async def trigger_render_video_worker(self, job_id: str) -> bool:
|
||||
"""Trigger render video worker (post-review)."""
|
||||
return await self.trigger_worker("render-video", job_id)
|
||||
@@ -166,3 +400,12 @@ def get_worker_service() -> WorkerService:
|
||||
_worker_service = WorkerService()
|
||||
return _worker_service
|
||||
|
||||
|
||||
def reset_worker_service() -> None:
|
||||
"""
|
||||
Reset the global worker service instance.
|
||||
|
||||
Used in tests to ensure clean state between test cases.
|
||||
"""
|
||||
global _worker_service
|
||||
_worker_service = None
|
||||
@@ -0,0 +1,499 @@
|
||||
"""
|
||||
Tests for audio search functionality (Batch 5).
|
||||
|
||||
Tests the audio search service, API routes, and job model fields.
|
||||
"""
|
||||
import pytest
|
||||
import os
|
||||
from unittest.mock import Mock, patch, MagicMock
|
||||
from datetime import datetime
|
||||
|
||||
from backend.models.job import Job, JobCreate, JobStatus, STATE_TRANSITIONS
|
||||
from backend.services.audio_search_service import (
|
||||
AudioSearchService,
|
||||
AudioSearchResult,
|
||||
AudioDownloadResult,
|
||||
AudioSearchError,
|
||||
NoResultsError,
|
||||
DownloadError,
|
||||
get_audio_search_service,
|
||||
)
|
||||
|
||||
|
||||
class TestJobModelAudioSearchFields:
|
||||
"""Test Job model has audio search fields."""
|
||||
|
||||
def test_job_has_audio_search_fields(self):
|
||||
"""Test Job model has audio search configuration fields."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.utcnow(),
|
||||
updated_at=datetime.utcnow(),
|
||||
artist="ABBA",
|
||||
title="Waterloo",
|
||||
audio_search_artist="ABBA",
|
||||
audio_search_title="Waterloo",
|
||||
auto_download=True,
|
||||
)
|
||||
|
||||
assert job.audio_search_artist == "ABBA"
|
||||
assert job.audio_search_title == "Waterloo"
|
||||
assert job.auto_download is True
|
||||
|
||||
def test_job_audio_search_fields_default_values(self):
|
||||
"""Test default values for audio search fields."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.utcnow(),
|
||||
updated_at=datetime.utcnow(),
|
||||
)
|
||||
|
||||
assert job.audio_search_artist is None
|
||||
assert job.audio_search_title is None
|
||||
assert job.auto_download is False
|
||||
|
||||
def test_job_create_has_audio_search_fields(self):
|
||||
"""Test JobCreate model has audio search fields."""
|
||||
job_create = JobCreate(
|
||||
artist="ABBA",
|
||||
title="Waterloo",
|
||||
audio_search_artist="ABBA",
|
||||
audio_search_title="Waterloo",
|
||||
auto_download=True,
|
||||
)
|
||||
|
||||
assert job_create.audio_search_artist == "ABBA"
|
||||
assert job_create.audio_search_title == "Waterloo"
|
||||
assert job_create.auto_download is True
|
||||
|
||||
|
||||
class TestJobStatusAudioSearchStates:
|
||||
"""Test Job status includes audio search states."""
|
||||
|
||||
def test_audio_search_statuses_exist(self):
|
||||
"""Test audio search status values exist."""
|
||||
assert hasattr(JobStatus, 'SEARCHING_AUDIO')
|
||||
assert hasattr(JobStatus, 'AWAITING_AUDIO_SELECTION')
|
||||
assert hasattr(JobStatus, 'DOWNLOADING_AUDIO')
|
||||
|
||||
assert JobStatus.SEARCHING_AUDIO == "searching_audio"
|
||||
assert JobStatus.AWAITING_AUDIO_SELECTION == "awaiting_audio_selection"
|
||||
assert JobStatus.DOWNLOADING_AUDIO == "downloading_audio"
|
||||
|
||||
def test_state_transitions_include_audio_search_flow(self):
|
||||
"""Test state transitions include audio search flow."""
|
||||
# PENDING can go to SEARCHING_AUDIO
|
||||
assert JobStatus.SEARCHING_AUDIO in STATE_TRANSITIONS[JobStatus.PENDING]
|
||||
|
||||
# SEARCHING_AUDIO can go to AWAITING_AUDIO_SELECTION or DOWNLOADING_AUDIO
|
||||
assert JobStatus.AWAITING_AUDIO_SELECTION in STATE_TRANSITIONS[JobStatus.SEARCHING_AUDIO]
|
||||
assert JobStatus.DOWNLOADING_AUDIO in STATE_TRANSITIONS[JobStatus.SEARCHING_AUDIO]
|
||||
|
||||
# AWAITING_AUDIO_SELECTION can go to DOWNLOADING_AUDIO
|
||||
assert JobStatus.DOWNLOADING_AUDIO in STATE_TRANSITIONS[JobStatus.AWAITING_AUDIO_SELECTION]
|
||||
|
||||
# DOWNLOADING_AUDIO can go to DOWNLOADING
|
||||
assert JobStatus.DOWNLOADING in STATE_TRANSITIONS[JobStatus.DOWNLOADING_AUDIO]
|
||||
|
||||
|
||||
class TestAudioSearchResult:
|
||||
"""Test AudioSearchResult dataclass.
|
||||
|
||||
AudioSearchResult is now imported from karaoke_gen.audio_fetcher.
|
||||
These tests verify the backend can use it correctly.
|
||||
"""
|
||||
|
||||
def test_create_audio_search_result(self):
|
||||
"""Test creating an AudioSearchResult."""
|
||||
result = AudioSearchResult(
|
||||
title="Waterloo",
|
||||
artist="ABBA",
|
||||
provider="YouTube",
|
||||
url="https://youtube.com/watch?v=abc123",
|
||||
duration=180,
|
||||
quality="FLAC",
|
||||
source_id="abc123",
|
||||
index=0,
|
||||
)
|
||||
|
||||
assert result.title == "Waterloo"
|
||||
assert result.artist == "ABBA"
|
||||
assert result.provider == "YouTube"
|
||||
assert result.url == "https://youtube.com/watch?v=abc123"
|
||||
assert result.duration == 180
|
||||
assert result.quality == "FLAC"
|
||||
assert result.source_id == "abc123"
|
||||
assert result.index == 0
|
||||
|
||||
def test_to_dict(self):
|
||||
"""Test converting to dict for serialization."""
|
||||
result = AudioSearchResult(
|
||||
title="Waterloo",
|
||||
artist="ABBA",
|
||||
provider="YouTube",
|
||||
url="https://youtube.com/watch?v=abc123",
|
||||
index=0,
|
||||
)
|
||||
|
||||
data = result.to_dict()
|
||||
|
||||
assert data['title'] == "Waterloo"
|
||||
assert data['artist'] == "ABBA"
|
||||
assert data['provider'] == "YouTube"
|
||||
assert data['url'] == "https://youtube.com/watch?v=abc123"
|
||||
assert data['index'] == 0
|
||||
# raw_result should NOT be in serialized dict
|
||||
assert 'raw_result' not in data
|
||||
|
||||
def test_from_dict(self):
|
||||
"""Test creating from dict."""
|
||||
data = {
|
||||
'title': "Waterloo",
|
||||
'artist': "ABBA",
|
||||
'provider': "YouTube",
|
||||
'url': "https://youtube.com/watch?v=abc123",
|
||||
'duration': 180,
|
||||
'quality': "FLAC",
|
||||
'source_id': "abc123",
|
||||
'index': 0,
|
||||
}
|
||||
|
||||
result = AudioSearchResult.from_dict(data)
|
||||
|
||||
assert result.title == "Waterloo"
|
||||
assert result.artist == "ABBA"
|
||||
assert result.provider == "YouTube"
|
||||
assert result.index == 0
|
||||
assert result.raw_result is None # Not set from dict
|
||||
|
||||
|
||||
class TestAudioSearchServiceInit:
|
||||
"""Test AudioSearchService initialization."""
|
||||
|
||||
@patch.dict('os.environ', {'REDACTED_API_KEY': '', 'OPS_API_KEY': ''}, clear=False)
|
||||
def test_init_with_no_keys(self):
|
||||
"""Test initialization without API keys."""
|
||||
# Pass explicit None to override environment
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
|
||||
# Service should have a FlacFetcher internally
|
||||
assert service._fetcher is not None
|
||||
assert service._cached_results == []
|
||||
|
||||
def test_init_with_keys(self):
|
||||
"""Test initialization with API keys."""
|
||||
service = AudioSearchService(
|
||||
redacted_api_key="test_redacted_key",
|
||||
ops_api_key="test_ops_key",
|
||||
)
|
||||
|
||||
# Keys are passed to FlacFetcher
|
||||
assert service._fetcher._redacted_api_key == "test_redacted_key"
|
||||
assert service._fetcher._ops_api_key == "test_ops_key"
|
||||
|
||||
def test_init_reads_from_environment(self):
|
||||
"""Test initialization reads keys from environment variables."""
|
||||
# Just test that the service can be initialized
|
||||
service = AudioSearchService()
|
||||
|
||||
# The service should have a FlacFetcher
|
||||
assert service._fetcher is not None
|
||||
|
||||
|
||||
class TestAudioSearchServiceSearch:
|
||||
"""Test AudioSearchService.search() method."""
|
||||
|
||||
def test_search_returns_results(self):
|
||||
"""Test search returns AudioSearchResult list."""
|
||||
# Create mock result
|
||||
mock_result = AudioSearchResult(
|
||||
title="Waterloo",
|
||||
artist="ABBA",
|
||||
provider="YouTube",
|
||||
url="https://youtube.com/watch?v=abc123",
|
||||
duration=180,
|
||||
quality="FLAC",
|
||||
source_id="abc123",
|
||||
index=0,
|
||||
)
|
||||
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.search.return_value = [mock_result]
|
||||
|
||||
results = service.search("ABBA", "Waterloo")
|
||||
|
||||
assert len(results) == 1
|
||||
assert results[0].title == "Waterloo"
|
||||
assert results[0].artist == "ABBA"
|
||||
assert results[0].provider == "YouTube"
|
||||
assert results[0].index == 0
|
||||
|
||||
def test_search_no_results_raises_error(self):
|
||||
"""Test search raises NoResultsError when no results."""
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.search.side_effect = NoResultsError("No results found")
|
||||
|
||||
with pytest.raises(NoResultsError) as exc_info:
|
||||
service.search("Unknown Artist", "Unknown Song")
|
||||
|
||||
assert "No results found" in str(exc_info.value)
|
||||
|
||||
def test_search_multiple_results(self):
|
||||
"""Test search returns multiple results with correct indices."""
|
||||
mock_results = []
|
||||
for i in range(3):
|
||||
mock_results.append(AudioSearchResult(
|
||||
title=f"Song {i}",
|
||||
artist="Artist",
|
||||
provider=["YouTube", "Redacted", "OPS"][i],
|
||||
url=f"https://example.com/{i}",
|
||||
duration=180 + i * 10,
|
||||
quality=["320kbps", "FLAC", "FLAC"][i],
|
||||
source_id=str(i),
|
||||
index=i,
|
||||
))
|
||||
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.search.return_value = mock_results
|
||||
|
||||
results = service.search("Artist", "Song")
|
||||
|
||||
assert len(results) == 3
|
||||
assert results[0].index == 0
|
||||
assert results[1].index == 1
|
||||
assert results[2].index == 2
|
||||
|
||||
|
||||
class TestAudioSearchServiceSelectBest:
|
||||
"""Test AudioSearchService.select_best() method."""
|
||||
|
||||
def test_select_best_delegates_to_fetcher(self):
|
||||
"""Test select_best uses FlacFetcher's select_best."""
|
||||
mock_results = [
|
||||
AudioSearchResult(
|
||||
title="Waterloo",
|
||||
artist="ABBA",
|
||||
provider="YouTube",
|
||||
url="https://youtube.com/watch?v=abc123",
|
||||
index=0,
|
||||
),
|
||||
AudioSearchResult(
|
||||
title="Waterloo",
|
||||
artist="ABBA",
|
||||
provider="Redacted",
|
||||
url="https://example.com/456",
|
||||
index=1,
|
||||
),
|
||||
]
|
||||
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.select_best.return_value = 1
|
||||
|
||||
best_index = service.select_best(mock_results)
|
||||
|
||||
assert best_index == 1
|
||||
service._fetcher.select_best.assert_called_once_with(mock_results)
|
||||
|
||||
def test_select_best_empty_list_returns_zero(self):
|
||||
"""Test select_best returns 0 for empty list."""
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.select_best.return_value = 0
|
||||
|
||||
best_index = service.select_best([])
|
||||
|
||||
assert best_index == 0
|
||||
|
||||
|
||||
class TestAudioSearchServiceDownload:
|
||||
"""Test AudioSearchService.download() method."""
|
||||
|
||||
def test_download_without_search_raises_error(self):
|
||||
"""Test download without prior search raises error."""
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
|
||||
with pytest.raises(DownloadError) as exc_info:
|
||||
service.download(0, "/tmp")
|
||||
|
||||
assert "No cached result" in str(exc_info.value)
|
||||
|
||||
def test_download_after_search(self):
|
||||
"""Test download after search works."""
|
||||
import tempfile
|
||||
|
||||
# Create a temp file to simulate downloaded file
|
||||
temp_dir = tempfile.mkdtemp()
|
||||
temp_file = os.path.join(temp_dir, "test.flac")
|
||||
with open(temp_file, 'w') as f:
|
||||
f.write("test")
|
||||
|
||||
mock_result = AudioSearchResult(
|
||||
title="Waterloo",
|
||||
artist="ABBA",
|
||||
provider="YouTube",
|
||||
url="https://youtube.com/watch?v=abc123",
|
||||
duration=180,
|
||||
quality="FLAC",
|
||||
source_id="abc123",
|
||||
index=0,
|
||||
)
|
||||
|
||||
mock_fetch_result = AudioDownloadResult(
|
||||
filepath=temp_file,
|
||||
artist="ABBA",
|
||||
title="Waterloo",
|
||||
provider="YouTube",
|
||||
duration=180,
|
||||
quality="FLAC",
|
||||
)
|
||||
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.search.return_value = [mock_result]
|
||||
service._fetcher.download.return_value = mock_fetch_result
|
||||
|
||||
service.search("ABBA", "Waterloo")
|
||||
result = service.download(0, temp_dir)
|
||||
|
||||
assert result.filepath == temp_file
|
||||
assert result.artist == "ABBA"
|
||||
assert result.title == "Waterloo"
|
||||
assert result.provider == "YouTube"
|
||||
|
||||
# Cleanup
|
||||
os.remove(temp_file)
|
||||
os.rmdir(temp_dir)
|
||||
|
||||
def test_download_invalid_index_raises_error(self):
|
||||
"""Test download with invalid index raises error."""
|
||||
mock_result = AudioSearchResult(
|
||||
title="Waterloo",
|
||||
artist="ABBA",
|
||||
provider="YouTube",
|
||||
url="https://youtube.com/watch?v=abc123",
|
||||
index=0,
|
||||
)
|
||||
|
||||
service = AudioSearchService(redacted_api_key=None, ops_api_key=None)
|
||||
service._fetcher = Mock()
|
||||
service._fetcher.search.return_value = [mock_result]
|
||||
|
||||
service.search("ABBA", "Waterloo")
|
||||
|
||||
with pytest.raises(DownloadError) as exc_info:
|
||||
service.download(99, "/tmp")
|
||||
|
||||
assert "No cached result for index 99" in str(exc_info.value)
|
||||
|
||||
|
||||
class TestGetAudioSearchService:
|
||||
"""Test get_audio_search_service singleton."""
|
||||
|
||||
def test_get_audio_search_service_returns_instance(self):
|
||||
"""Test get_audio_search_service returns an instance."""
|
||||
# Reset singleton
|
||||
import backend.services.audio_search_service as module
|
||||
module._audio_search_service = None
|
||||
|
||||
service = get_audio_search_service()
|
||||
|
||||
assert service is not None
|
||||
assert isinstance(service, AudioSearchService)
|
||||
|
||||
def test_get_audio_search_service_singleton(self):
|
||||
"""Test get_audio_search_service returns same instance."""
|
||||
service1 = get_audio_search_service()
|
||||
service2 = get_audio_search_service()
|
||||
|
||||
assert service1 is service2
|
||||
|
||||
|
||||
class TestFlacfetchIntegration:
|
||||
"""
|
||||
Integration tests that verify flacfetch imports work correctly.
|
||||
|
||||
These tests ensure the backend code is compatible with the installed
|
||||
flacfetch version and would catch issues like renamed classes.
|
||||
|
||||
Note: Some tests are marked with pytest.importorskip for flacfetch
|
||||
since it may not be installed in all test environments (e.g., CI).
|
||||
"""
|
||||
|
||||
def test_flacfetch_imports_work(self):
|
||||
"""Test that all flacfetch imports in audio_search_service work.
|
||||
|
||||
This test would have caught the YouTubeProvider -> YoutubeProvider rename.
|
||||
"""
|
||||
flacfetch = pytest.importorskip("flacfetch")
|
||||
|
||||
# These imports match what karaoke_gen.audio_fetcher does
|
||||
from flacfetch.core.manager import FetchManager
|
||||
from flacfetch.providers.youtube import YoutubeProvider
|
||||
from flacfetch.core.models import TrackQuery
|
||||
|
||||
# Verify the classes exist and are callable
|
||||
assert FetchManager is not None
|
||||
assert YoutubeProvider is not None
|
||||
assert TrackQuery is not None
|
||||
|
||||
def test_karaoke_gen_audio_fetcher_imports_work(self):
|
||||
"""Test that karaoke_gen.audio_fetcher imports work.
|
||||
|
||||
Since backend now imports from karaoke_gen, this is the critical test.
|
||||
This should work even without flacfetch installed (lazy import).
|
||||
"""
|
||||
from karaoke_gen.audio_fetcher import (
|
||||
FlacFetcher,
|
||||
AudioSearchResult,
|
||||
AudioFetchResult,
|
||||
AudioFetcherError,
|
||||
NoResultsError,
|
||||
DownloadError,
|
||||
)
|
||||
|
||||
assert FlacFetcher is not None
|
||||
assert AudioSearchResult is not None
|
||||
assert AudioFetchResult is not None
|
||||
assert AudioFetcherError is not None
|
||||
assert NoResultsError is not None
|
||||
assert DownloadError is not None
|
||||
|
||||
def test_audio_search_service_can_initialize_fetcher(self):
|
||||
"""Test AudioSearchService can initialize its FlacFetcher.
|
||||
|
||||
This verifies the actual initialization code path works.
|
||||
"""
|
||||
pytest.importorskip("flacfetch")
|
||||
|
||||
service = AudioSearchService(
|
||||
redacted_api_key=None,
|
||||
ops_api_key=None,
|
||||
)
|
||||
|
||||
# Verify FlacFetcher was initialized
|
||||
assert service._fetcher is not None
|
||||
|
||||
# FlacFetcher can initialize its manager (tests actual flacfetch)
|
||||
manager = service._fetcher._get_manager()
|
||||
assert manager is not None
|
||||
|
||||
def test_shared_classes_are_same_as_karaoke_gen(self):
|
||||
"""Verify backend uses the same classes as karaoke_gen."""
|
||||
from karaoke_gen.audio_fetcher import (
|
||||
AudioSearchResult as KGAudioSearchResult,
|
||||
AudioFetchResult as KGAudioFetchResult,
|
||||
NoResultsError as KGNoResultsError,
|
||||
DownloadError as KGDownloadError,
|
||||
)
|
||||
|
||||
# These should be the exact same classes, not copies
|
||||
assert AudioSearchResult is KGAudioSearchResult
|
||||
assert AudioDownloadResult is KGAudioFetchResult
|
||||
assert NoResultsError is KGNoResultsError
|
||||
assert DownloadError is KGDownloadError
|
||||
@@ -0,0 +1,363 @@
|
||||
"""
|
||||
Unit tests for audio analysis and editing services.
|
||||
|
||||
These tests verify the backend service wrappers that use the shared
|
||||
karaoke_gen.instrumental_review module with GCS integration.
|
||||
"""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import pytest
|
||||
from unittest.mock import MagicMock, patch, ANY
|
||||
|
||||
from pydub import AudioSegment
|
||||
from pydub.generators import Sine
|
||||
|
||||
from karaoke_gen.instrumental_review import MuteRegion
|
||||
|
||||
|
||||
class TestAudioAnalysisService:
|
||||
"""Tests for AudioAnalysisService."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_storage_service(self):
|
||||
"""Create a mock storage service."""
|
||||
mock = MagicMock()
|
||||
return mock
|
||||
|
||||
@pytest.fixture
|
||||
def temp_audio_file(self, tmp_path):
|
||||
"""Create a temporary audio file for testing."""
|
||||
audio_path = tmp_path / "test_audio.flac"
|
||||
|
||||
# Create a simple sine wave audio
|
||||
tone = Sine(440).to_audio_segment(duration=5000) - 15
|
||||
tone.export(str(audio_path), format="flac")
|
||||
|
||||
return str(audio_path)
|
||||
|
||||
@pytest.fixture
|
||||
def silent_audio_file(self, tmp_path):
|
||||
"""Create a silent audio file for testing."""
|
||||
audio_path = tmp_path / "silent_audio.flac"
|
||||
|
||||
audio = AudioSegment.silent(duration=5000, frame_rate=44100)
|
||||
audio.export(str(audio_path), format="flac")
|
||||
|
||||
return str(audio_path)
|
||||
|
||||
def test_service_initialization(self, mock_storage_service):
|
||||
"""Test service initializes with default parameters."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
service = AudioAnalysisService(storage_service=mock_storage_service)
|
||||
|
||||
assert service.storage_service == mock_storage_service
|
||||
assert service.analyzer.silence_threshold_db == -40.0
|
||||
|
||||
def test_service_custom_threshold(self, mock_storage_service):
|
||||
"""Test service accepts custom silence threshold."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
service = AudioAnalysisService(
|
||||
storage_service=mock_storage_service,
|
||||
silence_threshold_db=-35.0,
|
||||
)
|
||||
|
||||
assert service.analyzer.silence_threshold_db == -35.0
|
||||
|
||||
def test_analyze_backing_vocals_downloads_file(
|
||||
self, mock_storage_service, temp_audio_file
|
||||
):
|
||||
"""Test that analyze_backing_vocals downloads file from GCS."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
# Set up mock to copy the temp file to the download location
|
||||
def mock_download(gcs_path, local_path):
|
||||
import shutil
|
||||
shutil.copy(temp_audio_file, local_path)
|
||||
return local_path
|
||||
|
||||
mock_storage_service.download_file.side_effect = mock_download
|
||||
|
||||
service = AudioAnalysisService(storage_service=mock_storage_service)
|
||||
result = service.analyze_backing_vocals(
|
||||
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
|
||||
job_id="test-job",
|
||||
)
|
||||
|
||||
# Verify download was called
|
||||
mock_storage_service.download_file.assert_called_once()
|
||||
|
||||
# Verify result is valid
|
||||
assert result is not None
|
||||
assert hasattr(result, 'has_audible_content')
|
||||
|
||||
def test_analyze_silent_audio_returns_no_audible(
|
||||
self, mock_storage_service, silent_audio_file
|
||||
):
|
||||
"""Test that silent audio returns has_audible_content=False."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
def mock_download(gcs_path, local_path):
|
||||
import shutil
|
||||
shutil.copy(silent_audio_file, local_path)
|
||||
return local_path
|
||||
|
||||
mock_storage_service.download_file.side_effect = mock_download
|
||||
|
||||
service = AudioAnalysisService(storage_service=mock_storage_service)
|
||||
result = service.analyze_backing_vocals(
|
||||
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
|
||||
job_id="test-job",
|
||||
)
|
||||
|
||||
assert result.has_audible_content is False
|
||||
assert len(result.audible_segments) == 0
|
||||
|
||||
def test_analyze_and_generate_waveform_uploads_image(
|
||||
self, mock_storage_service, temp_audio_file
|
||||
):
|
||||
"""Test that analyze_and_generate_waveform uploads waveform to GCS."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
def mock_download(gcs_path, local_path):
|
||||
import shutil
|
||||
shutil.copy(temp_audio_file, local_path)
|
||||
return local_path
|
||||
|
||||
mock_storage_service.download_file.side_effect = mock_download
|
||||
mock_storage_service.upload_file.return_value = "jobs/test/analysis/waveform.png"
|
||||
|
||||
service = AudioAnalysisService(storage_service=mock_storage_service)
|
||||
result, waveform_path = service.analyze_and_generate_waveform(
|
||||
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
|
||||
job_id="test-job",
|
||||
gcs_waveform_destination="jobs/test/analysis/waveform.png",
|
||||
)
|
||||
|
||||
# Verify upload was called
|
||||
mock_storage_service.upload_file.assert_called_once()
|
||||
|
||||
# Verify waveform path is returned
|
||||
assert waveform_path == "jobs/test/analysis/waveform.png"
|
||||
|
||||
def test_get_waveform_data_returns_amplitudes(
|
||||
self, mock_storage_service, temp_audio_file
|
||||
):
|
||||
"""Test that get_waveform_data returns amplitude data."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
def mock_download(gcs_path, local_path):
|
||||
import shutil
|
||||
shutil.copy(temp_audio_file, local_path)
|
||||
return local_path
|
||||
|
||||
mock_storage_service.download_file.side_effect = mock_download
|
||||
|
||||
service = AudioAnalysisService(storage_service=mock_storage_service)
|
||||
amplitudes, duration = service.get_waveform_data(
|
||||
gcs_audio_path="jobs/test/stems/backing_vocals.flac",
|
||||
job_id="test-job",
|
||||
num_points=100,
|
||||
)
|
||||
|
||||
# Verify we get amplitude data
|
||||
assert isinstance(amplitudes, list)
|
||||
assert len(amplitudes) > 0
|
||||
assert all(isinstance(a, float) for a in amplitudes)
|
||||
|
||||
# Verify duration is reasonable
|
||||
assert duration > 0
|
||||
assert duration <= 10 # Our test file is 5 seconds
|
||||
|
||||
|
||||
class TestAudioEditingService:
|
||||
"""Tests for AudioEditingService."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_storage_service(self):
|
||||
"""Create a mock storage service."""
|
||||
mock = MagicMock()
|
||||
return mock
|
||||
|
||||
@pytest.fixture
|
||||
def temp_audio_files(self, tmp_path):
|
||||
"""Create temporary audio files for testing."""
|
||||
clean_path = tmp_path / "clean_instrumental.flac"
|
||||
backing_path = tmp_path / "backing_vocals.flac"
|
||||
|
||||
# Create clean instrumental (chord)
|
||||
tone1 = Sine(220).to_audio_segment(duration=5000) - 15
|
||||
tone2 = Sine(330).to_audio_segment(duration=5000) - 15
|
||||
clean = tone1.overlay(tone2)
|
||||
clean.export(str(clean_path), format="flac")
|
||||
|
||||
# Create backing vocals
|
||||
backing = Sine(440).to_audio_segment(duration=5000) - 20
|
||||
backing.export(str(backing_path), format="flac")
|
||||
|
||||
return str(clean_path), str(backing_path)
|
||||
|
||||
def test_service_initialization(self, mock_storage_service):
|
||||
"""Test service initializes correctly."""
|
||||
from backend.services.audio_editing_service import AudioEditingService
|
||||
|
||||
service = AudioEditingService(storage_service=mock_storage_service)
|
||||
|
||||
assert service.storage_service == mock_storage_service
|
||||
assert service.editor.output_format == "flac"
|
||||
|
||||
def test_create_custom_instrumental_downloads_files(
|
||||
self, mock_storage_service, temp_audio_files
|
||||
):
|
||||
"""Test that create_custom_instrumental downloads input files."""
|
||||
from backend.services.audio_editing_service import AudioEditingService
|
||||
|
||||
clean_path, backing_path = temp_audio_files
|
||||
|
||||
# Track which files were downloaded
|
||||
downloaded_files = []
|
||||
|
||||
def mock_download(gcs_path, local_path):
|
||||
downloaded_files.append(gcs_path)
|
||||
# Copy the appropriate test file
|
||||
import shutil
|
||||
if 'clean' in gcs_path:
|
||||
shutil.copy(clean_path, local_path)
|
||||
else:
|
||||
shutil.copy(backing_path, local_path)
|
||||
return local_path
|
||||
|
||||
mock_storage_service.download_file.side_effect = mock_download
|
||||
mock_storage_service.upload_file.return_value = "jobs/test/stems/custom.flac"
|
||||
|
||||
service = AudioEditingService(storage_service=mock_storage_service)
|
||||
|
||||
mute_regions = [MuteRegion(start_seconds=1.0, end_seconds=2.0)]
|
||||
|
||||
result = service.create_custom_instrumental(
|
||||
gcs_clean_instrumental_path="jobs/test/stems/clean.flac",
|
||||
gcs_backing_vocals_path="jobs/test/stems/backing.flac",
|
||||
mute_regions=mute_regions,
|
||||
gcs_output_path="jobs/test/stems/custom.flac",
|
||||
job_id="test-job",
|
||||
)
|
||||
|
||||
# Verify both files were downloaded
|
||||
assert len(downloaded_files) == 2
|
||||
assert mock_storage_service.upload_file.called
|
||||
|
||||
def test_create_custom_instrumental_uploads_result(
|
||||
self, mock_storage_service, temp_audio_files
|
||||
):
|
||||
"""Test that create_custom_instrumental uploads the result."""
|
||||
from backend.services.audio_editing_service import AudioEditingService
|
||||
|
||||
clean_path, backing_path = temp_audio_files
|
||||
|
||||
def mock_download(gcs_path, local_path):
|
||||
import shutil
|
||||
if 'clean' in gcs_path:
|
||||
shutil.copy(clean_path, local_path)
|
||||
else:
|
||||
shutil.copy(backing_path, local_path)
|
||||
return local_path
|
||||
|
||||
mock_storage_service.download_file.side_effect = mock_download
|
||||
mock_storage_service.upload_file.return_value = "jobs/test/stems/custom.flac"
|
||||
|
||||
service = AudioEditingService(storage_service=mock_storage_service)
|
||||
|
||||
mute_regions = [MuteRegion(start_seconds=1.0, end_seconds=2.0)]
|
||||
|
||||
result = service.create_custom_instrumental(
|
||||
gcs_clean_instrumental_path="jobs/test/stems/clean.flac",
|
||||
gcs_backing_vocals_path="jobs/test/stems/backing.flac",
|
||||
mute_regions=mute_regions,
|
||||
gcs_output_path="jobs/test/stems/custom.flac",
|
||||
job_id="test-job",
|
||||
)
|
||||
|
||||
# Verify upload was called with correct destination
|
||||
mock_storage_service.upload_file.assert_called_once()
|
||||
call_args = mock_storage_service.upload_file.call_args
|
||||
assert call_args[0][1] == "jobs/test/stems/custom.flac"
|
||||
|
||||
# Verify result has correct path
|
||||
assert result.output_path == "jobs/test/stems/custom.flac"
|
||||
|
||||
def test_create_custom_instrumental_returns_statistics(
|
||||
self, mock_storage_service, temp_audio_files
|
||||
):
|
||||
"""Test that create_custom_instrumental returns correct statistics."""
|
||||
from backend.services.audio_editing_service import AudioEditingService
|
||||
|
||||
clean_path, backing_path = temp_audio_files
|
||||
|
||||
def mock_download(gcs_path, local_path):
|
||||
import shutil
|
||||
if 'clean' in gcs_path:
|
||||
shutil.copy(clean_path, local_path)
|
||||
else:
|
||||
shutil.copy(backing_path, local_path)
|
||||
return local_path
|
||||
|
||||
mock_storage_service.download_file.side_effect = mock_download
|
||||
mock_storage_service.upload_file.return_value = "jobs/test/stems/custom.flac"
|
||||
|
||||
service = AudioEditingService(storage_service=mock_storage_service)
|
||||
|
||||
mute_regions = [
|
||||
MuteRegion(start_seconds=1.0, end_seconds=2.0),
|
||||
MuteRegion(start_seconds=3.0, end_seconds=4.0),
|
||||
]
|
||||
|
||||
result = service.create_custom_instrumental(
|
||||
gcs_clean_instrumental_path="jobs/test/stems/clean.flac",
|
||||
gcs_backing_vocals_path="jobs/test/stems/backing.flac",
|
||||
mute_regions=mute_regions,
|
||||
gcs_output_path="jobs/test/stems/custom.flac",
|
||||
job_id="test-job",
|
||||
)
|
||||
|
||||
# Verify statistics
|
||||
assert len(result.mute_regions_applied) == 2
|
||||
assert result.total_muted_duration_seconds == 2.0 # 1s + 1s
|
||||
assert result.output_duration_seconds > 0
|
||||
|
||||
def test_validate_mute_regions_returns_errors_for_invalid(
|
||||
self, mock_storage_service
|
||||
):
|
||||
"""Test that validate_mute_regions catches invalid regions."""
|
||||
from backend.services.audio_editing_service import AudioEditingService
|
||||
|
||||
service = AudioEditingService(storage_service=mock_storage_service)
|
||||
|
||||
mute_regions = [
|
||||
MuteRegion(start_seconds=0.0, end_seconds=1.0), # Valid
|
||||
MuteRegion(start_seconds=100.0, end_seconds=110.0), # Beyond duration
|
||||
]
|
||||
|
||||
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=60.0)
|
||||
|
||||
# Should have error for region exceeding duration
|
||||
assert len(errors) == 1
|
||||
assert "exceeds" in errors[0].lower()
|
||||
|
||||
def test_validate_mute_regions_returns_empty_for_valid(
|
||||
self, mock_storage_service
|
||||
):
|
||||
"""Test that validate_mute_regions returns empty for valid regions."""
|
||||
from backend.services.audio_editing_service import AudioEditingService
|
||||
|
||||
service = AudioEditingService(storage_service=mock_storage_service)
|
||||
|
||||
mute_regions = [
|
||||
MuteRegion(start_seconds=0.0, end_seconds=1.0),
|
||||
MuteRegion(start_seconds=10.0, end_seconds=15.0),
|
||||
]
|
||||
|
||||
errors = service.validate_mute_regions(mute_regions, total_duration_seconds=60.0)
|
||||
|
||||
assert len(errors) == 0
|
||||
@@ -0,0 +1,457 @@
|
||||
"""
|
||||
API endpoint tests for instrumental review functionality.
|
||||
|
||||
Tests for:
|
||||
- GET /api/jobs/{job_id}/instrumental-analysis
|
||||
- GET /api/jobs/{job_id}/audio-stream/{stem_type}
|
||||
- POST /api/jobs/{job_id}/create-custom-instrumental
|
||||
- GET /api/jobs/{job_id}/waveform-data
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from datetime import datetime
|
||||
from unittest.mock import MagicMock, patch, AsyncMock
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from backend.models.job import Job, JobStatus
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job_manager():
|
||||
"""Create a mock job manager."""
|
||||
with patch('backend.api.routes.jobs.job_manager') as mock:
|
||||
yield mock
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_storage_service():
|
||||
"""Create a mock storage service."""
|
||||
with patch('backend.api.routes.jobs.StorageService') as mock:
|
||||
mock_instance = MagicMock()
|
||||
mock_instance.generate_signed_url.return_value = "https://storage.googleapis.com/signed-url"
|
||||
mock.return_value = mock_instance
|
||||
yield mock_instance
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sample_job():
|
||||
"""Create a sample job for testing."""
|
||||
return Job(
|
||||
job_id="test-job-123",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
|
||||
created_at=datetime.now(),
|
||||
updated_at=datetime.now(),
|
||||
file_urls={
|
||||
"stems": {
|
||||
"instrumental_clean": "jobs/test/stems/instrumental_clean.flac",
|
||||
"backing_vocals": "jobs/test/stems/backing_vocals.flac",
|
||||
"instrumental_with_backing": "jobs/test/stems/instrumental_with_backing.flac",
|
||||
},
|
||||
"analysis": {
|
||||
"backing_vocals_waveform": "jobs/test/analysis/waveform.png",
|
||||
},
|
||||
},
|
||||
state_data={
|
||||
"backing_vocals_analysis": {
|
||||
"has_audible_content": True,
|
||||
"total_duration_seconds": 180.0,
|
||||
"audible_segments": [
|
||||
{
|
||||
"start_seconds": 10.0,
|
||||
"end_seconds": 20.0,
|
||||
"duration_seconds": 10.0,
|
||||
"avg_amplitude_db": -25.0,
|
||||
},
|
||||
{
|
||||
"start_seconds": 60.0,
|
||||
"end_seconds": 80.0,
|
||||
"duration_seconds": 20.0,
|
||||
"avg_amplitude_db": -30.0,
|
||||
},
|
||||
],
|
||||
"recommended_selection": "review_needed",
|
||||
"total_audible_duration_seconds": 30.0,
|
||||
"audible_percentage": 16.67,
|
||||
"silence_threshold_db": -40.0,
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def silent_job():
|
||||
"""Create a job with silent backing vocals."""
|
||||
return Job(
|
||||
job_id="test-job-silent",
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
|
||||
created_at=datetime.now(),
|
||||
updated_at=datetime.now(),
|
||||
file_urls={
|
||||
"stems": {
|
||||
"instrumental_clean": "jobs/test/stems/instrumental_clean.flac",
|
||||
"backing_vocals": "jobs/test/stems/backing_vocals.flac",
|
||||
"instrumental_with_backing": "jobs/test/stems/instrumental_with_backing.flac",
|
||||
},
|
||||
"analysis": {
|
||||
"backing_vocals_waveform": "jobs/test/analysis/waveform.png",
|
||||
},
|
||||
},
|
||||
state_data={
|
||||
"backing_vocals_analysis": {
|
||||
"has_audible_content": False,
|
||||
"total_duration_seconds": 180.0,
|
||||
"audible_segments": [],
|
||||
"recommended_selection": "clean",
|
||||
"total_audible_duration_seconds": 0.0,
|
||||
"audible_percentage": 0.0,
|
||||
"silence_threshold_db": -40.0,
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
class TestGetInstrumentalAnalysis:
|
||||
"""Tests for GET /api/jobs/{job_id}/instrumental-analysis."""
|
||||
|
||||
def test_get_analysis_returns_data(self, mock_job_manager, mock_storage_service, sample_job):
|
||||
"""GET /instrumental-analysis should return analysis data."""
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
from backend.api.routes.jobs import get_instrumental_analysis
|
||||
import asyncio
|
||||
|
||||
result = asyncio.get_event_loop().run_until_complete(
|
||||
get_instrumental_analysis("test-job-123")
|
||||
)
|
||||
|
||||
assert result["job_id"] == "test-job-123"
|
||||
assert result["artist"] == "Test Artist"
|
||||
assert result["title"] == "Test Song"
|
||||
assert result["analysis"]["has_audible_content"] is True
|
||||
assert len(result["analysis"]["audible_segments"]) == 2
|
||||
|
||||
def test_get_analysis_includes_audio_urls(self, mock_job_manager, mock_storage_service, sample_job):
|
||||
"""GET /instrumental-analysis should include audio URLs."""
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
from backend.api.routes.jobs import get_instrumental_analysis
|
||||
import asyncio
|
||||
|
||||
result = asyncio.get_event_loop().run_until_complete(
|
||||
get_instrumental_analysis("test-job-123")
|
||||
)
|
||||
|
||||
assert "audio_urls" in result
|
||||
assert "clean_instrumental" in result["audio_urls"]
|
||||
assert "backing_vocals" in result["audio_urls"]
|
||||
assert "with_backing" in result["audio_urls"]
|
||||
|
||||
def test_get_analysis_includes_waveform_url(self, mock_job_manager, sample_job):
|
||||
"""GET /instrumental-analysis should include waveform URL."""
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
# Need to mock StorageService at module level for the signed URL
|
||||
with patch('backend.api.routes.jobs.StorageService') as mock_storage_cls:
|
||||
mock_storage = MagicMock()
|
||||
mock_storage.generate_signed_url.return_value = "https://storage.googleapis.com/signed-url"
|
||||
mock_storage_cls.return_value = mock_storage
|
||||
|
||||
from backend.api.routes.jobs import get_instrumental_analysis
|
||||
import asyncio
|
||||
|
||||
result = asyncio.get_event_loop().run_until_complete(
|
||||
get_instrumental_analysis("test-job-123")
|
||||
)
|
||||
|
||||
assert "waveform_url" in result
|
||||
# Waveform URL should be generated from the file_urls
|
||||
assert result["waveform_url"] == "https://storage.googleapis.com/signed-url"
|
||||
|
||||
def test_get_analysis_silent_audio(self, mock_job_manager, mock_storage_service, silent_job):
|
||||
"""GET /instrumental-analysis should correctly report silent audio."""
|
||||
mock_job_manager.get_job.return_value = silent_job
|
||||
|
||||
from backend.api.routes.jobs import get_instrumental_analysis
|
||||
import asyncio
|
||||
|
||||
result = asyncio.get_event_loop().run_until_complete(
|
||||
get_instrumental_analysis("test-job-silent")
|
||||
)
|
||||
|
||||
assert result["analysis"]["has_audible_content"] is False
|
||||
assert result["analysis"]["recommended_selection"] == "clean"
|
||||
assert len(result["analysis"]["audible_segments"]) == 0
|
||||
|
||||
def test_get_analysis_job_not_found(self, mock_job_manager):
|
||||
"""GET /instrumental-analysis should return 404 for non-existent job."""
|
||||
mock_job_manager.get_job.return_value = None
|
||||
|
||||
from backend.api.routes.jobs import get_instrumental_analysis
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
get_instrumental_analysis("non-existent")
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 404
|
||||
|
||||
def test_get_analysis_wrong_status(self, mock_job_manager, sample_job):
|
||||
"""GET /instrumental-analysis should return 400 for wrong job status."""
|
||||
sample_job.status = JobStatus.PENDING
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
from backend.api.routes.jobs import get_instrumental_analysis
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
get_instrumental_analysis("test-job-123")
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 400
|
||||
|
||||
|
||||
class TestAudioStream:
|
||||
"""Tests for GET /api/jobs/{job_id}/audio-stream/{stem_type}."""
|
||||
|
||||
def test_stream_backing_vocals_valid(self, mock_job_manager, sample_job):
|
||||
"""Should return redirect for backing vocals stream."""
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
from backend.api.routes.jobs import stream_audio
|
||||
import asyncio
|
||||
|
||||
# This will try to stream from GCS which we can't test easily
|
||||
# Instead, verify the function exists and has correct signature
|
||||
assert callable(stream_audio)
|
||||
|
||||
def test_stream_invalid_stem_type(self, mock_job_manager, sample_job):
|
||||
"""Should return 400 for invalid stem type."""
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
from backend.api.routes.jobs import stream_audio
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
stream_audio("test-job-123", "invalid_stem")
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 400
|
||||
|
||||
def test_stream_job_not_found(self, mock_job_manager):
|
||||
"""Should return 404 for non-existent job."""
|
||||
mock_job_manager.get_job.return_value = None
|
||||
|
||||
from backend.api.routes.jobs import stream_audio
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
stream_audio("non-existent", "backing_vocals")
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 404
|
||||
|
||||
|
||||
class TestCreateCustomInstrumental:
|
||||
"""Tests for POST /api/jobs/{job_id}/create-custom-instrumental."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_audio_editing_service(self):
|
||||
"""Mock the audio editing service."""
|
||||
with patch('backend.api.routes.jobs.AudioEditingService') as mock:
|
||||
mock_instance = MagicMock()
|
||||
mock_instance.create_custom_instrumental.return_value = MagicMock(
|
||||
output_path="jobs/test/stems/custom_instrumental.flac",
|
||||
mute_regions_applied=[
|
||||
MagicMock(start_seconds=10.0, end_seconds=20.0),
|
||||
],
|
||||
total_muted_duration_seconds=10.0,
|
||||
output_duration_seconds=180.0,
|
||||
)
|
||||
mock.return_value = mock_instance
|
||||
yield mock_instance
|
||||
|
||||
def test_create_custom_validates_mute_regions(self):
|
||||
"""Should validate mute regions at model level."""
|
||||
from backend.models.requests import CreateCustomInstrumentalRequest, MuteRegionRequest
|
||||
from pydantic import ValidationError
|
||||
|
||||
# Empty mute regions should fail at model validation
|
||||
with pytest.raises(ValidationError) as exc_info:
|
||||
CreateCustomInstrumentalRequest(mute_regions=[])
|
||||
|
||||
assert "At least one mute region is required" in str(exc_info.value)
|
||||
|
||||
def test_create_custom_job_not_found(self, mock_job_manager):
|
||||
"""Should return 404 for non-existent job."""
|
||||
mock_job_manager.get_job.return_value = None
|
||||
|
||||
from backend.api.routes.jobs import create_custom_instrumental
|
||||
from backend.models.requests import CreateCustomInstrumentalRequest, MuteRegionRequest
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
request = CreateCustomInstrumentalRequest(
|
||||
mute_regions=[MuteRegionRequest(start_seconds=10.0, end_seconds=20.0)]
|
||||
)
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
create_custom_instrumental("non-existent", request)
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 404
|
||||
|
||||
def test_create_custom_wrong_status(self, mock_job_manager, sample_job):
|
||||
"""Should return 400 for wrong job status."""
|
||||
sample_job.status = JobStatus.PENDING
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
from backend.api.routes.jobs import create_custom_instrumental
|
||||
from backend.models.requests import CreateCustomInstrumentalRequest, MuteRegionRequest
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
request = CreateCustomInstrumentalRequest(
|
||||
mute_regions=[MuteRegionRequest(start_seconds=10.0, end_seconds=20.0)]
|
||||
)
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
create_custom_instrumental("test-job-123", request)
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 400
|
||||
|
||||
|
||||
class TestGetWaveformData:
|
||||
"""Tests for GET /api/jobs/{job_id}/waveform-data."""
|
||||
|
||||
def test_get_waveform_data_job_not_found(self, mock_job_manager):
|
||||
"""Should return 404 for non-existent job."""
|
||||
mock_job_manager.get_job.return_value = None
|
||||
|
||||
from backend.api.routes.jobs import get_waveform_data
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
get_waveform_data("non-existent")
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 404
|
||||
|
||||
def test_get_waveform_data_wrong_status(self, mock_job_manager, sample_job):
|
||||
"""Should return 400 for wrong job status."""
|
||||
sample_job.status = JobStatus.PENDING
|
||||
mock_job_manager.get_job.return_value = sample_job
|
||||
|
||||
from backend.api.routes.jobs import get_waveform_data
|
||||
from fastapi import HTTPException
|
||||
import asyncio
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
asyncio.get_event_loop().run_until_complete(
|
||||
get_waveform_data("test-job-123")
|
||||
)
|
||||
|
||||
assert exc_info.value.status_code == 400
|
||||
|
||||
def test_waveform_data_endpoint_exists(self):
|
||||
"""Verify the endpoint function exists and is callable."""
|
||||
from backend.api.routes.jobs import get_waveform_data
|
||||
|
||||
assert callable(get_waveform_data)
|
||||
# Verify it's an async function
|
||||
import inspect
|
||||
assert inspect.iscoroutinefunction(get_waveform_data)
|
||||
|
||||
|
||||
class TestRequestModels:
|
||||
"""Tests for API request models."""
|
||||
|
||||
def test_mute_region_request_valid(self):
|
||||
"""MuteRegionRequest should accept valid values."""
|
||||
from backend.models.requests import MuteRegionRequest
|
||||
|
||||
region = MuteRegionRequest(start_seconds=10.0, end_seconds=20.0)
|
||||
|
||||
assert region.start_seconds == 10.0
|
||||
assert region.end_seconds == 20.0
|
||||
|
||||
def test_mute_region_request_invalid_start(self):
|
||||
"""MuteRegionRequest should reject negative start."""
|
||||
from backend.models.requests import MuteRegionRequest
|
||||
from pydantic import ValidationError
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
MuteRegionRequest(start_seconds=-1.0, end_seconds=20.0)
|
||||
|
||||
def test_mute_region_request_invalid_order(self):
|
||||
"""MuteRegionRequest should reject end before start."""
|
||||
from backend.models.requests import MuteRegionRequest
|
||||
from pydantic import ValidationError
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
MuteRegionRequest(start_seconds=20.0, end_seconds=10.0)
|
||||
|
||||
def test_create_custom_instrumental_request_valid(self):
|
||||
"""CreateCustomInstrumentalRequest should accept valid mute regions."""
|
||||
from backend.models.requests import (
|
||||
CreateCustomInstrumentalRequest,
|
||||
MuteRegionRequest,
|
||||
)
|
||||
|
||||
request = CreateCustomInstrumentalRequest(
|
||||
mute_regions=[
|
||||
MuteRegionRequest(start_seconds=10.0, end_seconds=20.0),
|
||||
MuteRegionRequest(start_seconds=60.0, end_seconds=80.0),
|
||||
]
|
||||
)
|
||||
|
||||
assert len(request.mute_regions) == 2
|
||||
|
||||
|
||||
class TestInstrumentalSelectionExtension:
|
||||
"""Tests for extended instrumental selection (including 'custom')."""
|
||||
|
||||
def test_instrumental_selection_accepts_custom(self):
|
||||
"""InstrumentalSelection should accept 'custom' as valid selection."""
|
||||
from backend.models.requests import InstrumentalSelection
|
||||
|
||||
selection = InstrumentalSelection(selection="custom")
|
||||
assert selection.selection == "custom"
|
||||
|
||||
def test_instrumental_selection_accepts_clean(self):
|
||||
"""InstrumentalSelection should accept 'clean' as valid selection."""
|
||||
from backend.models.requests import InstrumentalSelection
|
||||
|
||||
selection = InstrumentalSelection(selection="clean")
|
||||
assert selection.selection == "clean"
|
||||
|
||||
def test_instrumental_selection_accepts_with_backing(self):
|
||||
"""InstrumentalSelection should accept 'with_backing' as valid selection."""
|
||||
from backend.models.requests import InstrumentalSelection
|
||||
|
||||
selection = InstrumentalSelection(selection="with_backing")
|
||||
assert selection.selection == "with_backing"
|
||||
|
||||
def test_instrumental_selection_rejects_invalid(self):
|
||||
"""InstrumentalSelection should reject invalid values."""
|
||||
from backend.models.requests import InstrumentalSelection
|
||||
from pydantic import ValidationError
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
InstrumentalSelection(selection="invalid_option")
|
||||
@@ -3,6 +3,9 @@ Unit tests for internal.py API routes using FastAPI TestClient.
|
||||
|
||||
Internal routes are used for worker communication within Cloud Run.
|
||||
These endpoints require admin authentication.
|
||||
|
||||
With Cloud Tasks integration, these endpoints include idempotency checks
|
||||
to prevent duplicate processing on retries.
|
||||
"""
|
||||
import os
|
||||
import pytest
|
||||
@@ -17,9 +20,20 @@ from backend.models.job import Job, JobStatus
|
||||
os.environ.setdefault('ADMIN_TOKENS', 'test-admin-token')
|
||||
|
||||
|
||||
def _create_mock_job(job_id: str = "test123") -> Job:
|
||||
"""Create a mock Job object for testing."""
|
||||
return Job(
|
||||
job_id=job_id,
|
||||
status=JobStatus.PROCESSING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
state_data={}, # Empty state_data - no worker progress yet
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
"""Create TestClient with mocked workers."""
|
||||
"""Create TestClient with mocked workers and job manager."""
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
@@ -28,11 +42,19 @@ def client():
|
||||
mock_lyrics = AsyncMock()
|
||||
mock_screens = AsyncMock()
|
||||
mock_video = AsyncMock()
|
||||
mock_render = AsyncMock()
|
||||
|
||||
# Mock JobManager for idempotency checks
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = _create_mock_job()
|
||||
mock_job_manager.update_state_data.return_value = None
|
||||
|
||||
with patch('backend.api.routes.internal.process_audio_separation', mock_audio), \
|
||||
patch('backend.api.routes.internal.process_lyrics_transcription', mock_lyrics), \
|
||||
patch('backend.api.routes.internal.generate_screens', mock_screens), \
|
||||
patch('backend.api.routes.internal.generate_video', mock_video), \
|
||||
patch('backend.api.routes.internal.process_render_video', mock_render), \
|
||||
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
@@ -175,6 +197,36 @@ class TestVideoWorkerEndpoint:
|
||||
assert response.status_code == 401
|
||||
|
||||
|
||||
class TestRenderVideoWorkerEndpoint:
|
||||
"""Tests for POST /api/internal/workers/render-video."""
|
||||
|
||||
def test_render_video_worker_returns_200(self, client, auth_headers):
|
||||
"""Test render-video worker endpoint returns 200 with auth."""
|
||||
response = client.post(
|
||||
"/api/internal/workers/render-video",
|
||||
headers=auth_headers,
|
||||
json={"job_id": "test123"}
|
||||
)
|
||||
assert response.status_code == 200
|
||||
|
||||
def test_render_video_worker_requires_job_id(self, client, auth_headers):
|
||||
"""Test render-video worker requires job_id."""
|
||||
response = client.post(
|
||||
"/api/internal/workers/render-video",
|
||||
headers=auth_headers,
|
||||
json={}
|
||||
)
|
||||
assert response.status_code == 422
|
||||
|
||||
def test_render_video_worker_requires_auth(self, client):
|
||||
"""Test render-video worker requires authentication."""
|
||||
response = client.post(
|
||||
"/api/internal/workers/render-video",
|
||||
json={"job_id": "test123"}
|
||||
)
|
||||
assert response.status_code == 401
|
||||
|
||||
|
||||
class TestWorkerResponseFormat:
|
||||
"""Tests for worker response format."""
|
||||
|
||||
@@ -198,3 +250,94 @@ class TestWorkerResponseFormat:
|
||||
data = response.json()
|
||||
assert "status" in data or "message" in data
|
||||
|
||||
|
||||
class TestWorkerIdempotency:
|
||||
"""Tests for worker idempotency checks (Cloud Tasks retry support)."""
|
||||
|
||||
def test_audio_worker_returns_already_running_when_in_progress(self, auth_headers):
|
||||
"""Test audio worker returns already_running if worker is already running."""
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
# Create job with audio already running
|
||||
mock_job = _create_mock_job()
|
||||
mock_job.state_data = {'audio_progress': {'stage': 'running'}}
|
||||
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = mock_job
|
||||
|
||||
with patch('backend.api.routes.internal.process_audio_separation', AsyncMock()), \
|
||||
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
from backend.main import app
|
||||
client = TestClient(app)
|
||||
|
||||
response = client.post(
|
||||
"/api/internal/workers/audio",
|
||||
headers=auth_headers,
|
||||
json={"job_id": "test123"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "already_running"
|
||||
|
||||
def test_audio_worker_returns_already_complete_when_finished(self, auth_headers):
|
||||
"""Test audio worker returns already_complete if worker already finished."""
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
# Create job with audio already complete
|
||||
mock_job = _create_mock_job()
|
||||
mock_job.state_data = {'audio_progress': {'stage': 'complete'}}
|
||||
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = mock_job
|
||||
|
||||
with patch('backend.api.routes.internal.process_audio_separation', AsyncMock()), \
|
||||
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
from backend.main import app
|
||||
client = TestClient(app)
|
||||
|
||||
response = client.post(
|
||||
"/api/internal/workers/audio",
|
||||
headers=auth_headers,
|
||||
json={"job_id": "test123"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "already_complete"
|
||||
|
||||
def test_audio_worker_returns_not_found_when_job_missing(self, auth_headers):
|
||||
"""Test audio worker returns not_found if job doesn't exist."""
|
||||
mock_creds = MagicMock()
|
||||
mock_creds.universe_domain = 'googleapis.com'
|
||||
|
||||
# Return None for job not found
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = None
|
||||
|
||||
with patch('backend.api.routes.internal.process_audio_separation', AsyncMock()), \
|
||||
patch('backend.api.routes.internal.JobManager', return_value=mock_job_manager), \
|
||||
patch('backend.services.firestore_service.firestore'), \
|
||||
patch('backend.services.storage_service.storage'), \
|
||||
patch('google.auth.default', return_value=(mock_creds, 'test-project')):
|
||||
from backend.main import app
|
||||
client = TestClient(app)
|
||||
|
||||
response = client.post(
|
||||
"/api/internal/workers/audio",
|
||||
headers=auth_headers,
|
||||
json={"job_id": "nonexistent"}
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "not_found"
|
||||
|
||||
@@ -577,6 +577,342 @@ class TestJobModelIntegration:
|
||||
assert restored_job.state_data["lyrics_metadata"]["line_count"] == 50
|
||||
|
||||
|
||||
class TestExistingInstrumentalModelFields:
|
||||
"""Test existing instrumental field in Job and JobCreate models (Batch 3)."""
|
||||
|
||||
def test_job_has_existing_instrumental_gcs_path(self):
|
||||
"""Test that Job model has existing_instrumental_gcs_path field."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
existing_instrumental_gcs_path="uploads/test123/audio/existing_instrumental.flac"
|
||||
)
|
||||
|
||||
assert hasattr(job, 'existing_instrumental_gcs_path')
|
||||
assert job.existing_instrumental_gcs_path == "uploads/test123/audio/existing_instrumental.flac"
|
||||
|
||||
def test_job_existing_instrumental_defaults_to_none(self):
|
||||
"""Test that existing_instrumental_gcs_path defaults to None."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC)
|
||||
)
|
||||
|
||||
assert job.existing_instrumental_gcs_path is None
|
||||
|
||||
def test_job_create_has_existing_instrumental_gcs_path(self):
|
||||
"""Test that JobCreate model has existing_instrumental_gcs_path field."""
|
||||
from backend.models.job import JobCreate
|
||||
|
||||
job_create = JobCreate(
|
||||
artist="Artist",
|
||||
title="Title",
|
||||
existing_instrumental_gcs_path="uploads/test/audio/existing_instrumental.mp3"
|
||||
)
|
||||
|
||||
assert job_create.existing_instrumental_gcs_path == "uploads/test/audio/existing_instrumental.mp3"
|
||||
|
||||
def test_job_create_existing_instrumental_optional(self):
|
||||
"""Test that existing_instrumental_gcs_path is optional in JobCreate."""
|
||||
from backend.models.job import JobCreate
|
||||
|
||||
job_create = JobCreate(
|
||||
artist="Artist",
|
||||
title="Title"
|
||||
)
|
||||
|
||||
assert job_create.existing_instrumental_gcs_path is None
|
||||
|
||||
def test_existing_instrumental_roundtrip_serialization(self):
|
||||
"""Test that existing_instrumental_gcs_path survives serialization/deserialization."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
existing_instrumental_gcs_path="uploads/test123/audio/existing_instrumental.wav"
|
||||
)
|
||||
|
||||
# Serialize to dict (simulates writing to Firestore)
|
||||
job_dict = job.model_dump()
|
||||
|
||||
# Convert to JSON and back (simulates Firestore storage)
|
||||
json_str = json.dumps(job_dict, default=str)
|
||||
restored_dict = json.loads(json_str)
|
||||
|
||||
# Deserialize back to Job (simulates reading from Firestore)
|
||||
restored_job = Job(**restored_dict)
|
||||
|
||||
assert restored_job.existing_instrumental_gcs_path == "uploads/test123/audio/existing_instrumental.wav"
|
||||
|
||||
def test_job_with_full_audio_config(self):
|
||||
"""Test Job with complete audio configuration including existing instrumental."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.AUDIO_COMPLETE,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
# Audio model configuration
|
||||
clean_instrumental_model="model_bs_roformer_ep_317_sdr_12.9755.ckpt",
|
||||
backing_vocals_models=["mel_band_roformer_karaoke.ckpt"],
|
||||
other_stems_models=["htdemucs_6s.yaml"],
|
||||
# Existing instrumental
|
||||
existing_instrumental_gcs_path="uploads/test123/audio/my_instrumental.flac",
|
||||
)
|
||||
|
||||
assert job.clean_instrumental_model == "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
assert job.existing_instrumental_gcs_path == "uploads/test123/audio/my_instrumental.flac"
|
||||
|
||||
|
||||
class TestTwoPhaseWorkflowFields:
|
||||
"""Test two-phase workflow fields in Job and JobCreate models (Batch 6)."""
|
||||
|
||||
def test_job_has_prep_only_field(self):
|
||||
"""Test that Job model has prep_only field."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
prep_only=True
|
||||
)
|
||||
|
||||
assert hasattr(job, 'prep_only')
|
||||
assert job.prep_only is True
|
||||
|
||||
def test_job_prep_only_defaults_to_false(self):
|
||||
"""Test that prep_only defaults to False."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC)
|
||||
)
|
||||
|
||||
assert job.prep_only is False
|
||||
|
||||
def test_job_has_finalise_only_field(self):
|
||||
"""Test that Job model has finalise_only field."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
finalise_only=True
|
||||
)
|
||||
|
||||
assert hasattr(job, 'finalise_only')
|
||||
assert job.finalise_only is True
|
||||
|
||||
def test_job_finalise_only_defaults_to_false(self):
|
||||
"""Test that finalise_only defaults to False."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC)
|
||||
)
|
||||
|
||||
assert job.finalise_only is False
|
||||
|
||||
def test_job_has_keep_brand_code_field(self):
|
||||
"""Test that Job model has keep_brand_code field."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
keep_brand_code="NOMAD-1234"
|
||||
)
|
||||
|
||||
assert hasattr(job, 'keep_brand_code')
|
||||
assert job.keep_brand_code == "NOMAD-1234"
|
||||
|
||||
def test_job_keep_brand_code_defaults_to_none(self):
|
||||
"""Test that keep_brand_code defaults to None."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC)
|
||||
)
|
||||
|
||||
assert job.keep_brand_code is None
|
||||
|
||||
def test_job_create_has_two_phase_workflow_fields(self):
|
||||
"""Test that JobCreate model has two-phase workflow fields."""
|
||||
job_create = JobCreate(
|
||||
artist="Artist",
|
||||
title="Title",
|
||||
prep_only=True,
|
||||
finalise_only=False,
|
||||
keep_brand_code="BRAND-0001"
|
||||
)
|
||||
|
||||
assert job_create.prep_only is True
|
||||
assert job_create.finalise_only is False
|
||||
assert job_create.keep_brand_code == "BRAND-0001"
|
||||
|
||||
def test_two_phase_workflow_roundtrip_serialization(self):
|
||||
"""Test that two-phase workflow fields survive serialization/deserialization."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
prep_only=True,
|
||||
finalise_only=False,
|
||||
keep_brand_code="NOMAD-5678"
|
||||
)
|
||||
|
||||
# Serialize to dict (simulates writing to Firestore)
|
||||
job_dict = job.model_dump()
|
||||
|
||||
# Convert to JSON and back (simulates Firestore storage)
|
||||
json_str = json.dumps(job_dict, default=str)
|
||||
restored_dict = json.loads(json_str)
|
||||
|
||||
# Deserialize back to Job (simulates reading from Firestore)
|
||||
restored_job = Job(**restored_dict)
|
||||
|
||||
assert restored_job.prep_only is True
|
||||
assert restored_job.finalise_only is False
|
||||
assert restored_job.keep_brand_code == "NOMAD-5678"
|
||||
|
||||
|
||||
class TestPrepCompleteStatus:
|
||||
"""Test PREP_COMPLETE status and state transitions (Batch 6)."""
|
||||
|
||||
def test_prep_complete_status_exists(self):
|
||||
"""Test that PREP_COMPLETE status exists in JobStatus enum."""
|
||||
assert hasattr(JobStatus, 'PREP_COMPLETE')
|
||||
assert JobStatus.PREP_COMPLETE.value == "prep_complete"
|
||||
|
||||
def test_job_can_have_prep_complete_status(self):
|
||||
"""Test that Job can be created with PREP_COMPLETE status."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PREP_COMPLETE,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC)
|
||||
)
|
||||
|
||||
assert job.status == JobStatus.PREP_COMPLETE
|
||||
|
||||
def test_state_transitions_review_complete_to_prep_complete(self):
|
||||
"""Test that REVIEW_COMPLETE can transition to PREP_COMPLETE."""
|
||||
from backend.models.job import STATE_TRANSITIONS
|
||||
|
||||
valid_transitions = STATE_TRANSITIONS.get(JobStatus.REVIEW_COMPLETE, [])
|
||||
assert JobStatus.PREP_COMPLETE in valid_transitions
|
||||
|
||||
def test_state_transitions_prep_complete_to_awaiting_instrumental(self):
|
||||
"""Test that PREP_COMPLETE can transition to AWAITING_INSTRUMENTAL_SELECTION."""
|
||||
from backend.models.job import STATE_TRANSITIONS
|
||||
|
||||
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PREP_COMPLETE, [])
|
||||
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
|
||||
|
||||
def test_state_transitions_pending_to_awaiting_instrumental(self):
|
||||
"""Test that PENDING can transition to AWAITING_INSTRUMENTAL_SELECTION (for finalise-only)."""
|
||||
from backend.models.job import STATE_TRANSITIONS
|
||||
|
||||
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PENDING, [])
|
||||
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
|
||||
|
||||
def test_critical_statuses_include_prep_complete(self):
|
||||
"""Test that PREP_COMPLETE is in the list of job statuses."""
|
||||
all_statuses = [status.value for status in JobStatus]
|
||||
assert "prep_complete" in all_statuses
|
||||
|
||||
|
||||
class TestJobWithFullTwoPhaseConfig:
|
||||
"""Integration tests for two-phase workflow job configuration."""
|
||||
|
||||
def test_prep_only_job_full_config(self):
|
||||
"""Test creating a prep-only job with full configuration."""
|
||||
job = Job(
|
||||
job_id="prep123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
prep_only=True,
|
||||
enable_cdg=True,
|
||||
enable_txt=True,
|
||||
brand_prefix="NOMAD",
|
||||
)
|
||||
|
||||
assert job.prep_only is True
|
||||
assert job.finalise_only is False
|
||||
assert job.brand_prefix == "NOMAD"
|
||||
|
||||
def test_finalise_only_job_full_config(self):
|
||||
"""Test creating a finalise-only job with full configuration."""
|
||||
job = Job(
|
||||
job_id="final123",
|
||||
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
finalise_only=True,
|
||||
keep_brand_code="NOMAD-1234",
|
||||
enable_youtube_upload=True,
|
||||
dropbox_path="/Karaoke/Tracks",
|
||||
state_data={
|
||||
"audio_complete": True,
|
||||
"lyrics_complete": True,
|
||||
"finalise_only": True,
|
||||
}
|
||||
)
|
||||
|
||||
assert job.finalise_only is True
|
||||
assert job.keep_brand_code == "NOMAD-1234"
|
||||
assert job.state_data["finalise_only"] is True
|
||||
|
||||
def test_prep_complete_job_with_file_urls(self):
|
||||
"""Test a job that has reached PREP_COMPLETE status with all prep outputs."""
|
||||
job = Job(
|
||||
job_id="prepcomp123",
|
||||
status=JobStatus.PREP_COMPLETE,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="ABBA",
|
||||
title="Waterloo",
|
||||
prep_only=True,
|
||||
file_urls={
|
||||
"stems": {
|
||||
"instrumental_clean": "gs://bucket/jobs/prepcomp123/stems/instrumental_clean.flac",
|
||||
"instrumental_with_backing": "gs://bucket/jobs/prepcomp123/stems/instrumental_with_backing.flac",
|
||||
},
|
||||
"videos": {
|
||||
"with_vocals": "gs://bucket/jobs/prepcomp123/videos/with_vocals.mkv",
|
||||
},
|
||||
"screens": {
|
||||
"title": "gs://bucket/jobs/prepcomp123/screens/title.mov",
|
||||
"end": "gs://bucket/jobs/prepcomp123/screens/end.mov",
|
||||
},
|
||||
"lyrics": {
|
||||
"lrc": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.lrc",
|
||||
"ass": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.ass",
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
assert job.status == JobStatus.PREP_COMPLETE
|
||||
assert job.prep_only is True
|
||||
assert "with_vocals" in job.file_urls["videos"]
|
||||
assert "title" in job.file_urls["screens"]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
pytest.main([__file__, "-v"])
|
||||
|
||||
@@ -12,6 +12,7 @@ from datetime import datetime, UTC
|
||||
import sys
|
||||
sys.modules['google.cloud.firestore'] = MagicMock()
|
||||
sys.modules['google.cloud.storage'] = MagicMock()
|
||||
sys.modules['google.cloud.tasks_v2'] = MagicMock()
|
||||
|
||||
from backend.services.auth_service import AuthService, UserType
|
||||
from backend.services.storage_service import StorageService
|
||||
@@ -239,6 +240,190 @@ class TestWorkerService:
|
||||
assert "/api/internal/workers/video" in str(call_args)
|
||||
|
||||
|
||||
class TestWorkerServiceCloudTasks:
|
||||
"""Test WorkerService Cloud Tasks integration."""
|
||||
|
||||
def test_should_use_cloud_tasks_default_false(self):
|
||||
"""Test that Cloud Tasks is disabled by default."""
|
||||
from backend.services.worker_service import WorkerService, reset_worker_service
|
||||
reset_worker_service()
|
||||
|
||||
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
|
||||
mock_getenv.side_effect = lambda k, d=None: {
|
||||
'PORT': '8000',
|
||||
}.get(k, d)
|
||||
|
||||
with patch('backend.services.worker_service.get_settings') as mock_settings:
|
||||
mock_settings.return_value.admin_tokens = None
|
||||
mock_settings.return_value.google_cloud_project = "test-project"
|
||||
mock_settings.return_value.enable_cloud_tasks = False
|
||||
mock_settings.return_value.gcp_region = "us-central1"
|
||||
mock_settings.return_value.use_cloud_run_jobs_for_video = False
|
||||
|
||||
service = WorkerService()
|
||||
assert service._use_cloud_tasks is False
|
||||
|
||||
def test_should_use_cloud_tasks_enabled(self):
|
||||
"""Test that Cloud Tasks can be enabled via settings."""
|
||||
from backend.services.worker_service import WorkerService, reset_worker_service
|
||||
reset_worker_service()
|
||||
|
||||
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
|
||||
mock_getenv.side_effect = lambda k, d=None: {
|
||||
'PORT': '8000',
|
||||
}.get(k, d)
|
||||
|
||||
with patch('backend.services.worker_service.get_settings') as mock_settings:
|
||||
mock_settings.return_value.admin_tokens = None
|
||||
mock_settings.return_value.google_cloud_project = "test-project"
|
||||
mock_settings.return_value.enable_cloud_tasks = True
|
||||
mock_settings.return_value.gcp_region = "us-central1"
|
||||
mock_settings.return_value.use_cloud_run_jobs_for_video = False
|
||||
|
||||
service = WorkerService()
|
||||
assert service._use_cloud_tasks is True
|
||||
|
||||
def test_worker_queues_mapping(self):
|
||||
"""Test that all worker types have queue mappings."""
|
||||
from backend.services.worker_service import WORKER_QUEUES
|
||||
|
||||
# Verify all expected workers have queues
|
||||
assert "audio" in WORKER_QUEUES
|
||||
assert "lyrics" in WORKER_QUEUES
|
||||
assert "screens" in WORKER_QUEUES
|
||||
assert "render-video" in WORKER_QUEUES
|
||||
assert "video" in WORKER_QUEUES
|
||||
|
||||
# Verify queue names are correct
|
||||
assert WORKER_QUEUES["audio"] == "audio-worker-queue"
|
||||
assert WORKER_QUEUES["lyrics"] == "lyrics-worker-queue"
|
||||
assert WORKER_QUEUES["screens"] == "screens-worker-queue"
|
||||
assert WORKER_QUEUES["render-video"] == "render-worker-queue"
|
||||
assert WORKER_QUEUES["video"] == "video-worker-queue"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trigger_worker_uses_http_when_cloud_tasks_disabled(self):
|
||||
"""Test that trigger_worker uses HTTP when Cloud Tasks is disabled."""
|
||||
from backend.services.worker_service import WorkerService, reset_worker_service
|
||||
reset_worker_service()
|
||||
|
||||
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
|
||||
mock_getenv.side_effect = lambda k, d=None: {
|
||||
'PORT': '8000',
|
||||
}.get(k, d)
|
||||
|
||||
with patch('backend.services.worker_service.get_settings') as mock_settings:
|
||||
mock_settings.return_value.admin_tokens = "test-token"
|
||||
mock_settings.return_value.google_cloud_project = "test-project"
|
||||
mock_settings.return_value.enable_cloud_tasks = False
|
||||
mock_settings.return_value.gcp_region = "us-central1"
|
||||
mock_settings.return_value.use_cloud_run_jobs_for_video = False
|
||||
|
||||
with patch('backend.services.worker_service.httpx.AsyncClient') as mock_client_cls:
|
||||
mock_client = AsyncMock()
|
||||
mock_client_cls.return_value.__aenter__.return_value = mock_client
|
||||
|
||||
mock_response = Mock()
|
||||
mock_response.status_code = 200
|
||||
mock_client.post.return_value = mock_response
|
||||
|
||||
service = WorkerService()
|
||||
assert service._use_cloud_tasks is False
|
||||
|
||||
result = await service.trigger_worker("audio", "test-job-123")
|
||||
|
||||
# Verify HTTP was used
|
||||
assert result is True
|
||||
mock_client.post.assert_called_once()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trigger_worker_uses_cloud_tasks_when_enabled(self):
|
||||
"""Test that trigger_worker uses Cloud Tasks when enabled."""
|
||||
from backend.services.worker_service import WorkerService, reset_worker_service
|
||||
reset_worker_service()
|
||||
|
||||
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
|
||||
mock_getenv.side_effect = lambda k, d=None: {
|
||||
'CLOUD_RUN_SERVICE_URL': 'https://api.example.com',
|
||||
}.get(k, d)
|
||||
|
||||
with patch('backend.services.worker_service.get_settings') as mock_settings:
|
||||
mock_settings.return_value.admin_tokens = "test-token"
|
||||
mock_settings.return_value.google_cloud_project = "test-project"
|
||||
mock_settings.return_value.enable_cloud_tasks = True
|
||||
mock_settings.return_value.gcp_region = "us-central1"
|
||||
mock_settings.return_value.use_cloud_run_jobs_for_video = False
|
||||
|
||||
# Mock Cloud Tasks client
|
||||
mock_tasks_client = MagicMock()
|
||||
mock_tasks_client.queue_path.return_value = "projects/test-project/locations/us-central1/queues/audio-worker-queue"
|
||||
mock_task_response = MagicMock()
|
||||
mock_task_response.name = "projects/test-project/locations/us-central1/queues/audio-worker-queue/tasks/abc123"
|
||||
mock_tasks_client.create_task.return_value = mock_task_response
|
||||
|
||||
# Mock the google.cloud.tasks_v2 module which is imported inside the method
|
||||
mock_tasks_module = MagicMock()
|
||||
mock_tasks_module.CloudTasksClient.return_value = mock_tasks_client
|
||||
mock_tasks_module.HttpMethod.POST = "POST"
|
||||
|
||||
with patch.dict('sys.modules', {'google.cloud.tasks_v2': mock_tasks_module}):
|
||||
with patch.dict(sys.modules, {'google.cloud': MagicMock()}):
|
||||
service = WorkerService()
|
||||
service._tasks_client = mock_tasks_client # Inject mocked client
|
||||
|
||||
result = await service.trigger_worker("audio", "test-job-123")
|
||||
|
||||
# Verify Cloud Tasks was used
|
||||
assert result is True
|
||||
mock_tasks_client.create_task.assert_called_once()
|
||||
|
||||
# Verify task payload
|
||||
call_args = mock_tasks_client.create_task.call_args
|
||||
assert call_args is not None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trigger_worker_returns_false_for_unknown_worker_type(self):
|
||||
"""Test that trigger_worker returns False for unknown worker types."""
|
||||
from backend.services.worker_service import WorkerService, reset_worker_service
|
||||
reset_worker_service()
|
||||
|
||||
with patch('backend.services.worker_service.os.getenv') as mock_getenv:
|
||||
mock_getenv.side_effect = lambda k, d=None: {
|
||||
'CLOUD_RUN_SERVICE_URL': 'https://api.example.com',
|
||||
}.get(k, d)
|
||||
|
||||
with patch('backend.services.worker_service.get_settings') as mock_settings:
|
||||
mock_settings.return_value.admin_tokens = "test-token"
|
||||
mock_settings.return_value.google_cloud_project = "test-project"
|
||||
mock_settings.return_value.enable_cloud_tasks = True
|
||||
mock_settings.return_value.gcp_region = "us-central1"
|
||||
mock_settings.return_value.use_cloud_run_jobs_for_video = False
|
||||
|
||||
service = WorkerService()
|
||||
|
||||
result = await service.trigger_worker("unknown-worker", "test-job-123")
|
||||
|
||||
assert result is False
|
||||
|
||||
def test_reset_worker_service(self):
|
||||
"""Test that reset_worker_service resets the singleton."""
|
||||
from backend.services.worker_service import get_worker_service, reset_worker_service
|
||||
|
||||
with patch('backend.services.worker_service.get_settings') as mock_settings:
|
||||
mock_settings.return_value.admin_tokens = None
|
||||
mock_settings.return_value.google_cloud_project = "test-project"
|
||||
mock_settings.return_value.enable_cloud_tasks = False
|
||||
mock_settings.return_value.gcp_region = "us-central1"
|
||||
mock_settings.return_value.use_cloud_run_jobs_for_video = False
|
||||
|
||||
service1 = get_worker_service()
|
||||
reset_worker_service()
|
||||
service2 = get_worker_service()
|
||||
|
||||
# After reset, should be different instances
|
||||
assert service1 is not service2
|
||||
|
||||
|
||||
class TestFirestoreService:
|
||||
"""Test FirestoreService database operations."""
|
||||
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
"""
|
||||
Unit tests for backend validation script (validate.py).
|
||||
|
||||
Tests the validation functions that check for import errors, syntax errors,
|
||||
configuration issues, and FastAPI app creation.
|
||||
"""
|
||||
import pytest
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
|
||||
class TestValidateImports:
|
||||
"""Tests for the validate_imports function."""
|
||||
|
||||
def test_validate_imports_success(self):
|
||||
"""Test that validate_imports succeeds when all modules import."""
|
||||
from backend.validate import validate_imports
|
||||
|
||||
# This should succeed since we're in a valid test environment
|
||||
result = validate_imports()
|
||||
assert result is True
|
||||
|
||||
def test_validate_imports_with_failure(self):
|
||||
"""Test that validate_imports handles import failures."""
|
||||
from backend.validate import validate_imports
|
||||
|
||||
# Mock importlib to simulate a failure
|
||||
with patch('backend.validate.importlib.import_module') as mock_import:
|
||||
mock_import.side_effect = ImportError("Module not found")
|
||||
|
||||
result = validate_imports()
|
||||
assert result is False
|
||||
|
||||
|
||||
class TestValidateSyntax:
|
||||
"""Tests for the validate_syntax function."""
|
||||
|
||||
def test_validate_syntax_success(self):
|
||||
"""Test that validate_syntax succeeds with valid Python files."""
|
||||
from backend.validate import validate_syntax
|
||||
|
||||
# This should succeed since the backend code is valid
|
||||
result = validate_syntax()
|
||||
assert result is True
|
||||
|
||||
def test_validate_syntax_handles_invalid_file(self):
|
||||
"""Test that validate_syntax detects syntax errors."""
|
||||
import tempfile
|
||||
import os
|
||||
from pathlib import Path
|
||||
from backend.validate import validate_syntax
|
||||
|
||||
# We can't easily inject invalid files into the backend dir,
|
||||
# but we can verify the function runs and returns True for valid files
|
||||
result = validate_syntax()
|
||||
assert result is True
|
||||
|
||||
|
||||
class TestValidateConfig:
|
||||
"""Tests for the validate_config function."""
|
||||
|
||||
def test_validate_config_success(self):
|
||||
"""Test that validate_config succeeds with valid configuration."""
|
||||
from backend.validate import validate_config
|
||||
|
||||
# This should succeed in test environment
|
||||
result = validate_config()
|
||||
assert result is True
|
||||
|
||||
def test_validate_config_failure(self):
|
||||
"""Test that validate_config handles configuration errors."""
|
||||
from backend.validate import validate_config
|
||||
|
||||
# Patch at the source module
|
||||
with patch('backend.config.get_settings') as mock_settings:
|
||||
mock_settings.side_effect = Exception("Config error")
|
||||
|
||||
result = validate_config()
|
||||
assert result is False
|
||||
|
||||
|
||||
class TestValidateFastapiApp:
|
||||
"""Tests for the validate_fastapi_app function."""
|
||||
|
||||
def test_validate_fastapi_app_success(self):
|
||||
"""Test that validate_fastapi_app succeeds."""
|
||||
from backend.validate import validate_fastapi_app
|
||||
|
||||
# This should succeed since the FastAPI app is valid
|
||||
result = validate_fastapi_app()
|
||||
assert result is True
|
||||
|
||||
def test_validate_fastapi_app_failure(self):
|
||||
"""Test that validate_fastapi_app handles app creation errors."""
|
||||
from backend.validate import validate_fastapi_app
|
||||
|
||||
with patch.dict('sys.modules', {'backend.main': MagicMock(app=MagicMock(title="Test", version="1.0", routes=[]))}):
|
||||
# Even with a mock, the function should work
|
||||
result = validate_fastapi_app()
|
||||
# The original module is still accessible, so this should still pass
|
||||
assert result is True
|
||||
|
||||
|
||||
class TestMain:
|
||||
"""Tests for the main function."""
|
||||
|
||||
def test_main_runs(self):
|
||||
"""Test that main runs without errors."""
|
||||
from backend.validate import main
|
||||
|
||||
# In a valid test environment, main should return 0 (success)
|
||||
result = main()
|
||||
assert result == 0
|
||||
|
||||
def test_main_returns_1_on_failure(self):
|
||||
"""Test that main returns 1 when validations fail."""
|
||||
from backend.validate import main
|
||||
|
||||
# Mock one validation to fail
|
||||
with patch('backend.validate.validate_imports', return_value=False):
|
||||
result = main()
|
||||
assert result == 1
|
||||
|
||||
|
||||
class TestValidateModule:
|
||||
"""Test the validate module can be imported and has expected functions."""
|
||||
|
||||
def test_module_imports(self):
|
||||
"""Test that the validate module can be imported."""
|
||||
import backend.validate
|
||||
assert backend.validate is not None
|
||||
|
||||
def test_has_validate_imports(self):
|
||||
"""Test that validate_imports function exists."""
|
||||
from backend.validate import validate_imports
|
||||
assert callable(validate_imports)
|
||||
|
||||
def test_has_validate_syntax(self):
|
||||
"""Test that validate_syntax function exists."""
|
||||
from backend.validate import validate_syntax
|
||||
assert callable(validate_syntax)
|
||||
|
||||
def test_has_validate_config(self):
|
||||
"""Test that validate_config function exists."""
|
||||
from backend.validate import validate_config
|
||||
assert callable(validate_config)
|
||||
|
||||
def test_has_validate_fastapi_app(self):
|
||||
"""Test that validate_fastapi_app function exists."""
|
||||
from backend.validate import validate_fastapi_app
|
||||
assert callable(validate_fastapi_app)
|
||||
|
||||
def test_has_main(self):
|
||||
"""Test that main function exists."""
|
||||
from backend.validate import main
|
||||
assert callable(main)
|
||||
|
||||
@@ -0,0 +1,365 @@
|
||||
"""
|
||||
Unit tests for backend/workers/worker_logging.py.
|
||||
|
||||
Tests the job logging utilities including JobLogger, JobLogHandler,
|
||||
and job_logging_context for concurrent job isolation.
|
||||
"""
|
||||
import pytest
|
||||
import logging
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
|
||||
class TestJobLoggingContext:
|
||||
"""Tests for job_logging_context context manager."""
|
||||
|
||||
def test_job_logging_context_sets_and_resets_job_id(self):
|
||||
"""Test that job_logging_context sets and resets the current job ID."""
|
||||
from backend.workers.worker_logging import job_logging_context, _current_job_id
|
||||
|
||||
# Initially no job
|
||||
assert _current_job_id.get() is None
|
||||
|
||||
# Inside context, job is set
|
||||
with job_logging_context("job123"):
|
||||
assert _current_job_id.get() == "job123"
|
||||
|
||||
# After context, job is reset
|
||||
assert _current_job_id.get() is None
|
||||
|
||||
def test_job_logging_context_nested(self):
|
||||
"""Test nested job_logging_context calls."""
|
||||
from backend.workers.worker_logging import job_logging_context, _current_job_id
|
||||
|
||||
with job_logging_context("outer_job"):
|
||||
assert _current_job_id.get() == "outer_job"
|
||||
|
||||
with job_logging_context("inner_job"):
|
||||
assert _current_job_id.get() == "inner_job"
|
||||
|
||||
# After inner context, outer job is restored
|
||||
assert _current_job_id.get() == "outer_job"
|
||||
|
||||
assert _current_job_id.get() is None
|
||||
|
||||
def test_job_logging_context_handles_exception(self):
|
||||
"""Test that job_logging_context resets even on exception."""
|
||||
from backend.workers.worker_logging import job_logging_context, _current_job_id
|
||||
|
||||
try:
|
||||
with job_logging_context("job123"):
|
||||
assert _current_job_id.get() == "job123"
|
||||
raise ValueError("Test exception")
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
# Job should still be reset after exception
|
||||
assert _current_job_id.get() is None
|
||||
|
||||
|
||||
class TestJobLogHandler:
|
||||
"""Tests for JobLogHandler class."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job_manager(self):
|
||||
"""Create a mock JobManager."""
|
||||
manager = MagicMock()
|
||||
manager.append_worker_log.return_value = None
|
||||
return manager
|
||||
|
||||
def test_job_log_handler_init(self, mock_job_manager):
|
||||
"""Test JobLogHandler initialization."""
|
||||
from backend.workers.worker_logging import JobLogHandler
|
||||
|
||||
handler = JobLogHandler(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
assert handler.job_id == "job123"
|
||||
assert handler.worker_name == "audio"
|
||||
assert handler.job_manager == mock_job_manager
|
||||
assert handler.level == logging.INFO
|
||||
|
||||
def test_job_log_handler_custom_level(self, mock_job_manager):
|
||||
"""Test JobLogHandler with custom level."""
|
||||
from backend.workers.worker_logging import JobLogHandler
|
||||
|
||||
handler = JobLogHandler(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager,
|
||||
level=logging.DEBUG
|
||||
)
|
||||
|
||||
assert handler.level == logging.DEBUG
|
||||
|
||||
def test_job_log_handler_emit_logs_to_firestore(self, mock_job_manager):
|
||||
"""Test that emit() calls job_manager.append_worker_log."""
|
||||
from backend.workers.worker_logging import JobLogHandler, job_logging_context
|
||||
|
||||
handler = JobLogHandler(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
handler.setFormatter(logging.Formatter('%(message)s'))
|
||||
|
||||
# Create a log record
|
||||
record = logging.LogRecord(
|
||||
name="test",
|
||||
level=logging.INFO,
|
||||
pathname="",
|
||||
lineno=0,
|
||||
msg="Test message",
|
||||
args=(),
|
||||
exc_info=None
|
||||
)
|
||||
|
||||
# Emit within the job context
|
||||
with job_logging_context("job123"):
|
||||
handler.emit(record)
|
||||
|
||||
# Should have called append_worker_log
|
||||
mock_job_manager.append_worker_log.assert_called()
|
||||
|
||||
def test_job_log_handler_filters_other_job_context(self, mock_job_manager):
|
||||
"""Test that handler filters logs from other job contexts."""
|
||||
from backend.workers.worker_logging import JobLogHandler, job_logging_context
|
||||
|
||||
handler = JobLogHandler(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
record = logging.LogRecord(
|
||||
name="test",
|
||||
level=logging.INFO,
|
||||
pathname="",
|
||||
lineno=0,
|
||||
msg="Test message",
|
||||
args=(),
|
||||
exc_info=None
|
||||
)
|
||||
|
||||
# Emit from a different job's context
|
||||
with job_logging_context("different_job"):
|
||||
handler.emit(record)
|
||||
|
||||
# Should NOT have called append_worker_log
|
||||
mock_job_manager.append_worker_log.assert_not_called()
|
||||
|
||||
def test_job_log_handler_deduplication(self, mock_job_manager):
|
||||
"""Test that handler deduplicates repeated messages."""
|
||||
from backend.workers.worker_logging import JobLogHandler, job_logging_context
|
||||
|
||||
handler = JobLogHandler(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
handler.setFormatter(logging.Formatter('%(message)s'))
|
||||
|
||||
record = logging.LogRecord(
|
||||
name="test",
|
||||
level=logging.INFO,
|
||||
pathname="",
|
||||
lineno=0,
|
||||
msg="Duplicate message",
|
||||
args=(),
|
||||
exc_info=None
|
||||
)
|
||||
|
||||
with job_logging_context("job123"):
|
||||
# Emit the same record twice
|
||||
handler.emit(record)
|
||||
handler.emit(record)
|
||||
|
||||
# Should only be called once due to deduplication
|
||||
assert mock_job_manager.append_worker_log.call_count == 1
|
||||
|
||||
|
||||
class TestJobLogger:
|
||||
"""Tests for JobLogger class."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job_manager(self):
|
||||
"""Create a mock JobManager."""
|
||||
manager = MagicMock()
|
||||
manager.append_worker_log.return_value = None
|
||||
return manager
|
||||
|
||||
def test_job_logger_init(self, mock_job_manager):
|
||||
"""Test JobLogger initialization."""
|
||||
from backend.workers.worker_logging import JobLogger
|
||||
|
||||
logger = JobLogger(
|
||||
job_id="job123",
|
||||
worker_name="lyrics",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
assert logger.job_id == "job123"
|
||||
assert logger.worker_name == "lyrics"
|
||||
assert logger.job_manager == mock_job_manager
|
||||
|
||||
def test_job_logger_info(self, mock_job_manager):
|
||||
"""Test JobLogger.info() method."""
|
||||
from backend.workers.worker_logging import JobLogger
|
||||
|
||||
logger = JobLogger(
|
||||
job_id="job123",
|
||||
worker_name="lyrics",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
logger.info("Processing started")
|
||||
|
||||
mock_job_manager.append_worker_log.assert_called_with(
|
||||
job_id="job123",
|
||||
worker="lyrics",
|
||||
level="INFO",
|
||||
message="Processing started"
|
||||
)
|
||||
|
||||
def test_job_logger_warning(self, mock_job_manager):
|
||||
"""Test JobLogger.warning() method."""
|
||||
from backend.workers.worker_logging import JobLogger
|
||||
|
||||
logger = JobLogger(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
logger.warning("Low memory")
|
||||
|
||||
mock_job_manager.append_worker_log.assert_called_with(
|
||||
job_id="job123",
|
||||
worker="audio",
|
||||
level="WARNING",
|
||||
message="Low memory"
|
||||
)
|
||||
|
||||
def test_job_logger_error(self, mock_job_manager):
|
||||
"""Test JobLogger.error() method."""
|
||||
from backend.workers.worker_logging import JobLogger
|
||||
|
||||
logger = JobLogger(
|
||||
job_id="job123",
|
||||
worker_name="video",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
logger.error("Processing failed")
|
||||
|
||||
mock_job_manager.append_worker_log.assert_called_with(
|
||||
job_id="job123",
|
||||
worker="video",
|
||||
level="ERROR",
|
||||
message="Processing failed"
|
||||
)
|
||||
|
||||
def test_job_logger_debug(self, mock_job_manager):
|
||||
"""Test JobLogger.debug() method."""
|
||||
from backend.workers.worker_logging import JobLogger
|
||||
|
||||
logger = JobLogger(
|
||||
job_id="job123",
|
||||
worker_name="screens",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
logger.debug("Debug info")
|
||||
|
||||
mock_job_manager.append_worker_log.assert_called_with(
|
||||
job_id="job123",
|
||||
worker="screens",
|
||||
level="DEBUG",
|
||||
message="Debug info"
|
||||
)
|
||||
|
||||
def test_job_logger_with_format_args(self, mock_job_manager):
|
||||
"""Test JobLogger with format arguments."""
|
||||
from backend.workers.worker_logging import JobLogger
|
||||
|
||||
logger = JobLogger(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
logger.info("Processing %s of %d", "audio", 10)
|
||||
|
||||
mock_job_manager.append_worker_log.assert_called_with(
|
||||
job_id="job123",
|
||||
worker="audio",
|
||||
level="INFO",
|
||||
message="Processing audio of 10"
|
||||
)
|
||||
|
||||
def test_job_logger_handles_firestore_error(self, mock_job_manager):
|
||||
"""Test that JobLogger handles Firestore errors gracefully."""
|
||||
from backend.workers.worker_logging import JobLogger
|
||||
|
||||
mock_job_manager.append_worker_log.side_effect = Exception("Firestore error")
|
||||
|
||||
logger = JobLogger(
|
||||
job_id="job123",
|
||||
worker_name="audio",
|
||||
job_manager=mock_job_manager
|
||||
)
|
||||
|
||||
# Should not raise exception
|
||||
logger.info("Test message")
|
||||
|
||||
|
||||
class TestCreateJobLogger:
|
||||
"""Tests for create_job_logger function."""
|
||||
|
||||
def test_create_job_logger(self):
|
||||
"""Test create_job_logger creates a JobLogger."""
|
||||
from backend.workers.worker_logging import create_job_logger, JobLogger
|
||||
|
||||
# Patch at the source module where JobManager is imported
|
||||
with patch('backend.services.job_manager.JobManager'):
|
||||
logger = create_job_logger("job123", "audio")
|
||||
|
||||
assert isinstance(logger, JobLogger)
|
||||
assert logger.job_id == "job123"
|
||||
assert logger.worker_name == "audio"
|
||||
|
||||
|
||||
class TestSetupJobLogging:
|
||||
"""Tests for setup_job_logging function."""
|
||||
|
||||
def test_setup_job_logging_returns_handler(self):
|
||||
"""Test setup_job_logging returns a JobLogHandler."""
|
||||
from backend.workers.worker_logging import setup_job_logging, JobLogHandler
|
||||
|
||||
# Patch at the source module
|
||||
with patch('backend.services.job_manager.JobManager'):
|
||||
handler = setup_job_logging("job123", "lyrics", "test_logger_wl1")
|
||||
|
||||
assert isinstance(handler, JobLogHandler)
|
||||
assert handler.job_id == "job123"
|
||||
assert handler.worker_name == "lyrics"
|
||||
|
||||
# Clean up - remove the handler
|
||||
logging.getLogger("test_logger_wl1").removeHandler(handler)
|
||||
|
||||
def test_setup_job_logging_adds_handler_to_loggers(self):
|
||||
"""Test setup_job_logging adds handler to specified loggers."""
|
||||
from backend.workers.worker_logging import setup_job_logging
|
||||
|
||||
with patch('backend.services.job_manager.JobManager'):
|
||||
handler = setup_job_logging("job123", "audio", "test_logger_wl2", "test_logger_wl3")
|
||||
|
||||
# Check handlers are added
|
||||
assert handler in logging.getLogger("test_logger_wl2").handlers
|
||||
assert handler in logging.getLogger("test_logger_wl3").handlers
|
||||
|
||||
# Clean up
|
||||
logging.getLogger("test_logger_wl2").removeHandler(handler)
|
||||
logging.getLogger("test_logger_wl3").removeHandler(handler)
|
||||
@@ -353,6 +353,108 @@ class TestLyricsWorker:
|
||||
mock_job_manager.mark_job_failed.assert_called()
|
||||
|
||||
|
||||
class TestLyricsWorkerConfiguration:
|
||||
"""Tests for lyrics worker configuration parameters."""
|
||||
|
||||
def test_create_lyrics_processor_with_defaults(self):
|
||||
"""Test creating LyricsProcessor with default parameters."""
|
||||
with patch('backend.workers.lyrics_worker.LyricsProcessor') as mock_processor:
|
||||
from backend.workers.lyrics_worker import create_lyrics_processor
|
||||
|
||||
result = create_lyrics_processor()
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
assert call_kwargs['lyrics_file'] is None
|
||||
assert call_kwargs['subtitle_offset_ms'] == 0
|
||||
|
||||
def test_create_lyrics_processor_with_lyrics_file(self):
|
||||
"""Test creating LyricsProcessor with custom lyrics file."""
|
||||
with patch('backend.workers.lyrics_worker.LyricsProcessor') as mock_processor:
|
||||
from backend.workers.lyrics_worker import create_lyrics_processor
|
||||
|
||||
result = create_lyrics_processor(
|
||||
lyrics_file="/path/to/lyrics.txt",
|
||||
subtitle_offset_ms=0
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
assert call_kwargs['lyrics_file'] == "/path/to/lyrics.txt"
|
||||
|
||||
def test_create_lyrics_processor_with_subtitle_offset(self):
|
||||
"""Test creating LyricsProcessor with subtitle offset."""
|
||||
with patch('backend.workers.lyrics_worker.LyricsProcessor') as mock_processor:
|
||||
from backend.workers.lyrics_worker import create_lyrics_processor
|
||||
|
||||
result = create_lyrics_processor(
|
||||
subtitle_offset_ms=500
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
assert call_kwargs['subtitle_offset_ms'] == 500
|
||||
|
||||
|
||||
class TestLyricsOverrideParameters:
|
||||
"""Tests for lyrics artist/title override functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job_with_overrides(self):
|
||||
"""Create a mock job with lyrics override fields."""
|
||||
return Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Beatles, The",
|
||||
title="Hey Jude - 2009 Remaster",
|
||||
lyrics_artist="The Beatles",
|
||||
lyrics_title="Hey Jude",
|
||||
subtitle_offset_ms=250,
|
||||
input_media_gcs_path="uploads/test123/song.flac"
|
||||
)
|
||||
|
||||
def test_job_uses_lyrics_artist_override(self, mock_job_with_overrides):
|
||||
"""Test that job uses lyrics_artist when searching for lyrics."""
|
||||
job = mock_job_with_overrides
|
||||
|
||||
# Use override if present, else fall back to main artist
|
||||
lyrics_search_artist = job.lyrics_artist or job.artist
|
||||
|
||||
assert lyrics_search_artist == "The Beatles"
|
||||
assert lyrics_search_artist != job.artist # Override is different
|
||||
|
||||
def test_job_uses_lyrics_title_override(self, mock_job_with_overrides):
|
||||
"""Test that job uses lyrics_title when searching for lyrics."""
|
||||
job = mock_job_with_overrides
|
||||
|
||||
# Use override if present, else fall back to main title
|
||||
lyrics_search_title = job.lyrics_title or job.title
|
||||
|
||||
assert lyrics_search_title == "Hey Jude"
|
||||
assert lyrics_search_title != job.title # Override is different
|
||||
|
||||
def test_job_falls_back_when_no_override(self):
|
||||
"""Test that job falls back to main artist/title when no override."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
input_media_gcs_path="uploads/test123/song.flac"
|
||||
)
|
||||
|
||||
# When override is None, use main values
|
||||
lyrics_search_artist = job.lyrics_artist or job.artist
|
||||
lyrics_search_title = job.lyrics_title or job.title
|
||||
|
||||
assert lyrics_search_artist == "Test Artist"
|
||||
assert lyrics_search_title == "Test Song"
|
||||
|
||||
|
||||
class TestScreensWorker:
|
||||
"""Tests for screens_worker.py functions.
|
||||
|
||||
@@ -379,6 +481,71 @@ class TestVideoWorker:
|
||||
assert hasattr(video_worker, 'logger')
|
||||
|
||||
|
||||
class TestRenderVideoWorkerConfiguration:
|
||||
"""Tests for render_video_worker subtitle_offset_ms support."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job_with_offset(self):
|
||||
"""Create a mock job with subtitle offset."""
|
||||
return Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.RENDERING_VIDEO,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
subtitle_offset_ms=500,
|
||||
input_media_gcs_path="uploads/test123/song.flac"
|
||||
)
|
||||
|
||||
def test_job_has_subtitle_offset_ms(self, mock_job_with_offset):
|
||||
"""Test that job has subtitle_offset_ms field."""
|
||||
assert mock_job_with_offset.subtitle_offset_ms == 500
|
||||
|
||||
def test_subtitle_offset_from_job(self, mock_job_with_offset):
|
||||
"""Test extracting subtitle offset from job with getattr."""
|
||||
job = mock_job_with_offset
|
||||
|
||||
# This mirrors the logic in render_video_worker.py
|
||||
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
|
||||
|
||||
assert subtitle_offset == 500
|
||||
|
||||
def test_subtitle_offset_default_zero(self):
|
||||
"""Test that subtitle offset defaults to 0 when not set."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.RENDERING_VIDEO,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
input_media_gcs_path="uploads/test123/song.flac"
|
||||
)
|
||||
|
||||
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
|
||||
|
||||
assert subtitle_offset == 0
|
||||
|
||||
def test_subtitle_offset_negative_value(self):
|
||||
"""Test that subtitle offset can be negative (advance subtitles)."""
|
||||
job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.RENDERING_VIDEO,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
subtitle_offset_ms=-250, # Negative = advance subtitles
|
||||
input_media_gcs_path="uploads/test123/song.flac"
|
||||
)
|
||||
|
||||
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
|
||||
|
||||
# Negative values should be preserved (not converted to 0)
|
||||
assert subtitle_offset == -250
|
||||
|
||||
|
||||
class TestDownloadHelpers:
|
||||
"""Tests for download helper functions in workers."""
|
||||
|
||||
@@ -406,3 +573,377 @@ class TestDownloadHelpers:
|
||||
# Should have downloaded from GCS
|
||||
mock_storage.download_file.assert_called()
|
||||
|
||||
|
||||
class TestAudioWorkerModelConfiguration:
|
||||
"""Tests for audio worker model configuration parameters."""
|
||||
|
||||
def test_create_audio_processor_with_defaults(self):
|
||||
"""Test creating AudioProcessor with default models."""
|
||||
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
|
||||
from backend.workers.audio_worker import create_audio_processor
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = create_audio_processor(temp_dir)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
|
||||
# Should use default models
|
||||
assert call_kwargs['clean_instrumental_model'] == "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
assert call_kwargs['backing_vocals_models'] == ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
assert call_kwargs['other_stems_models'] == ["htdemucs_6s.yaml"]
|
||||
|
||||
def test_create_audio_processor_with_custom_clean_model(self):
|
||||
"""Test creating AudioProcessor with custom clean instrumental model."""
|
||||
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
|
||||
from backend.workers.audio_worker import create_audio_processor
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = create_audio_processor(
|
||||
temp_dir,
|
||||
clean_instrumental_model="custom_clean_model.ckpt"
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
|
||||
# Should use custom clean model
|
||||
assert call_kwargs['clean_instrumental_model'] == "custom_clean_model.ckpt"
|
||||
# Other models should still be defaults
|
||||
assert call_kwargs['backing_vocals_models'] == ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
|
||||
def test_create_audio_processor_with_custom_backing_models(self):
|
||||
"""Test creating AudioProcessor with custom backing vocals models."""
|
||||
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
|
||||
from backend.workers.audio_worker import create_audio_processor
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = create_audio_processor(
|
||||
temp_dir,
|
||||
backing_vocals_models=["custom_bv1.ckpt", "custom_bv2.ckpt"]
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
|
||||
# Should use custom backing vocals models
|
||||
assert call_kwargs['backing_vocals_models'] == ["custom_bv1.ckpt", "custom_bv2.ckpt"]
|
||||
|
||||
def test_create_audio_processor_with_custom_other_stems_models(self):
|
||||
"""Test creating AudioProcessor with custom other stems models."""
|
||||
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
|
||||
from backend.workers.audio_worker import create_audio_processor
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = create_audio_processor(
|
||||
temp_dir,
|
||||
other_stems_models=["custom_demucs.yaml"]
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
|
||||
# Should use custom other stems models
|
||||
assert call_kwargs['other_stems_models'] == ["custom_demucs.yaml"]
|
||||
|
||||
def test_create_audio_processor_with_all_custom_models(self):
|
||||
"""Test creating AudioProcessor with all custom models."""
|
||||
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
|
||||
from backend.workers.audio_worker import create_audio_processor
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = create_audio_processor(
|
||||
temp_dir,
|
||||
clean_instrumental_model="custom_clean.ckpt",
|
||||
backing_vocals_models=["custom_bv.ckpt"],
|
||||
other_stems_models=["custom_stems.yaml"]
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
|
||||
# All models should be custom
|
||||
assert call_kwargs['clean_instrumental_model'] == "custom_clean.ckpt"
|
||||
assert call_kwargs['backing_vocals_models'] == ["custom_bv.ckpt"]
|
||||
assert call_kwargs['other_stems_models'] == ["custom_stems.yaml"]
|
||||
|
||||
def test_job_model_fields_are_passed_to_processor(self):
|
||||
"""Test that job model fields can be passed to create_audio_processor."""
|
||||
mock_job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test",
|
||||
title="Test",
|
||||
clean_instrumental_model="job_clean.ckpt",
|
||||
backing_vocals_models=["job_bv.ckpt"],
|
||||
other_stems_models=["job_stems.yaml"]
|
||||
)
|
||||
|
||||
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
|
||||
from backend.workers.audio_worker import create_audio_processor
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# Simulate passing job model fields to create_audio_processor
|
||||
result = create_audio_processor(
|
||||
temp_dir,
|
||||
clean_instrumental_model=mock_job.clean_instrumental_model,
|
||||
backing_vocals_models=mock_job.backing_vocals_models,
|
||||
other_stems_models=mock_job.other_stems_models
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
|
||||
# Models from job should be used
|
||||
assert call_kwargs['clean_instrumental_model'] == "job_clean.ckpt"
|
||||
assert call_kwargs['backing_vocals_models'] == ["job_bv.ckpt"]
|
||||
assert call_kwargs['other_stems_models'] == ["job_stems.yaml"]
|
||||
|
||||
def test_none_model_values_use_defaults(self):
|
||||
"""Test that None model values fall back to defaults."""
|
||||
mock_job = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PENDING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test",
|
||||
title="Test",
|
||||
clean_instrumental_model=None, # Not specified
|
||||
backing_vocals_models=None, # Not specified
|
||||
other_stems_models=None # Not specified
|
||||
)
|
||||
|
||||
with patch('backend.workers.audio_worker.AudioProcessor') as mock_processor:
|
||||
from backend.workers.audio_worker import create_audio_processor
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = create_audio_processor(
|
||||
temp_dir,
|
||||
clean_instrumental_model=mock_job.clean_instrumental_model,
|
||||
backing_vocals_models=mock_job.backing_vocals_models,
|
||||
other_stems_models=mock_job.other_stems_models
|
||||
)
|
||||
|
||||
mock_processor.assert_called_once()
|
||||
call_kwargs = mock_processor.call_args[1]
|
||||
|
||||
# Should fall back to defaults
|
||||
assert call_kwargs['clean_instrumental_model'] == "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
assert call_kwargs['backing_vocals_models'] == ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
assert call_kwargs['other_stems_models'] == ["htdemucs_6s.yaml"]
|
||||
|
||||
|
||||
class TestDownloadFromUrl:
|
||||
"""Tests for download_from_url function - URL-based audio download."""
|
||||
|
||||
def test_download_from_url_function_exists(self):
|
||||
"""Test that download_from_url function exists in audio_worker."""
|
||||
from backend.workers.audio_worker import download_from_url
|
||||
assert callable(download_from_url)
|
||||
|
||||
def test_download_from_url_signature(self):
|
||||
"""Test that download_from_url has the expected signature."""
|
||||
import inspect
|
||||
from backend.workers.audio_worker import download_from_url
|
||||
|
||||
sig = inspect.signature(download_from_url)
|
||||
params = list(sig.parameters.keys())
|
||||
|
||||
# Should have these parameters
|
||||
assert 'url' in params
|
||||
assert 'temp_dir' in params
|
||||
assert 'artist' in params
|
||||
assert 'title' in params
|
||||
assert 'job_manager' in params
|
||||
assert 'job_id' in params
|
||||
|
||||
def test_download_from_url_is_async(self):
|
||||
"""Test that download_from_url is an async function."""
|
||||
import inspect
|
||||
from backend.workers.audio_worker import download_from_url
|
||||
|
||||
assert inspect.iscoroutinefunction(download_from_url)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_from_url_returns_none_for_invalid_url(self):
|
||||
"""Test that download_from_url handles errors gracefully."""
|
||||
from backend.workers.audio_worker import download_from_url
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
# This should fail gracefully (no yt-dlp in test env or invalid URL)
|
||||
result = await download_from_url(
|
||||
url='not-a-valid-url',
|
||||
temp_dir=temp_dir,
|
||||
artist='Test',
|
||||
title='Test'
|
||||
)
|
||||
|
||||
# Should return None on error (graceful failure)
|
||||
assert result is None
|
||||
|
||||
|
||||
class TestDownloadAudioWithUrl:
|
||||
"""Tests for download_audio function with URL-based jobs."""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job_with_url(self):
|
||||
"""Create a mock job with a URL."""
|
||||
return Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PROCESSING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
url="https://www.youtube.com/watch?v=test123"
|
||||
)
|
||||
|
||||
@pytest.fixture
|
||||
def mock_job_manager(self, mock_job_with_url):
|
||||
"""Create a mock JobManager."""
|
||||
manager = MagicMock()
|
||||
manager.get_job.return_value = mock_job_with_url
|
||||
manager.update_job.return_value = None
|
||||
return manager
|
||||
|
||||
@pytest.fixture
|
||||
def mock_storage(self):
|
||||
"""Create a mock StorageService."""
|
||||
storage = MagicMock()
|
||||
return storage
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_audio_from_url(self, mock_job_with_url, mock_job_manager, mock_storage):
|
||||
"""Test download_audio routes to URL download when job has URL."""
|
||||
with patch('backend.workers.audio_worker.download_from_url', return_value='/tmp/test.wav') as mock_download:
|
||||
from backend.workers.audio_worker import download_audio
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = await download_audio(
|
||||
"test123",
|
||||
temp_dir,
|
||||
mock_storage,
|
||||
mock_job_with_url,
|
||||
job_manager_instance=mock_job_manager
|
||||
)
|
||||
|
||||
# Should have called download_from_url
|
||||
mock_download.assert_called_once()
|
||||
# Check positional args
|
||||
args = mock_download.call_args[0]
|
||||
assert args[0] == "https://www.youtube.com/watch?v=test123" # url
|
||||
assert args[1] == temp_dir # temp_dir
|
||||
assert args[2] == "Test Artist" # artist
|
||||
assert args[3] == "Test Song" # title
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_audio_from_gcs_when_no_url(self, mock_job_manager, mock_storage):
|
||||
"""Test download_audio downloads from GCS when job has no URL."""
|
||||
mock_job_no_url = Job(
|
||||
job_id="test123",
|
||||
status=JobStatus.PROCESSING,
|
||||
created_at=datetime.now(UTC),
|
||||
updated_at=datetime.now(UTC),
|
||||
artist="Test Artist",
|
||||
title="Test Song",
|
||||
input_media_gcs_path="uploads/test123/song.flac"
|
||||
)
|
||||
|
||||
with patch('backend.workers.audio_worker.download_from_url') as mock_url_download:
|
||||
from backend.workers.audio_worker import download_audio
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
result = await download_audio(
|
||||
"test123",
|
||||
temp_dir,
|
||||
mock_storage,
|
||||
mock_job_no_url,
|
||||
job_manager_instance=mock_job_manager
|
||||
)
|
||||
|
||||
# Should NOT have called download_from_url
|
||||
mock_url_download.assert_not_called()
|
||||
|
||||
# Should have called storage.download_file
|
||||
mock_storage.download_file.assert_called()
|
||||
|
||||
|
||||
class TestBackingVocalsAnalysis:
|
||||
"""Tests for backing vocals analysis in render_video_worker."""
|
||||
|
||||
def test_analyze_backing_vocals_function_exists(self):
|
||||
"""Test that _analyze_backing_vocals function exists in render_video_worker."""
|
||||
from backend.workers.render_video_worker import _analyze_backing_vocals
|
||||
assert callable(_analyze_backing_vocals)
|
||||
|
||||
def test_analyze_backing_vocals_is_async(self):
|
||||
"""Test that _analyze_backing_vocals is an async function."""
|
||||
import inspect
|
||||
from backend.workers.render_video_worker import _analyze_backing_vocals
|
||||
assert inspect.iscoroutinefunction(_analyze_backing_vocals)
|
||||
|
||||
def test_render_video_worker_imports_analysis_service(self):
|
||||
"""Test that render_video_worker can import AudioAnalysisService."""
|
||||
# This verifies the import path is correct
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
assert AudioAnalysisService is not None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_analyze_backing_vocals_handles_missing_job(self):
|
||||
"""Test that analysis returns early when job not found (no error stored)."""
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = None # No job found
|
||||
mock_job_manager.update_state_data = MagicMock()
|
||||
|
||||
mock_storage = MagicMock()
|
||||
mock_logger = MagicMock()
|
||||
|
||||
from backend.workers.render_video_worker import _analyze_backing_vocals
|
||||
|
||||
# Should not raise when job not found, just log warning and return
|
||||
await _analyze_backing_vocals(
|
||||
"nonexistent", mock_job_manager, mock_storage, mock_logger
|
||||
)
|
||||
|
||||
# Should log warning but not update state (early return)
|
||||
mock_logger.warning.assert_called()
|
||||
# No state_data update since we return early
|
||||
mock_job_manager.update_state_data.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_analyze_backing_vocals_handles_missing_stems(self):
|
||||
"""Test that analysis returns early when stems not found (no error stored)."""
|
||||
mock_job = MagicMock()
|
||||
mock_job.file_urls = {} # No stems
|
||||
|
||||
mock_job_manager = MagicMock()
|
||||
mock_job_manager.get_job.return_value = mock_job
|
||||
mock_job_manager.update_state_data = MagicMock()
|
||||
|
||||
mock_storage = MagicMock()
|
||||
mock_logger = MagicMock()
|
||||
|
||||
from backend.workers.render_video_worker import _analyze_backing_vocals
|
||||
|
||||
# Should not raise when stems not found, just log warning and return
|
||||
await _analyze_backing_vocals(
|
||||
"test123", mock_job_manager, mock_storage, mock_logger
|
||||
)
|
||||
|
||||
# Should log warning but not update state (early return)
|
||||
mock_logger.warning.assert_called()
|
||||
# No state_data update since we return early
|
||||
mock_job_manager.update_state_data.assert_not_called()
|
||||
|
||||
def test_analysis_service_can_be_instantiated(self):
|
||||
"""Test that AudioAnalysisService can be instantiated."""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
mock_storage = MagicMock()
|
||||
service = AudioAnalysisService(storage_service=mock_storage)
|
||||
|
||||
assert service is not None
|
||||
assert service.storage_service == mock_storage
|
||||
|
||||
@@ -19,7 +19,7 @@ from backend.models.job import JobStatus
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.config import get_settings
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
|
||||
|
||||
# Import from karaoke_gen package
|
||||
from karaoke_gen.audio_processor import AudioProcessor
|
||||
@@ -34,7 +34,7 @@ AUDIO_WORKER_LOGGERS = [
|
||||
]
|
||||
|
||||
|
||||
async def download_from_url(url: str, temp_dir: str, artist: str, title: str) -> Optional[str]:
|
||||
async def download_from_url(url: str, temp_dir: str, artist: str, title: str, job_manager: JobManager = None, job_id: str = None) -> Optional[str]:
|
||||
"""
|
||||
Download audio from a URL (YouTube, etc.) using karaoke_gen.FileHandler.
|
||||
|
||||
@@ -43,11 +43,16 @@ async def download_from_url(url: str, temp_dir: str, artist: str, title: str) ->
|
||||
- Cookie support for authenticated downloads
|
||||
- Retry logic
|
||||
|
||||
If artist and/or title are not provided, attempts to extract them from
|
||||
the URL metadata.
|
||||
|
||||
Args:
|
||||
url: URL to download from
|
||||
temp_dir: Temporary directory to save to
|
||||
artist: Artist name for filename
|
||||
title: Song title for filename
|
||||
artist: Artist name for filename (can be None for auto-detection)
|
||||
title: Song title for filename (can be None for auto-detection)
|
||||
job_manager: Optional JobManager to update job with detected metadata
|
||||
job_id: Optional job ID to update
|
||||
|
||||
Returns:
|
||||
Path to downloaded audio file, or None if failed
|
||||
@@ -64,6 +69,34 @@ async def download_from_url(url: str, temp_dir: str, artist: str, title: str) ->
|
||||
dry_run=False
|
||||
)
|
||||
|
||||
# Try to extract metadata if artist or title not provided
|
||||
if not artist or not title:
|
||||
logger.info(f"Extracting metadata from URL: {url}")
|
||||
metadata = file_handler.extract_metadata_from_url(url)
|
||||
|
||||
if metadata:
|
||||
if not artist:
|
||||
artist = metadata.get('artist', 'Unknown')
|
||||
logger.info(f"Auto-detected artist: {artist}")
|
||||
if not title:
|
||||
title = metadata.get('title', 'Unknown')
|
||||
logger.info(f"Auto-detected title: {title}")
|
||||
|
||||
# Update job with detected metadata if job_manager provided
|
||||
if job_manager and job_id:
|
||||
update_data = {}
|
||||
if artist:
|
||||
update_data['artist'] = artist
|
||||
if title:
|
||||
update_data['title'] = title
|
||||
if update_data:
|
||||
job_manager.update_job(job_id, update_data)
|
||||
logger.info(f"Updated job {job_id} with detected metadata")
|
||||
else:
|
||||
logger.warning("Could not extract metadata from URL, using defaults")
|
||||
artist = artist or "Unknown"
|
||||
title = title or "Unknown"
|
||||
|
||||
# Create output filename (without extension)
|
||||
safe_artist = sanitize_filename(artist) if artist else "Unknown"
|
||||
safe_title = sanitize_filename(title) if title else "Unknown"
|
||||
@@ -112,7 +145,12 @@ async def download_from_url(url: str, temp_dir: str, artist: str, title: str) ->
|
||||
return None
|
||||
|
||||
|
||||
def create_audio_processor(temp_dir: str) -> AudioProcessor:
|
||||
def create_audio_processor(
|
||||
temp_dir: str,
|
||||
clean_instrumental_model: Optional[str] = None,
|
||||
backing_vocals_models: Optional[list] = None,
|
||||
other_stems_models: Optional[list] = None
|
||||
) -> AudioProcessor:
|
||||
"""
|
||||
Create an AudioProcessor instance configured for remote API processing.
|
||||
|
||||
@@ -120,10 +158,13 @@ def create_audio_processor(temp_dir: str) -> AudioProcessor:
|
||||
- Uses remote Modal API (via AUDIO_SEPARATOR_API_URL env var)
|
||||
- No local models needed (model_file_dir=None)
|
||||
- FLAC output format for quality
|
||||
- Standard model configurations from karaoke-gen CLI
|
||||
- Model configurations from job or CLI defaults
|
||||
|
||||
Args:
|
||||
temp_dir: Temporary directory for processing
|
||||
clean_instrumental_model: Model for clean instrumental separation (optional, uses default if not provided)
|
||||
backing_vocals_models: List of models for backing vocals separation (optional, uses default if not provided)
|
||||
other_stems_models: List of models for other stems separation (optional, uses default if not provided)
|
||||
|
||||
Returns:
|
||||
Configured AudioProcessor instance
|
||||
@@ -132,10 +173,10 @@ def create_audio_processor(temp_dir: str) -> AudioProcessor:
|
||||
audio_logger = logging.getLogger("karaoke_gen.audio_processor")
|
||||
audio_logger.setLevel(logging.INFO)
|
||||
|
||||
# Model configurations (same as CLI defaults)
|
||||
clean_instrumental_model = "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
backing_vocals_models = ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
other_stems_models = ["htdemucs_6s.yaml"] # For 6-stem separation (bass, drums, etc.)
|
||||
# Model configurations - use provided values or CLI defaults
|
||||
effective_clean_model = clean_instrumental_model or "model_bs_roformer_ep_317_sdr_12.9755.ckpt"
|
||||
effective_backing_models = backing_vocals_models or ["mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt"]
|
||||
effective_other_models = other_stems_models or ["htdemucs_6s.yaml"] # For 6-stem separation (bass, drums, etc.)
|
||||
|
||||
# FFmpeg command for combining audio files (must be a string, not a list)
|
||||
ffmpeg_base_command = "ffmpeg -hide_banner -loglevel error -nostats -y"
|
||||
@@ -146,9 +187,9 @@ def create_audio_processor(temp_dir: str) -> AudioProcessor:
|
||||
log_formatter=None, # Not needed for our use case
|
||||
model_file_dir=None, # No local models, using remote API
|
||||
lossless_output_format="FLAC",
|
||||
clean_instrumental_model=clean_instrumental_model,
|
||||
backing_vocals_models=backing_vocals_models,
|
||||
other_stems_models=other_stems_models,
|
||||
clean_instrumental_model=effective_clean_model,
|
||||
backing_vocals_models=effective_backing_models,
|
||||
other_stems_models=effective_other_models,
|
||||
ffmpeg_base_command=ffmpeg_base_command
|
||||
)
|
||||
|
||||
@@ -200,71 +241,88 @@ async def process_audio_separation(job_id: str) -> bool:
|
||||
job_log.info(f"Created temp directory: {temp_dir}")
|
||||
|
||||
try:
|
||||
job_log.info(f"Starting audio separation for {job.artist} - {job.title}")
|
||||
logger.info(f"Starting audio separation for job {job_id}")
|
||||
|
||||
# Ensure AUDIO_SEPARATOR_API_URL is set
|
||||
api_url = os.environ.get("AUDIO_SEPARATOR_API_URL")
|
||||
if not api_url:
|
||||
raise Exception("AUDIO_SEPARATOR_API_URL environment variable not set. "
|
||||
"Cannot perform audio separation without remote API access.")
|
||||
job_log.info(f"Audio separator API: {api_url}")
|
||||
|
||||
# Download audio file from GCS
|
||||
job_log.info("Downloading audio file from GCS...")
|
||||
audio_path = await download_audio(job_id, temp_dir, storage, job)
|
||||
if not audio_path:
|
||||
raise Exception("Failed to download audio file")
|
||||
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
# The status is managed at a higher level - workers just track their progress
|
||||
job_manager.update_state_data(job_id, 'audio_progress', {
|
||||
'stage': 'separating_stage1',
|
||||
'progress': 10,
|
||||
'message': 'Starting audio separation (Stage 1: Clean instrumental)'
|
||||
})
|
||||
|
||||
# Create AudioProcessor instance (reuses karaoke_gen code)
|
||||
job_log.info("Creating AudioProcessor instance...")
|
||||
audio_processor = create_audio_processor(temp_dir)
|
||||
|
||||
# Format artist-title for file naming (matches CLI behavior)
|
||||
artist_title = f"{job.artist} - {job.title}"
|
||||
|
||||
# Run audio separation (calls Modal API internally)
|
||||
# This returns a dict with paths to all separated stems
|
||||
job_log.info("Starting audio separation (this may take 5-10 minutes)...")
|
||||
job_log.info(" Stage 1: Clean instrumental separation (MDX models)")
|
||||
job_log.info(" Stage 2: Backing vocals separation (Demucs model)")
|
||||
logger.info(f"Job {job_id}: Calling audio_processor.process_audio_separation()")
|
||||
separation_result = audio_processor.process_audio_separation(
|
||||
audio_file=audio_path,
|
||||
artist_title=artist_title,
|
||||
track_output_dir=temp_dir
|
||||
)
|
||||
|
||||
job_log.info("Audio separation complete!")
|
||||
job_log.info(f" Generated {len(separation_result)} stem files")
|
||||
logger.info(f"Job {job_id}: Audio separation complete, organizing results")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
job_manager.update_state_data(job_id, 'audio_progress', {
|
||||
'stage': 'audio_complete',
|
||||
'progress': 45,
|
||||
'message': 'Audio separation complete, uploading stems'
|
||||
})
|
||||
|
||||
# Upload all stems to GCS
|
||||
await upload_separation_results(job_id, separation_result, storage, job_manager)
|
||||
|
||||
logger.info(f"Job {job_id}: All stems uploaded successfully")
|
||||
|
||||
# Mark audio processing complete
|
||||
# This will check if lyrics are also complete and transition to next stage if so
|
||||
job_manager.mark_audio_complete(job_id)
|
||||
|
||||
return True
|
||||
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
|
||||
# This ensures logs from third-party libraries (karaoke_gen.audio_processor) are
|
||||
# only captured by this job's handler, not handlers from other concurrent jobs
|
||||
with job_logging_context(job_id):
|
||||
job_log.info(f"Starting audio separation for {job.artist} - {job.title}")
|
||||
logger.info(f"Starting audio separation for job {job_id}")
|
||||
|
||||
# Ensure AUDIO_SEPARATOR_API_URL is set
|
||||
api_url = os.environ.get("AUDIO_SEPARATOR_API_URL")
|
||||
if not api_url:
|
||||
raise Exception("AUDIO_SEPARATOR_API_URL environment variable not set. "
|
||||
"Cannot perform audio separation without remote API access.")
|
||||
job_log.info(f"Audio separator API: {api_url}")
|
||||
|
||||
# Download audio file from GCS or URL
|
||||
job_log.info("Downloading audio file...")
|
||||
audio_path = await download_audio(job_id, temp_dir, storage, job, job_manager_instance=job_manager)
|
||||
if not audio_path:
|
||||
raise Exception("Failed to download audio file")
|
||||
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
# The status is managed at a higher level - workers just track their progress
|
||||
job_manager.update_state_data(job_id, 'audio_progress', {
|
||||
'stage': 'separating_stage1',
|
||||
'progress': 10,
|
||||
'message': 'Starting audio separation (Stage 1: Clean instrumental)'
|
||||
})
|
||||
|
||||
# Create AudioProcessor instance (reuses karaoke_gen code)
|
||||
# Use model configuration from job if provided, otherwise use defaults
|
||||
job_log.info("Creating AudioProcessor instance...")
|
||||
if job.clean_instrumental_model:
|
||||
job_log.info(f" Using clean instrumental model: {job.clean_instrumental_model}")
|
||||
if job.backing_vocals_models:
|
||||
job_log.info(f" Using backing vocals models: {job.backing_vocals_models}")
|
||||
if job.other_stems_models:
|
||||
job_log.info(f" Using other stems models: {job.other_stems_models}")
|
||||
|
||||
audio_processor = create_audio_processor(
|
||||
temp_dir,
|
||||
clean_instrumental_model=job.clean_instrumental_model,
|
||||
backing_vocals_models=job.backing_vocals_models,
|
||||
other_stems_models=job.other_stems_models
|
||||
)
|
||||
|
||||
# Format artist-title for file naming (matches CLI behavior)
|
||||
artist_title = f"{job.artist} - {job.title}"
|
||||
|
||||
# Run audio separation (calls Modal API internally)
|
||||
# This returns a dict with paths to all separated stems
|
||||
job_log.info("Starting audio separation (this may take 5-10 minutes)...")
|
||||
job_log.info(" Stage 1: Clean instrumental separation (MDX models)")
|
||||
job_log.info(" Stage 2: Backing vocals separation (Demucs model)")
|
||||
logger.info(f"Job {job_id}: Calling audio_processor.process_audio_separation()")
|
||||
separation_result = audio_processor.process_audio_separation(
|
||||
audio_file=audio_path,
|
||||
artist_title=artist_title,
|
||||
track_output_dir=temp_dir
|
||||
)
|
||||
|
||||
job_log.info("Audio separation complete!")
|
||||
job_log.info(f" Generated {len(separation_result)} stem files")
|
||||
logger.info(f"Job {job_id}: Audio separation complete, organizing results")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
job_manager.update_state_data(job_id, 'audio_progress', {
|
||||
'stage': 'audio_complete',
|
||||
'progress': 45,
|
||||
'message': 'Audio separation complete, uploading stems'
|
||||
})
|
||||
|
||||
# Upload all stems to GCS
|
||||
await upload_separation_results(job_id, separation_result, storage, job_manager)
|
||||
|
||||
logger.info(f"Job {job_id}: All stems uploaded successfully")
|
||||
|
||||
# Mark audio processing complete
|
||||
# This will check if lyrics are also complete and transition to next stage if so
|
||||
job_manager.mark_audio_complete(job_id)
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Job {job_id}: Audio separation failed: {e}", exc_info=True)
|
||||
@@ -293,7 +351,8 @@ async def download_audio(
|
||||
job_id: str,
|
||||
temp_dir: str,
|
||||
storage: StorageService,
|
||||
job
|
||||
job,
|
||||
job_manager_instance: JobManager = None
|
||||
) -> Optional[str]:
|
||||
"""
|
||||
Download or fetch audio file to local temp directory.
|
||||
@@ -302,6 +361,13 @@ async def download_audio(
|
||||
1. Uploaded file: Download from GCS using input_media_gcs_path
|
||||
2. URL (YouTube, etc.): Download using yt-dlp or other tools
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
temp_dir: Temporary directory to save to
|
||||
storage: StorageService instance
|
||||
job: Job object with URL or GCS path
|
||||
job_manager_instance: Optional JobManager instance for updating job metadata
|
||||
|
||||
Returns:
|
||||
Path to downloaded audio file, or None if failed
|
||||
"""
|
||||
@@ -326,7 +392,18 @@ async def download_audio(
|
||||
# Case 3: Fresh URL that needs downloading
|
||||
if job.url:
|
||||
logger.info(f"Job {job_id}: Downloading from URL: {job.url}")
|
||||
local_path = await download_from_url(job.url, temp_dir, job.artist, job.title)
|
||||
|
||||
# Use provided job_manager or create new one
|
||||
jm = job_manager_instance or JobManager()
|
||||
|
||||
local_path = await download_from_url(
|
||||
job.url,
|
||||
temp_dir,
|
||||
job.artist,
|
||||
job.title,
|
||||
job_manager=jm,
|
||||
job_id=job_id
|
||||
)
|
||||
|
||||
if local_path and os.path.exists(local_path):
|
||||
# Upload to GCS and update job
|
||||
@@ -334,9 +411,8 @@ async def download_audio(
|
||||
url = storage.upload_file(local_path, gcs_path)
|
||||
|
||||
# Update job with GCS path for lyrics worker
|
||||
job_manager = JobManager()
|
||||
job_manager.firestore.update_job(job_id, {'input_media_gcs_path': gcs_path})
|
||||
job_manager.update_file_url(job_id, 'input', 'audio', url)
|
||||
jm.update_job(job_id, {'input_media_gcs_path': gcs_path})
|
||||
jm.update_file_url(job_id, 'input', 'audio', url)
|
||||
|
||||
logger.info(f"Job {job_id}: Downloaded and uploaded audio to GCS: {gcs_path}")
|
||||
return local_path
|
||||
|
||||
@@ -26,7 +26,7 @@ from backend.models.job import JobStatus
|
||||
from karaoke_gen.utils import sanitize_filename
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
|
||||
from backend.workers.style_helper import load_style_config
|
||||
|
||||
# Import from karaoke_gen package
|
||||
@@ -36,27 +36,21 @@ from karaoke_gen.lyrics_processor import LyricsProcessor
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Loggers to capture for lyrics worker (includes LyricsTranscriber internals)
|
||||
# Loggers to capture for lyrics worker (top-level package loggers only)
|
||||
# Note: We only capture at top-level package loggers to avoid duplicate log entries.
|
||||
# Logs from sub-modules (e.g., lyrics_transcriber.correction.anchor_sequence) will
|
||||
# propagate up to their parent logger (lyrics_transcriber) where they get captured.
|
||||
LYRICS_WORKER_LOGGERS = [
|
||||
"karaoke_gen.lyrics_processor",
|
||||
"lyrics_transcriber",
|
||||
"lyrics_transcriber.core",
|
||||
"lyrics_transcriber.core.controller",
|
||||
"lyrics_transcriber.lyrics",
|
||||
"lyrics_transcriber.lyrics.genius",
|
||||
"lyrics_transcriber.lyrics.spotify",
|
||||
"lyrics_transcriber.lyrics.musixmatch",
|
||||
"lyrics_transcriber.lyrics.lrclib",
|
||||
"lyrics_transcriber.correction",
|
||||
"lyrics_transcriber.correction.corrector",
|
||||
"lyrics_transcriber.correction.anchor_sequence",
|
||||
"lyrics_transcriber.transcribers",
|
||||
"lyrics_transcriber.transcribers.audioshake",
|
||||
"lyrics_transcriber.output",
|
||||
]
|
||||
|
||||
|
||||
def create_lyrics_processor(style_params_json: Optional[str] = None) -> LyricsProcessor:
|
||||
def create_lyrics_processor(
|
||||
style_params_json: Optional[str] = None,
|
||||
lyrics_file: Optional[str] = None,
|
||||
subtitle_offset_ms: int = 0
|
||||
) -> LyricsProcessor:
|
||||
"""
|
||||
Create a LyricsProcessor instance configured for Cloud Run processing.
|
||||
|
||||
@@ -68,6 +62,8 @@ def create_lyrics_processor(style_params_json: Optional[str] = None) -> LyricsPr
|
||||
|
||||
Args:
|
||||
style_params_json: Optional path to style parameters JSON file
|
||||
lyrics_file: Optional path to user-provided lyrics file (overrides API fetch)
|
||||
subtitle_offset_ms: Offset for subtitle timing in milliseconds
|
||||
|
||||
Returns:
|
||||
Configured LyricsProcessor instance
|
||||
@@ -79,11 +75,11 @@ def create_lyrics_processor(style_params_json: Optional[str] = None) -> LyricsPr
|
||||
return LyricsProcessor(
|
||||
logger=lyrics_logger,
|
||||
style_params_json=style_params_json,
|
||||
lyrics_file=None, # Will fetch from APIs
|
||||
lyrics_file=lyrics_file, # Use user-provided lyrics if available
|
||||
skip_transcription=False, # We want transcription
|
||||
skip_transcription_review=True, # Skip interactive review (use React UI instead)
|
||||
render_video=False, # Skip video generation for now (will be done after review)
|
||||
subtitle_offset_ms=0 # No offset by default
|
||||
subtitle_offset_ms=subtitle_offset_ms
|
||||
)
|
||||
|
||||
|
||||
@@ -131,126 +127,162 @@ async def process_lyrics_transcription(job_id: str) -> bool:
|
||||
temp_dir = None
|
||||
|
||||
try:
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
logger.error(f"Job {job_id} not found")
|
||||
job_log.error(f"Job {job_id} not found in Firestore!")
|
||||
return False
|
||||
|
||||
# Create temporary working directory
|
||||
temp_dir = tempfile.mkdtemp(prefix=f"karaoke_lyrics_{job_id}_")
|
||||
job_log.info(f"Created temp directory: {temp_dir}")
|
||||
job_log.info(f"Starting lyrics transcription for {job.artist} - {job.title}")
|
||||
job_log.info(f"Log capture enabled for: {', '.join(LYRICS_WORKER_LOGGERS)}")
|
||||
logger.info(f"Starting lyrics transcription for job {job_id}")
|
||||
|
||||
# Log environment configuration
|
||||
job_log.info("Checking API configuration...")
|
||||
if os.environ.get("GENIUS_API_TOKEN"):
|
||||
job_log.info(" Genius API: configured")
|
||||
else:
|
||||
job_log.warning(" Genius API: NOT configured")
|
||||
if os.environ.get("SPOTIFY_COOKIE_SP_DC"):
|
||||
job_log.info(" Spotify: configured")
|
||||
else:
|
||||
job_log.warning(" Spotify: NOT configured")
|
||||
if os.environ.get("RAPIDAPI_KEY"):
|
||||
job_log.info(" Musixmatch (RapidAPI): configured")
|
||||
else:
|
||||
job_log.warning(" Musixmatch (RapidAPI): NOT configured")
|
||||
|
||||
# Ensure required environment variables are set
|
||||
if not os.environ.get("AUDIOSHAKE_API_TOKEN"):
|
||||
job_log.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
|
||||
logger.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
|
||||
|
||||
# Download audio file from GCS (waits for audio worker if URL job)
|
||||
job_log.info("Downloading audio file from GCS...")
|
||||
audio_path = await download_audio(job_id, temp_dir, storage, job, job_manager)
|
||||
if not audio_path:
|
||||
raise Exception("Failed to download audio file")
|
||||
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
# The status is managed at a higher level - workers just track their progress
|
||||
job_manager.update_state_data(job_id, 'lyrics_progress', {
|
||||
'stage': 'transcribing',
|
||||
'progress': 10,
|
||||
'message': 'Starting lyrics transcription via AudioShake'
|
||||
})
|
||||
|
||||
# Load style configuration (downloads assets from GCS if available)
|
||||
# This is needed for max_line_length and other style settings in LyricsTranscriber
|
||||
job_log.info("Loading style configuration for lyrics processing...")
|
||||
style_config = await load_style_config(job, storage, temp_dir)
|
||||
style_params_json_path = style_config.get_style_params_path() if style_config.has_custom_styles() else None
|
||||
|
||||
if style_params_json_path:
|
||||
job_log.info(f"Using custom style params: {style_params_json_path}")
|
||||
# Log the contents of the style params for debugging
|
||||
try:
|
||||
with open(style_params_json_path, 'r') as f:
|
||||
style_content = json.load(f)
|
||||
job_log.info(f"Style params sections: {list(style_content.keys())}")
|
||||
if 'karaoke' in style_content:
|
||||
karaoke_style = style_content['karaoke']
|
||||
job_log.info(f" karaoke.background_image: {karaoke_style.get('background_image', 'NOT SET')}")
|
||||
job_log.info(f" karaoke.font_path: {karaoke_style.get('font_path', 'NOT SET')}")
|
||||
except Exception as e:
|
||||
job_log.warning(f"Could not read style params for logging: {e}")
|
||||
else:
|
||||
job_log.info("No custom style params found, using defaults")
|
||||
|
||||
# Create LyricsProcessor instance (reuses karaoke_gen code)
|
||||
job_log.info("Creating LyricsProcessor instance...")
|
||||
job_log.info(f" style_params_json: {style_params_json_path}")
|
||||
lyrics_processor = create_lyrics_processor(style_params_json=style_params_json_path)
|
||||
|
||||
# Run transcription + correction
|
||||
# This will:
|
||||
# 1. Fetch reference lyrics from Genius/Spotify/Musixmatch/LRCLib
|
||||
# 2. Transcribe audio via AudioShake API
|
||||
# 3. Run automatic correction with lyrics_transcriber
|
||||
# 4. Generate corrections JSON (no interactive review)
|
||||
# 5. Skip video generation (render_video=False)
|
||||
job_log.info("Starting LyricsTranscriber processing...")
|
||||
job_log.info(f" Artist: {job.artist}")
|
||||
job_log.info(f" Title: {job.title}")
|
||||
logger.info(f"Job {job_id}: Calling lyrics_processor.transcribe_lyrics()")
|
||||
|
||||
# Run transcription in thread pool to avoid blocking the event loop
|
||||
# (transcribe_lyrics is synchronous and takes 1-2 minutes)
|
||||
result = await asyncio.to_thread(
|
||||
lyrics_processor.transcribe_lyrics,
|
||||
input_audio_wav=audio_path,
|
||||
artist=job.artist,
|
||||
title=job.title,
|
||||
track_output_dir=temp_dir
|
||||
)
|
||||
|
||||
job_log.info("Transcription processing complete")
|
||||
logger.info(f"Job {job_id}: Transcription complete, uploading results")
|
||||
|
||||
# Upload lyrics results to GCS
|
||||
job_log.info("Uploading lyrics results to GCS...")
|
||||
await upload_lyrics_results(job_id, temp_dir, result, storage, job_manager, job_log)
|
||||
|
||||
job_log.info("All lyrics data uploaded successfully")
|
||||
logger.info(f"Job {job_id}: All lyrics data uploaded successfully")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
job_manager.update_state_data(job_id, 'lyrics_progress', {
|
||||
'stage': 'lyrics_complete',
|
||||
'progress': 45,
|
||||
'message': 'Lyrics transcription complete'
|
||||
})
|
||||
|
||||
# Mark lyrics processing complete
|
||||
# This will check if audio is also complete and transition to next stage if so
|
||||
job_log.info("Lyrics worker complete, checking if audio is also done...")
|
||||
job_manager.mark_lyrics_complete(job_id)
|
||||
|
||||
return True
|
||||
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
|
||||
# This ensures logs from third-party libraries (lyrics_transcriber) are only captured
|
||||
# by this job's handler, not handlers from other concurrent jobs
|
||||
with job_logging_context(job_id):
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
logger.error(f"Job {job_id} not found")
|
||||
job_log.error(f"Job {job_id} not found in Firestore!")
|
||||
return False
|
||||
|
||||
# Create temporary working directory
|
||||
temp_dir = tempfile.mkdtemp(prefix=f"karaoke_lyrics_{job_id}_")
|
||||
job_log.info(f"Created temp directory: {temp_dir}")
|
||||
job_log.info(f"Starting lyrics transcription for {job.artist} - {job.title}")
|
||||
job_log.info(f"Log capture enabled for: {', '.join(LYRICS_WORKER_LOGGERS)}")
|
||||
logger.info(f"Starting lyrics transcription for job {job_id}")
|
||||
|
||||
# Log environment configuration
|
||||
job_log.info("Checking API configuration...")
|
||||
if os.environ.get("GENIUS_API_TOKEN"):
|
||||
job_log.info(" Genius API: configured")
|
||||
else:
|
||||
job_log.warning(" Genius API: NOT configured")
|
||||
if os.environ.get("SPOTIFY_COOKIE_SP_DC"):
|
||||
job_log.info(" Spotify: configured")
|
||||
else:
|
||||
job_log.warning(" Spotify: NOT configured")
|
||||
if os.environ.get("RAPIDAPI_KEY"):
|
||||
job_log.info(" Musixmatch (RapidAPI): configured")
|
||||
else:
|
||||
job_log.warning(" Musixmatch (RapidAPI): NOT configured")
|
||||
|
||||
# Ensure required environment variables are set
|
||||
if not os.environ.get("AUDIOSHAKE_API_TOKEN"):
|
||||
job_log.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
|
||||
logger.warning("AUDIOSHAKE_API_TOKEN not set - transcription may fail")
|
||||
|
||||
# Download audio file from GCS (waits for audio worker if URL job)
|
||||
job_log.info("Downloading audio file from GCS...")
|
||||
audio_path = await download_audio(job_id, temp_dir, storage, job, job_manager)
|
||||
if not audio_path:
|
||||
raise Exception("Failed to download audio file")
|
||||
job_log.info(f"Audio downloaded: {os.path.basename(audio_path)}")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
# The status is managed at a higher level - workers just track their progress
|
||||
job_manager.update_state_data(job_id, 'lyrics_progress', {
|
||||
'stage': 'transcribing',
|
||||
'progress': 10,
|
||||
'message': 'Starting lyrics transcription via AudioShake'
|
||||
})
|
||||
|
||||
# Load style configuration (downloads assets from GCS if available)
|
||||
# This is needed for max_line_length and other style settings in LyricsTranscriber
|
||||
job_log.info("Loading style configuration for lyrics processing...")
|
||||
style_config = await load_style_config(job, storage, temp_dir)
|
||||
style_params_json_path = style_config.get_style_params_path() if style_config.has_custom_styles() else None
|
||||
|
||||
if style_params_json_path:
|
||||
job_log.info(f"Using custom style params: {style_params_json_path}")
|
||||
# Log the contents of the style params for debugging
|
||||
try:
|
||||
with open(style_params_json_path, 'r') as f:
|
||||
style_content = json.load(f)
|
||||
job_log.info(f"Style params sections: {list(style_content.keys())}")
|
||||
if 'karaoke' in style_content:
|
||||
karaoke_style = style_content['karaoke']
|
||||
job_log.info(f" karaoke.background_image: {karaoke_style.get('background_image', 'NOT SET')}")
|
||||
job_log.info(f" karaoke.font_path: {karaoke_style.get('font_path', 'NOT SET')}")
|
||||
except Exception as e:
|
||||
job_log.warning(f"Could not read style params for logging: {e}")
|
||||
else:
|
||||
job_log.info("No custom style params found, using defaults")
|
||||
|
||||
# Download user-provided lyrics file if available
|
||||
lyrics_file_path = None
|
||||
if hasattr(job, 'lyrics_file_gcs_path') and job.lyrics_file_gcs_path:
|
||||
lyrics_file_path = os.path.join(temp_dir, "user_lyrics.txt")
|
||||
job_log.info(f"Downloading user-provided lyrics file: {job.lyrics_file_gcs_path}")
|
||||
storage.download_file(job.lyrics_file_gcs_path, lyrics_file_path)
|
||||
job_log.info(f"User lyrics file downloaded to: {lyrics_file_path}")
|
||||
|
||||
# Get lyrics configuration from job
|
||||
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
|
||||
if subtitle_offset != 0:
|
||||
job_log.info(f"Subtitle offset: {subtitle_offset}ms")
|
||||
|
||||
# Create LyricsProcessor instance (reuses karaoke_gen code)
|
||||
job_log.info("Creating LyricsProcessor instance...")
|
||||
job_log.info(f" style_params_json: {style_params_json_path}")
|
||||
job_log.info(f" lyrics_file: {lyrics_file_path}")
|
||||
job_log.info(f" subtitle_offset_ms: {subtitle_offset}")
|
||||
lyrics_processor = create_lyrics_processor(
|
||||
style_params_json=style_params_json_path,
|
||||
lyrics_file=lyrics_file_path,
|
||||
subtitle_offset_ms=subtitle_offset
|
||||
)
|
||||
|
||||
# Run transcription + correction
|
||||
# This will:
|
||||
# 1. Fetch reference lyrics from Genius/Spotify/Musixmatch/LRCLib
|
||||
# 2. Transcribe audio via AudioShake API
|
||||
# 3. Run automatic correction with lyrics_transcriber
|
||||
# 4. Generate corrections JSON (no interactive review)
|
||||
# 5. Skip video generation (render_video=False)
|
||||
|
||||
# Use lyrics_artist/lyrics_title overrides if provided, else fall back to job artist/title
|
||||
lyrics_search_artist = getattr(job, 'lyrics_artist', None) or job.artist
|
||||
lyrics_search_title = getattr(job, 'lyrics_title', None) or job.title
|
||||
|
||||
job_log.info("Starting LyricsTranscriber processing...")
|
||||
job_log.info(f" Artist: {job.artist}")
|
||||
job_log.info(f" Title: {job.title}")
|
||||
if lyrics_search_artist != job.artist:
|
||||
job_log.info(f" Lyrics search artist override: {lyrics_search_artist}")
|
||||
if lyrics_search_title != job.title:
|
||||
job_log.info(f" Lyrics search title override: {lyrics_search_title}")
|
||||
logger.info(f"Job {job_id}: Calling lyrics_processor.transcribe_lyrics()")
|
||||
|
||||
# Run transcription in thread pool to avoid blocking the event loop
|
||||
# (transcribe_lyrics is synchronous and takes 1-2 minutes)
|
||||
# Note: We pass the original artist/title for file naming, but the LyricsProcessor
|
||||
# will use lyrics_artist/lyrics_title for the search (set via constructor args)
|
||||
result = await asyncio.to_thread(
|
||||
lyrics_processor.transcribe_lyrics,
|
||||
input_audio_wav=audio_path,
|
||||
artist=job.artist, # Original artist for file naming
|
||||
title=job.title, # Original title for file naming
|
||||
track_output_dir=temp_dir,
|
||||
lyrics_artist=lyrics_search_artist, # Override for lyrics search
|
||||
lyrics_title=lyrics_search_title # Override for lyrics search
|
||||
)
|
||||
|
||||
job_log.info("Transcription processing complete")
|
||||
logger.info(f"Job {job_id}: Transcription complete, uploading results")
|
||||
|
||||
# Upload lyrics results to GCS
|
||||
job_log.info("Uploading lyrics results to GCS...")
|
||||
await upload_lyrics_results(job_id, temp_dir, result, storage, job_manager, job_log)
|
||||
|
||||
job_log.info("All lyrics data uploaded successfully")
|
||||
logger.info(f"Job {job_id}: All lyrics data uploaded successfully")
|
||||
|
||||
# Update progress using state_data (don't change status during parallel processing)
|
||||
job_manager.update_state_data(job_id, 'lyrics_progress', {
|
||||
'stage': 'lyrics_complete',
|
||||
'progress': 45,
|
||||
'message': 'Lyrics transcription complete'
|
||||
})
|
||||
|
||||
# Mark lyrics processing complete
|
||||
# This will check if audio is also complete and transition to next stage if so
|
||||
job_log.info("Lyrics worker complete, checking if audio is also done...")
|
||||
job_manager.mark_lyrics_complete(job_id)
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
job_log.error(f"Lyrics transcription failed: {str(e)}", exc_info=True)
|
||||
@@ -415,11 +447,24 @@ async def upload_lyrics_results(
|
||||
corrections_data = json.load(f)
|
||||
|
||||
# Store metadata in state_data
|
||||
job_manager.update_state_data(job_id, 'lyrics_metadata', {
|
||||
# Include countdown padding info for instrumental synchronization
|
||||
lyrics_metadata = {
|
||||
'segment_count': len(corrections_data.get('corrected_segments', [])),
|
||||
'has_corrections': True,
|
||||
'ready_for_review': True
|
||||
})
|
||||
'ready_for_review': True,
|
||||
}
|
||||
|
||||
# Add countdown padding info from transcription result
|
||||
# This is needed by video_worker to pad instrumentals for sync
|
||||
if transcription_result.get("countdown_padding_added"):
|
||||
lyrics_metadata['has_countdown_padding'] = True
|
||||
lyrics_metadata['countdown_padding_seconds'] = transcription_result.get("countdown_padding_seconds", 3.0)
|
||||
logger.info(f"Job {job_id}: Countdown padding detected: {lyrics_metadata['countdown_padding_seconds']}s")
|
||||
else:
|
||||
lyrics_metadata['has_countdown_padding'] = False
|
||||
lyrics_metadata['countdown_padding_seconds'] = 0.0
|
||||
|
||||
job_manager.update_state_data(job_id, 'lyrics_metadata', lyrics_metadata)
|
||||
except Exception as e:
|
||||
logger.warning(f"Job {job_id}: Could not parse corrections JSON: {e}")
|
||||
else:
|
||||
|
||||
@@ -26,7 +26,7 @@ from backend.models.job import JobStatus
|
||||
from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.config import get_settings
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
|
||||
|
||||
# Import from lyrics_transcriber (submodule)
|
||||
from lyrics_transcriber.output.generator import OutputGenerator
|
||||
@@ -85,189 +85,213 @@ async def process_render_video(job_id: str) -> bool:
|
||||
logger.info(f"Job {job_id}: Starting video render (post-review)")
|
||||
|
||||
try:
|
||||
# Transition to RENDERING_VIDEO
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.RENDERING_VIDEO,
|
||||
progress=75,
|
||||
message="Rendering karaoke video with corrected lyrics"
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
job_log.info(f"Created temp directory: {temp_dir}")
|
||||
# 1. Download corrected corrections data
|
||||
corrections_path = os.path.join(temp_dir, "corrections.json")
|
||||
|
||||
# Try updated corrections first (from human review), fall back to original
|
||||
corrections_gcs_updated = f"jobs/{job_id}/lyrics/corrections_updated.json"
|
||||
corrections_gcs_original = f"jobs/{job_id}/lyrics/corrections.json"
|
||||
|
||||
# Get GCS path from file_urls
|
||||
corrections_url = job.file_urls.get('lyrics', {}).get('corrections_updated')
|
||||
if not corrections_url:
|
||||
corrections_url = job.file_urls.get('lyrics', {}).get('corrections')
|
||||
|
||||
if not corrections_url:
|
||||
# Try direct GCS paths
|
||||
if storage.file_exists(corrections_gcs_updated):
|
||||
corrections_gcs = corrections_gcs_updated
|
||||
elif storage.file_exists(corrections_gcs_original):
|
||||
corrections_gcs = corrections_gcs_original
|
||||
else:
|
||||
raise FileNotFoundError(f"No corrections file found for job {job_id}")
|
||||
else:
|
||||
# Extract GCS path from URL
|
||||
corrections_gcs = _extract_gcs_path(corrections_url)
|
||||
|
||||
# 2. Load the ORIGINAL corrections (has full structure)
|
||||
original_corrections_gcs = f"jobs/{job_id}/lyrics/corrections.json"
|
||||
original_corrections_path = os.path.join(temp_dir, "corrections_original.json")
|
||||
|
||||
job_log.info(f"Downloading original corrections from {original_corrections_gcs}")
|
||||
logger.info(f"Job {job_id}: Downloading original corrections from {original_corrections_gcs}")
|
||||
storage.download_file(original_corrections_gcs, original_corrections_path)
|
||||
|
||||
with open(original_corrections_path, 'r', encoding='utf-8') as f:
|
||||
original_data = json.load(f)
|
||||
|
||||
# 3. Check if there are updated corrections (from review UI)
|
||||
# The frontend sends only partial data: {corrections, corrected_segments}
|
||||
updated_corrections_gcs = f"jobs/{job_id}/lyrics/corrections_updated.json"
|
||||
|
||||
if storage.file_exists(updated_corrections_gcs):
|
||||
job_log.info("Found updated corrections from review, merging")
|
||||
logger.info(f"Job {job_id}: Found updated corrections, merging")
|
||||
updated_path = os.path.join(temp_dir, "corrections_updated.json")
|
||||
storage.download_file(updated_corrections_gcs, updated_path)
|
||||
|
||||
with open(updated_path, 'r', encoding='utf-8') as f:
|
||||
updated_data = json.load(f)
|
||||
|
||||
# Merge: update the original with the user's corrections
|
||||
if 'corrections' in updated_data:
|
||||
original_data['corrections'] = updated_data['corrections']
|
||||
if 'corrected_segments' in updated_data:
|
||||
original_data['corrected_segments'] = updated_data['corrected_segments']
|
||||
|
||||
job_log.info("Merged user corrections into original data")
|
||||
logger.info(f"Job {job_id}: Merged user corrections into original data")
|
||||
|
||||
# 4. Convert to CorrectionResult
|
||||
correction_result = CorrectionResult.from_dict(original_data)
|
||||
job_log.info(f"Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
|
||||
logger.info(f"Job {job_id}: Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
|
||||
|
||||
# 3. Download audio file
|
||||
audio_path = os.path.join(temp_dir, "audio.flac")
|
||||
audio_gcs_path = job.input_media_gcs_path
|
||||
|
||||
if not audio_gcs_path:
|
||||
raise FileNotFoundError(f"No input audio path for job {job_id}")
|
||||
|
||||
job_log.info(f"Downloading audio from {audio_gcs_path}")
|
||||
logger.info(f"Job {job_id}: Downloading audio from {audio_gcs_path}")
|
||||
storage.download_file(audio_gcs_path, audio_path)
|
||||
job_log.info(f"Audio downloaded: {os.path.getsize(audio_path)} bytes")
|
||||
|
||||
# 4. Get or create styles using the unified style loader
|
||||
job_log.info("Loading style configuration...")
|
||||
job_log.info(f" job.style_params_gcs_path: {job.style_params_gcs_path}")
|
||||
job_log.info(f" job.style_assets: {list(job.style_assets.keys()) if job.style_assets else 'None'}")
|
||||
|
||||
styles_path, style_data = load_styles_from_gcs(
|
||||
style_params_gcs_path=job.style_params_gcs_path,
|
||||
style_assets=job.style_assets,
|
||||
temp_dir=temp_dir,
|
||||
download_func=storage.download_file,
|
||||
logger=job_log,
|
||||
)
|
||||
|
||||
# 5. Configure OutputGenerator
|
||||
output_dir = os.path.join(temp_dir, "output")
|
||||
cache_dir = os.path.join(temp_dir, "cache")
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
os.makedirs(cache_dir, exist_ok=True)
|
||||
|
||||
job_log.info(f"Using styles from: {styles_path}")
|
||||
|
||||
config = OutputConfig(
|
||||
output_dir=output_dir,
|
||||
cache_dir=cache_dir,
|
||||
output_styles_json=styles_path,
|
||||
render_video=True,
|
||||
generate_cdg=False, # CDG optional, generated separately
|
||||
generate_plain_text=True,
|
||||
generate_lrc=True,
|
||||
video_resolution="4k",
|
||||
subtitle_offset_ms=0
|
||||
)
|
||||
|
||||
job_log.info(f"OutputConfig: output_styles_json={config.output_styles_json}, render_video={config.render_video}")
|
||||
|
||||
output_generator = OutputGenerator(config, logger)
|
||||
|
||||
# 6. Generate outputs (video, LRC, ASS, etc.)
|
||||
output_prefix = f"{job.artist or 'Unknown'} - {job.title or 'Unknown'}"
|
||||
job_log.info(f"Generating outputs with prefix '{output_prefix}'")
|
||||
logger.info(f"Job {job_id}: Generating outputs with prefix '{output_prefix}'")
|
||||
|
||||
outputs = output_generator.generate_outputs(
|
||||
transcription_corrected=correction_result,
|
||||
lyrics_results={}, # Reference lyrics already in correction_result
|
||||
output_prefix=output_prefix,
|
||||
audio_filepath=audio_path,
|
||||
artist=job.artist,
|
||||
title=job.title
|
||||
)
|
||||
|
||||
# 7. Upload video to GCS
|
||||
if outputs.video and os.path.exists(outputs.video):
|
||||
video_size = os.path.getsize(outputs.video)
|
||||
job_log.info(f"Video generated: {video_size} bytes")
|
||||
video_gcs_path = f"jobs/{job_id}/videos/with_vocals.mkv"
|
||||
video_url = storage.upload_file(outputs.video, video_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'videos', 'with_vocals', video_url)
|
||||
job_log.info(f"Uploaded with_vocals.mkv to GCS")
|
||||
logger.info(f"Job {job_id}: Uploaded with_vocals.mkv ({video_size} bytes)")
|
||||
else:
|
||||
job_log.error("Video generation failed - no output file produced!")
|
||||
raise Exception("Video generation failed - no output file produced")
|
||||
|
||||
# 8. Upload LRC file
|
||||
if outputs.lrc and os.path.exists(outputs.lrc):
|
||||
lrc_gcs_path = f"jobs/{job_id}/lyrics/karaoke.lrc"
|
||||
lrc_url = storage.upload_file(outputs.lrc, lrc_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'lyrics', 'lrc', lrc_url)
|
||||
job_log.info("Uploaded karaoke.lrc")
|
||||
logger.info(f"Job {job_id}: Uploaded karaoke.lrc")
|
||||
|
||||
# 9. Upload ASS subtitle file
|
||||
if outputs.ass and os.path.exists(outputs.ass):
|
||||
ass_gcs_path = f"jobs/{job_id}/lyrics/karaoke.ass"
|
||||
ass_url = storage.upload_file(outputs.ass, ass_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'lyrics', 'ass', ass_url)
|
||||
job_log.info("Uploaded karaoke.ass")
|
||||
logger.info(f"Job {job_id}: Uploaded karaoke.ass")
|
||||
|
||||
# 10. Upload corrected text files
|
||||
if outputs.corrected_txt and os.path.exists(outputs.corrected_txt):
|
||||
txt_gcs_path = f"jobs/{job_id}/lyrics/corrected.txt"
|
||||
txt_url = storage.upload_file(outputs.corrected_txt, txt_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'lyrics', 'corrected_txt', txt_url)
|
||||
job_log.info("Uploaded corrected.txt")
|
||||
logger.info(f"Job {job_id}: Uploaded corrected.txt")
|
||||
|
||||
# 11. Transition to awaiting instrumental selection
|
||||
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
|
||||
# This ensures logs from third-party libraries (lyrics_transcriber.output) are only
|
||||
# captured by this job's handler, not handlers from other concurrent jobs
|
||||
with job_logging_context(job_id):
|
||||
# Transition to RENDERING_VIDEO
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
|
||||
progress=80,
|
||||
message="Video rendered - select your instrumental"
|
||||
new_status=JobStatus.RENDERING_VIDEO,
|
||||
progress=75,
|
||||
message="Rendering karaoke video with corrected lyrics"
|
||||
)
|
||||
|
||||
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
|
||||
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
|
||||
return True
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
job_log.info(f"Created temp directory: {temp_dir}")
|
||||
# 1. Download corrected corrections data
|
||||
corrections_path = os.path.join(temp_dir, "corrections.json")
|
||||
|
||||
# Try updated corrections first (from human review), fall back to original
|
||||
corrections_gcs_updated = f"jobs/{job_id}/lyrics/corrections_updated.json"
|
||||
corrections_gcs_original = f"jobs/{job_id}/lyrics/corrections.json"
|
||||
|
||||
# Get GCS path from file_urls
|
||||
corrections_url = job.file_urls.get('lyrics', {}).get('corrections_updated')
|
||||
if not corrections_url:
|
||||
corrections_url = job.file_urls.get('lyrics', {}).get('corrections')
|
||||
|
||||
if not corrections_url:
|
||||
# Try direct GCS paths
|
||||
if storage.file_exists(corrections_gcs_updated):
|
||||
corrections_gcs = corrections_gcs_updated
|
||||
elif storage.file_exists(corrections_gcs_original):
|
||||
corrections_gcs = corrections_gcs_original
|
||||
else:
|
||||
raise FileNotFoundError(f"No corrections file found for job {job_id}")
|
||||
else:
|
||||
# Extract GCS path from URL
|
||||
corrections_gcs = _extract_gcs_path(corrections_url)
|
||||
|
||||
# 2. Load the ORIGINAL corrections (has full structure)
|
||||
original_corrections_gcs = f"jobs/{job_id}/lyrics/corrections.json"
|
||||
original_corrections_path = os.path.join(temp_dir, "corrections_original.json")
|
||||
|
||||
job_log.info(f"Downloading original corrections from {original_corrections_gcs}")
|
||||
logger.info(f"Job {job_id}: Downloading original corrections from {original_corrections_gcs}")
|
||||
storage.download_file(original_corrections_gcs, original_corrections_path)
|
||||
|
||||
with open(original_corrections_path, 'r', encoding='utf-8') as f:
|
||||
original_data = json.load(f)
|
||||
|
||||
# 3. Check if there are updated corrections (from review UI)
|
||||
# The frontend sends only partial data: {corrections, corrected_segments}
|
||||
updated_corrections_gcs = f"jobs/{job_id}/lyrics/corrections_updated.json"
|
||||
|
||||
if storage.file_exists(updated_corrections_gcs):
|
||||
job_log.info("Found updated corrections from review, merging")
|
||||
logger.info(f"Job {job_id}: Found updated corrections, merging")
|
||||
updated_path = os.path.join(temp_dir, "corrections_updated.json")
|
||||
storage.download_file(updated_corrections_gcs, updated_path)
|
||||
|
||||
with open(updated_path, 'r', encoding='utf-8') as f:
|
||||
updated_data = json.load(f)
|
||||
|
||||
# Merge: update the original with the user's corrections
|
||||
if 'corrections' in updated_data:
|
||||
original_data['corrections'] = updated_data['corrections']
|
||||
if 'corrected_segments' in updated_data:
|
||||
original_data['corrected_segments'] = updated_data['corrected_segments']
|
||||
|
||||
job_log.info("Merged user corrections into original data")
|
||||
logger.info(f"Job {job_id}: Merged user corrections into original data")
|
||||
|
||||
# 4. Convert to CorrectionResult
|
||||
correction_result = CorrectionResult.from_dict(original_data)
|
||||
job_log.info(f"Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
|
||||
logger.info(f"Job {job_id}: Loaded CorrectionResult with {len(correction_result.corrected_segments)} segments")
|
||||
|
||||
# 3. Download audio file
|
||||
audio_path = os.path.join(temp_dir, "audio.flac")
|
||||
audio_gcs_path = job.input_media_gcs_path
|
||||
|
||||
if not audio_gcs_path:
|
||||
raise FileNotFoundError(f"No input audio path for job {job_id}")
|
||||
|
||||
job_log.info(f"Downloading audio from {audio_gcs_path}")
|
||||
logger.info(f"Job {job_id}: Downloading audio from {audio_gcs_path}")
|
||||
storage.download_file(audio_gcs_path, audio_path)
|
||||
job_log.info(f"Audio downloaded: {os.path.getsize(audio_path)} bytes")
|
||||
|
||||
# 4. Get or create styles using the unified style loader
|
||||
job_log.info("Loading style configuration...")
|
||||
job_log.info(f" job.style_params_gcs_path: {job.style_params_gcs_path}")
|
||||
job_log.info(f" job.style_assets: {list(job.style_assets.keys()) if job.style_assets else 'None'}")
|
||||
|
||||
styles_path, style_data = load_styles_from_gcs(
|
||||
style_params_gcs_path=job.style_params_gcs_path,
|
||||
style_assets=job.style_assets,
|
||||
temp_dir=temp_dir,
|
||||
download_func=storage.download_file,
|
||||
logger=job_log,
|
||||
)
|
||||
|
||||
# 5. Configure OutputGenerator
|
||||
output_dir = os.path.join(temp_dir, "output")
|
||||
cache_dir = os.path.join(temp_dir, "cache")
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
os.makedirs(cache_dir, exist_ok=True)
|
||||
|
||||
job_log.info(f"Using styles from: {styles_path}")
|
||||
|
||||
# Get subtitle offset from job (user-specified timing adjustment)
|
||||
subtitle_offset = getattr(job, 'subtitle_offset_ms', 0) or 0
|
||||
if subtitle_offset != 0:
|
||||
job_log.info(f"Applying subtitle offset: {subtitle_offset}ms")
|
||||
|
||||
config = OutputConfig(
|
||||
output_dir=output_dir,
|
||||
cache_dir=cache_dir,
|
||||
output_styles_json=styles_path,
|
||||
render_video=True,
|
||||
generate_cdg=False, # CDG optional, generated separately
|
||||
generate_plain_text=True,
|
||||
generate_lrc=True,
|
||||
video_resolution="4k",
|
||||
subtitle_offset_ms=subtitle_offset
|
||||
)
|
||||
|
||||
job_log.info(f"OutputConfig: output_styles_json={config.output_styles_json}, render_video={config.render_video}")
|
||||
|
||||
output_generator = OutputGenerator(config, logger)
|
||||
|
||||
# 6. Generate outputs (video, LRC, ASS, etc.)
|
||||
output_prefix = f"{job.artist or 'Unknown'} - {job.title or 'Unknown'}"
|
||||
job_log.info(f"Generating outputs with prefix '{output_prefix}'")
|
||||
logger.info(f"Job {job_id}: Generating outputs with prefix '{output_prefix}'")
|
||||
|
||||
outputs = output_generator.generate_outputs(
|
||||
transcription_corrected=correction_result,
|
||||
lyrics_results={}, # Reference lyrics already in correction_result
|
||||
output_prefix=output_prefix,
|
||||
audio_filepath=audio_path,
|
||||
artist=job.artist,
|
||||
title=job.title
|
||||
)
|
||||
|
||||
# 7. Upload video to GCS
|
||||
if outputs.video and os.path.exists(outputs.video):
|
||||
video_size = os.path.getsize(outputs.video)
|
||||
job_log.info(f"Video generated: {video_size} bytes")
|
||||
video_gcs_path = f"jobs/{job_id}/videos/with_vocals.mkv"
|
||||
video_url = storage.upload_file(outputs.video, video_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'videos', 'with_vocals', video_url)
|
||||
job_log.info(f"Uploaded with_vocals.mkv to GCS")
|
||||
logger.info(f"Job {job_id}: Uploaded with_vocals.mkv ({video_size} bytes)")
|
||||
else:
|
||||
job_log.error("Video generation failed - no output file produced!")
|
||||
raise Exception("Video generation failed - no output file produced")
|
||||
|
||||
# 8. Upload LRC file
|
||||
if outputs.lrc and os.path.exists(outputs.lrc):
|
||||
lrc_gcs_path = f"jobs/{job_id}/lyrics/karaoke.lrc"
|
||||
lrc_url = storage.upload_file(outputs.lrc, lrc_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'lyrics', 'lrc', lrc_url)
|
||||
job_log.info("Uploaded karaoke.lrc")
|
||||
logger.info(f"Job {job_id}: Uploaded karaoke.lrc")
|
||||
|
||||
# 9. Upload ASS subtitle file
|
||||
if outputs.ass and os.path.exists(outputs.ass):
|
||||
ass_gcs_path = f"jobs/{job_id}/lyrics/karaoke.ass"
|
||||
ass_url = storage.upload_file(outputs.ass, ass_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'lyrics', 'ass', ass_url)
|
||||
job_log.info("Uploaded karaoke.ass")
|
||||
logger.info(f"Job {job_id}: Uploaded karaoke.ass")
|
||||
|
||||
# 10. Upload corrected text files
|
||||
if outputs.corrected_txt and os.path.exists(outputs.corrected_txt):
|
||||
txt_gcs_path = f"jobs/{job_id}/lyrics/corrected.txt"
|
||||
txt_url = storage.upload_file(outputs.corrected_txt, txt_gcs_path)
|
||||
job_manager.update_file_url(job_id, 'lyrics', 'corrected_txt', txt_url)
|
||||
job_log.info("Uploaded corrected.txt")
|
||||
logger.info(f"Job {job_id}: Uploaded corrected.txt")
|
||||
|
||||
# 11. Analyze backing vocals for intelligent instrumental selection
|
||||
job_log.info("Analyzing backing vocals for instrumental selection...")
|
||||
await _analyze_backing_vocals(job_id, job_manager, storage, job_log)
|
||||
|
||||
# 12. Transition based on prep_only flag
|
||||
if getattr(job, 'prep_only', False):
|
||||
# Prep-only mode: stop here and mark as prep complete
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.PREP_COMPLETE,
|
||||
progress=100,
|
||||
message="Prep phase complete - download outputs to continue locally"
|
||||
)
|
||||
job_log.info("=== RENDER VIDEO WORKER COMPLETE (PREP ONLY) ===")
|
||||
logger.info(f"Job {job_id}: Prep-only video render complete")
|
||||
else:
|
||||
# Normal mode: proceed to instrumental selection
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
|
||||
progress=80,
|
||||
message="Video rendered - select your instrumental"
|
||||
)
|
||||
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
|
||||
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
job_log.error(f"Video render failed: {e}")
|
||||
@@ -293,5 +317,109 @@ def _extract_gcs_path(url: str) -> str:
|
||||
return url
|
||||
|
||||
|
||||
async def _analyze_backing_vocals(
|
||||
job_id: str,
|
||||
job_manager: JobManager,
|
||||
storage: StorageService,
|
||||
job_log: logging.Logger,
|
||||
) -> None:
|
||||
"""
|
||||
Analyze backing vocals to help with intelligent instrumental selection.
|
||||
|
||||
This function:
|
||||
1. Downloads the backing vocals stem from GCS
|
||||
2. Runs audio analysis to detect audible content
|
||||
3. Generates a waveform visualization image
|
||||
4. Stores analysis results and waveform URL in job state
|
||||
|
||||
The analysis data is then used by the frontend to provide an
|
||||
intelligent instrumental selection experience.
|
||||
"""
|
||||
from backend.services.audio_analysis_service import AudioAnalysisService
|
||||
|
||||
try:
|
||||
# Get the job to access file URLs
|
||||
job = job_manager.get_job(job_id)
|
||||
if not job:
|
||||
job_log.warning(f"Could not get job {job_id} for backing vocals analysis")
|
||||
return
|
||||
|
||||
# Get backing vocals path
|
||||
backing_vocals_path = job.file_urls.get('stems', {}).get('backing_vocals')
|
||||
if not backing_vocals_path:
|
||||
job_log.warning("No backing vocals file found - skipping analysis")
|
||||
return
|
||||
|
||||
job_log.info(f"Analyzing backing vocals: {backing_vocals_path}")
|
||||
|
||||
# Create analysis service and run analysis
|
||||
analysis_service = AudioAnalysisService()
|
||||
|
||||
# Define output path for waveform
|
||||
waveform_gcs_path = f"jobs/{job_id}/analysis/backing_vocals_waveform.png"
|
||||
|
||||
# Run analysis and generate waveform
|
||||
result, waveform_path = analysis_service.analyze_and_generate_waveform(
|
||||
gcs_audio_path=backing_vocals_path,
|
||||
job_id=job_id,
|
||||
gcs_waveform_destination=waveform_gcs_path,
|
||||
)
|
||||
|
||||
# Store analysis results in job state_data
|
||||
analysis_data = {
|
||||
'has_audible_content': result.has_audible_content,
|
||||
'total_duration_seconds': result.total_duration_seconds,
|
||||
'audible_segments': [
|
||||
{
|
||||
'start_seconds': seg.start_seconds,
|
||||
'end_seconds': seg.end_seconds,
|
||||
'duration_seconds': seg.duration_seconds,
|
||||
'avg_amplitude_db': seg.avg_amplitude_db,
|
||||
'peak_amplitude_db': seg.peak_amplitude_db,
|
||||
}
|
||||
for seg in result.audible_segments
|
||||
],
|
||||
'recommended_selection': result.recommended_selection.value,
|
||||
'total_audible_duration_seconds': result.total_audible_duration_seconds,
|
||||
'audible_percentage': result.audible_percentage,
|
||||
'silence_threshold_db': result.silence_threshold_db,
|
||||
}
|
||||
|
||||
job_manager.update_state_data(job_id, 'backing_vocals_analysis', analysis_data)
|
||||
|
||||
# Store waveform URL
|
||||
job_manager.update_file_url(job_id, 'analysis', 'backing_vocals_waveform', waveform_path)
|
||||
|
||||
job_log.info(
|
||||
f"Backing vocals analysis complete: "
|
||||
f"has_audible={result.has_audible_content}, "
|
||||
f"segments={result.segment_count}, "
|
||||
f"recommendation={result.recommended_selection.value}"
|
||||
)
|
||||
|
||||
# Log segment details if there are any
|
||||
if result.audible_segments:
|
||||
job_log.info(f"Audible segments ({len(result.audible_segments)}):")
|
||||
for i, seg in enumerate(result.audible_segments[:5]): # Log first 5
|
||||
job_log.info(
|
||||
f" [{i+1}] {seg.start_seconds:.1f}s - {seg.end_seconds:.1f}s "
|
||||
f"({seg.duration_seconds:.1f}s, avg: {seg.avg_amplitude_db:.1f}dB)"
|
||||
)
|
||||
if len(result.audible_segments) > 5:
|
||||
job_log.info(f" ... and {len(result.audible_segments) - 5} more")
|
||||
|
||||
except Exception as e:
|
||||
# Log the error but don't fail the job - analysis is a nice-to-have
|
||||
job_log.warning(f"Backing vocals analysis failed (non-fatal): {e}")
|
||||
logger.warning(f"Job {job_id}: Backing vocals analysis failed: {e}")
|
||||
|
||||
# Store empty analysis so the frontend knows analysis was attempted
|
||||
job_manager.update_state_data(job_id, 'backing_vocals_analysis', {
|
||||
'has_audible_content': None,
|
||||
'analysis_error': str(e),
|
||||
'recommended_selection': 'review_needed',
|
||||
})
|
||||
|
||||
|
||||
# For compatibility with worker service
|
||||
render_video_worker = process_render_video
|
||||
@@ -35,7 +35,7 @@ from backend.services.job_manager import JobManager
|
||||
from backend.services.storage_service import StorageService
|
||||
from backend.config import get_settings
|
||||
from backend.workers.style_helper import load_style_config, StyleConfig
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
|
||||
|
||||
# Import from karaoke_gen package
|
||||
from karaoke_gen.video_generator import VideoGenerator
|
||||
@@ -88,94 +88,98 @@ async def generate_screens(job_id: str) -> bool:
|
||||
temp_dir = tempfile.mkdtemp(prefix=f"karaoke_screens_{job_id}_")
|
||||
|
||||
try:
|
||||
job_log.info(f"Starting screen generation for {job.artist} - {job.title}")
|
||||
logger.info(f"Starting screen generation for job {job_id}")
|
||||
|
||||
# Transition to GENERATING_SCREENS state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.GENERATING_SCREENS,
|
||||
progress=50,
|
||||
message="Generating title and end screens"
|
||||
)
|
||||
|
||||
# Log style assets info
|
||||
style_assets = getattr(job, 'style_assets', {}) or {}
|
||||
job_log.info(f"Style assets from job: {list(style_assets.keys()) if style_assets else 'None'}")
|
||||
if style_assets:
|
||||
for key, path in style_assets.items():
|
||||
job_log.info(f" {key}: {path}")
|
||||
|
||||
# Load style configuration (downloads assets from GCS if available)
|
||||
job_log.info("Loading style configuration from GCS...")
|
||||
style_config = await load_style_config(job, storage, temp_dir)
|
||||
if style_config.has_custom_styles():
|
||||
job_log.info("Using CUSTOM style configuration")
|
||||
logger.info(f"Job {job_id}: Using custom style configuration")
|
||||
else:
|
||||
job_log.warning("Using DEFAULT style configuration (no custom styles found)")
|
||||
logger.info(f"Job {job_id}: Using default style configuration")
|
||||
|
||||
# Initialize video generator
|
||||
video_generator = _create_video_generator(temp_dir)
|
||||
|
||||
# Generate title screen with style config
|
||||
job_log.info("Generating title screen...")
|
||||
title_screen_path = await _generate_title_screen(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
video_generator=video_generator,
|
||||
style_config=style_config,
|
||||
temp_dir=temp_dir,
|
||||
job_log=job_log
|
||||
)
|
||||
|
||||
if not title_screen_path:
|
||||
raise Exception("Title screen generation failed")
|
||||
job_log.info(f"Title screen generated: {title_screen_path}")
|
||||
|
||||
# Generate end screen with style config
|
||||
job_log.info("Generating end screen...")
|
||||
end_screen_path = await _generate_end_screen(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
video_generator=video_generator,
|
||||
style_config=style_config,
|
||||
temp_dir=temp_dir,
|
||||
job_log=job_log
|
||||
)
|
||||
|
||||
if not end_screen_path:
|
||||
raise Exception("End screen generation failed")
|
||||
job_log.info(f"End screen generated: {end_screen_path}")
|
||||
|
||||
# Upload screens to GCS
|
||||
await _upload_screens(
|
||||
job_id=job_id,
|
||||
job_manager=job_manager,
|
||||
storage=storage,
|
||||
title_screen_path=title_screen_path,
|
||||
end_screen_path=end_screen_path
|
||||
)
|
||||
|
||||
# Apply countdown padding if needed
|
||||
# (This is handled automatically by checking lyrics metadata)
|
||||
await _apply_countdown_padding_if_needed(job_id, job_manager, job)
|
||||
|
||||
# Transition to AWAITING_REVIEW
|
||||
# Human must review lyrics before video can be rendered
|
||||
logger.info(f"Job {job_id}: Screens generated, awaiting lyrics review")
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.AWAITING_REVIEW,
|
||||
progress=55,
|
||||
message="Ready for lyrics review. Please review and correct lyrics."
|
||||
)
|
||||
|
||||
# TODO: Send notification to user that selection is needed
|
||||
# await notify_user(job_id, "instrumental_selection_ready")
|
||||
|
||||
return True
|
||||
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
|
||||
# This ensures logs from third-party libraries (karaoke_gen.video_generator) are only
|
||||
# captured by this job's handler, not handlers from other concurrent jobs
|
||||
with job_logging_context(job_id):
|
||||
job_log.info(f"Starting screen generation for {job.artist} - {job.title}")
|
||||
logger.info(f"Starting screen generation for job {job_id}")
|
||||
|
||||
# Transition to GENERATING_SCREENS state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.GENERATING_SCREENS,
|
||||
progress=50,
|
||||
message="Generating title and end screens"
|
||||
)
|
||||
|
||||
# Log style assets info
|
||||
style_assets = getattr(job, 'style_assets', {}) or {}
|
||||
job_log.info(f"Style assets from job: {list(style_assets.keys()) if style_assets else 'None'}")
|
||||
if style_assets:
|
||||
for key, path in style_assets.items():
|
||||
job_log.info(f" {key}: {path}")
|
||||
|
||||
# Load style configuration (downloads assets from GCS if available)
|
||||
job_log.info("Loading style configuration from GCS...")
|
||||
style_config = await load_style_config(job, storage, temp_dir)
|
||||
if style_config.has_custom_styles():
|
||||
job_log.info("Using CUSTOM style configuration")
|
||||
logger.info(f"Job {job_id}: Using custom style configuration")
|
||||
else:
|
||||
job_log.warning("Using DEFAULT style configuration (no custom styles found)")
|
||||
logger.info(f"Job {job_id}: Using default style configuration")
|
||||
|
||||
# Initialize video generator
|
||||
video_generator = _create_video_generator(temp_dir)
|
||||
|
||||
# Generate title screen with style config
|
||||
job_log.info("Generating title screen...")
|
||||
title_screen_path = await _generate_title_screen(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
video_generator=video_generator,
|
||||
style_config=style_config,
|
||||
temp_dir=temp_dir,
|
||||
job_log=job_log
|
||||
)
|
||||
|
||||
if not title_screen_path:
|
||||
raise Exception("Title screen generation failed")
|
||||
job_log.info(f"Title screen generated: {title_screen_path}")
|
||||
|
||||
# Generate end screen with style config
|
||||
job_log.info("Generating end screen...")
|
||||
end_screen_path = await _generate_end_screen(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
video_generator=video_generator,
|
||||
style_config=style_config,
|
||||
temp_dir=temp_dir,
|
||||
job_log=job_log
|
||||
)
|
||||
|
||||
if not end_screen_path:
|
||||
raise Exception("End screen generation failed")
|
||||
job_log.info(f"End screen generated: {end_screen_path}")
|
||||
|
||||
# Upload screens to GCS
|
||||
await _upload_screens(
|
||||
job_id=job_id,
|
||||
job_manager=job_manager,
|
||||
storage=storage,
|
||||
title_screen_path=title_screen_path,
|
||||
end_screen_path=end_screen_path
|
||||
)
|
||||
|
||||
# Apply countdown padding if needed
|
||||
# (This is handled automatically by checking lyrics metadata)
|
||||
await _apply_countdown_padding_if_needed(job_id, job_manager, job)
|
||||
|
||||
# Transition to AWAITING_REVIEW
|
||||
# Human must review lyrics before video can be rendered
|
||||
logger.info(f"Job {job_id}: Screens generated, awaiting lyrics review")
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.AWAITING_REVIEW,
|
||||
progress=55,
|
||||
message="Ready for lyrics review. Please review and correct lyrics."
|
||||
)
|
||||
|
||||
# TODO: Send notification to user that selection is needed
|
||||
# await notify_user(job_id, "instrumental_selection_ready")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Job {job_id}: Screen generation failed: {e}", exc_info=True)
|
||||
|
||||
@@ -31,7 +31,7 @@ from backend.services.rclone_service import get_rclone_service
|
||||
from backend.services.youtube_service import get_youtube_service
|
||||
from backend.config import get_settings
|
||||
from backend.workers.style_helper import load_style_config
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging
|
||||
from backend.workers.worker_logging import create_job_logger, setup_job_logging, job_logging_context
|
||||
|
||||
# Import from karaoke_gen package - reuse existing implementation
|
||||
from karaoke_gen.karaoke_finalise.karaoke_finalise import KaraokeFinalise
|
||||
@@ -105,149 +105,177 @@ async def generate_video(job_id: str) -> bool:
|
||||
job_log.warning("YouTube credentials not available - upload will be skipped")
|
||||
|
||||
try:
|
||||
job_log.info(f"Starting video finalization for {job.artist} - {job.title}")
|
||||
logger.info(f"Starting video generation for job {job_id}")
|
||||
|
||||
# Transition to GENERATING_VIDEO state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.GENERATING_VIDEO,
|
||||
progress=70,
|
||||
message="Preparing files for video generation"
|
||||
)
|
||||
|
||||
# Download and set up files in the format KaraokeFinalise expects
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
job_log.info("Downloading files from GCS (this may take a few minutes for large files)...")
|
||||
await _setup_working_directory(job_id, job, storage, temp_dir, base_name, job_log)
|
||||
job_log.info("All files downloaded successfully")
|
||||
|
||||
# Load style config for CDG styles
|
||||
job_log.info("Loading CDG style configuration...")
|
||||
style_config = await load_style_config(job, storage, temp_dir)
|
||||
cdg_styles = style_config.get_cdg_styles()
|
||||
if cdg_styles:
|
||||
job_log.info("CDG styles loaded from custom configuration")
|
||||
else:
|
||||
job_log.info("Using default CDG styles")
|
||||
|
||||
# Change to working directory (KaraokeFinalise works in cwd)
|
||||
os.chdir(temp_dir)
|
||||
|
||||
# Get the selected instrumental file path
|
||||
instrumental_selection = job.state_data['instrumental_selection']
|
||||
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
|
||||
instrumental_file = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
|
||||
|
||||
# Transition to encoding state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.ENCODING,
|
||||
progress=75,
|
||||
message="Encoding videos (15-20 min)"
|
||||
)
|
||||
|
||||
# Log finalization parameters
|
||||
job_log.info("Creating KaraokeFinalise with parameters:")
|
||||
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
|
||||
job_log.info(f" enable_txt: {getattr(job, 'enable_txt', False)}")
|
||||
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
|
||||
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
|
||||
job_log.info(f" instrumental: {instrumental_selection}")
|
||||
|
||||
# Create KaraokeFinalise with ALL the parameters from the job
|
||||
# This reuses all existing functionality!
|
||||
|
||||
# Set up YouTube description file if template is provided
|
||||
youtube_desc_path = None
|
||||
if youtube_credentials and getattr(job, 'youtube_description_template', None):
|
||||
youtube_desc_path = os.path.join(temp_dir, "youtube_description.txt")
|
||||
with open(youtube_desc_path, 'w') as f:
|
||||
f.write(job.youtube_description_template)
|
||||
job_log.info("YouTube description template written to temp file")
|
||||
|
||||
finalise = KaraokeFinalise(
|
||||
logger=logger,
|
||||
log_level=logging.INFO,
|
||||
dry_run=False,
|
||||
instrumental_format="flac",
|
||||
# CDG/TXT generation
|
||||
enable_cdg=getattr(job, 'enable_cdg', False),
|
||||
enable_txt=getattr(job, 'enable_txt', False),
|
||||
cdg_styles=cdg_styles,
|
||||
# Brand code and organization (server-side mode uses rclone)
|
||||
brand_prefix=getattr(job, 'brand_prefix', None),
|
||||
organised_dir=None, # Not used in cloud - files stay in GCS
|
||||
organised_dir_rclone_root=getattr(job, 'organised_dir_rclone_root', None),
|
||||
public_share_dir=None, # Not used in cloud
|
||||
# Notifications
|
||||
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
|
||||
# YouTube upload (server-side with pre-loaded credentials)
|
||||
youtube_client_secrets_file=None, # Not used with pre-stored credentials
|
||||
youtube_description_file=youtube_desc_path,
|
||||
user_youtube_credentials=youtube_credentials, # Pre-loaded from Secret Manager
|
||||
rclone_destination=None,
|
||||
email_template_file=None,
|
||||
# Server-side optimizations
|
||||
non_interactive=True,
|
||||
server_side_mode=True,
|
||||
selected_instrumental_file=instrumental_file,
|
||||
)
|
||||
|
||||
# Call process() - this does ALL the work:
|
||||
# - Encodes to 4 video formats
|
||||
# - Generates CDG/TXT packages if enabled
|
||||
# - Posts Discord notification if configured
|
||||
# - Handles brand code generation
|
||||
job_log.info("Starting KaraokeFinalise.process() - this may take 15-20 minutes...")
|
||||
logger.info(f"Job {job_id}: Starting KaraokeFinalise.process()")
|
||||
result = finalise.process()
|
||||
|
||||
job_log.info("KaraokeFinalise.process() complete!")
|
||||
if result.get('brand_code'):
|
||||
job_log.info(f"Brand code: {result.get('brand_code')}")
|
||||
logger.info(f"Job {job_id}: KaraokeFinalise.process() complete")
|
||||
logger.info(f"Job {job_id}: Brand code: {result.get('brand_code')}")
|
||||
|
||||
# Native API distribution uploads (used by remote CLI instead of rclone)
|
||||
await _handle_native_distribution(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
job_log=job_log,
|
||||
job_manager=job_manager,
|
||||
temp_dir=temp_dir,
|
||||
result=result,
|
||||
)
|
||||
|
||||
# Upload generated files to GCS
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.PACKAGING,
|
||||
progress=95,
|
||||
message="Uploading final files"
|
||||
)
|
||||
|
||||
await _upload_results(job_id, job_manager, storage, temp_dir, result)
|
||||
|
||||
# Mark job as complete
|
||||
logger.info(f"Job {job_id}: Video generation complete")
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.COMPLETE,
|
||||
progress=100,
|
||||
message="Karaoke generation complete!"
|
||||
)
|
||||
|
||||
# Store result metadata in job
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
'brand_code': result.get('brand_code'),
|
||||
'youtube_url': result.get('youtube_url'),
|
||||
}
|
||||
})
|
||||
|
||||
return True
|
||||
# Use job_logging_context for proper log isolation when multiple jobs run concurrently
|
||||
# This ensures logs from third-party libraries (karaoke_gen.karaoke_finalise) are only
|
||||
# captured by this job's handler, not handlers from other concurrent jobs
|
||||
with job_logging_context(job_id):
|
||||
job_log.info(f"Starting video finalization for {job.artist} - {job.title}")
|
||||
logger.info(f"Starting video generation for job {job_id}")
|
||||
|
||||
# Transition to GENERATING_VIDEO state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.GENERATING_VIDEO,
|
||||
progress=70,
|
||||
message="Preparing files for video generation"
|
||||
)
|
||||
|
||||
# Download and set up files in the format KaraokeFinalise expects
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
job_log.info("Downloading files from GCS (this may take a few minutes for large files)...")
|
||||
await _setup_working_directory(job_id, job, storage, temp_dir, base_name, job_log)
|
||||
job_log.info("All files downloaded successfully")
|
||||
|
||||
# Load style config for CDG styles
|
||||
job_log.info("Loading CDG style configuration...")
|
||||
style_config = await load_style_config(job, storage, temp_dir)
|
||||
cdg_styles = style_config.get_cdg_styles()
|
||||
if cdg_styles:
|
||||
job_log.info("CDG styles loaded from custom configuration")
|
||||
else:
|
||||
job_log.info("Using default CDG styles")
|
||||
|
||||
# Change to working directory (KaraokeFinalise works in cwd)
|
||||
os.chdir(temp_dir)
|
||||
|
||||
# Get the selected instrumental file path
|
||||
# Batch 3: If user provided existing instrumental, use that; otherwise use AI-separated
|
||||
instrumental_selection = job.state_data['instrumental_selection']
|
||||
existing_instrumental_path = getattr(job, 'existing_instrumental_gcs_path', None)
|
||||
|
||||
if existing_instrumental_path:
|
||||
# User provided existing instrumental
|
||||
ext = Path(existing_instrumental_path).suffix.lower()
|
||||
instrumental_file = os.path.join(temp_dir, f"{base_name} (Instrumental User){ext}")
|
||||
instrumental_source = "user-provided"
|
||||
else:
|
||||
# Use AI-separated instrumental
|
||||
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
|
||||
instrumental_file = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
|
||||
instrumental_source = instrumental_selection
|
||||
|
||||
# Transition to encoding state
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.ENCODING,
|
||||
progress=75,
|
||||
message="Encoding videos (15-20 min)"
|
||||
)
|
||||
|
||||
# Get countdown padding info from lyrics metadata
|
||||
# This ensures instrumental is padded to match vocals if countdown was added
|
||||
lyrics_metadata = job.state_data.get('lyrics_metadata', {})
|
||||
countdown_padding_seconds = None
|
||||
if lyrics_metadata.get('has_countdown_padding'):
|
||||
countdown_padding_seconds = lyrics_metadata.get('countdown_padding_seconds', 3.0)
|
||||
job_log.info(f"Countdown padding detected: {countdown_padding_seconds}s - instrumental will be padded if needed")
|
||||
|
||||
# Log finalization parameters
|
||||
job_log.info("Creating KaraokeFinalise with parameters:")
|
||||
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
|
||||
job_log.info(f" enable_txt: {getattr(job, 'enable_txt', False)}")
|
||||
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
|
||||
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
|
||||
job_log.info(f" instrumental source: {instrumental_source}")
|
||||
job_log.info(f" countdown_padding_seconds: {countdown_padding_seconds}")
|
||||
if existing_instrumental_path:
|
||||
job_log.info(f" using user-provided instrumental (selection was: {instrumental_selection})")
|
||||
|
||||
# Create KaraokeFinalise with ALL the parameters from the job
|
||||
# This reuses all existing functionality!
|
||||
|
||||
# Set up YouTube description file if template is provided
|
||||
youtube_desc_path = None
|
||||
if youtube_credentials and getattr(job, 'youtube_description_template', None):
|
||||
youtube_desc_path = os.path.join(temp_dir, "youtube_description.txt")
|
||||
with open(youtube_desc_path, 'w') as f:
|
||||
f.write(job.youtube_description_template)
|
||||
job_log.info("YouTube description template written to temp file")
|
||||
|
||||
finalise = KaraokeFinalise(
|
||||
logger=logger,
|
||||
log_level=logging.INFO,
|
||||
dry_run=False,
|
||||
instrumental_format="flac",
|
||||
# CDG/TXT generation
|
||||
enable_cdg=getattr(job, 'enable_cdg', False),
|
||||
enable_txt=getattr(job, 'enable_txt', False),
|
||||
cdg_styles=cdg_styles,
|
||||
# Brand code and organization (server-side mode uses rclone)
|
||||
brand_prefix=getattr(job, 'brand_prefix', None),
|
||||
organised_dir=None, # Not used in cloud - files stay in GCS
|
||||
organised_dir_rclone_root=getattr(job, 'organised_dir_rclone_root', None),
|
||||
public_share_dir=None, # Not used in cloud
|
||||
# Notifications
|
||||
discord_webhook_url=getattr(job, 'discord_webhook_url', None),
|
||||
# YouTube upload (server-side with pre-loaded credentials)
|
||||
youtube_client_secrets_file=None, # Not used with pre-stored credentials
|
||||
youtube_description_file=youtube_desc_path,
|
||||
user_youtube_credentials=youtube_credentials, # Pre-loaded from Secret Manager
|
||||
rclone_destination=None,
|
||||
email_template_file=None,
|
||||
# Server-side optimizations
|
||||
non_interactive=True,
|
||||
server_side_mode=True,
|
||||
selected_instrumental_file=instrumental_file,
|
||||
# Audio synchronization - ensure instrumental matches vocal padding
|
||||
countdown_padding_seconds=countdown_padding_seconds,
|
||||
)
|
||||
|
||||
# Call process() - this does ALL the work:
|
||||
# - Encodes to 4 video formats
|
||||
# - Generates CDG/TXT packages if enabled
|
||||
# - Posts Discord notification if configured
|
||||
# - Handles brand code generation
|
||||
job_log.info("Starting KaraokeFinalise.process() - this may take 15-20 minutes...")
|
||||
logger.info(f"Job {job_id}: Starting KaraokeFinalise.process()")
|
||||
result = finalise.process()
|
||||
|
||||
job_log.info("KaraokeFinalise.process() complete!")
|
||||
if result.get('brand_code'):
|
||||
job_log.info(f"Brand code: {result.get('brand_code')}")
|
||||
logger.info(f"Job {job_id}: KaraokeFinalise.process() complete")
|
||||
logger.info(f"Job {job_id}: Brand code: {result.get('brand_code')}")
|
||||
|
||||
# Native API distribution uploads (used by remote CLI instead of rclone)
|
||||
await _handle_native_distribution(
|
||||
job_id=job_id,
|
||||
job=job,
|
||||
job_log=job_log,
|
||||
job_manager=job_manager,
|
||||
temp_dir=temp_dir,
|
||||
result=result,
|
||||
)
|
||||
|
||||
# Upload generated files to GCS
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.PACKAGING,
|
||||
progress=95,
|
||||
message="Uploading final files"
|
||||
)
|
||||
|
||||
await _upload_results(job_id, job_manager, storage, temp_dir, result)
|
||||
|
||||
# Mark job as complete
|
||||
logger.info(f"Job {job_id}: Video generation complete")
|
||||
job_manager.transition_to_state(
|
||||
job_id=job_id,
|
||||
new_status=JobStatus.COMPLETE,
|
||||
progress=100,
|
||||
message="Karaoke generation complete!"
|
||||
)
|
||||
|
||||
# Store result metadata in job
|
||||
job_manager.update_job(job_id, {
|
||||
'state_data': {
|
||||
**job.state_data,
|
||||
'brand_code': result.get('brand_code'),
|
||||
'youtube_url': result.get('youtube_url'),
|
||||
}
|
||||
})
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Job {job_id}: Video generation failed: {e}", exc_info=True)
|
||||
@@ -307,6 +335,13 @@ async def _handle_native_distribution(
|
||||
brand_code = result.get('brand_code')
|
||||
base_name = f"{job.artist} - {job.title}"
|
||||
|
||||
# Check if we should preserve existing brand code (Batch 6: --keep-brand-code)
|
||||
keep_brand_code = getattr(job, 'keep_brand_code', None)
|
||||
if keep_brand_code:
|
||||
brand_code = keep_brand_code
|
||||
result['brand_code'] = brand_code
|
||||
job_log.info(f"Using preserved brand code: {brand_code}")
|
||||
|
||||
# Upload to Dropbox using native SDK
|
||||
dropbox_path = getattr(job, 'dropbox_path', None)
|
||||
brand_prefix = getattr(job, 'brand_prefix', None)
|
||||
@@ -487,15 +522,26 @@ async def _setup_working_directory(
|
||||
storage.download_file(lyrics_video_url, lyrics_video_path)
|
||||
log_progress("Downloaded karaoke video")
|
||||
|
||||
# Download selected instrumental
|
||||
# Download instrumental - either user-provided existing instrumental or AI-separated
|
||||
existing_instrumental_path = getattr(job, 'existing_instrumental_gcs_path', None)
|
||||
instrumental_selection = job.state_data['instrumental_selection']
|
||||
instrumental_key = 'instrumental_clean' if instrumental_selection == 'clean' else 'instrumental_with_backing'
|
||||
instrumental_url = job.file_urls['stems'][instrumental_key]
|
||||
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
|
||||
instrumental_path = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
|
||||
log_progress(f"Downloading {instrumental_selection} instrumental audio...")
|
||||
storage.download_file(instrumental_url, instrumental_path)
|
||||
log_progress(f"Downloaded instrumental ({instrumental_selection})")
|
||||
|
||||
if existing_instrumental_path:
|
||||
# Batch 3: Use user-provided existing instrumental
|
||||
ext = Path(existing_instrumental_path).suffix.lower()
|
||||
instrumental_path = os.path.join(temp_dir, f"{base_name} (Instrumental User){ext}")
|
||||
log_progress(f"Downloading user-provided existing instrumental...")
|
||||
storage.download_file(existing_instrumental_path, instrumental_path)
|
||||
log_progress("Downloaded user-provided instrumental")
|
||||
else:
|
||||
# Use AI-separated instrumental based on selection
|
||||
instrumental_key = 'instrumental_clean' if instrumental_selection == 'clean' else 'instrumental_with_backing'
|
||||
instrumental_url = job.file_urls['stems'][instrumental_key]
|
||||
instrumental_suffix = "Clean" if instrumental_selection == 'clean' else "Backing"
|
||||
instrumental_path = os.path.join(temp_dir, f"{base_name} (Instrumental {instrumental_suffix}).flac")
|
||||
log_progress(f"Downloading {instrumental_selection} instrumental audio...")
|
||||
storage.download_file(instrumental_url, instrumental_path)
|
||||
log_progress(f"Downloaded instrumental ({instrumental_selection})")
|
||||
|
||||
# Download LRC file if available (needed for CDG/TXT)
|
||||
lyrics_urls = job.file_urls.get('lyrics', {})
|
||||
@@ -549,3 +595,64 @@ async def _upload_results(
|
||||
logger.info(f"Job {job_id}: Uploaded {file_key}")
|
||||
except Exception as e:
|
||||
logger.error(f"Job {job_id}: Failed to upload {file_key}: {e}")
|
||||
|
||||
|
||||
# ==================== CLI Entry Point for Cloud Run Jobs ====================
|
||||
# This allows the video worker to be run as a standalone Cloud Run Job.
|
||||
# Usage: python -m backend.workers.video_worker --job-id <job_id>
|
||||
|
||||
def main():
|
||||
"""
|
||||
CLI entry point for running video worker as a Cloud Run Job.
|
||||
|
||||
This is used when video encoding needs more than 30 minutes
|
||||
(the Cloud Tasks timeout limit). Cloud Run Jobs support up to 24 hours.
|
||||
|
||||
Usage:
|
||||
python -m backend.workers.video_worker --job-id abc123
|
||||
|
||||
Environment Variables:
|
||||
GOOGLE_CLOUD_PROJECT: GCP project ID (required)
|
||||
GCS_BUCKET_NAME: Storage bucket name (required)
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import sys
|
||||
|
||||
# Set up argument parser
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Video encoding worker for karaoke generation"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--job-id",
|
||||
required=True,
|
||||
help="Job ID to process"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
job_id = args.job_id
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
)
|
||||
|
||||
logger.info(f"Starting video worker CLI for job {job_id}")
|
||||
|
||||
# Run the async worker function
|
||||
try:
|
||||
success = asyncio.run(generate_video(job_id))
|
||||
if success:
|
||||
logger.info(f"Video generation completed successfully for job {job_id}")
|
||||
sys.exit(0)
|
||||
else:
|
||||
logger.error(f"Video generation failed for job {job_id}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
logger.error(f"Video worker crashed: {e}", exc_info=True)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -7,11 +7,48 @@ for remote debugging via the CLI.
|
||||
Two approaches are provided:
|
||||
1. JobLogger - A custom logger class for explicit logging in workers
|
||||
2. JobLogHandler - A logging.Handler that captures logs from any logger (including dependencies)
|
||||
|
||||
IMPORTANT: Uses contextvars to ensure log isolation between concurrent jobs.
|
||||
When multiple jobs run in parallel on the same Cloud Run instance, each job's
|
||||
worker logs are correctly filtered to only include logs from that job.
|
||||
"""
|
||||
import contextvars
|
||||
import logging
|
||||
from contextlib import contextmanager
|
||||
from typing import Optional, Set
|
||||
|
||||
|
||||
# Context variable to track the current job being processed
|
||||
# This is used to filter logs when multiple jobs run concurrently
|
||||
_current_job_id: contextvars.ContextVar[Optional[str]] = contextvars.ContextVar(
|
||||
'current_job_id', default=None
|
||||
)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def job_logging_context(job_id: str):
|
||||
"""
|
||||
Context manager to set the current job ID for log filtering.
|
||||
|
||||
When multiple jobs run concurrently on the same Cloud Run instance,
|
||||
this ensures that each job's handler only captures logs from its own
|
||||
processing thread/context.
|
||||
|
||||
Usage:
|
||||
with job_logging_context(job_id):
|
||||
# All logs emitted here will be associated with this job_id
|
||||
process_job(job_id)
|
||||
|
||||
Args:
|
||||
job_id: The job ID to set as the current context
|
||||
"""
|
||||
token = _current_job_id.set(job_id)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
_current_job_id.reset(token)
|
||||
|
||||
|
||||
class JobLogHandler(logging.Handler):
|
||||
"""
|
||||
A logging handler that captures log records and stores them in Firestore.
|
||||
@@ -53,6 +90,15 @@ class JobLogHandler(logging.Handler):
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
"""Process a log record."""
|
||||
try:
|
||||
# CRITICAL: Filter by current job context to prevent log mixing
|
||||
# When multiple jobs run concurrently, each attaches handlers to
|
||||
# the same global loggers. Without this check, logs from Job A
|
||||
# would be captured by Job B's handler and vice versa.
|
||||
current_job = _current_job_id.get()
|
||||
if current_job is not None and current_job != self.job_id:
|
||||
# This log is from a different job's context, skip it
|
||||
return
|
||||
|
||||
# Format the log message
|
||||
message = self.format(record)
|
||||
|
||||
@@ -182,6 +228,10 @@ def setup_job_logging(job_id: str, worker_name: str, *logger_names: str) -> JobL
|
||||
This adds a JobLogHandler to capture logs from specified loggers
|
||||
(including dependency loggers like lyrics_transcriber).
|
||||
|
||||
IMPORTANT: When using this function, wrap your job processing code in
|
||||
`job_logging_context(job_id)` to ensure proper log isolation when multiple
|
||||
jobs run concurrently on the same Cloud Run instance.
|
||||
|
||||
Args:
|
||||
job_id: Job ID
|
||||
worker_name: Worker name
|
||||
@@ -199,7 +249,10 @@ def setup_job_logging(job_id: str, worker_name: str, *logger_names: str) -> JobL
|
||||
"lyrics_transcriber", # Capture LyricsTranscriber logs
|
||||
)
|
||||
|
||||
# ... do work ...
|
||||
# IMPORTANT: Use job_logging_context for proper isolation
|
||||
with job_logging_context(job_id):
|
||||
# ... do work ...
|
||||
pass
|
||||
|
||||
# Optional: remove handler when done
|
||||
for name in logger_names:
|
||||
|
||||
@@ -1,22 +1,23 @@
|
||||
# Cloud Build configuration for base image
|
||||
#
|
||||
# This builds the base image with all Python dependencies pre-installed.
|
||||
# Run this manually when poetry.lock changes:
|
||||
# Run this manually when poetry.lock OR Dockerfile.base changes:
|
||||
# gcloud builds submit --config=cloudbuild-base.yaml --project=nomadkaraoke
|
||||
#
|
||||
# The base image is used by the main Dockerfile for fast app builds.
|
||||
# The poetry.lock hash is stored as a label for change detection.
|
||||
# A combined hash of poetry.lock + Dockerfile.base is stored as a label for change detection.
|
||||
|
||||
steps:
|
||||
# Calculate poetry.lock hash for change detection
|
||||
# Calculate combined hash for change detection (poetry.lock + Dockerfile.base)
|
||||
- name: 'gcr.io/cloud-builders/docker'
|
||||
entrypoint: 'bash'
|
||||
args:
|
||||
- '-c'
|
||||
- |
|
||||
POETRY_LOCK_HASH=$$(sha256sum poetry.lock | cut -d' ' -f1)
|
||||
echo "Poetry.lock hash: $$POETRY_LOCK_HASH"
|
||||
echo "$$POETRY_LOCK_HASH" > /workspace/POETRY_LOCK_HASH
|
||||
# Hash both files to detect changes in either
|
||||
CONTENT_HASH=$$(cat poetry.lock backend/Dockerfile.base | sha256sum | cut -d' ' -f1)
|
||||
echo "Combined content hash (poetry.lock + Dockerfile.base): $$CONTENT_HASH"
|
||||
echo "$$CONTENT_HASH" > /workspace/CONTENT_HASH
|
||||
|
||||
# Build base image with all dependencies
|
||||
- name: 'gcr.io/cloud-builders/docker'
|
||||
@@ -24,11 +25,11 @@ steps:
|
||||
args:
|
||||
- '-c'
|
||||
- |
|
||||
POETRY_LOCK_HASH=$$(cat /workspace/POETRY_LOCK_HASH)
|
||||
CONTENT_HASH=$$(cat /workspace/CONTENT_HASH)
|
||||
docker build \
|
||||
--build-arg BUILD_DATE=$BUILD_ID \
|
||||
--build-arg BUILD_ID=$BUILD_ID \
|
||||
--build-arg POETRY_LOCK_HASH=$$POETRY_LOCK_HASH \
|
||||
--build-arg BASE_CONTENT_HASH=$$CONTENT_HASH \
|
||||
-t us-central1-docker.pkg.dev/$PROJECT_ID/karaoke-repo/karaoke-backend-base:latest \
|
||||
-t us-central1-docker.pkg.dev/$PROJECT_ID/karaoke-repo/karaoke-backend-base:$BUILD_ID \
|
||||
-f backend/Dockerfile.base \
|
||||
|
||||
@@ -54,6 +54,7 @@ steps:
|
||||
- 'us-central1-docker.pkg.dev/$PROJECT_ID/karaoke-repo/karaoke-backend:latest'
|
||||
|
||||
# Deploy to Cloud Run with version from pyproject.toml
|
||||
# Uses Cloud Tasks for worker coordination (scalable architecture)
|
||||
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
|
||||
entrypoint: 'bash'
|
||||
args:
|
||||
@@ -71,11 +72,11 @@ steps:
|
||||
--cpu-boost \
|
||||
--execution-environment gen2 \
|
||||
--timeout 1800 \
|
||||
--concurrency 4 \
|
||||
--max-instances 10 \
|
||||
--concurrency 1 \
|
||||
--max-instances 50 \
|
||||
--min-instances 0 \
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=$PROJECT_ID,GCS_BUCKET_NAME=karaoke-gen-storage-$PROJECT_ID,FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest"
|
||||
--set-env-vars "GOOGLE_CLOUD_PROJECT=$PROJECT_ID,GCS_BUCKET_NAME=karaoke-gen-storage-$PROJECT_ID,FIRESTORE_COLLECTION=jobs,ENVIRONMENT=production,AUDIO_SEPARATOR_API_URL=https://nomadkaraoke--audio-separator-api.modal.run,KARAOKE_GEN_VERSION=$$VERSION,DEFAULT_DROPBOX_PATH=/MediaUnsynced/Karaoke/Tracks-Organized,DEFAULT_GDRIVE_FOLDER_ID=1laRKAyxo0v817SstfM5XkpbWiNKNAMSX,ENABLE_CLOUD_TASKS=true,CLOUD_RUN_SERVICE_URL=https://api.nomadkaraoke.com,GCP_REGION=us-central1" \
|
||||
--set-secrets "AUDIOSHAKE_API_TOKEN=audioshake-api-key:latest,GENIUS_API_TOKEN=genius-api-key:latest,SPOTIFY_COOKIE_SP_DC=spotify-cookie:latest,RAPIDAPI_KEY=rapidapi-key:latest,DEFAULT_DISCORD_WEBHOOK_URL=discord-releases-webhook:latest,ADMIN_TOKENS=admin-tokens:latest,REDACTED_API_KEY=redacted-api-key:latest,OPS_API_KEY=ops-api-key:latest,FLACFETCH_API_URL=flacfetch-api-url:latest,FLACFETCH_API_KEY=flacfetch-api-key:latest"
|
||||
|
||||
images:
|
||||
- 'us-central1-docker.pkg.dev/$PROJECT_ID/karaoke-repo/karaoke-backend:latest'
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
# Codecov Configuration
|
||||
# Docs: https://docs.codecov.com/docs/codecovyml-reference
|
||||
|
||||
coverage:
|
||||
# Coverage value precision (2 decimal places)
|
||||
precision: 2
|
||||
round: down
|
||||
# Green range starts at 70% (our target)
|
||||
range: "70...100"
|
||||
|
||||
status:
|
||||
# Project-level coverage status check
|
||||
project:
|
||||
default:
|
||||
target: 70% # Pass if coverage >= 70%
|
||||
threshold: 2% # Allow 2% drop without failing
|
||||
if_ci_failed: error
|
||||
|
||||
# Patch coverage (new/changed lines only)
|
||||
patch:
|
||||
default:
|
||||
target: 70% # New code should also meet 70%
|
||||
threshold: 5% # More lenient for patches
|
||||
|
||||
# Disable PR comments entirely since we're satisfied with 70%+ coverage
|
||||
# Status checks will still appear on the PR
|
||||
comment: false
|
||||
|
||||
# Alternative: Uncomment below to only comment when coverage changes
|
||||
# comment:
|
||||
# layout: "diff, flags, files"
|
||||
# behavior: default
|
||||
# require_changes: true # Only comment if coverage actually changes
|
||||
# require_base: true # Require base report to exist
|
||||
|
||||
@@ -1,10 +1,32 @@
|
||||
# Backend Feature Parity Plan
|
||||
|
||||
**Last Updated:** 2024-12-10
|
||||
**Status:** ✅ Core Feature Parity Achieved (v0.71.0)
|
||||
**Last Updated:** 2024-12-11
|
||||
**Status:** ✅ Core Feature Parity Achieved (v0.71.0) | Batch 1 ✅ | Batch 2 ✅ | Batch 3 ✅ | Batch 4 ✅ | Batch 5 ✅
|
||||
|
||||
This document tracks the progress toward complete feature parity between the local `karaoke-gen` CLI and the cloud backend, enabling the `karaoke-gen-remote` CLI to have equivalent functionality.
|
||||
|
||||
### Quick Parity Summary
|
||||
|
||||
| Category | Supported | Needed | Parity | Notes |
|
||||
|----------|-----------|--------|--------|-------|
|
||||
| Core Processing | 12 | 12 | **100%** | ✅ Complete |
|
||||
| Distribution | 5 | 5 | **100%** | ✅ Complete |
|
||||
| Lyrics Configuration | 5 | 5 | **100%** | ✅ **Batch 1 Complete** (L1-L4) |
|
||||
| Style Configuration | 1 | 4 | **25%** | S2-S4 are MEDIUM priority |
|
||||
| Workflow Control | 0 | 3 | **0%** | W1/W2 HIGH, W7 MEDIUM (skip flags deferred) |
|
||||
| Audio Processing | 4 | 4 | **100%** | ✅ **Batch 2 & 3 Complete** (AP1-AP3, AP5) |
|
||||
| Input Modes | 3 | 3 | **100%** | ✅ **Batch 4 & 5 Complete** (P2, P3) |
|
||||
| Audio Fetching | 1 | 1 | **100%** | ✅ **Batch 5 Complete** (A1) |
|
||||
| Flags to Remove | - | - | - | I3-I6, AP6 (simplify codebase) |
|
||||
|
||||
**Overall:** Core workflow complete. See [CLI Parameter Parity Analysis](#-cli-parameter-parity-analysis) for detailed parameter list with implementation plans.
|
||||
|
||||
**Priority Summary:**
|
||||
- **HIGH**: L1-L4 ✅, AP1-AP5 ✅, P3 ✅, A1 ✅ | P2 (YouTube URLs), W1/W2 (prep/finalise phases), F15 (keep-brand-code)
|
||||
- **MEDIUM**: S2-S4 (style overrides/video bg), W7 (edit-lyrics), F14 (email template)
|
||||
- **LOW/DEFERRED**: W3-W6 (skip flags), D2-D3 (debugging)
|
||||
- **REMOVE**: I3, I4, I5, I6, AP6 (unnecessary options)
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Milestone: First Successful End-to-End Run
|
||||
@@ -39,6 +61,521 @@ karaoke-gen-remote \
|
||||
|
||||
---
|
||||
|
||||
## 📊 CLI Parameter Parity Analysis
|
||||
|
||||
This section provides a comprehensive comparison of all CLI parameters between `karaoke-gen` (local) and `karaoke-gen-remote`, including detailed explanations of what each parameter does and how it could work remotely.
|
||||
|
||||
**Legend:**
|
||||
- ✅ Fully supported
|
||||
- ⚠️ Partially supported (limited functionality)
|
||||
- ❌ Not supported (ignored or errors)
|
||||
- 🔹 Remote-only (not applicable to local CLI)
|
||||
- N/A Not applicable to this mode
|
||||
|
||||
---
|
||||
|
||||
### Positional Arguments
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| P1 | `<file>` | ✅ | ✅ | **Local audio file path.** Copies file to output directory, converts to WAV for processing. Supports MP3, FLAC, WAV, M4A, OGG, AAC. |
|
||||
| P2 | `<url>` (YouTube) | ✅ | ✅ | **YouTube/online URL.** Uses yt-dlp to download audio and extract metadata (artist/title from video title). |
|
||||
| P3 | `<artist> <title>` | ✅ | ✅ | **Audio search mode.** When no file is provided, uses flacfetch to search Deezer/Tidal/etc for high-quality audio and downloads it. |
|
||||
| P4 | `<folder>` | ✅ | ❌ | **Batch folder processing.** Process all audio files in a folder, using `--filename_pattern` to extract track titles from filenames. |
|
||||
|
||||
**Remote Implementation Status:** ✅ **BATCH 4 & 5 COMPLETE** (v0.71.42)
|
||||
- **P2 YouTube URLs**: ✅ IMPLEMENTED. Backend accepts URL via `/api/jobs/create-from-url` endpoint. Uses yt-dlp to download audio and extract metadata (artist/title). Remote CLI detects URLs and routes to URL endpoint.
|
||||
- **P3 Flacfetch search**: ✅ IMPLEMENTED. Remote CLI submits artist+title, backend searches via flacfetch. Returns results for interactive selection (or auto-selects with `-y`/`--auto-download`). New job statuses: `searching_audio`, `awaiting_audio_selection`, `downloading_audio`.
|
||||
- **P4 Folder batch**: LOW PRIORITY. Could accept ZIP upload, process each as separate job. Defer for now.
|
||||
|
||||
---
|
||||
|
||||
### Workflow Control
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| W1 | `--prep-only` | ✅ | ❌ | **Stop after preparation phase.** Downloads audio, separates stems, transcribes lyrics, creates title/end screens, but does NOT run finalisation (encoding, YouTube upload, etc.). Useful for reviewing intermediate outputs before committing to final render. |
|
||||
| W2 | `--finalise-only` | ✅ | ❌ | **Run only finalisation phase.** Must be run from a directory that was previously prepared with `--prep-only`. Picks up where prep left off: encodes videos, creates CDG/TXT packages, uploads to YouTube/Dropbox, etc. |
|
||||
| W3 | `--skip-transcription` | ✅ | ❌ | **Skip automatic lyrics transcription.** Skips the AudioShake transcription and auto-correction steps. Use this if you want to manually provide lyrics or if transcription keeps failing. Lyrics review UI will still open if you have existing lyrics files. |
|
||||
| W4 | `--skip-separation` | ✅ | ❌ | **Skip audio separation.** Skips the AI-powered stem separation (Modal API). Useful when re-processing a track where you already have the stems from a previous run, or when combined with `--existing_instrumental`. |
|
||||
| W5 | `--skip-lyrics` | ✅ | ❌ | **Skip all lyrics processing.** Skips fetching lyrics from Genius/Spotify, transcription, and review. Output will have no lyrics overlay - just instrumental video with title/end screens. |
|
||||
| W6 | `--lyrics-only` | ✅ | ❌ | **Process only lyrics.** Sets `--skip-separation` and skips title/end screen generation. Useful for re-doing just the lyrics on an existing track without re-running expensive audio separation. |
|
||||
| W7 | `--edit-lyrics` | ✅ | ❌ | **Edit lyrics of existing track.** Run from inside an existing track directory (e.g., `NOMAD-1234 - Artist - Title/`). Backs up existing outputs, re-runs lyrics transcription with the existing audio, then re-renders and re-uploads. Uses `--keep-brand-code` implicitly. |
|
||||
| W8 | `--resume` | N/A | 🔹 | **Resume monitoring a job.** Reconnects to an existing remote job and continues monitoring its progress, handling review/instrumental selection if needed. |
|
||||
| W9 | `--cancel` | N/A | 🔹 | **Cancel a running job.** Stops processing but keeps the job record in Firestore. Useful for aborting stuck jobs. |
|
||||
| W10 | `--retry` | N/A | 🔹 | **Retry a failed job.** Restarts processing from the last successful checkpoint. Only works on jobs with `failed` status. |
|
||||
| W11 | `--delete` | N/A | 🔹 | **Delete a job.** Permanently removes job record from Firestore and all associated files from GCS. |
|
||||
| W12 | `--list` | N/A | 🔹 | **List all jobs.** Shows all jobs with their status, artist, title. Supports filtering with `--filter-environment` and `--filter-client-id`. |
|
||||
|
||||
**Remote Implementation Plan:**
|
||||
- **W1 `--prep-only`**: HIGH PRIORITY. Remote CLI submits job with `prep_only=true`. Backend runs all prep tasks (audio separation, lyrics transcription with review UI, initial render). After completion, CLI downloads the entire output folder and exits. No finalisation (encoding, YouTube, Dropbox, etc.) runs. Use case: Client needs custom edits to stems or lyrics before final render.
|
||||
- **W2 `--finalise-only`**: HIGH PRIORITY. Remote CLI checks current directory for expected prep output files. Uploads ALL files from that folder to backend (including any manual customizations user made since prep). Backend runs finalisation and distribution steps. Use case: User edited instrumental manually, now wants cloud to handle encoding/distribution.
|
||||
- **W3-W6 Skip flags**: LOW PRIORITY. Rarely used; typically all together to regenerate title/end screens with different artist/title. Defer until pipeline refactor provides cleaner stage-targeting.
|
||||
- **W7 `--edit-lyrics`**: MEDIUM PRIORITY. Re-runs lyrics worker with existing audio/stems. Creates NEW YouTube upload (doesn't delete/replace existing). Backend should handle gracefully if video with identical title already exists (user will have manually deleted old one). Use case: Customer found typo in published video.
|
||||
|
||||
---
|
||||
|
||||
### Logging & Debugging
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| D1 | `--log_level` | ✅ | ✅ | **Set logging verbosity.** Options: `debug`, `info`, `warning`, `error`. Default: `info`. Affects console output detail level. |
|
||||
| D2 | `--dry_run` | ✅ | ❌ | **Simulate without changes.** Runs through the workflow logic but doesn't actually download, process, or create files. Useful for testing argument combinations. |
|
||||
| D3 | `--render_bounding_boxes` | ✅ | ❌ | **Debug text positioning.** Renders red bounding boxes around text regions in title/end screen images. Helps debug custom style configurations. |
|
||||
|
||||
**Remote Implementation Plan:**
|
||||
- **D2 `--dry_run`**: LOW PRIORITY. Backend could accept `dry_run=true`, validate inputs and return what WOULD happen without processing. Nice to have for testing.
|
||||
- **D3 `--render_bounding_boxes`**: LOW PRIORITY. Backend could enable this, useful for testing styles before full render. Nice to have for debugging style configs.
|
||||
|
||||
---
|
||||
|
||||
### Input/Output Configuration
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| I1 | `--filename_pattern` | ✅ | N/A | **Regex for batch processing.** Python regex with named group `(?P<title>...)` to extract track title from filenames. Used with folder input. Example: `'(?P<index>\d+) - (?P<title>.+).mp3'` |
|
||||
| I2 | `--output_dir` | ✅ | ✅ | **Output directory.** Where to write output files. Local: creates subdirectory per track. Remote: where to download completed files. Default: current directory. |
|
||||
| I3 | `--no_track_subfolders` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Flat output structure. Not needed - always create subfolders. |
|
||||
| I4 | `--lossless_output_format` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Stem audio format. FLAC is always fine, no need for WAV option. |
|
||||
| I5 | `--output_png` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Always output both PNG and JPG, no need for flag. |
|
||||
| I6 | `--output_jpg` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** Always output both PNG and JPG, no need for flag. |
|
||||
|
||||
**Remote Implementation Plan:**
|
||||
- **I3, I4, I5, I6**: REMOVE from local CLI entirely. Not worth maintaining. Backend always uses FLAC, outputs both PNG+JPG, creates subfolders.
|
||||
|
||||
---
|
||||
|
||||
### Audio Fetching Configuration
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| A1 | `--auto-download` | ✅ | ✅ | **Auto-select audio source.** When using artist+title search mode (flacfetch), automatically select the best quality source instead of prompting for manual selection. Useful for scripted/batch processing. |
|
||||
|
||||
**Remote Implementation Status:** ✅ **BATCH 5 COMPLETE** (v0.71.28)
|
||||
- **A1 `--auto-download`**: ✅ IMPLEMENTED. Backend supports BOTH modes:
|
||||
- **Interactive (default)**: API returns search results to client. Client displays options, user selects via `/api/audio-search/{job_id}/select` endpoint.
|
||||
- **Auto-download (`--auto-download` or `-y`)**: Backend auto-selects best quality source using flacfetch's ranking.
|
||||
|
||||
---
|
||||
|
||||
### Audio Processing Configuration
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| AP1 | `--clean_instrumental_model` | ✅ | ✅ | **Stage 1 separation model.** AI model for initial vocal/instrumental separation. Default: `model_bs_roformer_ep_317_sdr_12.9755.ckpt`. Different models have different quality/speed tradeoffs. |
|
||||
| AP2 | `--backing_vocals_models` | ✅ | ✅ | **Stage 2 separation model(s).** AI model(s) for separating lead vocals from backing vocals. Default: `mel_band_roformer_karaoke_aufr33_viperx_sdr_10.1956.ckpt`. |
|
||||
| AP3 | `--other_stems_models` | ✅ | ✅ | **Multi-stem separation model.** AI model for separating drums, bass, guitar, piano, other. Default: `htdemucs_6s.yaml`. |
|
||||
| AP4 | `--model_file_dir` | ✅ | N/A | **Model cache directory.** Where to store/load AI model files. Default: `/tmp/audio-separator-models/`. |
|
||||
| AP5 | `--existing_instrumental` | ✅ | ✅ | **Use pre-made instrumental.** Path to an existing instrumental file to use instead of running AI separation. Useful when you have a better instrumental from another source (official instrumental release, manual editing, etc.). Backend still runs separation for stems, but uses provided instrumental for final video. |
|
||||
| AP6 | `--instrumental_format` | 🗑️ | N/A | **REMOVE FROM LOCAL CLI.** FLAC is always fine, no need for format option. |
|
||||
|
||||
**Remote Implementation Status:** ✅ **BATCH 2 & 3 COMPLETE** (PR #9 v0.71.24, PR #10 v0.71.26)
|
||||
- **AP1 `--clean_instrumental_model`**: ✅ IMPLEMENTED. Sent as form field to backend. Audio worker passes to Modal API.
|
||||
- **AP2 `--backing_vocals_models`**: ✅ IMPLEMENTED. Sent as comma-separated list, parsed and passed to Modal API.
|
||||
- **AP3 `--other_stems_models`**: ✅ IMPLEMENTED. Sent as comma-separated list, parsed and passed to Modal API.
|
||||
- **AP4 `--model_file_dir`**: N/A for remote - Modal API handles model caching internally.
|
||||
- **AP5 `--existing_instrumental`**: ✅ IMPLEMENTED (Batch 3). User uploads instrumental file with job. Backend validates duration match (±0.5s), still runs separation for stems, but uses provided instrumental for final video encoding.
|
||||
- **AP6**: REMOVE from local CLI - FLAC is always fine.
|
||||
|
||||
---
|
||||
|
||||
### Lyrics Configuration
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| L1 | `--lyrics_artist` | ✅ | ✅ | **Override artist for lyrics search.** Use different artist name when searching Genius/Spotify/Musixmatch for lyrics. Useful for covers, remixes, or when artist name differs in lyrics databases. Example: `--lyrics_artist="The Beatles"` when your file says "Beatles, The". |
|
||||
| L2 | `--lyrics_title` | ✅ | ✅ | **Override title for lyrics search.** Use different song title when searching for lyrics. Useful for songs with alternate titles, subtitle variations, or "(Remastered)" suffixes. Example: `--lyrics_title="Hey Jude"` when file says "Hey Jude - 2009 Remaster". |
|
||||
| L3 | `--lyrics_file` | ✅ | ✅ | **Use existing lyrics file.** Path to a text file containing lyrics to use instead of fetching from online sources. Supports TXT, DOCX, RTF formats. Useful when online lyrics are wrong or unavailable. Transcription still runs to generate timestamps. |
|
||||
| L4 | `--subtitle_offset_ms` | ✅ | ✅ | **Adjust subtitle timing.** Shift all subtitle timestamps by N milliseconds. Positive values delay subtitles, negative values advance them. Useful when audio has intro padding or sync drift. Example: `--subtitle_offset_ms=500` delays by 0.5 seconds. |
|
||||
| L5 | `--skip_transcription_review` | ✅ | ⚠️ | **Skip review UI.** Don't open the browser-based lyrics review interface after transcription. Use existing auto-corrected lyrics as-is. Remote: use `-y` flag instead for non-interactive mode. |
|
||||
|
||||
**Remote Implementation Status:** ✅ **BATCH 1 COMPLETE** (PR #8, v0.71.22)
|
||||
- **L1 `--lyrics_artist`**: ✅ IMPLEMENTED. Sent as form field to backend. Lyrics worker uses for Genius/Spotify/AudioShake queries.
|
||||
- **L2 `--lyrics_title`**: ✅ IMPLEMENTED. Sent as form field to backend. Lyrics worker uses for search instead of main title.
|
||||
- **L3 `--lyrics_file`**: ✅ IMPLEMENTED. Uploaded with job, stored in GCS. Lyrics worker reads it instead of fetching online.
|
||||
- **L4 `--subtitle_offset_ms`**: ✅ IMPLEMENTED. Sent to backend, render worker applies offset when burning subtitles.
|
||||
- **L5 `--skip_transcription_review`**: Already works via `-y` flag which auto-accepts corrections.
|
||||
|
||||
---
|
||||
|
||||
### Style Configuration
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| S1 | `--style_params_json` | ✅ | ✅ | **Style configuration file.** Path to JSON file defining visual style: background images, fonts, colors, text positioning for intro/karaoke/end screens and CDG. All referenced image/font files are auto-uploaded in remote mode. |
|
||||
| S2 | `--style_override` | ✅ | ❌ | **Quick style tweaks.** Override specific style parameters without editing JSON. Can be used multiple times. Format: `section.key=value`. Example: `--style_override 'intro.background_image=/path/to/new_bg.png'`. Creates a temp JSON file with overrides merged. |
|
||||
| S3 | `--background_video` | ✅ | ❌ | **Video background for karaoke.** Path to video file to use as animated background instead of static image for the karaoke section. Video is looped/trimmed to match audio duration, with lyrics overlaid. |
|
||||
| S4 | `--background_video_darkness` | ✅ | ❌ | **Dim video background.** Darkness overlay percentage (0-100) applied to video background so lyrics are more readable. 0 = no darkening, 100 = completely black. Default: 0. |
|
||||
|
||||
**Remote Implementation Plan:**
|
||||
- **S2 `--style_override`**: MEDIUM PRIORITY. Send override strings to backend, merge server-side before passing to workers. Works same as local (creates merged config). Use case: Web UI will let users tweak font colors etc., needs to pass adjustments to backend.
|
||||
- **S3 `--background_video`**: MEDIUM PRIORITY. Upload video file with job. Backend processes with VideoBackgroundProcessor (loop/trim to audio duration). Requires larger file upload limits. Future: Web UI could allow YouTube URL or direct upload for video background.
|
||||
- **S4 `--background_video_darkness`**: Depends on S3. Send percentage to backend, apply overlay during video processing.
|
||||
|
||||
---
|
||||
|
||||
### Finalisation Configuration
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| F1 | `--enable_cdg` | ✅ | ✅ | **Generate CDG+MP3 package.** Create CD+Graphics karaoke format files (used by karaoke machines). Produces ZIP containing `.cdg` and `.mp3` files. Requires `cdg` section in style_params_json. |
|
||||
| F2 | `--enable_txt` | ✅ | ✅ | **Generate TXT+MP3 package.** Create text-based karaoke format with timed lyrics. Produces ZIP containing `.txt` (with timestamps) and `.mp3` files. |
|
||||
| F3 | `--brand_prefix` | ✅ | ✅ | **Brand code prefix.** Your brand identifier for sequential numbering. When set, output folder is renamed to `PREFIX-XXXX - Artist - Title` where XXXX is the next sequential number found in `organised_dir` or Dropbox. Example: `NOMAD` produces `NOMAD-1234`. |
|
||||
| F4 | `--organised_dir` | ✅ | N/A | **Local organized output.** Local filesystem path where final track folders are moved after processing. Used with `--brand_prefix` to maintain organized library. Example: `/Volumes/Media/Karaoke/Tracks-Organized/`. |
|
||||
| F5 | `--organised_dir_rclone_root` | ✅ | ✅ | **Rclone path for Dropbox.** Rclone remote path that maps to your organised_dir. Used to calculate next brand code number and for rclone-based uploads. Example: `dropbox:Media/Karaoke/Tracks-Organized`. |
|
||||
| F6 | `--public_share_dir` | ✅ | N/A | **Local public share folder.** Local filesystem path where shareable versions (720p video, CDG, TXT) are copied for public distribution. |
|
||||
| F7 | `--enable_youtube_upload` | ✅ | ✅ | **Upload to YouTube.** After encoding, upload the lossy 4K video to YouTube. Local: uses OAuth flow with client secrets. Remote: uses server-side stored credentials. |
|
||||
| F8 | `--youtube_client_secrets_file` | ✅ | N/A | **YouTube OAuth credentials.** Local path to Google OAuth client secrets JSON file for YouTube API. Only needed for local mode - remote uses server-stored credentials. |
|
||||
| F9 | `--youtube_description_file` | ✅ | ✅ | **YouTube video description.** Path to text file containing YouTube video description template. Supports placeholders like `{artist}`, `{title}`, `{brand_code}`. |
|
||||
| F10 | `--rclone_destination` | ✅ | N/A | **Rclone sync destination.** Rclone remote path to sync public_share_dir to. Used for Google Drive public sharing. Example: `googledrive:KaraokePublic`. |
|
||||
| F11 | `--dropbox_path` | N/A | 🔹 | **Dropbox folder (native API).** Remote-only: Dropbox folder path for organized output upload using native Dropbox API. Example: `/Karaoke/Tracks-Organized`. |
|
||||
| F12 | `--gdrive_folder_id` | N/A | 🔹 | **Google Drive folder (native API).** Remote-only: Google Drive folder ID for public share uploads using native API. Example: `1abc123xyz`. |
|
||||
| F13 | `--discord_webhook_url` | ✅ | ✅ | **Discord notification.** Webhook URL for sending completion notifications to Discord. Posts message with track info, brand code, and links. |
|
||||
| F14 | `--email_template_file` | ✅ | ❌ | **Email draft template.** Path to email template file for creating Gmail drafts with track info. Supports placeholders. Uses pyperclip for manual email composition. |
|
||||
| F15 | `--keep-brand-code` | ✅ | ❌ | **Preserve existing brand code.** When run from an existing track directory (with brand code in folder name), use that brand code instead of calculating a new one. Implicitly enabled with `--edit-lyrics`. |
|
||||
| F16 | `-y` / `--yes` | ✅ | ✅ | **Non-interactive mode.** Auto-accept all prompts: lyrics corrections, instrumental selection, confirmations. Useful for automated/CI pipelines. Remote: auto-completes review, selects clean instrumental. |
|
||||
| F17 | `--test_email_template` | ✅ | N/A | **Test email template.** Debug mode: test the email template rendering with fake data without processing any track. |
|
||||
|
||||
**Remote Implementation Plan:**
|
||||
- **F14 `--email_template_file`**: MEDIUM PRIORITY. Two behaviors needed:
|
||||
1. **Finalisation**: Same as local - use template to create Gmail draft (requires Gmail API OAuth). User just changes recipient and sends.
|
||||
2. **API endpoint**: Expose endpoint to fetch templated message text for a job. Use case: Web frontend button to copy message for pasting into Fiverr, etc.
|
||||
- **F15 `--keep-brand-code`**: HIGH PRIORITY (for `--finalise-only` workflow). When using `--finalise-only` remotely, user may be re-processing an already-published track. Must preserve existing brand code so updated files overwrite in Dropbox/GDrive rather than creating new sequence number. Implementation: Remote CLI reads brand code from local folder name, sends to backend.
|
||||
|
||||
---
|
||||
|
||||
### Remote CLI Specific Options
|
||||
|
||||
| # | Parameter | Local | Remote | Description |
|
||||
|---|-----------|-------|--------|-------------|
|
||||
| R1 | `--service-url` | N/A | 🔹 | **Backend service URL.** URL of the karaoke-gen cloud backend. Can also be set via `KARAOKE_GEN_URL` environment variable. Required for remote mode. |
|
||||
| R2 | `--review-ui-url` | N/A | 🔹 | **Lyrics review UI URL.** URL of the hosted lyrics review web application. Default: `https://lyrics.nomadkaraoke.com`. |
|
||||
| R3 | `--poll-interval` | N/A | 🔹 | **Status poll frequency.** Seconds between job status checks while monitoring. Default: 5 seconds. |
|
||||
| R4 | `--environment` | N/A | 🔹 | **Job environment tag.** Tag jobs with environment label (test/production/development) for filtering and cleanup. Sent as `X-Environment` header. |
|
||||
| R5 | `--client-id` | N/A | 🔹 | **Client identifier tag.** Tag jobs with client/user identifier for filtering. Sent as `X-Client-ID` header. |
|
||||
| R6 | `--filter-environment` | N/A | 🔹 | **Filter by environment.** When using `--list` or `--bulk-delete`, only show/delete jobs matching this environment. |
|
||||
| R7 | `--filter-client-id` | N/A | 🔹 | **Filter by client ID.** When using `--list` or `--bulk-delete`, only show/delete jobs matching this client ID. |
|
||||
| R8 | `--bulk-delete` | N/A | 🔹 | **Bulk delete jobs.** Delete all jobs matching filter criteria. Requires at least one filter (`--filter-environment` or `--filter-client-id`). |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Implementation Batches
|
||||
|
||||
The following batches are ordered by priority and grouped by similar functionality. Each batch is designed to be a self-contained unit of work that can be tackled by a single agent session.
|
||||
|
||||
### 📋 Instructions for Implementing a Batch
|
||||
|
||||
**Before starting a batch, ensure you follow these steps:**
|
||||
|
||||
#### 1. Test Coverage Requirements
|
||||
All code changes must have **>70% test coverage** on modified files. Run:
|
||||
```bash
|
||||
python -m pytest backend/tests/ --cov=backend.models.job --cov=backend.api.routes.file_upload --cov=backend.workers.<worker_name> --cov-report=term-missing
|
||||
```
|
||||
|
||||
**Required tests for each batch:**
|
||||
- **Model changes:** Test new fields exist, have correct defaults, serialize properly
|
||||
- **API changes:** Test form validation (valid/invalid inputs), file upload handling
|
||||
- **Worker changes:** Test parameter passing, configuration, error handling
|
||||
- **CLI changes:** Test submit_job() passes parameters correctly
|
||||
|
||||
#### 2. Documentation Updates (REQUIRED)
|
||||
As part of your PR, you MUST update this document:
|
||||
- [ ] Update the **Quick Parity Summary** table to reflect new parity percentages
|
||||
- [ ] Mark parameters as ✅ in the relevant **CLI Parameter Parity Analysis** section
|
||||
- [ ] Update **Remote Implementation Plan/Status** to show ✅ IMPLEMENTED
|
||||
- [ ] Mark the batch as **✅ COMPLETE** with PR number in the **Summary Table**
|
||||
- [ ] Add entry to **Recently Completed** table with version and PR number
|
||||
- [ ] Remove items from **Not Yet Implemented** list
|
||||
|
||||
#### 3. Version Bump
|
||||
Bump the patch version in `pyproject.toml` before committing.
|
||||
|
||||
#### 4. PR Requirements
|
||||
- Title: `feat: Batch N - <description>`
|
||||
- Include: Summary of changes, files modified, tests added
|
||||
- Link to this document in PR description
|
||||
|
||||
---
|
||||
|
||||
### Batch 1: Lyrics Configuration (HIGH) ✅ COMPLETE
|
||||
**Parameters:** L1 `--lyrics_artist`, L2 `--lyrics_title`, L3 `--lyrics_file`, L4 `--subtitle_offset_ms`
|
||||
|
||||
**Status:** ✅ **IMPLEMENTED** - PR #8 merged (v0.71.22)
|
||||
|
||||
**Implementation Summary:**
|
||||
- Remote CLI: Added 4 form fields to `submit_job()`, uploads lyrics file if provided
|
||||
- Backend: Added 4 Form parameters to `/api/jobs/upload` endpoint
|
||||
- Backend: Lyrics worker downloads user lyrics file, passes overrides to LyricsProcessor
|
||||
- Backend: Render worker applies subtitle offset via OutputConfig
|
||||
|
||||
**Files modified:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - submit_job() with lyrics params
|
||||
- `backend/api/routes/file_upload.py` - Form fields + lyrics file upload
|
||||
- `backend/models/job.py` - lyrics_artist, lyrics_title, lyrics_file_gcs_path, subtitle_offset_ms fields
|
||||
- `backend/workers/lyrics_worker.py` - override values passed to LyricsProcessor
|
||||
- `backend/workers/render_video_worker.py` - subtitle_offset_ms applied
|
||||
|
||||
**Tests added:** 25+ unit tests covering lyrics file validation, Job model fields, worker configuration
|
||||
|
||||
**Complexity:** Low-Medium (straightforward form fields + file upload)
|
||||
|
||||
---
|
||||
|
||||
### Batch 2: Audio Separation Model Selection (HIGH) ✅ COMPLETE
|
||||
**Parameters:** AP1 `--clean_instrumental_model`, AP2 `--backing_vocals_models`, AP3 `--other_stems_models`
|
||||
|
||||
**Status:** ✅ **IMPLEMENTED** - PR #9 merged (v0.71.24)
|
||||
|
||||
**Implementation Summary:**
|
||||
- Remote CLI: Added 3 form fields to `submit_job()`, sends models as comma-separated strings
|
||||
- Backend: Added 3 Form parameters to `/api/jobs/upload` endpoint, parses comma-separated lists
|
||||
- Backend: Job model stores model configuration as optional fields
|
||||
- Backend: Audio worker passes model names to AudioProcessor which uses Modal API
|
||||
|
||||
**Files modified:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - submit_job() with model params
|
||||
- `backend/api/routes/file_upload.py` - Form fields + comma parsing
|
||||
- `backend/models/job.py` - clean_instrumental_model, backing_vocals_models, other_stems_models fields
|
||||
- `backend/workers/audio_worker.py` - create_audio_processor() accepts model params from job
|
||||
|
||||
**Tests added:** 15+ unit tests covering Job model fields, form parsing, AudioProcessor configuration
|
||||
|
||||
**Complexity:** Low (Modal API already supports dynamic model selection)
|
||||
|
||||
---
|
||||
|
||||
### Batch 3: Existing Instrumental Support (HIGH) ✅ COMPLETE
|
||||
**Parameters:** AP5 `--existing_instrumental`
|
||||
|
||||
**Status:** ✅ **IMPLEMENTED** - PR #10 merged (v0.71.26)
|
||||
|
||||
**Implementation Summary:**
|
||||
- Remote CLI: Added `--existing_instrumental` parameter, uploads file as `existing_instrumental` type
|
||||
- Backend: Added `existing_instrumental` to VALID_FILE_TYPES with audio extension validation
|
||||
- Backend: Duration validation (±0.5s tolerance) at uploads-complete endpoint using pydub
|
||||
- Backend: Job model has `existing_instrumental_gcs_path` field
|
||||
- Backend: Video worker downloads user-provided instrumental if present, uses it for final encoding
|
||||
|
||||
**Files modified:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - submit_job() with existing_instrumental param
|
||||
- `backend/api/routes/file_upload.py` - VALID_FILE_TYPES, GCS path generation, duration validation
|
||||
- `backend/models/job.py` - existing_instrumental_gcs_path field
|
||||
- `backend/workers/video_worker.py` - downloads and uses user-provided instrumental
|
||||
|
||||
**Tests added:** 20+ unit tests covering model fields, file type validation, GCS path generation, duration validation
|
||||
|
||||
**Complexity:** Medium (file upload + validation + conditional logic)
|
||||
|
||||
---
|
||||
|
||||
### Batch 4: YouTube URL Input (HIGH) ✅ COMPLETE
|
||||
**Parameters:** P2 `<url>` (YouTube/online URL)
|
||||
|
||||
**Status:** ✅ **IMPLEMENTED** (v0.71.26)
|
||||
|
||||
**Implementation Summary:**
|
||||
- Remote CLI: Detects URLs via `is_url()`, routes to `submit_job_from_url()` method
|
||||
- Backend: New `/api/jobs/create-from-url` endpoint accepts URL and optional artist/title
|
||||
- Backend: Audio worker downloads from URL using yt-dlp, auto-detects artist/title from metadata
|
||||
- FileHandler: Added `download_video()` and `extract_metadata_from_url()` methods
|
||||
- Dependencies: yt-dlp added to pyproject.toml
|
||||
|
||||
**Files modified:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - URL detection, new `submit_job_from_url()` method
|
||||
- `karaoke_gen/file_handler.py` - `download_video()` and `extract_metadata_from_url()` methods
|
||||
- `backend/api/routes/file_upload.py` - `CreateJobFromUrlRequest`, `CreateJobFromUrlResponse`, `/api/jobs/create-from-url` endpoint
|
||||
- `backend/workers/audio_worker.py` - Enhanced `download_from_url()` with metadata extraction
|
||||
- `pyproject.toml` - Added yt-dlp dependency
|
||||
|
||||
**Tests added:** 20+ unit tests covering URL validation, request/response models, Job URL field
|
||||
|
||||
**Complexity:** Medium-High (new input mode, yt-dlp integration)
|
||||
|
||||
---
|
||||
|
||||
### Batch 5: Flacfetch Audio Search (HIGH) ✅ COMPLETE
|
||||
**Parameters:** P3 `<artist> <title>` (search mode), A1 `--auto-download`
|
||||
|
||||
**Status:** ✅ **IMPLEMENTED** - PR #13 merged (v0.71.28)
|
||||
|
||||
**Implementation Summary:**
|
||||
- Remote CLI: Detects artist+title without file, calls `/api/audio-search/search` endpoint
|
||||
- Backend: New `AudioSearchService` integrates flacfetch for searching
|
||||
- Backend: New job statuses `searching_audio`, `awaiting_audio_selection`, `downloading_audio`
|
||||
- Backend: Returns search results for interactive selection, or auto-selects with `auto_download=true`
|
||||
- Remote CLI: Displays results interactively, user selects, sends to `/api/audio-search/{job_id}/select`
|
||||
- Backend: Downloads selected audio to GCS, proceeds with normal job flow
|
||||
- Support `--auto-download` / `-y` for non-interactive auto-selection
|
||||
|
||||
**Files modified:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - search flow, display results, `search_audio()`, `select_audio_source()`, `handle_audio_selection()`
|
||||
- `backend/api/routes/audio_search.py` - NEW: search and select endpoints
|
||||
- `backend/services/audio_search_service.py` - NEW: flacfetch integration service
|
||||
- `backend/models/job.py` - New statuses and fields (`audio_search_artist`, `audio_search_title`, `auto_download`)
|
||||
- `backend/main.py` - Register audio_search router
|
||||
|
||||
**Tests added:** 20+ unit tests covering Job model fields, AudioSearchService, state transitions
|
||||
|
||||
**Complexity:** High (new interactive API flow, new service integration)
|
||||
|
||||
---
|
||||
|
||||
### Batch 6: Two-Phase Workflow (HIGH) ✅ COMPLETE
|
||||
**Parameters:** W1 `--prep-only`, W2 `--finalise-only`, F15 `--keep-brand-code`
|
||||
|
||||
**Status:** ✅ Complete (2025-12-11)
|
||||
|
||||
**Implementation Summary:**
|
||||
- **prep-only:** Added `prep_only` field to Job model. When set, backend transitions to new `PREP_COMPLETE` status after review and video rendering (with_vocals.mkv). CLI downloads all prep outputs.
|
||||
- **finalise-only:** Added new endpoint `/api/jobs/create-finalise-only` for uploading prep outputs. Remote CLI detects files in prep folder, uploads them via signed URLs, and monitors finalisation.
|
||||
- **keep-brand-code:** Remote CLI extracts brand code from folder name (e.g., "NOMAD-1234 - Artist - Title"), sends to backend. Video worker uses preserved brand code for distribution.
|
||||
|
||||
**Files modified:**
|
||||
- `backend/models/job.py` - Added `PREP_COMPLETE` status, `prep_only`, `finalise_only`, `keep_brand_code` fields, updated state transitions
|
||||
- `backend/api/routes/file_upload.py` - Added `prep_only`, `keep_brand_code` to job creation, new finalise-only endpoints
|
||||
- `backend/workers/render_video_worker.py` - Conditional transition to PREP_COMPLETE for prep-only jobs
|
||||
- `backend/workers/video_worker.py` - Use preserved brand code when set
|
||||
- `backend/services/job_manager.py` - Pass new fields through job creation
|
||||
- `karaoke_gen/utils/remote_cli.py` - Full implementation of prep-only, finalise-only, and keep-brand-code flows
|
||||
|
||||
**Tests added:**
|
||||
- `backend/tests/test_models.py` - Tests for new fields and PREP_COMPLETE status
|
||||
- `backend/tests/test_file_upload.py` - Tests for finalise-only models and endpoints
|
||||
|
||||
**Complexity:** High (significant workflow changes, new upload flow)
|
||||
|
||||
---
|
||||
|
||||
### Batch 7: Style Overrides (MEDIUM)
|
||||
**Parameters:** S2 `--style_override`
|
||||
|
||||
**Scope:**
|
||||
- Remote CLI: Parse override strings, send as array to backend
|
||||
- Backend: Accept style_override array
|
||||
- Backend: Merge overrides with uploaded style JSON before processing
|
||||
- Handle asset references in overrides (upload referenced files)
|
||||
|
||||
**Files to modify:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - parse and send overrides
|
||||
- `backend/api/routes/file_upload.py` - accept overrides
|
||||
- `backend/services/style_service.py` or similar - merge logic
|
||||
|
||||
**Complexity:** Medium (string parsing, merge logic, asset handling)
|
||||
|
||||
---
|
||||
|
||||
### Batch 8: Video Background (MEDIUM)
|
||||
**Parameters:** S3 `--background_video`, S4 `--background_video_darkness`
|
||||
|
||||
**Scope:**
|
||||
- Remote CLI: Upload video file (larger file support needed)
|
||||
- Backend: Accept video upload, store in GCS
|
||||
- Backend: Render worker uses VideoBackgroundProcessor
|
||||
- Backend: Apply darkness overlay
|
||||
|
||||
**Files to modify:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - upload video file
|
||||
- `backend/api/routes/file_upload.py` - accept video upload (larger limits)
|
||||
- `backend/workers/render_worker.py` - integrate VideoBackgroundProcessor
|
||||
|
||||
**Complexity:** Medium (large file upload, video processing)
|
||||
|
||||
---
|
||||
|
||||
### Batch 9: Edit Lyrics Mode (MEDIUM)
|
||||
**Parameters:** W7 `--edit-lyrics`
|
||||
|
||||
**Scope:**
|
||||
- Remote CLI: Accept job_id for existing job, trigger edit mode
|
||||
- Backend: Load existing job's audio/stems from GCS
|
||||
- Backend: Re-run lyrics worker only
|
||||
- Backend: Re-render with new lyrics
|
||||
- Backend: Create NEW YouTube upload (handle duplicate title gracefully)
|
||||
|
||||
**Files to modify:**
|
||||
- `karaoke_gen/utils/remote_cli.py` - edit mode flow
|
||||
- `backend/api/routes/` - edit endpoint or flag
|
||||
- `backend/workers/orchestrator.py` - partial re-processing logic
|
||||
|
||||
**Complexity:** Medium-High (partial re-processing, state management)
|
||||
|
||||
---
|
||||
|
||||
### Batch 10: Email Template + Cleanup (LOW)
|
||||
**Parameters:** F14 `--email_template_file`, D2 `--dry_run`, D3 `--render_bounding_boxes`
|
||||
**Removal:** I3 `--no_track_subfolders`, I4 `--lossless_output_format`, I5 `--output_png`, I6 `--output_jpg`, AP6 `--instrumental_format`
|
||||
|
||||
**Scope:**
|
||||
- **Email template:** Upload template, backend creates Gmail draft + new API endpoint to fetch formatted text
|
||||
- **Dry run:** Backend validates inputs, returns plan without executing
|
||||
- **Bounding boxes:** Enable debug rendering in screens worker
|
||||
- **Removal:** Remove 5 unnecessary CLI flags from local CLI
|
||||
|
||||
**Files to modify:**
|
||||
- `karaoke_gen/utils/cli_args.py` - remove flags
|
||||
- `karaoke_gen/utils/remote_cli.py` - email template upload
|
||||
- `backend/api/routes/` - email template endpoint
|
||||
- `backend/services/email_service.py` - Gmail API integration
|
||||
|
||||
**Complexity:** Mixed (Gmail OAuth is complex, removals are simple)
|
||||
|
||||
---
|
||||
|
||||
### Summary Table
|
||||
|
||||
| Batch | Parameters | Priority | Complexity | Status |
|
||||
|-------|------------|----------|------------|--------|
|
||||
| 1 | L1-L4 (lyrics config) | HIGH | Low-Medium | ✅ **COMPLETE** (PR #8) |
|
||||
| 2 | AP1-AP3 (model selection) | HIGH | Low | ✅ **COMPLETE** (PR #9) |
|
||||
| 3 | AP5 (existing instrumental) | HIGH | Medium | ✅ **COMPLETE** (PR #10) |
|
||||
| 4 | P2 (YouTube URLs) | HIGH | Medium-High | ✅ **COMPLETE** |
|
||||
| 5 | P3, A1 (flacfetch search) | HIGH | High | ✅ **COMPLETE** (PR #13) |
|
||||
| 6 | W1, W2, F15 (two-phase) | HIGH | High | ⏳ Pending |
|
||||
| 7 | S2 (style overrides) | MEDIUM | Medium | ⏳ Pending |
|
||||
| 8 | S3, S4 (video background) | MEDIUM | Medium | ⏳ Pending |
|
||||
| 9 | W7 (edit-lyrics) | MEDIUM | Medium-High | ⏳ Pending |
|
||||
| 10 | F14, D2, D3, removals | LOW | Mixed | ⏳ Pending |
|
||||
|
||||
**Recommended Order:** ~~1~~ → ~~2~~ → ~~3~~ → ~~4~~ → ~~5~~ → 6 → 7 → 8 → 9 → 10
|
||||
|
||||
Batches 1-4 are quick wins that add significant value. Batches 5-6 are larger but high priority. Batches 7-10 can be done as time permits.
|
||||
|
||||
---
|
||||
|
||||
## ✅ Implementation Decisions (Answered 2024-12-10)
|
||||
|
||||
All questions have been answered and incorporated into the "Remote Implementation Plan" sections above. Summary of key decisions:
|
||||
|
||||
### Workflow Control
|
||||
- **W1/W2**: YES, two-phase workflow needed. `--prep-only` downloads all prep outputs after review, `--finalise-only` uploads local folder (with any manual edits) for cloud finalisation.
|
||||
- **W3-W6 Skip flags**: LOW PRIORITY - rarely used, defer until pipeline refactor provides cleaner stage-targeting.
|
||||
- **W7 Edit-lyrics**: Creates NEW YouTube upload (user deletes old one manually). Backend handles duplicate title gracefully.
|
||||
|
||||
### Input Modes
|
||||
- **P2 YouTube**: HIGH PRIORITY - needed for niche live versions only on YouTube.
|
||||
- **P3 Flacfetch**: HIGH PRIORITY - MUST be interactive via API (not auto-select) for future web UI audio source picker.
|
||||
|
||||
### Audio Processing
|
||||
- **AP1-AP3 Models**: HIGH PRIORITY - all model options must work remotely (Modal API can load any model on demand).
|
||||
- **AP5 Existing Instrumental**: Still run separation (for consistent stem downloads), but use provided instrumental for final video. Validate duration match (±0.5s) at job start before proceeding.
|
||||
|
||||
### I/O & Audio Format Options
|
||||
- **I3, I4, I5, I6, AP6**: REMOVE from local CLI entirely - unnecessary options, simplifies codebase.
|
||||
|
||||
### Audio Fetching
|
||||
- **A1 `--auto-download`**: Backend must support BOTH interactive (default, for web UI) AND auto-select (with `-y` flag) modes.
|
||||
|
||||
### Style
|
||||
- **S2 Style Override**: MEDIUM PRIORITY - needed for web UI style tweaks (font colors, etc.).
|
||||
- **S3/S4 Background Video**: MEDIUM PRIORITY - implement, will eventually allow YouTube URL or direct upload in web UI.
|
||||
|
||||
### Finalisation
|
||||
- **F14 Email Template**: BOTH - create Gmail draft (like local) AND expose API endpoint to fetch templated text (for Fiverr copy-paste).
|
||||
- **F15 Keep-brand-code**: HIGH PRIORITY for `--finalise-only` workflow - preserves brand code so updates overwrite existing Dropbox/GDrive files.
|
||||
|
||||
---
|
||||
|
||||
## Vision
|
||||
|
||||
The karaoke-gen system supports multiple interfaces to the same core functionality:
|
||||
@@ -105,14 +642,27 @@ The karaoke-gen system supports multiple interfaces to the same core functionali
|
||||
| Google Drive upload | ✅ | Service account credentials |
|
||||
| Discord notification | ✅ | Webhook URL from job or default |
|
||||
|
||||
### ⏳ Not Yet Implemented (Lower Priority)
|
||||
### ✅ Recently Completed
|
||||
|
||||
| Feature | Version | PR | Notes |
|
||||
|---------|---------|-----|-------|
|
||||
| Flacfetch audio search | v0.71.42 | #13 | `artist title` (search mode), `--auto-download` - search and download audio by artist+title |
|
||||
| YouTube URL input | v0.71.26 | - | `<url>` positional argument for YouTube/online URLs |
|
||||
| Existing instrumental | v0.71.26 | #10 | `--existing_instrumental` - upload and use user-provided instrumental |
|
||||
| Audio model selection | v0.71.24 | #9 | `--clean_instrumental_model`, `--backing_vocals_models`, `--other_stems_models` |
|
||||
| Lyrics override params | v0.71.22 | #8 | `--lyrics_artist`, `--lyrics_title`, `--lyrics_file`, `--subtitle_offset_ms` |
|
||||
|
||||
### ⏳ Not Yet Implemented
|
||||
|
||||
| Feature | Priority | Notes |
|
||||
|---------|----------|-------|
|
||||
| Style override | MEDIUM | `--style_override` for quick tweaks |
|
||||
| Skip separation/transcription | MEDIUM | Workflow control for re-processing |
|
||||
| Edit lyrics mode | MEDIUM | `--edit-lyrics` for fixing existing tracks |
|
||||
| Background video | MEDIUM | `--background_video` requires video upload |
|
||||
| Two-phase workflow | HIGH | `--prep-only`, `--finalise-only` |
|
||||
| Batch folder processing | LOW | Process multiple files at once |
|
||||
| Gmail draft creation | LOW | Nice-to-have, not blocking |
|
||||
| YouTube URL input | LOW | Requires yt-dlp in container |
|
||||
| Batch processing | LOW | Queue management needed |
|
||||
| Existing instrumental support | LOW | For re-processing existing tracks |
|
||||
|
||||
---
|
||||
|
||||
@@ -131,6 +681,8 @@ The karaoke-gen system supports multiple interfaces to the same core functionali
|
||||
| `dropbox-oauth-credentials` | ✅ | Dropbox API OAuth tokens |
|
||||
| `gdrive-service-account` | ✅ | Google Drive service account |
|
||||
| `discord-webhook-url` | ✅ | Default Discord webhook |
|
||||
| `redacted-api-key` | ⏳ | Redacted tracker API key (flacfetch audio search) |
|
||||
| `ops-api-key` | ⏳ | OPS/Orpheus tracker API key (flacfetch audio search) |
|
||||
|
||||
### Default Distribution Settings
|
||||
|
||||
@@ -168,96 +720,17 @@ karaoke-gen-remote \
|
||||
|
||||
---
|
||||
|
||||
## Future Architecture: Shared Pipeline
|
||||
## Future Architecture
|
||||
|
||||
Once feature parity is achieved, the codebase should be refactored toward a **shared pipeline architecture** where both local and remote execution use the same abstractions.
|
||||
For the comprehensive architecture plan covering both **horizontal scalability** (Cloud Tasks, Cloud Run Jobs) and **software architecture** (shared pipeline, SOLID, DRY), see:
|
||||
|
||||
### Current State (Divergent Paths)
|
||||
**[SCALABLE-ARCHITECTURE-PLAN.md](./SCALABLE-ARCHITECTURE-PLAN.md)**
|
||||
|
||||
```
|
||||
LOCAL CLI CLOUD BACKEND
|
||||
───────── ─────────────
|
||||
KaraokeGen.process() API Routes
|
||||
│ │
|
||||
├─► AudioProcessor ├─► audio_worker.py
|
||||
│ └── Modal API or local │ └── Modal API
|
||||
│ │
|
||||
├─► LyricsProcessor ├─► lyrics_worker.py
|
||||
│ └── Orchestrates everything │ └── Transcription only
|
||||
│ including video generation │
|
||||
│ ├─► screens_worker.py
|
||||
│ │
|
||||
│ ├─► render_video_worker.py
|
||||
│ │ └── OutputGenerator directly
|
||||
│ │
|
||||
└─► KaraokeFinalise └─► video_worker.py
|
||||
└── Encoding, distribution └── KaraokeFinalise
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- Video generation called differently (via LyricsProcessor vs OutputGenerator directly)
|
||||
- LyricsProcessor does too many things (fetching, transcription, video, file management)
|
||||
- Testing requires mocking different things for local vs remote
|
||||
- Bug fixes may need to be applied in multiple places
|
||||
|
||||
### Target State (Shared Pipeline)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ SHARED PIPELINE ARCHITECTURE │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ AudioInput → Separation → Transcription → Review → Render → Finalize │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ ▼ ▼ ▼ ▼ ▼ ▼ │
|
||||
│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │
|
||||
│ │ Stage │ │ Stage │ │ Stage │ │ Stage │ │ Stage │ │ Stage │ │
|
||||
│ │ API │ │ API │ │ API │ │ API │ │ API │ │ API │ │
|
||||
│ └────┬───┘ └────┬───┘ └────┬───┘ └────┬───┘ └────┬───┘ └────┬───┘ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ ┌────┴───────────┴────────────┴────────────┴──────────┴──────────┴────┐ │
|
||||
│ │ EXECUTION LAYER │ │
|
||||
│ │ │ │
|
||||
│ │ Local Mode: │ Remote Mode: │ │
|
||||
│ │ - Direct calls │ - HTTP to backend │ │
|
||||
│ │ - Local GPU/CPU │ - Workers + Modal │ │
|
||||
│ │ - Blocking │ - Async + polling │ │
|
||||
│ └──────────────────────┴───────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Single source of truth for each pipeline stage
|
||||
- Each stage independently testable
|
||||
- Same business logic regardless of execution mode
|
||||
- Easier to add new stages or modify existing ones
|
||||
- DRY - fix bugs once, works everywhere
|
||||
|
||||
### Implementation Plan for Shared Pipeline
|
||||
|
||||
This refactor should be done in a separate branch after the v0.71.0 release:
|
||||
|
||||
1. **Extract stage interfaces**
|
||||
```python
|
||||
class PipelineStage(Protocol):
|
||||
async def execute(self, context: PipelineContext) -> StageResult:
|
||||
...
|
||||
```
|
||||
|
||||
2. **Create execution adapters**
|
||||
```python
|
||||
class LocalExecutor:
|
||||
"""Runs stages directly in-process"""
|
||||
|
||||
class RemoteExecutor:
|
||||
"""Runs stages via backend API/workers"""
|
||||
```
|
||||
|
||||
3. **Refactor incrementally**
|
||||
- Start with one stage (e.g., Separation)
|
||||
- Prove the pattern works
|
||||
- Migrate other stages
|
||||
Key goals:
|
||||
- **100+ concurrent jobs** with no performance degradation
|
||||
- **Dedicated resources per job** (no CPU/memory contention)
|
||||
- **Shared pipeline architecture** - single source of truth for business logic
|
||||
- **DRY codebase** - bug fixes apply to both local and remote execution
|
||||
|
||||
---
|
||||
|
||||
@@ -355,6 +828,50 @@ DEFAULT_DISCORD_WEBHOOK_URL=https://discord.com/...
|
||||
|
||||
## Recent Changes Log
|
||||
|
||||
### 2024-12-11: v0.71.28 - Batch 5 Complete (Flacfetch Audio Search)
|
||||
|
||||
**PR #13: Flacfetch Audio Search for Remote CLI**
|
||||
|
||||
- ✅ `artist title` (search mode) - Search for audio by artist and title without providing a file
|
||||
- ✅ `--auto-download` - Auto-select best audio source (skip interactive selection)
|
||||
- New job statuses: `searching_audio`, `awaiting_audio_selection`, `downloading_audio`
|
||||
- New API endpoints: `POST /api/audio-search/search`, `GET /api/audio-search/{job_id}/results`, `POST /api/audio-search/{job_id}/select`
|
||||
- Interactive mode: Returns search results for user selection
|
||||
- Auto mode: Uses flacfetch's quality ranking to auto-select best source
|
||||
- Added 20+ unit tests for new functionality
|
||||
- Files: `remote_cli.py`, `audio_search.py` (new route), `audio_search_service.py` (new service), `job.py`
|
||||
|
||||
### 2024-12-11: v0.71.26 - Batch 3 Complete (Existing Instrumental Support)
|
||||
|
||||
**PR #10: Existing Instrumental Support for Remote CLI**
|
||||
|
||||
- ✅ `--existing_instrumental` - Upload and use user-provided instrumental file
|
||||
- Duration validation (±0.5s tolerance) before processing starts
|
||||
- Backend still runs AI separation for stem downloads, but uses provided instrumental for final video
|
||||
- Added 20+ unit tests for new functionality
|
||||
- Files: `remote_cli.py`, `file_upload.py`, `job.py`, `video_worker.py`
|
||||
|
||||
### 2024-12-11: v0.71.24 - Batch 2 Complete (Audio Model Selection)
|
||||
|
||||
**PR #9: Audio Model Selection for Remote CLI**
|
||||
|
||||
- ✅ `--clean_instrumental_model` - Custom Stage 1 separation model
|
||||
- ✅ `--backing_vocals_models` - Custom backing vocals separation models
|
||||
- ✅ `--other_stems_models` - Custom multi-stem separation models (bass, drums, etc.)
|
||||
- Added 15+ unit tests for new functionality
|
||||
- Files: `remote_cli.py`, `file_upload.py`, `job.py`, `audio_worker.py`
|
||||
|
||||
### 2024-12-10: v0.71.22 - Batch 1 Complete (Lyrics Configuration)
|
||||
|
||||
**PR #8: Lyrics Configuration for Remote CLI**
|
||||
|
||||
- ✅ `--lyrics_artist` - Override artist name for lyrics search
|
||||
- ✅ `--lyrics_title` - Override title for lyrics search
|
||||
- ✅ `--lyrics_file` - Upload custom lyrics file (TXT, DOCX, RTF)
|
||||
- ✅ `--subtitle_offset_ms` - Adjust subtitle timing offset
|
||||
- Added 25+ unit tests for new functionality
|
||||
- Files: `remote_cli.py`, `file_upload.py`, `job.py`, `lyrics_worker.py`, `render_video_worker.py`
|
||||
|
||||
### 2024-12-10: v0.71.0 - Core Feature Parity Complete
|
||||
|
||||
**First successful end-to-end remote run!**
|
||||
@@ -400,14 +917,47 @@ DEFAULT_DISCORD_WEBHOOK_URL=https://discord.com/...
|
||||
2. ✅ Tag v0.71.0 release
|
||||
3. Update PyPI package
|
||||
|
||||
### Short-Term
|
||||
1. Monitor production usage
|
||||
2. Address any edge cases found
|
||||
3. Improve error messages and logging
|
||||
### Short-Term: Feature Parity (Priority Order)
|
||||
|
||||
### Long-Term (Shared Pipeline Refactor)
|
||||
1. Create new branch for shared pipeline architecture
|
||||
2. Extract stage interfaces
|
||||
3. Implement execution adapters
|
||||
4. Migrate stages incrementally
|
||||
5. Achieve true code sharing between local and remote
|
||||
1. ✅ **Lyrics Configuration** (Batch 1 - COMPLETE)
|
||||
- `--lyrics_artist`, `--lyrics_title`, `--lyrics_file`, `--subtitle_offset_ms`
|
||||
- PR #8, v0.71.22
|
||||
|
||||
2. ✅ **Audio Model Selection** (Batch 2 - COMPLETE)
|
||||
- `--clean_instrumental_model`, `--backing_vocals_models`, `--other_stems_models`
|
||||
- PR #9, v0.71.24
|
||||
|
||||
3. ✅ **Existing Instrumental** (Batch 3 - COMPLETE)
|
||||
- Added `--existing_instrumental` upload support
|
||||
- PR #10, v0.71.26
|
||||
|
||||
4. **YouTube URL Input** (Batch 4 - NEXT)
|
||||
- Accept YouTube URLs as input
|
||||
- Requires: yt-dlp in container
|
||||
- Enables: Processing niche live versions only on YouTube
|
||||
|
||||
5. ✅ **Flacfetch Audio Search** (Batch 5 - COMPLETE)
|
||||
- Interactive audio source selection via API
|
||||
- v0.71.28
|
||||
|
||||
### Medium-Term: Workflow Features
|
||||
|
||||
1. **Two-Phase Workflow** (Batch 6) ✅ COMPLETE
|
||||
- `--prep-only` and `--finalise-only` modes
|
||||
|
||||
3. **Style Override** (Batch 7)
|
||||
- Quick style tweaks without editing JSON
|
||||
|
||||
4. **Background Video Support** (Batch 8)
|
||||
- Upload and use video backgrounds
|
||||
|
||||
5. **Edit Lyrics Mode** (Batch 9)
|
||||
- Allow fixing lyrics on completed jobs
|
||||
|
||||
### Long-Term (Scalable Architecture)
|
||||
|
||||
See **[SCALABLE-ARCHITECTURE-PLAN.md](./SCALABLE-ARCHITECTURE-PLAN.md)** for the comprehensive plan covering:
|
||||
1. Cloud Tasks integration for horizontal scalability
|
||||
2. Cloud Run Jobs for long-running video encoding
|
||||
3. Shared pipeline architecture for code reuse
|
||||
4. Implementation roadmap and migration strategy
|
||||
@@ -0,0 +1,141 @@
|
||||
# Refactor: Unify Audio Fetcher Code
|
||||
|
||||
**Created:** 2025-12-20
|
||||
**Completed:** 2025-12-20
|
||||
**Priority:** MEDIUM (prevents future bugs like YouTubeProvider -> YoutubeProvider)
|
||||
**Status:** ✅ Completed
|
||||
|
||||
## Problem
|
||||
|
||||
The codebase has **two separate implementations** of flacfetch integration:
|
||||
1. `karaoke_gen/audio_fetcher.py` - Used by local CLI
|
||||
2. `backend/services/audio_search_service.py` - Used by cloud backend
|
||||
|
||||
This violates DRY and caused a bug where the backend's import wasn't updated when flacfetch renamed `YouTubeProvider` to `YoutubeProvider`.
|
||||
|
||||
## Current State
|
||||
|
||||
### Duplicated Code
|
||||
|
||||
| Component | Local (`audio_fetcher.py`) | Backend (`audio_search_service.py`) |
|
||||
|-----------|---------------------------|-------------------------------------|
|
||||
| `_get_manager()` | ✅ 40 lines | ❌ 35 lines (copy) |
|
||||
| Provider setup | ✅ Redacted, OPS, YouTube | ❌ Same, diverged |
|
||||
| `search()` | ✅ Full implementation | ❌ Similar, diverged |
|
||||
| `download()` | ✅ With downloaders | ⚠️ Different, simpler |
|
||||
| `AudioSearchResult` | ✅ Dataclass | ❌ Duplicate, different fields |
|
||||
| `AudioDownloadResult` | ✅ (`AudioFetchResult`) | ❌ Duplicate |
|
||||
| Exceptions | ✅ 4 types | ❌ 3 types, different names |
|
||||
|
||||
### Drift Examples
|
||||
- Backend missing `raw_result` field on `AudioSearchResult`
|
||||
- Backend has `index` field, local doesn't
|
||||
- Backend exceptions inherit from `AudioSearchError`, local from `AudioFetcherError`
|
||||
- Backend missing `UserCancelledError`
|
||||
- Backend missing downloader registration (TorrentDownloader, YoutubeDownloader)
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
### Option 1: Backend imports from karaoke_gen (Recommended)
|
||||
|
||||
The backend already depends on `karaoke_gen` (installed via `pip install -e .` in Dockerfile).
|
||||
|
||||
```python
|
||||
# backend/services/audio_search_service.py - REFACTORED
|
||||
|
||||
from karaoke_gen.audio_fetcher import (
|
||||
FlacFetcher,
|
||||
AudioSearchResult,
|
||||
AudioFetchResult,
|
||||
AudioFetcherError,
|
||||
NoResultsError,
|
||||
DownloadError,
|
||||
)
|
||||
|
||||
class AudioSearchService:
|
||||
"""
|
||||
Thin wrapper around karaoke_gen.audio_fetcher.FlacFetcher.
|
||||
|
||||
Adds backend-specific functionality:
|
||||
- Firestore serialization (to_dict/from_dict)
|
||||
- GCS integration for downloads
|
||||
- Job state management
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
redacted_api_key: Optional[str] = None,
|
||||
ops_api_key: Optional[str] = None,
|
||||
):
|
||||
self._fetcher = FlacFetcher(
|
||||
redacted_api_key=redacted_api_key,
|
||||
ops_api_key=ops_api_key,
|
||||
)
|
||||
|
||||
def search(self, artist: str, title: str) -> List[AudioSearchResult]:
|
||||
"""Delegate to FlacFetcher.search()."""
|
||||
return self._fetcher.search(artist, title)
|
||||
|
||||
def select_best(self, results: List[AudioSearchResult]) -> int:
|
||||
"""Delegate to FlacFetcher.select_best()."""
|
||||
return self._fetcher.select_best(results)
|
||||
|
||||
def download(
|
||||
self,
|
||||
result: AudioSearchResult,
|
||||
output_dir: str,
|
||||
output_filename: Optional[str] = None,
|
||||
) -> AudioFetchResult:
|
||||
"""Delegate to FlacFetcher.download()."""
|
||||
return self._fetcher.download(result, output_dir, output_filename)
|
||||
```
|
||||
|
||||
### Changes Required to karaoke_gen.audio_fetcher
|
||||
|
||||
1. **Add `to_dict()` method to `AudioSearchResult`** - For Firestore serialization
|
||||
2. **Add `from_dict()` classmethod** - For deserialization
|
||||
3. **Ensure `select_best()` is a standalone method** - Currently buried in interactive flow
|
||||
4. **Export all public classes from `__init__.py`**
|
||||
|
||||
### Changes Required to backend
|
||||
|
||||
1. **Remove duplicate dataclasses** - Use imports from karaoke_gen
|
||||
2. **Remove duplicate `_get_manager()`** - Delegate to FlacFetcher
|
||||
3. **Remove duplicate exception classes** - Use karaoke_gen exceptions
|
||||
4. **Update tests** - Point to real classes, not mocks
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Single source of truth** - Bug fixes apply everywhere
|
||||
2. **No more drift** - Dataclasses, exceptions stay in sync
|
||||
3. **Better test coverage** - karaoke_gen has integration tests for flacfetch imports
|
||||
4. **Less code to maintain** - Backend becomes thin wrapper
|
||||
|
||||
## Migration Steps
|
||||
|
||||
1. [x] Add `to_dict()` / `from_dict()` to `AudioSearchResult` in karaoke_gen
|
||||
2. [x] Extract `select_best()` as standalone method in FlacFetcher
|
||||
3. [x] Update karaoke_gen `__init__.py` exports (with lazy imports)
|
||||
4. [x] Refactor backend to import from karaoke_gen
|
||||
5. [x] Update backend tests
|
||||
6. [x] Delete duplicate code from backend (~150 lines removed)
|
||||
7. [x] Add integration test that catches import divergence
|
||||
|
||||
## Related Files
|
||||
|
||||
- `karaoke_gen/audio_fetcher.py` - Source of truth (keep)
|
||||
- `backend/services/audio_search_service.py` - Refactor to thin wrapper
|
||||
- `backend/api/routes/audio_search.py` - Update imports
|
||||
- `backend/tests/test_audio_search.py` - Update tests
|
||||
|
||||
## Estimated Effort
|
||||
|
||||
- **Time:** 2-4 hours
|
||||
- **Risk:** Low (well-defined changes, good test coverage)
|
||||
- **Dependencies:** None
|
||||
|
||||
## Related Architecture Docs
|
||||
|
||||
- `docs/01-reference/ARCHITECTURE.md` - "LyricsTranscriber Integration" section shows similar pattern
|
||||
- `docs/00-current-plan/SCALABLE-ARCHITECTURE-PLAN.md` - "Shared pipeline architecture" goal
|
||||
|
||||
@@ -0,0 +1,827 @@
|
||||
# Scalable Architecture Plan
|
||||
|
||||
**Last Updated:** 2025-12-11
|
||||
**Status:** Phase 1 Complete - Cloud Tasks Enabled
|
||||
|
||||
This document outlines the plan for evolving karaoke-gen's cloud backend to support **100+ concurrent jobs** with no performance degradation, while also achieving a clean, maintainable codebase with proper separation of concerns.
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
After thorough codebase analysis, I recommend a **phased approach** that prioritizes infrastructure scalability first (Cloud Tasks), followed by incremental adoption of the existing pipeline architecture for code sharing.
|
||||
|
||||
### Key Findings
|
||||
|
||||
1. **The infrastructure problem is well-isolated** - WorkerService already abstracts worker triggering behind HTTP calls
|
||||
2. **Pipeline architecture exists but is underutilized** - `karaoke_gen/pipeline/` has well-designed stages and executors, but backend workers don't use them
|
||||
3. **RemoteExecutor is a placeholder** - It submits jobs to the backend but doesn't execute stages remotely
|
||||
4. **Workers are self-contained** - Each worker only needs `job_id` and fetches everything from Firestore/GCS
|
||||
|
||||
### Recommended Priority
|
||||
|
||||
1. **Phase 1 (HIGH): Cloud Tasks Integration** - Solves the scalability problem immediately
|
||||
2. **Phase 2 (MEDIUM): Cloud Run Jobs for Video** - Removes 30-min timeout constraint
|
||||
3. **Phase 3 (LOW): Pipeline Unification** - Nice-to-have for code maintainability
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
### Infrastructure Goals
|
||||
1. **Horizontal scalability** - Each job gets dedicated resources (no resource contention)
|
||||
2. **Maximum processing speed** - Same speed for 1 job or 100 concurrent jobs
|
||||
3. **Reliability** - Automatic retries, no lost jobs from container restarts
|
||||
4. **Cost efficiency** - Scale to zero when idle, pay only for active processing
|
||||
|
||||
### Software Architecture Goals
|
||||
1. **SOLID principles** - Single responsibility, clean interfaces
|
||||
2. **DRY** - Share code between local CLI and remote backend
|
||||
3. **Testability** - Each component independently testable
|
||||
4. **Maintainability** - Bug fixes apply everywhere, easy to add features
|
||||
|
||||
---
|
||||
|
||||
## Current Architecture Analysis
|
||||
|
||||
### What Works Well
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ CURRENT ARCHITECTURE (v0.71.x) │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ✅ Heavy work already offloaded to external services: │
|
||||
│ │
|
||||
│ Audio Separation ──────────────────▶ Modal API (GPU, ~5-8 min) │
|
||||
│ Lyrics Transcription ──────────────▶ AudioShake API (~2-3 min) │
|
||||
│ │
|
||||
│ ✅ State persistence in Firestore (survives restarts) │
|
||||
│ ✅ Files in GCS (no local storage dependencies) │
|
||||
│ ✅ Internal HTTP endpoints exist for worker triggers │
|
||||
│ ✅ Workers are self-contained (only need job_id) │
|
||||
│ ✅ WorkerService abstracts HTTP calls │
|
||||
│ ✅ Pipeline stages exist in karaoke_gen/pipeline/ │
|
||||
│ ✅ LocalExecutor for CLI, RemoteExecutor placeholder ready │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Current Code Structure
|
||||
|
||||
```
|
||||
WorkerService (backend/services/worker_service.py)
|
||||
│
|
||||
└── trigger_worker(worker_type, job_id)
|
||||
│
|
||||
└── HTTP POST /api/internal/workers/{worker_type}
|
||||
│
|
||||
└── BackgroundTasks.add_task(process_xxx, job_id) ⚠️ PROBLEM
|
||||
```
|
||||
|
||||
The **single point of change** is `WorkerService.trigger_worker()`. Currently it:
|
||||
1. Makes HTTP POST to internal endpoint
|
||||
2. Internal endpoint adds to FastAPI BackgroundTasks
|
||||
3. BackgroundTask runs in same container (problem!)
|
||||
|
||||
### Current Limitations
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ CURRENT FLOW (PROBLEMATIC) │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ POST /api/jobs/upload │
|
||||
│ │ │
|
||||
│ ├──▶ Save job to Firestore ✅ │
|
||||
│ │ │
|
||||
│ └──▶ BackgroundTasks.add_task(trigger_workers) │
|
||||
│ │ │
|
||||
│ └──▶ HTTP POST /api/internal/workers/audio │
|
||||
│ │ │
|
||||
│ └──▶ BackgroundTasks.add_task(process_audio) ⚠️ │
|
||||
│ │ │
|
||||
│ └── Runs IN THE SAME CONTAINER │
|
||||
│ as the API request handler │
|
||||
│ │
|
||||
│ PROBLEMS: │
|
||||
│ ⚠️ Container may be killed before background tasks complete │
|
||||
│ ⚠️ No guaranteed delivery - if container crashes, job is lost │
|
||||
│ ⚠️ Multiple concurrent jobs share CPU/memory (resource contention) │
|
||||
│ ⚠️ Long-running video encoding (15-20 min) vulnerable to timeouts │
|
||||
│ ⚠️ FFmpeg renders compete for resources when jobs run in parallel │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Existing Pipeline Architecture (Underutilized)
|
||||
|
||||
The codebase already has a well-designed pipeline system:
|
||||
|
||||
```
|
||||
karaoke_gen/pipeline/
|
||||
├── base.py # PipelineStage, StageResult, PipelineExecutor abstracts
|
||||
├── context.py # PipelineContext for shared state
|
||||
├── executors/
|
||||
│ ├── local.py # LocalExecutor - runs stages in-process (for CLI)
|
||||
│ └── remote.py # RemoteExecutor - placeholder for backend integration
|
||||
└── stages/
|
||||
├── separation.py # SeparationStage - wraps AudioProcessor
|
||||
├── transcription.py # TranscriptionStage - wraps LyricsProcessor
|
||||
├── screens.py # ScreensStage - title/end screens
|
||||
├── render.py # RenderStage - lyrics video
|
||||
└── finalize.py # FinalizeStage - wraps KaraokeFinalise
|
||||
```
|
||||
|
||||
**However**: Backend workers (`backend/workers/*.py`) don't use these stages - they directly call the underlying processors. This creates two parallel implementations.
|
||||
|
||||
---
|
||||
|
||||
## Recommended Architecture
|
||||
|
||||
### Decision: Cloud Tasks First
|
||||
|
||||
**Why Cloud Tasks over Pub/Sub or Workflows:**
|
||||
|
||||
| Feature | Cloud Tasks | Pub/Sub | Workflows |
|
||||
|---------|-------------|---------|-----------|
|
||||
| HTTP delivery | ✅ Native | Requires push subscription | N/A |
|
||||
| Retry control | ✅ Built-in | Manual | Built-in |
|
||||
| Rate limiting | ✅ Built-in | Manual | N/A |
|
||||
| Timeout (max) | 30 min | 10 min ack deadline | 1 year |
|
||||
| Complexity | Low | Medium | High |
|
||||
| Cost | Very low | Low | Medium |
|
||||
| Existing code changes | Minimal | Medium | High |
|
||||
|
||||
**Cloud Tasks is the right choice because:**
|
||||
1. Workers already have HTTP endpoints - Cloud Tasks delivers HTTP requests
|
||||
2. Need rate limiting to protect Modal/AudioShake APIs
|
||||
3. Built-in retries with configurable backoff
|
||||
4. Minimal code changes (update WorkerService only)
|
||||
|
||||
### Target Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ CLOUD TASKS + CLOUD RUN JOBS ARCHITECTURE │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ POST /api/jobs/upload │
|
||||
│ │ │
|
||||
│ ├──▶ Save job to Firestore │
|
||||
│ │ │
|
||||
│ └──▶ WorkerService.trigger_worker() │
|
||||
│ │ │
|
||||
│ │ if ENABLE_CLOUD_TASKS: │
|
||||
│ │ └──▶ Enqueue Cloud Task │
|
||||
│ │ │ │
|
||||
│ │ ├──▶ audio-worker-queue │
|
||||
│ │ │ {job_id, target: /internal/workers/audio} │
|
||||
│ │ │ │
|
||||
│ │ └──▶ lyrics-worker-queue │
|
||||
│ │ {job_id, target: /internal/workers/lyrics}│
|
||||
│ │ │
|
||||
│ │ else: (development/testing) │
|
||||
│ │ └──▶ HTTP POST directly (existing behavior) │
|
||||
│ │ │
|
||||
│ ┌──────────▼────────────────────────────────────────────────────────┐ │
|
||||
│ │ Cloud Tasks Queue │ │
|
||||
│ │ │ │
|
||||
│ │ • Guaranteed delivery (retries on failure) │ │
|
||||
│ │ • Rate limiting / concurrency control │ │
|
||||
│ │ • Task deduplication │ │
|
||||
│ │ • Configurable timeouts (up to 30 min per task) │ │
|
||||
│ └─────────────────────────────────┬───────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Cloud Run (Auto-Scaled Worker Instances) │ │
|
||||
│ │ │ │
|
||||
│ │ Instance 1 Instance 2 Instance 3 ... │ │
|
||||
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
|
||||
│ │ │ Job A │ │ Job B │ │ Job C │ │ │
|
||||
│ │ │ Audio │ │ Audio │ │ Lyrics │ │ │
|
||||
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
|
||||
│ │ │ │
|
||||
│ │ Each task gets a DEDICATED container instance │ │
|
||||
│ │ • Full CPU/memory for that task alone │ │
|
||||
│ │ • No resource contention with other jobs │ │
|
||||
│ │ • Container lifecycle tied to task completion │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ For video encoding (>30 min timeout): │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Cloud Run Jobs (Long-Running) │ │
|
||||
│ │ │ │
|
||||
│ │ • Up to 24 hour timeout │ │
|
||||
│ │ • Separate from API service │ │
|
||||
│ │ • Higher CPU/memory (4 vCPU, 8GB) │ │
|
||||
│ │ • Triggered via gcloud or API │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Cloud Tasks Integration (HIGH Priority)
|
||||
|
||||
**Goal:** Enable 100+ concurrent jobs with no performance degradation
|
||||
|
||||
**Effort:** 2-3 days | **Risk:** Low (feature-flagged, easy rollback)
|
||||
|
||||
#### Step 1.1: Add Cloud Tasks Infrastructure (Pulumi)
|
||||
|
||||
```python
|
||||
# infrastructure/__main__.py additions
|
||||
|
||||
from pulumi_gcp import cloudtasks
|
||||
|
||||
# Create Cloud Tasks queues
|
||||
audio_worker_queue = cloudtasks.Queue(
|
||||
"audio-worker-queue",
|
||||
name="audio-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=10, # Protect Modal API
|
||||
max_concurrent_dispatches=50, # Max parallel audio workers
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=3,
|
||||
min_backoff="10s",
|
||||
max_backoff="300s",
|
||||
max_retry_duration="1800s", # 30 min total retry window
|
||||
),
|
||||
)
|
||||
|
||||
lyrics_worker_queue = cloudtasks.Queue(
|
||||
"lyrics-worker-queue",
|
||||
name="lyrics-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=10,
|
||||
max_concurrent_dispatches=50,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=3,
|
||||
min_backoff="10s",
|
||||
max_backoff="300s",
|
||||
),
|
||||
)
|
||||
|
||||
screens_worker_queue = cloudtasks.Queue(
|
||||
"screens-worker-queue",
|
||||
name="screens-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=50, # Fast, CPU-light
|
||||
max_concurrent_dispatches=100,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=3,
|
||||
min_backoff="5s",
|
||||
max_backoff="60s",
|
||||
),
|
||||
)
|
||||
|
||||
render_worker_queue = cloudtasks.Queue(
|
||||
"render-worker-queue",
|
||||
name="render-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=5, # CPU-intensive
|
||||
max_concurrent_dispatches=20,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=2,
|
||||
min_backoff="30s",
|
||||
max_backoff="300s",
|
||||
max_retry_duration="3600s", # 60 min for render
|
||||
),
|
||||
)
|
||||
|
||||
video_worker_queue = cloudtasks.Queue(
|
||||
"video-worker-queue",
|
||||
name="video-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=3, # Very CPU-intensive
|
||||
max_concurrent_dispatches=10,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=2,
|
||||
min_backoff="60s",
|
||||
max_backoff="600s",
|
||||
max_retry_duration="7200s", # 2 hour total (video is long)
|
||||
),
|
||||
)
|
||||
|
||||
# Grant Cloud Tasks permission to invoke Cloud Run
|
||||
cloud_tasks_invoker = gcp.projects.IAMMember(
|
||||
"cloud-tasks-invoker",
|
||||
project=project_id,
|
||||
role="roles/run.invoker",
|
||||
member=f"serviceAccount:service-{project.number}@gcp-sa-cloudtasks.iam.gserviceaccount.com",
|
||||
)
|
||||
|
||||
# Export queue names
|
||||
pulumi.export("audio_worker_queue", audio_worker_queue.name)
|
||||
pulumi.export("lyrics_worker_queue", lyrics_worker_queue.name)
|
||||
pulumi.export("screens_worker_queue", screens_worker_queue.name)
|
||||
pulumi.export("render_worker_queue", render_worker_queue.name)
|
||||
pulumi.export("video_worker_queue", video_worker_queue.name)
|
||||
```
|
||||
|
||||
#### Step 1.2: Update WorkerService for Cloud Tasks
|
||||
|
||||
```python
|
||||
# backend/services/worker_service.py - Updated
|
||||
|
||||
import logging
|
||||
import os
|
||||
import json
|
||||
from typing import Optional
|
||||
from google.cloud import tasks_v2
|
||||
from google.protobuf import timestamp_pb2
|
||||
import httpx
|
||||
|
||||
from backend.config import get_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Queue mapping
|
||||
WORKER_QUEUES = {
|
||||
"audio": "audio-worker-queue",
|
||||
"lyrics": "lyrics-worker-queue",
|
||||
"screens": "screens-worker-queue",
|
||||
"render-video": "render-worker-queue",
|
||||
"video": "video-worker-queue",
|
||||
}
|
||||
|
||||
class WorkerService:
|
||||
"""
|
||||
Service for coordinating background workers.
|
||||
|
||||
Supports two modes:
|
||||
- Cloud Tasks (production): Guaranteed delivery, auto-scaling
|
||||
- Direct HTTP (development): Faster iteration, simpler debugging
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.settings = get_settings()
|
||||
self._base_url = self._get_base_url()
|
||||
self._admin_token = self._get_admin_token()
|
||||
self._use_cloud_tasks = self._should_use_cloud_tasks()
|
||||
self._tasks_client = None
|
||||
|
||||
def _should_use_cloud_tasks(self) -> bool:
|
||||
"""Check if Cloud Tasks should be used."""
|
||||
# Feature flag with environment variable
|
||||
enable_flag = os.getenv("ENABLE_CLOUD_TASKS", "false").lower()
|
||||
return enable_flag in ("true", "1", "yes")
|
||||
|
||||
@property
|
||||
def tasks_client(self):
|
||||
"""Lazy-init Cloud Tasks client."""
|
||||
if self._tasks_client is None and self._use_cloud_tasks:
|
||||
self._tasks_client = tasks_v2.CloudTasksClient()
|
||||
return self._tasks_client
|
||||
|
||||
def _get_admin_token(self) -> Optional[str]:
|
||||
admin_tokens_str = self.settings.admin_tokens or ""
|
||||
tokens = [t.strip() for t in admin_tokens_str.split(",") if t.strip()]
|
||||
return tokens[0] if tokens else None
|
||||
|
||||
def _get_base_url(self) -> str:
|
||||
# Check for test environment override
|
||||
test_url = os.getenv("TEST_SERVER_URL")
|
||||
if test_url:
|
||||
return test_url
|
||||
|
||||
# Production: Cloud Run service URL
|
||||
service_url = os.getenv("CLOUD_RUN_SERVICE_URL")
|
||||
if service_url:
|
||||
return service_url
|
||||
|
||||
# Development: localhost
|
||||
port = os.getenv("PORT", "8000")
|
||||
return f"http://localhost:{port}"
|
||||
|
||||
async def trigger_worker(
|
||||
self,
|
||||
worker_type: str,
|
||||
job_id: str,
|
||||
timeout_seconds: int = 30
|
||||
) -> bool:
|
||||
"""
|
||||
Trigger a background worker.
|
||||
|
||||
In production (ENABLE_CLOUD_TASKS=true):
|
||||
- Enqueues task to Cloud Tasks queue
|
||||
- Cloud Tasks calls internal endpoint with retries
|
||||
|
||||
In development:
|
||||
- Direct HTTP call to internal endpoint
|
||||
- Faster iteration, but no retry guarantees
|
||||
"""
|
||||
if self._use_cloud_tasks:
|
||||
return await self._enqueue_cloud_task(worker_type, job_id)
|
||||
else:
|
||||
return await self._trigger_http(worker_type, job_id, timeout_seconds)
|
||||
|
||||
async def _enqueue_cloud_task(self, worker_type: str, job_id: str) -> bool:
|
||||
"""Enqueue task to Cloud Tasks for guaranteed delivery."""
|
||||
try:
|
||||
queue_name = WORKER_QUEUES.get(worker_type)
|
||||
if not queue_name:
|
||||
logger.error(f"Unknown worker type: {worker_type}")
|
||||
return False
|
||||
|
||||
project = self.settings.google_cloud_project
|
||||
location = "us-central1"
|
||||
|
||||
# Build queue path
|
||||
parent = self.tasks_client.queue_path(project, location, queue_name)
|
||||
|
||||
# Build task
|
||||
task = {
|
||||
"http_request": {
|
||||
"http_method": tasks_v2.HttpMethod.POST,
|
||||
"url": f"{self._base_url}/api/internal/workers/{worker_type}",
|
||||
"headers": {
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
"body": json.dumps({"job_id": job_id}).encode(),
|
||||
# Use OIDC token for Cloud Run authentication
|
||||
"oidc_token": {
|
||||
"service_account_email": f"karaoke-backend@{project}.iam.gserviceaccount.com",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
# Add admin auth header if available
|
||||
if self._admin_token:
|
||||
task["http_request"]["headers"]["Authorization"] = f"Bearer {self._admin_token}"
|
||||
|
||||
# Create task
|
||||
response = self.tasks_client.create_task(parent=parent, task=task)
|
||||
logger.info(f"Created Cloud Task for {worker_type} worker, job {job_id}: {response.name}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to enqueue Cloud Task for {worker_type}/{job_id}: {e}", exc_info=True)
|
||||
return False
|
||||
|
||||
async def _trigger_http(
|
||||
self,
|
||||
worker_type: str,
|
||||
job_id: str,
|
||||
timeout_seconds: int = 30
|
||||
) -> bool:
|
||||
"""Direct HTTP trigger (for development)."""
|
||||
try:
|
||||
headers = {}
|
||||
if self._admin_token:
|
||||
headers["Authorization"] = f"Bearer {self._admin_token}"
|
||||
|
||||
async with httpx.AsyncClient(timeout=timeout_seconds) as client:
|
||||
url = f"{self._base_url}/api/internal/workers/{worker_type}"
|
||||
response = await client.post(url, json={"job_id": job_id}, headers=headers)
|
||||
|
||||
if response.status_code == 200:
|
||||
logger.info(f"Triggered {worker_type} worker for job {job_id}")
|
||||
return True
|
||||
else:
|
||||
logger.error(f"Failed to trigger {worker_type}: HTTP {response.status_code}")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error triggering {worker_type} worker: {e}", exc_info=True)
|
||||
return False
|
||||
|
||||
# Convenience methods (unchanged)
|
||||
async def trigger_audio_worker(self, job_id: str) -> bool:
|
||||
return await self.trigger_worker("audio", job_id)
|
||||
|
||||
async def trigger_lyrics_worker(self, job_id: str) -> bool:
|
||||
return await self.trigger_worker("lyrics", job_id)
|
||||
|
||||
async def trigger_screens_worker(self, job_id: str) -> bool:
|
||||
return await self.trigger_worker("screens", job_id)
|
||||
|
||||
async def trigger_video_worker(self, job_id: str) -> bool:
|
||||
return await self.trigger_worker("video", job_id)
|
||||
|
||||
async def trigger_render_video_worker(self, job_id: str) -> bool:
|
||||
return await self.trigger_worker("render-video", job_id)
|
||||
```
|
||||
|
||||
#### Step 1.3: Update Internal Endpoints for Cloud Tasks
|
||||
|
||||
The internal endpoints need minor updates to handle Cloud Tasks requests:
|
||||
|
||||
```python
|
||||
# backend/api/routes/internal.py - Updates
|
||||
|
||||
# Add idempotency check to prevent duplicate processing
|
||||
@router.post("/workers/audio", response_model=WorkerResponse)
|
||||
async def trigger_audio_worker(
|
||||
request: WorkerRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
auth_data: Tuple[str, UserType, int] = Depends(require_admin)
|
||||
):
|
||||
"""Trigger audio separation worker for a job."""
|
||||
job_id = request.job_id
|
||||
|
||||
# Idempotency: Check if already processing
|
||||
job_manager = JobManager()
|
||||
job = job_manager.get_job(job_id)
|
||||
|
||||
if job and job.state_data.get('audio_progress', {}).get('stage') == 'running':
|
||||
logger.info(f"Audio worker already running for job {job_id}, skipping")
|
||||
return WorkerResponse(
|
||||
status="already_running",
|
||||
job_id=job_id,
|
||||
message="Audio worker already in progress"
|
||||
)
|
||||
|
||||
# Mark as running before starting (for idempotency)
|
||||
job_manager.update_state_data(job_id, 'audio_progress', {'stage': 'running'})
|
||||
|
||||
logger.info(f"Triggering audio worker for job {job_id}")
|
||||
background_tasks.add_task(process_audio_separation, job_id)
|
||||
|
||||
return WorkerResponse(
|
||||
status="started",
|
||||
job_id=job_id,
|
||||
message="Audio separation worker started"
|
||||
)
|
||||
```
|
||||
|
||||
#### Step 1.4: Cloud Run Service Account Permissions
|
||||
|
||||
```python
|
||||
# infrastructure/__main__.py additions
|
||||
|
||||
# Grant Cloud Tasks admin permission to the service account
|
||||
# (needed to create tasks)
|
||||
cloud_tasks_admin = gcp.projects.IAMMember(
|
||||
"karaoke-backend-cloudtasks-admin",
|
||||
project=project_id,
|
||||
role="roles/cloudtasks.enqueuer",
|
||||
member=service_account.email.apply(lambda email: f"serviceAccount:{email}"),
|
||||
)
|
||||
```
|
||||
|
||||
### Phase 2: Cloud Run Jobs for Video Worker (MEDIUM Priority)
|
||||
|
||||
**Goal:** Remove 30-minute timeout constraint for video encoding
|
||||
|
||||
**Effort:** 1-2 days | **Risk:** Low
|
||||
|
||||
The video worker (final encoding) can take 15-20 minutes, sometimes longer. Cloud Tasks has a 30-minute max timeout. For safety, we use Cloud Run Jobs.
|
||||
|
||||
#### Step 2.1: Create Cloud Run Job
|
||||
|
||||
```yaml
|
||||
# cloudbuild-video-job.yaml
|
||||
steps:
|
||||
- name: 'gcr.io/cloud-builders/gcloud'
|
||||
args:
|
||||
- 'run'
|
||||
- 'jobs'
|
||||
- 'create'
|
||||
- 'video-encoding-job'
|
||||
- '--image=${_IMAGE}'
|
||||
- '--region=us-central1'
|
||||
- '--cpu=4'
|
||||
- '--memory=8Gi'
|
||||
- '--max-retries=2'
|
||||
- '--task-timeout=3600' # 1 hour max
|
||||
- '--service-account=karaoke-backend@${PROJECT_ID}.iam.gserviceaccount.com'
|
||||
- '--set-env-vars=GOOGLE_CLOUD_PROJECT=${PROJECT_ID}'
|
||||
```
|
||||
|
||||
#### Step 2.2: Update Video Worker Trigger
|
||||
|
||||
```python
|
||||
# backend/services/worker_service.py - Addition
|
||||
|
||||
async def trigger_video_worker_job(self, job_id: str) -> bool:
|
||||
"""Trigger video worker as Cloud Run Job (for long-running encoding)."""
|
||||
if not self._use_cloud_tasks:
|
||||
# Development: use HTTP trigger
|
||||
return await self._trigger_http("video", job_id)
|
||||
|
||||
try:
|
||||
from google.cloud import run_v2
|
||||
|
||||
client = run_v2.JobsClient()
|
||||
project = self.settings.google_cloud_project
|
||||
location = "us-central1"
|
||||
|
||||
job_name = f"projects/{project}/locations/{location}/jobs/video-encoding-job"
|
||||
|
||||
# Override container args with job_id
|
||||
request = run_v2.RunJobRequest(
|
||||
name=job_name,
|
||||
overrides=run_v2.RunJobRequest.Overrides(
|
||||
container_overrides=[
|
||||
run_v2.RunJobRequest.Overrides.ContainerOverride(
|
||||
args=["--job-id", job_id],
|
||||
)
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
operation = client.run_job(request=request)
|
||||
logger.info(f"Started Cloud Run Job for video encoding: {job_id}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start video encoding job: {e}", exc_info=True)
|
||||
return False
|
||||
```
|
||||
|
||||
### Phase 3: Pipeline Unification (LOW Priority - Future)
|
||||
|
||||
**Goal:** Share code between CLI and backend workers
|
||||
|
||||
**Effort:** 1-2 weeks | **Risk:** Medium (requires careful refactoring)
|
||||
|
||||
This phase can be deferred - the infrastructure scaling (Phase 1-2) solves the immediate problem.
|
||||
|
||||
**When to consider this:**
|
||||
- When adding new features that need to work in both CLI and backend
|
||||
- When a bug fix needs to be applied in multiple places
|
||||
- When the divergence between workers and pipeline stages causes maintenance burden
|
||||
|
||||
**Approach:**
|
||||
1. Make backend workers call pipeline stages instead of processors directly
|
||||
2. Have RemoteExecutor use Cloud Tasks to trigger stage execution
|
||||
3. Both CLI and backend use the same pipeline stages
|
||||
|
||||
---
|
||||
|
||||
## Resource Configuration
|
||||
|
||||
### Cloud Tasks Queues
|
||||
|
||||
| Queue | Max Concurrent | Rate Limit | Timeout | Retries | Notes |
|
||||
|-------|---------------|------------|---------|---------|-------|
|
||||
| audio-worker | 50 | 10/sec | 15 min | 3 | Mostly I/O (Modal API) |
|
||||
| lyrics-worker | 50 | 10/sec | 10 min | 3 | Mostly I/O (AudioShake) |
|
||||
| screens-worker | 100 | 50/sec | 2 min | 3 | FFmpeg (quick) |
|
||||
| render-worker | 20 | 5/sec | 30 min | 2 | LyricsTranscriber + FFmpeg |
|
||||
| video-worker | 10 | 3/sec | 30 min | 2 | Heavy FFmpeg (or use Jobs) |
|
||||
|
||||
### Cloud Run Instance Scaling
|
||||
|
||||
| Worker | CPU | Memory | Max Instances | Notes |
|
||||
|--------|-----|--------|---------------|-------|
|
||||
| audio | 2 | 2Gi | 50 | Mostly waiting on Modal API |
|
||||
| lyrics | 2 | 2Gi | 50 | Mostly waiting on AudioShake |
|
||||
| screens | 2 | 2Gi | 100 | Quick FFmpeg |
|
||||
| render | 4 | 4Gi | 20 | CPU-intensive |
|
||||
| video (Job) | 4 | 8Gi | 10 | Very CPU-intensive |
|
||||
|
||||
### Cost Estimate (100 concurrent jobs/day)
|
||||
|
||||
```
|
||||
Cloud Tasks: ~$0.40/million tasks ≈ $0.01/day
|
||||
Cloud Run: ~$0.00002/vCPU-sec ≈ $5-10/day (100 jobs)
|
||||
Cloud Run Jobs: ~$0.00002/vCPU-sec ≈ $2-5/day (100 jobs)
|
||||
Modal (GPU): ~$2.50/hour ≈ $20-40/day (100 jobs)
|
||||
AudioShake: Usage-based ≈ $10-20/day (100 jobs)
|
||||
|
||||
Total: ~$40-80/day for 100 concurrent jobs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Zero-Downtime Migration
|
||||
|
||||
```
|
||||
Week 1: Deploy Cloud Tasks infrastructure (Pulumi)
|
||||
└── Queues exist but aren't used yet (ENABLE_CLOUD_TASKS=false)
|
||||
|
||||
Week 2: Test in staging
|
||||
└── Deploy to staging with ENABLE_CLOUD_TASKS=true
|
||||
└── Run 10-20 test jobs concurrently
|
||||
└── Monitor for issues
|
||||
|
||||
Week 3: Enable in production
|
||||
└── Set ENABLE_CLOUD_TASKS=true
|
||||
└── Monitor metrics and logs
|
||||
└── Have rollback ready
|
||||
|
||||
Week 4: Cleanup
|
||||
└── Remove feature flag after stable week
|
||||
└── Update documentation
|
||||
```
|
||||
|
||||
### Rollback Plan
|
||||
|
||||
If issues occur after enabling Cloud Tasks:
|
||||
1. Set `ENABLE_CLOUD_TASKS=false` environment variable
|
||||
2. Redeploy (takes ~2 minutes)
|
||||
3. New jobs use direct HTTP (old behavior)
|
||||
4. In-flight Cloud Tasks jobs continue to completion
|
||||
5. No data loss - Firestore has job state
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
```bash
|
||||
# Run unit tests (no Cloud Tasks)
|
||||
pytest backend/tests/ -v
|
||||
|
||||
# Run integration tests with Cloud Tasks (staging)
|
||||
ENABLE_CLOUD_TASKS=true pytest backend/tests/integration/ -v
|
||||
|
||||
# Load test (staging)
|
||||
# Submit 50 concurrent jobs, verify all complete
|
||||
for i in {1..50}; do
|
||||
curl -X POST "https://staging.api.nomadkaraoke.com/api/jobs/upload" \
|
||||
-F "file=@test.flac" -F "artist=Test" -F "title=Job$i" &
|
||||
done
|
||||
wait
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Phase 1 (Cloud Tasks)
|
||||
- [x] Cloud Tasks queues created and configured via Pulumi
|
||||
- [x] WorkerService updated to support Cloud Tasks mode
|
||||
- [x] Deployment configured with ENABLE_CLOUD_TASKS=true
|
||||
- [x] Container concurrency set to 1 for true isolation
|
||||
- [ ] 50 concurrent jobs complete without failures (to be validated)
|
||||
- [ ] Processing time per job unchanged (same as 1 job)
|
||||
- [ ] No resource contention (CPU/memory per job consistent)
|
||||
- [ ] Automatic retry on transient failures works
|
||||
- [ ] Feature flag rollback works
|
||||
|
||||
### Phase 2 (Cloud Run Jobs)
|
||||
- [ ] Video encoding >30 min completes successfully
|
||||
- [ ] Jobs triggered correctly from WorkerService
|
||||
- [ ] Progress updates visible in CLI
|
||||
|
||||
### Phase 3 (Pipeline Unification)
|
||||
- [ ] Single stage implementation used by both CLIs
|
||||
- [ ] Bug fixes apply to both local and remote execution
|
||||
- [ ] Each stage independently testable
|
||||
- [ ] 80%+ code coverage on pipeline module
|
||||
|
||||
---
|
||||
|
||||
## Files to Modify
|
||||
|
||||
### Phase 1
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `infrastructure/__main__.py` | Add Cloud Tasks queues, IAM permissions |
|
||||
| `backend/services/worker_service.py` | Add Cloud Tasks client, feature flag |
|
||||
| `backend/api/routes/internal.py` | Add idempotency checks |
|
||||
| `backend/config.py` | Add ENABLE_CLOUD_TASKS setting |
|
||||
| `backend/requirements.txt` | Add google-cloud-tasks |
|
||||
|
||||
### Phase 2
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `cloudbuild-video-job.yaml` | New file for Cloud Run Job |
|
||||
| `backend/services/worker_service.py` | Add Cloud Run Jobs trigger |
|
||||
| `backend/workers/video_worker.py` | CLI entry point for Jobs |
|
||||
|
||||
---
|
||||
|
||||
## Related Documents
|
||||
|
||||
- [ARCHITECTURE.md](../01-reference/ARCHITECTURE.md) - Current system architecture
|
||||
- [BACKEND-FEATURE-PARITY-PLAN.md](./BACKEND-FEATURE-PARITY-PLAN.md) - Feature parity roadmap
|
||||
- [WORKER-IMPLEMENTATION-PLAN.md](./WORKER-IMPLEMENTATION-PLAN.md) - Worker details
|
||||
|
||||
---
|
||||
|
||||
## Changelog
|
||||
|
||||
### 2025-12-11: Analysis Complete - Updated Plan
|
||||
|
||||
- **Analyzed codebase thoroughly** - found existing pipeline architecture underutilized
|
||||
- **Confirmed Cloud Tasks as best approach** - minimal code changes, single point of change
|
||||
- **Identified WorkerService as sole modification point** - clean abstraction exists
|
||||
- **Decided against immediate pipeline unification** - infrastructure scaling is the priority
|
||||
- **Added detailed implementation code** - ready to start Phase 1
|
||||
- **Created phased approach** - Phase 1 (Cloud Tasks) → Phase 2 (Jobs) → Phase 3 (Pipeline)
|
||||
|
||||
### 2025-12-11: Initial Plan
|
||||
- Documented current architecture limitations
|
||||
- Proposed Cloud Tasks + Cloud Run Jobs hybrid
|
||||
- Outlined shared pipeline architecture
|
||||
- Created implementation roadmap
|
||||
@@ -0,0 +1,729 @@
|
||||
# Flacfetch HTTP API Service Implementation Plan
|
||||
|
||||
**Created:** 2025-12-20
|
||||
**Updated:** 2025-12-20
|
||||
**Status:** In Progress
|
||||
**Priority:** Medium
|
||||
**Estimated Effort:** 3-4 days
|
||||
|
||||
## Problem Statement
|
||||
|
||||
BitTorrent downloads don't work reliably in Cloud Run due to:
|
||||
- No inbound peer connections (NAT/firewall)
|
||||
- Ephemeral containers with no stable IP
|
||||
- DHT peer discovery blocked
|
||||
- Container timeout limits (max 60 min)
|
||||
|
||||
This prevents downloading lossless audio from private trackers (Redacted, OPS) via the remote backend.
|
||||
|
||||
## Solution Overview
|
||||
|
||||
Add an HTTP API mode to **flacfetch** (our audio fetching library) and deploy it on a dedicated GCP VM with:
|
||||
- Full BitTorrent peer connectivity via Transmission daemon
|
||||
- Static IP for tracker whitelist (seedbox-like)
|
||||
- Indefinite seeding of downloaded torrents
|
||||
- Automatic disk cleanup when space runs low
|
||||
- YouTube downloads also handled (single service for all audio acquisition)
|
||||
|
||||
This approach provides clean separation of concerns:
|
||||
- **flacfetch**: Handles ALL audio acquisition (torrents + YouTube)
|
||||
- **karaoke-gen backend**: Orchestrates karaoke generation pipeline, calls flacfetch API
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ FLACFETCH SERVICE ARCHITECTURE │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Cloud Run Backend Flacfetch VM (e2-small) │
|
||||
│ ┌────────────────────┐ ┌─────────────────────────┐ │
|
||||
│ │ │ │ │ │
|
||||
│ │ karaoke-gen │ 1. Search │ flacfetch HTTP API │ │
|
||||
│ │ AudioSearchService │ ────────────▶ │ POST /search │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ │ 2. Results │ Providers: │ │
|
||||
│ │ │ ◀──────────── │ • Redacted (FLAC) │ │
|
||||
│ │ │ │ • OPS (FLAC) │ │
|
||||
│ │ │ 3. Download │ • YouTube (fallback) │ │
|
||||
│ │ │ ────────────▶ │ │ │
|
||||
│ │ │ │ POST /download │ │
|
||||
│ │ │ 4. Status │ GET /download/{id} │ │
|
||||
│ │ │ ◀──────────── │ │ │
|
||||
│ │ │ (polling) │ Transmission Daemon │ │
|
||||
│ │ │ │ • Full peer connect │ │
|
||||
│ │ │ 5. Complete │ • DHT enabled │ │
|
||||
│ │ │ ◀──────────── │ • Indefinite seeding │ │
|
||||
│ │ │ (file path) │ │ │
|
||||
│ └────────────────────┘ └───────────┬─────────────┘ │
|
||||
│ │ │ │
|
||||
│ │ │ 6. Upload to GCS │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Google Cloud Storage │ │
|
||||
│ │ │ │
|
||||
│ │ uploads/{job_id}/audio/{filename}.flac │ │
|
||||
│ │ │ │
|
||||
│ └─────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### 1. HTTP API in flacfetch (not separate service)
|
||||
- **Why**: Clean separation of concerns, reusable by future projects, single source of truth
|
||||
- **How**: Add `flacfetch/api/` module with FastAPI, new CLI mode `flacfetch serve`
|
||||
|
||||
### 2. Indefinite Seeding
|
||||
- **Why**: VM is always-on anyway, helps maintain tracker ratio, acts like a seedbox
|
||||
- **Current behavior**: Torrent removed after download (line 317-321 in torrent.py)
|
||||
- **New behavior**: Torrent kept seeding, only removed by cleanup or explicit delete
|
||||
|
||||
### 3. Static IP
|
||||
- **Why**: Required for tracker account IP whitelist, consistent for firewall rules
|
||||
- **Cost**: ~$3/month additional
|
||||
|
||||
### 4. Disk Space Management
|
||||
- **Strategy**: Auto-cleanup oldest torrents when free space < 5GB threshold
|
||||
- **API**: Manual cleanup endpoints + automatic background cleanup
|
||||
|
||||
### 5. Instance Type: e2-small
|
||||
- **Specs**: 0.5 vCPU, 2GB RAM, ~$12/month
|
||||
- **Why**: Sufficient for I/O-bound workload, newer C3/N4 require 2+ vCPUs (overkill)
|
||||
- **Disk**: 30GB SSD (20GB for torrents + headroom)
|
||||
|
||||
### 6. Authentication
|
||||
- **Method**: Static API key in `X-API-Key` header
|
||||
- **Firewall**: Permissive (allows future web UI), auth enforced at app level
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Add HTTP API to flacfetch
|
||||
|
||||
**Location:** `./flacfetch/flacfetch/api/`
|
||||
|
||||
**New Directory Structure:**
|
||||
```
|
||||
flacfetch/
|
||||
├── flacfetch/
|
||||
│ ├── api/ # NEW
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── main.py # FastAPI app entry point
|
||||
│ │ ├── auth.py # API key authentication
|
||||
│ │ ├── models.py # Pydantic request/response models
|
||||
│ │ ├── routes/
|
||||
│ │ │ ├── __init__.py
|
||||
│ │ │ ├── search.py # POST /search
|
||||
│ │ │ ├── download.py # POST /download, GET /download/{id}/status
|
||||
│ │ │ ├── torrents.py # GET /torrents, DELETE /torrents/{id}
|
||||
│ │ │ └── health.py # GET /health, GET /disk-usage
|
||||
│ │ └── services/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── download_manager.py # Track active downloads
|
||||
│ │ └── disk_manager.py # Disk cleanup logic
|
||||
│ ├── core/
|
||||
│ ├── downloaders/
|
||||
│ │ └── torrent.py # MODIFIED: optional keep-seeding mode
|
||||
│ ├── interface/
|
||||
│ │ └── cli.py # MODIFIED: add 'serve' subcommand
|
||||
│ └── providers/
|
||||
└── pyproject.toml # MODIFIED: add fastapi, uvicorn deps
|
||||
```
|
||||
|
||||
**API Endpoints:**
|
||||
|
||||
```yaml
|
||||
# All endpoints require X-API-Key header
|
||||
|
||||
# Search for audio
|
||||
POST /search
|
||||
Request:
|
||||
{ "artist": "ABBA", "title": "Waterloo" }
|
||||
Response:
|
||||
{
|
||||
"search_id": "abc123",
|
||||
"results": [
|
||||
{
|
||||
"index": 0,
|
||||
"title": "Waterloo",
|
||||
"artist": "ABBA",
|
||||
"provider": "Redacted",
|
||||
"quality": "FLAC 16bit CD",
|
||||
"seeders": 45,
|
||||
"size_bytes": 31457280,
|
||||
"target_file": "02. Waterloo.flac",
|
||||
...
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
|
||||
# Start download
|
||||
POST /download
|
||||
Request:
|
||||
{
|
||||
"search_id": "abc123", # From previous search
|
||||
"result_index": 0, # Which result to download
|
||||
"output_filename": "ABBA - Waterloo", # Optional
|
||||
"upload_to_gcs": true, # Upload to GCS when complete
|
||||
"gcs_path": "uploads/job123/audio/" # Required if upload_to_gcs
|
||||
}
|
||||
Response:
|
||||
{
|
||||
"download_id": "dl_xyz789",
|
||||
"status": "queued"
|
||||
}
|
||||
|
||||
# Check download status
|
||||
GET /download/{download_id}/status
|
||||
Response:
|
||||
{
|
||||
"download_id": "dl_xyz789",
|
||||
"status": "downloading", # queued, downloading, uploading, seeding, complete, failed
|
||||
"progress": 45.2,
|
||||
"peers": 3,
|
||||
"download_speed_kbps": 1250.5,
|
||||
"eta_seconds": 120,
|
||||
"provider": "Redacted",
|
||||
"title": "ABBA - Waterloo",
|
||||
"output_path": null, # Set when complete (local path)
|
||||
"gcs_path": null, # Set when uploaded to GCS
|
||||
"error": null
|
||||
}
|
||||
|
||||
# List all torrents (for management)
|
||||
GET /torrents
|
||||
Response:
|
||||
{
|
||||
"torrents": [
|
||||
{
|
||||
"id": 1,
|
||||
"name": "ABBA - Waterloo [FLAC]",
|
||||
"status": "seeding",
|
||||
"progress": 100.0,
|
||||
"size_bytes": 314572800,
|
||||
"uploaded_bytes": 628145600, # Seeding stats
|
||||
"ratio": 2.0,
|
||||
"added_at": "2025-12-20T10:30:00Z"
|
||||
},
|
||||
...
|
||||
],
|
||||
"total_size_bytes": 5368709120,
|
||||
"count": 15
|
||||
}
|
||||
|
||||
# Delete a torrent (manual cleanup)
|
||||
DELETE /torrents/{torrent_id}
|
||||
Query params: ?delete_data=true # Also delete downloaded files
|
||||
Response:
|
||||
{ "status": "deleted" }
|
||||
|
||||
# Trigger cleanup (free disk space)
|
||||
POST /torrents/cleanup
|
||||
Request:
|
||||
{
|
||||
"strategy": "oldest", # oldest, largest, lowest_ratio
|
||||
"target_free_gb": 10 # Keep removing until this much free
|
||||
}
|
||||
Response:
|
||||
{
|
||||
"removed_count": 3,
|
||||
"freed_bytes": 1073741824,
|
||||
"free_space_gb": 10.5
|
||||
}
|
||||
|
||||
# Health check
|
||||
GET /health
|
||||
Response:
|
||||
{
|
||||
"status": "healthy",
|
||||
"transmission": {
|
||||
"available": true,
|
||||
"version": "4.0.5",
|
||||
"active_torrents": 5
|
||||
},
|
||||
"disk": {
|
||||
"total_gb": 30,
|
||||
"used_gb": 18,
|
||||
"free_gb": 12
|
||||
},
|
||||
"providers": {
|
||||
"redacted": true,
|
||||
"ops": true,
|
||||
"youtube": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Torrent Downloader Changes:**
|
||||
|
||||
```python
|
||||
# flacfetch/downloaders/torrent.py
|
||||
|
||||
class TorrentDownloader(Downloader):
|
||||
def __init__(self, ..., keep_seeding: bool = False):
|
||||
"""
|
||||
Args:
|
||||
keep_seeding: If True, don't remove torrent after download.
|
||||
Used in server mode for indefinite seeding.
|
||||
"""
|
||||
self.keep_seeding = keep_seeding
|
||||
|
||||
def download(self, release, output_path, output_filename=None):
|
||||
# ... existing download logic ...
|
||||
|
||||
# After download complete:
|
||||
if self.keep_seeding:
|
||||
# Don't remove, just return the file path
|
||||
logger.info(f"Download complete, keeping torrent for seeding: {torrent.name}")
|
||||
else:
|
||||
# Original behavior: stop and remove
|
||||
self.client.stop_torrent(torrent.id)
|
||||
self.client.remove_torrent(torrent.id, delete_data=True)
|
||||
```
|
||||
|
||||
**CLI Changes:**
|
||||
|
||||
```bash
|
||||
# New serve subcommand
|
||||
flacfetch serve --port 8080 --host 0.0.0.0
|
||||
|
||||
# Or via environment
|
||||
FLACFETCH_API_PORT=8080 flacfetch serve
|
||||
```
|
||||
|
||||
### Phase 2: Infrastructure (Pulumi)
|
||||
|
||||
**Location:** `./infrastructure/__main__.py`
|
||||
|
||||
**New Resources:**
|
||||
```python
|
||||
# Static IP for flacfetch service (for tracker whitelist)
|
||||
flacfetch_ip = gcp.compute.Address(
|
||||
"flacfetch-ip",
|
||||
name="flacfetch-static-ip",
|
||||
region="us-central1",
|
||||
)
|
||||
|
||||
# Service account for flacfetch VM
|
||||
flacfetch_sa = gcp.serviceaccount.Account(
|
||||
"flacfetch-sa",
|
||||
account_id="flacfetch-service",
|
||||
display_name="Flacfetch Service Account",
|
||||
)
|
||||
|
||||
# GCS write permissions for flacfetch
|
||||
gcp.storage.BucketIAMMember(
|
||||
"flacfetch-gcs-writer",
|
||||
bucket=bucket.name,
|
||||
role="roles/storage.objectCreator",
|
||||
member=pulumi.Output.concat("serviceAccount:", flacfetch_sa.email),
|
||||
)
|
||||
|
||||
# Flacfetch VM
|
||||
flacfetch_vm = gcp.compute.Instance(
|
||||
"flacfetch-service",
|
||||
machine_type="e2-small",
|
||||
zone="us-central1-a",
|
||||
boot_disk=gcp.compute.InstanceBootDiskArgs(
|
||||
initialize_params=gcp.compute.InstanceBootDiskInitializeParamsArgs(
|
||||
image="debian-cloud/debian-12",
|
||||
size=30, # GB
|
||||
type="pd-ssd",
|
||||
),
|
||||
),
|
||||
network_interfaces=[gcp.compute.InstanceNetworkInterfaceArgs(
|
||||
network="default",
|
||||
access_configs=[gcp.compute.InstanceNetworkInterfaceAccessConfigArgs(
|
||||
nat_ip=flacfetch_ip.address, # Static IP
|
||||
)],
|
||||
)],
|
||||
service_account=gcp.compute.InstanceServiceAccountArgs(
|
||||
email=flacfetch_sa.email,
|
||||
scopes=["cloud-platform"],
|
||||
),
|
||||
metadata_startup_script=FLACFETCH_STARTUP_SCRIPT,
|
||||
tags=["flacfetch-service"],
|
||||
)
|
||||
|
||||
# Firewall for flacfetch
|
||||
flacfetch_firewall = gcp.compute.Firewall(
|
||||
"flacfetch-firewall",
|
||||
network="default",
|
||||
allows=[
|
||||
# BitTorrent peer connections
|
||||
gcp.compute.FirewallAllowArgs(protocol="tcp", ports=["51413"]),
|
||||
gcp.compute.FirewallAllowArgs(protocol="udp", ports=["51413"]),
|
||||
# Flacfetch HTTP API (auth required at app level)
|
||||
gcp.compute.FirewallAllowArgs(protocol="tcp", ports=["8080"]),
|
||||
],
|
||||
source_ranges=["0.0.0.0/0"], # Permissive, auth at app level
|
||||
target_tags=["flacfetch-service"],
|
||||
)
|
||||
|
||||
# Store API key in Secret Manager
|
||||
flacfetch_api_key = gcp.secretmanager.Secret(
|
||||
"flacfetch-api-key",
|
||||
secret_id="flacfetch-api-key",
|
||||
replication=gcp.secretmanager.SecretReplicationArgs(
|
||||
auto=gcp.secretmanager.SecretReplicationAutoArgs(),
|
||||
),
|
||||
)
|
||||
|
||||
# Export for use
|
||||
pulumi.export("flacfetch_static_ip", flacfetch_ip.address)
|
||||
pulumi.export("flacfetch_service_url", pulumi.Output.concat("http://", flacfetch_ip.address, ":8080"))
|
||||
```
|
||||
|
||||
**Startup Script:**
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# Install dependencies
|
||||
apt-get update
|
||||
apt-get install -y python3-pip python3-venv transmission-daemon ffmpeg git
|
||||
|
||||
# Configure Transmission
|
||||
systemctl stop transmission-daemon
|
||||
cat > /etc/transmission-daemon/settings.json << 'EOF'
|
||||
{
|
||||
"download-dir": "/var/lib/transmission-daemon/downloads",
|
||||
"incomplete-dir": "/var/lib/transmission-daemon/.incomplete",
|
||||
"incomplete-dir-enabled": true,
|
||||
"rpc-authentication-required": false,
|
||||
"rpc-bind-address": "127.0.0.1",
|
||||
"rpc-enabled": true,
|
||||
"rpc-port": 9091,
|
||||
"rpc-whitelist-enabled": false,
|
||||
"peer-port": 51413,
|
||||
"port-forwarding-enabled": false,
|
||||
"speed-limit-down": 0,
|
||||
"speed-limit-down-enabled": false,
|
||||
"speed-limit-up": 0,
|
||||
"speed-limit-up-enabled": false,
|
||||
"ratio-limit-enabled": false,
|
||||
"umask": 2
|
||||
}
|
||||
EOF
|
||||
systemctl start transmission-daemon
|
||||
|
||||
# Install flacfetch
|
||||
cd /opt
|
||||
git clone https://github.com/nomadkaraoke/flacfetch.git
|
||||
cd flacfetch
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -e ".[api]"
|
||||
|
||||
# Get secrets from metadata/Secret Manager
|
||||
FLACFETCH_API_KEY=$(gcloud secrets versions access latest --secret=flacfetch-api-key)
|
||||
REDACTED_API_KEY=$(gcloud secrets versions access latest --secret=redacted-api-key)
|
||||
OPS_API_KEY=$(gcloud secrets versions access latest --secret=ops-api-key)
|
||||
GCS_BUCKET=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/gcs-bucket" -H "Metadata-Flavor: Google")
|
||||
|
||||
# Create systemd service
|
||||
cat > /etc/systemd/system/flacfetch.service << EOF
|
||||
[Unit]
|
||||
Description=Flacfetch HTTP API Service
|
||||
After=network.target transmission-daemon.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=root
|
||||
WorkingDirectory=/opt/flacfetch
|
||||
Environment="FLACFETCH_API_KEY=${FLACFETCH_API_KEY}"
|
||||
Environment="REDACTED_API_KEY=${REDACTED_API_KEY}"
|
||||
Environment="OPS_API_KEY=${OPS_API_KEY}"
|
||||
Environment="GCS_BUCKET=${GCS_BUCKET}"
|
||||
Environment="FLACFETCH_KEEP_SEEDING=true"
|
||||
Environment="FLACFETCH_MIN_FREE_GB=5"
|
||||
ExecStart=/opt/flacfetch/venv/bin/flacfetch serve --host 0.0.0.0 --port 8080
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl enable flacfetch
|
||||
systemctl start flacfetch
|
||||
```
|
||||
|
||||
### Phase 3: Backend Integration
|
||||
|
||||
**New File:** `backend/services/flacfetch_client.py`
|
||||
|
||||
```python
|
||||
"""
|
||||
Client for the remote flacfetch HTTP API service.
|
||||
|
||||
Used when FLACFETCH_API_URL is configured, otherwise falls back to local flacfetch.
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Optional, List, Dict, Any
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class FlacfetchServiceError(Exception):
|
||||
"""Error communicating with flacfetch service."""
|
||||
pass
|
||||
|
||||
|
||||
class FlacfetchClient:
|
||||
"""Client for remote flacfetch HTTP API."""
|
||||
|
||||
def __init__(self, base_url: str, api_key: str, timeout: int = 30):
|
||||
self.base_url = base_url.rstrip('/')
|
||||
self.api_key = api_key
|
||||
self.timeout = timeout
|
||||
|
||||
def _headers(self) -> Dict[str, str]:
|
||||
return {"X-API-Key": self.api_key}
|
||||
|
||||
async def health_check(self) -> Dict[str, Any]:
|
||||
"""Check if service is healthy."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.get(
|
||||
f"{self.base_url}/health",
|
||||
headers=self._headers(),
|
||||
timeout=10,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
async def search(self, artist: str, title: str) -> Dict[str, Any]:
|
||||
"""Search for audio."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.post(
|
||||
f"{self.base_url}/search",
|
||||
headers=self._headers(),
|
||||
json={"artist": artist, "title": title},
|
||||
timeout=self.timeout,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
async def download(
|
||||
self,
|
||||
search_id: str,
|
||||
result_index: int,
|
||||
output_filename: Optional[str] = None,
|
||||
gcs_path: Optional[str] = None,
|
||||
) -> str:
|
||||
"""
|
||||
Start download and return download_id.
|
||||
|
||||
If gcs_path is provided, file will be uploaded to GCS.
|
||||
"""
|
||||
async with httpx.AsyncClient() as client:
|
||||
payload = {
|
||||
"search_id": search_id,
|
||||
"result_index": result_index,
|
||||
}
|
||||
if output_filename:
|
||||
payload["output_filename"] = output_filename
|
||||
if gcs_path:
|
||||
payload["upload_to_gcs"] = True
|
||||
payload["gcs_path"] = gcs_path
|
||||
|
||||
resp = await client.post(
|
||||
f"{self.base_url}/download",
|
||||
headers=self._headers(),
|
||||
json=payload,
|
||||
timeout=self.timeout,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()["download_id"]
|
||||
|
||||
async def get_download_status(self, download_id: str) -> Dict[str, Any]:
|
||||
"""Get download status."""
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.get(
|
||||
f"{self.base_url}/download/{download_id}/status",
|
||||
headers=self._headers(),
|
||||
timeout=10,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
async def wait_for_download(
|
||||
self,
|
||||
download_id: str,
|
||||
timeout: int = 600,
|
||||
poll_interval: int = 2,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Wait for download to complete.
|
||||
|
||||
Returns final status dict with output_path or gcs_path.
|
||||
Raises FlacfetchServiceError on failure or timeout.
|
||||
"""
|
||||
elapsed = 0
|
||||
while elapsed < timeout:
|
||||
status = await self.get_download_status(download_id)
|
||||
|
||||
if status["status"] == "complete":
|
||||
return status
|
||||
elif status["status"] == "failed":
|
||||
raise FlacfetchServiceError(f"Download failed: {status.get('error')}")
|
||||
|
||||
await asyncio.sleep(poll_interval)
|
||||
elapsed += poll_interval
|
||||
|
||||
raise FlacfetchServiceError(f"Download timed out after {timeout}s")
|
||||
|
||||
|
||||
# Singleton
|
||||
_client: Optional[FlacfetchClient] = None
|
||||
|
||||
|
||||
def get_flacfetch_client() -> Optional[FlacfetchClient]:
|
||||
"""Get flacfetch client if configured."""
|
||||
global _client
|
||||
if _client is None:
|
||||
from backend.config import get_settings
|
||||
settings = get_settings()
|
||||
if settings.flacfetch_api_url and settings.flacfetch_api_key:
|
||||
_client = FlacfetchClient(
|
||||
base_url=settings.flacfetch_api_url,
|
||||
api_key=settings.flacfetch_api_key,
|
||||
)
|
||||
return _client
|
||||
```
|
||||
|
||||
**Modify:** `backend/services/audio_search_service.py` to use remote client when available.
|
||||
|
||||
**Modify:** `backend/config.py` to add new settings:
|
||||
```python
|
||||
flacfetch_api_url: Optional[str] = None # e.g., http://10.0.0.5:8080
|
||||
flacfetch_api_key: Optional[str] = None
|
||||
```
|
||||
|
||||
### Phase 4: Monitoring & Reliability
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Add Cloud Monitoring alert for VM CPU/memory
|
||||
- [ ] Add alert for disk space < 5GB
|
||||
- [ ] Add alert for flacfetch service down (health check fails)
|
||||
- [ ] Configure Cloud Logging export from VM
|
||||
- [ ] Add Grafana dashboard (optional)
|
||||
|
||||
**Automatic Disk Cleanup:**
|
||||
The disk manager runs as a background task in the flacfetch API service:
|
||||
```python
|
||||
# Check every 5 minutes, cleanup if free space < threshold
|
||||
async def background_cleanup_task():
|
||||
while True:
|
||||
await asyncio.sleep(300)
|
||||
free_gb = get_free_space_gb()
|
||||
if free_gb < MIN_FREE_GB:
|
||||
cleanup_oldest_torrents(target_free_gb=MIN_FREE_GB + 5)
|
||||
```
|
||||
|
||||
### Phase 5: Testing & Documentation
|
||||
|
||||
**Testing:**
|
||||
- [ ] Unit tests for flacfetch API endpoints
|
||||
- [ ] Unit tests for disk manager
|
||||
- [ ] Integration test: search → download → GCS upload
|
||||
- [ ] Mock tests for karaoke-gen backend client
|
||||
|
||||
**Documentation:**
|
||||
- [ ] Update `docs/01-reference/ARCHITECTURE.md` with flacfetch service
|
||||
- [ ] Add `flacfetch/README.md` HTTP API section
|
||||
- [ ] Document deployment process
|
||||
- [ ] Document environment variables
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Flacfetch Service (VM)
|
||||
```bash
|
||||
# API Configuration
|
||||
FLACFETCH_API_KEY=<secret> # Required for API auth
|
||||
FLACFETCH_API_PORT=8080 # Default: 8080
|
||||
FLACFETCH_API_HOST=0.0.0.0 # Default: 0.0.0.0
|
||||
|
||||
# Tracker API Keys
|
||||
REDACTED_API_KEY=<key> # For Redacted provider
|
||||
OPS_API_KEY=<key> # For OPS provider
|
||||
|
||||
# GCS Upload (optional)
|
||||
GCS_BUCKET=karaoke-uploads # For uploading completed downloads
|
||||
GOOGLE_CLOUD_PROJECT=nomadkaraoke
|
||||
|
||||
# Transmission
|
||||
TRANSMISSION_HOST=localhost # Default: localhost
|
||||
TRANSMISSION_PORT=9091 # Default: 9091
|
||||
|
||||
# Behavior
|
||||
FLACFETCH_KEEP_SEEDING=true # Keep torrents seeding after download
|
||||
FLACFETCH_MIN_FREE_GB=5 # Minimum free disk space before cleanup
|
||||
FLACFETCH_DOWNLOAD_DIR=/var/lib/transmission-daemon/downloads
|
||||
```
|
||||
|
||||
### Cloud Run Backend
|
||||
```bash
|
||||
# New variables
|
||||
FLACFETCH_API_URL=http://10.128.0.5:8080 # Internal IP of flacfetch VM
|
||||
FLACFETCH_API_KEY=<same-secret>
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **API Authentication**: Static API key in `X-API-Key` header
|
||||
2. **Firewall**: Permissive for future web UI, auth enforced at application level
|
||||
3. **GCS**: Service account with `storage.objectCreator` role only
|
||||
4. **Transmission**: RPC bound to localhost only, no external access
|
||||
5. **Static IP**: Can be whitelisted on tracker accounts
|
||||
|
||||
## Rollback Plan
|
||||
|
||||
If flacfetch service fails:
|
||||
1. Set `FLACFETCH_API_URL=""` in Cloud Run (disables remote service)
|
||||
2. Backend falls back to local flacfetch (YouTube only in Cloud Run)
|
||||
3. Users see warning that lossless downloads may be slower/unavailable
|
||||
|
||||
## Cost Estimate
|
||||
|
||||
| Resource | Monthly Cost |
|
||||
|----------|--------------|
|
||||
| e2-small VM | ~$12 |
|
||||
| Static IP | ~$3 |
|
||||
| 30GB SSD | ~$5 |
|
||||
| Network egress | ~$2-5 |
|
||||
| **Total** | **~$22-25/month** |
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Flacfetch HTTP API accepts search requests and returns results
|
||||
- [ ] Downloads complete successfully for Redacted/OPS sources
|
||||
- [ ] Downloaded files upload to GCS automatically
|
||||
- [ ] Torrents continue seeding after download
|
||||
- [ ] Disk cleanup triggers automatically when space is low
|
||||
- [ ] Remote CLI shows download progress from flacfetch service
|
||||
- [ ] Fallback to local flacfetch works when service unavailable
|
||||
- [ ] Static IP can be whitelisted on tracker accounts
|
||||
|
||||
## Timeline
|
||||
|
||||
| Phase | Duration | Dependencies |
|
||||
|-------|----------|--------------|
|
||||
| Phase 1: Flacfetch HTTP API | 1.5 days | None |
|
||||
| Phase 2: Infrastructure | 0.5 day | Phase 1 |
|
||||
| Phase 3: Backend Integration | 0.5 day | Phase 1, 2 |
|
||||
| Phase 4: Monitoring | 0.5 day | Phase 2 |
|
||||
| Phase 5: Testing & Docs | 0.5 day | Phase 1-3 |
|
||||
| **Total** | **3.5 days** | |
|
||||
|
||||
## References
|
||||
|
||||
- [Transmission RPC Spec](https://github.com/transmission/transmission/blob/main/docs/rpc-spec.md)
|
||||
- [transmission-rpc Python library](https://github.com/Trim21/transmission-rpc)
|
||||
- [FastAPI Documentation](https://fastapi.tiangolo.com/)
|
||||
- [flacfetch source code](./flacfetch/)
|
||||
@@ -0,0 +1,603 @@
|
||||
"use client"
|
||||
|
||||
import { useParams, useRouter } from "next/navigation"
|
||||
import { useState, useEffect, useCallback } from "react"
|
||||
import { AppHeader } from "@/components/app-header"
|
||||
import { ProtectedRoute } from "@/components/protected-route"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import { Alert, AlertDescription } from "@/components/ui/alert"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import { RadioGroup, RadioGroupItem } from "@/components/ui/radio-group"
|
||||
import { Label } from "@/components/ui/label"
|
||||
import { Skeleton } from "@/components/ui/skeleton"
|
||||
import {
|
||||
WaveformDisplay,
|
||||
AudioPlayer,
|
||||
RegionSelector,
|
||||
} from "@/components/instrumental-review"
|
||||
import {
|
||||
ArrowLeft,
|
||||
AlertCircle,
|
||||
CheckCircle2,
|
||||
Loader2,
|
||||
Music,
|
||||
VolumeX,
|
||||
Volume2,
|
||||
} from "lucide-react"
|
||||
|
||||
interface AudibleSegment {
|
||||
start_seconds: number
|
||||
end_seconds: number
|
||||
duration_seconds: number
|
||||
avg_amplitude_db: number
|
||||
peak_amplitude_db?: number
|
||||
}
|
||||
|
||||
interface MuteRegion {
|
||||
start_seconds: number
|
||||
end_seconds: number
|
||||
}
|
||||
|
||||
interface AnalysisData {
|
||||
has_audible_content: boolean
|
||||
total_duration_seconds: number
|
||||
audible_segments: AudibleSegment[]
|
||||
recommended_selection: string
|
||||
total_audible_duration_seconds: number
|
||||
audible_percentage: number
|
||||
silence_threshold_db: number
|
||||
}
|
||||
|
||||
interface InstrumentalAnalysisResponse {
|
||||
job_id: string
|
||||
artist: string
|
||||
title: string
|
||||
status: string
|
||||
analysis: AnalysisData
|
||||
audio_urls: {
|
||||
clean_instrumental: string | null
|
||||
backing_vocals: string | null
|
||||
with_backing: string | null
|
||||
custom_instrumental: string | null
|
||||
}
|
||||
waveform_url: string | null
|
||||
has_custom_instrumental: boolean
|
||||
}
|
||||
|
||||
// Get API base URL from environment or default to relative path
|
||||
const API_BASE = process.env.NEXT_PUBLIC_API_URL || "/api"
|
||||
|
||||
function InstrumentalReviewContent() {
|
||||
const params = useParams()
|
||||
const router = useRouter()
|
||||
const jobId = params.id as string
|
||||
|
||||
// State
|
||||
const [isLoading, setIsLoading] = useState(true)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const [analysisData, setAnalysisData] = useState<InstrumentalAnalysisResponse | null>(null)
|
||||
const [waveformData, setWaveformData] = useState<{ amplitudes: number[]; duration: number } | null>(null)
|
||||
|
||||
const [selectedOption, setSelectedOption] = useState<string>("clean")
|
||||
const [muteRegions, setMuteRegions] = useState<MuteRegion[]>([])
|
||||
const [isSelectionMode, setIsSelectionMode] = useState(false)
|
||||
const [currentTime, setCurrentTime] = useState(0)
|
||||
const [seekTo, setSeekTo] = useState<number | undefined>(undefined)
|
||||
|
||||
const [isCreatingCustom, setIsCreatingCustom] = useState(false)
|
||||
const [isSubmitting, setIsSubmitting] = useState(false)
|
||||
const [activeAudio, setActiveAudio] = useState<"clean" | "backing" | "with_backing" | "custom">("backing")
|
||||
|
||||
// Fetch analysis data
|
||||
const fetchAnalysis = useCallback(async () => {
|
||||
try {
|
||||
setIsLoading(true)
|
||||
setError(null)
|
||||
|
||||
const response = await fetch(`${API_BASE}/jobs/${jobId}/instrumental-analysis`)
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}))
|
||||
throw new Error(errorData.detail || `Failed to fetch analysis: ${response.status}`)
|
||||
}
|
||||
|
||||
const data: InstrumentalAnalysisResponse = await response.json()
|
||||
setAnalysisData(data)
|
||||
|
||||
// Set initial selection based on recommendation
|
||||
const recommendation = data.analysis.recommended_selection
|
||||
if (recommendation === "clean") {
|
||||
setSelectedOption("clean")
|
||||
} else if (recommendation === "with_backing") {
|
||||
setSelectedOption("with_backing")
|
||||
} else {
|
||||
// "review_needed" - default to with_backing but user should review
|
||||
setSelectedOption("with_backing")
|
||||
}
|
||||
|
||||
// Fetch waveform data for interactive display
|
||||
const waveformResponse = await fetch(`${API_BASE}/jobs/${jobId}/waveform-data?num_points=600`)
|
||||
if (waveformResponse.ok) {
|
||||
const waveform = await waveformResponse.json()
|
||||
setWaveformData(waveform)
|
||||
}
|
||||
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "Failed to load analysis data")
|
||||
} finally {
|
||||
setIsLoading(false)
|
||||
}
|
||||
}, [jobId])
|
||||
|
||||
useEffect(() => {
|
||||
fetchAnalysis()
|
||||
}, [fetchAnalysis])
|
||||
|
||||
// Handle seeking from waveform click
|
||||
const handleSeek = (time: number) => {
|
||||
setSeekTo(time)
|
||||
// Reset seekTo after a short delay
|
||||
setTimeout(() => setSeekTo(undefined), 100)
|
||||
}
|
||||
|
||||
// Handle adding a mute region
|
||||
const handleAddRegion = (region: MuteRegion) => {
|
||||
setMuteRegions((prev) => {
|
||||
// Check for overlaps and merge if needed
|
||||
const newRegions = [...prev, region].sort((a, b) => a.start_seconds - b.start_seconds)
|
||||
|
||||
// Merge overlapping regions
|
||||
const merged: MuteRegion[] = []
|
||||
for (const r of newRegions) {
|
||||
if (merged.length === 0) {
|
||||
merged.push(r)
|
||||
} else {
|
||||
const last = merged[merged.length - 1]
|
||||
if (r.start_seconds <= last.end_seconds) {
|
||||
last.end_seconds = Math.max(last.end_seconds, r.end_seconds)
|
||||
} else {
|
||||
merged.push(r)
|
||||
}
|
||||
}
|
||||
}
|
||||
return merged
|
||||
})
|
||||
setIsSelectionMode(false)
|
||||
}
|
||||
|
||||
// Handle removing a mute region
|
||||
const handleRemoveRegion = (index: number) => {
|
||||
setMuteRegions((prev) => prev.filter((_, i) => i !== index))
|
||||
}
|
||||
|
||||
// Handle clearing all mute regions
|
||||
const handleClearAll = () => {
|
||||
setMuteRegions([])
|
||||
}
|
||||
|
||||
// Create custom instrumental
|
||||
const handleCreateCustom = async () => {
|
||||
if (muteRegions.length === 0) {
|
||||
setError("Please select at least one region to mute")
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
setIsCreatingCustom(true)
|
||||
setError(null)
|
||||
|
||||
const response = await fetch(`${API_BASE}/jobs/${jobId}/create-custom-instrumental`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
mute_regions: muteRegions.map((r) => ({
|
||||
start_seconds: r.start_seconds,
|
||||
end_seconds: r.end_seconds,
|
||||
})),
|
||||
}),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}))
|
||||
throw new Error(errorData.detail || "Failed to create custom instrumental")
|
||||
}
|
||||
|
||||
// Refresh analysis to get updated URLs
|
||||
await fetchAnalysis()
|
||||
setSelectedOption("custom")
|
||||
setActiveAudio("custom")
|
||||
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "Failed to create custom instrumental")
|
||||
} finally {
|
||||
setIsCreatingCustom(false)
|
||||
}
|
||||
}
|
||||
|
||||
// Submit selection
|
||||
const handleSubmit = async () => {
|
||||
let selection = selectedOption
|
||||
|
||||
// If "review" is selected, determine based on mute regions
|
||||
if (selectedOption === "review") {
|
||||
if (muteRegions.length > 0 && analysisData?.has_custom_instrumental) {
|
||||
selection = "custom"
|
||||
} else {
|
||||
selection = "with_backing"
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
setIsSubmitting(true)
|
||||
setError(null)
|
||||
|
||||
const response = await fetch(`${API_BASE}/jobs/${jobId}/select-instrumental`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ selection }),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}))
|
||||
throw new Error(errorData.detail || "Failed to submit selection")
|
||||
}
|
||||
|
||||
// Navigate back to job page
|
||||
router.push(`/jobs/${jobId}`)
|
||||
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "Failed to submit selection")
|
||||
} finally {
|
||||
setIsSubmitting(false)
|
||||
}
|
||||
}
|
||||
|
||||
// Get current audio URL based on active selection
|
||||
const getCurrentAudioUrl = () => {
|
||||
if (!analysisData) return null
|
||||
switch (activeAudio) {
|
||||
case "clean":
|
||||
return analysisData.audio_urls.clean_instrumental
|
||||
case "backing":
|
||||
return analysisData.audio_urls.backing_vocals
|
||||
case "with_backing":
|
||||
return analysisData.audio_urls.with_backing
|
||||
case "custom":
|
||||
return analysisData.audio_urls.custom_instrumental
|
||||
default:
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
// Loading state
|
||||
if (isLoading) {
|
||||
return (
|
||||
<div className="min-h-screen bg-background">
|
||||
<AppHeader />
|
||||
<main className="container mx-auto px-4 py-8 max-w-5xl">
|
||||
<Skeleton className="h-8 w-48 mb-4" />
|
||||
<Skeleton className="h-4 w-96 mb-8" />
|
||||
<Skeleton className="h-[200px] w-full mb-4" />
|
||||
<Skeleton className="h-32 w-full" />
|
||||
</main>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// Error state
|
||||
if (error && !analysisData) {
|
||||
return (
|
||||
<div className="min-h-screen bg-background">
|
||||
<AppHeader />
|
||||
<main className="container mx-auto px-4 py-8 max-w-5xl">
|
||||
<Alert variant="destructive">
|
||||
<AlertCircle className="h-4 w-4" />
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
<Button variant="outline" className="mt-4" onClick={() => router.push(`/jobs/${jobId}`)}>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Job
|
||||
</Button>
|
||||
</main>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
const analysis = analysisData?.analysis
|
||||
|
||||
return (
|
||||
<div className="min-h-screen bg-background">
|
||||
<AppHeader />
|
||||
|
||||
<main className="container mx-auto px-4 py-8 max-w-6xl">
|
||||
{/* Header */}
|
||||
<Button variant="ghost" className="mb-6" onClick={() => router.push(`/jobs/${jobId}`)}>
|
||||
<ArrowLeft className="w-4 h-4 mr-2" />
|
||||
Back to Job
|
||||
</Button>
|
||||
|
||||
<div className="mb-8">
|
||||
<h1 className="text-3xl font-bold mb-2">
|
||||
Select Instrumental Track
|
||||
</h1>
|
||||
<p className="text-muted-foreground">
|
||||
{analysisData?.artist} - {analysisData?.title}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Error Alert */}
|
||||
{error && (
|
||||
<Alert variant="destructive" className="mb-6">
|
||||
<AlertCircle className="h-4 w-4" />
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{/* Analysis Summary */}
|
||||
{analysis && (
|
||||
<Card className="mb-6">
|
||||
<CardHeader>
|
||||
<CardTitle className="text-lg flex items-center gap-2">
|
||||
<Music className="h-5 w-5" />
|
||||
Backing Vocals Analysis
|
||||
</CardTitle>
|
||||
<CardDescription>
|
||||
Automated analysis of the backing vocals stem
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="flex flex-wrap gap-4">
|
||||
<div className="flex items-center gap-2">
|
||||
{analysis.has_audible_content ? (
|
||||
<Volume2 className="h-4 w-4 text-yellow-500" />
|
||||
) : (
|
||||
<VolumeX className="h-4 w-4 text-green-500" />
|
||||
)}
|
||||
<span className="text-sm">
|
||||
{analysis.has_audible_content
|
||||
? "Backing vocals detected"
|
||||
: "No backing vocals detected"}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{analysis.has_audible_content && (
|
||||
<>
|
||||
<Badge variant="outline">
|
||||
{analysis.audible_segments.length} audible segments
|
||||
</Badge>
|
||||
<Badge variant="outline">
|
||||
{analysis.audible_percentage.toFixed(1)}% of track
|
||||
</Badge>
|
||||
<Badge variant="outline">
|
||||
{analysis.total_audible_duration_seconds.toFixed(1)}s total
|
||||
</Badge>
|
||||
</>
|
||||
)}
|
||||
|
||||
<Badge
|
||||
variant={analysis.recommended_selection === "clean" ? "default" : "secondary"}
|
||||
>
|
||||
Recommended: {analysis.recommended_selection === "clean" ? "Clean Instrumental" : "Review Needed"}
|
||||
</Badge>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Waveform Display */}
|
||||
{waveformData && analysis && (
|
||||
<Card className="mb-6">
|
||||
<CardHeader>
|
||||
<CardTitle className="text-lg">Backing Vocals Waveform</CardTitle>
|
||||
<CardDescription>
|
||||
Click to seek • {isSelectionMode ? "Drag to select mute region" : "Click 'Add Region' to start selecting"}
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<WaveformDisplay
|
||||
amplitudes={waveformData.amplitudes}
|
||||
duration={waveformData.duration}
|
||||
currentTime={currentTime}
|
||||
segments={analysis.audible_segments}
|
||||
muteRegions={muteRegions}
|
||||
onSeek={handleSeek}
|
||||
onRegionSelect={handleAddRegion}
|
||||
isSelecting={isSelectionMode}
|
||||
width={900}
|
||||
height={150}
|
||||
className="w-full overflow-x-auto"
|
||||
/>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
<div className="grid lg:grid-cols-2 gap-6 mb-6">
|
||||
{/* Audio Players */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-lg">Audio Preview</CardTitle>
|
||||
<CardDescription>
|
||||
Listen to different instrumental options
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
{/* Audio source selector */}
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button
|
||||
variant={activeAudio === "backing" ? "default" : "outline"}
|
||||
size="sm"
|
||||
onClick={() => setActiveAudio("backing")}
|
||||
>
|
||||
Backing Vocals
|
||||
</Button>
|
||||
<Button
|
||||
variant={activeAudio === "clean" ? "default" : "outline"}
|
||||
size="sm"
|
||||
onClick={() => setActiveAudio("clean")}
|
||||
>
|
||||
Clean Instrumental
|
||||
</Button>
|
||||
<Button
|
||||
variant={activeAudio === "with_backing" ? "default" : "outline"}
|
||||
size="sm"
|
||||
onClick={() => setActiveAudio("with_backing")}
|
||||
>
|
||||
With Backing
|
||||
</Button>
|
||||
{analysisData?.has_custom_instrumental && (
|
||||
<Button
|
||||
variant={activeAudio === "custom" ? "default" : "outline"}
|
||||
size="sm"
|
||||
onClick={() => setActiveAudio("custom")}
|
||||
>
|
||||
Custom
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Audio player */}
|
||||
{getCurrentAudioUrl() && (
|
||||
<AudioPlayer
|
||||
src={getCurrentAudioUrl()!}
|
||||
onTimeUpdate={setCurrentTime}
|
||||
seekTo={seekTo}
|
||||
label={
|
||||
activeAudio === "backing"
|
||||
? "Backing Vocals Only"
|
||||
: activeAudio === "clean"
|
||||
? "Clean Instrumental"
|
||||
: activeAudio === "custom"
|
||||
? "Custom Instrumental"
|
||||
: "Instrumental + Backing Vocals"
|
||||
}
|
||||
/>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Region Selector */}
|
||||
<RegionSelector
|
||||
muteRegions={muteRegions}
|
||||
audibleSegments={analysis?.audible_segments}
|
||||
onAddRegion={handleAddRegion}
|
||||
onRemoveRegion={handleRemoveRegion}
|
||||
onClearAll={handleClearAll}
|
||||
onSeekTo={handleSeek}
|
||||
isSelectionMode={isSelectionMode}
|
||||
onToggleSelectionMode={() => setIsSelectionMode(!isSelectionMode)}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* Create Custom Button */}
|
||||
{muteRegions.length > 0 && !analysisData?.has_custom_instrumental && (
|
||||
<Card className="mb-6">
|
||||
<CardContent className="py-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<p className="font-medium">Ready to create custom instrumental</p>
|
||||
<p className="text-sm text-muted-foreground">
|
||||
{muteRegions.length} region{muteRegions.length > 1 ? "s" : ""} will be muted
|
||||
</p>
|
||||
</div>
|
||||
<Button onClick={handleCreateCustom} disabled={isCreatingCustom}>
|
||||
{isCreatingCustom ? (
|
||||
<>
|
||||
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
|
||||
Creating...
|
||||
</>
|
||||
) : (
|
||||
"Create Custom Instrumental"
|
||||
)}
|
||||
</Button>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Selection Options */}
|
||||
<Card className="mb-6">
|
||||
<CardHeader>
|
||||
<CardTitle className="text-lg">Final Selection</CardTitle>
|
||||
<CardDescription>
|
||||
Choose which instrumental to use for your karaoke video
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<RadioGroup value={selectedOption} onValueChange={setSelectedOption}>
|
||||
<div className="space-y-4">
|
||||
<div className="flex items-center space-x-3">
|
||||
<RadioGroupItem value="clean" id="clean" />
|
||||
<Label htmlFor="clean" className="flex-1 cursor-pointer">
|
||||
<div className="font-medium">Clean Instrumental</div>
|
||||
<div className="text-sm text-muted-foreground">
|
||||
Use the instrumental with no backing vocals at all
|
||||
</div>
|
||||
</Label>
|
||||
{analysis?.recommended_selection === "clean" && (
|
||||
<Badge variant="default" className="bg-green-600">
|
||||
<CheckCircle2 className="h-3 w-3 mr-1" />
|
||||
Recommended
|
||||
</Badge>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="flex items-center space-x-3">
|
||||
<RadioGroupItem value="with_backing" id="with_backing" />
|
||||
<Label htmlFor="with_backing" className="flex-1 cursor-pointer">
|
||||
<div className="font-medium">Instrumental with Backing Vocals</div>
|
||||
<div className="text-sm text-muted-foreground">
|
||||
Use the instrumental with all backing vocals included
|
||||
</div>
|
||||
</Label>
|
||||
</div>
|
||||
|
||||
{analysisData?.has_custom_instrumental && (
|
||||
<div className="flex items-center space-x-3">
|
||||
<RadioGroupItem value="custom" id="custom" />
|
||||
<Label htmlFor="custom" className="flex-1 cursor-pointer">
|
||||
<div className="font-medium">Custom Instrumental</div>
|
||||
<div className="text-sm text-muted-foreground">
|
||||
Use your custom instrumental with {muteRegions.length} muted region{muteRegions.length > 1 ? "s" : ""}
|
||||
</div>
|
||||
</Label>
|
||||
<Badge variant="secondary">Custom</Badge>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</RadioGroup>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Submit Button */}
|
||||
<div className="flex gap-4">
|
||||
<Button
|
||||
size="lg"
|
||||
onClick={handleSubmit}
|
||||
disabled={isSubmitting}
|
||||
className="flex-1 max-w-md"
|
||||
>
|
||||
{isSubmitting ? (
|
||||
<>
|
||||
<Loader2 className="h-4 w-4 mr-2 animate-spin" />
|
||||
Submitting...
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<CheckCircle2 className="h-4 w-4 mr-2" />
|
||||
Confirm Selection & Continue
|
||||
</>
|
||||
)}
|
||||
</Button>
|
||||
</div>
|
||||
</main>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export default function InstrumentalReviewPage() {
|
||||
return (
|
||||
<ProtectedRoute>
|
||||
<InstrumentalReviewContent />
|
||||
</ProtectedRoute>
|
||||
)
|
||||
}
|
||||
@@ -190,8 +190,15 @@ function JobDetailContent() {
|
||||
{job.status === "awaiting_instrumental" && (
|
||||
<Alert className="bg-secondary/10 border-secondary/30">
|
||||
<AlertCircle className="h-4 w-4 text-secondary" />
|
||||
<AlertDescription className="text-foreground">
|
||||
Multiple instrumental versions were detected. Please select your preferred version to continue.
|
||||
<AlertDescription className="text-foreground flex items-center justify-between">
|
||||
<span>Multiple instrumental versions were detected. Please select your preferred version to continue.</span>
|
||||
<Button
|
||||
size="sm"
|
||||
onClick={() => router.push(`/jobs/${job.id}/instrumental-review`)}
|
||||
className="ml-4"
|
||||
>
|
||||
Select Instrumental
|
||||
</Button>
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
"use client"
|
||||
|
||||
import { useRef, useState, useEffect, useCallback } from "react"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Slider } from "@/components/ui/slider"
|
||||
import { Play, Pause, SkipBack, SkipForward, Volume2, VolumeX } from "lucide-react"
|
||||
|
||||
interface AudioPlayerProps {
|
||||
src: string
|
||||
onTimeUpdate?: (currentTime: number) => void
|
||||
onDurationChange?: (duration: number) => void
|
||||
seekTo?: number
|
||||
label?: string
|
||||
className?: string
|
||||
}
|
||||
|
||||
export function AudioPlayer({
|
||||
src,
|
||||
onTimeUpdate,
|
||||
onDurationChange,
|
||||
seekTo,
|
||||
label,
|
||||
className = "",
|
||||
}: AudioPlayerProps) {
|
||||
const audioRef = useRef<HTMLAudioElement>(null)
|
||||
const [isPlaying, setIsPlaying] = useState(false)
|
||||
const [currentTime, setCurrentTime] = useState(0)
|
||||
const [duration, setDuration] = useState(0)
|
||||
const [volume, setVolume] = useState(1)
|
||||
const [isMuted, setIsMuted] = useState(false)
|
||||
const [isLoading, setIsLoading] = useState(true)
|
||||
|
||||
// Handle external seek requests
|
||||
useEffect(() => {
|
||||
if (seekTo !== undefined && audioRef.current && !isNaN(seekTo)) {
|
||||
audioRef.current.currentTime = seekTo
|
||||
}
|
||||
}, [seekTo])
|
||||
|
||||
// Handle audio events
|
||||
useEffect(() => {
|
||||
const audio = audioRef.current
|
||||
if (!audio) return
|
||||
|
||||
const handleTimeUpdate = () => {
|
||||
setCurrentTime(audio.currentTime)
|
||||
onTimeUpdate?.(audio.currentTime)
|
||||
}
|
||||
|
||||
const handleDurationChange = () => {
|
||||
setDuration(audio.duration)
|
||||
onDurationChange?.(audio.duration)
|
||||
}
|
||||
|
||||
const handleLoadedData = () => {
|
||||
setIsLoading(false)
|
||||
setDuration(audio.duration)
|
||||
onDurationChange?.(audio.duration)
|
||||
}
|
||||
|
||||
const handleEnded = () => {
|
||||
setIsPlaying(false)
|
||||
}
|
||||
|
||||
const handleLoadStart = () => {
|
||||
setIsLoading(true)
|
||||
}
|
||||
|
||||
const handleCanPlay = () => {
|
||||
setIsLoading(false)
|
||||
}
|
||||
|
||||
// Sync isPlaying state with actual audio element state
|
||||
const handlePlay = () => {
|
||||
setIsPlaying(true)
|
||||
}
|
||||
|
||||
const handlePause = () => {
|
||||
setIsPlaying(false)
|
||||
}
|
||||
|
||||
audio.addEventListener("timeupdate", handleTimeUpdate)
|
||||
audio.addEventListener("durationchange", handleDurationChange)
|
||||
audio.addEventListener("loadeddata", handleLoadedData)
|
||||
audio.addEventListener("ended", handleEnded)
|
||||
audio.addEventListener("loadstart", handleLoadStart)
|
||||
audio.addEventListener("canplay", handleCanPlay)
|
||||
audio.addEventListener("play", handlePlay)
|
||||
audio.addEventListener("pause", handlePause)
|
||||
|
||||
return () => {
|
||||
audio.removeEventListener("timeupdate", handleTimeUpdate)
|
||||
audio.removeEventListener("durationchange", handleDurationChange)
|
||||
audio.removeEventListener("loadeddata", handleLoadedData)
|
||||
audio.removeEventListener("ended", handleEnded)
|
||||
audio.removeEventListener("loadstart", handleLoadStart)
|
||||
audio.removeEventListener("canplay", handleCanPlay)
|
||||
audio.removeEventListener("play", handlePlay)
|
||||
audio.removeEventListener("pause", handlePause)
|
||||
}
|
||||
}, [onTimeUpdate, onDurationChange])
|
||||
|
||||
// Update volume
|
||||
useEffect(() => {
|
||||
if (audioRef.current) {
|
||||
audioRef.current.volume = isMuted ? 0 : volume
|
||||
}
|
||||
}, [volume, isMuted])
|
||||
|
||||
const togglePlay = useCallback(async () => {
|
||||
const audio = audioRef.current
|
||||
if (!audio) return
|
||||
|
||||
try {
|
||||
if (audio.paused) {
|
||||
await audio.play()
|
||||
} else {
|
||||
audio.pause()
|
||||
}
|
||||
// State will be updated by the play/pause event listeners
|
||||
} catch (error) {
|
||||
// Play was prevented (e.g., user hasn't interacted with page yet)
|
||||
console.warn("Audio playback was prevented:", error)
|
||||
}
|
||||
}, [])
|
||||
|
||||
const handleSeek = useCallback((value: number[]) => {
|
||||
const audio = audioRef.current
|
||||
if (!audio) return
|
||||
|
||||
const newTime = value[0]
|
||||
audio.currentTime = newTime
|
||||
setCurrentTime(newTime)
|
||||
}, [])
|
||||
|
||||
const skipBackward = useCallback(() => {
|
||||
const audio = audioRef.current
|
||||
if (!audio) return
|
||||
|
||||
audio.currentTime = Math.max(0, audio.currentTime - 5)
|
||||
}, [])
|
||||
|
||||
const skipForward = useCallback(() => {
|
||||
const audio = audioRef.current
|
||||
if (!audio) return
|
||||
|
||||
audio.currentTime = Math.min(duration, audio.currentTime + 5)
|
||||
}, [duration])
|
||||
|
||||
const toggleMute = useCallback(() => {
|
||||
setIsMuted(!isMuted)
|
||||
}, [isMuted])
|
||||
|
||||
const handleVolumeChange = useCallback((value: number[]) => {
|
||||
setVolume(value[0])
|
||||
if (value[0] > 0 && isMuted) {
|
||||
setIsMuted(false)
|
||||
}
|
||||
}, [isMuted])
|
||||
|
||||
const formatTime = (time: number) => {
|
||||
if (isNaN(time)) return "0:00"
|
||||
const minutes = Math.floor(time / 60)
|
||||
const seconds = Math.floor(time % 60)
|
||||
return `${minutes}:${seconds.toString().padStart(2, "0")}`
|
||||
}
|
||||
|
||||
return (
|
||||
<div className={`bg-card rounded-lg p-4 ${className}`}>
|
||||
<audio ref={audioRef} src={src} preload="metadata" />
|
||||
|
||||
{label && (
|
||||
<div className="text-sm font-medium text-muted-foreground mb-3">
|
||||
{label}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Progress bar */}
|
||||
<div className="mb-4">
|
||||
<Slider
|
||||
value={[currentTime]}
|
||||
max={duration || 100}
|
||||
step={0.1}
|
||||
onValueChange={handleSeek}
|
||||
disabled={isLoading}
|
||||
className="w-full"
|
||||
/>
|
||||
<div className="flex justify-between text-xs text-muted-foreground mt-1">
|
||||
<span>{formatTime(currentTime)}</span>
|
||||
<span>{formatTime(duration)}</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Controls */}
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={skipBackward}
|
||||
disabled={isLoading}
|
||||
className="h-8 w-8"
|
||||
>
|
||||
<SkipBack className="h-4 w-4" />
|
||||
</Button>
|
||||
|
||||
<Button
|
||||
variant="default"
|
||||
size="icon"
|
||||
onClick={togglePlay}
|
||||
disabled={isLoading}
|
||||
className="h-10 w-10"
|
||||
>
|
||||
{isPlaying ? (
|
||||
<Pause className="h-5 w-5" />
|
||||
) : (
|
||||
<Play className="h-5 w-5 ml-0.5" />
|
||||
)}
|
||||
</Button>
|
||||
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={skipForward}
|
||||
disabled={isLoading}
|
||||
className="h-8 w-8"
|
||||
>
|
||||
<SkipForward className="h-4 w-4" />
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Volume control */}
|
||||
<div className="flex items-center gap-2">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={toggleMute}
|
||||
className="h-8 w-8"
|
||||
>
|
||||
{isMuted || volume === 0 ? (
|
||||
<VolumeX className="h-4 w-4" />
|
||||
) : (
|
||||
<Volume2 className="h-4 w-4" />
|
||||
)}
|
||||
</Button>
|
||||
<Slider
|
||||
value={[isMuted ? 0 : volume]}
|
||||
max={1}
|
||||
step={0.1}
|
||||
onValueChange={handleVolumeChange}
|
||||
className="w-20"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{isLoading && (
|
||||
<div className="text-xs text-muted-foreground text-center mt-2">
|
||||
Loading audio...
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
export { WaveformDisplay } from "./waveform-display"
|
||||
export { AudioPlayer } from "./audio-player"
|
||||
export { RegionSelector } from "./region-selector"
|
||||
@@ -0,0 +1,222 @@
|
||||
"use client"
|
||||
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"
|
||||
import { Badge } from "@/components/ui/badge"
|
||||
import { Trash2, VolumeX, Plus } from "lucide-react"
|
||||
|
||||
interface MuteRegion {
|
||||
start_seconds: number
|
||||
end_seconds: number
|
||||
}
|
||||
|
||||
interface AudibleSegment {
|
||||
start_seconds: number
|
||||
end_seconds: number
|
||||
duration_seconds: number
|
||||
avg_amplitude_db: number
|
||||
}
|
||||
|
||||
interface RegionSelectorProps {
|
||||
muteRegions: MuteRegion[]
|
||||
audibleSegments?: AudibleSegment[]
|
||||
onAddRegion: (region: MuteRegion) => void
|
||||
onRemoveRegion: (index: number) => void
|
||||
onClearAll: () => void
|
||||
onSeekTo?: (time: number) => void
|
||||
isSelectionMode: boolean
|
||||
onToggleSelectionMode: () => void
|
||||
className?: string
|
||||
}
|
||||
|
||||
export function RegionSelector({
|
||||
muteRegions,
|
||||
audibleSegments = [],
|
||||
onAddRegion,
|
||||
onRemoveRegion,
|
||||
onClearAll,
|
||||
onSeekTo,
|
||||
isSelectionMode,
|
||||
onToggleSelectionMode,
|
||||
className = "",
|
||||
}: RegionSelectorProps) {
|
||||
const formatTime = (seconds: number) => {
|
||||
const mins = Math.floor(seconds / 60)
|
||||
const secs = Math.floor(seconds % 60)
|
||||
const ms = Math.floor((seconds % 1) * 100)
|
||||
return `${mins}:${secs.toString().padStart(2, "0")}.${ms.toString().padStart(2, "0")}`
|
||||
}
|
||||
|
||||
const formatDuration = (seconds: number) => {
|
||||
if (seconds < 1) {
|
||||
return `${Math.round(seconds * 1000)}ms`
|
||||
}
|
||||
return `${seconds.toFixed(1)}s`
|
||||
}
|
||||
|
||||
const totalMutedDuration = muteRegions.reduce(
|
||||
(sum, r) => sum + (r.end_seconds - r.start_seconds),
|
||||
0
|
||||
)
|
||||
|
||||
// Quick mute: add mute region for an audible segment
|
||||
const handleQuickMute = (segment: AudibleSegment) => {
|
||||
onAddRegion({
|
||||
start_seconds: segment.start_seconds,
|
||||
end_seconds: segment.end_seconds,
|
||||
})
|
||||
}
|
||||
|
||||
// Check if a segment is already muted
|
||||
const isSegmentMuted = (segment: AudibleSegment) => {
|
||||
return muteRegions.some(
|
||||
(r) =>
|
||||
r.start_seconds <= segment.start_seconds &&
|
||||
r.end_seconds >= segment.end_seconds
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className={`space-y-4 ${className}`}>
|
||||
{/* Selection Mode Toggle */}
|
||||
<Card>
|
||||
<CardHeader className="pb-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<CardTitle className="text-lg">Mute Regions</CardTitle>
|
||||
<Button
|
||||
variant={isSelectionMode ? "default" : "outline"}
|
||||
size="sm"
|
||||
onClick={onToggleSelectionMode}
|
||||
>
|
||||
<Plus className="h-4 w-4 mr-2" />
|
||||
{isSelectionMode ? "Click waveform to select" : "Add Region"}
|
||||
</Button>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
{muteRegions.length === 0 ? (
|
||||
<p className="text-sm text-muted-foreground">
|
||||
No mute regions selected. Click "Add Region" and drag on the waveform to select areas to mute.
|
||||
</p>
|
||||
) : (
|
||||
<div className="space-y-2">
|
||||
{muteRegions.map((region, index) => (
|
||||
<div
|
||||
key={index}
|
||||
className="flex items-center justify-between p-2 bg-muted/50 rounded-lg group"
|
||||
>
|
||||
<div className="flex items-center gap-3">
|
||||
<VolumeX className="h-4 w-4 text-destructive" />
|
||||
<button
|
||||
onClick={() => onSeekTo?.(region.start_seconds)}
|
||||
className="text-sm font-mono hover:text-primary transition-colors"
|
||||
>
|
||||
{formatTime(region.start_seconds)}
|
||||
</button>
|
||||
<span className="text-muted-foreground">→</span>
|
||||
<button
|
||||
onClick={() => onSeekTo?.(region.end_seconds)}
|
||||
className="text-sm font-mono hover:text-primary transition-colors"
|
||||
>
|
||||
{formatTime(region.end_seconds)}
|
||||
</button>
|
||||
<Badge variant="secondary" className="ml-2">
|
||||
{formatDuration(region.end_seconds - region.start_seconds)}
|
||||
</Badge>
|
||||
</div>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={() => onRemoveRegion(index)}
|
||||
className="h-8 w-8 opacity-0 group-hover:opacity-100 transition-opacity"
|
||||
>
|
||||
<Trash2 className="h-4 w-4 text-destructive" />
|
||||
</Button>
|
||||
</div>
|
||||
))}
|
||||
|
||||
<div className="flex items-center justify-between pt-2 border-t mt-3">
|
||||
<span className="text-sm text-muted-foreground">
|
||||
Total muted: {formatDuration(totalMutedDuration)}
|
||||
</span>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={onClearAll}
|
||||
className="text-destructive hover:text-destructive"
|
||||
>
|
||||
Clear All
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Detected Audible Segments */}
|
||||
{audibleSegments.length > 0 && (
|
||||
<Card>
|
||||
<CardHeader className="pb-3">
|
||||
<CardTitle className="text-lg">Detected Backing Vocals</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<p className="text-sm text-muted-foreground mb-3">
|
||||
Click a segment to jump to that time, or click the mute button to add it as a mute region.
|
||||
</p>
|
||||
<div className="space-y-2 max-h-64 overflow-y-auto">
|
||||
{audibleSegments.map((segment, index) => {
|
||||
const isMuted = isSegmentMuted(segment)
|
||||
return (
|
||||
<div
|
||||
key={index}
|
||||
className={`flex items-center justify-between p-2 rounded-lg transition-colors ${
|
||||
isMuted ? "bg-destructive/10" : "bg-muted/50 hover:bg-muted"
|
||||
}`}
|
||||
>
|
||||
<button
|
||||
onClick={() => onSeekTo?.(segment.start_seconds)}
|
||||
className="flex items-center gap-3 text-left"
|
||||
>
|
||||
<span className="text-sm font-mono">
|
||||
{formatTime(segment.start_seconds)} - {formatTime(segment.end_seconds)}
|
||||
</span>
|
||||
<Badge variant="outline" className="text-xs">
|
||||
{formatDuration(segment.duration_seconds)}
|
||||
</Badge>
|
||||
<Badge
|
||||
variant="outline"
|
||||
className={`text-xs ${
|
||||
segment.avg_amplitude_db > -20
|
||||
? "border-yellow-500 text-yellow-500"
|
||||
: "border-muted-foreground"
|
||||
}`}
|
||||
>
|
||||
{segment.avg_amplitude_db.toFixed(1)}dB
|
||||
</Badge>
|
||||
</button>
|
||||
{!isMuted && (
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={() => handleQuickMute(segment)}
|
||||
className="text-xs"
|
||||
>
|
||||
<VolumeX className="h-3 w-3 mr-1" />
|
||||
Mute
|
||||
</Button>
|
||||
)}
|
||||
{isMuted && (
|
||||
<Badge variant="destructive" className="text-xs">
|
||||
Muted
|
||||
</Badge>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,259 @@
|
||||
"use client"
|
||||
|
||||
import { useRef, useEffect, useState, useCallback } from "react"
|
||||
|
||||
interface AudibleSegment {
|
||||
start_seconds: number
|
||||
end_seconds: number
|
||||
duration_seconds: number
|
||||
avg_amplitude_db: number
|
||||
}
|
||||
|
||||
interface MuteRegion {
|
||||
start_seconds: number
|
||||
end_seconds: number
|
||||
}
|
||||
|
||||
interface WaveformDisplayProps {
|
||||
amplitudes: number[]
|
||||
duration: number
|
||||
currentTime?: number
|
||||
segments?: AudibleSegment[]
|
||||
muteRegions?: MuteRegion[]
|
||||
onSeek?: (time: number) => void
|
||||
onRegionSelect?: (region: MuteRegion) => void
|
||||
isSelecting?: boolean
|
||||
width?: number
|
||||
height?: number
|
||||
className?: string
|
||||
}
|
||||
|
||||
export function WaveformDisplay({
|
||||
amplitudes,
|
||||
duration,
|
||||
currentTime = 0,
|
||||
segments = [],
|
||||
muteRegions = [],
|
||||
onSeek,
|
||||
onRegionSelect,
|
||||
isSelecting = false,
|
||||
width = 800,
|
||||
height = 150,
|
||||
className = "",
|
||||
}: WaveformDisplayProps) {
|
||||
const canvasRef = useRef<HTMLCanvasElement>(null)
|
||||
const [isDragging, setIsDragging] = useState(false)
|
||||
const [dragStart, setDragStart] = useState<number | null>(null)
|
||||
const [dragEnd, setDragEnd] = useState<number | null>(null)
|
||||
const [hoveredTime, setHoveredTime] = useState<number | null>(null)
|
||||
|
||||
// Convert time to x position
|
||||
const timeToX = useCallback((time: number) => {
|
||||
return (time / duration) * width
|
||||
}, [duration, width])
|
||||
|
||||
// Convert x position to time
|
||||
const xToTime = useCallback((x: number) => {
|
||||
return (x / width) * duration
|
||||
}, [duration, width])
|
||||
|
||||
// Draw the waveform
|
||||
useEffect(() => {
|
||||
const canvas = canvasRef.current
|
||||
if (!canvas) return
|
||||
|
||||
const ctx = canvas.getContext("2d")
|
||||
if (!ctx) return
|
||||
|
||||
// Clear canvas
|
||||
ctx.fillStyle = "#1a1a2e"
|
||||
ctx.fillRect(0, 0, width, height)
|
||||
|
||||
const barWidth = width / amplitudes.length
|
||||
const centerY = height / 2
|
||||
|
||||
// Draw mute regions background
|
||||
ctx.fillStyle = "rgba(255, 107, 107, 0.2)"
|
||||
for (const region of muteRegions) {
|
||||
const startX = timeToX(region.start_seconds)
|
||||
const endX = timeToX(region.end_seconds)
|
||||
ctx.fillRect(startX, 0, endX - startX, height)
|
||||
}
|
||||
|
||||
// Draw current drag selection
|
||||
if (isDragging && dragStart !== null && dragEnd !== null) {
|
||||
const startX = Math.min(timeToX(dragStart), timeToX(dragEnd))
|
||||
const endX = Math.max(timeToX(dragStart), timeToX(dragEnd))
|
||||
ctx.fillStyle = "rgba(255, 107, 107, 0.3)"
|
||||
ctx.fillRect(startX, 0, endX - startX, height)
|
||||
}
|
||||
|
||||
// Draw segment highlights
|
||||
for (const segment of segments) {
|
||||
const startX = timeToX(segment.start_seconds)
|
||||
const endX = timeToX(segment.end_seconds)
|
||||
const startIdx = Math.floor((segment.start_seconds / duration) * amplitudes.length)
|
||||
const endIdx = Math.ceil((segment.end_seconds / duration) * amplitudes.length)
|
||||
|
||||
ctx.fillStyle = "rgba(233, 69, 96, 0.4)"
|
||||
for (let i = startIdx; i < endIdx && i < amplitudes.length; i++) {
|
||||
const amplitude = amplitudes[i]
|
||||
const barHeight = amplitude * height * 0.8
|
||||
const x = i * barWidth
|
||||
ctx.fillRect(x, centerY - barHeight / 2, barWidth - 1, barHeight)
|
||||
}
|
||||
}
|
||||
|
||||
// Draw main waveform
|
||||
ctx.fillStyle = "#4a90d9"
|
||||
for (let i = 0; i < amplitudes.length; i++) {
|
||||
const amplitude = amplitudes[i]
|
||||
const barHeight = amplitude * height * 0.8
|
||||
const x = i * barWidth
|
||||
ctx.fillRect(x, centerY - barHeight / 2, barWidth - 1, barHeight)
|
||||
}
|
||||
|
||||
// Draw silence threshold line
|
||||
ctx.strokeStyle = "rgba(255, 255, 255, 0.2)"
|
||||
ctx.setLineDash([5, 5])
|
||||
ctx.beginPath()
|
||||
const thresholdY = height * 0.5 // Visual representation
|
||||
ctx.moveTo(0, thresholdY)
|
||||
ctx.lineTo(width, thresholdY)
|
||||
ctx.stroke()
|
||||
ctx.setLineDash([])
|
||||
|
||||
// Draw playhead
|
||||
if (currentTime > 0) {
|
||||
const playheadX = timeToX(currentTime)
|
||||
ctx.strokeStyle = "#ffffff"
|
||||
ctx.lineWidth = 2
|
||||
ctx.beginPath()
|
||||
ctx.moveTo(playheadX, 0)
|
||||
ctx.lineTo(playheadX, height)
|
||||
ctx.stroke()
|
||||
}
|
||||
|
||||
// Draw hover position
|
||||
if (hoveredTime !== null && !isDragging) {
|
||||
const hoverX = timeToX(hoveredTime)
|
||||
ctx.strokeStyle = "rgba(255, 255, 255, 0.5)"
|
||||
ctx.lineWidth = 1
|
||||
ctx.beginPath()
|
||||
ctx.moveTo(hoverX, 0)
|
||||
ctx.lineTo(hoverX, height)
|
||||
ctx.stroke()
|
||||
}
|
||||
|
||||
// Draw time axis
|
||||
ctx.fillStyle = "#ffffff"
|
||||
ctx.font = "10px sans-serif"
|
||||
ctx.textAlign = "center"
|
||||
|
||||
const tickInterval = duration > 300 ? 60 : duration > 60 ? 30 : 10
|
||||
for (let t = 0; t <= duration; t += tickInterval) {
|
||||
const x = timeToX(t)
|
||||
const minutes = Math.floor(t / 60)
|
||||
const seconds = Math.floor(t % 60)
|
||||
const label = `${minutes}:${seconds.toString().padStart(2, "0")}`
|
||||
ctx.fillText(label, x, height - 5)
|
||||
|
||||
// Tick mark
|
||||
ctx.strokeStyle = "rgba(255, 255, 255, 0.3)"
|
||||
ctx.beginPath()
|
||||
ctx.moveTo(x, height - 20)
|
||||
ctx.lineTo(x, height - 15)
|
||||
ctx.stroke()
|
||||
}
|
||||
|
||||
}, [amplitudes, duration, currentTime, segments, muteRegions, width, height, isDragging, dragStart, dragEnd, hoveredTime, timeToX])
|
||||
|
||||
// Handle mouse events for seeking and region selection
|
||||
const handleMouseDown = (e: React.MouseEvent<HTMLCanvasElement>) => {
|
||||
const canvas = canvasRef.current
|
||||
if (!canvas) return
|
||||
|
||||
const rect = canvas.getBoundingClientRect()
|
||||
const x = e.clientX - rect.left
|
||||
const time = xToTime(x)
|
||||
|
||||
if (isSelecting) {
|
||||
setIsDragging(true)
|
||||
setDragStart(time)
|
||||
setDragEnd(time)
|
||||
} else if (onSeek) {
|
||||
onSeek(time)
|
||||
}
|
||||
}
|
||||
|
||||
const handleMouseMove = (e: React.MouseEvent<HTMLCanvasElement>) => {
|
||||
const canvas = canvasRef.current
|
||||
if (!canvas) return
|
||||
|
||||
const rect = canvas.getBoundingClientRect()
|
||||
const x = e.clientX - rect.left
|
||||
const time = xToTime(x)
|
||||
|
||||
setHoveredTime(time)
|
||||
|
||||
if (isDragging && isSelecting) {
|
||||
setDragEnd(time)
|
||||
}
|
||||
}
|
||||
|
||||
const handleMouseUp = () => {
|
||||
if (isDragging && dragStart !== null && dragEnd !== null && onRegionSelect) {
|
||||
const start = Math.min(dragStart, dragEnd)
|
||||
const end = Math.max(dragStart, dragEnd)
|
||||
|
||||
// Only create region if it's at least 0.1 seconds
|
||||
if (end - start >= 0.1) {
|
||||
onRegionSelect({ start_seconds: start, end_seconds: end })
|
||||
}
|
||||
}
|
||||
|
||||
setIsDragging(false)
|
||||
setDragStart(null)
|
||||
setDragEnd(null)
|
||||
}
|
||||
|
||||
const handleMouseLeave = () => {
|
||||
setHoveredTime(null)
|
||||
if (isDragging) {
|
||||
handleMouseUp()
|
||||
}
|
||||
}
|
||||
|
||||
// Format time for tooltip
|
||||
const formatTime = (time: number) => {
|
||||
const minutes = Math.floor(time / 60)
|
||||
const seconds = Math.floor(time % 60)
|
||||
const ms = Math.floor((time % 1) * 100)
|
||||
return `${minutes}:${seconds.toString().padStart(2, "0")}.${ms.toString().padStart(2, "0")}`
|
||||
}
|
||||
|
||||
return (
|
||||
<div className={`relative ${className}`}>
|
||||
<canvas
|
||||
ref={canvasRef}
|
||||
width={width}
|
||||
height={height}
|
||||
className={`rounded-lg ${isSelecting ? "cursor-crosshair" : "cursor-pointer"}`}
|
||||
onMouseDown={handleMouseDown}
|
||||
onMouseMove={handleMouseMove}
|
||||
onMouseUp={handleMouseUp}
|
||||
onMouseLeave={handleMouseLeave}
|
||||
/>
|
||||
|
||||
{/* Time tooltip */}
|
||||
{hoveredTime !== null && (
|
||||
<div
|
||||
className="absolute top-0 transform -translate-x-1/2 bg-black/80 text-white text-xs px-2 py-1 rounded pointer-events-none"
|
||||
style={{ left: timeToX(hoveredTime) }}
|
||||
>
|
||||
{formatTime(hoveredTime)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -94,6 +94,14 @@ echo -n "your-genius-key" | \
|
||||
# Add Audio Separator API URL
|
||||
echo -n "https://your-modal-url" | \
|
||||
gcloud secrets versions add audio-separator-api-url --data-file=-
|
||||
|
||||
# Add Redacted API key (for flacfetch audio search - private tracker)
|
||||
echo -n "your-redacted-api-key" | \
|
||||
gcloud secrets versions add redacted-api-key --data-file=-
|
||||
|
||||
# Add OPS API key (for flacfetch audio search - private tracker)
|
||||
echo -n "your-ops-api-key" | \
|
||||
gcloud secrets versions add ops-api-key --data-file=-
|
||||
```
|
||||
|
||||
Or use Pulumi config secrets:
|
||||
|
||||
@@ -2,10 +2,20 @@
|
||||
Pulumi infrastructure for Karaoke Generator backend on Google Cloud Platform.
|
||||
|
||||
This replaces the manual setup scripts with proper Infrastructure as Code.
|
||||
|
||||
Infrastructure includes:
|
||||
- Firestore database for job state
|
||||
- Cloud Storage bucket for files
|
||||
- Artifact Registry for Docker images
|
||||
- Service accounts and IAM
|
||||
- GitHub Actions Workload Identity Federation
|
||||
- Cloud Tasks queues for worker coordination (Phase 1 scalability)
|
||||
- Secret Manager secrets
|
||||
- Cloud Run domain mapping
|
||||
"""
|
||||
import pulumi
|
||||
import pulumi_gcp as gcp
|
||||
from pulumi_gcp import storage, firestore, secretmanager, artifactregistry, cloudrun, serviceaccount
|
||||
from pulumi_gcp import storage, firestore, secretmanager, artifactregistry, cloudrun, serviceaccount, cloudtasks
|
||||
|
||||
# Get the current GCP project
|
||||
project = gcp.organizations.get_project()
|
||||
@@ -227,6 +237,167 @@ github_actions_wif_binding = gcp.serviceaccount.IAMBinding(
|
||||
)],
|
||||
)
|
||||
|
||||
# ==================== Cloud Tasks Queues (Phase 1 Scalability) ====================
|
||||
# These queues provide guaranteed delivery and horizontal scaling for worker tasks.
|
||||
# Workers are triggered via Cloud Tasks instead of BackgroundTasks, ensuring:
|
||||
# - Tasks survive container restarts
|
||||
# - Each task gets dedicated resources
|
||||
# - Automatic retries on failure
|
||||
# - Rate limiting to protect external APIs
|
||||
|
||||
# Audio worker queue - calls Modal API for audio separation
|
||||
audio_worker_queue = cloudtasks.Queue(
|
||||
"audio-worker-queue",
|
||||
name="audio-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=10, # Protect Modal API
|
||||
max_concurrent_dispatches=50, # Max parallel audio workers
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=3,
|
||||
min_backoff="10s",
|
||||
max_backoff="300s",
|
||||
max_retry_duration="1800s", # 30 min total retry window
|
||||
),
|
||||
)
|
||||
|
||||
# Lyrics worker queue - calls AudioShake API for transcription
|
||||
lyrics_worker_queue = cloudtasks.Queue(
|
||||
"lyrics-worker-queue",
|
||||
name="lyrics-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=10,
|
||||
max_concurrent_dispatches=50,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=3,
|
||||
min_backoff="10s",
|
||||
max_backoff="300s",
|
||||
max_retry_duration="1200s", # 20 min total retry window
|
||||
),
|
||||
)
|
||||
|
||||
# Screens worker queue - generates title/end screens (fast, CPU-light)
|
||||
screens_worker_queue = cloudtasks.Queue(
|
||||
"screens-worker-queue",
|
||||
name="screens-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=50, # Fast operations
|
||||
max_concurrent_dispatches=100,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=3,
|
||||
min_backoff="5s",
|
||||
max_backoff="60s",
|
||||
max_retry_duration="300s", # 5 min total retry window
|
||||
),
|
||||
)
|
||||
|
||||
# Render worker queue - LyricsTranscriber + FFmpeg (CPU-intensive)
|
||||
render_worker_queue = cloudtasks.Queue(
|
||||
"render-worker-queue",
|
||||
name="render-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=5, # CPU-intensive
|
||||
max_concurrent_dispatches=20,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=2,
|
||||
min_backoff="30s",
|
||||
max_backoff="300s",
|
||||
max_retry_duration="3600s", # 60 min for render
|
||||
),
|
||||
)
|
||||
|
||||
# Video worker queue - final encoding (very CPU-intensive, longest running)
|
||||
# Note: For tasks >30 min, consider Cloud Run Jobs instead
|
||||
video_worker_queue = cloudtasks.Queue(
|
||||
"video-worker-queue",
|
||||
name="video-worker-queue",
|
||||
location="us-central1",
|
||||
rate_limits=cloudtasks.QueueRateLimitsArgs(
|
||||
max_dispatches_per_second=3, # Very CPU-intensive
|
||||
max_concurrent_dispatches=10,
|
||||
),
|
||||
retry_config=cloudtasks.QueueRetryConfigArgs(
|
||||
max_attempts=2,
|
||||
min_backoff="60s",
|
||||
max_backoff="600s",
|
||||
max_retry_duration="7200s", # 2 hour total (video is long)
|
||||
),
|
||||
)
|
||||
|
||||
# Grant Cloud Tasks permission to invoke Cloud Run service
|
||||
# Cloud Tasks service agent needs to authenticate to Cloud Run
|
||||
cloud_tasks_invoker = gcp.projects.IAMMember(
|
||||
"cloud-tasks-invoker",
|
||||
project=project_id,
|
||||
role="roles/run.invoker",
|
||||
member=f"serviceAccount:service-{project.number}@gcp-sa-cloudtasks.iam.gserviceaccount.com",
|
||||
)
|
||||
|
||||
# Grant backend service account permission to enqueue Cloud Tasks
|
||||
cloud_tasks_enqueuer = gcp.projects.IAMMember(
|
||||
"karaoke-backend-cloudtasks-enqueuer",
|
||||
project=project_id,
|
||||
role="roles/cloudtasks.enqueuer",
|
||||
member=service_account.email.apply(lambda email: f"serviceAccount:{email}"),
|
||||
)
|
||||
|
||||
# ==================== Cloud Run Job for Video Encoding (Phase 2 Scalability) ====================
|
||||
# Cloud Run Jobs provide up to 24-hour execution time for long-running video encoding.
|
||||
# This is used when video encoding might exceed the 30-minute Cloud Tasks timeout.
|
||||
# Triggered programmatically via WorkerService when ENABLE_CLOUD_TASKS=true and
|
||||
# USE_CLOUD_RUN_JOBS_FOR_VIDEO=true
|
||||
|
||||
video_encoding_job = gcp.cloudrunv2.Job(
|
||||
"video-encoding-job",
|
||||
name="video-encoding-job",
|
||||
location="us-central1",
|
||||
template=gcp.cloudrunv2.JobTemplateArgs(
|
||||
template=gcp.cloudrunv2.JobTemplateTemplateArgs(
|
||||
# Use the same container as the backend service
|
||||
# The job runs with --job-id argument to process a specific job
|
||||
containers=[gcp.cloudrunv2.JobTemplateTemplateContainerArgs(
|
||||
image=f"us-central1-docker.pkg.dev/{project_id}/karaoke-repo/karaoke-backend:latest",
|
||||
args=["python", "-m", "backend.workers.video_worker"],
|
||||
resources=gcp.cloudrunv2.JobTemplateTemplateContainerResourcesArgs(
|
||||
limits={
|
||||
"cpu": "4", # 4 vCPUs for encoding
|
||||
"memory": "8Gi", # 8GB RAM
|
||||
},
|
||||
),
|
||||
envs=[
|
||||
gcp.cloudrunv2.JobTemplateTemplateContainerEnvArgs(
|
||||
name="GOOGLE_CLOUD_PROJECT",
|
||||
value=project_id,
|
||||
),
|
||||
gcp.cloudrunv2.JobTemplateTemplateContainerEnvArgs(
|
||||
name="GCS_BUCKET_NAME",
|
||||
value=bucket.name,
|
||||
),
|
||||
],
|
||||
)],
|
||||
service_account=service_account.email,
|
||||
timeout="3600s", # 1 hour max per task
|
||||
max_retries=2,
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
# Grant backend service account permission to run Cloud Run Jobs
|
||||
cloud_run_jobs_developer = gcp.projects.IAMMember(
|
||||
"karaoke-backend-run-jobs-developer",
|
||||
project=project_id,
|
||||
role="roles/run.developer",
|
||||
member=service_account.email.apply(lambda email: f"serviceAccount:{email}"),
|
||||
)
|
||||
|
||||
# ==================== Secrets ====================
|
||||
# Create secrets (you'll need to add the actual secret values manually or via pulumi config)
|
||||
config = pulumi.Config()
|
||||
|
||||
@@ -257,6 +428,24 @@ audio_separator_secret = secretmanager.Secret(
|
||||
),
|
||||
)
|
||||
|
||||
# Redacted API Key Secret (for flacfetch audio search)
|
||||
redacted_api_key_secret = secretmanager.Secret(
|
||||
"redacted-api-key",
|
||||
secret_id="redacted-api-key",
|
||||
replication=secretmanager.SecretReplicationArgs(
|
||||
auto=secretmanager.SecretReplicationAutoArgs(),
|
||||
),
|
||||
)
|
||||
|
||||
# OPS API Key Secret (for flacfetch audio search)
|
||||
ops_api_key_secret = secretmanager.Secret(
|
||||
"ops-api-key",
|
||||
secret_id="ops-api-key",
|
||||
replication=secretmanager.SecretReplicationArgs(
|
||||
auto=secretmanager.SecretReplicationAutoArgs(),
|
||||
),
|
||||
)
|
||||
|
||||
# Create Cloud Run Domain Mapping
|
||||
# Maps api.nomadkaraoke.com to the karaoke-backend service
|
||||
domain_mapping = cloudrun.DomainMapping(
|
||||
@@ -286,6 +475,16 @@ pulumi.export("backend_default_url", "https://karaoke-backend-ipzqd2k4yq-uc.a.ru
|
||||
pulumi.export("github_actions_service_account", github_actions_sa.email)
|
||||
pulumi.export("workload_identity_provider", workload_identity_provider.name)
|
||||
|
||||
# Cloud Tasks queue exports (Phase 1 scalability)
|
||||
pulumi.export("audio_worker_queue", audio_worker_queue.name)
|
||||
pulumi.export("lyrics_worker_queue", lyrics_worker_queue.name)
|
||||
pulumi.export("screens_worker_queue", screens_worker_queue.name)
|
||||
pulumi.export("render_worker_queue", render_worker_queue.name)
|
||||
pulumi.export("video_worker_queue", video_worker_queue.name)
|
||||
|
||||
# Cloud Run Job export (Phase 2 scalability)
|
||||
pulumi.export("video_encoding_job", video_encoding_job.name)
|
||||
|
||||
# Export DNS configuration needed for Cloudflare
|
||||
# These are the records to add to your Cloudflare DNS
|
||||
domain_mapping.statuses.apply(
|
||||
@@ -298,3 +497,244 @@ domain_mapping.statuses.apply(
|
||||
})
|
||||
)
|
||||
|
||||
# ==================== Flacfetch Service (Torrent Downloads) ====================
|
||||
# A dedicated VM running flacfetch with Transmission daemon for BitTorrent downloads.
|
||||
# This is required because Cloud Run doesn't support inbound peer connections for torrents.
|
||||
# The VM has a static IP for tracker whitelist and runs indefinitely for seeding.
|
||||
|
||||
# Static IP for flacfetch (for tracker account whitelist)
|
||||
flacfetch_ip = gcp.compute.Address(
|
||||
"flacfetch-ip",
|
||||
name="flacfetch-static-ip",
|
||||
region="us-central1",
|
||||
address_type="EXTERNAL",
|
||||
description="Static IP for flacfetch service (torrent downloads)",
|
||||
)
|
||||
|
||||
# Service account for flacfetch VM
|
||||
flacfetch_sa = serviceaccount.Account(
|
||||
"flacfetch-sa",
|
||||
account_id="flacfetch-service",
|
||||
display_name="Flacfetch Service Account",
|
||||
description="Service account for flacfetch torrent/audio download VM",
|
||||
)
|
||||
|
||||
# Grant flacfetch service account GCS write permissions
|
||||
flacfetch_storage_iam = gcp.storage.BucketIAMMember(
|
||||
"flacfetch-storage-writer",
|
||||
bucket=bucket.name,
|
||||
role="roles/storage.objectCreator",
|
||||
member=flacfetch_sa.email.apply(lambda email: f"serviceAccount:{email}"),
|
||||
)
|
||||
|
||||
# Grant flacfetch service account GCS read permissions (for uploads folder)
|
||||
flacfetch_storage_reader_iam = gcp.storage.BucketIAMMember(
|
||||
"flacfetch-storage-reader",
|
||||
bucket=bucket.name,
|
||||
role="roles/storage.objectViewer",
|
||||
member=flacfetch_sa.email.apply(lambda email: f"serviceAccount:{email}"),
|
||||
)
|
||||
|
||||
# Grant flacfetch service account Secret Manager access
|
||||
flacfetch_secrets_iam = gcp.projects.IAMMember(
|
||||
"flacfetch-secrets-access",
|
||||
project=project_id,
|
||||
role="roles/secretmanager.secretAccessor",
|
||||
member=flacfetch_sa.email.apply(lambda email: f"serviceAccount:{email}"),
|
||||
)
|
||||
|
||||
# Flacfetch API key secret (for authenticating API requests)
|
||||
flacfetch_api_key_secret = secretmanager.Secret(
|
||||
"flacfetch-api-key",
|
||||
secret_id="flacfetch-api-key",
|
||||
replication=secretmanager.SecretReplicationArgs(
|
||||
auto=secretmanager.SecretReplicationAutoArgs(),
|
||||
),
|
||||
)
|
||||
|
||||
# Flacfetch API URL secret (the static IP address of the flacfetch VM)
|
||||
# This is stored as a secret to avoid committing infrastructure IPs to the repo
|
||||
flacfetch_api_url_secret = secretmanager.Secret(
|
||||
"flacfetch-api-url",
|
||||
secret_id="flacfetch-api-url",
|
||||
replication=secretmanager.SecretReplicationArgs(
|
||||
auto=secretmanager.SecretReplicationAutoArgs(),
|
||||
),
|
||||
)
|
||||
|
||||
# Startup script for the flacfetch VM
|
||||
# This installs and configures Transmission daemon and the flacfetch service
|
||||
FLACFETCH_STARTUP_SCRIPT = """#!/bin/bash
|
||||
set -e
|
||||
|
||||
# Log to a file for debugging
|
||||
exec > >(tee /var/log/flacfetch-startup.log) 2>&1
|
||||
echo "Starting flacfetch setup at $(date)"
|
||||
|
||||
# Install dependencies
|
||||
apt-get update
|
||||
apt-get install -y python3-pip python3-venv transmission-daemon ffmpeg git curl
|
||||
|
||||
# Configure Transmission
|
||||
echo "Configuring Transmission daemon..."
|
||||
systemctl stop transmission-daemon || true
|
||||
|
||||
# Create transmission directories
|
||||
mkdir -p /var/lib/transmission-daemon/downloads
|
||||
mkdir -p /var/lib/transmission-daemon/.config/transmission-daemon
|
||||
chown -R debian-transmission:debian-transmission /var/lib/transmission-daemon
|
||||
|
||||
# Configure transmission settings
|
||||
cat > /var/lib/transmission-daemon/.config/transmission-daemon/settings.json << 'SETTINGS'
|
||||
{
|
||||
"download-dir": "/var/lib/transmission-daemon/downloads",
|
||||
"incomplete-dir": "/var/lib/transmission-daemon/.incomplete",
|
||||
"incomplete-dir-enabled": true,
|
||||
"rpc-authentication-required": false,
|
||||
"rpc-bind-address": "127.0.0.1",
|
||||
"rpc-enabled": true,
|
||||
"rpc-port": 9091,
|
||||
"rpc-whitelist-enabled": false,
|
||||
"peer-port": 51413,
|
||||
"port-forwarding-enabled": false,
|
||||
"speed-limit-down": 0,
|
||||
"speed-limit-down-enabled": false,
|
||||
"speed-limit-up": 0,
|
||||
"speed-limit-up-enabled": false,
|
||||
"ratio-limit-enabled": false,
|
||||
"umask": 2,
|
||||
"encryption": 1
|
||||
}
|
||||
SETTINGS
|
||||
|
||||
chown debian-transmission:debian-transmission /var/lib/transmission-daemon/.config/transmission-daemon/settings.json
|
||||
systemctl start transmission-daemon
|
||||
echo "Transmission daemon started"
|
||||
|
||||
# Install flacfetch
|
||||
echo "Installing flacfetch..."
|
||||
cd /opt
|
||||
|
||||
# Clone flacfetch (we'll use the repo from GitHub)
|
||||
if [ ! -d "flacfetch" ]; then
|
||||
git clone https://github.com/nomadkaraoke/flacfetch.git
|
||||
fi
|
||||
cd flacfetch
|
||||
|
||||
# Create and activate virtual environment
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install flacfetch with API dependencies
|
||||
pip install -e ".[api]"
|
||||
|
||||
# Get secrets from Secret Manager
|
||||
echo "Fetching secrets from Secret Manager..."
|
||||
FLACFETCH_API_KEY=$(gcloud secrets versions access latest --secret=flacfetch-api-key 2>/dev/null || echo "")
|
||||
REDACTED_API_KEY=$(gcloud secrets versions access latest --secret=redacted-api-key 2>/dev/null || echo "")
|
||||
OPS_API_KEY=$(gcloud secrets versions access latest --secret=ops-api-key 2>/dev/null || echo "")
|
||||
|
||||
# Get bucket name from project metadata
|
||||
GCS_BUCKET=$(gcloud compute project-info describe --format='value(commonInstanceMetadata.items.gcs-bucket)' 2>/dev/null || echo "")
|
||||
if [ -z "$GCS_BUCKET" ]; then
|
||||
# Fallback: construct from project ID
|
||||
PROJECT_ID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google")
|
||||
GCS_BUCKET="karaoke-gen-storage-${PROJECT_ID}"
|
||||
fi
|
||||
|
||||
echo "Using GCS bucket: $GCS_BUCKET"
|
||||
|
||||
# Create systemd service for flacfetch
|
||||
echo "Creating systemd service..."
|
||||
cat > /etc/systemd/system/flacfetch.service << SYSTEMD
|
||||
[Unit]
|
||||
Description=Flacfetch HTTP API Service
|
||||
After=network.target transmission-daemon.service
|
||||
Requires=transmission-daemon.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=root
|
||||
WorkingDirectory=/opt/flacfetch
|
||||
Environment="FLACFETCH_API_KEY=${FLACFETCH_API_KEY}"
|
||||
Environment="REDACTED_API_KEY=${REDACTED_API_KEY}"
|
||||
Environment="OPS_API_KEY=${OPS_API_KEY}"
|
||||
Environment="GCS_BUCKET=${GCS_BUCKET}"
|
||||
Environment="FLACFETCH_KEEP_SEEDING=true"
|
||||
Environment="FLACFETCH_MIN_FREE_GB=5"
|
||||
Environment="FLACFETCH_DOWNLOAD_DIR=/var/lib/transmission-daemon/downloads"
|
||||
Environment="TRANSMISSION_HOST=localhost"
|
||||
Environment="TRANSMISSION_PORT=9091"
|
||||
ExecStart=/opt/flacfetch/venv/bin/flacfetch serve --host 0.0.0.0 --port 8080
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
SYSTEMD
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl enable flacfetch
|
||||
systemctl start flacfetch
|
||||
|
||||
echo "Flacfetch service started at $(date)"
|
||||
echo "Setup complete!"
|
||||
"""
|
||||
|
||||
# Flacfetch VM instance
|
||||
flacfetch_vm = gcp.compute.Instance(
|
||||
"flacfetch-service",
|
||||
name="flacfetch-service",
|
||||
machine_type="e2-small", # 0.5 vCPU, 2GB RAM - sufficient for I/O-bound workload
|
||||
zone="us-central1-a",
|
||||
boot_disk=gcp.compute.InstanceBootDiskArgs(
|
||||
initialize_params=gcp.compute.InstanceBootDiskInitializeParamsArgs(
|
||||
image="debian-cloud/debian-12",
|
||||
size=30, # GB - for torrent storage + headroom
|
||||
type="pd-ssd", # SSD for faster I/O
|
||||
),
|
||||
),
|
||||
network_interfaces=[gcp.compute.InstanceNetworkInterfaceArgs(
|
||||
network="default",
|
||||
access_configs=[gcp.compute.InstanceNetworkInterfaceAccessConfigArgs(
|
||||
nat_ip=flacfetch_ip.address, # Static IP
|
||||
)],
|
||||
)],
|
||||
service_account=gcp.compute.InstanceServiceAccountArgs(
|
||||
email=flacfetch_sa.email,
|
||||
scopes=["cloud-platform"],
|
||||
),
|
||||
metadata_startup_script=FLACFETCH_STARTUP_SCRIPT,
|
||||
metadata={
|
||||
"gcs-bucket": bucket.name, # Pass bucket name to startup script
|
||||
},
|
||||
tags=["flacfetch-service"],
|
||||
allow_stopping_for_update=True,
|
||||
)
|
||||
|
||||
# Firewall rule for flacfetch
|
||||
# - Port 51413 TCP/UDP: BitTorrent peer connections
|
||||
# - Port 8080 TCP: Flacfetch HTTP API (auth required at app level)
|
||||
flacfetch_firewall = gcp.compute.Firewall(
|
||||
"flacfetch-firewall",
|
||||
name="flacfetch-firewall",
|
||||
network="default",
|
||||
allows=[
|
||||
# BitTorrent peer connections
|
||||
gcp.compute.FirewallAllowArgs(protocol="tcp", ports=["51413"]),
|
||||
gcp.compute.FirewallAllowArgs(protocol="udp", ports=["51413"]),
|
||||
# Flacfetch HTTP API (auth required at app level)
|
||||
gcp.compute.FirewallAllowArgs(protocol="tcp", ports=["8080"]),
|
||||
],
|
||||
source_ranges=["0.0.0.0/0"], # Permissive, auth enforced at app level
|
||||
target_tags=["flacfetch-service"],
|
||||
description="Firewall rules for flacfetch torrent/API service",
|
||||
)
|
||||
|
||||
# Export flacfetch values
|
||||
pulumi.export("flacfetch_static_ip", flacfetch_ip.address)
|
||||
pulumi.export("flacfetch_service_url", flacfetch_ip.address.apply(
|
||||
lambda ip: f"http://{ip}:8080"
|
||||
))
|
||||
pulumi.export("flacfetch_service_account", flacfetch_sa.email)
|
||||
|
||||
@@ -4,4 +4,35 @@ import warnings
|
||||
warnings.filterwarnings("ignore", category=SyntaxWarning, module="pydub.*")
|
||||
warnings.filterwarnings("ignore", category=SyntaxWarning, module="syrics.*")
|
||||
|
||||
from .karaoke_gen import KaraokePrep
|
||||
# Lazy imports to avoid loading heavy dependencies when not needed.
|
||||
# KaraokePrep has many dependencies (lyrics_transcriber, etc.) which may not
|
||||
# be available in all contexts (e.g., backend which only needs audio_fetcher).
|
||||
#
|
||||
# Use explicit imports instead:
|
||||
# from karaoke_gen.karaoke_gen import KaraokePrep
|
||||
# from karaoke_gen.audio_fetcher import FlacFetcher, AudioSearchResult, ...
|
||||
|
||||
__all__ = [
|
||||
"KaraokePrep",
|
||||
# Audio fetcher
|
||||
"FlacFetcher",
|
||||
"AudioSearchResult",
|
||||
"AudioFetchResult",
|
||||
"AudioFetcherError",
|
||||
"NoResultsError",
|
||||
"DownloadError",
|
||||
"UserCancelledError",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name):
|
||||
"""Lazy import for heavy modules."""
|
||||
if name == "KaraokePrep":
|
||||
from .karaoke_gen import KaraokePrep
|
||||
return KaraokePrep
|
||||
elif name in ("FlacFetcher", "AudioSearchResult", "AudioFetchResult",
|
||||
"AudioFetcherError", "NoResultsError", "DownloadError",
|
||||
"UserCancelledError"):
|
||||
from . import audio_fetcher
|
||||
return getattr(audio_fetcher, name)
|
||||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||||
@@ -7,15 +7,29 @@ using flacfetch, replacing the previous direct yt-dlp usage.
|
||||
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass
|
||||
from typing import List, Optional
|
||||
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
|
||||
from dataclasses import dataclass, asdict, field
|
||||
from typing import List, Optional, Dict, Any
|
||||
|
||||
# Global flag to track if user requested cancellation via Ctrl+C
|
||||
_interrupt_requested = False
|
||||
|
||||
|
||||
@dataclass
|
||||
class AudioSearchResult:
|
||||
"""Represents a single search result for audio."""
|
||||
"""Represents a single search result for audio.
|
||||
|
||||
Used by both local CLI and cloud backend. Supports serialization
|
||||
for Firestore storage via to_dict()/from_dict().
|
||||
|
||||
For rich display, this class can serialize the full flacfetch Release
|
||||
data so remote CLIs can use flacfetch's shared display functions.
|
||||
"""
|
||||
|
||||
title: str
|
||||
artist: str
|
||||
@@ -24,13 +38,75 @@ class AudioSearchResult:
|
||||
duration: Optional[int] = None # Duration in seconds
|
||||
quality: Optional[str] = None # e.g., "FLAC", "320kbps", etc.
|
||||
source_id: Optional[str] = None # Unique ID from the source
|
||||
# Raw result object from the provider (for download)
|
||||
raw_result: Optional[object] = None
|
||||
index: int = 0 # Index in the results list (for API selection)
|
||||
seeders: Optional[int] = None # Number of seeders (for torrent sources)
|
||||
target_file: Optional[str] = None # Target filename in the release
|
||||
# Raw result object from the provider (for download) - not serialized
|
||||
raw_result: Optional[object] = field(default=None, repr=False)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert to dict for JSON/Firestore serialization.
|
||||
|
||||
Includes full flacfetch Release data if available, enabling
|
||||
remote CLIs to use flacfetch's shared display functions.
|
||||
"""
|
||||
result = {
|
||||
"title": self.title,
|
||||
"artist": self.artist,
|
||||
"url": self.url,
|
||||
"provider": self.provider,
|
||||
"duration": self.duration,
|
||||
"quality": self.quality,
|
||||
"source_id": self.source_id,
|
||||
"index": self.index,
|
||||
"seeders": self.seeders,
|
||||
"target_file": self.target_file,
|
||||
}
|
||||
|
||||
# If we have a raw_result (flacfetch Release), include its full data
|
||||
# This enables rich display on the remote CLI
|
||||
if self.raw_result:
|
||||
try:
|
||||
release_dict = self.raw_result.to_dict()
|
||||
# Merge Release fields into result (they may override basic fields)
|
||||
for key in ['year', 'label', 'edition_info', 'release_type', 'channel',
|
||||
'view_count', 'size_bytes', 'target_file_size', 'track_pattern',
|
||||
'match_score', 'formatted_size', 'formatted_duration',
|
||||
'formatted_views', 'is_lossless', 'quality_str', 'quality']:
|
||||
if key in release_dict:
|
||||
# Use 'quality_data' for the quality dict to avoid confusion with quality string
|
||||
result_key = 'quality_data' if key == 'quality' else key
|
||||
result[result_key] = release_dict[key]
|
||||
except AttributeError:
|
||||
pass # raw_result doesn't have to_dict() method
|
||||
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> "AudioSearchResult":
|
||||
"""Create from dict (e.g., from Firestore)."""
|
||||
return cls(
|
||||
title=data.get("title", ""),
|
||||
artist=data.get("artist", ""),
|
||||
url=data.get("url", ""),
|
||||
provider=data.get("provider", "Unknown"),
|
||||
duration=data.get("duration"),
|
||||
quality=data.get("quality"),
|
||||
source_id=data.get("source_id"),
|
||||
index=data.get("index", 0),
|
||||
seeders=data.get("seeders"),
|
||||
target_file=data.get("target_file"),
|
||||
raw_result=None, # Not stored in serialized form
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class AudioFetchResult:
|
||||
"""Result of an audio fetch operation."""
|
||||
"""Result of an audio fetch operation.
|
||||
|
||||
Used by both local CLI and cloud backend. Supports serialization
|
||||
for Firestore storage via to_dict()/from_dict().
|
||||
"""
|
||||
|
||||
filepath: str
|
||||
artist: str
|
||||
@@ -38,6 +114,22 @@ class AudioFetchResult:
|
||||
provider: str
|
||||
duration: Optional[int] = None
|
||||
quality: Optional[str] = None
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert to dict for JSON/Firestore serialization."""
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> "AudioFetchResult":
|
||||
"""Create from dict (e.g., from Firestore)."""
|
||||
return cls(
|
||||
filepath=data.get("filepath", ""),
|
||||
artist=data.get("artist", ""),
|
||||
title=data.get("title", ""),
|
||||
provider=data.get("provider", "Unknown"),
|
||||
duration=data.get("duration"),
|
||||
quality=data.get("quality"),
|
||||
)
|
||||
|
||||
|
||||
class AudioFetcherError(Exception):
|
||||
@@ -58,6 +150,19 @@ class DownloadError(AudioFetcherError):
|
||||
pass
|
||||
|
||||
|
||||
class UserCancelledError(AudioFetcherError):
|
||||
"""Raised when user explicitly cancels the operation (e.g., enters 0 or Ctrl+C)."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def _check_interrupt():
|
||||
"""Check if interrupt was requested and raise UserCancelledError if so."""
|
||||
global _interrupt_requested
|
||||
if _interrupt_requested:
|
||||
raise UserCancelledError("Operation cancelled by user")
|
||||
|
||||
|
||||
class AudioFetcher(ABC):
|
||||
"""Abstract base class for audio fetching implementations."""
|
||||
|
||||
@@ -140,6 +245,8 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
|
||||
This provides access to multiple audio sources including private music trackers
|
||||
and YouTube, with intelligent prioritization of high-quality sources.
|
||||
|
||||
Also exported as FlacFetcher for shorter name.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
@@ -163,31 +270,101 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
self._ops_api_key = ops_api_key or os.environ.get("OPS_API_KEY")
|
||||
self._provider_priority = provider_priority
|
||||
self._manager = None
|
||||
self._transmission_available = None # Cached result of Transmission check
|
||||
|
||||
def _check_transmission_available(self) -> bool:
|
||||
"""
|
||||
Check if Transmission daemon is available for torrent downloads.
|
||||
|
||||
This prevents adding tracker providers (Redacted/OPS) when Transmission
|
||||
isn't running, which would result in search results that can't be downloaded.
|
||||
|
||||
Returns:
|
||||
True if Transmission is available and responsive, False otherwise.
|
||||
"""
|
||||
if self._transmission_available is not None:
|
||||
self.logger.info(f"[Transmission] Using cached status: available={self._transmission_available}")
|
||||
return self._transmission_available
|
||||
|
||||
host = os.environ.get("TRANSMISSION_HOST", "localhost")
|
||||
port = int(os.environ.get("TRANSMISSION_PORT", "9091"))
|
||||
self.logger.info(f"[Transmission] Checking availability at {host}:{port}")
|
||||
|
||||
try:
|
||||
import transmission_rpc
|
||||
self.logger.info(f"[Transmission] transmission_rpc imported successfully")
|
||||
|
||||
client = transmission_rpc.Client(host=host, port=port, timeout=5)
|
||||
self.logger.info(f"[Transmission] Client created, calling session_stats()...")
|
||||
|
||||
# Simple test to verify connection works
|
||||
stats = client.session_stats()
|
||||
self.logger.info(f"[Transmission] Connected! Download dir: {getattr(stats, 'download_dir', 'unknown')}")
|
||||
|
||||
self._transmission_available = True
|
||||
except ImportError as e:
|
||||
self._transmission_available = False
|
||||
self.logger.warning(f"[Transmission] transmission_rpc not installed: {e}")
|
||||
except Exception as e:
|
||||
self._transmission_available = False
|
||||
self.logger.warning(f"[Transmission] Connection failed to {host}:{port}: {type(e).__name__}: {e}")
|
||||
|
||||
self.logger.info(f"[Transmission] Final status: available={self._transmission_available}")
|
||||
return self._transmission_available
|
||||
|
||||
def _get_manager(self):
|
||||
"""Lazily initialize and return the FetchManager."""
|
||||
if self._manager is None:
|
||||
# Import flacfetch here to avoid import errors if not installed
|
||||
from flacfetch.core.manager import FetchManager
|
||||
from flacfetch.providers.youtube import YouTubeProvider
|
||||
from flacfetch.providers.youtube import YoutubeProvider
|
||||
from flacfetch.downloaders.youtube import YoutubeDownloader
|
||||
|
||||
# Try to import TorrentDownloader (has optional dependencies)
|
||||
TorrentDownloader = None
|
||||
try:
|
||||
from flacfetch.downloaders.torrent import TorrentDownloader
|
||||
except ImportError:
|
||||
self.logger.debug("TorrentDownloader not available (missing dependencies)")
|
||||
|
||||
self._manager = FetchManager()
|
||||
|
||||
# Add providers based on available API keys
|
||||
if self._redacted_api_key:
|
||||
# Only add tracker providers if we can actually download from them
|
||||
# This requires both TorrentDownloader and a running Transmission daemon
|
||||
has_torrent_downloader = TorrentDownloader is not None
|
||||
transmission_available = self._check_transmission_available()
|
||||
can_use_trackers = has_torrent_downloader and transmission_available
|
||||
|
||||
self.logger.info(
|
||||
f"[FlacFetcher] Provider setup: TorrentDownloader={has_torrent_downloader}, "
|
||||
f"Transmission={transmission_available}, can_use_trackers={can_use_trackers}"
|
||||
)
|
||||
|
||||
if not can_use_trackers and (self._redacted_api_key or self._ops_api_key):
|
||||
self.logger.warning(
|
||||
"[FlacFetcher] Tracker providers (Redacted/OPS) DISABLED: "
|
||||
f"TorrentDownloader={has_torrent_downloader}, Transmission={transmission_available}. "
|
||||
"Only YouTube sources will be used."
|
||||
)
|
||||
|
||||
# Add providers and downloaders based on available API keys
|
||||
if self._redacted_api_key and can_use_trackers:
|
||||
from flacfetch.providers.redacted import RedactedProvider
|
||||
|
||||
self._manager.add_provider(RedactedProvider(api_key=self._redacted_api_key))
|
||||
self.logger.debug("Added Redacted provider")
|
||||
self._manager.register_downloader("Redacted", TorrentDownloader())
|
||||
self.logger.info("[FlacFetcher] Added Redacted provider with TorrentDownloader")
|
||||
|
||||
if self._ops_api_key:
|
||||
if self._ops_api_key and can_use_trackers:
|
||||
from flacfetch.providers.ops import OPSProvider
|
||||
|
||||
self._manager.add_provider(OPSProvider(api_key=self._ops_api_key))
|
||||
self.logger.debug("Added OPS provider")
|
||||
self._manager.register_downloader("OPS", TorrentDownloader())
|
||||
self.logger.info("[FlacFetcher] Added OPS provider with TorrentDownloader")
|
||||
|
||||
# Always add YouTube as a fallback provider
|
||||
self._manager.add_provider(YouTubeProvider())
|
||||
# Always add YouTube as a fallback provider with its downloader
|
||||
self._manager.add_provider(YoutubeProvider())
|
||||
self._manager.register_downloader("YouTube", YoutubeDownloader())
|
||||
self.logger.debug("Added YouTube provider")
|
||||
|
||||
return self._manager
|
||||
@@ -219,16 +396,23 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
|
||||
# Convert to our AudioSearchResult format
|
||||
search_results = []
|
||||
for result in results:
|
||||
for i, result in enumerate(results):
|
||||
# Get quality as string if it's a Quality object
|
||||
quality = getattr(result, "quality", None)
|
||||
quality_str = str(quality) if quality else None
|
||||
|
||||
search_results.append(
|
||||
AudioSearchResult(
|
||||
title=getattr(result, "title", title),
|
||||
artist=getattr(result, "artist", artist),
|
||||
url=getattr(result, "url", ""),
|
||||
provider=getattr(result, "provider", "Unknown"),
|
||||
duration=getattr(result, "duration", None),
|
||||
quality=getattr(result, "quality", None),
|
||||
source_id=getattr(result, "id", None),
|
||||
url=getattr(result, "download_url", "") or "",
|
||||
provider=getattr(result, "source_name", "Unknown"),
|
||||
duration=getattr(result, "duration_seconds", None),
|
||||
quality=quality_str,
|
||||
source_id=getattr(result, "info_hash", None),
|
||||
index=i, # Set index for API selection
|
||||
seeders=getattr(result, "seeders", None),
|
||||
target_file=getattr(result, "target_file", None),
|
||||
raw_result=result,
|
||||
)
|
||||
)
|
||||
@@ -265,7 +449,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
if output_filename is None:
|
||||
output_filename = f"{result.artist} - {result.title}"
|
||||
|
||||
self.logger.info(f"Downloading: {result.artist} - {result.title} from {result.provider}")
|
||||
self.logger.info(f"Downloading: {result.artist} - {result.title} from {result.provider or 'Unknown'}")
|
||||
|
||||
try:
|
||||
# Use flacfetch to download
|
||||
@@ -292,6 +476,40 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
except Exception as e:
|
||||
raise DownloadError(f"Failed to download {result.artist} - {result.title}: {e}") from e
|
||||
|
||||
def select_best(self, results: List[AudioSearchResult]) -> int:
|
||||
"""
|
||||
Select the best result from a list of search results.
|
||||
|
||||
Uses flacfetch's built-in quality ranking to determine the best source.
|
||||
This is useful for automated/non-interactive usage.
|
||||
|
||||
Args:
|
||||
results: List of AudioSearchResult objects from search()
|
||||
|
||||
Returns:
|
||||
Index of the best result in the list
|
||||
"""
|
||||
if not results:
|
||||
return 0
|
||||
|
||||
manager = self._get_manager()
|
||||
|
||||
# Get raw results that have raw_result set
|
||||
raw_results = [r.raw_result for r in results if r.raw_result is not None]
|
||||
|
||||
if raw_results:
|
||||
try:
|
||||
best = manager.select_best(raw_results)
|
||||
# Find index of best result
|
||||
for i, r in enumerate(results):
|
||||
if r.raw_result == best:
|
||||
return i
|
||||
except Exception as e:
|
||||
self.logger.warning(f"select_best failed, using first result: {e}")
|
||||
|
||||
# Fallback: return first result
|
||||
return 0
|
||||
|
||||
def search_and_download(
|
||||
self,
|
||||
artist: str,
|
||||
@@ -319,6 +537,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
Raises:
|
||||
NoResultsError: If no results are found
|
||||
DownloadError: If download fails
|
||||
UserCancelledError: If user cancels (Ctrl+C or enters 0)
|
||||
"""
|
||||
from flacfetch.core.models import TrackQuery
|
||||
|
||||
@@ -326,7 +545,9 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
query = TrackQuery(artist=artist, title=title)
|
||||
|
||||
self.logger.info(f"Searching for: {artist} - {title}")
|
||||
results = manager.search(query)
|
||||
|
||||
# Run search in a thread so we can handle Ctrl+C
|
||||
results = self._interruptible_search(manager, query)
|
||||
|
||||
if not results:
|
||||
raise NoResultsError(f"No results found for: {artist} - {title}")
|
||||
@@ -336,11 +557,13 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
if auto_select:
|
||||
# Auto mode: select best result based on flacfetch's ranking
|
||||
selected = manager.select_best(results)
|
||||
self.logger.info(f"Auto-selected: {getattr(selected, 'title', title)} from {getattr(selected, 'provider', 'Unknown')}")
|
||||
self.logger.info(f"Auto-selected: {getattr(selected, 'title', title)} from {getattr(selected, 'source_name', 'Unknown')}")
|
||||
else:
|
||||
# Interactive mode: present options to user
|
||||
selected = self._interactive_select(results, artist, title)
|
||||
|
||||
# Note: _interactive_select now raises UserCancelledError instead of returning None
|
||||
# This check is kept as a safety net
|
||||
if selected is None:
|
||||
raise NoResultsError(f"No result selected for: {artist} - {title}")
|
||||
|
||||
@@ -351,69 +574,262 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
if output_filename is None:
|
||||
output_filename = f"{artist} - {title}"
|
||||
|
||||
self.logger.info(f"Downloading from {getattr(selected, 'provider', 'Unknown')}...")
|
||||
self.logger.info(f"Downloading from {getattr(selected, 'source_name', 'Unknown')}...")
|
||||
|
||||
try:
|
||||
filepath = manager.download(
|
||||
# Use interruptible download so Ctrl+C works during torrent downloads
|
||||
filepath = self._interruptible_download(
|
||||
manager,
|
||||
selected,
|
||||
output_path=output_dir,
|
||||
output_filename=output_filename,
|
||||
)
|
||||
|
||||
if filepath is None:
|
||||
if not filepath:
|
||||
raise DownloadError(f"Download returned no file path for: {artist} - {title}")
|
||||
|
||||
self.logger.info(f"Downloaded to: {filepath}")
|
||||
|
||||
# Get quality as string if it's a Quality object
|
||||
quality = getattr(selected, "quality", None)
|
||||
quality_str = str(quality) if quality else None
|
||||
|
||||
return AudioFetchResult(
|
||||
filepath=filepath,
|
||||
artist=artist,
|
||||
title=title,
|
||||
provider=getattr(selected, "provider", "Unknown"),
|
||||
duration=getattr(selected, "duration", None),
|
||||
quality=getattr(selected, "quality", None),
|
||||
provider=getattr(selected, "source_name", "Unknown"),
|
||||
duration=getattr(selected, "duration_seconds", None),
|
||||
quality=quality_str,
|
||||
)
|
||||
|
||||
except (UserCancelledError, KeyboardInterrupt):
|
||||
# Let cancellation exceptions propagate without wrapping
|
||||
raise
|
||||
except Exception as e:
|
||||
raise DownloadError(f"Failed to download {artist} - {title}: {e}") from e
|
||||
|
||||
def _interruptible_search(self, manager, query) -> list:
|
||||
"""
|
||||
Run search in a way that can be interrupted by Ctrl+C.
|
||||
|
||||
The flacfetch search is a blocking network operation that doesn't
|
||||
respond to SIGINT while running. This method runs it in a background
|
||||
thread and periodically checks for interrupts.
|
||||
|
||||
Args:
|
||||
manager: The FetchManager instance
|
||||
query: The TrackQuery to search for
|
||||
|
||||
Returns:
|
||||
List of search results
|
||||
|
||||
Raises:
|
||||
UserCancelledError: If user presses Ctrl+C during search
|
||||
"""
|
||||
global _interrupt_requested
|
||||
_interrupt_requested = False
|
||||
result_container = {"results": None, "error": None}
|
||||
|
||||
def do_search():
|
||||
try:
|
||||
result_container["results"] = manager.search(query)
|
||||
except Exception as e:
|
||||
result_container["error"] = e
|
||||
|
||||
# Set up signal handler for immediate response to Ctrl+C
|
||||
original_handler = signal.getsignal(signal.SIGINT)
|
||||
|
||||
def interrupt_handler(signum, frame):
|
||||
global _interrupt_requested
|
||||
_interrupt_requested = True
|
||||
# Print immediately so user knows it was received
|
||||
print("\nCancelling... please wait", file=sys.stderr)
|
||||
|
||||
signal.signal(signal.SIGINT, interrupt_handler)
|
||||
|
||||
try:
|
||||
# Start search in background thread
|
||||
search_thread = threading.Thread(target=do_search, daemon=True)
|
||||
search_thread.start()
|
||||
|
||||
# Wait for completion with periodic interrupt checks
|
||||
while search_thread.is_alive():
|
||||
search_thread.join(timeout=0.1) # Check every 100ms
|
||||
if _interrupt_requested:
|
||||
# Don't wait for thread - it's a daemon and will be killed
|
||||
raise UserCancelledError("Search cancelled by user (Ctrl+C)")
|
||||
|
||||
# Check for errors from the search
|
||||
if result_container["error"] is not None:
|
||||
raise result_container["error"]
|
||||
|
||||
return result_container["results"]
|
||||
|
||||
finally:
|
||||
# Restore original signal handler
|
||||
signal.signal(signal.SIGINT, original_handler)
|
||||
_interrupt_requested = False
|
||||
|
||||
def _interruptible_download(self, manager, selected, output_path: str, output_filename: str) -> str:
|
||||
"""
|
||||
Run download in a way that can be interrupted by Ctrl+C.
|
||||
|
||||
The flacfetch/transmission download is a blocking operation that doesn't
|
||||
respond to SIGINT while running (especially for torrent downloads).
|
||||
This method runs it in a background thread and periodically checks for interrupts.
|
||||
|
||||
Args:
|
||||
manager: The FetchManager instance
|
||||
selected: The selected result to download
|
||||
output_path: Directory to save the file
|
||||
output_filename: Filename to save as
|
||||
|
||||
Returns:
|
||||
Path to the downloaded file
|
||||
|
||||
Raises:
|
||||
UserCancelledError: If user presses Ctrl+C during download
|
||||
DownloadError: If download fails
|
||||
"""
|
||||
global _interrupt_requested
|
||||
_interrupt_requested = False
|
||||
result_container = {"filepath": None, "error": None}
|
||||
was_cancelled = False
|
||||
|
||||
def do_download():
|
||||
try:
|
||||
result_container["filepath"] = manager.download(
|
||||
selected,
|
||||
output_path=output_path,
|
||||
output_filename=output_filename,
|
||||
)
|
||||
except Exception as e:
|
||||
result_container["error"] = e
|
||||
|
||||
# Set up signal handler for immediate response to Ctrl+C
|
||||
original_handler = signal.getsignal(signal.SIGINT)
|
||||
|
||||
def interrupt_handler(signum, frame):
|
||||
global _interrupt_requested
|
||||
_interrupt_requested = True
|
||||
# Print immediately so user knows it was received
|
||||
print("\nCancelling download... please wait (may take a few seconds)", file=sys.stderr)
|
||||
|
||||
signal.signal(signal.SIGINT, interrupt_handler)
|
||||
|
||||
try:
|
||||
# Start download in background thread
|
||||
download_thread = threading.Thread(target=do_download, daemon=True)
|
||||
download_thread.start()
|
||||
|
||||
# Wait for completion with periodic interrupt checks
|
||||
while download_thread.is_alive():
|
||||
download_thread.join(timeout=0.2) # Check every 200ms
|
||||
if _interrupt_requested:
|
||||
was_cancelled = True
|
||||
# Clean up any pending torrents before raising
|
||||
self._cleanup_transmission_torrents(selected)
|
||||
raise UserCancelledError("Download cancelled by user (Ctrl+C)")
|
||||
|
||||
# Check for errors from the download
|
||||
if result_container["error"] is not None:
|
||||
raise result_container["error"]
|
||||
|
||||
return result_container["filepath"]
|
||||
|
||||
finally:
|
||||
# Restore original signal handler
|
||||
signal.signal(signal.SIGINT, original_handler)
|
||||
_interrupt_requested = False
|
||||
|
||||
def _cleanup_transmission_torrents(self, selected) -> None:
|
||||
"""
|
||||
Clean up any torrents in Transmission that were started for this download.
|
||||
|
||||
Called when a download is cancelled to remove incomplete torrents and their data.
|
||||
|
||||
Args:
|
||||
selected: The selected result that was being downloaded
|
||||
"""
|
||||
try:
|
||||
import transmission_rpc
|
||||
host = os.environ.get("TRANSMISSION_HOST", "localhost")
|
||||
port = int(os.environ.get("TRANSMISSION_PORT", "9091"))
|
||||
client = transmission_rpc.Client(host=host, port=port, timeout=5)
|
||||
|
||||
# Get the release name to match against torrents
|
||||
release_name = getattr(selected, 'name', None) or getattr(selected, 'title', None)
|
||||
if not release_name:
|
||||
self.logger.debug("[Transmission] No release name to match for cleanup")
|
||||
return
|
||||
|
||||
# Find and remove matching incomplete torrents
|
||||
torrents = client.get_torrents()
|
||||
for torrent in torrents:
|
||||
# Match by name similarity and incomplete status
|
||||
if torrent.progress < 100 and release_name.lower() in torrent.name.lower():
|
||||
self.logger.info(f"[Transmission] Removing cancelled torrent: {torrent.name}")
|
||||
client.remove_torrent(torrent.id, delete_data=True)
|
||||
|
||||
except Exception as e:
|
||||
# Don't fail the cancellation if cleanup fails
|
||||
self.logger.debug(f"[Transmission] Cleanup failed (non-fatal): {e}")
|
||||
|
||||
def _interactive_select(self, results: list, artist: str, title: str) -> object:
|
||||
"""
|
||||
Present search results to the user for interactive selection.
|
||||
|
||||
Uses flacfetch's built-in CLIHandler for rich, colorized output.
|
||||
|
||||
Args:
|
||||
results: List of search results from flacfetch
|
||||
results: List of Release objects from flacfetch
|
||||
artist: The artist name being searched
|
||||
title: The track title being searched
|
||||
|
||||
Returns:
|
||||
The selected result object
|
||||
The selected Release object
|
||||
|
||||
Raises:
|
||||
UserCancelledError: If user cancels selection
|
||||
"""
|
||||
print(f"\n{'=' * 60}")
|
||||
print(f"Search Results for: {artist} - {title}")
|
||||
print(f"{'=' * 60}\n")
|
||||
try:
|
||||
# Use flacfetch's built-in CLIHandler for rich display
|
||||
from flacfetch.interface.cli import CLIHandler
|
||||
|
||||
for i, result in enumerate(results, 1):
|
||||
provider = getattr(result, "provider", "Unknown")
|
||||
result_title = getattr(result, "title", "Unknown")
|
||||
result_artist = getattr(result, "artist", "Unknown")
|
||||
quality = getattr(result, "quality", "")
|
||||
duration = getattr(result, "duration", None)
|
||||
handler = CLIHandler(target_artist=artist)
|
||||
result = handler.select_release(results)
|
||||
if result is None:
|
||||
# User selected 0 to cancel
|
||||
raise UserCancelledError("Selection cancelled by user")
|
||||
return result
|
||||
except ImportError:
|
||||
# Fallback to basic display if CLIHandler not available
|
||||
return self._basic_interactive_select(results, artist, title)
|
||||
except (KeyboardInterrupt, EOFError):
|
||||
raise UserCancelledError("Selection cancelled by user (Ctrl+C)")
|
||||
except (AttributeError, TypeError):
|
||||
# Fallback if results aren't proper Release objects (e.g., in tests)
|
||||
return self._basic_interactive_select(results, artist, title)
|
||||
|
||||
# Format duration if available
|
||||
duration_str = ""
|
||||
if duration:
|
||||
minutes = duration // 60
|
||||
seconds = duration % 60
|
||||
duration_str = f" [{minutes}:{seconds:02d}]"
|
||||
def _basic_interactive_select(self, results: list, artist: str, title: str) -> object:
|
||||
"""
|
||||
Basic fallback for interactive selection without rich formatting.
|
||||
|
||||
quality_str = f" ({quality})" if quality else ""
|
||||
Args:
|
||||
results: List of Release objects from flacfetch
|
||||
artist: The artist name being searched
|
||||
title: The track title being searched
|
||||
|
||||
print(f" {i}. [{provider}] {result_artist} - {result_title}{quality_str}{duration_str}")
|
||||
|
||||
print()
|
||||
print(" 0. Cancel")
|
||||
print()
|
||||
Returns:
|
||||
The selected Release object
|
||||
|
||||
Raises:
|
||||
UserCancelledError: If user cancels selection
|
||||
"""
|
||||
# Use flacfetch's shared display function
|
||||
from flacfetch import print_releases
|
||||
print_releases(results, target_artist=artist, use_colors=True)
|
||||
|
||||
while True:
|
||||
try:
|
||||
@@ -421,7 +837,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
|
||||
if choice == "0":
|
||||
self.logger.info("User cancelled selection")
|
||||
return None
|
||||
raise UserCancelledError("Selection cancelled by user")
|
||||
|
||||
choice_num = int(choice)
|
||||
if 1 <= choice_num <= len(results):
|
||||
@@ -435,7 +851,11 @@ class FlacFetchAudioFetcher(AudioFetcher):
|
||||
print("Please enter a valid number")
|
||||
except KeyboardInterrupt:
|
||||
print("\nCancelled")
|
||||
return None
|
||||
raise UserCancelledError("Selection cancelled by user (Ctrl+C)")
|
||||
|
||||
|
||||
# Alias for shorter name - used by backend and other consumers
|
||||
FlacFetcher = FlacFetchAudioFetcher
|
||||
|
||||
|
||||
def create_audio_fetcher(
|
||||
|
||||
@@ -771,8 +771,16 @@ class AudioProcessor:
|
||||
padded_result["other_stems"] = separation_result.get("other_stems", {})
|
||||
padded_result["backing_vocals"] = separation_result.get("backing_vocals", {})
|
||||
|
||||
self.logger.info(
|
||||
f"✓ Countdown padding applied to {len(padded_result['combined_instrumentals']) + 1} instrumental file(s)"
|
||||
)
|
||||
# Count actual padded files (don't assume clean instrumental was padded)
|
||||
padded_count = (1 if padded_result["clean_instrumental"].get("instrumental") else 0) + len(padded_result["combined_instrumentals"])
|
||||
|
||||
if padded_count > 0:
|
||||
self.logger.info(
|
||||
f"✓ Countdown padding applied to {padded_count} instrumental file(s)"
|
||||
)
|
||||
else:
|
||||
self.logger.warning(
|
||||
"No instrumental files found to pad. Check that audio separation completed successfully."
|
||||
)
|
||||
|
||||
return padded_result
|
||||
@@ -5,6 +5,12 @@ import shutil
|
||||
import tempfile
|
||||
from .utils import sanitize_filename
|
||||
|
||||
try:
|
||||
import yt_dlp
|
||||
YT_DLP_AVAILABLE = True
|
||||
except ImportError:
|
||||
YT_DLP_AVAILABLE = False
|
||||
|
||||
|
||||
# Placeholder class or functions for file handling
|
||||
class FileHandler:
|
||||
@@ -71,6 +77,192 @@ class FileHandler:
|
||||
|
||||
return target_path
|
||||
|
||||
def download_video(self, url, output_filename_no_extension, cookies_str=None):
|
||||
"""
|
||||
Download audio from a URL (YouTube, etc.) using yt-dlp.
|
||||
|
||||
This method downloads the best quality audio from a URL and saves it
|
||||
to the specified output path. It handles YouTube and other video platforms
|
||||
supported by yt-dlp.
|
||||
|
||||
Args:
|
||||
url: URL to download from (YouTube, Vimeo, etc.)
|
||||
output_filename_no_extension: Output filename without extension
|
||||
cookies_str: Optional cookies string for authenticated downloads
|
||||
|
||||
Returns:
|
||||
Path to downloaded audio file, or None if failed
|
||||
"""
|
||||
if not YT_DLP_AVAILABLE:
|
||||
self.logger.error("yt-dlp is not installed. Install with: pip install yt-dlp")
|
||||
return None
|
||||
|
||||
self.logger.info(f"Downloading audio from URL: {url}")
|
||||
|
||||
# Configure yt-dlp options
|
||||
ydl_opts = {
|
||||
'format': 'bestaudio/best',
|
||||
'outtmpl': output_filename_no_extension + '.%(ext)s',
|
||||
'postprocessors': [{
|
||||
'key': 'FFmpegExtractAudio',
|
||||
'preferredcodec': 'best',
|
||||
'preferredquality': '0', # Best quality
|
||||
}],
|
||||
'quiet': True,
|
||||
'no_warnings': True,
|
||||
'extract_flat': False,
|
||||
# Anti-detection options
|
||||
'user_agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'retries': 3,
|
||||
'fragment_retries': 3,
|
||||
'http_headers': {
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
|
||||
},
|
||||
}
|
||||
|
||||
# Handle cookies if provided - use safe tempfile pattern to avoid leaks
|
||||
cookie_file_path = None
|
||||
if cookies_str:
|
||||
try:
|
||||
# Use context manager to safely write cookies file
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as cookie_file:
|
||||
cookie_file.write(cookies_str)
|
||||
cookie_file_path = cookie_file.name
|
||||
ydl_opts['cookiefile'] = cookie_file_path
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Failed to write cookies file: {e}")
|
||||
cookie_file_path = None
|
||||
|
||||
try:
|
||||
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
|
||||
# Extract info first to get actual filename
|
||||
info = ydl.extract_info(url, download=True)
|
||||
|
||||
if info is None:
|
||||
self.logger.error("Failed to extract info from URL")
|
||||
return None
|
||||
|
||||
# Find the downloaded file
|
||||
# The actual filename might differ from template due to post-processing
|
||||
downloaded_file = None
|
||||
|
||||
# Check common extensions
|
||||
for ext in ['m4a', 'opus', 'webm', 'mp3', 'flac', 'wav', 'ogg', 'aac']:
|
||||
candidate = f"{output_filename_no_extension}.{ext}"
|
||||
if os.path.exists(candidate):
|
||||
downloaded_file = candidate
|
||||
break
|
||||
|
||||
if downloaded_file is None:
|
||||
# Try to find any audio file with matching prefix
|
||||
import glob
|
||||
matches = glob.glob(f"{output_filename_no_extension}.*")
|
||||
audio_extensions = ['.m4a', '.opus', '.webm', '.mp3', '.flac', '.wav', '.ogg', '.aac']
|
||||
for match in matches:
|
||||
if any(match.endswith(ext) for ext in audio_extensions):
|
||||
downloaded_file = match
|
||||
break
|
||||
|
||||
if downloaded_file and os.path.exists(downloaded_file):
|
||||
self.logger.info(f"Successfully downloaded: {downloaded_file}")
|
||||
return downloaded_file
|
||||
else:
|
||||
self.logger.error("Downloaded file not found after yt-dlp completed")
|
||||
return None
|
||||
|
||||
except yt_dlp.DownloadError as e:
|
||||
self.logger.error(f"yt-dlp download error: {e}")
|
||||
return None
|
||||
except Exception as e:
|
||||
self.logger.error(f"Failed to download from URL: {e}")
|
||||
return None
|
||||
finally:
|
||||
# Clean up cookie file if we created one
|
||||
if cookie_file_path is not None:
|
||||
try:
|
||||
os.unlink(cookie_file_path)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def extract_metadata_from_url(self, url):
|
||||
"""
|
||||
Extract metadata (artist, title) from a URL without downloading.
|
||||
|
||||
Uses yt-dlp to fetch video metadata including title, uploader/artist,
|
||||
and other information that can be used for the karaoke generation.
|
||||
|
||||
Args:
|
||||
url: URL to extract metadata from
|
||||
|
||||
Returns:
|
||||
Dict with 'artist', 'title', 'duration', and 'raw_info', or None if failed
|
||||
"""
|
||||
if not YT_DLP_AVAILABLE:
|
||||
self.logger.error("yt-dlp is not installed. Install with: pip install yt-dlp")
|
||||
return None
|
||||
|
||||
self.logger.info(f"Extracting metadata from URL: {url}")
|
||||
|
||||
ydl_opts = {
|
||||
'quiet': True,
|
||||
'no_warnings': True,
|
||||
'extract_flat': False,
|
||||
'skip_download': True,
|
||||
}
|
||||
|
||||
try:
|
||||
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
|
||||
info = ydl.extract_info(url, download=False)
|
||||
|
||||
if info is None:
|
||||
self.logger.error("Failed to extract metadata from URL")
|
||||
return None
|
||||
|
||||
# Try to extract artist and title from various fields
|
||||
raw_title = info.get('title', '')
|
||||
uploader = info.get('uploader', '') or info.get('channel', '') or info.get('artist', '')
|
||||
duration = info.get('duration', 0)
|
||||
|
||||
# Attempt to parse "Artist - Title" format from title
|
||||
artist = None
|
||||
title = raw_title
|
||||
|
||||
if ' - ' in raw_title:
|
||||
parts = raw_title.split(' - ', 1)
|
||||
if len(parts) == 2:
|
||||
artist = parts[0].strip()
|
||||
title = parts[1].strip()
|
||||
|
||||
# Fall back to uploader as artist if not found in title
|
||||
if not artist:
|
||||
artist = uploader
|
||||
|
||||
# Clean up title (remove common suffixes like "(Official Video)")
|
||||
title_cleanup_patterns = [
|
||||
'(official video)', '(official music video)', '(official audio)',
|
||||
'(lyric video)', '(lyrics)', '(visualizer)', '(music video)',
|
||||
'[official video]', '[official music video]', '[official audio]',
|
||||
'(hd)', '(4k)', '(remastered)', '| official video', '| official audio',
|
||||
]
|
||||
title_lower = title.lower()
|
||||
for pattern in title_cleanup_patterns:
|
||||
if pattern in title_lower:
|
||||
idx = title_lower.find(pattern)
|
||||
title = title[:idx].strip()
|
||||
title_lower = title.lower()
|
||||
|
||||
return {
|
||||
'artist': artist,
|
||||
'title': title,
|
||||
'duration': duration,
|
||||
'raw_info': info,
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Failed to extract metadata from URL: {e}")
|
||||
return None
|
||||
|
||||
def extract_still_image_from_video(self, input_filename, output_filename_no_extension):
|
||||
output_filename = output_filename_no_extension + ".png"
|
||||
self.logger.info(f"Extracting still image from position 30s input media")
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
"""
|
||||
Instrumental Review Module - Shared core for both local and remote CLI.
|
||||
|
||||
This module provides audio analysis and editing functionality for instrumental
|
||||
selection in karaoke generation. It's designed to be:
|
||||
- Pure Python with no cloud dependencies (GCS, etc.)
|
||||
- Reusable by both local CLI (karaoke-gen) and remote backend (Cloud Run)
|
||||
- Easy to test without mocking cloud services
|
||||
|
||||
Classes:
|
||||
AudioAnalyzer: Analyzes backing vocals audio for audible content
|
||||
AudioEditor: Creates custom instrumentals by muting regions
|
||||
WaveformGenerator: Generates waveform visualization images
|
||||
InstrumentalReviewServer: Local HTTP server for browser-based review
|
||||
|
||||
Models:
|
||||
AnalysisResult: Result of audio analysis
|
||||
AudibleSegment: A detected segment of audible content
|
||||
MuteRegion: A region to mute in the backing vocals
|
||||
RecommendedSelection: Enum of selection recommendations
|
||||
"""
|
||||
|
||||
from .models import (
|
||||
AnalysisResult,
|
||||
AudibleSegment,
|
||||
MuteRegion,
|
||||
RecommendedSelection,
|
||||
)
|
||||
from .analyzer import AudioAnalyzer
|
||||
from .editor import AudioEditor
|
||||
from .waveform import WaveformGenerator
|
||||
from .server import InstrumentalReviewServer
|
||||
|
||||
__all__ = [
|
||||
# Models
|
||||
"AnalysisResult",
|
||||
"AudibleSegment",
|
||||
"MuteRegion",
|
||||
"RecommendedSelection",
|
||||
# Classes
|
||||
"AudioAnalyzer",
|
||||
"AudioEditor",
|
||||
"WaveformGenerator",
|
||||
"InstrumentalReviewServer",
|
||||
]
|
||||
@@ -0,0 +1,408 @@
|
||||
"""
|
||||
Audio analyzer for detecting audible content in backing vocals.
|
||||
|
||||
This module provides the AudioAnalyzer class which analyzes audio files
|
||||
to detect segments of audible content above a silence threshold. It's used
|
||||
to help determine whether backing vocals should be included in the final
|
||||
karaoke instrumental.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
from pydub import AudioSegment
|
||||
|
||||
from .models import AnalysisResult, AudibleSegment, RecommendedSelection
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AudioAnalyzer:
|
||||
"""
|
||||
Analyzes audio files for backing vocals content.
|
||||
|
||||
This class is pure Python with no cloud dependencies. It works with
|
||||
local file paths and uses pydub for audio loading and analysis.
|
||||
|
||||
The analyzer detects segments of audible content (above a silence threshold)
|
||||
and provides recommendations for instrumental selection based on the
|
||||
analysis results.
|
||||
|
||||
Attributes:
|
||||
silence_threshold_db: Amplitude threshold below which audio is
|
||||
considered silent (default: -40.0 dB)
|
||||
min_segment_duration_ms: Minimum duration for a segment to be
|
||||
considered audible (default: 100ms)
|
||||
merge_gap_ms: Maximum gap between segments to merge them
|
||||
(default: 500ms)
|
||||
window_ms: Analysis window size in milliseconds (default: 50ms)
|
||||
|
||||
Example:
|
||||
>>> analyzer = AudioAnalyzer(silence_threshold_db=-40.0)
|
||||
>>> result = analyzer.analyze("/path/to/backing_vocals.flac")
|
||||
>>> if result.has_audible_content:
|
||||
... print(f"Found {result.segment_count} audible segments")
|
||||
... for seg in result.audible_segments:
|
||||
... print(f" {seg.start_seconds:.1f}s - {seg.end_seconds:.1f}s")
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
silence_threshold_db: float = -40.0,
|
||||
min_segment_duration_ms: int = 100,
|
||||
merge_gap_ms: int = 500,
|
||||
window_ms: int = 50,
|
||||
):
|
||||
"""
|
||||
Initialize the audio analyzer.
|
||||
|
||||
Args:
|
||||
silence_threshold_db: Amplitude threshold below which audio is
|
||||
considered silent. Default is -40.0 dB.
|
||||
min_segment_duration_ms: Minimum duration for a segment to be
|
||||
reported as audible. Segments shorter than this are ignored.
|
||||
Default is 100ms.
|
||||
merge_gap_ms: If two audible segments are separated by a gap
|
||||
shorter than this, they are merged into one segment.
|
||||
Default is 500ms.
|
||||
window_ms: Size of the analysis window in milliseconds.
|
||||
Smaller windows give more precise timing but slower analysis.
|
||||
Default is 50ms.
|
||||
"""
|
||||
self.silence_threshold_db = silence_threshold_db
|
||||
self.min_segment_duration_ms = min_segment_duration_ms
|
||||
self.merge_gap_ms = merge_gap_ms
|
||||
self.window_ms = window_ms
|
||||
|
||||
def analyze(self, audio_path: str) -> AnalysisResult:
|
||||
"""
|
||||
Analyze an audio file for audible content.
|
||||
|
||||
This method loads the audio file, calculates amplitude levels across
|
||||
the duration, and identifies segments where the amplitude exceeds
|
||||
the silence threshold.
|
||||
|
||||
Args:
|
||||
audio_path: Path to the audio file to analyze. Supports formats
|
||||
that pydub/ffmpeg can read (FLAC, WAV, MP3, etc.)
|
||||
|
||||
Returns:
|
||||
AnalysisResult containing:
|
||||
- has_audible_content: Whether any audible content was found
|
||||
- total_duration_seconds: Total duration of the audio
|
||||
- audible_segments: List of detected audible segments
|
||||
- recommended_selection: Recommendation for which instrumental
|
||||
- Various statistics about the audible content
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If the audio file doesn't exist
|
||||
Exception: If the audio file cannot be loaded
|
||||
"""
|
||||
path = Path(audio_path)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Audio file not found: {audio_path}")
|
||||
|
||||
logger.info(f"Analyzing audio file: {audio_path}")
|
||||
|
||||
# Load audio file
|
||||
audio = AudioSegment.from_file(audio_path)
|
||||
total_duration_ms = len(audio)
|
||||
total_duration_seconds = total_duration_ms / 1000.0
|
||||
|
||||
logger.debug(f"Audio duration: {total_duration_seconds:.2f}s, "
|
||||
f"channels: {audio.channels}, "
|
||||
f"sample_rate: {audio.frame_rate}")
|
||||
|
||||
# Convert to mono for consistent analysis
|
||||
if audio.channels > 1:
|
||||
audio = audio.set_channels(1)
|
||||
|
||||
# Analyze amplitude in windows
|
||||
audible_windows = self._find_audible_windows(audio)
|
||||
|
||||
# Merge adjacent windows into segments
|
||||
raw_segments = self._windows_to_segments(audible_windows, audio)
|
||||
|
||||
# Merge close segments and filter short ones
|
||||
segments = self._merge_and_filter_segments(raw_segments)
|
||||
|
||||
# Calculate statistics
|
||||
total_audible_ms = sum(
|
||||
seg.duration_seconds * 1000 for seg in segments
|
||||
)
|
||||
total_audible_seconds = total_audible_ms / 1000.0
|
||||
audible_percentage = (
|
||||
(total_audible_seconds / total_duration_seconds * 100)
|
||||
if total_duration_seconds > 0 else 0.0
|
||||
)
|
||||
|
||||
has_audible_content = len(segments) > 0
|
||||
|
||||
# Determine recommendation
|
||||
recommended_selection = self._get_recommendation(
|
||||
has_audible_content,
|
||||
segments,
|
||||
audible_percentage
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"Analysis complete: {len(segments)} segments, "
|
||||
f"{audible_percentage:.1f}% audible, "
|
||||
f"recommendation: {recommended_selection.value}"
|
||||
)
|
||||
|
||||
return AnalysisResult(
|
||||
has_audible_content=has_audible_content,
|
||||
total_duration_seconds=total_duration_seconds,
|
||||
audible_segments=segments,
|
||||
recommended_selection=recommended_selection,
|
||||
silence_threshold_db=self.silence_threshold_db,
|
||||
total_audible_duration_seconds=total_audible_seconds,
|
||||
audible_percentage=audible_percentage,
|
||||
)
|
||||
|
||||
def get_amplitude_envelope(
|
||||
self,
|
||||
audio_path: str,
|
||||
window_ms: int = 100,
|
||||
normalize: bool = True,
|
||||
) -> List[float]:
|
||||
"""
|
||||
Get the amplitude envelope for waveform visualization.
|
||||
|
||||
This method returns a list of amplitude values suitable for
|
||||
rendering a waveform display. Each value represents the RMS
|
||||
amplitude of a window of audio.
|
||||
|
||||
Args:
|
||||
audio_path: Path to the audio file
|
||||
window_ms: Size of each window in milliseconds. Smaller values
|
||||
give more detail but larger data. Default is 100ms.
|
||||
normalize: If True, normalize amplitudes to 0.0-1.0 range.
|
||||
Default is True.
|
||||
|
||||
Returns:
|
||||
List of amplitude values (floats). If normalize=True, values
|
||||
are in the range [0.0, 1.0]. Otherwise, values are in dBFS.
|
||||
"""
|
||||
path = Path(audio_path)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Audio file not found: {audio_path}")
|
||||
|
||||
audio = AudioSegment.from_file(audio_path)
|
||||
|
||||
# Convert to mono
|
||||
if audio.channels > 1:
|
||||
audio = audio.set_channels(1)
|
||||
|
||||
amplitudes = []
|
||||
duration_ms = len(audio)
|
||||
|
||||
for start_ms in range(0, duration_ms, window_ms):
|
||||
end_ms = min(start_ms + window_ms, duration_ms)
|
||||
window = audio[start_ms:end_ms]
|
||||
|
||||
# Get RMS amplitude in dBFS
|
||||
if window.rms > 0:
|
||||
db = 20 * math.log10(window.rms / window.max_possible_amplitude)
|
||||
else:
|
||||
db = -100.0 # Effectively silent
|
||||
|
||||
amplitudes.append(db)
|
||||
|
||||
if normalize and amplitudes:
|
||||
# Normalize to 0.0 - 1.0 range
|
||||
# Map from [silence_threshold, 0] to [0, 1]
|
||||
min_db = self.silence_threshold_db
|
||||
max_db = 0.0
|
||||
amplitudes = [
|
||||
max(0.0, min(1.0, (db - min_db) / (max_db - min_db)))
|
||||
for db in amplitudes
|
||||
]
|
||||
|
||||
return amplitudes
|
||||
|
||||
def _find_audible_windows(
|
||||
self,
|
||||
audio: AudioSegment
|
||||
) -> List[Tuple[int, float, float]]:
|
||||
"""
|
||||
Find windows with amplitude above the silence threshold.
|
||||
|
||||
Returns a list of tuples: (start_ms, avg_db, peak_db)
|
||||
"""
|
||||
audible_windows = []
|
||||
duration_ms = len(audio)
|
||||
|
||||
for start_ms in range(0, duration_ms, self.window_ms):
|
||||
end_ms = min(start_ms + self.window_ms, duration_ms)
|
||||
window = audio[start_ms:end_ms]
|
||||
|
||||
# Calculate RMS amplitude in dB
|
||||
if window.rms > 0:
|
||||
avg_db = 20 * math.log10(window.rms / window.max_possible_amplitude)
|
||||
# Peak is approximated as max sample value
|
||||
peak_db = window.dBFS if hasattr(window, 'dBFS') else avg_db
|
||||
else:
|
||||
avg_db = -100.0
|
||||
peak_db = -100.0
|
||||
|
||||
if avg_db > self.silence_threshold_db:
|
||||
audible_windows.append((start_ms, avg_db, peak_db))
|
||||
|
||||
return audible_windows
|
||||
|
||||
def _windows_to_segments(
|
||||
self,
|
||||
audible_windows: List[Tuple[int, float, float]],
|
||||
audio: AudioSegment
|
||||
) -> List[AudibleSegment]:
|
||||
"""
|
||||
Convert list of audible windows into contiguous segments.
|
||||
"""
|
||||
if not audible_windows:
|
||||
return []
|
||||
|
||||
segments = []
|
||||
segment_start_ms = audible_windows[0][0]
|
||||
segment_dbs = [audible_windows[0][1]]
|
||||
segment_peaks = [audible_windows[0][2]]
|
||||
last_end_ms = audible_windows[0][0] + self.window_ms
|
||||
|
||||
for i in range(1, len(audible_windows)):
|
||||
start_ms, avg_db, peak_db = audible_windows[i]
|
||||
|
||||
# Check if this window is contiguous with the previous
|
||||
gap_ms = start_ms - last_end_ms
|
||||
|
||||
if gap_ms <= self.window_ms:
|
||||
# Extend current segment
|
||||
segment_dbs.append(avg_db)
|
||||
segment_peaks.append(peak_db)
|
||||
last_end_ms = start_ms + self.window_ms
|
||||
else:
|
||||
# Save current segment and start a new one
|
||||
segments.append(self._create_segment(
|
||||
segment_start_ms, last_end_ms, segment_dbs, segment_peaks
|
||||
))
|
||||
|
||||
segment_start_ms = start_ms
|
||||
segment_dbs = [avg_db]
|
||||
segment_peaks = [peak_db]
|
||||
last_end_ms = start_ms + self.window_ms
|
||||
|
||||
# Don't forget the last segment
|
||||
segments.append(self._create_segment(
|
||||
segment_start_ms, last_end_ms, segment_dbs, segment_peaks
|
||||
))
|
||||
|
||||
return segments
|
||||
|
||||
def _create_segment(
|
||||
self,
|
||||
start_ms: int,
|
||||
end_ms: int,
|
||||
dbs: List[float],
|
||||
peaks: List[float]
|
||||
) -> AudibleSegment:
|
||||
"""Create an AudibleSegment from window data."""
|
||||
return AudibleSegment(
|
||||
start_seconds=start_ms / 1000.0,
|
||||
end_seconds=end_ms / 1000.0,
|
||||
duration_seconds=(end_ms - start_ms) / 1000.0,
|
||||
avg_amplitude_db=sum(dbs) / len(dbs) if dbs else -100.0,
|
||||
peak_amplitude_db=max(peaks) if peaks else -100.0,
|
||||
)
|
||||
|
||||
def _merge_and_filter_segments(
|
||||
self,
|
||||
segments: List[AudibleSegment]
|
||||
) -> List[AudibleSegment]:
|
||||
"""
|
||||
Merge segments that are close together and filter out short ones.
|
||||
"""
|
||||
if not segments:
|
||||
return []
|
||||
|
||||
# Sort by start time
|
||||
segments = sorted(segments, key=lambda s: s.start_seconds)
|
||||
|
||||
# Merge segments with small gaps
|
||||
merged = []
|
||||
current = segments[0]
|
||||
|
||||
for next_seg in segments[1:]:
|
||||
gap_ms = (next_seg.start_seconds - current.end_seconds) * 1000
|
||||
|
||||
if gap_ms <= self.merge_gap_ms:
|
||||
# Merge segments
|
||||
combined_duration = (
|
||||
next_seg.end_seconds - current.start_seconds
|
||||
)
|
||||
# Weight average amplitude by duration
|
||||
total_duration = (
|
||||
current.duration_seconds + next_seg.duration_seconds
|
||||
)
|
||||
weighted_avg_db = (
|
||||
(current.avg_amplitude_db * current.duration_seconds +
|
||||
next_seg.avg_amplitude_db * next_seg.duration_seconds)
|
||||
/ total_duration
|
||||
) if total_duration > 0 else -100.0
|
||||
|
||||
current = AudibleSegment(
|
||||
start_seconds=current.start_seconds,
|
||||
end_seconds=next_seg.end_seconds,
|
||||
duration_seconds=combined_duration,
|
||||
avg_amplitude_db=weighted_avg_db,
|
||||
peak_amplitude_db=max(
|
||||
current.peak_amplitude_db,
|
||||
next_seg.peak_amplitude_db
|
||||
),
|
||||
)
|
||||
else:
|
||||
merged.append(current)
|
||||
current = next_seg
|
||||
|
||||
merged.append(current)
|
||||
|
||||
# Filter out segments shorter than minimum duration
|
||||
min_duration_seconds = self.min_segment_duration_ms / 1000.0
|
||||
filtered = [
|
||||
seg for seg in merged
|
||||
if seg.duration_seconds >= min_duration_seconds
|
||||
]
|
||||
|
||||
return filtered
|
||||
|
||||
def _get_recommendation(
|
||||
self,
|
||||
has_audible_content: bool,
|
||||
segments: List[AudibleSegment],
|
||||
audible_percentage: float
|
||||
) -> RecommendedSelection:
|
||||
"""
|
||||
Determine the recommended instrumental selection.
|
||||
|
||||
Logic:
|
||||
- If no audible content: recommend clean instrumental
|
||||
- If audible content covers > 20% of the audio: likely has
|
||||
meaningful backing vocals, recommend review
|
||||
- Otherwise: minimal content, recommend clean
|
||||
"""
|
||||
if not has_audible_content:
|
||||
return RecommendedSelection.CLEAN
|
||||
|
||||
# If there's significant audible content, recommend review
|
||||
if audible_percentage > 20.0:
|
||||
return RecommendedSelection.REVIEW_NEEDED
|
||||
|
||||
# If there are loud segments, recommend review
|
||||
loud_segments = [seg for seg in segments if seg.is_loud]
|
||||
if loud_segments:
|
||||
return RecommendedSelection.REVIEW_NEEDED
|
||||
|
||||
# Minimal content - recommend clean
|
||||
return RecommendedSelection.CLEAN
|
||||
@@ -0,0 +1,322 @@
|
||||
"""
|
||||
Audio editor for creating custom instrumentals with muted regions.
|
||||
|
||||
This module provides the AudioEditor class which creates custom instrumental
|
||||
tracks by muting specified regions of backing vocals audio and combining
|
||||
it with the clean instrumental.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
from pydub import AudioSegment
|
||||
|
||||
from .models import CustomInstrumentalResult, MuteRegion
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AudioEditor:
|
||||
"""
|
||||
Creates custom instrumentals by muting regions of backing vocals.
|
||||
|
||||
This class is pure Python with no cloud dependencies. It works with
|
||||
local file paths and uses pydub for audio manipulation.
|
||||
|
||||
The editor takes a clean instrumental and backing vocals track,
|
||||
applies silence to specified regions of the backing vocals, and
|
||||
combines them to create a custom instrumental track.
|
||||
|
||||
Example:
|
||||
>>> editor = AudioEditor()
|
||||
>>> mute_regions = [
|
||||
... MuteRegion(start_seconds=45.0, end_seconds=48.5),
|
||||
... MuteRegion(start_seconds=120.0, end_seconds=125.0),
|
||||
... ]
|
||||
>>> result = editor.create_custom_instrumental(
|
||||
... clean_instrumental_path="/path/to/clean.flac",
|
||||
... backing_vocals_path="/path/to/backing.flac",
|
||||
... mute_regions=mute_regions,
|
||||
... output_path="/path/to/custom.flac"
|
||||
... )
|
||||
>>> print(f"Created: {result.output_path}")
|
||||
"""
|
||||
|
||||
def __init__(self, output_format: str = "flac"):
|
||||
"""
|
||||
Initialize the audio editor.
|
||||
|
||||
Args:
|
||||
output_format: Output audio format. Default is "flac".
|
||||
Supported formats depend on ffmpeg installation.
|
||||
"""
|
||||
self.output_format = output_format
|
||||
|
||||
def create_custom_instrumental(
|
||||
self,
|
||||
clean_instrumental_path: str,
|
||||
backing_vocals_path: str,
|
||||
mute_regions: List[MuteRegion],
|
||||
output_path: str,
|
||||
) -> CustomInstrumentalResult:
|
||||
"""
|
||||
Create a custom instrumental by muting regions of backing vocals.
|
||||
|
||||
This method:
|
||||
1. Loads the clean instrumental and backing vocals tracks
|
||||
2. Applies silence to the specified regions of the backing vocals
|
||||
3. Combines the clean instrumental with the edited backing vocals
|
||||
4. Exports the result to the specified output path
|
||||
|
||||
Args:
|
||||
clean_instrumental_path: Path to the clean instrumental audio file
|
||||
backing_vocals_path: Path to the backing vocals audio file
|
||||
mute_regions: List of regions to mute in the backing vocals
|
||||
output_path: Path where the output file should be saved
|
||||
|
||||
Returns:
|
||||
CustomInstrumentalResult containing the output path and statistics
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If input files don't exist
|
||||
ValueError: If mute regions are invalid
|
||||
"""
|
||||
# Validate inputs
|
||||
if not Path(clean_instrumental_path).exists():
|
||||
raise FileNotFoundError(
|
||||
f"Clean instrumental not found: {clean_instrumental_path}"
|
||||
)
|
||||
if not Path(backing_vocals_path).exists():
|
||||
raise FileNotFoundError(
|
||||
f"Backing vocals not found: {backing_vocals_path}"
|
||||
)
|
||||
|
||||
# Normalize and validate mute regions
|
||||
normalized_regions = self._normalize_mute_regions(mute_regions)
|
||||
|
||||
logger.info(
|
||||
f"Creating custom instrumental with {len(normalized_regions)} "
|
||||
f"mute regions"
|
||||
)
|
||||
|
||||
# Load audio files
|
||||
logger.debug(f"Loading clean instrumental: {clean_instrumental_path}")
|
||||
clean_instrumental = AudioSegment.from_file(clean_instrumental_path)
|
||||
|
||||
logger.debug(f"Loading backing vocals: {backing_vocals_path}")
|
||||
backing_vocals = AudioSegment.from_file(backing_vocals_path)
|
||||
|
||||
# Ensure same duration (use shorter one)
|
||||
clean_duration_ms = len(clean_instrumental)
|
||||
backing_duration_ms = len(backing_vocals)
|
||||
|
||||
if abs(clean_duration_ms - backing_duration_ms) > 1000:
|
||||
logger.warning(
|
||||
f"Duration mismatch: clean={clean_duration_ms}ms, "
|
||||
f"backing={backing_duration_ms}ms. Using shorter duration."
|
||||
)
|
||||
|
||||
target_duration_ms = min(clean_duration_ms, backing_duration_ms)
|
||||
clean_instrumental = clean_instrumental[:target_duration_ms]
|
||||
backing_vocals = backing_vocals[:target_duration_ms]
|
||||
|
||||
# Apply mute regions to backing vocals
|
||||
edited_backing = self._apply_mute_regions(
|
||||
backing_vocals, normalized_regions
|
||||
)
|
||||
|
||||
# Combine: clean instrumental + edited backing vocals
|
||||
# The backing vocals are mixed on top of the clean instrumental
|
||||
combined = clean_instrumental.overlay(edited_backing)
|
||||
|
||||
# Ensure output directory exists
|
||||
output_dir = Path(output_path).parent
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Export
|
||||
logger.info(f"Exporting custom instrumental to: {output_path}")
|
||||
combined.export(output_path, format=self.output_format)
|
||||
|
||||
# Calculate statistics with clamping to actual audio duration
|
||||
output_duration_seconds = len(combined) / 1000.0
|
||||
total_muted_ms = sum(
|
||||
(min(r.end_seconds, output_duration_seconds) - max(r.start_seconds, 0)) * 1000
|
||||
for r in normalized_regions
|
||||
if r.start_seconds < output_duration_seconds # Skip regions entirely outside audio
|
||||
)
|
||||
|
||||
return CustomInstrumentalResult(
|
||||
output_path=output_path,
|
||||
mute_regions_applied=normalized_regions,
|
||||
total_muted_duration_seconds=max(0, total_muted_ms / 1000.0),
|
||||
output_duration_seconds=output_duration_seconds,
|
||||
)
|
||||
|
||||
def apply_mute_to_single_track(
|
||||
self,
|
||||
audio_path: str,
|
||||
mute_regions: List[MuteRegion],
|
||||
output_path: str,
|
||||
) -> str:
|
||||
"""
|
||||
Apply mute regions to a single audio track.
|
||||
|
||||
This is useful for muting sections of backing vocals without
|
||||
combining with the clean instrumental.
|
||||
|
||||
Args:
|
||||
audio_path: Path to the input audio file
|
||||
mute_regions: List of regions to mute
|
||||
output_path: Path where the output file should be saved
|
||||
|
||||
Returns:
|
||||
Path to the output file
|
||||
"""
|
||||
if not Path(audio_path).exists():
|
||||
raise FileNotFoundError(f"Audio file not found: {audio_path}")
|
||||
|
||||
normalized_regions = self._normalize_mute_regions(mute_regions)
|
||||
|
||||
logger.info(f"Applying {len(normalized_regions)} mute regions to audio")
|
||||
|
||||
audio = AudioSegment.from_file(audio_path)
|
||||
edited = self._apply_mute_regions(audio, normalized_regions)
|
||||
|
||||
output_dir = Path(output_path).parent
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
edited.export(output_path, format=self.output_format)
|
||||
|
||||
return output_path
|
||||
|
||||
def _normalize_mute_regions(
|
||||
self,
|
||||
regions: List[MuteRegion]
|
||||
) -> List[MuteRegion]:
|
||||
"""
|
||||
Normalize mute regions: sort, validate, and merge overlapping ones.
|
||||
"""
|
||||
if not regions:
|
||||
return []
|
||||
|
||||
# Validate regions
|
||||
for region in regions:
|
||||
if region.start_seconds < 0:
|
||||
raise ValueError(
|
||||
f"Mute region start cannot be negative: {region.start_seconds}"
|
||||
)
|
||||
if region.end_seconds <= region.start_seconds:
|
||||
raise ValueError(
|
||||
f"Mute region end ({region.end_seconds}) must be after "
|
||||
f"start ({region.start_seconds})"
|
||||
)
|
||||
|
||||
# Sort by start time
|
||||
sorted_regions = sorted(regions, key=lambda r: r.start_seconds)
|
||||
|
||||
# Merge overlapping regions
|
||||
merged = [sorted_regions[0]]
|
||||
|
||||
for region in sorted_regions[1:]:
|
||||
last = merged[-1]
|
||||
|
||||
# Check if overlapping or adjacent
|
||||
if region.start_seconds <= last.end_seconds:
|
||||
# Merge
|
||||
merged[-1] = MuteRegion(
|
||||
start_seconds=last.start_seconds,
|
||||
end_seconds=max(last.end_seconds, region.end_seconds)
|
||||
)
|
||||
else:
|
||||
merged.append(region)
|
||||
|
||||
return merged
|
||||
|
||||
def _apply_mute_regions(
|
||||
self,
|
||||
audio: AudioSegment,
|
||||
regions: List[MuteRegion]
|
||||
) -> AudioSegment:
|
||||
"""
|
||||
Apply silence to the specified regions of an audio segment.
|
||||
"""
|
||||
if not regions:
|
||||
return audio
|
||||
|
||||
duration_ms = len(audio)
|
||||
result = audio
|
||||
|
||||
for region in regions:
|
||||
start_ms = int(region.start_seconds * 1000)
|
||||
end_ms = int(region.end_seconds * 1000)
|
||||
|
||||
# Clamp to audio boundaries
|
||||
start_ms = max(0, start_ms)
|
||||
end_ms = min(duration_ms, end_ms)
|
||||
|
||||
if start_ms >= end_ms:
|
||||
continue
|
||||
|
||||
# Create silence segment
|
||||
silence_duration = end_ms - start_ms
|
||||
silence = AudioSegment.silent(
|
||||
duration=silence_duration,
|
||||
frame_rate=audio.frame_rate
|
||||
)
|
||||
|
||||
# Replace the region with silence
|
||||
before = result[:start_ms]
|
||||
after = result[end_ms:]
|
||||
result = before + silence + after
|
||||
|
||||
logger.debug(
|
||||
f"Muted region: {start_ms/1000:.2f}s - {end_ms/1000:.2f}s"
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
def preview_with_mutes(
|
||||
self,
|
||||
clean_instrumental_path: str,
|
||||
backing_vocals_path: str,
|
||||
mute_regions: List[MuteRegion],
|
||||
output_path: Optional[str] = None,
|
||||
) -> AudioSegment:
|
||||
"""
|
||||
Create a preview of the custom instrumental (in memory).
|
||||
|
||||
This is useful for creating temporary previews without saving
|
||||
to disk. If output_path is provided, the preview is also saved.
|
||||
|
||||
Args:
|
||||
clean_instrumental_path: Path to the clean instrumental
|
||||
backing_vocals_path: Path to the backing vocals
|
||||
mute_regions: Regions to mute
|
||||
output_path: Optional path to save the preview
|
||||
|
||||
Returns:
|
||||
AudioSegment of the preview
|
||||
"""
|
||||
clean_instrumental = AudioSegment.from_file(clean_instrumental_path)
|
||||
backing_vocals = AudioSegment.from_file(backing_vocals_path)
|
||||
|
||||
# Match durations
|
||||
target_duration = min(len(clean_instrumental), len(backing_vocals))
|
||||
clean_instrumental = clean_instrumental[:target_duration]
|
||||
backing_vocals = backing_vocals[:target_duration]
|
||||
|
||||
# Apply mutes
|
||||
normalized = self._normalize_mute_regions(mute_regions)
|
||||
edited_backing = self._apply_mute_regions(backing_vocals, normalized)
|
||||
|
||||
# Combine
|
||||
combined = clean_instrumental.overlay(edited_backing)
|
||||
|
||||
if output_path:
|
||||
Path(output_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
combined.export(output_path, format=self.output_format)
|
||||
|
||||
return combined
|
||||
@@ -0,0 +1,171 @@
|
||||
"""
|
||||
Pydantic models for the instrumental review module.
|
||||
|
||||
These models define the data structures used for audio analysis results,
|
||||
mute regions, and recommendations. They're designed to be serializable
|
||||
to JSON for API responses.
|
||||
"""
|
||||
|
||||
from enum import Enum
|
||||
from typing import List, Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class RecommendedSelection(str, Enum):
|
||||
"""Recommendation for which instrumental to use."""
|
||||
|
||||
CLEAN = "clean"
|
||||
"""Use clean instrumental - no backing vocals detected or recommended."""
|
||||
|
||||
WITH_BACKING = "with_backing"
|
||||
"""Use instrumental with backing vocals - backing vocals sound good."""
|
||||
|
||||
REVIEW_NEEDED = "review_needed"
|
||||
"""Human review needed - backing vocals detected but quality uncertain."""
|
||||
|
||||
|
||||
class AudibleSegment(BaseModel):
|
||||
"""A detected segment of audible content in the backing vocals."""
|
||||
|
||||
start_seconds: float = Field(
|
||||
...,
|
||||
ge=0,
|
||||
description="Start time of the segment in seconds"
|
||||
)
|
||||
end_seconds: float = Field(
|
||||
...,
|
||||
ge=0,
|
||||
description="End time of the segment in seconds"
|
||||
)
|
||||
duration_seconds: float = Field(
|
||||
...,
|
||||
ge=0,
|
||||
description="Duration of the segment in seconds"
|
||||
)
|
||||
avg_amplitude_db: float = Field(
|
||||
...,
|
||||
description="Average amplitude of the segment in dB"
|
||||
)
|
||||
peak_amplitude_db: float = Field(
|
||||
default=0.0,
|
||||
description="Peak amplitude of the segment in dB"
|
||||
)
|
||||
|
||||
@property
|
||||
def is_loud(self) -> bool:
|
||||
"""Check if this segment is relatively loud (> -20dB average)."""
|
||||
return self.avg_amplitude_db > -20.0
|
||||
|
||||
|
||||
class MuteRegion(BaseModel):
|
||||
"""A region to mute in the backing vocals audio."""
|
||||
|
||||
start_seconds: float = Field(
|
||||
...,
|
||||
ge=0,
|
||||
description="Start time of the region to mute in seconds"
|
||||
)
|
||||
end_seconds: float = Field(
|
||||
...,
|
||||
ge=0,
|
||||
description="End time of the region to mute in seconds"
|
||||
)
|
||||
|
||||
@property
|
||||
def duration_seconds(self) -> float:
|
||||
"""Duration of the mute region in seconds."""
|
||||
return self.end_seconds - self.start_seconds
|
||||
|
||||
def overlaps(self, other: "MuteRegion") -> bool:
|
||||
"""Check if this region overlaps with another."""
|
||||
return not (self.end_seconds <= other.start_seconds or
|
||||
self.start_seconds >= other.end_seconds)
|
||||
|
||||
def merge(self, other: "MuteRegion") -> "MuteRegion":
|
||||
"""Merge this region with another overlapping region."""
|
||||
return MuteRegion(
|
||||
start_seconds=min(self.start_seconds, other.start_seconds),
|
||||
end_seconds=max(self.end_seconds, other.end_seconds)
|
||||
)
|
||||
|
||||
|
||||
class AnalysisResult(BaseModel):
|
||||
"""Result of analyzing backing vocals audio for audible content."""
|
||||
|
||||
has_audible_content: bool = Field(
|
||||
...,
|
||||
description="Whether any audible content was detected"
|
||||
)
|
||||
total_duration_seconds: float = Field(
|
||||
...,
|
||||
ge=0,
|
||||
description="Total duration of the audio file in seconds"
|
||||
)
|
||||
audible_segments: List[AudibleSegment] = Field(
|
||||
default_factory=list,
|
||||
description="List of detected audible segments"
|
||||
)
|
||||
recommended_selection: RecommendedSelection = Field(
|
||||
...,
|
||||
description="Recommended instrumental selection based on analysis"
|
||||
)
|
||||
silence_threshold_db: float = Field(
|
||||
default=-40.0,
|
||||
description="Threshold used to detect silence (in dB)"
|
||||
)
|
||||
total_audible_duration_seconds: float = Field(
|
||||
default=0.0,
|
||||
ge=0,
|
||||
description="Total duration of all audible segments combined"
|
||||
)
|
||||
audible_percentage: float = Field(
|
||||
default=0.0,
|
||||
ge=0,
|
||||
le=100,
|
||||
description="Percentage of audio that is audible"
|
||||
)
|
||||
waveform_data: Optional[List[float]] = Field(
|
||||
default=None,
|
||||
description="Amplitude envelope data for waveform rendering (optional)"
|
||||
)
|
||||
|
||||
@property
|
||||
def segment_count(self) -> int:
|
||||
"""Number of audible segments detected."""
|
||||
return len(self.audible_segments)
|
||||
|
||||
def get_segments_in_range(
|
||||
self,
|
||||
start_seconds: float,
|
||||
end_seconds: float
|
||||
) -> List[AudibleSegment]:
|
||||
"""Get segments that overlap with the given time range."""
|
||||
return [
|
||||
seg for seg in self.audible_segments
|
||||
if not (seg.end_seconds <= start_seconds or
|
||||
seg.start_seconds >= end_seconds)
|
||||
]
|
||||
|
||||
|
||||
class CustomInstrumentalResult(BaseModel):
|
||||
"""Result of creating a custom instrumental with muted regions."""
|
||||
|
||||
output_path: str = Field(
|
||||
...,
|
||||
description="Path to the created custom instrumental file"
|
||||
)
|
||||
mute_regions_applied: List[MuteRegion] = Field(
|
||||
default_factory=list,
|
||||
description="List of mute regions that were applied"
|
||||
)
|
||||
total_muted_duration_seconds: float = Field(
|
||||
default=0.0,
|
||||
ge=0,
|
||||
description="Total duration of muted audio in seconds"
|
||||
)
|
||||
output_duration_seconds: float = Field(
|
||||
default=0.0,
|
||||
ge=0,
|
||||
description="Duration of the output file in seconds"
|
||||
)
|
||||
@@ -0,0 +1,423 @@
|
||||
"""
|
||||
Local FastAPI server for instrumental review.
|
||||
|
||||
This module provides a local HTTP server that serves the instrumental review
|
||||
UI for local CLI usage. It provides the same API endpoints as the cloud backend
|
||||
to enable UI reuse.
|
||||
|
||||
Similar pattern to LyricsTranscriber's ReviewServer.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
import threading
|
||||
import webbrowser
|
||||
from typing import List, Optional
|
||||
|
||||
from fastapi import FastAPI, HTTPException, UploadFile, File
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, HTMLResponse
|
||||
from pydantic import BaseModel
|
||||
import shutil
|
||||
import tempfile
|
||||
import uvicorn
|
||||
|
||||
from pydub import AudioSegment
|
||||
|
||||
from karaoke_gen.instrumental_review import (
|
||||
AnalysisResult,
|
||||
AudioAnalyzer,
|
||||
AudioEditor,
|
||||
MuteRegion,
|
||||
WaveformGenerator,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Request/Response Models
|
||||
class MuteRegionRequest(BaseModel):
|
||||
start_seconds: float
|
||||
end_seconds: float
|
||||
|
||||
|
||||
class CreateCustomRequest(BaseModel):
|
||||
mute_regions: List[MuteRegionRequest]
|
||||
|
||||
|
||||
class SelectionRequest(BaseModel):
|
||||
selection: str
|
||||
|
||||
|
||||
class InstrumentalReviewServer:
|
||||
"""
|
||||
Local FastAPI server for instrumental review UI.
|
||||
|
||||
This server provides a web interface for reviewing and selecting
|
||||
instrumental tracks in the local CLI workflow. It serves the same
|
||||
API endpoints as the cloud backend to enable UI reuse.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
output_dir: str,
|
||||
base_name: str,
|
||||
analysis: AnalysisResult,
|
||||
waveform_path: str,
|
||||
backing_vocals_path: str,
|
||||
clean_instrumental_path: str,
|
||||
with_backing_path: Optional[str] = None,
|
||||
original_audio_path: Optional[str] = None,
|
||||
):
|
||||
"""
|
||||
Initialize the review server.
|
||||
|
||||
Args:
|
||||
output_dir: Directory containing the audio files
|
||||
base_name: Base name for output files (e.g., "Artist - Title")
|
||||
analysis: Analysis result from AudioAnalyzer
|
||||
waveform_path: Path to the waveform image
|
||||
backing_vocals_path: Path to the backing vocals audio file
|
||||
clean_instrumental_path: Path to the clean instrumental audio file
|
||||
with_backing_path: Path to the instrumental with backing vocals
|
||||
original_audio_path: Path to the original audio file (with vocals)
|
||||
"""
|
||||
self.output_dir = output_dir
|
||||
self.base_name = base_name
|
||||
self.analysis = analysis
|
||||
self.waveform_path = waveform_path
|
||||
self.backing_vocals_path = backing_vocals_path
|
||||
self.clean_instrumental_path = clean_instrumental_path
|
||||
self.with_backing_path = with_backing_path
|
||||
self.original_audio_path = original_audio_path
|
||||
self.custom_instrumental_path: Optional[str] = None
|
||||
self.uploaded_instrumental_path: Optional[str] = None
|
||||
self.selection: Optional[str] = None
|
||||
|
||||
self._app: Optional[FastAPI] = None
|
||||
self._server_thread: Optional[threading.Thread] = None
|
||||
self._selection_event = threading.Event()
|
||||
self._shutdown_event = threading.Event()
|
||||
|
||||
def _create_app(self) -> FastAPI:
|
||||
"""Create and configure the FastAPI application."""
|
||||
app = FastAPI(title="Instrumental Review", docs_url=None, redoc_url=None)
|
||||
|
||||
# Configure CORS
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# Register routes
|
||||
self._register_routes(app)
|
||||
|
||||
return app
|
||||
|
||||
def _register_routes(self, app: FastAPI) -> None:
|
||||
"""Register API routes."""
|
||||
|
||||
@app.get("/")
|
||||
async def serve_frontend():
|
||||
"""Serve the frontend HTML."""
|
||||
return HTMLResponse(content=self._get_frontend_html())
|
||||
|
||||
@app.get("/api/jobs/local/instrumental-analysis")
|
||||
async def get_analysis():
|
||||
"""Get analysis data for the instrumental review."""
|
||||
return {
|
||||
"job_id": "local",
|
||||
"artist": self.base_name.split(" - ")[0] if " - " in self.base_name else "",
|
||||
"title": self.base_name.split(" - ")[1] if " - " in self.base_name else self.base_name,
|
||||
"status": "awaiting_instrumental_selection",
|
||||
"analysis": {
|
||||
"has_audible_content": self.analysis.has_audible_content,
|
||||
"total_duration_seconds": self.analysis.total_duration_seconds,
|
||||
"audible_segments": [
|
||||
{
|
||||
"start_seconds": seg.start_seconds,
|
||||
"end_seconds": seg.end_seconds,
|
||||
"duration_seconds": seg.duration_seconds,
|
||||
"avg_amplitude_db": seg.avg_amplitude_db,
|
||||
"peak_amplitude_db": seg.peak_amplitude_db,
|
||||
}
|
||||
for seg in self.analysis.audible_segments
|
||||
],
|
||||
"recommended_selection": self.analysis.recommended_selection.value,
|
||||
"total_audible_duration_seconds": self.analysis.total_audible_duration_seconds,
|
||||
"audible_percentage": self.analysis.audible_percentage,
|
||||
"silence_threshold_db": self.analysis.silence_threshold_db,
|
||||
},
|
||||
"audio_urls": {
|
||||
"clean_instrumental": "/api/audio/clean_instrumental" if self.clean_instrumental_path else None,
|
||||
"backing_vocals": "/api/audio/backing_vocals" if self.backing_vocals_path else None,
|
||||
"with_backing": "/api/audio/with_backing" if self.with_backing_path else None,
|
||||
"custom_instrumental": "/api/audio/custom_instrumental" if self.custom_instrumental_path else None,
|
||||
"uploaded_instrumental": "/api/audio/uploaded_instrumental" if self.uploaded_instrumental_path else None,
|
||||
"original": "/api/audio/original" if self.original_audio_path else None,
|
||||
},
|
||||
"waveform_url": "/api/waveform" if self.waveform_path else None,
|
||||
"has_custom_instrumental": self.custom_instrumental_path is not None,
|
||||
"has_uploaded_instrumental": self.uploaded_instrumental_path is not None,
|
||||
"has_original": self.original_audio_path is not None,
|
||||
}
|
||||
|
||||
@app.get("/api/jobs/local/waveform-data")
|
||||
async def get_waveform_data(num_points: int = 600):
|
||||
"""Get waveform amplitude data for client-side rendering."""
|
||||
# Validate num_points parameter
|
||||
if num_points <= 0 or num_points > 10000:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="num_points must be between 1 and 10000"
|
||||
)
|
||||
|
||||
if not self.backing_vocals_path or not os.path.exists(self.backing_vocals_path):
|
||||
raise HTTPException(status_code=404, detail="Backing vocals file not found")
|
||||
|
||||
try:
|
||||
generator = WaveformGenerator()
|
||||
amplitudes, duration = generator.generate_data_only(self.backing_vocals_path, num_points)
|
||||
return {"amplitudes": amplitudes, "duration": duration}
|
||||
except Exception as e:
|
||||
logger.exception(f"Error generating waveform data: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e)) from e
|
||||
|
||||
@app.get("/api/audio/{stem_type}")
|
||||
async def stream_audio(stem_type: str):
|
||||
"""Stream audio file."""
|
||||
path_map = {
|
||||
"clean_instrumental": self.clean_instrumental_path,
|
||||
"backing_vocals": self.backing_vocals_path,
|
||||
"with_backing": self.with_backing_path,
|
||||
"custom_instrumental": self.custom_instrumental_path,
|
||||
"uploaded_instrumental": self.uploaded_instrumental_path,
|
||||
"original": self.original_audio_path,
|
||||
}
|
||||
|
||||
audio_path = path_map.get(stem_type)
|
||||
if not audio_path or not os.path.exists(audio_path):
|
||||
raise HTTPException(status_code=404, detail=f"Audio file not found: {stem_type}")
|
||||
|
||||
# Determine content type
|
||||
ext = os.path.splitext(audio_path)[1].lower()
|
||||
content_types = {
|
||||
".flac": "audio/flac",
|
||||
".mp3": "audio/mpeg",
|
||||
".wav": "audio/wav",
|
||||
}
|
||||
content_type = content_types.get(ext, "application/octet-stream")
|
||||
|
||||
return FileResponse(audio_path, media_type=content_type)
|
||||
|
||||
@app.get("/api/waveform")
|
||||
async def get_waveform_image():
|
||||
"""Serve waveform image."""
|
||||
if not self.waveform_path or not os.path.exists(self.waveform_path):
|
||||
raise HTTPException(status_code=404, detail="Waveform image not found")
|
||||
return FileResponse(self.waveform_path, media_type="image/png")
|
||||
|
||||
@app.post("/api/jobs/local/create-custom-instrumental")
|
||||
async def create_custom_instrumental(request: CreateCustomRequest):
|
||||
"""Create a custom instrumental with muted regions."""
|
||||
if not request.mute_regions:
|
||||
raise HTTPException(status_code=400, detail="No mute regions provided")
|
||||
|
||||
try:
|
||||
mute_regions = [
|
||||
MuteRegion(
|
||||
start_seconds=r.start_seconds,
|
||||
end_seconds=r.end_seconds,
|
||||
)
|
||||
for r in request.mute_regions
|
||||
]
|
||||
|
||||
editor = AudioEditor()
|
||||
output_path = os.path.join(
|
||||
self.output_dir,
|
||||
f"{self.base_name} (Instrumental Custom).flac"
|
||||
)
|
||||
|
||||
result = editor.create_custom_instrumental(
|
||||
clean_instrumental_path=self.clean_instrumental_path,
|
||||
backing_vocals_path=self.backing_vocals_path,
|
||||
mute_regions=mute_regions,
|
||||
output_path=output_path,
|
||||
)
|
||||
|
||||
self.custom_instrumental_path = result.output_path
|
||||
|
||||
return {
|
||||
"status": "success",
|
||||
"custom_instrumental_url": "/api/audio/custom_instrumental",
|
||||
"statistics": {
|
||||
"mute_regions_applied": len(result.mute_regions_applied),
|
||||
"total_muted_duration_seconds": result.total_muted_duration_seconds,
|
||||
"output_duration_seconds": result.output_duration_seconds,
|
||||
},
|
||||
}
|
||||
except Exception as e:
|
||||
logger.exception(f"Error creating custom instrumental: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e)) from e
|
||||
|
||||
@app.post("/api/jobs/local/upload-instrumental")
|
||||
async def upload_instrumental(file: UploadFile = File(...)):
|
||||
"""Upload a custom instrumental audio file."""
|
||||
# Validate file type
|
||||
allowed_extensions = {".flac", ".mp3", ".wav", ".m4a", ".ogg"}
|
||||
ext = os.path.splitext(file.filename or "")[1].lower()
|
||||
if ext not in allowed_extensions:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid file type. Allowed: {', '.join(allowed_extensions)}"
|
||||
)
|
||||
|
||||
tmp_path = None
|
||||
file_moved = False
|
||||
try:
|
||||
# Save to temp file first to validate
|
||||
with tempfile.NamedTemporaryFile(delete=False, suffix=ext) as tmp:
|
||||
shutil.copyfileobj(file.file, tmp)
|
||||
tmp_path = tmp.name
|
||||
|
||||
# Load and check duration
|
||||
uploaded_audio = AudioSegment.from_file(tmp_path)
|
||||
uploaded_duration = len(uploaded_audio) / 1000.0 # ms to seconds
|
||||
|
||||
expected_duration = self.analysis.total_duration_seconds
|
||||
duration_diff = abs(uploaded_duration - expected_duration)
|
||||
|
||||
if duration_diff > 0.5:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Duration mismatch: uploaded file is {uploaded_duration:.2f}s, "
|
||||
f"expected {expected_duration:.2f}s (±0.5s allowed)"
|
||||
)
|
||||
|
||||
# Move to final location
|
||||
output_path = os.path.join(
|
||||
self.output_dir,
|
||||
f"{self.base_name} (Instrumental Uploaded){ext}"
|
||||
)
|
||||
shutil.move(tmp_path, output_path)
|
||||
file_moved = True
|
||||
self.uploaded_instrumental_path = output_path
|
||||
|
||||
return {
|
||||
"status": "success",
|
||||
"uploaded_instrumental_url": "/api/audio/uploaded_instrumental",
|
||||
"duration_seconds": uploaded_duration,
|
||||
"filename": file.filename,
|
||||
}
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.exception(f"Error uploading instrumental: {e}")
|
||||
raise HTTPException(status_code=500, detail=str(e)) from e
|
||||
finally:
|
||||
# Clean up temp file if it wasn't moved
|
||||
if tmp_path and not file_moved and os.path.exists(tmp_path):
|
||||
try:
|
||||
os.unlink(tmp_path)
|
||||
except OSError:
|
||||
pass # Best effort cleanup
|
||||
|
||||
@app.post("/api/jobs/local/select-instrumental")
|
||||
async def select_instrumental(request: SelectionRequest):
|
||||
"""Submit instrumental selection."""
|
||||
if request.selection not in ("clean", "with_backing", "custom", "uploaded", "original"):
|
||||
raise HTTPException(status_code=400, detail=f"Invalid selection: {request.selection}")
|
||||
|
||||
self.selection = request.selection
|
||||
self._selection_event.set()
|
||||
|
||||
return {"status": "success", "selection": request.selection}
|
||||
|
||||
@staticmethod
|
||||
def _get_static_dir() -> Path:
|
||||
"""Get the path to the static assets directory."""
|
||||
return Path(__file__).parent / "static"
|
||||
|
||||
def _get_frontend_html(self) -> str:
|
||||
"""Return the frontend HTML by reading from the static file."""
|
||||
static_file = self._get_static_dir() / "index.html"
|
||||
if static_file.exists():
|
||||
return static_file.read_text(encoding="utf-8")
|
||||
else:
|
||||
# Fallback error message if file is missing
|
||||
return """<!DOCTYPE html>
|
||||
<html>
|
||||
<head><title>Error</title></head>
|
||||
<body style="background:#1a1a1a;color:#fff;font-family:sans-serif;padding:2rem;">
|
||||
<h1>Frontend assets not found</h1>
|
||||
<p>The static/index.html file is missing from the instrumental_review module.</p>
|
||||
</body>
|
||||
</html>"""
|
||||
|
||||
def start_and_open_browser(self, port: int = 8765) -> str:
|
||||
"""
|
||||
Start server, open browser, and block until selection is submitted.
|
||||
|
||||
Args:
|
||||
port: Port to run the server on
|
||||
|
||||
Returns:
|
||||
The user's selection ("clean", "with_backing", or "custom")
|
||||
"""
|
||||
self._app = self._create_app()
|
||||
|
||||
# Run uvicorn in a separate thread
|
||||
config = uvicorn.Config(
|
||||
self._app,
|
||||
host="127.0.0.1",
|
||||
port=port,
|
||||
log_level="warning",
|
||||
)
|
||||
server = uvicorn.Server(config)
|
||||
|
||||
def run_server():
|
||||
server.run()
|
||||
|
||||
self._server_thread = threading.Thread(target=run_server, daemon=True)
|
||||
self._server_thread.start()
|
||||
|
||||
# Wait a moment for server to start
|
||||
import time
|
||||
time.sleep(0.5)
|
||||
|
||||
url = f"http://localhost:{port}/"
|
||||
logger.info(f"Instrumental review server started at {url}")
|
||||
|
||||
# Open browser
|
||||
webbrowser.open(url)
|
||||
|
||||
# Wait for selection
|
||||
logger.info("Waiting for instrumental selection...")
|
||||
self._selection_event.wait()
|
||||
|
||||
# Give a moment for response to be sent
|
||||
time.sleep(0.5)
|
||||
|
||||
return self.get_selection()
|
||||
|
||||
def stop(self) -> None:
|
||||
"""Stop the server."""
|
||||
self._shutdown_event.set()
|
||||
|
||||
def get_selection(self) -> str:
|
||||
"""Get the user's selection."""
|
||||
if self.selection is None:
|
||||
raise ValueError("No selection has been made")
|
||||
return self.selection
|
||||
|
||||
def get_custom_instrumental_path(self) -> Optional[str]:
|
||||
"""Get the path to the custom instrumental if one was created."""
|
||||
return self.custom_instrumental_path
|
||||
|
||||
def get_uploaded_instrumental_path(self) -> Optional[str]:
|
||||
"""Get the path to the uploaded instrumental if one was uploaded."""
|
||||
return self.uploaded_instrumental_path
|
||||
@@ -0,0 +1,409 @@
|
||||
"""
|
||||
Waveform visualization generator for audio files.
|
||||
|
||||
This module provides the WaveformGenerator class which creates waveform
|
||||
images suitable for display in the instrumental review UI.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
import matplotlib
|
||||
matplotlib.use('Agg') # Use non-interactive backend
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
from pydub import AudioSegment
|
||||
|
||||
from .models import AudibleSegment, MuteRegion
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class WaveformGenerator:
|
||||
"""
|
||||
Generates waveform visualization images from audio files.
|
||||
|
||||
This class creates PNG images showing the amplitude envelope of an
|
||||
audio file over time. It can highlight detected audible segments
|
||||
and mute regions with different colors.
|
||||
|
||||
The generated images are suitable for display in web UIs and can
|
||||
be used for interactive seeking (click-to-seek) functionality.
|
||||
|
||||
Attributes:
|
||||
width: Width of the output image in pixels (default: 1200)
|
||||
height: Height of the output image in pixels (default: 200)
|
||||
background_color: Background color (default: "#1a1a2e")
|
||||
waveform_color: Main waveform color (default: "#4a90d9")
|
||||
segment_color: Color for audible segments (default: "#e94560")
|
||||
mute_color: Color for mute regions (default: "#ff6b6b")
|
||||
time_axis_color: Color for time axis (default: "#ffffff")
|
||||
|
||||
Example:
|
||||
>>> generator = WaveformGenerator(width=1200, height=200)
|
||||
>>> generator.generate(
|
||||
... audio_path="/path/to/backing_vocals.flac",
|
||||
... output_path="/path/to/waveform.png",
|
||||
... segments=analysis_result.audible_segments
|
||||
... )
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
width: int = 1200,
|
||||
height: int = 200,
|
||||
background_color: str = "#1a1a2e",
|
||||
waveform_color: str = "#4a90d9",
|
||||
segment_color: str = "#e94560",
|
||||
mute_color: str = "#ff6b6b",
|
||||
time_axis_color: str = "#ffffff",
|
||||
dpi: int = 100,
|
||||
):
|
||||
"""
|
||||
Initialize the waveform generator.
|
||||
|
||||
Args:
|
||||
width: Width of the output image in pixels
|
||||
height: Height of the output image in pixels
|
||||
background_color: Background color (hex or named color)
|
||||
waveform_color: Main waveform color
|
||||
segment_color: Color for highlighting audible segments
|
||||
mute_color: Color for highlighting mute regions
|
||||
time_axis_color: Color for time axis labels
|
||||
dpi: DPI for the output image
|
||||
"""
|
||||
self.width = width
|
||||
self.height = height
|
||||
self.background_color = background_color
|
||||
self.waveform_color = waveform_color
|
||||
self.segment_color = segment_color
|
||||
self.mute_color = mute_color
|
||||
self.time_axis_color = time_axis_color
|
||||
self.dpi = dpi
|
||||
|
||||
def generate(
|
||||
self,
|
||||
audio_path: str,
|
||||
output_path: str,
|
||||
segments: Optional[List[AudibleSegment]] = None,
|
||||
mute_regions: Optional[List[MuteRegion]] = None,
|
||||
show_time_axis: bool = True,
|
||||
silence_threshold_db: float = -40.0,
|
||||
) -> str:
|
||||
"""
|
||||
Generate a waveform image from an audio file.
|
||||
|
||||
Args:
|
||||
audio_path: Path to the audio file
|
||||
output_path: Path where the PNG image will be saved
|
||||
segments: Optional list of audible segments to highlight
|
||||
mute_regions: Optional list of mute regions to highlight
|
||||
show_time_axis: Whether to show time axis labels
|
||||
silence_threshold_db: Threshold for visual reference line
|
||||
|
||||
Returns:
|
||||
Path to the generated image file
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If the audio file doesn't exist
|
||||
"""
|
||||
path = Path(audio_path)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Audio file not found: {audio_path}")
|
||||
|
||||
logger.info(f"Generating waveform for: {audio_path}")
|
||||
|
||||
# Load audio
|
||||
audio = AudioSegment.from_file(audio_path)
|
||||
duration_seconds = len(audio) / 1000.0
|
||||
|
||||
# Convert to mono if needed
|
||||
if audio.channels > 1:
|
||||
audio = audio.set_channels(1)
|
||||
|
||||
# Get amplitude envelope
|
||||
envelope = self._get_envelope(audio)
|
||||
|
||||
# Create the figure
|
||||
fig, ax = self._create_figure(duration_seconds, show_time_axis)
|
||||
|
||||
# Draw waveform
|
||||
self._draw_waveform(ax, envelope, duration_seconds)
|
||||
|
||||
# Highlight mute regions (if any) - draw first so waveform is on top
|
||||
if mute_regions:
|
||||
self._draw_mute_regions(ax, mute_regions, duration_seconds)
|
||||
|
||||
# Highlight audible segments (if any)
|
||||
if segments:
|
||||
self._draw_segments(ax, segments, envelope, duration_seconds)
|
||||
|
||||
# Draw silence threshold reference line
|
||||
self._draw_threshold_line(ax, silence_threshold_db, duration_seconds)
|
||||
|
||||
# Save the figure
|
||||
output_dir = Path(output_path).parent
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
fig.savefig(
|
||||
output_path,
|
||||
facecolor=self.background_color,
|
||||
edgecolor='none',
|
||||
bbox_inches='tight',
|
||||
pad_inches=0.1,
|
||||
)
|
||||
plt.close(fig)
|
||||
|
||||
logger.info(f"Waveform saved to: {output_path}")
|
||||
return output_path
|
||||
|
||||
def generate_data_only(
|
||||
self,
|
||||
audio_path: str,
|
||||
num_points: int = 500,
|
||||
) -> Tuple[List[float], float]:
|
||||
"""
|
||||
Generate waveform data without creating an image.
|
||||
|
||||
This is useful for sending data to a frontend that will
|
||||
render the waveform itself (e.g., using Canvas or SVG).
|
||||
|
||||
Args:
|
||||
audio_path: Path to the audio file
|
||||
num_points: Number of data points to return
|
||||
|
||||
Returns:
|
||||
Tuple of (amplitude_values, duration_seconds)
|
||||
Amplitude values are normalized to 0.0-1.0 range.
|
||||
"""
|
||||
path = Path(audio_path)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Audio file not found: {audio_path}")
|
||||
|
||||
audio = AudioSegment.from_file(audio_path)
|
||||
duration_seconds = len(audio) / 1000.0
|
||||
|
||||
if audio.channels > 1:
|
||||
audio = audio.set_channels(1)
|
||||
|
||||
# Calculate window size to get desired number of points
|
||||
duration_ms = len(audio)
|
||||
window_ms = max(1, duration_ms // num_points)
|
||||
|
||||
amplitudes = []
|
||||
for start_ms in range(0, duration_ms, window_ms):
|
||||
end_ms = min(start_ms + window_ms, duration_ms)
|
||||
window = audio[start_ms:end_ms]
|
||||
|
||||
if window.rms > 0:
|
||||
db = 20 * math.log10(window.rms / window.max_possible_amplitude)
|
||||
else:
|
||||
db = -100.0
|
||||
|
||||
# Normalize to 0-1 range (mapping -60dB to 0dB -> 0 to 1)
|
||||
normalized = max(0.0, min(1.0, (db + 60) / 60))
|
||||
amplitudes.append(normalized)
|
||||
|
||||
return amplitudes, duration_seconds
|
||||
|
||||
def _get_envelope(
|
||||
self,
|
||||
audio: AudioSegment,
|
||||
window_ms: int = 50,
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Extract amplitude envelope from audio.
|
||||
|
||||
Returns array of amplitude values in dB.
|
||||
"""
|
||||
duration_ms = len(audio)
|
||||
amplitudes = []
|
||||
|
||||
for start_ms in range(0, duration_ms, window_ms):
|
||||
end_ms = min(start_ms + window_ms, duration_ms)
|
||||
window = audio[start_ms:end_ms]
|
||||
|
||||
if window.rms > 0:
|
||||
db = 20 * math.log10(window.rms / window.max_possible_amplitude)
|
||||
else:
|
||||
db = -100.0
|
||||
|
||||
amplitudes.append(db)
|
||||
|
||||
return np.array(amplitudes)
|
||||
|
||||
def _create_figure(
|
||||
self,
|
||||
duration_seconds: float,
|
||||
show_time_axis: bool,
|
||||
) -> Tuple[plt.Figure, plt.Axes]:
|
||||
"""
|
||||
Create matplotlib figure and axes.
|
||||
"""
|
||||
fig_width = self.width / self.dpi
|
||||
fig_height = self.height / self.dpi
|
||||
|
||||
fig, ax = plt.subplots(figsize=(fig_width, fig_height), dpi=self.dpi)
|
||||
|
||||
# Set background
|
||||
fig.patch.set_facecolor(self.background_color)
|
||||
ax.set_facecolor(self.background_color)
|
||||
|
||||
# Configure axes
|
||||
ax.set_xlim(0, duration_seconds)
|
||||
ax.set_ylim(-60, 0) # dB range
|
||||
|
||||
# Remove spines
|
||||
for spine in ax.spines.values():
|
||||
spine.set_visible(False)
|
||||
|
||||
# Configure ticks
|
||||
if show_time_axis:
|
||||
ax.tick_params(
|
||||
axis='x',
|
||||
colors=self.time_axis_color,
|
||||
labelsize=8,
|
||||
)
|
||||
ax.tick_params(axis='y', left=False, labelleft=False)
|
||||
|
||||
# Set time axis ticks
|
||||
self._set_time_ticks(ax, duration_seconds)
|
||||
else:
|
||||
ax.tick_params(
|
||||
axis='both',
|
||||
left=False,
|
||||
bottom=False,
|
||||
labelleft=False,
|
||||
labelbottom=False,
|
||||
)
|
||||
|
||||
return fig, ax
|
||||
|
||||
def _set_time_ticks(self, ax: plt.Axes, duration_seconds: float):
|
||||
"""
|
||||
Set appropriate time axis tick marks.
|
||||
"""
|
||||
if duration_seconds <= 60:
|
||||
# Under 1 minute: tick every 10 seconds
|
||||
tick_interval = 10
|
||||
elif duration_seconds <= 300:
|
||||
# Under 5 minutes: tick every 30 seconds
|
||||
tick_interval = 30
|
||||
else:
|
||||
# Over 5 minutes: tick every minute
|
||||
tick_interval = 60
|
||||
|
||||
ticks = np.arange(0, duration_seconds + 1, tick_interval)
|
||||
ax.set_xticks(ticks)
|
||||
|
||||
# Format tick labels as MM:SS
|
||||
labels = []
|
||||
for t in ticks:
|
||||
minutes = int(t // 60)
|
||||
seconds = int(t % 60)
|
||||
labels.append(f"{minutes}:{seconds:02d}")
|
||||
ax.set_xticklabels(labels)
|
||||
|
||||
def _draw_waveform(
|
||||
self,
|
||||
ax: plt.Axes,
|
||||
envelope: np.ndarray,
|
||||
duration_seconds: float,
|
||||
):
|
||||
"""
|
||||
Draw the main waveform.
|
||||
"""
|
||||
num_points = len(envelope)
|
||||
time_points = np.linspace(0, duration_seconds, num_points)
|
||||
|
||||
# Draw as filled area
|
||||
ax.fill_between(
|
||||
time_points,
|
||||
envelope,
|
||||
-60, # Bottom of range
|
||||
color=self.waveform_color,
|
||||
alpha=0.7,
|
||||
)
|
||||
|
||||
# Draw outline
|
||||
ax.plot(
|
||||
time_points,
|
||||
envelope,
|
||||
color=self.waveform_color,
|
||||
linewidth=0.5,
|
||||
alpha=0.9,
|
||||
)
|
||||
|
||||
def _draw_segments(
|
||||
self,
|
||||
ax: plt.Axes,
|
||||
segments: List[AudibleSegment],
|
||||
envelope: np.ndarray,
|
||||
duration_seconds: float,
|
||||
):
|
||||
"""
|
||||
Highlight audible segments on the waveform.
|
||||
"""
|
||||
num_points = len(envelope)
|
||||
time_points = np.linspace(0, duration_seconds, num_points)
|
||||
|
||||
for segment in segments:
|
||||
# Find indices corresponding to this segment
|
||||
start_idx = int(segment.start_seconds / duration_seconds * num_points)
|
||||
end_idx = int(segment.end_seconds / duration_seconds * num_points)
|
||||
|
||||
start_idx = max(0, min(start_idx, num_points - 1))
|
||||
end_idx = max(0, min(end_idx, num_points))
|
||||
|
||||
if start_idx >= end_idx:
|
||||
continue
|
||||
|
||||
segment_time = time_points[start_idx:end_idx]
|
||||
segment_envelope = envelope[start_idx:end_idx]
|
||||
|
||||
# Highlight this segment with a different color
|
||||
ax.fill_between(
|
||||
segment_time,
|
||||
segment_envelope,
|
||||
-60,
|
||||
color=self.segment_color,
|
||||
alpha=0.6,
|
||||
)
|
||||
|
||||
def _draw_mute_regions(
|
||||
self,
|
||||
ax: plt.Axes,
|
||||
mute_regions: List[MuteRegion],
|
||||
duration_seconds: float,
|
||||
):
|
||||
"""
|
||||
Draw mute region overlays.
|
||||
"""
|
||||
for region in mute_regions:
|
||||
ax.axvspan(
|
||||
region.start_seconds,
|
||||
region.end_seconds,
|
||||
color=self.mute_color,
|
||||
alpha=0.3,
|
||||
zorder=0,
|
||||
)
|
||||
|
||||
def _draw_threshold_line(
|
||||
self,
|
||||
ax: plt.Axes,
|
||||
threshold_db: float,
|
||||
duration_seconds: float,
|
||||
):
|
||||
"""
|
||||
Draw a reference line at the silence threshold.
|
||||
"""
|
||||
ax.axhline(
|
||||
y=threshold_db,
|
||||
color=self.time_axis_color,
|
||||
linestyle='--',
|
||||
linewidth=0.5,
|
||||
alpha=0.3,
|
||||
)
|
||||
@@ -47,6 +47,7 @@ class KaraokeFinalise:
|
||||
user_youtube_credentials=None, # Add support for pre-stored credentials
|
||||
server_side_mode=False, # New parameter for server-side deployment
|
||||
selected_instrumental_file=None, # Add support for pre-selected instrumental file
|
||||
countdown_padding_seconds=None, # Padding applied to vocals; instrumental must match
|
||||
):
|
||||
self.log_level = log_level
|
||||
self.log_formatter = log_formatter
|
||||
@@ -54,6 +55,9 @@ class KaraokeFinalise:
|
||||
if logger is None:
|
||||
self.logger = logging.getLogger(__name__)
|
||||
self.logger.setLevel(log_level)
|
||||
# Prevent log propagation to root logger to avoid duplicate logs
|
||||
# when external packages (like lyrics_converter) configure root logger handlers
|
||||
self.logger.propagate = False
|
||||
|
||||
self.log_handler = logging.StreamHandler()
|
||||
|
||||
@@ -105,6 +109,7 @@ class KaraokeFinalise:
|
||||
self.user_youtube_credentials = user_youtube_credentials
|
||||
self.server_side_mode = server_side_mode
|
||||
self.selected_instrumental_file = selected_instrumental_file
|
||||
self.countdown_padding_seconds = countdown_padding_seconds
|
||||
|
||||
self.suffixes = {
|
||||
"title_mov": " (Title).mov",
|
||||
@@ -421,6 +426,15 @@ class KaraokeFinalise:
|
||||
# Check if any videos were found
|
||||
if "items" in response and len(response["items"]) > 0:
|
||||
for item in response["items"]:
|
||||
# YouTube search API sometimes returns results from other channels even with channelId filter
|
||||
# Verify the video actually belongs to our channel
|
||||
result_channel_id = item["snippet"]["channelId"]
|
||||
if result_channel_id != channel_id:
|
||||
self.logger.debug(
|
||||
f"Skipping video from different channel: {item['snippet']['title']} (channel: {result_channel_id})"
|
||||
)
|
||||
continue
|
||||
|
||||
found_title = item["snippet"]["title"]
|
||||
|
||||
# In server-side mode, require an exact match to avoid false positives.
|
||||
@@ -720,6 +734,32 @@ class KaraokeFinalise:
|
||||
artist, title = base_name.split(" - ", 1)
|
||||
return base_name, artist, title
|
||||
|
||||
def _pad_audio_file(self, input_audio, output_audio, padding_seconds):
|
||||
"""
|
||||
Pad an audio file by prepending silence at the beginning.
|
||||
|
||||
Uses the same ffmpeg approach as LyricsTranscriber's CountdownProcessor
|
||||
to ensure consistent padding behavior.
|
||||
|
||||
Args:
|
||||
input_audio: Path to input audio file
|
||||
output_audio: Path for the padded output file
|
||||
padding_seconds: Amount of silence to prepend (in seconds)
|
||||
"""
|
||||
self.logger.info(f"Padding audio file with {padding_seconds}s of silence")
|
||||
|
||||
# Use ffmpeg to prepend silence - this matches the approach in audio_processor.py
|
||||
# adelay filter adds delay in milliseconds
|
||||
delay_ms = int(padding_seconds * 1000)
|
||||
|
||||
ffmpeg_command = (
|
||||
f'{self.ffmpeg_base_command} -i "{input_audio}" '
|
||||
f'-af "adelay={delay_ms}|{delay_ms}" '
|
||||
f'"{output_audio}"'
|
||||
)
|
||||
|
||||
self.execute_command(ffmpeg_command, f"Padding audio with {padding_seconds}s silence")
|
||||
|
||||
def execute_command(self, command, description):
|
||||
"""Execute a shell command and log the output. For general commands (rclone, etc.)"""
|
||||
self.logger.info(f"{description}")
|
||||
@@ -764,11 +804,32 @@ class KaraokeFinalise:
|
||||
|
||||
def remux_with_instrumental(self, with_vocals_file, instrumental_audio, output_file):
|
||||
"""Remux the video with instrumental audio to create karaoke version"""
|
||||
# Safety net: If countdown padding was applied to vocals, ensure instrumental is padded too
|
||||
actual_instrumental = instrumental_audio
|
||||
if self.countdown_padding_seconds and self.countdown_padding_seconds > 0:
|
||||
# Check if the instrumental file is already padded (has "(Padded)" in name)
|
||||
if "(Padded)" not in instrumental_audio:
|
||||
self.logger.warning(
|
||||
f"Countdown padding ({self.countdown_padding_seconds}s) was applied to vocals, "
|
||||
f"but instrumental doesn't appear to be padded. Creating padded version..."
|
||||
)
|
||||
# Create a padded version of the instrumental
|
||||
base, ext = os.path.splitext(instrumental_audio)
|
||||
padded_instrumental = f"{base} (Padded){ext}"
|
||||
|
||||
if not os.path.exists(padded_instrumental):
|
||||
self._pad_audio_file(instrumental_audio, padded_instrumental, self.countdown_padding_seconds)
|
||||
self.logger.info(f"Created padded instrumental: {padded_instrumental}")
|
||||
|
||||
actual_instrumental = padded_instrumental
|
||||
else:
|
||||
self.logger.info(f"Using already-padded instrumental: {instrumental_audio}")
|
||||
|
||||
# This operation is primarily I/O bound (remuxing), so hardware acceleration doesn't provide significant benefit
|
||||
# Keep the existing approach but use the new execute method
|
||||
ffmpeg_command = (
|
||||
f'{self.ffmpeg_base_command} -an -i "{with_vocals_file}" '
|
||||
f'-vn -i "{instrumental_audio}" -c:v copy -c:a pcm_s16le "{output_file}"'
|
||||
f'-vn -i "{actual_instrumental}" -c:v copy -c:a pcm_s16le "{output_file}"'
|
||||
)
|
||||
self.execute_command(ffmpeg_command, "Remuxing video with instrumental audio")
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ from .audio_processor import AudioProcessor
|
||||
from .lyrics_processor import LyricsProcessor
|
||||
from .video_generator import VideoGenerator
|
||||
from .video_background_processor import VideoBackgroundProcessor
|
||||
from .audio_fetcher import create_audio_fetcher, AudioFetcherError, NoResultsError
|
||||
from .audio_fetcher import create_audio_fetcher, AudioFetcherError, NoResultsError, UserCancelledError
|
||||
|
||||
|
||||
class KaraokePrep:
|
||||
@@ -84,6 +84,9 @@ class KaraokePrep:
|
||||
if logger is None:
|
||||
self.logger = logging.getLogger(__name__)
|
||||
self.logger.setLevel(log_level)
|
||||
# Prevent log propagation to root logger to avoid duplicate logs
|
||||
# when external packages (like lyrics_converter) configure root logger handlers
|
||||
self.logger.propagate = False
|
||||
|
||||
self.log_handler = logging.StreamHandler()
|
||||
|
||||
@@ -256,6 +259,101 @@ class KaraokePrep:
|
||||
self.artist = metadata_result["artist"]
|
||||
self.title = metadata_result["title"]
|
||||
|
||||
def _scan_directory_for_instrumentals(self, track_output_dir, artist_title):
|
||||
"""
|
||||
Scan the directory for existing instrumental files and build a separated_audio structure.
|
||||
|
||||
This is used when transcription was skipped (existing files found) but we need to
|
||||
pad instrumentals due to countdown padding.
|
||||
|
||||
Args:
|
||||
track_output_dir: The track output directory to scan
|
||||
artist_title: The "{artist} - {title}" string for matching files
|
||||
|
||||
Returns:
|
||||
Dictionary with separated_audio structure containing found instrumental paths
|
||||
"""
|
||||
self.logger.info(f"Scanning directory for existing instrumentals: {track_output_dir}")
|
||||
|
||||
separated_audio = {
|
||||
"clean_instrumental": {},
|
||||
"backing_vocals": {},
|
||||
"other_stems": {},
|
||||
"combined_instrumentals": {},
|
||||
}
|
||||
|
||||
# Search patterns for instrumental files
|
||||
# Files are named like: "{artist} - {title} (Instrumental {model}).flac"
|
||||
# Or with backing vocals: "{artist} - {title} (Instrumental +BV {model}).flac"
|
||||
|
||||
# Look for files in the track output directory
|
||||
search_dir = track_output_dir
|
||||
|
||||
# Find all instrumental files (not padded ones - we want the originals)
|
||||
instrumental_pattern = os.path.join(search_dir, f"{artist_title} (Instrumental*.flac")
|
||||
instrumental_files = glob.glob(instrumental_pattern)
|
||||
|
||||
# Also check for wav files
|
||||
instrumental_pattern_wav = os.path.join(search_dir, f"{artist_title} (Instrumental*.wav")
|
||||
instrumental_files.extend(glob.glob(instrumental_pattern_wav))
|
||||
|
||||
self.logger.debug(f"Found {len(instrumental_files)} instrumental files")
|
||||
|
||||
for filepath in instrumental_files:
|
||||
filename = os.path.basename(filepath)
|
||||
|
||||
# Skip already padded files
|
||||
if "(Padded)" in filename:
|
||||
self.logger.debug(f"Skipping already padded file: {filename}")
|
||||
continue
|
||||
|
||||
# Determine if it's a combined instrumental (+BV) or clean instrumental
|
||||
if "+BV" in filename or "+bv" in filename.lower():
|
||||
# Combined instrumental with backing vocals
|
||||
# Extract model name from filename
|
||||
# Pattern: "(Instrumental +BV {model}).flac"
|
||||
model_match = re.search(r'\(Instrumental \+BV ([^)]+)\)', filename)
|
||||
if model_match:
|
||||
model_name = model_match.group(1).strip()
|
||||
separated_audio["combined_instrumentals"][model_name] = filepath
|
||||
self.logger.info(f"Found combined instrumental: {filename}")
|
||||
else:
|
||||
# Clean instrumental (no backing vocals)
|
||||
# Pattern: "(Instrumental {model}).flac"
|
||||
model_match = re.search(r'\(Instrumental ([^)]+)\)', filename)
|
||||
if model_match:
|
||||
# Use as clean instrumental if we don't have one yet
|
||||
if not separated_audio["clean_instrumental"].get("instrumental"):
|
||||
separated_audio["clean_instrumental"]["instrumental"] = filepath
|
||||
self.logger.info(f"Found clean instrumental: {filename}")
|
||||
else:
|
||||
# Additional clean instrumentals go to combined_instrumentals for padding
|
||||
model_name = model_match.group(1).strip()
|
||||
separated_audio["combined_instrumentals"][model_name] = filepath
|
||||
self.logger.info(f"Found additional instrumental: {filename}")
|
||||
|
||||
# Also look for backing vocals files
|
||||
backing_vocals_pattern = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.flac")
|
||||
backing_vocals_files = glob.glob(backing_vocals_pattern)
|
||||
backing_vocals_pattern_wav = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.wav")
|
||||
backing_vocals_files.extend(glob.glob(backing_vocals_pattern_wav))
|
||||
|
||||
for filepath in backing_vocals_files:
|
||||
filename = os.path.basename(filepath)
|
||||
model_match = re.search(r'\(Backing Vocals ([^)]+)\)', filename)
|
||||
if model_match:
|
||||
model_name = model_match.group(1).strip()
|
||||
if model_name not in separated_audio["backing_vocals"]:
|
||||
separated_audio["backing_vocals"][model_name] = {"backing_vocals": filepath}
|
||||
self.logger.info(f"Found backing vocals: {filename}")
|
||||
|
||||
# Log summary
|
||||
clean_count = 1 if separated_audio["clean_instrumental"].get("instrumental") else 0
|
||||
combined_count = len(separated_audio["combined_instrumentals"])
|
||||
self.logger.info(f"Directory scan complete: {clean_count} clean instrumental, {combined_count} combined instrumentals")
|
||||
|
||||
return separated_audio
|
||||
|
||||
async def prep_single_track(self):
|
||||
# Add signal handler at the start
|
||||
loop = asyncio.get_running_loop()
|
||||
@@ -419,6 +517,9 @@ class KaraokePrep:
|
||||
# No still image for audio-only downloads
|
||||
processed_track["input_still_image"] = None
|
||||
|
||||
except UserCancelledError:
|
||||
# User cancelled - propagate up to CLI for graceful exit
|
||||
raise
|
||||
except NoResultsError as e:
|
||||
self.logger.error(f"No audio found: {e}")
|
||||
return None
|
||||
@@ -761,6 +862,18 @@ class KaraokePrep:
|
||||
f"Applying {padding_seconds}s padding to all instrumental files to sync with vocal countdown"
|
||||
)
|
||||
|
||||
# If separated_audio is empty (e.g., transcription was skipped but existing files have countdown),
|
||||
# scan the directory for existing instrumental files
|
||||
has_instrumentals = (
|
||||
processed_track["separated_audio"].get("clean_instrumental", {}).get("instrumental") or
|
||||
processed_track["separated_audio"].get("combined_instrumentals")
|
||||
)
|
||||
if not has_instrumentals:
|
||||
self.logger.info("No instrumentals in separated_audio, scanning directory for existing files...")
|
||||
processed_track["separated_audio"] = self._scan_directory_for_instrumentals(
|
||||
track_output_dir, artist_title
|
||||
)
|
||||
|
||||
# Apply padding using AudioProcessor
|
||||
padded_separation_result = self.audio_processor.apply_countdown_padding_to_instrumentals(
|
||||
separation_result=processed_track["separated_audio"],
|
||||
|
||||
@@ -11,6 +11,9 @@ from .utils import sanitize_filename
|
||||
|
||||
# Placeholder class or functions for lyrics processing
|
||||
class LyricsProcessor:
|
||||
# Standard countdown padding duration used by LyricsTranscriber
|
||||
COUNTDOWN_PADDING_SECONDS = 3.0
|
||||
|
||||
def __init__(
|
||||
self, logger, style_params_json, lyrics_file, skip_transcription, skip_transcription_review, render_video, subtitle_offset_ms
|
||||
):
|
||||
@@ -22,6 +25,60 @@ class LyricsProcessor:
|
||||
self.render_video = render_video
|
||||
self.subtitle_offset_ms = subtitle_offset_ms
|
||||
|
||||
def _detect_countdown_padding_from_lrc(self, lrc_filepath):
|
||||
"""
|
||||
Detect if countdown padding was applied by checking the first lyric timestamp in the LRC file.
|
||||
|
||||
LRC format timestamps look like: [mm:ss.xx] or [mm:ss.xxx]
|
||||
If the first lyric timestamp is >= 3.0 seconds, countdown padding was likely applied.
|
||||
|
||||
Args:
|
||||
lrc_filepath: Path to the LRC file
|
||||
|
||||
Returns:
|
||||
Tuple of (countdown_padding_added: bool, countdown_padding_seconds: float)
|
||||
"""
|
||||
try:
|
||||
with open(lrc_filepath, 'r', encoding='utf-8') as f:
|
||||
content = f.read()
|
||||
|
||||
# Find all timestamp patterns in the LRC file
|
||||
# LRC timestamps: [mm:ss.xx] or [mm:ss.xxx]
|
||||
timestamp_pattern = r'\[(\d{1,2}):(\d{2})\.(\d{2,3})\]'
|
||||
matches = re.findall(timestamp_pattern, content)
|
||||
|
||||
if not matches:
|
||||
self.logger.debug("No timestamps found in LRC file")
|
||||
return (False, 0.0)
|
||||
|
||||
# Find the first non-metadata timestamp (metadata like [ar:Artist] doesn't have decimal)
|
||||
# We already filtered for decimal timestamps in our pattern
|
||||
first_timestamp = matches[0]
|
||||
minutes = int(first_timestamp[0])
|
||||
seconds = int(first_timestamp[1])
|
||||
# Handle both .xx and .xxx formats
|
||||
centiseconds = first_timestamp[2]
|
||||
if len(centiseconds) == 2:
|
||||
milliseconds = int(centiseconds) * 10
|
||||
else:
|
||||
milliseconds = int(centiseconds)
|
||||
|
||||
first_lyric_time = minutes * 60 + seconds + milliseconds / 1000.0
|
||||
|
||||
self.logger.debug(f"First lyric timestamp in LRC: {first_lyric_time:.3f}s")
|
||||
|
||||
# If first lyric is at or after 3 seconds, countdown padding was applied
|
||||
# Use a small buffer (2.5s) to account for songs that naturally start a bit late
|
||||
if first_lyric_time >= 2.5:
|
||||
self.logger.info(f"Detected countdown padding from LRC: first lyric at {first_lyric_time:.2f}s")
|
||||
return (True, self.COUNTDOWN_PADDING_SECONDS)
|
||||
|
||||
return (False, 0.0)
|
||||
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Failed to detect countdown padding from LRC file: {e}")
|
||||
return (False, 0.0)
|
||||
|
||||
def find_best_split_point(self, line):
|
||||
"""
|
||||
Find the best split point in a line based on the specified criteria.
|
||||
@@ -138,23 +195,43 @@ class LyricsProcessor:
|
||||
lyrics_video_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (With Vocals).mkv")
|
||||
lyrics_lrc_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (Karaoke).lrc")
|
||||
|
||||
# If files exist in parent directory, return early
|
||||
# If files exist in parent directory, return early (but detect countdown padding first)
|
||||
if os.path.exists(parent_video_path) and os.path.exists(parent_lrc_path):
|
||||
self.logger.info(f"Found existing video and LRC files in parent directory, skipping transcription")
|
||||
self.logger.info("Found existing video and LRC files in parent directory, skipping transcription")
|
||||
|
||||
# Detect countdown padding from existing LRC file
|
||||
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
|
||||
|
||||
if countdown_padding_added:
|
||||
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
|
||||
|
||||
return {
|
||||
"lrc_filepath": parent_lrc_path,
|
||||
"ass_filepath": parent_video_path,
|
||||
"countdown_padding_added": countdown_padding_added,
|
||||
"countdown_padding_seconds": countdown_padding_seconds,
|
||||
"padded_audio_filepath": None, # Original padded audio may not exist
|
||||
}
|
||||
|
||||
# If files exist in lyrics directory, copy to parent and return
|
||||
# If files exist in lyrics directory, copy to parent and return (but detect countdown padding first)
|
||||
if os.path.exists(lyrics_video_path) and os.path.exists(lyrics_lrc_path):
|
||||
self.logger.info(f"Found existing video and LRC files in lyrics directory, copying to parent")
|
||||
self.logger.info("Found existing video and LRC files in lyrics directory, copying to parent")
|
||||
os.makedirs(track_output_dir, exist_ok=True)
|
||||
shutil.copy2(lyrics_video_path, parent_video_path)
|
||||
shutil.copy2(lyrics_lrc_path, parent_lrc_path)
|
||||
|
||||
# Detect countdown padding from existing LRC file
|
||||
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
|
||||
|
||||
if countdown_padding_added:
|
||||
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
|
||||
|
||||
return {
|
||||
"lrc_filepath": parent_lrc_path,
|
||||
"ass_filepath": parent_video_path,
|
||||
"countdown_padding_added": countdown_padding_added,
|
||||
"countdown_padding_seconds": countdown_padding_seconds,
|
||||
"padded_audio_filepath": None, # Original padded audio may not exist
|
||||
}
|
||||
|
||||
# Create lyrics directory if it doesn't exist
|
||||
|
||||
@@ -19,6 +19,9 @@ from karaoke_gen.karaoke_finalise import KaraokeFinalise
|
||||
# Global logger
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.setLevel(logging.INFO) # Set initial log level
|
||||
# Prevent log propagation to root logger to avoid duplicate logs
|
||||
# when external packages (like lyrics_converter) configure root logger handlers
|
||||
logger.propagate = False
|
||||
|
||||
|
||||
async def process_track_prep(row, args, logger, log_formatter):
|
||||
|
||||
@@ -208,6 +208,11 @@ def create_parser(prog: str = "karaoke-gen") -> argparse.ArgumentParser:
|
||||
default="flac",
|
||||
help="Optional: format / file extension for instrumental track to use for remux (default: %(default)s). Example: --instrumental_format=mp3",
|
||||
)
|
||||
audio_group.add_argument(
|
||||
"--skip_instrumental_review",
|
||||
action="store_true",
|
||||
help="Optional: Skip the interactive instrumental review UI and use the old numeric selection. Example: --skip_instrumental_review",
|
||||
)
|
||||
|
||||
# Lyrics Configuration
|
||||
lyrics_group = parser.add_argument_group("Lyrics Configuration")
|
||||
@@ -347,8 +352,10 @@ def create_parser(prog: str = "karaoke-gen") -> argparse.ArgumentParser:
|
||||
)
|
||||
remote_group.add_argument(
|
||||
"--review-ui-url",
|
||||
default=os.environ.get('REVIEW_UI_URL', 'https://lyrics.nomadkaraoke.com'),
|
||||
help="Lyrics review UI URL (default: https://lyrics.nomadkaraoke.com)",
|
||||
default=os.environ.get('REVIEW_UI_URL', os.environ.get('LYRICS_REVIEW_UI_URL', 'local')),
|
||||
help="Lyrics review UI URL. Use 'local' for bundled frontend (default), "
|
||||
"'http://localhost:5173' for Vite dev server, or 'https://lyrics.nomadkaraoke.com' for hosted. "
|
||||
"(env: REVIEW_UI_URL or LYRICS_REVIEW_UI_URL)",
|
||||
)
|
||||
remote_group.add_argument(
|
||||
"--poll-interval",
|
||||
|
||||
@@ -14,14 +14,285 @@ import sys
|
||||
import json
|
||||
import asyncio
|
||||
import time
|
||||
import glob
|
||||
import pyperclip
|
||||
from karaoke_gen import KaraokePrep
|
||||
from karaoke_gen.karaoke_finalise import KaraokeFinalise
|
||||
from karaoke_gen.audio_fetcher import UserCancelledError
|
||||
from karaoke_gen.instrumental_review import (
|
||||
AudioAnalyzer,
|
||||
WaveformGenerator,
|
||||
InstrumentalReviewServer,
|
||||
)
|
||||
from .cli_args import create_parser, process_style_overrides, is_url, is_file
|
||||
|
||||
|
||||
def _resolve_path_for_cwd(path: str, track_dir: str) -> str:
|
||||
"""
|
||||
Resolve a path that may have been created relative to the original working directory.
|
||||
|
||||
After os.chdir(track_dir), paths like './TrackDir/stems/file.flac' become invalid.
|
||||
This function converts such paths to work from the new current directory.
|
||||
|
||||
Args:
|
||||
path: The path to resolve (may be relative or absolute)
|
||||
track_dir: The track directory we've chdir'd into
|
||||
|
||||
Returns:
|
||||
A path that's valid from the current working directory
|
||||
"""
|
||||
if os.path.isabs(path):
|
||||
return path
|
||||
|
||||
# Normalize both paths for comparison
|
||||
norm_path = os.path.normpath(path)
|
||||
norm_track_dir = os.path.normpath(track_dir)
|
||||
|
||||
# If path starts with track_dir, strip it to get the relative path from within track_dir
|
||||
# e.g., './Four Lanes Male Choir - The White Rose/stems/file.flac' -> 'stems/file.flac'
|
||||
if norm_path.startswith(norm_track_dir + os.sep):
|
||||
return norm_path[len(norm_track_dir) + 1:]
|
||||
elif norm_path.startswith(norm_track_dir):
|
||||
return norm_path[len(norm_track_dir):].lstrip(os.sep) or '.'
|
||||
|
||||
# If path doesn't start with track_dir, it might already be relative to track_dir
|
||||
# or it's a path that doesn't need transformation
|
||||
return path
|
||||
|
||||
|
||||
def auto_select_instrumental(track: dict, track_dir: str, logger: logging.Logger) -> str:
|
||||
"""
|
||||
Auto-select the best instrumental file when --skip_instrumental_review is used.
|
||||
|
||||
Selection priority:
|
||||
1. Padded combined instrumental (+BV) - synchronized with vocals + backing vocals
|
||||
2. Non-padded combined instrumental (+BV) - has backing vocals
|
||||
3. Padded clean instrumental - synchronized with vocals
|
||||
4. Non-padded clean instrumental - basic instrumental
|
||||
|
||||
Args:
|
||||
track: The track dictionary from KaraokePrep containing separated audio info
|
||||
track_dir: The track output directory (we're already chdir'd into it)
|
||||
logger: Logger instance
|
||||
|
||||
Returns:
|
||||
Path to the selected instrumental file
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If no suitable instrumental file can be found
|
||||
"""
|
||||
separated = track.get("separated_audio", {})
|
||||
|
||||
# Look for combined instrumentals first (they include backing vocals)
|
||||
combined = separated.get("combined_instrumentals", {})
|
||||
for model, path in combined.items():
|
||||
if path:
|
||||
resolved = _resolve_path_for_cwd(path, track_dir)
|
||||
# Prefer padded version if it exists
|
||||
base, ext = os.path.splitext(resolved)
|
||||
padded = f"{base} (Padded){ext}"
|
||||
if os.path.exists(padded):
|
||||
logger.info(f"Auto-selected padded combined instrumental: {padded}")
|
||||
return padded
|
||||
if os.path.exists(resolved):
|
||||
logger.info(f"Auto-selected combined instrumental: {resolved}")
|
||||
return resolved
|
||||
|
||||
# Fall back to clean instrumental
|
||||
clean = separated.get("clean_instrumental", {})
|
||||
if clean.get("instrumental"):
|
||||
resolved = _resolve_path_for_cwd(clean["instrumental"], track_dir)
|
||||
# Prefer padded version if it exists
|
||||
base, ext = os.path.splitext(resolved)
|
||||
padded = f"{base} (Padded){ext}"
|
||||
if os.path.exists(padded):
|
||||
logger.info(f"Auto-selected padded clean instrumental: {padded}")
|
||||
return padded
|
||||
if os.path.exists(resolved):
|
||||
logger.info(f"Auto-selected clean instrumental: {resolved}")
|
||||
return resolved
|
||||
|
||||
# If separated_audio doesn't have what we need, search the directory
|
||||
# This handles edge cases and custom instrumentals
|
||||
logger.info("No instrumental found in separated_audio, searching directory...")
|
||||
instrumental_files = glob.glob("*(Instrumental*.flac") + glob.glob("*(Instrumental*.wav")
|
||||
|
||||
# Sort to prefer padded versions and combined instrumentals
|
||||
padded_combined = [f for f in instrumental_files if "(Padded)" in f and "+BV" in f]
|
||||
if padded_combined:
|
||||
logger.info(f"Auto-selected from directory: {padded_combined[0]}")
|
||||
return padded_combined[0]
|
||||
|
||||
padded_files = [f for f in instrumental_files if "(Padded)" in f]
|
||||
if padded_files:
|
||||
logger.info(f"Auto-selected from directory: {padded_files[0]}")
|
||||
return padded_files[0]
|
||||
|
||||
combined_files = [f for f in instrumental_files if "+BV" in f]
|
||||
if combined_files:
|
||||
logger.info(f"Auto-selected from directory: {combined_files[0]}")
|
||||
return combined_files[0]
|
||||
|
||||
if instrumental_files:
|
||||
logger.info(f"Auto-selected from directory: {instrumental_files[0]}")
|
||||
return instrumental_files[0]
|
||||
|
||||
raise FileNotFoundError(
|
||||
"No instrumental file found. Audio separation may have failed. "
|
||||
"Check the stems/ directory for separated audio files."
|
||||
)
|
||||
|
||||
|
||||
def run_instrumental_review(track: dict, logger: logging.Logger) -> str | None:
|
||||
"""
|
||||
Run the instrumental review UI to let user select the best instrumental track.
|
||||
|
||||
This analyzes the backing vocals, generates a waveform, and opens a browser
|
||||
with an interactive UI for reviewing and selecting the instrumental.
|
||||
|
||||
Args:
|
||||
track: The track dictionary from KaraokePrep containing separated audio info
|
||||
logger: Logger instance
|
||||
|
||||
Returns:
|
||||
Path to the selected instrumental file, or None to use the old numeric selection
|
||||
"""
|
||||
track_dir = track.get("track_output_dir", ".")
|
||||
artist = track.get("artist", "")
|
||||
title = track.get("title", "")
|
||||
base_name = f"{artist} - {title}"
|
||||
|
||||
# Get separation results
|
||||
separated = track.get("separated_audio", {})
|
||||
if not separated:
|
||||
logger.info("No separated audio found, skipping instrumental review UI")
|
||||
return None
|
||||
|
||||
# Find the backing vocals file
|
||||
# Note: Paths in separated_audio may be relative to the original working directory,
|
||||
# but we've already chdir'd into track_dir. Use _resolve_path_for_cwd to fix paths.
|
||||
backing_vocals_path = None
|
||||
backing_vocals_result = separated.get("backing_vocals", {})
|
||||
for model, paths in backing_vocals_result.items():
|
||||
if paths.get("backing_vocals"):
|
||||
backing_vocals_path = _resolve_path_for_cwd(paths["backing_vocals"], track_dir)
|
||||
break
|
||||
|
||||
if not backing_vocals_path or not os.path.exists(backing_vocals_path):
|
||||
logger.info("No backing vocals file found, skipping instrumental review UI")
|
||||
return None
|
||||
|
||||
# Find the clean instrumental file
|
||||
clean_result = separated.get("clean_instrumental", {})
|
||||
raw_clean_path = clean_result.get("instrumental")
|
||||
clean_instrumental_path = _resolve_path_for_cwd(raw_clean_path, track_dir) if raw_clean_path else None
|
||||
|
||||
if not clean_instrumental_path or not os.path.exists(clean_instrumental_path):
|
||||
logger.info("No clean instrumental file found, skipping instrumental review UI")
|
||||
return None
|
||||
|
||||
# Find the combined instrumental (with backing vocals) file - these have "(Padded)" suffix if padded
|
||||
combined_result = separated.get("combined_instrumentals", {})
|
||||
with_backing_path = None
|
||||
for model, path in combined_result.items():
|
||||
resolved_path = _resolve_path_for_cwd(path, track_dir) if path else None
|
||||
if resolved_path and os.path.exists(resolved_path):
|
||||
with_backing_path = resolved_path
|
||||
break
|
||||
|
||||
try:
|
||||
logger.info("=== Starting Instrumental Review ===")
|
||||
logger.info(f"Analyzing backing vocals: {backing_vocals_path}")
|
||||
|
||||
# Analyze backing vocals
|
||||
analyzer = AudioAnalyzer()
|
||||
analysis = analyzer.analyze(backing_vocals_path)
|
||||
|
||||
logger.info(f"Analysis complete:")
|
||||
logger.info(f" Has audible content: {analysis.has_audible_content}")
|
||||
logger.info(f" Total duration: {analysis.total_duration_seconds:.1f}s")
|
||||
logger.info(f" Audible segments: {len(analysis.audible_segments)}")
|
||||
logger.info(f" Recommendation: {analysis.recommended_selection.value}")
|
||||
|
||||
# Generate waveform
|
||||
# Note: We're already in track_dir after chdir, so use current directory
|
||||
logger.info("Generating waveform visualization...")
|
||||
waveform_generator = WaveformGenerator()
|
||||
waveform_path = f"{base_name} (Backing Vocals Waveform).png"
|
||||
waveform_generator.generate(
|
||||
audio_path=backing_vocals_path,
|
||||
output_path=waveform_path,
|
||||
segments=analysis.audible_segments,
|
||||
)
|
||||
|
||||
# Start the review server
|
||||
# Note: We're already in track_dir after chdir, so output_dir is "."
|
||||
logger.info("Starting instrumental review UI...")
|
||||
server = InstrumentalReviewServer(
|
||||
output_dir=".",
|
||||
base_name=base_name,
|
||||
analysis=analysis,
|
||||
waveform_path=waveform_path,
|
||||
backing_vocals_path=backing_vocals_path,
|
||||
clean_instrumental_path=clean_instrumental_path,
|
||||
with_backing_path=with_backing_path,
|
||||
)
|
||||
|
||||
# Start server and open browser, wait for selection
|
||||
server.start_and_open_browser()
|
||||
|
||||
logger.info("Waiting for instrumental selection in browser...")
|
||||
logger.info("(Close the browser tab or press Ctrl+C to cancel)")
|
||||
|
||||
try:
|
||||
# Wait for user selection (blocking)
|
||||
server._selection_event.wait()
|
||||
selection = server.get_selection()
|
||||
|
||||
logger.info(f"User selected: {selection}")
|
||||
|
||||
# Stop the server
|
||||
server.stop()
|
||||
|
||||
# Return the selected instrumental path
|
||||
if selection == "clean":
|
||||
return clean_instrumental_path
|
||||
elif selection == "with_backing":
|
||||
return with_backing_path
|
||||
elif selection == "custom":
|
||||
custom_path = server.get_custom_instrumental_path()
|
||||
if custom_path and os.path.exists(custom_path):
|
||||
return custom_path
|
||||
else:
|
||||
logger.warning("Custom instrumental not found, falling back to clean")
|
||||
return clean_instrumental_path
|
||||
elif selection == "uploaded":
|
||||
uploaded_path = server.get_uploaded_instrumental_path()
|
||||
if uploaded_path and os.path.exists(uploaded_path):
|
||||
return uploaded_path
|
||||
else:
|
||||
logger.warning("Uploaded instrumental not found, falling back to clean")
|
||||
return clean_instrumental_path
|
||||
else:
|
||||
logger.warning(f"Unknown selection: {selection}, falling back to numeric selection")
|
||||
return None
|
||||
|
||||
except KeyboardInterrupt:
|
||||
logger.info("Instrumental review cancelled by user")
|
||||
server.stop()
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error during instrumental review: {e}")
|
||||
logger.info("Falling back to numeric selection")
|
||||
return None
|
||||
|
||||
|
||||
async def async_main():
|
||||
logger = logging.getLogger(__name__)
|
||||
# Prevent log propagation to root logger to avoid duplicate logs
|
||||
# when external packages (like lyrics_converter) configure root logger handlers
|
||||
logger.propagate = False
|
||||
log_handler = logging.StreamHandler()
|
||||
log_formatter = logging.Formatter(fmt="%(asctime)s.%(msecs)03d - %(levelname)s - %(module)s - %(message)s", datefmt="%Y-%m-%d %H:%M:%S")
|
||||
log_handler.setFormatter(log_formatter)
|
||||
@@ -31,6 +302,11 @@ async def async_main():
|
||||
parser = create_parser(prog="karaoke-gen")
|
||||
args = parser.parse_args()
|
||||
|
||||
# Set review UI URL environment variable for the lyrics transcriber review server
|
||||
# This allows development against a local frontend dev server (e.g., http://localhost:5173)
|
||||
if hasattr(args, 'review_ui_url') and args.review_ui_url:
|
||||
os.environ['LYRICS_REVIEW_UI_URL'] = args.review_ui_url
|
||||
|
||||
# Process style overrides
|
||||
try:
|
||||
style_overrides = process_style_overrides(args.style_override, logger)
|
||||
@@ -122,7 +398,21 @@ async def async_main():
|
||||
kprep.input_media = input_audio_wav
|
||||
|
||||
# Run KaraokePrep
|
||||
tracks = await kprep.process()
|
||||
try:
|
||||
tracks = await kprep.process()
|
||||
except UserCancelledError:
|
||||
logger.info("Operation cancelled by user")
|
||||
return
|
||||
except KeyboardInterrupt:
|
||||
logger.info("Operation cancelled by user (Ctrl+C)")
|
||||
return
|
||||
|
||||
# Filter out None tracks (can happen if prep failed for some tracks)
|
||||
tracks = [t for t in tracks if t is not None] if tracks else []
|
||||
|
||||
if not tracks:
|
||||
logger.warning("No tracks to process")
|
||||
return
|
||||
|
||||
# Load CDG styles if CDG generation is enabled
|
||||
cdg_styles = None
|
||||
@@ -441,7 +731,21 @@ async def async_main():
|
||||
kprep = kprep_coroutine
|
||||
|
||||
# Create final tracks data structure
|
||||
tracks = await kprep.process()
|
||||
try:
|
||||
tracks = await kprep.process()
|
||||
except UserCancelledError:
|
||||
logger.info("Operation cancelled by user")
|
||||
return
|
||||
except KeyboardInterrupt:
|
||||
logger.info("Operation cancelled by user (Ctrl+C)")
|
||||
return
|
||||
|
||||
# Filter out None tracks (can happen if prep failed for some tracks)
|
||||
tracks = [t for t in tracks if t is not None] if tracks else []
|
||||
|
||||
if not tracks:
|
||||
logger.warning("No tracks to process")
|
||||
return
|
||||
|
||||
# If prep-only mode, we're done
|
||||
if args.prep_only:
|
||||
@@ -461,6 +765,67 @@ async def async_main():
|
||||
logger.info(f"Changing to directory: {track_dir}")
|
||||
os.chdir(track_dir)
|
||||
|
||||
# Select instrumental file - either via web UI or auto-selection
|
||||
# This ALWAYS produces a selected file - no silent fallback to legacy code
|
||||
selected_instrumental_file = None
|
||||
skip_review = getattr(args, 'skip_instrumental_review', False)
|
||||
|
||||
if skip_review:
|
||||
# Auto-select instrumental when review is skipped (non-interactive mode)
|
||||
logger.info("Instrumental review skipped (--skip_instrumental_review), auto-selecting instrumental file...")
|
||||
try:
|
||||
selected_instrumental_file = auto_select_instrumental(
|
||||
track=track,
|
||||
track_dir=track_dir,
|
||||
logger=logger,
|
||||
)
|
||||
except FileNotFoundError as e:
|
||||
logger.error(f"Failed to auto-select instrumental: {e}")
|
||||
logger.error("Check that audio separation completed successfully.")
|
||||
sys.exit(1)
|
||||
return # Explicit return for testing
|
||||
else:
|
||||
# Run instrumental review web UI
|
||||
selected_instrumental_file = run_instrumental_review(
|
||||
track=track,
|
||||
logger=logger,
|
||||
)
|
||||
|
||||
# If instrumental review failed/returned None, show error and exit
|
||||
# NO SILENT FALLBACK - we want to know if the new flow has issues
|
||||
if selected_instrumental_file is None:
|
||||
logger.error("")
|
||||
logger.error("=" * 70)
|
||||
logger.error("INSTRUMENTAL SELECTION FAILED")
|
||||
logger.error("=" * 70)
|
||||
logger.error("")
|
||||
logger.error("The instrumental review UI could not find the required files.")
|
||||
logger.error("")
|
||||
logger.error("Common causes:")
|
||||
logger.error(" - No backing vocals file was found (check stems/ directory)")
|
||||
logger.error(" - No clean instrumental was found (audio separation may have failed)")
|
||||
logger.error(" - Path resolution failed after directory change")
|
||||
logger.error("")
|
||||
logger.error("To investigate:")
|
||||
logger.error(" - Check the stems/ directory for: *Backing Vocals*.flac and *Instrumental*.flac")
|
||||
logger.error(" - Look for separation errors earlier in the log")
|
||||
logger.error(" - Verify audio separation completed without errors")
|
||||
logger.error("")
|
||||
logger.error("Workarounds:")
|
||||
logger.error(" - Re-run with --skip_instrumental_review to auto-select an instrumental")
|
||||
logger.error(" - Re-run the full pipeline to regenerate stems")
|
||||
logger.error("")
|
||||
sys.exit(1)
|
||||
return # Explicit return for testing
|
||||
|
||||
logger.info(f"Selected instrumental file: {selected_instrumental_file}")
|
||||
|
||||
# Get countdown padding info from track (if vocals were padded, instrumental must match)
|
||||
countdown_padding_seconds = None
|
||||
if track.get("countdown_padding_added", False):
|
||||
countdown_padding_seconds = track.get("countdown_padding_seconds", 3.0)
|
||||
logger.info(f"Countdown padding detected: {countdown_padding_seconds}s (will be applied to instrumental if needed)")
|
||||
|
||||
# Load CDG styles if CDG generation is enabled
|
||||
cdg_styles = None
|
||||
if args.enable_cdg:
|
||||
@@ -504,6 +869,8 @@ async def async_main():
|
||||
cdg_styles=cdg_styles,
|
||||
keep_brand_code=getattr(args, 'keep_brand_code', False),
|
||||
non_interactive=args.yes,
|
||||
selected_instrumental_file=selected_instrumental_file,
|
||||
countdown_padding_seconds=countdown_padding_seconds,
|
||||
)
|
||||
|
||||
try:
|
||||
|
||||
@@ -0,0 +1,336 @@
|
||||
# MidiCo Synchronizer Reference
|
||||
|
||||
This document provides a detailed reference for replicating the MidiCo Synchronizer user experience in our Nomad Karaoke lyrics review UI. The MidiCo Synchronizer is a native macOS application that provides a polished, intuitive interface for manually syncing lyrics to audio.
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Sometimes transcribed lyrics are correct and start well-synced, but progressively drift out of sync towards the second half of the song. We need an efficient way to fix timing for multiple segments without having to re-sync everything from scratch.
|
||||
|
||||
## Overview of MidiCo Interface
|
||||
|
||||
MidiCo consists of two windows:
|
||||
1. **Main Audio Player Window** - Controls audio playback
|
||||
2. **Synchronizer Window** - Timeline-based lyrics sync interface
|
||||
|
||||

|
||||
|
||||
---
|
||||
|
||||
## 1. Audio Player Integration
|
||||
|
||||
The main MidiCo window provides an audio player that stays synchronized with the Synchronizer:
|
||||
|
||||
- **Play/Pause/Stop buttons** - Standard playback controls
|
||||
- **Rewind/Fast-forward buttons** - Skip through the track
|
||||
- **Progress slider** - Shows current position (00:00 format) and remaining time (-03:45 format)
|
||||
- **Volume slider** - Control audio volume
|
||||
- **Tempo/Pitch sliders** - Adjust playback speed and pitch (useful for difficult sections)
|
||||
|
||||
**Key behavior:** The playhead position is always synced between the audio player and the Synchronizer timeline.
|
||||
|
||||
---
|
||||
|
||||
## 2. Synchronizer Primary View
|
||||
|
||||
The Synchronizer window contains the main timeline interface for syncing lyrics.
|
||||
|
||||
### 2.1 Top Control Buttons
|
||||
|
||||
Four main control buttons at the top:
|
||||
|
||||
| Button | Function |
|
||||
|--------|----------|
|
||||
| **Voice no.1** | Select which vocal track to sync (for multi-voice karaoke) |
|
||||
| **Start Sync** | Begin sync mode - starts audio, shows upcoming words, listens for spacebar |
|
||||
| **Clear Sync** | Delete all sync data, reset to unsynced state |
|
||||
| **Edit Lyrics** | Open modal to edit the lyrics text directly |
|
||||
| **Import Lrc** | Import timing data from an LRC file |
|
||||
|
||||
### 2.2 Timeline View
|
||||
|
||||
The main rectangular timeline area is the core of the interface.
|
||||
|
||||

|
||||
|
||||
#### Time Bar (Top Strip)
|
||||
- Thin bar at the top showing time in `MM:SS` format
|
||||
- Small tick marks for decisecond precision
|
||||
- Clicking anywhere on the time bar sets the playhead position (without changing playback state)
|
||||
|
||||
#### Playhead Marker
|
||||
- Small arrow/triangle indicator in the time bar
|
||||
- White vertical line spanning the full height of the timeline
|
||||
- Shows current playback position
|
||||
- Synced with the audio player position
|
||||
|
||||
#### Timeline Background
|
||||
- Light gray gradient background
|
||||
- **Clicking on the background deselects any selected words**
|
||||
- **Click-drag creates a selection rectangle** for selecting multiple word blocks
|
||||
|
||||
#### Zoom Control
|
||||
- Horizontal slider below the timeline labeled "Zoom"
|
||||
- Smooth continuous zoom (approximately 50 notches)
|
||||
- **Most zoomed in:** ~4.5 seconds visible edge-to-edge
|
||||
- **Most zoomed out:** ~24 seconds visible edge-to-edge
|
||||
- Smooth, responsive zooming is critical for fast songs
|
||||
|
||||

|
||||
|
||||
#### Horizontal Scrolling
|
||||
- Timeline scrolls horizontally to show different parts of the song
|
||||
- Small horizontal scrollbar at the bottom of the timeline area
|
||||
- During sync mode, the timeline auto-scrolls to follow the playhead
|
||||
|
||||
### 2.3 Word Blocks
|
||||
|
||||
Synced words appear as **red rectangular blocks** on the timeline:
|
||||
|
||||
- Each block shows the word text above the red bar
|
||||
- If word text is too long to fit the width of the red bar, the alignment with the red blocks becomes "best effort". Words don't overlap, they're all readable above the red bars for that segment
|
||||
- Block width corresponds to the word's duration
|
||||
- Block horizontal position corresponds to the word's start time
|
||||
|
||||
#### Two-Level Word Layout (Alternating Segments)
|
||||
|
||||
Segments alternate between two vertical positions on the timeline. Each consecutive segment (in time order) is placed on the opposite level from the previous one:
|
||||
|
||||
- **Level 0 (top)**: 1st, 3rd, 5th, 7th... segments
|
||||
- **Level 1 (bottom)**: 2nd, 4th, 6th, 8th... segments
|
||||
|
||||

|
||||
|
||||
This alternating layout is critical because:
|
||||
|
||||
1. **Text never overlaps between consecutive segments** - Since each segment is on a different level, the text labels above the red blocks from one segment won't collide with text from the next segment, even if the text extends past its block width.
|
||||
|
||||
2. **All words remain readable** - Text is allowed to extend horizontally beyond the red block boundaries. Because consecutive segments are always on different vertical levels, this extended text doesn't overlap with adjacent segments.
|
||||
|
||||

|
||||
|
||||
In the example above, notice how:
|
||||
- "Monica Mountains" (level 0) has text that extends beyond its block
|
||||
- "Where she then proceeded to pump her vehicle in and out of turns" (level 1) is on a different vertical level
|
||||
- "Sometimes dropping down to fifty miles per hour" (level 0) returns to the top level
|
||||
- "Only to immediately gun it back up to 90 again" (level 1) is back on the bottom level
|
||||
|
||||
This simple alternating pattern ensures that even long segment text remains fully readable without truncation.
|
||||
|
||||
**Edge case:** Overlap can still occur in rare situations - if one segment's text is extremely long AND the segment two positions later (on the same level) is very short/early. But this is uncommon in practice.
|
||||
|
||||
#### Word Block Selection
|
||||
|
||||
- **Single click** on a word block: Selects that block (shown with white 1px border)
|
||||
- **Click-drag on background**: Creates selection rectangle to select multiple blocks
|
||||
|
||||

|
||||
|
||||
#### Word Block Manipulation (Resize & Drag)
|
||||
|
||||
Selected word blocks can be manipulated directly on the timeline:
|
||||
|
||||
##### Resizing Word Blocks (Change Duration)
|
||||
|
||||
When a word block is selected and you hover over it, a **small white dot/handle** appears just inside the right edge of the block. This resize handle allows you to change the word's duration:
|
||||
|
||||
1. Select a word block (single click)
|
||||
2. Hover over the selected block to reveal the resize handle (white dot on right edge)
|
||||
3. Click and drag the handle left/right to shorten/lengthen the word's duration
|
||||
4. The word's end time changes; start time remains fixed
|
||||
|
||||
This is essential for fine-tuning word durations after initial sync, especially for:
|
||||
- Words that were synced too long or too short
|
||||
- Adjusting timing to better match the audio
|
||||
- Creating smoother transitions between words
|
||||
|
||||
##### Dragging Word Blocks (Change Position)
|
||||
|
||||
Selected word blocks can be dragged horizontally to change their position in time:
|
||||
|
||||
**Single word:**
|
||||
1. Select a word block
|
||||
2. Click and drag anywhere on the block (not on the resize handle)
|
||||
3. The entire word moves earlier/later in time (both start and end times shift)
|
||||
|
||||
**Multiple words:**
|
||||
1. Drag-select multiple word blocks (or Ctrl/Cmd+click to add to selection)
|
||||
2. Click and drag any of the selected blocks
|
||||
3. **All selected blocks move together**, maintaining their relative timing
|
||||
|
||||
This is particularly useful for:
|
||||
- Shifting an entire segment earlier or later when it's consistently off
|
||||
- Adjusting groups of words that were synced correctly relative to each other but offset from the audio
|
||||
- Fine-tuning timing without having to re-sync
|
||||
|
||||
**Example workflow:** If a segment of 10 words all come in 0.2 seconds too late:
|
||||
1. Drag-select all 10 word blocks
|
||||
2. Drag them 0.2 seconds earlier
|
||||
3. All words maintain their relative timing but are now properly aligned with the audio
|
||||
|
||||
### 2.4 Upcoming Words Display
|
||||
|
||||
During sync mode, unsynced words appear in **two locations**:
|
||||
|
||||

|
||||
|
||||
1. **Fixed position above time bar** (left side): Shows upcoming words as red/white blocks in a horizontal row. The next word to sync is highlighted in red, subsequent words in white.
|
||||
|
||||
2. **On the timeline, right of playhead**: Same words appear starting from the playhead position and extending to the right, with a pale yellow background and black text, moving along with the playhead during playback.
|
||||
|
||||
This dual display makes it easy to:
|
||||
- See what's coming next (fixed position - easy to read while focused on timing)
|
||||
- See where words will land on the timeline relative to already-synced words
|
||||
|
||||
---
|
||||
|
||||
## 3. Action Buttons (Icon Toolbar)
|
||||
|
||||
A row of small icon-only buttons appears above the timeline. Key buttons we need:
|
||||
|
||||
### Essential Buttons
|
||||
|
||||
| Icon | Tooltip | Function |
|
||||
|------|---------|----------|
|
||||
| **Rectangle with Pencil** | Split, spell or edit selected word | Opens popup to edit a single word. Useful for typos. Entering multiple space-separated words splits the selected word while preserving timing. |
|
||||
| **No entry symbol** | Unsynchronize from cursor position | All word blocks **after** the current playhead position are reset to unsynced state. Critical for fixing drift in later parts of a song. |
|
||||
| **Trash** | Delete selected | Deletes selected word(s) entirely. Useful for removing backing vocals or nonsense words. |
|
||||
|
||||
### Typical Workflow for Deleting Unwanted Words
|
||||
|
||||
1. Tap spacebar to sync the unwanted words (so they appear on timeline)
|
||||
2. Stop playback
|
||||
3. Click-drag to select the unwanted word blocks
|
||||
4. Click trash icon to delete
|
||||
5. Click Start Sync to resume syncing from where you left off
|
||||
|
||||
---
|
||||
|
||||
## 4. Sync Mode Operation
|
||||
|
||||
### Starting Sync
|
||||
|
||||
1. Click **Start Sync** button
|
||||
2. Audio begins playing from current playhead position
|
||||
3. Upcoming unsynced words appear in both display locations
|
||||
4. System begins listening for spacebar input
|
||||
|
||||
### Spacebar Timing
|
||||
|
||||
Two input modes:
|
||||
|
||||
**Tap (press and release quickly):**
|
||||
- Sets the **start time** of the next unsynced word to the current playhead position
|
||||
- The word's **end time** is set when the following word is synced
|
||||
- If gap between words > 1 second, previous word gets default 1 second duration
|
||||
|
||||
**Press-hold:**
|
||||
- Hold spacebar down when word starts
|
||||
- Release when word ends
|
||||
- Sets both start and end times explicitly
|
||||
|
||||
### Stopping Sync
|
||||
|
||||
- Click the **Stop** button (replaces Play when in sync mode)
|
||||
- Or click **Start Sync** again to toggle off
|
||||
- Sync progress is preserved
|
||||
|
||||
---
|
||||
|
||||
## 5. Edit Lyrics Modal
|
||||
|
||||

|
||||
|
||||
Accessed via the **Edit Lyrics** button. Provides:
|
||||
|
||||
- **Left panel**: Plain text editor with all lyrics, newlines separate segments
|
||||
- **Right panel**: Preview of how lyrics will appear (karaoke style rendering)
|
||||
- **Language selector**: For hyphenation support
|
||||
- **Hyphenate button**: Auto-hyphenate words for syllable-level sync
|
||||
- **Reset button**: Revert to original lyrics
|
||||
- **Cancel/OK buttons**: Discard or apply changes
|
||||
|
||||
### Important Behavior
|
||||
|
||||
Editing lyrics when there are already synced timestamps **usually messes up the syncs**. Users typically need to Clear Sync and start over after editing lyrics.
|
||||
|
||||
---
|
||||
|
||||
## 6. Apply Button
|
||||
|
||||
The **Apply** button in the bottom-right corner is critical:
|
||||
|
||||
- Changes made in the Synchronizer are **not saved** until Apply is clicked
|
||||
- This allows experimenting without fear of losing work
|
||||
- If sync goes badly wrong, user can close the Synchronizer without applying changes
|
||||
- Provides a safe "sandbox" for sync work
|
||||
|
||||
---
|
||||
|
||||
## 7. Performance Requirements
|
||||
|
||||
The entire interface must be:
|
||||
|
||||
- **Smooth and responsive** - No lag or stutter during playback
|
||||
- **Low latency** - Spacebar input must register with sub-100ms precision
|
||||
- **Efficient** - No unnecessary computation during sync (user is focused on timing)
|
||||
- **Native-app feel** - As expected from a macOS application
|
||||
|
||||
Since we're dealing with sub-second time precision for synced lyrics, performance is critical.
|
||||
|
||||
---
|
||||
|
||||
## 8. Implementation Notes for Our App
|
||||
|
||||
### Entry Point: "Edit All" Button
|
||||
|
||||
The current "Edit All" button should show a choice:
|
||||
1. **Replace all lyrics** - Shows the existing paste box flow (for completely new lyrics)
|
||||
2. **Re-sync existing lyrics** - Goes directly to the new Synchronizer view with all existing words/segments plotted on the timeline
|
||||
|
||||
This preserves existing sync data while allowing re-sync of later portions.
|
||||
|
||||
### Key Features to Implement
|
||||
|
||||
1. **Timeline with word blocks showing existing sync data**
|
||||
2. **Playhead synced with audio player**
|
||||
3. **Zoom slider (4.5s to 24s range)**
|
||||
4. **Horizontal scrolling**
|
||||
5. **Two-level word block layout**
|
||||
6. **Start Sync mode with spacebar capture**
|
||||
7. **Upcoming words display (fixed + timeline)**
|
||||
8. **"Unsynchronize from cursor" - critical for fixing drift**
|
||||
9. **Word selection (single click + drag-select)**
|
||||
10. **Delete selected words**
|
||||
11. **Edit single word popup**
|
||||
12. **Apply/Cancel for safe experimentation**
|
||||
|
||||
### Fixing Timing Drift (Primary Use Case)
|
||||
|
||||
The main workflow for fixing drift:
|
||||
|
||||
1. Open Synchronizer (with existing sync data loaded)
|
||||
2. Play audio to find where sync starts drifting
|
||||
3. Position playhead at last correctly-synced word
|
||||
4. Click "Unsynchronize from cursor position"
|
||||
5. Click "Start Sync" to resume syncing from that point
|
||||
6. Tap spacebar to re-sync the remaining words
|
||||
7. Click "Apply" when done
|
||||
|
||||
This workflow allows keeping good sync data while fixing only the problematic portion.
|
||||
|
||||
---
|
||||
|
||||
## Screenshots Reference
|
||||
|
||||
All screenshots are located in: `docs/midico-sync-screenshots/`
|
||||
|
||||
| Filename | Description |
|
||||
|----------|-------------|
|
||||
| `MidiCo-Synchronizer-2025-12-18-Empty-Timeline-Before-Any-Words-Synced.png` | Initial state with empty timeline |
|
||||
| `MidiCo-Synchronizer-2025-12-18-Start-Sync-Showing-Upcoming-Words.png` | Sync mode started, showing upcoming words in both locations |
|
||||
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Synced-And-Upcoming-Words.png` | Mid-sync showing both synced blocks and upcoming words |
|
||||
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Two-Levels-Word-Blocks.png` | Demonstration of two-level word block layout |
|
||||
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Max-Zoom-Level.png` | Maximum zoom level (~4.5 seconds visible) |
|
||||
| `MidiCo-Synchronizer-2025-12-18-Click-Drag-Word-Selection.png` | Selection rectangle for multi-word selection |
|
||||
| `MidiCo-Synchronizer-2025-12-18-Lyrics-Editor-Modal.png` | Edit Lyrics modal dialog |
|
||||
| `MidiCo-Synchronizer-2025-12-18-Timeline-With-Non-Overlapping-Segments.png` | Shows alternating segment levels with long text that doesn't overlap |
|
||||
|
After Width: | Height: | Size: 138 KiB |
|
After Width: | Height: | Size: 144 KiB |
|
After Width: | Height: | Size: 149 KiB |
|
After Width: | Height: | Size: 162 KiB |
|
After Width: | Height: | Size: 126 KiB |
|
After Width: | Height: | Size: 143 KiB |
|
After Width: | Height: | Size: 150 KiB |
|
After Width: | Height: | Size: 56 KiB |
@@ -1,9 +1,8 @@
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional, Tuple, Union
|
||||
from typing import Any, Dict, List, Optional, Set, Tuple, Union
|
||||
import logging
|
||||
from tqdm import tqdm
|
||||
from multiprocessing import Pool, cpu_count
|
||||
from functools import partial
|
||||
from pathlib import Path
|
||||
import json
|
||||
@@ -64,8 +63,70 @@ class AnchorSequenceFinder:
|
||||
# self.logger.debug(f"_find_ngrams called with {len(words)} words, n={n}")
|
||||
return [(words[i : i + n], i) for i in range(len(words) - n + 1)]
|
||||
|
||||
def _build_ngram_index(
|
||||
self,
|
||||
references: Dict[str, List[str]],
|
||||
n: int
|
||||
) -> Dict[Tuple[str, ...], Dict[str, List[int]]]:
|
||||
"""
|
||||
Build a hash-based index mapping n-grams to their positions in each reference.
|
||||
|
||||
Args:
|
||||
references: Dict mapping source names to lists of cleaned words
|
||||
n: The n-gram length to index
|
||||
|
||||
Returns:
|
||||
Dict mapping n-gram tuples to {source: [positions]} dicts
|
||||
"""
|
||||
index: Dict[Tuple[str, ...], Dict[str, List[int]]] = {}
|
||||
|
||||
for source, words in references.items():
|
||||
for i in range(len(words) - n + 1):
|
||||
ngram_tuple = tuple(words[i:i + n])
|
||||
if ngram_tuple not in index:
|
||||
index[ngram_tuple] = {}
|
||||
if source not in index[ngram_tuple]:
|
||||
index[ngram_tuple][source] = []
|
||||
index[ngram_tuple][source].append(i)
|
||||
|
||||
return index
|
||||
|
||||
def _find_matching_sources_indexed(
|
||||
self,
|
||||
ngram: List[str],
|
||||
ngram_index: Dict[Tuple[str, ...], Dict[str, List[int]]]
|
||||
) -> Dict[str, int]:
|
||||
"""
|
||||
Find which sources contain the given n-gram using pre-built index (O(1) lookup).
|
||||
|
||||
Args:
|
||||
ngram: List of words to find
|
||||
ngram_index: Pre-built index from _build_ngram_index()
|
||||
|
||||
Returns:
|
||||
Dict mapping source names to first unused position
|
||||
"""
|
||||
matches = {}
|
||||
ngram_tuple = tuple(ngram)
|
||||
|
||||
# O(1) lookup in the index
|
||||
if ngram_tuple not in ngram_index:
|
||||
return matches
|
||||
|
||||
source_positions = ngram_index[ngram_tuple]
|
||||
|
||||
# For each source that contains this n-gram, find first unused position
|
||||
for source, positions in source_positions.items():
|
||||
used = self.used_positions.get(source, set())
|
||||
for pos in positions:
|
||||
if pos not in used:
|
||||
matches[source] = pos
|
||||
break
|
||||
|
||||
return matches
|
||||
|
||||
def _find_matching_sources(self, ngram: List[str], references: Dict[str, List[str]], n: int) -> Dict[str, int]:
|
||||
"""Find which sources contain the given n-gram and at what positions."""
|
||||
"""Find which sources contain the given n-gram and at what positions (legacy O(n) method)."""
|
||||
# self.logger.debug(f"_find_matching_sources called for ngram: '{' '.join(ngram)}'")
|
||||
matches = {}
|
||||
all_positions = {source: [] for source in references}
|
||||
@@ -193,121 +254,59 @@ class AnchorSequenceFinder:
|
||||
ref_words: Dict[str, List[Word]],
|
||||
min_sources: int,
|
||||
) -> List[AnchorSequence]:
|
||||
"""Process a single n-gram length to find matching sequences with timeout and early termination."""
|
||||
self.logger.info(f"🔍 N-GRAM {n}: Starting processing with {len(trans_words)} transcription words")
|
||||
self.logger.info(f"🔍 N-GRAM {n}: Reference sources: {list(ref_texts_clean.keys())}")
|
||||
self.logger.info(f"🔍 N-GRAM {n}: Max iterations limit: {self.max_iterations_per_ngram}")
|
||||
"""Process a single n-gram length to find matching sequences using hash-based index."""
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Starting processing with {len(trans_words)} transcription words")
|
||||
|
||||
candidate_anchors = []
|
||||
used_positions = {source: set() for source in ref_texts_clean.keys()}
|
||||
used_trans_positions = set()
|
||||
used_trans_positions: Set[int] = set()
|
||||
|
||||
iteration_count = 0
|
||||
last_progress_check = 0
|
||||
last_anchor_count = 0
|
||||
stagnation_count = 0
|
||||
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Processing n-gram length {n} with max {self.max_iterations_per_ngram} iterations")
|
||||
# Build hash-based index for O(1) lookups
|
||||
ngram_index = self._build_ngram_index(ref_texts_clean, n)
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Built index with {len(ngram_index)} unique n-grams")
|
||||
|
||||
# Generate n-grams from transcribed text once
|
||||
# Generate n-grams from transcribed text
|
||||
trans_ngrams = self._find_ngrams(trans_words, n)
|
||||
self.logger.info(f"🔍 N-GRAM {n}: Generated {len(trans_ngrams)} n-grams for processing")
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Processing {len(trans_ngrams)} transcription n-grams")
|
||||
|
||||
# Process all n-grams efficiently in multiple passes
|
||||
found_new_match = True
|
||||
while found_new_match and iteration_count < self.max_iterations_per_ngram:
|
||||
found_new_match = False
|
||||
iteration_count += 1
|
||||
anchors_found_this_iteration = 0
|
||||
# Single pass through all transcription n-grams
|
||||
for ngram, trans_pos in trans_ngrams:
|
||||
# Skip if we've already used this transcription position
|
||||
if trans_pos in used_trans_positions:
|
||||
continue
|
||||
|
||||
# Log every 10th iteration to track progress
|
||||
if iteration_count % 10 == 0:
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Iteration {iteration_count}, anchors found: {len(candidate_anchors)}")
|
||||
|
||||
# Check for progress stagnation every N iterations
|
||||
if iteration_count - last_progress_check >= self.progress_check_interval:
|
||||
current_anchor_count = len(candidate_anchors)
|
||||
if current_anchor_count == last_anchor_count:
|
||||
stagnation_count += 1
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Stagnation check {stagnation_count}/3 at iteration {iteration_count}")
|
||||
if stagnation_count >= 3: # No progress for 3 consecutive checks
|
||||
self.logger.info(f"🔍 N-GRAM {n}: ⏹️ Early termination due to stagnation after {iteration_count} iterations")
|
||||
break
|
||||
else:
|
||||
stagnation_count = 0 # Reset stagnation counter
|
||||
|
||||
last_anchor_count = current_anchor_count
|
||||
last_progress_check = iteration_count
|
||||
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: iteration {iteration_count}, anchors: {current_anchor_count}, stagnation: {stagnation_count}")
|
||||
|
||||
# Process all n-grams in this iteration
|
||||
for ngram, trans_pos in trans_ngrams:
|
||||
# Skip if we've already used this transcription position
|
||||
if trans_pos in used_trans_positions:
|
||||
continue
|
||||
|
||||
# Get the actual words from the transcription at this position
|
||||
actual_words = [w.text.lower().strip('.,?!"\n') for w in all_words[trans_pos : trans_pos + n]]
|
||||
ngram_words = [w.lower() for w in ngram]
|
||||
|
||||
if actual_words != ngram_words:
|
||||
self.logger.error(f"🔍 N-GRAM {n}: ❌ Mismatch between ngram and actual words at position {trans_pos}:")
|
||||
self.logger.error(f"🔍 N-GRAM {n}: Ngram words: {ngram_words}")
|
||||
self.logger.error(f"🔍 N-GRAM {n}: Actual words: {actual_words}")
|
||||
self.logger.error(f"🔍 N-GRAM {n}: Full trans_words: {trans_words}")
|
||||
self.logger.error(f"🔍 N-GRAM {n}: Full all_words: {[w.text for w in all_words]}")
|
||||
raise AssertionError(
|
||||
f"Ngram words don't match actual words at position {trans_pos}. "
|
||||
f"This should never happen as trans_words should be derived from all_words."
|
||||
)
|
||||
|
||||
matches = self._find_matching_sources(ngram, ref_texts_clean, n)
|
||||
if len(matches) >= min_sources:
|
||||
# Log successful match
|
||||
if len(candidate_anchors) < 5: # Only log first few matches to avoid spam
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: ✅ Found match: '{' '.join(ngram)}' at pos {trans_pos} with {len(matches)} sources")
|
||||
|
||||
# Get Word IDs for transcribed words
|
||||
transcribed_word_ids = [w.id for w in all_words[trans_pos : trans_pos + n]]
|
||||
|
||||
# Get Word IDs for reference words
|
||||
reference_word_ids = {source: [w.id for w in ref_words[source][pos : pos + n]] for source, pos in matches.items()}
|
||||
|
||||
# Mark positions as used
|
||||
for source, pos in matches.items():
|
||||
used_positions[source].add(pos)
|
||||
used_trans_positions.add(trans_pos)
|
||||
|
||||
anchor = AnchorSequence(
|
||||
id=WordUtils.generate_id(),
|
||||
transcribed_word_ids=transcribed_word_ids,
|
||||
transcription_position=trans_pos,
|
||||
reference_positions=matches,
|
||||
reference_word_ids=reference_word_ids,
|
||||
confidence=len(matches) / len(ref_texts_clean),
|
||||
)
|
||||
candidate_anchors.append(anchor)
|
||||
anchors_found_this_iteration += 1
|
||||
found_new_match = True
|
||||
|
||||
# For efficiency, if we have very low iteration limits, find one match per iteration
|
||||
if self.max_iterations_per_ngram <= 10:
|
||||
break
|
||||
# Use indexed lookup (O(1) instead of O(n))
|
||||
matches = self._find_matching_sources_indexed(ngram, ngram_index)
|
||||
|
||||
# Log progress for this iteration
|
||||
if anchors_found_this_iteration > 0:
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Found {anchors_found_this_iteration} anchors in iteration {iteration_count}")
|
||||
|
||||
# Early termination if we've found enough anchors or processed all positions
|
||||
if len(used_trans_positions) >= len(trans_ngrams) or len(candidate_anchors) >= len(trans_ngrams):
|
||||
self.logger.info(f"🔍 N-GRAM {n}: ⏹️ Early termination - processed all positions after {iteration_count} iterations")
|
||||
break
|
||||
if len(matches) >= min_sources:
|
||||
# Get Word IDs for transcribed words
|
||||
transcribed_word_ids = [w.id for w in all_words[trans_pos : trans_pos + n]]
|
||||
|
||||
if iteration_count >= self.max_iterations_per_ngram:
|
||||
self.logger.warning(f"🔍 N-GRAM {n}: ⏰ Processing terminated after reaching max iterations ({self.max_iterations_per_ngram})")
|
||||
# Get Word IDs for reference words
|
||||
reference_word_ids = {
|
||||
source: [w.id for w in ref_words[source][pos : pos + n]]
|
||||
for source, pos in matches.items()
|
||||
}
|
||||
|
||||
# Mark transcription position as used
|
||||
used_trans_positions.add(trans_pos)
|
||||
|
||||
# Mark reference positions as used
|
||||
for source, pos in matches.items():
|
||||
if source not in self.used_positions:
|
||||
self.used_positions[source] = set()
|
||||
self.used_positions[source].add(pos)
|
||||
|
||||
anchor = AnchorSequence(
|
||||
id=WordUtils.generate_id(),
|
||||
transcribed_word_ids=transcribed_word_ids,
|
||||
transcription_position=trans_pos,
|
||||
reference_positions=matches,
|
||||
reference_word_ids=reference_word_ids,
|
||||
confidence=len(matches) / len(ref_texts_clean),
|
||||
)
|
||||
candidate_anchors.append(anchor)
|
||||
|
||||
self.logger.info(f"🔍 N-GRAM {n}: ✅ Completed processing after {iteration_count} iterations, found {len(candidate_anchors)} anchors")
|
||||
self.logger.debug(f"🔍 N-GRAM {n}: Found {len(candidate_anchors)} anchors")
|
||||
return candidate_anchors
|
||||
|
||||
def find_anchors(
|
||||
@@ -347,8 +346,10 @@ class AnchorSequenceFinder:
|
||||
self.logger.error("Could not serialize first cached anchor for logging")
|
||||
|
||||
# If not in cache or cache format invalid, perform the computation
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ❌ Cache miss - computing anchors with timeout {self.timeout_seconds}s")
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: Finding anchor sequences for transcription with length {len(transcribed)}")
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: Cache miss - computing anchors")
|
||||
|
||||
# Reset used positions for fresh computation
|
||||
self.used_positions = {}
|
||||
|
||||
# Check timeout before starting computation
|
||||
self._check_timeout(start_time, "anchor computation initialization")
|
||||
@@ -407,122 +408,45 @@ class AnchorSequenceFinder:
|
||||
min_sources=self.min_sources,
|
||||
)
|
||||
|
||||
# Process n-gram lengths in parallel with timeout
|
||||
# Process n-gram lengths sequentially (single-threaded for cloud compatibility)
|
||||
candidate_anchors = []
|
||||
pool_timeout = max(60, self.timeout_seconds // 2) if self.timeout_seconds > 0 else 300 # Use half the total timeout for pool operations
|
||||
|
||||
# Check timeout before parallel processing
|
||||
self._check_timeout(start_time, "parallel processing start")
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Timeout check passed - about to start parallel processing")
|
||||
# Check timeout before processing
|
||||
self._check_timeout(start_time, "n-gram processing start")
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: Starting sequential n-gram processing ({len(n_gram_lengths)} lengths)")
|
||||
|
||||
pool = None
|
||||
try:
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: 🚀 Starting parallel processing with {max(cpu_count() - 1, 1)} processes, pool timeout: {pool_timeout}s")
|
||||
pool = Pool(processes=max(cpu_count() - 1, 1))
|
||||
self.logger.debug(f"🔍 ANCHOR SEARCH: Pool created successfully")
|
||||
results = []
|
||||
|
||||
# Submit all jobs first
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: Submitting {len(n_gram_lengths)} n-gram processing jobs...")
|
||||
async_results = []
|
||||
for i, n in enumerate(n_gram_lengths):
|
||||
self.logger.debug(f"🔍 ANCHOR SEARCH: Submitting job {i+1}/{len(n_gram_lengths)} for n-gram length {n}")
|
||||
async_result = pool.apply_async(process_length_partial, (n,))
|
||||
async_results.append(async_result)
|
||||
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ All {len(async_results)} jobs submitted")
|
||||
|
||||
# Collect results with individual timeouts
|
||||
batch_results = []
|
||||
batch_size = 10
|
||||
|
||||
for i, async_result in enumerate(async_results):
|
||||
n_gram_length = n_gram_lengths[i]
|
||||
try:
|
||||
# Check remaining time for pool timeout (more lenient than overall timeout)
|
||||
batch_size = 10
|
||||
batch_results = []
|
||||
|
||||
for i, n in enumerate(n_gram_lengths):
|
||||
try:
|
||||
# Check timeout periodically
|
||||
if self.timeout_seconds > 0:
|
||||
elapsed_time = time.time() - start_time
|
||||
remaining_time = max(10, self.timeout_seconds - elapsed_time) if self.timeout_seconds > 0 else pool_timeout
|
||||
|
||||
self.logger.debug(f"🔍 ANCHOR SEARCH: Remaining time for n-gram {n_gram_length}: {remaining_time}s")
|
||||
|
||||
# Use a more lenient timeout for individual results to allow fallback
|
||||
individual_timeout = min(pool_timeout, remaining_time) if self.timeout_seconds > 0 else pool_timeout
|
||||
|
||||
result = async_result.get(timeout=individual_timeout)
|
||||
results.append(result)
|
||||
|
||||
# Batch logging - collect info for batched logging
|
||||
batch_results.append((n_gram_length, len(result)))
|
||||
|
||||
# Log progress every batch_size results or on the last result
|
||||
if (i + 1) % batch_size == 0 or (i + 1) == len(async_results):
|
||||
total_anchors_in_batch = sum(anchor_count for _, anchor_count in batch_results)
|
||||
n_gram_ranges = [str(ng) for ng, _ in batch_results]
|
||||
range_str = f"{n_gram_ranges[0]}-{n_gram_ranges[-1]}" if len(n_gram_ranges) > 1 else n_gram_ranges[0]
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Completed n-gram lengths {range_str} ({i+1-len(batch_results)+1}-{i+1}/{len(async_results)}) - found {total_anchors_in_batch} anchors total")
|
||||
batch_results = [] # Reset batch
|
||||
|
||||
except Exception as e:
|
||||
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ n-gram length {n_gram_length} failed or timed out: {str(e)}")
|
||||
results.append([]) # Add empty result to maintain order
|
||||
|
||||
# Add failed result to batch for logging
|
||||
batch_results.append((n_gram_length, 0))
|
||||
|
||||
# If we're running short on time, trigger fallback early
|
||||
if self.timeout_seconds > 0 and (time.time() - start_time) > (self.timeout_seconds * 0.8):
|
||||
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ Approaching timeout limit, triggering early fallback")
|
||||
# Raise exception to trigger fallback to sequential processing
|
||||
raise Exception("Parallel processing timeout, triggering fallback")
|
||||
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Parallel processing completed, combining results...")
|
||||
for anchors in results:
|
||||
if elapsed_time > self.timeout_seconds:
|
||||
self.logger.warning(f"🔍 ANCHOR SEARCH: ⏰ Timeout reached at n-gram {n}, stopping")
|
||||
break
|
||||
|
||||
anchors = self._process_ngram_length(
|
||||
n, trans_words, all_words, ref_texts_clean, ref_words, self.min_sources
|
||||
)
|
||||
candidate_anchors.extend(anchors)
|
||||
|
||||
# Explicitly cleanup pool to avoid hangs in containerized environments
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: 🧹 Cleaning up pool...")
|
||||
pool.close()
|
||||
pool.terminate()
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Pool cleanup completed")
|
||||
|
||||
# Batch logging
|
||||
batch_results.append((n, len(anchors)))
|
||||
|
||||
# Log progress every batch_size results or on the last result
|
||||
if (i + 1) % batch_size == 0 or (i + 1) == len(n_gram_lengths):
|
||||
total_anchors_in_batch = sum(anchor_count for _, anchor_count in batch_results)
|
||||
n_gram_ranges = [str(ng) for ng, _ in batch_results]
|
||||
range_str = f"{n_gram_ranges[0]}-{n_gram_ranges[-1]}" if len(n_gram_ranges) > 1 else n_gram_ranges[0]
|
||||
self.logger.debug(f"🔍 ANCHOR SEARCH: Completed n-gram lengths {range_str} - found {total_anchors_in_batch} anchors")
|
||||
batch_results = []
|
||||
|
||||
except AnchorSequenceTimeoutError:
|
||||
self.logger.error(f"🔍 ANCHOR SEARCH: ❌ Parallel processing timed out")
|
||||
# Re-raise timeout errors
|
||||
raise
|
||||
except Exception as e:
|
||||
self.logger.error(f"🔍 ANCHOR SEARCH: ❌ Parallel processing failed: {str(e)}")
|
||||
# Fall back to sequential processing with timeout checks
|
||||
self.logger.info("🔍 ANCHOR SEARCH: 🔄 Falling back to sequential processing")
|
||||
for n in n_gram_lengths:
|
||||
try:
|
||||
# Check timeout more leniently during sequential processing
|
||||
if self.timeout_seconds > 0:
|
||||
elapsed_time = time.time() - start_time
|
||||
# Allow more time for sequential processing (up to 2x the original timeout)
|
||||
if elapsed_time > (self.timeout_seconds * 2.0):
|
||||
self.logger.warning(f"🔍 ANCHOR SEARCH: ⏰ Sequential processing timeout for n-gram {n}")
|
||||
break
|
||||
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: 🔄 Sequential processing n-gram length {n}")
|
||||
|
||||
anchors = self._process_ngram_length(
|
||||
n, trans_words, all_words, ref_texts_clean, ref_words, self.min_sources
|
||||
)
|
||||
candidate_anchors.extend(anchors)
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Sequential n-gram {n} completed - found {len(anchors)} anchors")
|
||||
except Exception as e:
|
||||
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ Sequential processing failed for n-gram length {n}: {str(e)}")
|
||||
continue
|
||||
finally:
|
||||
# Always ensure pool is cleaned up to avoid hangs in containerized environments
|
||||
if pool is not None:
|
||||
try:
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: 🧹 Final pool cleanup...")
|
||||
pool.terminate()
|
||||
pool.join(timeout=5) # Wait max 5 seconds for workers to terminate
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Final pool cleanup completed")
|
||||
except Exception as cleanup_error:
|
||||
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ Pool cleanup error (ignored): {cleanup_error}")
|
||||
except Exception as e:
|
||||
self.logger.warning(f"🔍 ANCHOR SEARCH: ⚠️ n-gram length {n} failed: {str(e)}")
|
||||
batch_results.append((n, 0))
|
||||
continue
|
||||
|
||||
self.logger.info(f"🔍 ANCHOR SEARCH: ✅ Found {len(candidate_anchors)} candidate anchors in {time.time() - start_time:.1f}s")
|
||||
|
||||
@@ -669,123 +593,75 @@ class AnchorSequenceFinder:
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
# Try different pool sizes with timeout
|
||||
num_processes = max(cpu_count() - 1, 1) # Leave one CPU free
|
||||
self.logger.info(f"🔍 FILTERING: Using {num_processes} processes for scoring")
|
||||
|
||||
# Create a partial function with the context parameter fixed
|
||||
score_anchor_partial = partial(self._score_anchor_static, context=context)
|
||||
|
||||
# Use multiprocessing to score anchors in parallel with timeout
|
||||
# Score anchors sequentially using simple rule-based scoring
|
||||
# (Avoids expensive spaCy NLP and works in cloud environments)
|
||||
scored_anchors = []
|
||||
pool_timeout = 300 # 5 minutes for scoring phase
|
||||
|
||||
scoring_pool = None
|
||||
try:
|
||||
self.logger.info(f"🔍 FILTERING: 🚀 Starting parallel scoring with timeout {pool_timeout}s")
|
||||
scoring_pool = Pool(processes=num_processes)
|
||||
# Submit scoring jobs with timeout
|
||||
async_results = []
|
||||
batch_size = 50
|
||||
|
||||
self.logger.info(f"🔍 FILTERING: Splitting {len(anchors)} anchors into batches of {batch_size}")
|
||||
for i in range(0, len(anchors), batch_size):
|
||||
batch = anchors[i:i + batch_size]
|
||||
async_result = scoring_pool.apply_async(self._score_batch_static, (batch, context))
|
||||
async_results.append(async_result)
|
||||
|
||||
self.logger.info(f"🔍 FILTERING: Submitted {len(async_results)} scoring batches")
|
||||
|
||||
# Collect results with timeout
|
||||
for i, async_result in enumerate(async_results):
|
||||
try:
|
||||
self.logger.debug(f"🔍 FILTERING: ⏳ Collecting batch {i+1}/{len(async_results)}")
|
||||
batch_results = async_result.get(timeout=pool_timeout)
|
||||
scored_anchors.extend(batch_results)
|
||||
self.logger.debug(f"🔍 FILTERING: ✅ Completed scoring batch {i+1}/{len(async_results)}")
|
||||
except Exception as e:
|
||||
self.logger.warning(f"🔍 FILTERING: ⚠️ Scoring batch {i+1} failed or timed out: {str(e)}")
|
||||
# Add basic scores for failed batch
|
||||
start_idx = i * batch_size
|
||||
end_idx = min((i + 1) * batch_size, len(anchors))
|
||||
for j in range(start_idx, end_idx):
|
||||
if j < len(anchors):
|
||||
try:
|
||||
phrase_score = PhraseScore(
|
||||
total_score=1.0,
|
||||
natural_break_score=1.0,
|
||||
phrase_type=PhraseType.COMPLETE
|
||||
)
|
||||
scored_anchors.append(ScoredAnchor(anchor=anchors[j], phrase_score=phrase_score))
|
||||
except:
|
||||
continue
|
||||
|
||||
# Explicitly cleanup pool to avoid hangs in containerized environments
|
||||
self.logger.info(f"🔍 FILTERING: 🧹 Cleaning up scoring pool...")
|
||||
scoring_pool.close()
|
||||
scoring_pool.terminate()
|
||||
self.logger.info(f"🔍 FILTERING: ✅ Scoring pool cleanup completed")
|
||||
|
||||
except Exception as e:
|
||||
self.logger.warning(f"🔍 FILTERING: ❌ Parallel scoring failed: {str(e)}, falling back to basic scoring")
|
||||
# Fall back to basic scoring
|
||||
for anchor in anchors:
|
||||
try:
|
||||
phrase_score = PhraseScore(
|
||||
total_score=1.0,
|
||||
natural_break_score=1.0,
|
||||
phrase_type=PhraseType.COMPLETE
|
||||
)
|
||||
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
|
||||
except:
|
||||
continue
|
||||
finally:
|
||||
# Always ensure scoring pool is cleaned up to avoid hangs
|
||||
if scoring_pool is not None:
|
||||
try:
|
||||
self.logger.info(f"🔍 FILTERING: 🧹 Final scoring pool cleanup...")
|
||||
scoring_pool.terminate()
|
||||
scoring_pool.join(timeout=5) # Wait max 5 seconds for workers to terminate
|
||||
self.logger.info(f"🔍 FILTERING: ✅ Final scoring pool cleanup completed")
|
||||
except Exception as cleanup_error:
|
||||
self.logger.warning(f"🔍 FILTERING: ⚠️ Scoring pool cleanup error (ignored): {cleanup_error}")
|
||||
self.logger.debug(f"🔍 FILTERING: Scoring {len(anchors)} anchors sequentially")
|
||||
|
||||
parallel_time = time.time() - start_time
|
||||
self.logger.info(f"🔍 FILTERING: ✅ Parallel scoring completed in {parallel_time:.2f}s, scored {len(scored_anchors)} anchors")
|
||||
for i, anchor in enumerate(anchors):
|
||||
try:
|
||||
# Simple rule-based scoring based on anchor properties
|
||||
phrase_score = self._simple_score_anchor(anchor)
|
||||
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
|
||||
except Exception as e:
|
||||
# Fallback to default score on error
|
||||
self.logger.debug(f"🔍 FILTERING: Scoring failed for anchor {i}: {e}")
|
||||
phrase_score = PhraseScore(
|
||||
phrase_type=PhraseType.COMPLETE,
|
||||
natural_break_score=1.0,
|
||||
length_score=1.0
|
||||
)
|
||||
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
|
||||
|
||||
# Sort and filter as before
|
||||
self.logger.info(f"🔍 FILTERING: 🔄 Sorting anchors by priority...")
|
||||
scoring_time = time.time() - start_time
|
||||
self.logger.debug(f"🔍 FILTERING: Scoring completed in {scoring_time:.2f}s, scored {len(scored_anchors)} anchors")
|
||||
|
||||
# Sort anchors by priority (highest first)
|
||||
self.logger.debug(f"🔍 FILTERING: Sorting anchors by priority...")
|
||||
scored_anchors.sort(key=self._get_sequence_priority, reverse=True)
|
||||
self.logger.info(f"🔍 FILTERING: ✅ Sorting completed")
|
||||
|
||||
self.logger.info(f"🔍 FILTERING: 🔄 Filtering {len(scored_anchors)} overlapping sequences")
|
||||
# O(N) overlap filtering using covered positions set
|
||||
self.logger.debug(f"🔍 FILTERING: Filtering {len(scored_anchors)} overlapping sequences")
|
||||
filtered_scored = []
|
||||
|
||||
for i, scored_anchor in enumerate(scored_anchors):
|
||||
# Check timeout every 100 anchors using our timeout mechanism (more lenient)
|
||||
if i % 100 == 0 and i > 0:
|
||||
# Only check timeout if we're significantly over the limit
|
||||
if self.timeout_seconds > 0:
|
||||
elapsed_time = time.time() - start_time
|
||||
# Use a more lenient timeout for filtering (allow 50% more time)
|
||||
if elapsed_time > (self.timeout_seconds * 1.5):
|
||||
self.logger.warning(f"🔍 FILTERING: ⏰ Filtering timed out, returning {len(filtered_scored)} anchors out of {len(scored_anchors)}")
|
||||
break
|
||||
|
||||
self.logger.debug(f"🔍 FILTERING: Progress: {i}/{len(scored_anchors)} processed, {len(filtered_scored)} kept")
|
||||
covered_positions: Set[int] = set()
|
||||
|
||||
for scored_anchor in scored_anchors:
|
||||
anchor = scored_anchor.anchor
|
||||
start_pos = anchor.transcription_position
|
||||
end_pos = start_pos + anchor.length
|
||||
|
||||
overlaps = False
|
||||
for existing in filtered_scored:
|
||||
if self._sequences_overlap(scored_anchor.anchor, existing.anchor):
|
||||
overlaps = True
|
||||
break
|
||||
|
||||
if not overlaps:
|
||||
# Check if any position in this anchor's range is already covered
|
||||
anchor_positions = set(range(start_pos, end_pos))
|
||||
if not anchor_positions & covered_positions: # No overlap with covered
|
||||
filtered_scored.append(scored_anchor)
|
||||
covered_positions.update(anchor_positions)
|
||||
|
||||
self.logger.info(f"🔍 FILTERING: ✅ Filtering completed - kept {len(filtered_scored)} non-overlapping anchors out of {len(scored_anchors)}")
|
||||
self.logger.debug(f"🔍 FILTERING: Kept {len(filtered_scored)} non-overlapping anchors out of {len(scored_anchors)}")
|
||||
return filtered_scored
|
||||
|
||||
def _simple_score_anchor(self, anchor: AnchorSequence) -> PhraseScore:
|
||||
"""
|
||||
Simple rule-based scoring for anchors without expensive NLP.
|
||||
|
||||
Scoring criteria:
|
||||
- Longer sequences are preferred (length_score)
|
||||
- Sequences matching more reference sources are preferred (natural_break_score)
|
||||
- All sequences treated as COMPLETE type for simplicity
|
||||
"""
|
||||
# Length score: normalize to 0-1 range (3-15 words typical)
|
||||
length = anchor.length
|
||||
length_score = min(1.0, (length - 2) / 10.0) # 3 words = 0.1, 12 words = 1.0
|
||||
|
||||
# Source match score: more sources = higher score
|
||||
num_sources = len(anchor.reference_positions)
|
||||
natural_break_score = min(1.0, num_sources / 3.0) # 1 source = 0.33, 3+ sources = 1.0
|
||||
|
||||
return PhraseScore(
|
||||
phrase_type=PhraseType.COMPLETE,
|
||||
natural_break_score=natural_break_score,
|
||||
length_score=length_score
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _score_anchor_static(anchor: AnchorSequence, context: str) -> ScoredAnchor:
|
||||
"""Static version of _score_anchor for multiprocessing compatibility."""
|
||||
@@ -816,9 +692,9 @@ class AnchorSequenceFinder:
|
||||
except Exception:
|
||||
# Add basic score for failed anchor
|
||||
phrase_score = PhraseScore(
|
||||
total_score=1.0,
|
||||
phrase_type=PhraseType.COMPLETE,
|
||||
natural_break_score=1.0,
|
||||
phrase_type=PhraseType.COMPLETE
|
||||
length_score=1.0
|
||||
)
|
||||
scored_anchors.append(ScoredAnchor(anchor=anchor, phrase_score=phrase_score))
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"name": "lyrics-transcriber-frontend",
|
||||
"private": true,
|
||||
"homepage": "https://nomadkaraoke.github.io/lyrics-transcriber-frontend",
|
||||
"version": "0.80.0",
|
||||
"version": "0.82.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { Box, Button, Typography, useMediaQuery, useTheme, Switch, FormControlLabel, Tooltip, Paper, IconButton } from '@mui/material'
|
||||
import { Box, Button, Typography, useMediaQuery, useTheme, Switch, FormControlLabel, Tooltip, Paper, IconButton, Chip } from '@mui/material'
|
||||
import LockIcon from '@mui/icons-material/Lock'
|
||||
import UploadFileIcon from '@mui/icons-material/UploadFile'
|
||||
import FindReplaceIcon from '@mui/icons-material/FindReplace'
|
||||
@@ -7,6 +7,7 @@ import UndoIcon from '@mui/icons-material/Undo'
|
||||
import RedoIcon from '@mui/icons-material/Redo'
|
||||
import TimerIcon from '@mui/icons-material/Timer'
|
||||
import RestoreIcon from '@mui/icons-material/Restore'
|
||||
import RateReviewIcon from '@mui/icons-material/RateReview'
|
||||
import { CorrectionData, InteractionMode } from '../types'
|
||||
import CorrectionMetrics from './CorrectionMetrics'
|
||||
import AgenticCorrectionMetrics from './AgenticCorrectionMetrics'
|
||||
@@ -41,6 +42,8 @@ interface HeaderProps {
|
||||
onRedo: () => void
|
||||
canUndo: boolean
|
||||
canRedo: boolean
|
||||
annotationsEnabled?: boolean
|
||||
onAnnotationsToggle?: (enabled: boolean) => void
|
||||
}
|
||||
|
||||
export default function Header({
|
||||
@@ -65,6 +68,8 @@ export default function Header({
|
||||
onRedo,
|
||||
canUndo,
|
||||
canRedo,
|
||||
annotationsEnabled = true,
|
||||
onAnnotationsToggle,
|
||||
}: HeaderProps) {
|
||||
const theme = useTheme()
|
||||
const isMobile = useMediaQuery(theme.breakpoints.down('md'))
|
||||
@@ -135,17 +140,38 @@ export default function Header({
|
||||
<Typography variant="h4" sx={{ fontSize: isMobile ? '1.3rem' : '1.5rem' }}>
|
||||
Nomad Karaoke: Lyrics Transcription Review
|
||||
</Typography>
|
||||
{isReadOnly && (
|
||||
<Button
|
||||
variant="outlined"
|
||||
size="small"
|
||||
startIcon={<UploadFileIcon />}
|
||||
onClick={onFileLoad}
|
||||
fullWidth={isMobile}
|
||||
>
|
||||
Load File
|
||||
</Button>
|
||||
)}
|
||||
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
|
||||
{!isReadOnly && onAnnotationsToggle && (
|
||||
<Tooltip title={annotationsEnabled
|
||||
? "Click to disable annotation prompts when editing"
|
||||
: "Click to enable annotation prompts when editing"
|
||||
}>
|
||||
<Chip
|
||||
icon={<RateReviewIcon />}
|
||||
label={annotationsEnabled ? "Feedback On" : "Feedback Off"}
|
||||
onClick={() => onAnnotationsToggle(!annotationsEnabled)}
|
||||
color={annotationsEnabled ? "primary" : "default"}
|
||||
variant={annotationsEnabled ? "filled" : "outlined"}
|
||||
size="small"
|
||||
sx={{
|
||||
cursor: 'pointer',
|
||||
'& .MuiChip-icon': { fontSize: '1rem' }
|
||||
}}
|
||||
/>
|
||||
</Tooltip>
|
||||
)}
|
||||
{isReadOnly && (
|
||||
<Button
|
||||
variant="outlined"
|
||||
size="small"
|
||||
startIcon={<UploadFileIcon />}
|
||||
onClick={onFileLoad}
|
||||
fullWidth={isMobile}
|
||||
>
|
||||
Load File
|
||||
</Button>
|
||||
)}
|
||||
</Box>
|
||||
</Box>
|
||||
|
||||
<Box sx={{
|
||||
|
||||
@@ -200,6 +200,8 @@ interface MemoizedHeaderProps {
|
||||
canUndo: boolean
|
||||
canRedo: boolean
|
||||
onUnCorrectAll: () => void
|
||||
annotationsEnabled: boolean
|
||||
onAnnotationsToggle: (enabled: boolean) => void
|
||||
}
|
||||
|
||||
// Create a memoized Header component
|
||||
@@ -224,7 +226,9 @@ const MemoizedHeader = memo(function MemoizedHeader({
|
||||
onRedo,
|
||||
canUndo,
|
||||
canRedo,
|
||||
onUnCorrectAll
|
||||
onUnCorrectAll,
|
||||
annotationsEnabled,
|
||||
onAnnotationsToggle
|
||||
}: MemoizedHeaderProps) {
|
||||
return (
|
||||
<Header
|
||||
@@ -249,6 +253,8 @@ const MemoizedHeader = memo(function MemoizedHeader({
|
||||
canUndo={canUndo}
|
||||
canRedo={canRedo}
|
||||
onUnCorrectAll={onUnCorrectAll}
|
||||
annotationsEnabled={annotationsEnabled}
|
||||
onAnnotationsToggle={onAnnotationsToggle}
|
||||
/>
|
||||
);
|
||||
});
|
||||
@@ -295,13 +301,18 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
|
||||
wordIdsAffected: string[]
|
||||
gapId?: string
|
||||
} | null>(null)
|
||||
const [annotationsEnabled] = useState(() => {
|
||||
const [annotationsEnabled, setAnnotationsEnabled] = useState(() => {
|
||||
// Check localStorage for user preference
|
||||
const saved = localStorage.getItem('annotationsEnabled')
|
||||
return saved !== null ? saved === 'true' : true // Default: enabled
|
||||
// TODO: Add UI toggle to enable/disable via setAnnotationsEnabled
|
||||
})
|
||||
|
||||
// Persist annotation preference to localStorage
|
||||
const handleAnnotationsToggle = useCallback((enabled: boolean) => {
|
||||
setAnnotationsEnabled(enabled)
|
||||
localStorage.setItem('annotationsEnabled', String(enabled))
|
||||
}, [])
|
||||
|
||||
// Correction detail card state
|
||||
const [correctionDetailOpen, setCorrectionDetailOpen] = useState(false)
|
||||
const [selectedCorrection, setSelectedCorrection] = useState<{
|
||||
@@ -1164,6 +1175,8 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
|
||||
canUndo={canUndo}
|
||||
canRedo={canRedo}
|
||||
onUnCorrectAll={handleUnCorrectAll}
|
||||
annotationsEnabled={annotationsEnabled}
|
||||
onAnnotationsToggle={handleAnnotationsToggle}
|
||||
/>
|
||||
|
||||
<Grid container direction={isMobile ? 'column' : 'row'}>
|
||||
@@ -1313,6 +1326,7 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
|
||||
onPlaySegment={handlePlaySegment}
|
||||
currentTime={currentAudioTime}
|
||||
setModalSpacebarHandler={handleSetModalSpacebarHandler}
|
||||
existingSegments={data.corrected_segments}
|
||||
/>
|
||||
|
||||
{pendingAnnotation && (
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
import { memo } from 'react'
|
||||
import {
|
||||
Box,
|
||||
Button,
|
||||
Divider,
|
||||
Stack
|
||||
} from '@mui/material'
|
||||
import PlayCircleOutlineIcon from '@mui/icons-material/PlayCircleOutline'
|
||||
import StopIcon from '@mui/icons-material/Stop'
|
||||
import PauseIcon from '@mui/icons-material/Pause'
|
||||
import PlayArrowIcon from '@mui/icons-material/PlayArrow'
|
||||
import ClearAllIcon from '@mui/icons-material/ClearAll'
|
||||
import EditNoteIcon from '@mui/icons-material/EditNote'
|
||||
import BlockIcon from '@mui/icons-material/Block'
|
||||
import EditIcon from '@mui/icons-material/Edit'
|
||||
import DeleteIcon from '@mui/icons-material/Delete'
|
||||
|
||||
interface SyncControlsProps {
|
||||
// Main buttons
|
||||
isManualSyncing: boolean
|
||||
isPaused: boolean
|
||||
onStartSync: () => void
|
||||
onPauseSync: () => void
|
||||
onResumeSync: () => void
|
||||
onClearSync: () => void
|
||||
onEditLyrics: () => void
|
||||
|
||||
// Playback controls
|
||||
onPlay: () => void
|
||||
onStop: () => void
|
||||
isPlaying: boolean
|
||||
|
||||
// Word action buttons
|
||||
hasSelectedWords: boolean
|
||||
selectedWordCount: number
|
||||
onUnsyncFromCursor: () => void
|
||||
onEditSelectedWord: () => void
|
||||
onDeleteSelected: () => void
|
||||
|
||||
// Additional state
|
||||
canUnsyncFromCursor: boolean
|
||||
}
|
||||
|
||||
const SyncControls = memo(function SyncControls({
|
||||
isManualSyncing,
|
||||
isPaused,
|
||||
onStartSync,
|
||||
onPauseSync,
|
||||
onResumeSync,
|
||||
onClearSync,
|
||||
onEditLyrics,
|
||||
onPlay,
|
||||
onStop,
|
||||
isPlaying,
|
||||
hasSelectedWords,
|
||||
selectedWordCount,
|
||||
onUnsyncFromCursor,
|
||||
onEditSelectedWord,
|
||||
onDeleteSelected,
|
||||
canUnsyncFromCursor
|
||||
}: SyncControlsProps) {
|
||||
return (
|
||||
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 1.5 }}>
|
||||
{/* Row 1: Playback & Sync Mode Controls */}
|
||||
<Stack direction="row" spacing={1} alignItems="center" flexWrap="wrap">
|
||||
{/* Playback controls */}
|
||||
<Button
|
||||
variant="outlined"
|
||||
color="primary"
|
||||
onClick={onPlay}
|
||||
startIcon={<PlayArrowIcon />}
|
||||
size="small"
|
||||
disabled={isPlaying}
|
||||
>
|
||||
Play
|
||||
</Button>
|
||||
<Button
|
||||
variant="outlined"
|
||||
color="error"
|
||||
onClick={onStop}
|
||||
startIcon={<StopIcon />}
|
||||
size="small"
|
||||
disabled={!isPlaying}
|
||||
>
|
||||
Stop
|
||||
</Button>
|
||||
|
||||
<Divider orientation="vertical" flexItem sx={{ mx: 0.5 }} />
|
||||
|
||||
{/* Sync Mode controls */}
|
||||
{isManualSyncing ? (
|
||||
<>
|
||||
<Button
|
||||
variant="contained"
|
||||
color="error"
|
||||
onClick={onStartSync}
|
||||
startIcon={<StopIcon />}
|
||||
size="small"
|
||||
>
|
||||
Stop Sync
|
||||
</Button>
|
||||
<Button
|
||||
variant="outlined"
|
||||
color={isPaused ? 'success' : 'warning'}
|
||||
onClick={isPaused ? onResumeSync : onPauseSync}
|
||||
startIcon={isPaused ? <PlayArrowIcon /> : <PauseIcon />}
|
||||
size="small"
|
||||
>
|
||||
{isPaused ? 'Resume' : 'Pause'}
|
||||
</Button>
|
||||
</>
|
||||
) : (
|
||||
<Button
|
||||
variant="contained"
|
||||
color="primary"
|
||||
onClick={onStartSync}
|
||||
startIcon={<PlayCircleOutlineIcon />}
|
||||
size="small"
|
||||
>
|
||||
Start Sync
|
||||
</Button>
|
||||
)}
|
||||
</Stack>
|
||||
|
||||
{/* Row 2: Editing & Word Actions */}
|
||||
<Stack direction="row" spacing={1} alignItems="center" flexWrap="wrap">
|
||||
<Button
|
||||
variant="outlined"
|
||||
color="warning"
|
||||
onClick={onClearSync}
|
||||
startIcon={<ClearAllIcon />}
|
||||
size="small"
|
||||
disabled={isManualSyncing && !isPaused}
|
||||
>
|
||||
Clear Sync
|
||||
</Button>
|
||||
|
||||
<Button
|
||||
variant="outlined"
|
||||
onClick={onEditLyrics}
|
||||
startIcon={<EditNoteIcon />}
|
||||
size="small"
|
||||
disabled={isManualSyncing && !isPaused}
|
||||
>
|
||||
Edit Lyrics
|
||||
</Button>
|
||||
|
||||
<Divider orientation="vertical" flexItem sx={{ mx: 0.5 }} />
|
||||
|
||||
<Button
|
||||
variant="outlined"
|
||||
onClick={onUnsyncFromCursor}
|
||||
startIcon={<BlockIcon />}
|
||||
size="small"
|
||||
disabled={!canUnsyncFromCursor || (isManualSyncing && !isPaused)}
|
||||
>
|
||||
Unsync from Cursor
|
||||
</Button>
|
||||
|
||||
<Button
|
||||
variant="outlined"
|
||||
onClick={onEditSelectedWord}
|
||||
startIcon={<EditIcon />}
|
||||
size="small"
|
||||
disabled={!hasSelectedWords || selectedWordCount !== 1}
|
||||
>
|
||||
Edit Word
|
||||
</Button>
|
||||
|
||||
<Button
|
||||
variant="outlined"
|
||||
color="error"
|
||||
onClick={onDeleteSelected}
|
||||
startIcon={<DeleteIcon />}
|
||||
size="small"
|
||||
disabled={!hasSelectedWords || (isManualSyncing && !isPaused)}
|
||||
>
|
||||
Delete{hasSelectedWords && selectedWordCount > 0 ? ` (${selectedWordCount})` : ''}
|
||||
</Button>
|
||||
</Stack>
|
||||
</Box>
|
||||
)
|
||||
})
|
||||
|
||||
export default SyncControls
|
||||
@@ -0,0 +1,704 @@
|
||||
import { useRef, useEffect, useCallback, useState, memo } from 'react'
|
||||
import { Box, IconButton, Tooltip } from '@mui/material'
|
||||
import ArrowBackIcon from '@mui/icons-material/ArrowBack'
|
||||
import ArrowForwardIcon from '@mui/icons-material/ArrowForward'
|
||||
import { Word, LyricsSegment } from '../../types'
|
||||
|
||||
interface TimelineCanvasProps {
|
||||
words: Word[]
|
||||
segments: LyricsSegment[]
|
||||
visibleStartTime: number
|
||||
visibleEndTime: number
|
||||
currentTime: number
|
||||
selectedWordIds: Set<string>
|
||||
onWordClick: (wordId: string, event: React.MouseEvent) => void
|
||||
onBackgroundClick: () => void
|
||||
onTimeBarClick: (time: number) => void
|
||||
onSelectionComplete: (wordIds: string[]) => void
|
||||
onWordTimingChange: (wordId: string, newStartTime: number, newEndTime: number) => void
|
||||
onWordsMove: (updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }>) => void
|
||||
syncWordIndex: number
|
||||
isManualSyncing: boolean
|
||||
onScrollChange: (newStartTime: number) => void
|
||||
audioDuration: number
|
||||
zoomSeconds: number
|
||||
height?: number
|
||||
}
|
||||
|
||||
// Constants for rendering
|
||||
const TIME_BAR_HEIGHT = 28
|
||||
const WORD_BLOCK_HEIGHT = 24
|
||||
const WORD_LEVEL_SPACING = 50
|
||||
const CANVAS_PADDING = 8
|
||||
const TEXT_ABOVE_BLOCK = 14
|
||||
const RESIZE_HANDLE_SIZE = 8
|
||||
const RESIZE_HANDLE_HITAREA = 12
|
||||
const PLAYHEAD_COLOR = '#ffffff'
|
||||
const WORD_BLOCK_COLOR = '#d32f2f'
|
||||
const WORD_BLOCK_SELECTED_COLOR = '#b71c1c'
|
||||
const WORD_BLOCK_CURRENT_COLOR = '#f44336'
|
||||
const WORD_TEXT_CURRENT_COLOR = '#d32f2f'
|
||||
const UPCOMING_WORD_BG = '#fff9c4'
|
||||
const UPCOMING_WORD_TEXT = '#000000'
|
||||
const TIME_BAR_BG = '#f5f5f5'
|
||||
const TIME_BAR_TEXT = '#666666'
|
||||
const TIMELINE_BG = '#e0e0e0'
|
||||
|
||||
// Drag modes
|
||||
type DragMode = 'none' | 'selection' | 'resize' | 'move'
|
||||
|
||||
// Build a map of word ID to segment index
|
||||
function buildWordToSegmentMap(segments: LyricsSegment[]): Map<string, number> {
|
||||
const map = new Map<string, number>()
|
||||
segments.forEach((segment, idx) => {
|
||||
segment.words.forEach(word => {
|
||||
map.set(word.id, idx)
|
||||
})
|
||||
})
|
||||
return map
|
||||
}
|
||||
|
||||
// Calculate which vertical level a word should be on
|
||||
function calculateWordLevels(words: Word[], segments: LyricsSegment[]): Map<string, number> {
|
||||
const levels = new Map<string, number>()
|
||||
const wordToSegment = buildWordToSegmentMap(segments)
|
||||
|
||||
const segmentsWithTiming = segments
|
||||
.map((segment, idx) => {
|
||||
const timedWords = segment.words.filter(w => w.start_time !== null)
|
||||
const minStart = timedWords.length > 0
|
||||
? Math.min(...timedWords.map(w => w.start_time!))
|
||||
: Infinity
|
||||
return { idx, minStart }
|
||||
})
|
||||
.filter(s => s.minStart !== Infinity)
|
||||
.sort((a, b) => a.minStart - b.minStart)
|
||||
|
||||
const segmentLevels = new Map<number, number>()
|
||||
segmentsWithTiming.forEach(({ idx }, orderIndex) => {
|
||||
segmentLevels.set(idx, orderIndex % 2)
|
||||
})
|
||||
|
||||
for (const word of words) {
|
||||
const segmentIdx = wordToSegment.get(word.id)
|
||||
if (segmentIdx !== undefined && segmentLevels.has(segmentIdx)) {
|
||||
levels.set(word.id, segmentLevels.get(segmentIdx)!)
|
||||
} else {
|
||||
levels.set(word.id, 0)
|
||||
}
|
||||
}
|
||||
|
||||
return levels
|
||||
}
|
||||
|
||||
function formatTime(seconds: number): string {
|
||||
const mins = Math.floor(seconds / 60)
|
||||
const secs = Math.floor(seconds % 60)
|
||||
return `${mins.toString().padStart(2, '0')}:${secs.toString().padStart(2, '0')}`
|
||||
}
|
||||
|
||||
const TimelineCanvas = memo(function TimelineCanvas({
|
||||
words,
|
||||
segments,
|
||||
visibleStartTime,
|
||||
visibleEndTime,
|
||||
currentTime,
|
||||
selectedWordIds,
|
||||
onWordClick,
|
||||
onBackgroundClick,
|
||||
onTimeBarClick,
|
||||
onSelectionComplete,
|
||||
onWordTimingChange,
|
||||
onWordsMove,
|
||||
syncWordIndex,
|
||||
isManualSyncing,
|
||||
onScrollChange,
|
||||
audioDuration,
|
||||
zoomSeconds,
|
||||
height = 200
|
||||
}: TimelineCanvasProps) {
|
||||
const canvasRef = useRef<HTMLCanvasElement>(null)
|
||||
const containerRef = useRef<HTMLDivElement>(null)
|
||||
const [canvasWidth, setCanvasWidth] = useState(800)
|
||||
const animationFrameRef = useRef<number>()
|
||||
const wordLevelsRef = useRef<Map<string, number>>(new Map())
|
||||
|
||||
// Drag state
|
||||
const [dragMode, setDragMode] = useState<DragMode>('none')
|
||||
const dragStartRef = useRef<{ x: number; y: number; time: number } | null>(null)
|
||||
const dragWordIdRef = useRef<string | null>(null)
|
||||
const dragOriginalTimesRef = useRef<Map<string, { start: number; end: number }>>(new Map())
|
||||
|
||||
// Selection rectangle
|
||||
const [selectionRect, setSelectionRect] = useState<{
|
||||
startX: number; startY: number; endX: number; endY: number
|
||||
} | null>(null)
|
||||
|
||||
// Hover state for showing resize handle
|
||||
const [hoveredWordId, setHoveredWordId] = useState<string | null>(null)
|
||||
const [cursorStyle, setCursorStyle] = useState<string>('default')
|
||||
|
||||
// Update canvas width on resize
|
||||
useEffect(() => {
|
||||
const updateWidth = () => {
|
||||
if (containerRef.current) {
|
||||
setCanvasWidth(containerRef.current.clientWidth)
|
||||
}
|
||||
}
|
||||
|
||||
updateWidth()
|
||||
const resizeObserver = new ResizeObserver(updateWidth)
|
||||
if (containerRef.current) {
|
||||
resizeObserver.observe(containerRef.current)
|
||||
}
|
||||
|
||||
return () => resizeObserver.disconnect()
|
||||
}, [])
|
||||
|
||||
// Calculate word levels when words or segments change
|
||||
useEffect(() => {
|
||||
wordLevelsRef.current = calculateWordLevels(words, segments)
|
||||
}, [words, segments])
|
||||
|
||||
// Convert time to x position
|
||||
const timeToX = useCallback((time: number): number => {
|
||||
const duration = visibleEndTime - visibleStartTime
|
||||
if (duration <= 0) return 0
|
||||
return CANVAS_PADDING + ((time - visibleStartTime) / duration) * (canvasWidth - CANVAS_PADDING * 2)
|
||||
}, [visibleStartTime, visibleEndTime, canvasWidth])
|
||||
|
||||
// Convert x position to time
|
||||
const xToTime = useCallback((x: number): number => {
|
||||
const duration = visibleEndTime - visibleStartTime
|
||||
return visibleStartTime + ((x - CANVAS_PADDING) / (canvasWidth - CANVAS_PADDING * 2)) * duration
|
||||
}, [visibleStartTime, visibleEndTime, canvasWidth])
|
||||
|
||||
// Get word bounds
|
||||
const getWordBounds = useCallback((word: Word) => {
|
||||
if (word.start_time === null || word.end_time === null) return null
|
||||
|
||||
const level = wordLevelsRef.current.get(word.id) || 0
|
||||
const startX = timeToX(word.start_time)
|
||||
const endX = timeToX(word.end_time)
|
||||
const blockWidth = Math.max(endX - startX, 4)
|
||||
const y = TIME_BAR_HEIGHT + CANVAS_PADDING + TEXT_ABOVE_BLOCK + level * WORD_LEVEL_SPACING
|
||||
|
||||
return { startX, endX, blockWidth, y, level }
|
||||
}, [timeToX])
|
||||
|
||||
// Check if position is near resize handle
|
||||
const isNearResizeHandlePos = useCallback((word: Word, x: number, y: number): boolean => {
|
||||
const bounds = getWordBounds(word)
|
||||
if (!bounds) return false
|
||||
|
||||
const handleX = bounds.startX + bounds.blockWidth - RESIZE_HANDLE_SIZE / 2
|
||||
const handleY = bounds.y + WORD_BLOCK_HEIGHT / 2
|
||||
|
||||
return Math.abs(x - handleX) < RESIZE_HANDLE_HITAREA / 2 &&
|
||||
Math.abs(y - handleY) < RESIZE_HANDLE_HITAREA / 2
|
||||
}, [getWordBounds])
|
||||
|
||||
// Find word at position
|
||||
const findWordAtPosition = useCallback((x: number, y: number): Word | null => {
|
||||
for (const word of words) {
|
||||
const bounds = getWordBounds(word)
|
||||
if (!bounds) continue
|
||||
|
||||
if (x >= bounds.startX && x <= bounds.startX + bounds.blockWidth &&
|
||||
y >= bounds.y && y <= bounds.y + WORD_BLOCK_HEIGHT) {
|
||||
return word
|
||||
}
|
||||
}
|
||||
return null
|
||||
}, [words, getWordBounds])
|
||||
|
||||
// Find words in selection rectangle
|
||||
const findWordsInRect = useCallback((rect: { startX: number; startY: number; endX: number; endY: number }): string[] => {
|
||||
const rectLeft = Math.min(rect.startX, rect.endX)
|
||||
const rectRight = Math.max(rect.startX, rect.endX)
|
||||
const rectTop = Math.min(rect.startY, rect.endY)
|
||||
const rectBottom = Math.max(rect.startY, rect.endY)
|
||||
|
||||
const selectedIds: string[] = []
|
||||
|
||||
for (const word of words) {
|
||||
const bounds = getWordBounds(word)
|
||||
if (!bounds) continue
|
||||
|
||||
if (bounds.startX + bounds.blockWidth >= rectLeft && bounds.startX <= rectRight &&
|
||||
bounds.y + WORD_BLOCK_HEIGHT >= rectTop && bounds.y <= rectBottom) {
|
||||
selectedIds.push(word.id)
|
||||
}
|
||||
}
|
||||
|
||||
return selectedIds
|
||||
}, [words, getWordBounds])
|
||||
|
||||
// Draw the timeline
|
||||
const draw = useCallback(() => {
|
||||
const canvas = canvasRef.current
|
||||
if (!canvas) return
|
||||
|
||||
const ctx = canvas.getContext('2d')
|
||||
if (!ctx) return
|
||||
|
||||
const dpr = window.devicePixelRatio || 1
|
||||
canvas.width = canvasWidth * dpr
|
||||
canvas.height = height * dpr
|
||||
ctx.scale(dpr, dpr)
|
||||
|
||||
// Clear canvas
|
||||
ctx.fillStyle = TIMELINE_BG
|
||||
ctx.fillRect(0, 0, canvasWidth, height)
|
||||
|
||||
// Draw time bar background
|
||||
ctx.fillStyle = TIME_BAR_BG
|
||||
ctx.fillRect(0, 0, canvasWidth, TIME_BAR_HEIGHT)
|
||||
|
||||
// Draw time markers
|
||||
const duration = visibleEndTime - visibleStartTime
|
||||
const secondsPerTick = duration > 15 ? 2 : duration > 8 ? 1 : 0.5
|
||||
const startSecond = Math.ceil(visibleStartTime / secondsPerTick) * secondsPerTick
|
||||
|
||||
ctx.fillStyle = TIME_BAR_TEXT
|
||||
ctx.font = '11px system-ui, -apple-system, sans-serif'
|
||||
ctx.textAlign = 'center'
|
||||
|
||||
for (let t = startSecond; t <= visibleEndTime; t += secondsPerTick) {
|
||||
const x = timeToX(t)
|
||||
|
||||
ctx.beginPath()
|
||||
ctx.strokeStyle = '#999999'
|
||||
ctx.lineWidth = 1
|
||||
ctx.moveTo(x, TIME_BAR_HEIGHT - 6)
|
||||
ctx.lineTo(x, TIME_BAR_HEIGHT)
|
||||
ctx.stroke()
|
||||
|
||||
if (t % 1 === 0) {
|
||||
ctx.fillText(formatTime(t), x, TIME_BAR_HEIGHT - 10)
|
||||
}
|
||||
}
|
||||
|
||||
ctx.beginPath()
|
||||
ctx.strokeStyle = '#cccccc'
|
||||
ctx.lineWidth = 1
|
||||
ctx.moveTo(0, TIME_BAR_HEIGHT)
|
||||
ctx.lineTo(canvasWidth, TIME_BAR_HEIGHT)
|
||||
ctx.stroke()
|
||||
|
||||
const wordToSegment = buildWordToSegmentMap(segments)
|
||||
const syncedWords = words.filter(w => w.start_time !== null && w.end_time !== null)
|
||||
|
||||
const currentWordId = syncedWords.find(w =>
|
||||
currentTime >= w.start_time! && currentTime <= w.end_time!
|
||||
)?.id || null
|
||||
|
||||
// First pass: draw all blocks
|
||||
for (const word of syncedWords) {
|
||||
const bounds = getWordBounds(word)
|
||||
if (!bounds) continue
|
||||
|
||||
const isSelected = selectedWordIds.has(word.id)
|
||||
const isCurrent = word.id === currentWordId
|
||||
const isHovered = word.id === hoveredWordId
|
||||
|
||||
// Draw word block background
|
||||
if (isSelected) {
|
||||
ctx.fillStyle = WORD_BLOCK_SELECTED_COLOR
|
||||
} else if (isCurrent) {
|
||||
ctx.fillStyle = WORD_BLOCK_CURRENT_COLOR
|
||||
} else {
|
||||
ctx.fillStyle = WORD_BLOCK_COLOR
|
||||
}
|
||||
ctx.fillRect(bounds.startX, bounds.y, bounds.blockWidth, WORD_BLOCK_HEIGHT)
|
||||
|
||||
// Draw selection border
|
||||
if (isSelected) {
|
||||
ctx.strokeStyle = '#ffffff'
|
||||
ctx.lineWidth = 2
|
||||
ctx.strokeRect(bounds.startX, bounds.y, bounds.blockWidth, WORD_BLOCK_HEIGHT)
|
||||
|
||||
// Draw resize handle (white dot on right edge) for selected words when hovered
|
||||
if (isHovered || selectedWordIds.size === 1) {
|
||||
const handleX = bounds.startX + bounds.blockWidth - RESIZE_HANDLE_SIZE / 2
|
||||
const handleY = bounds.y + WORD_BLOCK_HEIGHT / 2
|
||||
|
||||
ctx.beginPath()
|
||||
ctx.fillStyle = '#ffffff'
|
||||
ctx.arc(handleX, handleY, RESIZE_HANDLE_SIZE / 2, 0, Math.PI * 2)
|
||||
ctx.fill()
|
||||
ctx.strokeStyle = '#666666'
|
||||
ctx.lineWidth = 1
|
||||
ctx.stroke()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Second pass: draw text
|
||||
const wordsBySegment = new Map<number, Word[]>()
|
||||
for (const word of syncedWords) {
|
||||
const segIdx = wordToSegment.get(word.id)
|
||||
if (segIdx !== undefined) {
|
||||
if (!wordsBySegment.has(segIdx)) {
|
||||
wordsBySegment.set(segIdx, [])
|
||||
}
|
||||
wordsBySegment.get(segIdx)!.push(word)
|
||||
}
|
||||
}
|
||||
|
||||
ctx.font = '11px system-ui, -apple-system, sans-serif'
|
||||
ctx.textAlign = 'left'
|
||||
|
||||
for (const [, segmentWords] of wordsBySegment) {
|
||||
const sortedWords = [...segmentWords].sort((a, b) =>
|
||||
(a.start_time || 0) - (b.start_time || 0)
|
||||
)
|
||||
|
||||
if (sortedWords.length === 0) continue
|
||||
|
||||
const level = wordLevelsRef.current.get(sortedWords[0].id) || 0
|
||||
const textY = TIME_BAR_HEIGHT + CANVAS_PADDING + TEXT_ABOVE_BLOCK + level * WORD_LEVEL_SPACING - 3
|
||||
|
||||
let rightmostTextEnd = -Infinity
|
||||
|
||||
for (const word of sortedWords) {
|
||||
const blockStartX = timeToX(word.start_time!)
|
||||
const textWidth = ctx.measureText(word.text).width
|
||||
const textStartX = Math.max(blockStartX, rightmostTextEnd + 3)
|
||||
|
||||
if (textStartX < canvasWidth - 10) {
|
||||
const isCurrent = word.id === currentWordId
|
||||
ctx.fillStyle = isCurrent ? WORD_TEXT_CURRENT_COLOR : '#333333'
|
||||
ctx.fillText(word.text, textStartX, textY)
|
||||
rightmostTextEnd = textStartX + textWidth
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Draw upcoming words during sync
|
||||
if (isManualSyncing && syncWordIndex >= 0) {
|
||||
const upcomingWords = words.slice(syncWordIndex).filter(w => w.start_time === null)
|
||||
const playheadX = timeToX(currentTime)
|
||||
let offsetX = playheadX + 10
|
||||
|
||||
ctx.font = '11px system-ui, -apple-system, sans-serif'
|
||||
|
||||
for (let i = 0; i < Math.min(upcomingWords.length, 12); i++) {
|
||||
const word = upcomingWords[i]
|
||||
const textWidth = ctx.measureText(word.text).width + 10
|
||||
|
||||
ctx.fillStyle = UPCOMING_WORD_BG
|
||||
ctx.fillRect(offsetX, TIME_BAR_HEIGHT + CANVAS_PADDING + WORD_LEVEL_SPACING + 60, textWidth, 20)
|
||||
|
||||
ctx.fillStyle = UPCOMING_WORD_TEXT
|
||||
ctx.textAlign = 'left'
|
||||
ctx.fillText(word.text, offsetX + 5, TIME_BAR_HEIGHT + CANVAS_PADDING + WORD_LEVEL_SPACING + 74)
|
||||
|
||||
offsetX += textWidth + 3
|
||||
if (offsetX > canvasWidth - 20) break
|
||||
}
|
||||
}
|
||||
|
||||
// Draw playhead
|
||||
if (currentTime >= visibleStartTime && currentTime <= visibleEndTime) {
|
||||
const playheadX = timeToX(currentTime)
|
||||
|
||||
ctx.beginPath()
|
||||
ctx.fillStyle = PLAYHEAD_COLOR
|
||||
ctx.strokeStyle = '#333333'
|
||||
ctx.lineWidth = 1
|
||||
ctx.moveTo(playheadX - 6, 2)
|
||||
ctx.lineTo(playheadX + 6, 2)
|
||||
ctx.lineTo(playheadX, TIME_BAR_HEIGHT - 4)
|
||||
ctx.closePath()
|
||||
ctx.fill()
|
||||
ctx.stroke()
|
||||
|
||||
ctx.beginPath()
|
||||
ctx.strokeStyle = PLAYHEAD_COLOR
|
||||
ctx.lineWidth = 2
|
||||
ctx.moveTo(playheadX, TIME_BAR_HEIGHT)
|
||||
ctx.lineTo(playheadX, height)
|
||||
ctx.stroke()
|
||||
|
||||
ctx.beginPath()
|
||||
ctx.strokeStyle = 'rgba(0,0,0,0.4)'
|
||||
ctx.lineWidth = 1
|
||||
ctx.moveTo(playheadX + 1, TIME_BAR_HEIGHT)
|
||||
ctx.lineTo(playheadX + 1, height)
|
||||
ctx.stroke()
|
||||
}
|
||||
|
||||
// Draw selection rectangle
|
||||
if (selectionRect) {
|
||||
ctx.fillStyle = 'rgba(25, 118, 210, 0.2)'
|
||||
ctx.strokeStyle = 'rgba(25, 118, 210, 0.8)'
|
||||
ctx.lineWidth = 1
|
||||
|
||||
const rectX = Math.min(selectionRect.startX, selectionRect.endX)
|
||||
const rectY = Math.min(selectionRect.startY, selectionRect.endY)
|
||||
const rectW = Math.abs(selectionRect.endX - selectionRect.startX)
|
||||
const rectH = Math.abs(selectionRect.endY - selectionRect.startY)
|
||||
|
||||
ctx.fillRect(rectX, rectY, rectW, rectH)
|
||||
ctx.strokeRect(rectX, rectY, rectW, rectH)
|
||||
}
|
||||
}, [
|
||||
canvasWidth, height, visibleStartTime, visibleEndTime, currentTime,
|
||||
words, segments, selectedWordIds, selectionRect, hoveredWordId,
|
||||
syncWordIndex, isManualSyncing, timeToX, getWordBounds
|
||||
])
|
||||
|
||||
// Animation frame
|
||||
useEffect(() => {
|
||||
const animate = () => {
|
||||
draw()
|
||||
animationFrameRef.current = requestAnimationFrame(animate)
|
||||
}
|
||||
animate()
|
||||
|
||||
return () => {
|
||||
if (animationFrameRef.current) {
|
||||
cancelAnimationFrame(animationFrameRef.current)
|
||||
}
|
||||
}
|
||||
}, [draw])
|
||||
|
||||
// Mouse handlers
|
||||
const handleMouseDown = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
|
||||
const rect = canvasRef.current?.getBoundingClientRect()
|
||||
if (!rect) return
|
||||
|
||||
const x = e.clientX - rect.left
|
||||
const y = e.clientY - rect.top
|
||||
const time = xToTime(x)
|
||||
|
||||
// Time bar click
|
||||
if (y < TIME_BAR_HEIGHT) {
|
||||
onTimeBarClick(Math.max(0, time))
|
||||
return
|
||||
}
|
||||
|
||||
const clickedWord = findWordAtPosition(x, y)
|
||||
|
||||
if (clickedWord && selectedWordIds.has(clickedWord.id)) {
|
||||
// Check if clicking on resize handle
|
||||
if (isNearResizeHandlePos(clickedWord, x, y)) {
|
||||
// Start resize
|
||||
setDragMode('resize')
|
||||
dragStartRef.current = { x, y, time }
|
||||
dragWordIdRef.current = clickedWord.id
|
||||
dragOriginalTimesRef.current = new Map([[clickedWord.id, {
|
||||
start: clickedWord.start_time!,
|
||||
end: clickedWord.end_time!
|
||||
}]])
|
||||
return
|
||||
}
|
||||
|
||||
// Start move (for all selected words)
|
||||
setDragMode('move')
|
||||
dragStartRef.current = { x, y, time }
|
||||
dragWordIdRef.current = clickedWord.id
|
||||
|
||||
// Store original times for all selected words
|
||||
const originalTimes = new Map<string, { start: number; end: number }>()
|
||||
for (const wordId of selectedWordIds) {
|
||||
const word = words.find(w => w.id === wordId)
|
||||
if (word && word.start_time !== null && word.end_time !== null) {
|
||||
originalTimes.set(wordId, { start: word.start_time, end: word.end_time })
|
||||
}
|
||||
}
|
||||
dragOriginalTimesRef.current = originalTimes
|
||||
return
|
||||
}
|
||||
|
||||
if (clickedWord) {
|
||||
// Click on unselected word - select it
|
||||
onWordClick(clickedWord.id, e)
|
||||
return
|
||||
}
|
||||
|
||||
// Background - start selection
|
||||
setDragMode('selection')
|
||||
dragStartRef.current = { x, y, time }
|
||||
setSelectionRect({ startX: x, startY: y, endX: x, endY: y })
|
||||
}, [xToTime, onTimeBarClick, findWordAtPosition, selectedWordIds, isNearResizeHandlePos, onWordClick, words])
|
||||
|
||||
const handleMouseMove = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
|
||||
const rect = canvasRef.current?.getBoundingClientRect()
|
||||
if (!rect) return
|
||||
|
||||
const x = e.clientX - rect.left
|
||||
const y = e.clientY - rect.top
|
||||
const time = xToTime(x)
|
||||
|
||||
// Update hover state and cursor
|
||||
if (dragMode === 'none') {
|
||||
const hoveredWord = findWordAtPosition(x, y)
|
||||
setHoveredWordId(hoveredWord?.id || null)
|
||||
|
||||
if (hoveredWord && selectedWordIds.has(hoveredWord.id)) {
|
||||
const nearHandle = isNearResizeHandlePos(hoveredWord, x, y)
|
||||
setCursorStyle(nearHandle ? 'ew-resize' : 'grab')
|
||||
} else if (hoveredWord) {
|
||||
setCursorStyle('pointer')
|
||||
} else if (y < TIME_BAR_HEIGHT) {
|
||||
setCursorStyle('pointer')
|
||||
} else {
|
||||
setCursorStyle('default')
|
||||
}
|
||||
}
|
||||
|
||||
if (!dragStartRef.current) return
|
||||
|
||||
if (dragMode === 'selection') {
|
||||
setSelectionRect({
|
||||
startX: dragStartRef.current.x,
|
||||
startY: dragStartRef.current.y,
|
||||
endX: x,
|
||||
endY: y
|
||||
})
|
||||
} else if (dragMode === 'resize' && dragWordIdRef.current) {
|
||||
// Resize the word
|
||||
const originalTimes = dragOriginalTimesRef.current.get(dragWordIdRef.current)
|
||||
if (originalTimes) {
|
||||
const deltaTime = time - dragStartRef.current.time
|
||||
const newEndTime = Math.max(originalTimes.start + 0.05, originalTimes.end + deltaTime)
|
||||
onWordTimingChange(dragWordIdRef.current, originalTimes.start, newEndTime)
|
||||
}
|
||||
setCursorStyle('ew-resize')
|
||||
} else if (dragMode === 'move') {
|
||||
// Move all selected words
|
||||
const deltaTime = time - dragStartRef.current.time
|
||||
const updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }> = []
|
||||
|
||||
for (const [wordId, originalTimes] of dragOriginalTimesRef.current) {
|
||||
// Ensure end time is always after start time (at least 0.05s duration)
|
||||
const newStartTime = Math.max(0, originalTimes.start + deltaTime)
|
||||
const newEndTime = Math.max(newStartTime + 0.05, originalTimes.end + deltaTime)
|
||||
updates.push({
|
||||
wordId,
|
||||
newStartTime,
|
||||
newEndTime
|
||||
})
|
||||
}
|
||||
|
||||
if (updates.length > 0) {
|
||||
onWordsMove(updates)
|
||||
}
|
||||
setCursorStyle('grabbing')
|
||||
}
|
||||
}, [dragMode, xToTime, findWordAtPosition, selectedWordIds, isNearResizeHandlePos, onWordTimingChange, onWordsMove])
|
||||
|
||||
const handleMouseUp = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
|
||||
const rect = canvasRef.current?.getBoundingClientRect()
|
||||
|
||||
if (dragMode === 'selection' && dragStartRef.current && rect) {
|
||||
const endX = e.clientX - rect.left
|
||||
const endY = e.clientY - rect.top
|
||||
|
||||
const dragDistance = Math.sqrt(
|
||||
Math.pow(endX - dragStartRef.current.x, 2) +
|
||||
Math.pow(endY - dragStartRef.current.y, 2)
|
||||
)
|
||||
|
||||
if (dragDistance < 5) {
|
||||
onBackgroundClick()
|
||||
} else {
|
||||
const finalRect = {
|
||||
startX: dragStartRef.current.x,
|
||||
startY: dragStartRef.current.y,
|
||||
endX,
|
||||
endY
|
||||
}
|
||||
const selectedIds = findWordsInRect(finalRect)
|
||||
if (selectedIds.length > 0) {
|
||||
onSelectionComplete(selectedIds)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Reset drag state
|
||||
setDragMode('none')
|
||||
dragStartRef.current = null
|
||||
dragWordIdRef.current = null
|
||||
dragOriginalTimesRef.current = new Map()
|
||||
setSelectionRect(null)
|
||||
setCursorStyle('default')
|
||||
}, [dragMode, onBackgroundClick, findWordsInRect, onSelectionComplete])
|
||||
|
||||
// Wheel handler
|
||||
const handleWheel = useCallback((e: React.WheelEvent<HTMLCanvasElement>) => {
|
||||
const delta = e.deltaX !== 0 ? e.deltaX : e.deltaY
|
||||
const scrollAmount = (delta / 100) * (zoomSeconds / 4)
|
||||
let newStart = Math.max(0, Math.min(audioDuration - zoomSeconds, visibleStartTime + scrollAmount))
|
||||
|
||||
if (newStart !== visibleStartTime) {
|
||||
onScrollChange(newStart)
|
||||
}
|
||||
}, [visibleStartTime, zoomSeconds, audioDuration, onScrollChange])
|
||||
|
||||
const handleScrollLeft = useCallback(() => {
|
||||
const newStart = Math.max(0, visibleStartTime - zoomSeconds * 0.25)
|
||||
onScrollChange(newStart)
|
||||
}, [visibleStartTime, zoomSeconds, onScrollChange])
|
||||
|
||||
const handleScrollRight = useCallback(() => {
|
||||
const newStart = Math.min(audioDuration - zoomSeconds, visibleStartTime + zoomSeconds * 0.25)
|
||||
onScrollChange(Math.max(0, newStart))
|
||||
}, [visibleStartTime, zoomSeconds, audioDuration, onScrollChange])
|
||||
|
||||
return (
|
||||
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 0.5 }}>
|
||||
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
|
||||
<Tooltip title="Scroll Left">
|
||||
<IconButton
|
||||
size="small"
|
||||
onClick={handleScrollLeft}
|
||||
disabled={visibleStartTime <= 0}
|
||||
>
|
||||
<ArrowBackIcon fontSize="small" />
|
||||
</IconButton>
|
||||
</Tooltip>
|
||||
|
||||
<Box
|
||||
ref={containerRef}
|
||||
sx={{
|
||||
flexGrow: 1,
|
||||
height,
|
||||
cursor: cursorStyle,
|
||||
borderRadius: 1,
|
||||
overflow: 'hidden'
|
||||
}}
|
||||
>
|
||||
<canvas
|
||||
ref={canvasRef}
|
||||
style={{
|
||||
width: '100%',
|
||||
height: '100%',
|
||||
display: 'block',
|
||||
cursor: cursorStyle
|
||||
}}
|
||||
onMouseDown={handleMouseDown}
|
||||
onMouseMove={handleMouseMove}
|
||||
onMouseUp={handleMouseUp}
|
||||
onMouseLeave={handleMouseUp}
|
||||
onWheel={handleWheel}
|
||||
/>
|
||||
</Box>
|
||||
|
||||
<Tooltip title="Scroll Right">
|
||||
<IconButton
|
||||
size="small"
|
||||
onClick={handleScrollRight}
|
||||
disabled={visibleStartTime >= audioDuration - zoomSeconds}
|
||||
>
|
||||
<ArrowForwardIcon fontSize="small" />
|
||||
</IconButton>
|
||||
</Tooltip>
|
||||
</Box>
|
||||
</Box>
|
||||
)
|
||||
})
|
||||
|
||||
export default TimelineCanvas
|
||||
@@ -0,0 +1,80 @@
|
||||
import { memo, useMemo } from 'react'
|
||||
import { Box, Typography } from '@mui/material'
|
||||
import { Word } from '../../types'
|
||||
|
||||
interface UpcomingWordsBarProps {
|
||||
words: Word[]
|
||||
syncWordIndex: number
|
||||
isManualSyncing: boolean
|
||||
maxWordsToShow?: number
|
||||
}
|
||||
|
||||
const UpcomingWordsBar = memo(function UpcomingWordsBar({
|
||||
words,
|
||||
syncWordIndex,
|
||||
isManualSyncing,
|
||||
maxWordsToShow = 20
|
||||
}: UpcomingWordsBarProps) {
|
||||
// Get upcoming unsynced words
|
||||
const upcomingWords = useMemo(() => {
|
||||
if (!isManualSyncing || syncWordIndex < 0) return []
|
||||
|
||||
return words
|
||||
.slice(syncWordIndex)
|
||||
.filter(w => w.start_time === null)
|
||||
.slice(0, maxWordsToShow)
|
||||
}, [words, syncWordIndex, isManualSyncing, maxWordsToShow])
|
||||
|
||||
const totalRemaining = useMemo(() => {
|
||||
if (!isManualSyncing || syncWordIndex < 0) return 0
|
||||
return words.slice(syncWordIndex).filter(w => w.start_time === null).length
|
||||
}, [words, syncWordIndex, isManualSyncing])
|
||||
|
||||
// Show empty placeholder when no upcoming words
|
||||
if (upcomingWords.length === 0) {
|
||||
return null
|
||||
}
|
||||
|
||||
return (
|
||||
<Box sx={{
|
||||
height: 44,
|
||||
bgcolor: 'grey.100',
|
||||
borderRadius: 1,
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
px: 1,
|
||||
gap: 0.5,
|
||||
overflow: 'hidden',
|
||||
boxSizing: 'border-box'
|
||||
}}>
|
||||
{upcomingWords.map((word, index) => (
|
||||
<Box
|
||||
key={word.id}
|
||||
sx={{
|
||||
px: 1,
|
||||
py: 0.5,
|
||||
borderRadius: 0.5,
|
||||
bgcolor: index === 0 ? 'error.main' : 'grey.300',
|
||||
color: index === 0 ? 'white' : 'text.primary',
|
||||
fontWeight: index === 0 ? 'bold' : 'normal',
|
||||
fontSize: '13px',
|
||||
fontFamily: 'system-ui, -apple-system, sans-serif',
|
||||
whiteSpace: 'nowrap',
|
||||
border: index === 0 ? '2px solid' : 'none',
|
||||
borderColor: 'error.dark'
|
||||
}}
|
||||
>
|
||||
{word.text}
|
||||
</Box>
|
||||
))}
|
||||
|
||||
{totalRemaining > maxWordsToShow && (
|
||||
<Typography variant="caption" color="text.secondary" sx={{ ml: 1 }}>
|
||||
+{totalRemaining - maxWordsToShow} more
|
||||
</Typography>
|
||||
)}
|
||||
</Box>
|
||||
)
|
||||
})
|
||||
|
||||
export default UpcomingWordsBar
|
||||
@@ -0,0 +1,905 @@
|
||||
import { useState, useCallback, useEffect, useRef, useMemo, memo } from 'react'
|
||||
import {
|
||||
Box,
|
||||
Typography,
|
||||
Slider,
|
||||
Dialog,
|
||||
DialogTitle,
|
||||
DialogContent,
|
||||
DialogActions,
|
||||
TextField,
|
||||
Button,
|
||||
Paper,
|
||||
Alert
|
||||
} from '@mui/material'
|
||||
import ZoomInIcon from '@mui/icons-material/ZoomIn'
|
||||
import ZoomOutIcon from '@mui/icons-material/ZoomOut'
|
||||
import { Word, LyricsSegment } from '../../types'
|
||||
import TimelineCanvas from './TimelineCanvas'
|
||||
import UpcomingWordsBar from './UpcomingWordsBar'
|
||||
import SyncControls from './SyncControls'
|
||||
|
||||
// Augment window type for audio functions
|
||||
declare global {
|
||||
interface Window {
|
||||
getAudioDuration?: () => number
|
||||
toggleAudioPlayback?: () => void
|
||||
isAudioPlaying?: boolean
|
||||
}
|
||||
}
|
||||
|
||||
interface LyricsSynchronizerProps {
|
||||
segments: LyricsSegment[]
|
||||
currentTime: number
|
||||
onPlaySegment?: (startTime: number) => void
|
||||
onSave: (segments: LyricsSegment[]) => void
|
||||
onCancel: () => void
|
||||
setModalSpacebarHandler: (handler: (() => (e: KeyboardEvent) => void) | undefined) => void
|
||||
}
|
||||
|
||||
// Constants for zoom
|
||||
const MIN_ZOOM_SECONDS = 4.5 // Most zoomed in - 4.5 seconds visible
|
||||
const MAX_ZOOM_SECONDS = 24 // Most zoomed out - 24 seconds visible
|
||||
const ZOOM_STEPS = 50 // Number of zoom levels
|
||||
|
||||
// Get all words from segments
|
||||
function getAllWords(segments: LyricsSegment[]): Word[] {
|
||||
return segments.flatMap(s => s.words)
|
||||
}
|
||||
|
||||
// Deep clone segments
|
||||
function cloneSegments(segments: LyricsSegment[]): LyricsSegment[] {
|
||||
return JSON.parse(JSON.stringify(segments))
|
||||
}
|
||||
|
||||
const LyricsSynchronizer = memo(function LyricsSynchronizer({
|
||||
segments: initialSegments,
|
||||
currentTime,
|
||||
onPlaySegment,
|
||||
onSave,
|
||||
onCancel,
|
||||
setModalSpacebarHandler
|
||||
}: LyricsSynchronizerProps) {
|
||||
// Working copy of segments
|
||||
const [workingSegments, setWorkingSegments] = useState<LyricsSegment[]>(() =>
|
||||
cloneSegments(initialSegments)
|
||||
)
|
||||
|
||||
// Get all words flattened
|
||||
const allWords = useMemo(() => getAllWords(workingSegments), [workingSegments])
|
||||
|
||||
// Audio duration
|
||||
const audioDuration = useMemo(() => {
|
||||
if (typeof window.getAudioDuration === 'function') {
|
||||
const duration = window.getAudioDuration()
|
||||
return duration > 0 ? duration : 300
|
||||
}
|
||||
return 300 // 5 minute fallback
|
||||
}, [])
|
||||
|
||||
// Zoom state (value is the visible time window in seconds)
|
||||
const [zoomSeconds, setZoomSeconds] = useState(12) // Default ~12 seconds visible
|
||||
|
||||
// Visible time range
|
||||
const [visibleStartTime, setVisibleStartTime] = useState(0)
|
||||
const visibleEndTime = useMemo(() =>
|
||||
Math.min(visibleStartTime + zoomSeconds, audioDuration),
|
||||
[visibleStartTime, zoomSeconds, audioDuration]
|
||||
)
|
||||
|
||||
// Manual sync state
|
||||
const [isManualSyncing, setIsManualSyncing] = useState(false)
|
||||
const [isPaused, setIsPaused] = useState(false)
|
||||
const [syncWordIndex, setSyncWordIndex] = useState(-1)
|
||||
const [isSpacebarPressed, setIsSpacebarPressed] = useState(false)
|
||||
const wordStartTimeRef = useRef<number | null>(null)
|
||||
const spacebarPressTimeRef = useRef<number | null>(null)
|
||||
const currentTimeRef = useRef(currentTime)
|
||||
|
||||
// Selection state
|
||||
const [selectedWordIds, setSelectedWordIds] = useState<Set<string>>(new Set())
|
||||
|
||||
// Edit lyrics modal state
|
||||
const [showEditLyricsModal, setShowEditLyricsModal] = useState(false)
|
||||
const [editLyricsText, setEditLyricsText] = useState('')
|
||||
|
||||
// Edit word modal state
|
||||
const [showEditWordModal, setShowEditWordModal] = useState(false)
|
||||
const [editWordText, setEditWordText] = useState('')
|
||||
const [editWordId, setEditWordId] = useState<string | null>(null)
|
||||
|
||||
// Keep currentTimeRef up to date
|
||||
useEffect(() => {
|
||||
currentTimeRef.current = currentTime
|
||||
}, [currentTime])
|
||||
|
||||
// Auto-scroll to follow playhead during sync
|
||||
useEffect(() => {
|
||||
if (isManualSyncing && !isPaused && currentTime > 0) {
|
||||
// If playhead is near the end of visible area, scroll forward
|
||||
if (currentTime > visibleEndTime - (zoomSeconds * 0.1)) {
|
||||
const newStart = Math.max(0, currentTime - zoomSeconds * 0.1)
|
||||
setVisibleStartTime(newStart)
|
||||
}
|
||||
// If playhead is before visible area, scroll back
|
||||
else if (currentTime < visibleStartTime) {
|
||||
setVisibleStartTime(Math.max(0, currentTime - 1))
|
||||
}
|
||||
}
|
||||
}, [currentTime, isManualSyncing, isPaused, visibleStartTime, visibleEndTime, zoomSeconds])
|
||||
|
||||
// Handle zoom slider change
|
||||
const handleZoomChange = useCallback((_: Event, value: number | number[]) => {
|
||||
const zoomValue = value as number
|
||||
// Map slider value (0-50) to zoom range (4.5-24 seconds)
|
||||
const newZoomSeconds = MIN_ZOOM_SECONDS + (zoomValue / ZOOM_STEPS) * (MAX_ZOOM_SECONDS - MIN_ZOOM_SECONDS)
|
||||
setZoomSeconds(newZoomSeconds)
|
||||
}, [])
|
||||
|
||||
// Get slider value from zoom seconds
|
||||
const sliderValue = useMemo(() => {
|
||||
return ((zoomSeconds - MIN_ZOOM_SECONDS) / (MAX_ZOOM_SECONDS - MIN_ZOOM_SECONDS)) * ZOOM_STEPS
|
||||
}, [zoomSeconds])
|
||||
|
||||
// Handle scroll change from timeline
|
||||
const handleScrollChange = useCallback((newStartTime: number) => {
|
||||
setVisibleStartTime(newStartTime)
|
||||
}, [])
|
||||
|
||||
// Update words in segments
|
||||
const updateWords = useCallback((newWords: Word[]) => {
|
||||
setWorkingSegments(prevSegments => {
|
||||
const newSegments = cloneSegments(prevSegments)
|
||||
|
||||
// Create a map of word id to word for quick lookup
|
||||
const wordMap = new Map(newWords.map(w => [w.id, w]))
|
||||
|
||||
// Update each segment's words
|
||||
for (const segment of newSegments) {
|
||||
segment.words = segment.words.map(w => wordMap.get(w.id) || w)
|
||||
|
||||
// Recalculate segment timing
|
||||
const timedWords = segment.words.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
)
|
||||
|
||||
if (timedWords.length > 0) {
|
||||
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
|
||||
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
|
||||
} else {
|
||||
segment.start_time = null
|
||||
segment.end_time = null
|
||||
}
|
||||
}
|
||||
|
||||
return newSegments
|
||||
})
|
||||
}, [])
|
||||
|
||||
// Check if audio is playing
|
||||
const [isPlaying, setIsPlaying] = useState(false)
|
||||
|
||||
// Update isPlaying state periodically
|
||||
useEffect(() => {
|
||||
const checkPlaying = () => {
|
||||
setIsPlaying(typeof window.isAudioPlaying === 'boolean' ? window.isAudioPlaying : false)
|
||||
}
|
||||
checkPlaying()
|
||||
const interval = setInterval(checkPlaying, 100)
|
||||
return () => clearInterval(interval)
|
||||
}, [])
|
||||
|
||||
// Play audio from current position
|
||||
const handlePlayAudio = useCallback(() => {
|
||||
if (onPlaySegment) {
|
||||
onPlaySegment(currentTimeRef.current)
|
||||
}
|
||||
}, [onPlaySegment])
|
||||
|
||||
// Stop audio playback - also exits sync mode
|
||||
const handleStopAudio = useCallback(() => {
|
||||
if (typeof window.toggleAudioPlayback === 'function' && window.isAudioPlaying) {
|
||||
window.toggleAudioPlayback()
|
||||
}
|
||||
// Also exit sync mode when stopping
|
||||
if (isManualSyncing) {
|
||||
setIsManualSyncing(false)
|
||||
setIsPaused(false)
|
||||
setIsSpacebarPressed(false)
|
||||
}
|
||||
}, [isManualSyncing])
|
||||
|
||||
// Start manual sync
|
||||
const handleStartSync = useCallback(() => {
|
||||
if (isManualSyncing) {
|
||||
// Stop sync
|
||||
setIsManualSyncing(false)
|
||||
setIsPaused(false)
|
||||
setSyncWordIndex(-1)
|
||||
setIsSpacebarPressed(false)
|
||||
|
||||
// Stop audio
|
||||
handleStopAudio()
|
||||
return
|
||||
}
|
||||
|
||||
// Find first unsynced word
|
||||
const firstUnsyncedIndex = allWords.findIndex(w =>
|
||||
w.start_time === null || w.end_time === null
|
||||
)
|
||||
|
||||
const startIndex = firstUnsyncedIndex !== -1 ? firstUnsyncedIndex : 0
|
||||
|
||||
setIsManualSyncing(true)
|
||||
setIsPaused(false)
|
||||
setSyncWordIndex(startIndex)
|
||||
setIsSpacebarPressed(false)
|
||||
|
||||
// Start playback
|
||||
if (onPlaySegment) {
|
||||
onPlaySegment(Math.max(0, currentTimeRef.current - 1))
|
||||
}
|
||||
}, [isManualSyncing, allWords, onPlaySegment, handleStopAudio])
|
||||
|
||||
// Pause sync
|
||||
const handlePauseSync = useCallback(() => {
|
||||
setIsPaused(true)
|
||||
handleStopAudio()
|
||||
}, [handleStopAudio])
|
||||
|
||||
// Resume sync
|
||||
const handleResumeSync = useCallback(() => {
|
||||
setIsPaused(false)
|
||||
|
||||
// Find first unsynced word from current position
|
||||
const firstUnsyncedIndex = allWords.findIndex(w =>
|
||||
w.start_time === null || w.end_time === null
|
||||
)
|
||||
|
||||
if (firstUnsyncedIndex !== -1 && firstUnsyncedIndex !== syncWordIndex) {
|
||||
setSyncWordIndex(firstUnsyncedIndex)
|
||||
}
|
||||
|
||||
// Resume playback
|
||||
if (onPlaySegment) {
|
||||
onPlaySegment(currentTimeRef.current)
|
||||
}
|
||||
}, [allWords, syncWordIndex, onPlaySegment])
|
||||
|
||||
// Clear all sync data
|
||||
const handleClearSync = useCallback(() => {
|
||||
setWorkingSegments(prevSegments => {
|
||||
const newSegments = cloneSegments(prevSegments)
|
||||
|
||||
for (const segment of newSegments) {
|
||||
for (const word of segment.words) {
|
||||
word.start_time = null
|
||||
word.end_time = null
|
||||
}
|
||||
segment.start_time = null
|
||||
segment.end_time = null
|
||||
}
|
||||
|
||||
return newSegments
|
||||
})
|
||||
|
||||
setSyncWordIndex(-1)
|
||||
}, [])
|
||||
|
||||
// Unsync from cursor position
|
||||
const handleUnsyncFromCursor = useCallback(() => {
|
||||
const cursorTime = currentTimeRef.current
|
||||
|
||||
setWorkingSegments(prevSegments => {
|
||||
const newSegments = cloneSegments(prevSegments)
|
||||
|
||||
for (const segment of newSegments) {
|
||||
for (const word of segment.words) {
|
||||
// Reset words that start after cursor position
|
||||
if (word.start_time !== null && word.start_time > cursorTime) {
|
||||
word.start_time = null
|
||||
word.end_time = null
|
||||
}
|
||||
}
|
||||
|
||||
// Recalculate segment timing
|
||||
const timedWords = segment.words.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
)
|
||||
|
||||
if (timedWords.length > 0) {
|
||||
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
|
||||
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
|
||||
} else {
|
||||
segment.start_time = null
|
||||
segment.end_time = null
|
||||
}
|
||||
}
|
||||
|
||||
return newSegments
|
||||
})
|
||||
}, [])
|
||||
|
||||
// Check if there are words after cursor that can be unsynced
|
||||
const canUnsyncFromCursor = useMemo(() => {
|
||||
const cursorTime = currentTimeRef.current
|
||||
return allWords.some(w =>
|
||||
w.start_time !== null && w.start_time > cursorTime
|
||||
)
|
||||
}, [allWords, currentTime])
|
||||
|
||||
// Open edit lyrics modal
|
||||
const handleEditLyrics = useCallback(() => {
|
||||
const text = workingSegments.map(s => s.text).join('\n')
|
||||
setEditLyricsText(text)
|
||||
setShowEditLyricsModal(true)
|
||||
}, [workingSegments])
|
||||
|
||||
// Save edited lyrics (warning: resets timing)
|
||||
const handleSaveEditedLyrics = useCallback(() => {
|
||||
const lines = editLyricsText.split('\n').filter(l => l.trim())
|
||||
|
||||
const newSegments: LyricsSegment[] = lines.map((line, idx) => {
|
||||
const words = line.trim().split(/\s+/).map((text, wIdx) => ({
|
||||
id: `word-${idx}-${wIdx}-${Date.now()}`,
|
||||
text,
|
||||
start_time: null,
|
||||
end_time: null,
|
||||
confidence: 1.0
|
||||
}))
|
||||
|
||||
return {
|
||||
id: `segment-${idx}-${Date.now()}`,
|
||||
text: line.trim(),
|
||||
words,
|
||||
start_time: null,
|
||||
end_time: null
|
||||
}
|
||||
})
|
||||
|
||||
setWorkingSegments(newSegments)
|
||||
setShowEditLyricsModal(false)
|
||||
setSyncWordIndex(-1)
|
||||
}, [editLyricsText])
|
||||
|
||||
// Open edit word modal
|
||||
const handleEditSelectedWord = useCallback(() => {
|
||||
if (selectedWordIds.size !== 1) return
|
||||
|
||||
const wordId = Array.from(selectedWordIds)[0]
|
||||
const word = allWords.find(w => w.id === wordId)
|
||||
|
||||
if (word) {
|
||||
setEditWordId(wordId)
|
||||
setEditWordText(word.text)
|
||||
setShowEditWordModal(true)
|
||||
}
|
||||
}, [selectedWordIds, allWords])
|
||||
|
||||
// Save edited word
|
||||
const handleSaveEditedWord = useCallback(() => {
|
||||
if (!editWordId) return
|
||||
|
||||
const newText = editWordText.trim()
|
||||
if (!newText) return
|
||||
|
||||
// Check if we're splitting into multiple words
|
||||
const newWords = newText.split(/\s+/)
|
||||
|
||||
if (newWords.length === 1) {
|
||||
// Simple rename
|
||||
const updatedWords = allWords.map(w =>
|
||||
w.id === editWordId ? { ...w, text: newWords[0] } : w
|
||||
)
|
||||
updateWords(updatedWords)
|
||||
} else {
|
||||
// Split word - preserve timing for first word, null for rest
|
||||
const originalWord = allWords.find(w => w.id === editWordId)
|
||||
if (!originalWord) return
|
||||
|
||||
setWorkingSegments(prevSegments => {
|
||||
const newSegments = cloneSegments(prevSegments)
|
||||
|
||||
for (const segment of newSegments) {
|
||||
const wordIndex = segment.words.findIndex(w => w.id === editWordId)
|
||||
if (wordIndex !== -1) {
|
||||
const newWordObjects = newWords.map((text, idx) => ({
|
||||
id: idx === 0 ? editWordId : `${editWordId}-split-${idx}`,
|
||||
text,
|
||||
start_time: idx === 0 ? originalWord.start_time : null,
|
||||
end_time: idx === 0 ? originalWord.end_time : null,
|
||||
confidence: 1.0
|
||||
}))
|
||||
|
||||
segment.words.splice(wordIndex, 1, ...newWordObjects)
|
||||
segment.text = segment.words.map(w => w.text).join(' ')
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return newSegments
|
||||
})
|
||||
}
|
||||
|
||||
setShowEditWordModal(false)
|
||||
setEditWordId(null)
|
||||
setEditWordText('')
|
||||
setSelectedWordIds(new Set())
|
||||
}, [editWordId, editWordText, allWords, updateWords])
|
||||
|
||||
// Delete selected words
|
||||
const handleDeleteSelected = useCallback(() => {
|
||||
if (selectedWordIds.size === 0) return
|
||||
|
||||
setWorkingSegments(prevSegments => {
|
||||
const newSegments = cloneSegments(prevSegments)
|
||||
|
||||
for (const segment of newSegments) {
|
||||
segment.words = segment.words.filter(w => !selectedWordIds.has(w.id))
|
||||
segment.text = segment.words.map(w => w.text).join(' ')
|
||||
|
||||
// Recalculate segment timing
|
||||
const timedWords = segment.words.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
)
|
||||
|
||||
if (timedWords.length > 0) {
|
||||
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
|
||||
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
|
||||
} else {
|
||||
segment.start_time = null
|
||||
segment.end_time = null
|
||||
}
|
||||
}
|
||||
|
||||
// Remove empty segments
|
||||
return newSegments.filter(s => s.words.length > 0)
|
||||
})
|
||||
|
||||
setSelectedWordIds(new Set())
|
||||
}, [selectedWordIds])
|
||||
|
||||
// Handle word click (selection)
|
||||
const handleWordClick = useCallback((wordId: string, event: React.MouseEvent) => {
|
||||
if (event.shiftKey || event.ctrlKey || event.metaKey) {
|
||||
// Add to selection
|
||||
setSelectedWordIds(prev => {
|
||||
const newSet = new Set(prev)
|
||||
if (newSet.has(wordId)) {
|
||||
newSet.delete(wordId)
|
||||
} else {
|
||||
newSet.add(wordId)
|
||||
}
|
||||
return newSet
|
||||
})
|
||||
} else {
|
||||
// Single selection
|
||||
setSelectedWordIds(new Set([wordId]))
|
||||
}
|
||||
}, [])
|
||||
|
||||
// Handle background click (deselect)
|
||||
const handleBackgroundClick = useCallback(() => {
|
||||
setSelectedWordIds(new Set())
|
||||
}, [])
|
||||
|
||||
// Handle single word timing change (from resize)
|
||||
const handleWordTimingChange = useCallback((wordId: string, newStartTime: number, newEndTime: number) => {
|
||||
setWorkingSegments(prevSegments => {
|
||||
const newSegments = cloneSegments(prevSegments)
|
||||
|
||||
for (const segment of newSegments) {
|
||||
const word = segment.words.find(w => w.id === wordId)
|
||||
if (word) {
|
||||
word.start_time = Math.max(0, newStartTime)
|
||||
word.end_time = Math.max(word.start_time + 0.05, newEndTime)
|
||||
|
||||
// Recalculate segment timing
|
||||
const timedWords = segment.words.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
)
|
||||
if (timedWords.length > 0) {
|
||||
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
|
||||
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return newSegments
|
||||
})
|
||||
}, [])
|
||||
|
||||
// Handle moving multiple words (from drag)
|
||||
const handleWordsMove = useCallback((updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }>) => {
|
||||
setWorkingSegments(prevSegments => {
|
||||
const newSegments = cloneSegments(prevSegments)
|
||||
|
||||
// Create a map for quick lookup
|
||||
const updateMap = new Map(updates.map(u => [u.wordId, u]))
|
||||
|
||||
for (const segment of newSegments) {
|
||||
for (const word of segment.words) {
|
||||
const update = updateMap.get(word.id)
|
||||
if (update) {
|
||||
word.start_time = update.newStartTime
|
||||
word.end_time = update.newEndTime
|
||||
}
|
||||
}
|
||||
|
||||
// Recalculate segment timing
|
||||
const timedWords = segment.words.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
)
|
||||
if (timedWords.length > 0) {
|
||||
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
|
||||
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
|
||||
}
|
||||
}
|
||||
|
||||
return newSegments
|
||||
})
|
||||
}, [])
|
||||
|
||||
// Handle time bar click (seek to position without playing)
|
||||
const handleTimeBarClick = useCallback((time: number) => {
|
||||
// Scroll the timeline to show this time centered
|
||||
const newStart = Math.max(0, time - zoomSeconds / 2)
|
||||
setVisibleStartTime(Math.min(newStart, Math.max(0, audioDuration - zoomSeconds)))
|
||||
|
||||
// Seek to the position (this will briefly start playback)
|
||||
if (onPlaySegment) {
|
||||
onPlaySegment(time)
|
||||
// Immediately stop playback after seeking
|
||||
setTimeout(() => {
|
||||
if (typeof window.toggleAudioPlayback === 'function' && window.isAudioPlaying) {
|
||||
window.toggleAudioPlayback()
|
||||
}
|
||||
}, 50)
|
||||
}
|
||||
}, [zoomSeconds, audioDuration, onPlaySegment])
|
||||
|
||||
// Handle selection complete from drag
|
||||
const handleSelectionComplete = useCallback((wordIds: string[]) => {
|
||||
setSelectedWordIds(new Set(wordIds))
|
||||
}, [])
|
||||
|
||||
// Handle spacebar for manual sync
|
||||
const handleKeyDown = useCallback((e: KeyboardEvent) => {
|
||||
if (e.code !== 'Space') return
|
||||
if (!isManualSyncing || isPaused) return
|
||||
if (syncWordIndex < 0 || syncWordIndex >= allWords.length) return
|
||||
|
||||
e.preventDefault()
|
||||
e.stopPropagation()
|
||||
|
||||
if (isSpacebarPressed) return
|
||||
|
||||
setIsSpacebarPressed(true)
|
||||
wordStartTimeRef.current = currentTimeRef.current
|
||||
spacebarPressTimeRef.current = Date.now()
|
||||
|
||||
// Set start time for current word
|
||||
const newWords = [...allWords]
|
||||
const currentWord = newWords[syncWordIndex]
|
||||
currentWord.start_time = currentTimeRef.current
|
||||
|
||||
// Handle previous word's end time
|
||||
if (syncWordIndex > 0) {
|
||||
const prevWord = newWords[syncWordIndex - 1]
|
||||
if (prevWord.start_time !== null && prevWord.end_time === null) {
|
||||
const gap = currentTimeRef.current - prevWord.start_time
|
||||
if (gap > 1.0) {
|
||||
prevWord.end_time = prevWord.start_time + 0.5
|
||||
} else {
|
||||
prevWord.end_time = currentTimeRef.current - 0.005
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
updateWords(newWords)
|
||||
}, [isManualSyncing, isPaused, syncWordIndex, allWords, isSpacebarPressed, updateWords])
|
||||
|
||||
const handleKeyUp = useCallback((e: KeyboardEvent) => {
|
||||
if (e.code !== 'Space') return
|
||||
if (!isManualSyncing || isPaused) return
|
||||
if (!isSpacebarPressed) return
|
||||
|
||||
e.preventDefault()
|
||||
e.stopPropagation()
|
||||
|
||||
setIsSpacebarPressed(false)
|
||||
|
||||
const pressDuration = spacebarPressTimeRef.current
|
||||
? Date.now() - spacebarPressTimeRef.current
|
||||
: 0
|
||||
const isTap = pressDuration < 200 // 200ms threshold
|
||||
|
||||
const newWords = [...allWords]
|
||||
const currentWord = newWords[syncWordIndex]
|
||||
|
||||
if (isTap) {
|
||||
// Short tap: default 500ms duration
|
||||
currentWord.end_time = (wordStartTimeRef.current || currentTimeRef.current) + 0.5
|
||||
} else {
|
||||
// Hold: use actual timing
|
||||
currentWord.end_time = currentTimeRef.current
|
||||
}
|
||||
|
||||
updateWords(newWords)
|
||||
|
||||
// Move to next word
|
||||
if (syncWordIndex < allWords.length - 1) {
|
||||
setSyncWordIndex(syncWordIndex + 1)
|
||||
} else {
|
||||
// All words synced
|
||||
setIsManualSyncing(false)
|
||||
setSyncWordIndex(-1)
|
||||
handleStopAudio()
|
||||
}
|
||||
|
||||
wordStartTimeRef.current = null
|
||||
spacebarPressTimeRef.current = null
|
||||
}, [isManualSyncing, isPaused, isSpacebarPressed, syncWordIndex, allWords, updateWords, handleStopAudio])
|
||||
|
||||
// Combined spacebar handler
|
||||
const handleSpacebar = useCallback((e: KeyboardEvent) => {
|
||||
if (e.type === 'keydown') {
|
||||
handleKeyDown(e)
|
||||
} else if (e.type === 'keyup') {
|
||||
handleKeyUp(e)
|
||||
}
|
||||
}, [handleKeyDown, handleKeyUp])
|
||||
|
||||
// Keep ref for handler
|
||||
const spacebarHandlerRef = useRef(handleSpacebar)
|
||||
spacebarHandlerRef.current = handleSpacebar
|
||||
|
||||
// Set up spacebar handler
|
||||
useEffect(() => {
|
||||
const handler = (e: KeyboardEvent) => {
|
||||
if (e.code === 'Space') {
|
||||
e.preventDefault()
|
||||
e.stopPropagation()
|
||||
spacebarHandlerRef.current(e)
|
||||
}
|
||||
}
|
||||
|
||||
setModalSpacebarHandler(() => handler)
|
||||
|
||||
return () => {
|
||||
setModalSpacebarHandler(undefined)
|
||||
}
|
||||
}, [setModalSpacebarHandler])
|
||||
|
||||
// Handle save
|
||||
const handleSave = useCallback(() => {
|
||||
onSave(workingSegments)
|
||||
}, [workingSegments, onSave])
|
||||
|
||||
// Progress stats
|
||||
const stats = useMemo(() => {
|
||||
const total = allWords.length
|
||||
const synced = allWords.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
).length
|
||||
return { total, synced, remaining: total - synced }
|
||||
}, [allWords])
|
||||
|
||||
// Get instruction text based on state
|
||||
const getInstructionText = useCallback(() => {
|
||||
if (isManualSyncing) {
|
||||
if (isSpacebarPressed) {
|
||||
// Always include secondary text to prevent layout shift
|
||||
return { primary: '⏱️ Holding... release when word ends', secondary: 'Release spacebar when the word finishes' }
|
||||
}
|
||||
if (stats.remaining === 0) {
|
||||
return { primary: '✅ All words synced!', secondary: 'Click "Stop Sync" then "Apply" to save' }
|
||||
}
|
||||
return { primary: '👆 Press SPACEBAR when you hear each word', secondary: 'Tap for short words, hold for longer words' }
|
||||
}
|
||||
if (stats.synced === 0) {
|
||||
return { primary: 'Click "Start Sync" to begin timing words', secondary: 'Audio will play and you\'ll tap spacebar for each word' }
|
||||
}
|
||||
if (stats.remaining > 0) {
|
||||
return { primary: `${stats.remaining} words remaining to sync`, secondary: 'Click "Start Sync" to continue, or "Unsync from Cursor" to re-sync from a point' }
|
||||
}
|
||||
return { primary: '✅ All words synced!', secondary: 'Click "Apply" to save changes, or make adjustments first' }
|
||||
}, [isManualSyncing, isSpacebarPressed, stats.synced, stats.remaining])
|
||||
|
||||
const instruction = getInstructionText()
|
||||
|
||||
return (
|
||||
<Box sx={{ display: 'flex', flexDirection: 'column', height: '100%', gap: 1 }}>
|
||||
{/* Stats bar - fixed height */}
|
||||
<Box sx={{ display: 'flex', justifyContent: 'flex-end', alignItems: 'center', height: 24 }}>
|
||||
<Typography variant="body2" color="text.secondary">
|
||||
{stats.synced} / {stats.total} words synced
|
||||
{stats.remaining > 0 && ` (${stats.remaining} remaining)`}
|
||||
</Typography>
|
||||
</Box>
|
||||
|
||||
{/* Instruction banner - fixed height to prevent layout shifts */}
|
||||
<Box
|
||||
sx={{
|
||||
height: 56,
|
||||
flexShrink: 0
|
||||
}}
|
||||
>
|
||||
<Paper
|
||||
sx={{
|
||||
p: 1.5,
|
||||
height: '100%',
|
||||
bgcolor: isManualSyncing ? 'info.main' : 'grey.100',
|
||||
color: isManualSyncing ? 'info.contrastText' : 'text.primary',
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
justifyContent: 'center',
|
||||
overflow: 'hidden',
|
||||
boxSizing: 'border-box'
|
||||
}}
|
||||
>
|
||||
<Typography variant="body2" sx={{ fontWeight: 500, lineHeight: 1.3 }}>
|
||||
{instruction.primary}
|
||||
</Typography>
|
||||
<Typography variant="caption" sx={{ opacity: 0.85, display: 'block', lineHeight: 1.3 }}>
|
||||
{instruction.secondary}
|
||||
</Typography>
|
||||
</Paper>
|
||||
</Box>
|
||||
|
||||
{/* Controls - fixed height section */}
|
||||
<Box sx={{ height: 88, flexShrink: 0 }}>
|
||||
<SyncControls
|
||||
isManualSyncing={isManualSyncing}
|
||||
isPaused={isPaused}
|
||||
onStartSync={handleStartSync}
|
||||
onPauseSync={handlePauseSync}
|
||||
onResumeSync={handleResumeSync}
|
||||
onClearSync={handleClearSync}
|
||||
onEditLyrics={handleEditLyrics}
|
||||
onPlay={handlePlayAudio}
|
||||
onStop={handleStopAudio}
|
||||
isPlaying={isPlaying}
|
||||
hasSelectedWords={selectedWordIds.size > 0}
|
||||
selectedWordCount={selectedWordIds.size}
|
||||
onUnsyncFromCursor={handleUnsyncFromCursor}
|
||||
onEditSelectedWord={handleEditSelectedWord}
|
||||
onDeleteSelected={handleDeleteSelected}
|
||||
canUnsyncFromCursor={canUnsyncFromCursor}
|
||||
/>
|
||||
</Box>
|
||||
|
||||
{/* Upcoming words bar - fixed height, immediately above timeline */}
|
||||
<Box sx={{ height: 44, flexShrink: 0 }}>
|
||||
{isManualSyncing && (
|
||||
<UpcomingWordsBar
|
||||
words={allWords}
|
||||
syncWordIndex={syncWordIndex}
|
||||
isManualSyncing={isManualSyncing}
|
||||
/>
|
||||
)}
|
||||
</Box>
|
||||
|
||||
{/* Timeline canvas */}
|
||||
<Box sx={{ flexGrow: 1, minHeight: 200 }}>
|
||||
<TimelineCanvas
|
||||
words={allWords}
|
||||
segments={workingSegments}
|
||||
visibleStartTime={visibleStartTime}
|
||||
visibleEndTime={visibleEndTime}
|
||||
currentTime={currentTime}
|
||||
selectedWordIds={selectedWordIds}
|
||||
onWordClick={handleWordClick}
|
||||
onBackgroundClick={handleBackgroundClick}
|
||||
onTimeBarClick={handleTimeBarClick}
|
||||
onSelectionComplete={handleSelectionComplete}
|
||||
onWordTimingChange={handleWordTimingChange}
|
||||
onWordsMove={handleWordsMove}
|
||||
syncWordIndex={syncWordIndex}
|
||||
isManualSyncing={isManualSyncing}
|
||||
onScrollChange={handleScrollChange}
|
||||
audioDuration={audioDuration}
|
||||
zoomSeconds={zoomSeconds}
|
||||
height={200}
|
||||
/>
|
||||
</Box>
|
||||
|
||||
{/* Zoom slider */}
|
||||
<Box sx={{ display: 'flex', alignItems: 'center', gap: 2, px: 2 }}>
|
||||
<ZoomInIcon color="action" fontSize="small" />
|
||||
<Slider
|
||||
value={sliderValue}
|
||||
onChange={handleZoomChange}
|
||||
min={0}
|
||||
max={ZOOM_STEPS}
|
||||
step={1}
|
||||
sx={{ flexGrow: 1 }}
|
||||
disabled={isManualSyncing && !isPaused}
|
||||
/>
|
||||
<ZoomOutIcon color="action" fontSize="small" />
|
||||
<Typography variant="caption" color="text.secondary" sx={{ minWidth: 60 }}>
|
||||
{zoomSeconds.toFixed(1)}s view
|
||||
</Typography>
|
||||
</Box>
|
||||
|
||||
{/* Action buttons */}
|
||||
<Box sx={{ display: 'flex', justifyContent: 'flex-end', gap: 2, pt: 2, borderTop: 1, borderColor: 'divider' }}>
|
||||
<Button onClick={onCancel} color="inherit">
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
onClick={handleSave}
|
||||
variant="contained"
|
||||
color="primary"
|
||||
disabled={isManualSyncing && !isPaused}
|
||||
>
|
||||
Apply
|
||||
</Button>
|
||||
</Box>
|
||||
|
||||
{/* Edit Lyrics Modal */}
|
||||
<Dialog
|
||||
open={showEditLyricsModal}
|
||||
onClose={() => setShowEditLyricsModal(false)}
|
||||
maxWidth="md"
|
||||
fullWidth
|
||||
>
|
||||
<DialogTitle>Edit Lyrics</DialogTitle>
|
||||
<DialogContent>
|
||||
<Alert severity="warning" sx={{ mb: 2 }}>
|
||||
Editing lyrics will reset all timing data. You will need to re-sync the entire song.
|
||||
</Alert>
|
||||
<TextField
|
||||
multiline
|
||||
rows={15}
|
||||
fullWidth
|
||||
value={editLyricsText}
|
||||
onChange={(e) => setEditLyricsText(e.target.value)}
|
||||
placeholder="Enter lyrics, one line per segment..."
|
||||
/>
|
||||
</DialogContent>
|
||||
<DialogActions>
|
||||
<Button onClick={() => setShowEditLyricsModal(false)}>Cancel</Button>
|
||||
<Button onClick={handleSaveEditedLyrics} variant="contained" color="warning">
|
||||
Save & Reset Timing
|
||||
</Button>
|
||||
</DialogActions>
|
||||
</Dialog>
|
||||
|
||||
{/* Edit Word Modal */}
|
||||
<Dialog
|
||||
open={showEditWordModal}
|
||||
onClose={() => setShowEditWordModal(false)}
|
||||
maxWidth="xs"
|
||||
fullWidth
|
||||
>
|
||||
<DialogTitle>Edit Word</DialogTitle>
|
||||
<DialogContent>
|
||||
<Typography variant="body2" color="text.secondary" sx={{ mb: 2 }}>
|
||||
Edit the word text. Enter multiple words separated by spaces to split.
|
||||
</Typography>
|
||||
<TextField
|
||||
fullWidth
|
||||
value={editWordText}
|
||||
onChange={(e) => setEditWordText(e.target.value)}
|
||||
autoFocus
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter') {
|
||||
handleSaveEditedWord()
|
||||
}
|
||||
}}
|
||||
/>
|
||||
</DialogContent>
|
||||
<DialogActions>
|
||||
<Button onClick={() => setShowEditWordModal(false)}>Cancel</Button>
|
||||
<Button onClick={handleSaveEditedWord} variant="contained">
|
||||
Save
|
||||
</Button>
|
||||
</DialogActions>
|
||||
</Dialog>
|
||||
</Box>
|
||||
)
|
||||
})
|
||||
|
||||
export default LyricsSynchronizer
|
||||
@@ -0,0 +1,127 @@
|
||||
import {
|
||||
Dialog,
|
||||
DialogTitle,
|
||||
DialogContent,
|
||||
DialogActions,
|
||||
IconButton,
|
||||
Box,
|
||||
Button,
|
||||
Typography,
|
||||
Paper
|
||||
} from '@mui/material'
|
||||
import CloseIcon from '@mui/icons-material/Close'
|
||||
import ContentPasteIcon from '@mui/icons-material/ContentPaste'
|
||||
import SyncIcon from '@mui/icons-material/Sync'
|
||||
|
||||
interface ModeSelectionModalProps {
|
||||
open: boolean
|
||||
onClose: () => void
|
||||
onSelectReplace: () => void // Goes to paste phase for replacing all lyrics
|
||||
onSelectResync: () => void // Goes directly to sync view with existing lyrics
|
||||
hasExistingLyrics: boolean // Whether there are existing lyrics to re-sync
|
||||
}
|
||||
|
||||
export default function ModeSelectionModal({
|
||||
open,
|
||||
onClose,
|
||||
onSelectReplace,
|
||||
onSelectResync,
|
||||
hasExistingLyrics
|
||||
}: ModeSelectionModalProps) {
|
||||
return (
|
||||
<Dialog
|
||||
open={open}
|
||||
onClose={onClose}
|
||||
maxWidth="sm"
|
||||
fullWidth
|
||||
>
|
||||
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
|
||||
<Box sx={{ flex: 1 }}>
|
||||
Edit All Lyrics
|
||||
</Box>
|
||||
<IconButton onClick={onClose} sx={{ ml: 'auto' }}>
|
||||
<CloseIcon />
|
||||
</IconButton>
|
||||
</DialogTitle>
|
||||
|
||||
<DialogContent dividers>
|
||||
<Typography variant="body1" sx={{ mb: 3 }}>
|
||||
Choose how you want to edit the lyrics:
|
||||
</Typography>
|
||||
|
||||
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 2 }}>
|
||||
{/* Re-sync Existing option - only show if there are existing lyrics */}
|
||||
{hasExistingLyrics && (
|
||||
<Paper
|
||||
sx={{
|
||||
p: 2,
|
||||
cursor: 'pointer',
|
||||
border: 2,
|
||||
borderColor: 'primary.main',
|
||||
'&:hover': {
|
||||
bgcolor: 'action.hover',
|
||||
borderColor: 'primary.dark'
|
||||
}
|
||||
}}
|
||||
onClick={onSelectResync}
|
||||
>
|
||||
<Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}>
|
||||
<SyncIcon color="primary" sx={{ fontSize: 40, mt: 0.5 }} />
|
||||
<Box>
|
||||
<Typography variant="h6" color="primary">
|
||||
Re-sync Existing Lyrics
|
||||
</Typography>
|
||||
<Typography variant="body2" color="text.secondary">
|
||||
Keep the current lyrics text and fix timing issues.
|
||||
Use this when lyrics are correct but timing has drifted,
|
||||
especially in the second half of the song.
|
||||
</Typography>
|
||||
<Typography variant="caption" color="success.main" sx={{ mt: 1, display: 'block' }}>
|
||||
Recommended for fixing timing drift
|
||||
</Typography>
|
||||
</Box>
|
||||
</Box>
|
||||
</Paper>
|
||||
)}
|
||||
|
||||
{/* Replace All option */}
|
||||
<Paper
|
||||
sx={{
|
||||
p: 2,
|
||||
cursor: 'pointer',
|
||||
border: 1,
|
||||
borderColor: 'divider',
|
||||
'&:hover': {
|
||||
bgcolor: 'action.hover',
|
||||
borderColor: 'text.secondary'
|
||||
}
|
||||
}}
|
||||
onClick={onSelectReplace}
|
||||
>
|
||||
<Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}>
|
||||
<ContentPasteIcon sx={{ fontSize: 40, mt: 0.5, color: 'text.secondary' }} />
|
||||
<Box>
|
||||
<Typography variant="h6">
|
||||
Replace All Lyrics
|
||||
</Typography>
|
||||
<Typography variant="body2" color="text.secondary">
|
||||
Paste completely new lyrics from clipboard and manually
|
||||
sync timing for all words from scratch.
|
||||
</Typography>
|
||||
<Typography variant="caption" color="warning.main" sx={{ mt: 1, display: 'block' }}>
|
||||
All existing timing data will be lost
|
||||
</Typography>
|
||||
</Box>
|
||||
</Box>
|
||||
</Paper>
|
||||
</Box>
|
||||
</DialogContent>
|
||||
|
||||
<DialogActions>
|
||||
<Button onClick={onClose} color="inherit">
|
||||
Cancel
|
||||
</Button>
|
||||
</DialogActions>
|
||||
</Dialog>
|
||||
)
|
||||
}
|
||||
@@ -7,19 +7,17 @@ import {
|
||||
Box,
|
||||
Button,
|
||||
Typography,
|
||||
TextField,
|
||||
Paper,
|
||||
Divider
|
||||
TextField
|
||||
} from '@mui/material'
|
||||
import CloseIcon from '@mui/icons-material/Close'
|
||||
import ContentPasteIcon from '@mui/icons-material/ContentPaste'
|
||||
import AutoFixHighIcon from '@mui/icons-material/AutoFixHigh'
|
||||
import ArrowBackIcon from '@mui/icons-material/ArrowBack'
|
||||
import { LyricsSegment, Word } from '../types'
|
||||
import { useState, useEffect, useCallback, useMemo, useRef, memo } from 'react'
|
||||
import { useState, useEffect, useCallback, useMemo } from 'react'
|
||||
import { nanoid } from 'nanoid'
|
||||
import useManualSync from '../hooks/useManualSync'
|
||||
import EditTimelineSection from './EditTimelineSection'
|
||||
import EditActionBar from './EditActionBar'
|
||||
import ModeSelectionModal from './ModeSelectionModal'
|
||||
import LyricsSynchronizer from './LyricsSynchronizer'
|
||||
|
||||
// Augment window type for audio functions
|
||||
declare global {
|
||||
@@ -30,6 +28,8 @@ declare global {
|
||||
}
|
||||
}
|
||||
|
||||
type ModalMode = 'selection' | 'replace' | 'resync'
|
||||
|
||||
interface ReplaceAllLyricsModalProps {
|
||||
open: boolean
|
||||
onClose: () => void
|
||||
@@ -37,6 +37,7 @@ interface ReplaceAllLyricsModalProps {
|
||||
onPlaySegment?: (startTime: number) => void
|
||||
currentTime?: number
|
||||
setModalSpacebarHandler: (handler: (() => (e: KeyboardEvent) => void) | undefined) => void
|
||||
existingSegments?: LyricsSegment[] // Current segments for re-sync mode
|
||||
}
|
||||
|
||||
export default function ReplaceAllLyricsModal({
|
||||
@@ -45,22 +46,21 @@ export default function ReplaceAllLyricsModal({
|
||||
onSave,
|
||||
onPlaySegment,
|
||||
currentTime = 0,
|
||||
setModalSpacebarHandler
|
||||
setModalSpacebarHandler,
|
||||
existingSegments = []
|
||||
}: ReplaceAllLyricsModalProps) {
|
||||
const [mode, setMode] = useState<ModalMode>('selection')
|
||||
const [inputText, setInputText] = useState('')
|
||||
const [isReplaced, setIsReplaced] = useState(false)
|
||||
const [globalSegment, setGlobalSegment] = useState<LyricsSegment | null>(null)
|
||||
const [originalSegments, setOriginalSegments] = useState<LyricsSegment[]>([])
|
||||
const [currentSegments, setCurrentSegments] = useState<LyricsSegment[]>([])
|
||||
const [newSegments, setNewSegments] = useState<LyricsSegment[]>([])
|
||||
|
||||
// Get the real audio duration, with fallback
|
||||
const getAudioDuration = useCallback(() => {
|
||||
if (window.getAudioDuration) {
|
||||
const duration = window.getAudioDuration()
|
||||
return duration > 0 ? duration : 600 // Use real duration or 10 min fallback
|
||||
// Reset state when modal opens
|
||||
useEffect(() => {
|
||||
if (open) {
|
||||
setMode('selection')
|
||||
setInputText('')
|
||||
setNewSegments([])
|
||||
}
|
||||
return 600 // 10 minute fallback if audio not loaded
|
||||
}, [])
|
||||
}, [open])
|
||||
|
||||
// Parse the input text to get line and word counts
|
||||
const parseInfo = useMemo(() => {
|
||||
@@ -79,61 +79,31 @@ export default function ReplaceAllLyricsModal({
|
||||
if (!inputText.trim()) return
|
||||
|
||||
const lines = inputText.trim().split('\n').filter(line => line.trim().length > 0)
|
||||
const newSegments: LyricsSegment[] = []
|
||||
const allWords: Word[] = []
|
||||
const segments: LyricsSegment[] = []
|
||||
|
||||
lines.forEach((line) => {
|
||||
const words = line.trim().split(/\s+/).filter(word => word.length > 0)
|
||||
const segmentWords: Word[] = []
|
||||
const segmentWords: Word[] = words.map((wordText) => ({
|
||||
id: nanoid(),
|
||||
text: wordText,
|
||||
start_time: null,
|
||||
end_time: null,
|
||||
confidence: 1.0,
|
||||
created_during_correction: true
|
||||
}))
|
||||
|
||||
words.forEach((wordText) => {
|
||||
const word: Word = {
|
||||
id: nanoid(),
|
||||
text: wordText,
|
||||
start_time: null,
|
||||
end_time: null,
|
||||
confidence: 1.0,
|
||||
created_during_correction: true
|
||||
}
|
||||
segmentWords.push(word)
|
||||
allWords.push(word)
|
||||
})
|
||||
|
||||
const segment: LyricsSegment = {
|
||||
segments.push({
|
||||
id: nanoid(),
|
||||
text: line.trim(),
|
||||
words: segmentWords,
|
||||
start_time: null,
|
||||
end_time: null
|
||||
}
|
||||
|
||||
newSegments.push(segment)
|
||||
})
|
||||
})
|
||||
|
||||
// Create a global segment with all words for manual sync
|
||||
// Set a very large end time to ensure manual sync doesn't stop prematurely
|
||||
const audioDuration = getAudioDuration()
|
||||
const endTime = Math.max(audioDuration, 3600) // At least 1 hour to prevent auto-stop
|
||||
|
||||
console.log('ReplaceAllLyricsModal - Creating global segment', {
|
||||
audioDuration,
|
||||
endTime,
|
||||
wordCount: allWords.length
|
||||
})
|
||||
|
||||
const globalSegment: LyricsSegment = {
|
||||
id: 'global-replacement',
|
||||
text: allWords.map(w => w.text).join(' '),
|
||||
words: allWords,
|
||||
start_time: 0,
|
||||
end_time: endTime
|
||||
}
|
||||
|
||||
setCurrentSegments(newSegments)
|
||||
setOriginalSegments(JSON.parse(JSON.stringify(newSegments)))
|
||||
setGlobalSegment(globalSegment)
|
||||
setIsReplaced(true)
|
||||
}, [inputText, getAudioDuration])
|
||||
setNewSegments(segments)
|
||||
setMode('resync') // Go to synchronizer with the new segments
|
||||
}, [inputText])
|
||||
|
||||
// Handle paste from clipboard
|
||||
const handlePasteFromClipboard = useCallback(async () => {
|
||||
@@ -146,311 +116,90 @@ export default function ReplaceAllLyricsModal({
|
||||
}
|
||||
}, [])
|
||||
|
||||
// Update segment when words change during manual sync
|
||||
const updateSegment = useCallback((newWords: Word[]) => {
|
||||
if (!globalSegment) return
|
||||
|
||||
const validStartTimes = newWords.map(w => w.start_time).filter((t): t is number => t !== null)
|
||||
const validEndTimes = newWords.map(w => w.end_time).filter((t): t is number => t !== null)
|
||||
|
||||
const segmentStartTime = validStartTimes.length > 0 ? Math.min(...validStartTimes) : null
|
||||
const segmentEndTime = validEndTimes.length > 0 ? Math.max(...validEndTimes) : null
|
||||
|
||||
const updatedGlobalSegment = {
|
||||
...globalSegment,
|
||||
words: newWords,
|
||||
text: newWords.map(w => w.text).join(' '),
|
||||
start_time: segmentStartTime,
|
||||
end_time: segmentEndTime
|
||||
}
|
||||
|
||||
// Batch state updates to prevent multiple re-renders
|
||||
setGlobalSegment(updatedGlobalSegment)
|
||||
|
||||
// Update individual segment timing as words get synced - but only calculate when needed
|
||||
const updatedSegments = currentSegments.map(segment => {
|
||||
// Find words that belong to this segment and have been timed
|
||||
const segmentWordsWithTiming = segment.words.map(segmentWord => {
|
||||
const globalWord = newWords.find(w => w.id === segmentWord.id)
|
||||
return globalWord || segmentWord
|
||||
})
|
||||
|
||||
// Calculate segment timing if all words have timing
|
||||
const wordsWithTiming = segmentWordsWithTiming.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
)
|
||||
|
||||
if (wordsWithTiming.length === segmentWordsWithTiming.length && wordsWithTiming.length > 0) {
|
||||
// All words in this segment have timing - update segment timing
|
||||
const segmentStart = Math.min(...wordsWithTiming.map(w => w.start_time!))
|
||||
const segmentEnd = Math.max(...wordsWithTiming.map(w => w.end_time!))
|
||||
|
||||
return {
|
||||
...segment,
|
||||
words: segmentWordsWithTiming,
|
||||
start_time: segmentStart,
|
||||
end_time: segmentEnd
|
||||
}
|
||||
} else {
|
||||
// Some words still don't have timing - just update the words
|
||||
return {
|
||||
...segment,
|
||||
words: segmentWordsWithTiming
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
setCurrentSegments(updatedSegments)
|
||||
}, [globalSegment, currentSegments])
|
||||
|
||||
// Use the manual sync hook
|
||||
const {
|
||||
isManualSyncing,
|
||||
isPaused,
|
||||
syncWordIndex,
|
||||
startManualSync,
|
||||
pauseManualSync,
|
||||
resumeManualSync,
|
||||
cleanupManualSync,
|
||||
handleSpacebar,
|
||||
isSpacebarPressed
|
||||
} = useManualSync({
|
||||
editedSegment: globalSegment,
|
||||
currentTime,
|
||||
onPlaySegment,
|
||||
updateSegment
|
||||
})
|
||||
|
||||
// Handle manual word updates (drag/resize in timeline)
|
||||
const handleWordUpdate = useCallback((wordIndex: number, updates: Partial<Word>) => {
|
||||
if (!globalSegment) return
|
||||
|
||||
// Only allow manual adjustments when manual sync is paused or not active
|
||||
if (isManualSyncing && !isPaused) {
|
||||
console.log('ReplaceAllLyricsModal - Ignoring word update during active manual sync')
|
||||
return
|
||||
}
|
||||
|
||||
console.log('ReplaceAllLyricsModal - Manual word update', {
|
||||
wordIndex,
|
||||
wordText: globalSegment.words[wordIndex]?.text,
|
||||
updates,
|
||||
isManualSyncing,
|
||||
isPaused
|
||||
})
|
||||
|
||||
// Update the word in the global segment
|
||||
const newWords = [...globalSegment.words]
|
||||
newWords[wordIndex] = {
|
||||
...newWords[wordIndex],
|
||||
...updates
|
||||
}
|
||||
|
||||
// Update the global segment through the existing updateSegment function
|
||||
updateSegment(newWords)
|
||||
}, [globalSegment, updateSegment, isManualSyncing, isPaused])
|
||||
|
||||
// Handle un-syncing a word (right-click context menu)
|
||||
const handleUnsyncWord = useCallback((wordIndex: number) => {
|
||||
if (!globalSegment) return
|
||||
|
||||
console.log('ReplaceAllLyricsModal - Un-syncing word', {
|
||||
wordIndex,
|
||||
wordText: globalSegment.words[wordIndex]?.text
|
||||
})
|
||||
|
||||
// Update the word to remove timing
|
||||
const newWords = [...globalSegment.words]
|
||||
newWords[wordIndex] = {
|
||||
...newWords[wordIndex],
|
||||
start_time: null,
|
||||
end_time: null
|
||||
}
|
||||
|
||||
// Update the global segment through the existing updateSegment function
|
||||
updateSegment(newWords)
|
||||
}, [globalSegment, updateSegment])
|
||||
|
||||
// Handle modal close
|
||||
const handleClose = useCallback(() => {
|
||||
cleanupManualSync()
|
||||
setMode('selection')
|
||||
setInputText('')
|
||||
setIsReplaced(false)
|
||||
setGlobalSegment(null)
|
||||
setOriginalSegments([])
|
||||
setCurrentSegments([])
|
||||
setNewSegments([])
|
||||
onClose()
|
||||
}, [onClose, cleanupManualSync])
|
||||
}, [onClose])
|
||||
|
||||
// Handle save
|
||||
const handleSave = useCallback(() => {
|
||||
if (!globalSegment || !currentSegments.length) return
|
||||
|
||||
// Distribute the timed words back to their original segments
|
||||
const finalSegments: LyricsSegment[] = []
|
||||
let wordIndex = 0
|
||||
|
||||
currentSegments.forEach((segment) => {
|
||||
const originalWordCount = segment.words.length
|
||||
const segmentWords = globalSegment.words.slice(wordIndex, wordIndex + originalWordCount)
|
||||
wordIndex += originalWordCount
|
||||
|
||||
if (segmentWords.length > 0) {
|
||||
// Recalculate segment start and end times
|
||||
const validStartTimes = segmentWords.map(w => w.start_time).filter((t): t is number => t !== null)
|
||||
const validEndTimes = segmentWords.map(w => w.end_time).filter((t): t is number => t !== null)
|
||||
|
||||
const segmentStartTime = validStartTimes.length > 0 ? Math.min(...validStartTimes) : null
|
||||
const segmentEndTime = validEndTimes.length > 0 ? Math.max(...validEndTimes) : null
|
||||
|
||||
finalSegments.push({
|
||||
...segment,
|
||||
words: segmentWords,
|
||||
text: segmentWords.map(w => w.text).join(' '),
|
||||
start_time: segmentStartTime,
|
||||
end_time: segmentEndTime
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
console.log('ReplaceAllLyricsModal - Saving new segments:', {
|
||||
originalSegmentCount: currentSegments.length,
|
||||
finalSegmentCount: finalSegments.length,
|
||||
totalWords: finalSegments.reduce((count, seg) => count + seg.words.length, 0)
|
||||
})
|
||||
|
||||
onSave(finalSegments)
|
||||
// Handle save from synchronizer
|
||||
const handleSave = useCallback((segments: LyricsSegment[]) => {
|
||||
onSave(segments)
|
||||
handleClose()
|
||||
}, [globalSegment, currentSegments, onSave, handleClose])
|
||||
}, [onSave, handleClose])
|
||||
|
||||
// Handle reset
|
||||
const handleReset = useCallback(() => {
|
||||
if (!originalSegments.length) return
|
||||
// Handle mode selection
|
||||
const handleSelectReplace = useCallback(() => {
|
||||
setMode('replace')
|
||||
}, [])
|
||||
|
||||
console.log('ReplaceAllLyricsModal - Resetting to original state')
|
||||
|
||||
// Reset all words to have null timing (ready for fresh manual sync)
|
||||
const resetWords = originalSegments.flatMap(segment =>
|
||||
segment.words.map(word => ({
|
||||
...word,
|
||||
start_time: null,
|
||||
end_time: null
|
||||
}))
|
||||
)
|
||||
|
||||
const audioDuration = getAudioDuration()
|
||||
const resetGlobalSegment: LyricsSegment = {
|
||||
id: 'global-replacement',
|
||||
text: resetWords.map(w => w.text).join(' '),
|
||||
words: resetWords,
|
||||
start_time: 0,
|
||||
end_time: Math.max(audioDuration, 3600) // At least 1 hour to prevent auto-stop
|
||||
}
|
||||
const handleSelectResync = useCallback(() => {
|
||||
setMode('resync')
|
||||
}, [])
|
||||
|
||||
// Also reset the current segments to have null timing
|
||||
const resetCurrentSegments = originalSegments.map(segment => ({
|
||||
...segment,
|
||||
words: segment.words.map(word => ({
|
||||
...word,
|
||||
start_time: null,
|
||||
end_time: null
|
||||
})),
|
||||
start_time: null,
|
||||
end_time: null
|
||||
}))
|
||||
// Handle back to selection
|
||||
const handleBackToSelection = useCallback(() => {
|
||||
setMode('selection')
|
||||
setInputText('')
|
||||
setNewSegments([])
|
||||
}, [])
|
||||
|
||||
setGlobalSegment(resetGlobalSegment)
|
||||
setCurrentSegments(resetCurrentSegments)
|
||||
}, [originalSegments, getAudioDuration])
|
||||
// Determine which segments to use for synchronizer
|
||||
const segmentsForSync = mode === 'resync' && newSegments.length > 0
|
||||
? newSegments
|
||||
: existingSegments
|
||||
|
||||
// Keep a ref to the current spacebar handler to avoid closure issues
|
||||
const spacebarHandlerRef = useRef(handleSpacebar)
|
||||
spacebarHandlerRef.current = handleSpacebar
|
||||
|
||||
// Update the spacebar handler when modal state changes
|
||||
useEffect(() => {
|
||||
if (open && isReplaced) {
|
||||
console.log('ReplaceAllLyricsModal - Setting up spacebar handler')
|
||||
|
||||
const handleKeyEvent = (e: KeyboardEvent) => {
|
||||
if (e.code === 'Space') {
|
||||
console.log('ReplaceAllLyricsModal - Spacebar captured in modal')
|
||||
e.preventDefault()
|
||||
e.stopPropagation()
|
||||
// Use the ref to get the current handler
|
||||
spacebarHandlerRef.current(e)
|
||||
}
|
||||
}
|
||||
|
||||
setModalSpacebarHandler(() => handleKeyEvent)
|
||||
|
||||
return () => {
|
||||
if (!open) {
|
||||
console.log('ReplaceAllLyricsModal - Clearing spacebar handler')
|
||||
setModalSpacebarHandler(undefined)
|
||||
}
|
||||
}
|
||||
} else if (open) {
|
||||
// Clear handler when not in replaced state
|
||||
setModalSpacebarHandler(undefined)
|
||||
}
|
||||
}, [open, isReplaced, setModalSpacebarHandler])
|
||||
|
||||
// Memoize timeline range to prevent recalculation
|
||||
const timeRange = useMemo(() => {
|
||||
const audioDuration = getAudioDuration()
|
||||
// Always use full song duration for replace-all mode
|
||||
return { start: 0, end: audioDuration }
|
||||
}, [getAudioDuration])
|
||||
|
||||
// Memoize the segment progress props to prevent unnecessary re-renders
|
||||
const segmentProgressProps = useMemo(() => ({
|
||||
currentSegments,
|
||||
globalSegment,
|
||||
syncWordIndex
|
||||
}), [currentSegments, globalSegment, syncWordIndex])
|
||||
// Check if we have existing lyrics
|
||||
const hasExistingLyrics = existingSegments.length > 0 &&
|
||||
existingSegments.some(s => s.words.length > 0)
|
||||
|
||||
return (
|
||||
<Dialog
|
||||
open={open}
|
||||
onClose={handleClose}
|
||||
maxWidth={false}
|
||||
fullWidth={true}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter' && !e.shiftKey && isReplaced) {
|
||||
e.preventDefault()
|
||||
handleSave()
|
||||
}
|
||||
}}
|
||||
PaperProps={{
|
||||
sx: {
|
||||
height: '90vh',
|
||||
margin: '5vh 2vh',
|
||||
maxWidth: 'calc(100vw - 4vh)',
|
||||
width: 'calc(100vw - 4vh)'
|
||||
}
|
||||
}}
|
||||
>
|
||||
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
|
||||
<Box sx={{ flex: 1 }}>
|
||||
Replace All Lyrics
|
||||
</Box>
|
||||
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
|
||||
<CloseIcon />
|
||||
</IconButton>
|
||||
</DialogTitle>
|
||||
<>
|
||||
{/* Mode Selection Modal */}
|
||||
<ModeSelectionModal
|
||||
open={open && mode === 'selection'}
|
||||
onClose={handleClose}
|
||||
onSelectReplace={handleSelectReplace}
|
||||
onSelectResync={handleSelectResync}
|
||||
hasExistingLyrics={hasExistingLyrics}
|
||||
/>
|
||||
|
||||
<DialogContent
|
||||
dividers
|
||||
sx={{
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
flexGrow: 1,
|
||||
overflow: 'hidden'
|
||||
{/* Replace All Lyrics Modal (Paste Phase) */}
|
||||
<Dialog
|
||||
open={open && mode === 'replace'}
|
||||
onClose={handleClose}
|
||||
maxWidth="md"
|
||||
fullWidth
|
||||
PaperProps={{
|
||||
sx: {
|
||||
height: '80vh',
|
||||
maxHeight: '80vh'
|
||||
}
|
||||
}}
|
||||
>
|
||||
{!isReplaced ? (
|
||||
// Step 1: Input new lyrics
|
||||
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
|
||||
<IconButton onClick={handleBackToSelection} size="small">
|
||||
<ArrowBackIcon />
|
||||
</IconButton>
|
||||
<Box sx={{ flex: 1 }}>
|
||||
Replace All Lyrics
|
||||
</Box>
|
||||
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
|
||||
<CloseIcon />
|
||||
</IconButton>
|
||||
</DialogTitle>
|
||||
|
||||
<DialogContent
|
||||
dividers
|
||||
sx={{
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
flexGrow: 1,
|
||||
overflow: 'hidden'
|
||||
}}
|
||||
>
|
||||
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 2, height: '100%' }}>
|
||||
<Typography variant="h6" gutterBottom>
|
||||
Paste your new lyrics below:
|
||||
@@ -492,197 +241,96 @@ export default function ReplaceAllLyricsModal({
|
||||
}
|
||||
}}
|
||||
/>
|
||||
|
||||
<Box sx={{ display: 'flex', justifyContent: 'flex-end', gap: 2 }}>
|
||||
<Button
|
||||
variant="contained"
|
||||
onClick={processLyrics}
|
||||
disabled={!inputText.trim()}
|
||||
startIcon={<AutoFixHighIcon />}
|
||||
>
|
||||
Replace All Lyrics
|
||||
</Button>
|
||||
</Box>
|
||||
</Box>
|
||||
) : (
|
||||
// Step 2: Manual sync interface
|
||||
<Box sx={{ display: 'flex', flexDirection: 'column', height: '100%', gap: 2 }}>
|
||||
<Paper sx={{ p: 2, bgcolor: 'background.paper' }}>
|
||||
<Typography variant="h6" gutterBottom>
|
||||
Lyrics Replaced Successfully
|
||||
</Typography>
|
||||
<Typography variant="body2" color="text.secondary">
|
||||
Created {currentSegments.length} segments with {globalSegment?.words.length} words total.
|
||||
Use Manual Sync to set timing for all words.
|
||||
</Typography>
|
||||
</Paper>
|
||||
</DialogContent>
|
||||
|
||||
<Divider />
|
||||
<DialogActions>
|
||||
<Button onClick={handleClose} color="inherit">
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
variant="contained"
|
||||
onClick={processLyrics}
|
||||
disabled={!inputText.trim()}
|
||||
startIcon={<AutoFixHighIcon />}
|
||||
>
|
||||
Continue to Sync
|
||||
</Button>
|
||||
</DialogActions>
|
||||
</Dialog>
|
||||
|
||||
{globalSegment && (
|
||||
<Box sx={{ display: 'flex', gap: 2, flexGrow: 1, minHeight: 0 }}>
|
||||
{/* Timeline Section */}
|
||||
<Box sx={{ flex: 2, display: 'flex', flexDirection: 'column', minHeight: 0 }}>
|
||||
<EditTimelineSection
|
||||
words={globalSegment.words}
|
||||
startTime={timeRange.start}
|
||||
endTime={timeRange.end}
|
||||
originalStartTime={0}
|
||||
originalEndTime={getAudioDuration()}
|
||||
currentStartTime={globalSegment.start_time}
|
||||
currentEndTime={globalSegment.end_time}
|
||||
currentTime={currentTime}
|
||||
isManualSyncing={isManualSyncing}
|
||||
syncWordIndex={syncWordIndex}
|
||||
isSpacebarPressed={isSpacebarPressed}
|
||||
onWordUpdate={handleWordUpdate}
|
||||
onUnsyncWord={handleUnsyncWord}
|
||||
onPlaySegment={onPlaySegment}
|
||||
onStopAudio={() => {
|
||||
// Stop audio playback using global function
|
||||
if (window.toggleAudioPlayback && window.isAudioPlaying) {
|
||||
window.toggleAudioPlayback()
|
||||
}
|
||||
}}
|
||||
startManualSync={startManualSync}
|
||||
pauseManualSync={pauseManualSync}
|
||||
resumeManualSync={resumeManualSync}
|
||||
isPaused={isPaused}
|
||||
isGlobal={true}
|
||||
defaultZoomLevel={10} // Show 10 seconds by default
|
||||
isReplaceAllMode={true} // Prevent zoom changes during sync
|
||||
/>
|
||||
</Box>
|
||||
|
||||
{/* Segment Progress Section */}
|
||||
<SegmentProgressPanel
|
||||
currentSegments={segmentProgressProps.currentSegments}
|
||||
globalSegment={segmentProgressProps.globalSegment}
|
||||
syncWordIndex={segmentProgressProps.syncWordIndex}
|
||||
/>
|
||||
</Box>
|
||||
)}
|
||||
</Box>
|
||||
)}
|
||||
</DialogContent>
|
||||
|
||||
<DialogActions>
|
||||
{isReplaced && (
|
||||
<EditActionBar
|
||||
onReset={handleReset}
|
||||
onClose={handleClose}
|
||||
onSave={handleSave}
|
||||
editedSegment={globalSegment}
|
||||
isGlobal={true}
|
||||
/>
|
||||
)}
|
||||
</DialogActions>
|
||||
</Dialog>
|
||||
)
|
||||
}
|
||||
|
||||
// Memoized Segment Progress Item to prevent unnecessary re-renders
|
||||
const SegmentProgressItem = memo(({
|
||||
segment,
|
||||
index,
|
||||
isActive
|
||||
}: {
|
||||
segment: LyricsSegment
|
||||
index: number
|
||||
isActive: boolean
|
||||
}) => {
|
||||
const wordsWithTiming = segment.words.filter(w =>
|
||||
w.start_time !== null && w.end_time !== null
|
||||
).length
|
||||
const totalWords = segment.words.length
|
||||
const isComplete = wordsWithTiming === totalWords
|
||||
|
||||
return (
|
||||
<Paper
|
||||
key={segment.id}
|
||||
ref={isActive ? (el) => {
|
||||
// Auto-scroll to active segment
|
||||
if (el) {
|
||||
el.scrollIntoView({
|
||||
behavior: 'smooth',
|
||||
block: 'center'
|
||||
})
|
||||
}
|
||||
} : undefined}
|
||||
sx={{
|
||||
p: 1,
|
||||
mb: 1,
|
||||
bgcolor: isActive ? 'primary.light' :
|
||||
isComplete ? 'success.light' : 'background.paper',
|
||||
border: isActive ? 2 : 1,
|
||||
borderColor: isActive ? 'primary.main' : 'divider'
|
||||
}}
|
||||
>
|
||||
<Typography
|
||||
variant="body2"
|
||||
sx={{
|
||||
fontWeight: isActive ? 'bold' : 'normal',
|
||||
mb: 0.5
|
||||
{/* Synchronizer Modal */}
|
||||
<Dialog
|
||||
open={open && mode === 'resync'}
|
||||
onClose={handleClose}
|
||||
maxWidth={false}
|
||||
fullWidth
|
||||
PaperProps={{
|
||||
sx: {
|
||||
height: '90vh',
|
||||
margin: '5vh 2vw',
|
||||
maxWidth: 'calc(100vw - 4vw)',
|
||||
width: 'calc(100vw - 4vw)'
|
||||
}
|
||||
}}
|
||||
>
|
||||
Segment {index + 1}: {segment.text.slice(0, 50)}
|
||||
{segment.text.length > 50 ? '...' : ''}
|
||||
</Typography>
|
||||
<Typography variant="caption" color="text.secondary">
|
||||
{wordsWithTiming}/{totalWords} words synced
|
||||
{isComplete && segment.start_time !== null && segment.end_time !== null && (
|
||||
<>
|
||||
<br />
|
||||
{segment.start_time.toFixed(2)}s - {segment.end_time.toFixed(2)}s
|
||||
</>
|
||||
)}
|
||||
</Typography>
|
||||
</Paper>
|
||||
)
|
||||
})
|
||||
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
|
||||
<IconButton onClick={handleBackToSelection} size="small">
|
||||
<ArrowBackIcon />
|
||||
</IconButton>
|
||||
<Box sx={{ flex: 1 }}>
|
||||
{newSegments.length > 0 ? 'Sync New Lyrics' : 'Re-sync Existing Lyrics'}
|
||||
</Box>
|
||||
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
|
||||
<CloseIcon />
|
||||
</IconButton>
|
||||
</DialogTitle>
|
||||
|
||||
// Memoized Segment Progress Panel
|
||||
const SegmentProgressPanel = memo(({
|
||||
currentSegments,
|
||||
globalSegment,
|
||||
syncWordIndex
|
||||
}: {
|
||||
currentSegments: LyricsSegment[]
|
||||
globalSegment: LyricsSegment | null
|
||||
syncWordIndex: number
|
||||
}) => {
|
||||
return (
|
||||
<Box sx={{ flex: 1, display: 'flex', flexDirection: 'column', minHeight: 0 }}>
|
||||
<Typography variant="h6" gutterBottom>
|
||||
Segment Progress
|
||||
</Typography>
|
||||
<Box sx={{
|
||||
overflow: 'auto',
|
||||
flexGrow: 1,
|
||||
border: 1,
|
||||
borderColor: 'divider',
|
||||
borderRadius: 1,
|
||||
p: 1
|
||||
}}>
|
||||
{currentSegments.map((segment, index) => {
|
||||
const isActive = Boolean(
|
||||
globalSegment &&
|
||||
syncWordIndex >= 0 &&
|
||||
syncWordIndex < globalSegment.words.length &&
|
||||
globalSegment.words[syncWordIndex] &&
|
||||
segment.words.some(w => w.id === globalSegment.words[syncWordIndex].id)
|
||||
)
|
||||
|
||||
return (
|
||||
<SegmentProgressItem
|
||||
key={segment.id}
|
||||
segment={segment}
|
||||
index={index}
|
||||
isActive={isActive}
|
||||
<DialogContent
|
||||
dividers
|
||||
sx={{
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
flexGrow: 1,
|
||||
overflow: 'hidden',
|
||||
p: 2
|
||||
}}
|
||||
>
|
||||
{segmentsForSync.length > 0 ? (
|
||||
<LyricsSynchronizer
|
||||
segments={segmentsForSync}
|
||||
currentTime={currentTime}
|
||||
onPlaySegment={onPlaySegment}
|
||||
onSave={handleSave}
|
||||
onCancel={handleClose}
|
||||
setModalSpacebarHandler={setModalSpacebarHandler}
|
||||
/>
|
||||
)
|
||||
})}
|
||||
</Box>
|
||||
</Box>
|
||||
) : (
|
||||
<Box sx={{
|
||||
display: 'flex',
|
||||
flexDirection: 'column',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'center',
|
||||
height: '100%',
|
||||
gap: 2
|
||||
}}>
|
||||
<Typography variant="h6" color="text.secondary">
|
||||
No lyrics to sync
|
||||
</Typography>
|
||||
<Typography variant="body2" color="text.secondary">
|
||||
Go back and paste new lyrics, or close this modal.
|
||||
</Typography>
|
||||
<Button
|
||||
variant="outlined"
|
||||
onClick={handleBackToSelection}
|
||||
startIcon={<ArrowBackIcon />}
|
||||
>
|
||||
Back to Selection
|
||||
</Button>
|
||||
</Box>
|
||||
)}
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</>
|
||||
)
|
||||
})
|
||||
}
|
||||
@@ -1 +1 @@
|
||||
{"root":["./src/app.tsx","./src/api.ts","./src/main.tsx","./src/theme.ts","./src/types.ts","./src/validation.ts","./src/vite-env.d.ts","./src/components/aifeedbackmodal.tsx","./src/components/addlyricsmodal.tsx","./src/components/agenticcorrectionmetrics.tsx","./src/components/audioplayer.tsx","./src/components/correctedwordwithactions.tsx","./src/components/correctionannotationmodal.tsx","./src/components/correctiondetailcard.tsx","./src/components/correctionmetrics.tsx","./src/components/durationtimelineview.tsx","./src/components/editactionbar.tsx","./src/components/editmodal.tsx","./src/components/edittimelinesection.tsx","./src/components/editwordlist.tsx","./src/components/fileupload.tsx","./src/components/findreplacemodal.tsx","./src/components/header.tsx","./src/components/lyricsanalyzer.tsx","./src/components/metricsdashboard.tsx","./src/components/modeselector.tsx","./src/components/modelselector.tsx","./src/components/previewvideosection.tsx","./src/components/referenceview.tsx","./src/components/replacealllyricsmodal.tsx","./src/components/reviewchangesmodal.tsx","./src/components/segmentdetailsmodal.tsx","./src/components/timelineeditor.tsx","./src/components/timingoffsetmodal.tsx","./src/components/transcriptionview.tsx","./src/components/worddivider.tsx","./src/components/shared/constants.ts","./src/components/shared/styles.ts","./src/components/shared/types.ts","./src/components/shared/components/highlightedtext.tsx","./src/components/shared/components/sourceselector.tsx","./src/components/shared/components/word.tsx","./src/components/shared/hooks/usewordclick.ts","./src/components/shared/utils/keyboardhandlers.ts","./src/components/shared/utils/localstorage.ts","./src/components/shared/utils/referencelinecalculator.ts","./src/components/shared/utils/segmentoperations.ts","./src/components/shared/utils/timingutils.ts","./src/components/shared/utils/wordutils.ts","./src/hooks/usemanualsync.ts","./src/types/global.d.ts"],"version":"5.6.3"}
|
||||
{"root":["./src/app.tsx","./src/api.ts","./src/main.tsx","./src/theme.ts","./src/types.ts","./src/validation.ts","./src/vite-env.d.ts","./src/components/aifeedbackmodal.tsx","./src/components/addlyricsmodal.tsx","./src/components/agenticcorrectionmetrics.tsx","./src/components/audioplayer.tsx","./src/components/correctedwordwithactions.tsx","./src/components/correctionannotationmodal.tsx","./src/components/correctiondetailcard.tsx","./src/components/correctionmetrics.tsx","./src/components/durationtimelineview.tsx","./src/components/editactionbar.tsx","./src/components/editmodal.tsx","./src/components/edittimelinesection.tsx","./src/components/editwordlist.tsx","./src/components/fileupload.tsx","./src/components/findreplacemodal.tsx","./src/components/header.tsx","./src/components/lyricsanalyzer.tsx","./src/components/metricsdashboard.tsx","./src/components/modeselectionmodal.tsx","./src/components/modeselector.tsx","./src/components/modelselector.tsx","./src/components/previewvideosection.tsx","./src/components/referenceview.tsx","./src/components/replacealllyricsmodal.tsx","./src/components/reviewchangesmodal.tsx","./src/components/segmentdetailsmodal.tsx","./src/components/timelineeditor.tsx","./src/components/timingoffsetmodal.tsx","./src/components/transcriptionview.tsx","./src/components/worddivider.tsx","./src/components/lyricssynchronizer/synccontrols.tsx","./src/components/lyricssynchronizer/timelinecanvas.tsx","./src/components/lyricssynchronizer/upcomingwordsbar.tsx","./src/components/lyricssynchronizer/index.tsx","./src/components/shared/constants.ts","./src/components/shared/styles.ts","./src/components/shared/types.ts","./src/components/shared/components/highlightedtext.tsx","./src/components/shared/components/sourceselector.tsx","./src/components/shared/components/word.tsx","./src/components/shared/hooks/usewordclick.ts","./src/components/shared/utils/keyboardhandlers.ts","./src/components/shared/utils/localstorage.ts","./src/components/shared/utils/referencelinecalculator.ts","./src/components/shared/utils/segmentoperations.ts","./src/components/shared/utils/timingutils.ts","./src/components/shared/utils/wordutils.ts","./src/hooks/usemanualsync.ts","./src/types/global.d.ts"],"version":"5.6.3"}
|
||||
@@ -0,0 +1 @@
|
||||
{"version":3,"file":"index-COYImAcx.js","sources":["../../node_modules/react/cjs/react.production.min.js","../../node_modules/react/index.js","../../node_modules/react/cjs/react-jsx-runtime.production.min.js","../../node_modules/react/jsx-runtime.js","../../node_modules/scheduler/cjs/scheduler.production.min.js","../../node_modules/scheduler/index.js","../../node_modules/react-dom/cjs/react-dom.production.min.js","../../node_modules/react-dom/index.js","../../node_modules/react-dom/client.js","../../node_modules/@mui/utils/esm/formatMuiErrorMessage/formatMuiErrorMessage.js","../../node_modules/@mui/material/styles/identifier.js","../../node_modules/@babel/runtime/helpers/esm/extends.js","../../node_modules/@emotion/sheet/dist/emotion-sheet.esm.js","../../node_modules/stylis/src/Enum.js","../../node_modules/stylis/src/Utility.js","../../node_modules/stylis/src/Tokenizer.js","../../node_modules/stylis/src/Parser.js","../../node_modules/stylis/src/Serializer.js","../../node_modules/stylis/src/Middleware.js","../../node_modules/@emotion/memoize/dist/emotion-memoize.esm.js","../../node_modules/@emotion/cache/dist/emotion-cache.browser.esm.js","../../node_modules/react-is/cjs/react-is.production.min.js","../../node_modules/react-is/index.js","../../node_modules/hoist-non-react-statics/dist/hoist-non-react-statics.cjs.js","../../node_modules/@emotion/utils/dist/emotion-utils.browser.esm.js","../../node_modules/@emotion/hash/dist/emotion-hash.esm.js","../../node_modules/@emotion/unitless/dist/emotion-unitless.esm.js","../../node_modules/@emotion/serialize/dist/emotion-serialize.esm.js","../../node_modules/@emotion/use-insertion-effect-with-fallbacks/dist/emotion-use-insertion-effect-with-fallbacks.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-element-f0de968e.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-react.browser.esm.js","../../node_modules/@emotion/is-prop-valid/dist/emotion-is-prop-valid.esm.js","../../node_modules/@emotion/styled/base/dist/emotion-styled-base.browser.esm.js","../../node_modules/@emotion/styled/dist/emotion-styled.browser.esm.js","../../node_modules/@mui/styled-engine/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/styled-engine/index.js","../../node_modules/@mui/utils/node_modules/react-is/cjs/react-is.production.js","../../node_modules/@mui/utils/node_modules/react-is/index.js","../../node_modules/@mui/utils/esm/deepmerge/deepmerge.js","../../node_modules/@mui/system/esm/createBreakpoints/createBreakpoints.js","../../node_modules/@mui/system/esm/cssContainerQueries/cssContainerQueries.js","../../node_modules/@mui/system/esm/createTheme/shape.js","../../node_modules/@mui/system/esm/merge/merge.js","../../node_modules/@mui/system/esm/breakpoints/breakpoints.js","../../node_modules/@mui/utils/esm/capitalize/capitalize.js","../../node_modules/@mui/system/esm/style/style.js","../../node_modules/@mui/system/esm/memoize/memoize.js","../../node_modules/@mui/system/esm/spacing/spacing.js","../../node_modules/@mui/system/esm/createTheme/createSpacing.js","../../node_modules/@mui/system/esm/compose/compose.js","../../node_modules/@mui/system/esm/borders/borders.js","../../node_modules/@mui/system/esm/cssGrid/cssGrid.js","../../node_modules/@mui/system/esm/palette/palette.js","../../node_modules/@mui/system/esm/sizing/sizing.js","../../node_modules/@mui/system/esm/styleFunctionSx/defaultSxConfig.js","../../node_modules/@mui/system/esm/styleFunctionSx/styleFunctionSx.js","../../node_modules/@mui/system/esm/createTheme/applyStyles.js","../../node_modules/@mui/system/esm/createTheme/createTheme.js","../../node_modules/@mui/system/esm/useThemeWithoutDefault/useThemeWithoutDefault.js","../../node_modules/@mui/system/esm/useTheme/useTheme.js","../../node_modules/@mui/system/esm/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/system/esm/styleFunctionSx/extendSxProp.js","../../node_modules/@mui/utils/esm/ClassNameGenerator/ClassNameGenerator.js","../../node_modules/clsx/dist/clsx.mjs","../../node_modules/@mui/system/esm/createBox/createBox.js","../../node_modules/@mui/utils/esm/generateUtilityClass/generateUtilityClass.js","../../node_modules/@mui/utils/esm/generateUtilityClasses/generateUtilityClasses.js","../../node_modules/@mui/system/esm/preprocessStyles.js","../../node_modules/@mui/system/esm/createStyled/createStyled.js","../../node_modules/@mui/system/esm/styled/styled.js","../../node_modules/@mui/utils/esm/resolveProps/resolveProps.js","../../node_modules/@mui/system/esm/useThemeProps/getThemeProps.js","../../node_modules/@mui/system/esm/useThemeProps/useThemeProps.js","../../node_modules/@mui/utils/esm/useEnhancedEffect/useEnhancedEffect.js","../../node_modules/@mui/system/esm/useMediaQuery/useMediaQuery.js","../../node_modules/@mui/utils/esm/clamp/clamp.js","../../node_modules/@mui/system/esm/colorManipulator/colorManipulator.js","../../node_modules/@mui/utils/esm/createChainedFunction/createChainedFunction.js","../../node_modules/@mui/utils/esm/debLine truncated
|
||||
@@ -1 +0,0 @@
|
||||
{"version":3,"file":"index-DdJTDWH3.js","sources":["../../node_modules/react/cjs/react.production.min.js","../../node_modules/react/index.js","../../node_modules/react/cjs/react-jsx-runtime.production.min.js","../../node_modules/react/jsx-runtime.js","../../node_modules/scheduler/cjs/scheduler.production.min.js","../../node_modules/scheduler/index.js","../../node_modules/react-dom/cjs/react-dom.production.min.js","../../node_modules/react-dom/index.js","../../node_modules/react-dom/client.js","../../node_modules/@mui/utils/esm/formatMuiErrorMessage/formatMuiErrorMessage.js","../../node_modules/@mui/material/styles/identifier.js","../../node_modules/@babel/runtime/helpers/esm/extends.js","../../node_modules/@emotion/sheet/dist/emotion-sheet.esm.js","../../node_modules/stylis/src/Enum.js","../../node_modules/stylis/src/Utility.js","../../node_modules/stylis/src/Tokenizer.js","../../node_modules/stylis/src/Parser.js","../../node_modules/stylis/src/Serializer.js","../../node_modules/stylis/src/Middleware.js","../../node_modules/@emotion/memoize/dist/emotion-memoize.esm.js","../../node_modules/@emotion/cache/dist/emotion-cache.browser.esm.js","../../node_modules/react-is/cjs/react-is.production.min.js","../../node_modules/react-is/index.js","../../node_modules/hoist-non-react-statics/dist/hoist-non-react-statics.cjs.js","../../node_modules/@emotion/utils/dist/emotion-utils.browser.esm.js","../../node_modules/@emotion/hash/dist/emotion-hash.esm.js","../../node_modules/@emotion/unitless/dist/emotion-unitless.esm.js","../../node_modules/@emotion/serialize/dist/emotion-serialize.esm.js","../../node_modules/@emotion/use-insertion-effect-with-fallbacks/dist/emotion-use-insertion-effect-with-fallbacks.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-element-f0de968e.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-react.browser.esm.js","../../node_modules/@emotion/is-prop-valid/dist/emotion-is-prop-valid.esm.js","../../node_modules/@emotion/styled/base/dist/emotion-styled-base.browser.esm.js","../../node_modules/@emotion/styled/dist/emotion-styled.browser.esm.js","../../node_modules/@mui/styled-engine/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/styled-engine/index.js","../../node_modules/@mui/utils/node_modules/react-is/cjs/react-is.production.js","../../node_modules/@mui/utils/node_modules/react-is/index.js","../../node_modules/@mui/utils/esm/deepmerge/deepmerge.js","../../node_modules/@mui/system/esm/createBreakpoints/createBreakpoints.js","../../node_modules/@mui/system/esm/cssContainerQueries/cssContainerQueries.js","../../node_modules/@mui/system/esm/createTheme/shape.js","../../node_modules/@mui/system/esm/merge/merge.js","../../node_modules/@mui/system/esm/breakpoints/breakpoints.js","../../node_modules/@mui/utils/esm/capitalize/capitalize.js","../../node_modules/@mui/system/esm/style/style.js","../../node_modules/@mui/system/esm/memoize/memoize.js","../../node_modules/@mui/system/esm/spacing/spacing.js","../../node_modules/@mui/system/esm/createTheme/createSpacing.js","../../node_modules/@mui/system/esm/compose/compose.js","../../node_modules/@mui/system/esm/borders/borders.js","../../node_modules/@mui/system/esm/cssGrid/cssGrid.js","../../node_modules/@mui/system/esm/palette/palette.js","../../node_modules/@mui/system/esm/sizing/sizing.js","../../node_modules/@mui/system/esm/styleFunctionSx/defaultSxConfig.js","../../node_modules/@mui/system/esm/styleFunctionSx/styleFunctionSx.js","../../node_modules/@mui/system/esm/createTheme/applyStyles.js","../../node_modules/@mui/system/esm/createTheme/createTheme.js","../../node_modules/@mui/system/esm/useThemeWithoutDefault/useThemeWithoutDefault.js","../../node_modules/@mui/system/esm/useTheme/useTheme.js","../../node_modules/@mui/system/esm/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/system/esm/styleFunctionSx/extendSxProp.js","../../node_modules/@mui/utils/esm/ClassNameGenerator/ClassNameGenerator.js","../../node_modules/clsx/dist/clsx.mjs","../../node_modules/@mui/system/esm/createBox/createBox.js","../../node_modules/@mui/utils/esm/generateUtilityClass/generateUtilityClass.js","../../node_modules/@mui/utils/esm/generateUtilityClasses/generateUtilityClasses.js","../../node_modules/@mui/system/esm/preprocessStyles.js","../../node_modules/@mui/system/esm/createStyled/createStyled.js","../../node_modules/@mui/system/esm/styled/styled.js","../../node_modules/@mui/utils/esm/resolveProps/resolveProps.js","../../node_modules/@mui/system/esm/useThemeProps/getThemeProps.js","../../node_modules/@mui/system/esm/useThemeProps/useThemeProps.js","../../node_modules/@mui/utils/esm/useEnhancedEffect/useEnhancedEffect.js","../../node_modules/@mui/system/esm/useMediaQuery/useMediaQuery.js","../../node_modules/@mui/utils/esm/clamp/clamp.js","../../node_modules/@mui/system/esm/colorManipulator/colorManipulator.js","../../node_modules/@mui/utils/esm/createChainedFunction/createChainedFunction.js","../../node_modules/@mui/utils/esm/debLine truncated
|
||||