Compare commits

...
Author SHA1 Message Date
Andrew Beveridge f99d6c1fb4 style: remove extraneous f-string prefixes
Address CodeRabbit nitpick comments - remove f-string prefixes from
strings without placeholders.
2025-12-19 18:42:03 -05:00
Andrew Beveridge f8a45bfbd2 fix: countdown padding synchronization for instrumental audio
When LyricsTranscriber adds 3-second countdown padding to vocals,
the instrumental audio must be padded to match. This was broken when
transcription was skipped due to existing files.

Changes:
- Add _detect_countdown_padding_from_lrc() to detect padding from existing LRC
- Update early returns in transcribe_lyrics() to include padding info
- Add _scan_directory_for_instrumentals() to find existing instrumentals
- Fix counting bug in apply_countdown_padding_to_instrumentals
- Add countdown_padding_seconds safety net to KaraokeFinalise
- Store padding metadata in lyrics_worker job state_data
- Pass padding info to video_worker for instrumental sync
- Add comprehensive unit tests for new functionality
2025-12-19 18:35:37 -05:00
Andrew Beveridge ☄️ f524796f7f Merge pull request #14 from nomadkaraoke/feature/batch-6-two-phase-workflow
feat: Batch 6 - Two-Phase Workflow (prep-only, finalise-only, keep-brand-code)
2025-12-19 00:07:15 -05:00
Andrew Beveridge ☄️ 8372ed211a Merge pull request #34 from nomadkaraoke/fix/youtube-duplicate-checker-channel-filter
Fix YouTube duplicate checker returning videos from other channels
2025-12-19 00:06:11 -05:00
Andrew Beveridge 78068ea1d8 Fix YouTube duplicate checker returning videos from other channels
The YouTube Data API v3 search endpoint's channelId parameter doesn't
always filter correctly and sometimes returns videos from other channels.

This fix verifies each search result's snippet.channelId matches the
user's channel ID before considering it as a potential match, preventing
false positives from unrelated videos with similar titles.

Added tests for the channel filtering behavior.
2025-12-18 23:28:23 -05:00
Andrew Beveridge 84d7c12f13 fix: Address CodeRabbit review comments for Batch 6 PR
- Add PREP_COMPLETE to RENDERING_VIDEO state transitions (job.py)
- Simplify instrumental detection logic for consistency (remote_cli.py)
- Add shell escaping (shlex.quote) for continuation command display (remote_cli.py)
- Update test for --finalise-only now that it's implemented (test_remote_cli.py)
- Add test for PREP_COMPLETE status (test_remote_cli.py)
- Bump version to 0.71.69
2025-12-18 23:22:05 -05:00
Andrew Beveridge 7a4ed9661c Merge origin/main into feature/batch-6-two-phase-workflow 2025-12-18 22:42:49 -05:00
Andrew Beveridge ☄️ dd62c32026 Merge pull request #33 from nomadkaraoke/fix/uploaded-instrumental-selection
Fix: Handle 'uploaded' selection in instrumental review CLI
2025-12-18 21:35:28 -05:00
Andrew Beveridge c130fa8a07 Merge remote-tracking branch 'origin/main' into fix/uploaded-instrumental-selection 2025-12-18 20:24:14 -05:00
Andrew Beveridge ☄️ e0d1fa7536 Merge pull request #32 from nomadkaraoke/replicate-midico-synchronizer
feat: Add MidiCo-style lyrics synchronizer for re-syncing existing lyrics
2025-12-18 20:23:47 -05:00
Andrew Beveridge 29ffbd4ae2 fix: handle 'uploaded' selection in instrumental review CLI
The instrumental review UI supports uploading custom instrumental files,
but gen_cli.py was missing the handler for the 'uploaded' selection type.

When a user uploaded a custom instrumental and selected it, the CLI would
log 'Unknown selection: uploaded' and crash with an unhelpful error message.

This fix adds proper handling for the 'uploaded' case, retrieving the
uploaded file path from the server just like the 'custom' case does.
2025-12-18 20:21:00 -05:00
Andrew Beveridge e440f0bbcd fix: Address CodeRabbit review comments
- Fix word move timing bug: ensure end time is always after start time
  (prevents invalid timings where end < start with very negative deltas)
- Add typeof checks before calling window methods for runtime type safety
2025-12-18 20:13:13 -05:00
Andrew Beveridge e0f5a84e54 feat: Add MidiCo-style lyrics synchronizer for re-syncing existing lyrics
Implements a new synchronizer UI that closely replicates MidiCo's UX:

- Mode selection modal: Choose between "Re-sync Existing Lyrics" or "Replace All Lyrics"
- Canvas-based timeline for high-performance word block rendering
- Two-level alternating segment layout to prevent text overlap
- Spacebar sync mode: tap for short words, hold for longer words
- Current word highlighting (red text + brighter block)
- Word block manipulation:
  - Resize words by dragging handle on right edge
  - Drag to move single or multiple selected words
- Scroll/zoom controls for timeline navigation
- Play/Stop/Pause controls for audio playback
- "Unsync from Cursor" to re-sync only the drifted portion
- Fixed-height layout sections to prevent UI jumping during sync

This addresses the core problem of lyrics that start well-synced but drift
progressively out of sync in the second half of a song.
2025-12-18 19:59:08 -05:00
Andrew Beveridge 8da57da767 Added reference doc explaining the plan to replace our "Edit All" functionality with something much more closely aligned with MidiCo functionality 2025-12-18 16:27:59 -05:00
Andrew Beveridge b840a12669 Added midico synchronizer reference guide 2025-12-18 16:02:19 -05:00
Andrew Beveridge ☄️ f7038ce72c Merge pull request #31 from nomadkaraoke/fix/ctrlc-audio-fetcher
Ensure we can quit during audio fetch
2025-12-18 12:04:35 -05:00
Andrew Beveridge 138ddeecf5 Ensure we can quit during audio fetch 2025-12-18 11:46:33 -05:00
Andrew Beveridge ☄️ a7d9bb40b9 Merge pull request #30 from nomadkaraoke/feature/local-review-ui
Support local review UI for frontend development
2025-12-18 02:16:55 -05:00
Andrew Beveridge ☄️ 20f842fa85 Merge pull request #29 from nomadkaraoke/feature/annotation-toggle
Add toggle to disable annotation prompts when editing lyrics
2025-12-18 02:07:51 -05:00
Andrew Beveridge 1b74199032 Default to bundled local frontend for lyrics review
- Change default from hosted https://lyrics.nomadkaraoke.com to local
- Local karaoke-gen now uses bundled frontend served by FastAPI
- No external dependency for local runs
- Can override with LYRICS_REVIEW_UI_URL env var if needed:
  - 'local' (default): bundled frontend at http://localhost:8000
  - 'http://localhost:5173': Vite dev server for frontend development
  - 'https://lyrics.nomadkaraoke.com': hosted version (legacy)

Also includes rebuilt frontend assets with annotation toggle feature.

Version bump: 0.71.65 -> 0.71.66
2025-12-18 01:58:53 -05:00
Andrew Beveridge e75c5b2723 Support local review UI for development via --review-ui-url
- Add --review-ui-url argument to local gen_cli (was remote-only)
- Set LYRICS_REVIEW_UI_URL env var from CLI argument
- Allows pointing to local Vite dev server (http://localhost:5173)
- Supports both REVIEW_UI_URL and LYRICS_REVIEW_UI_URL env vars

Usage for local frontend development:
  karaoke-gen --review-ui-url http://localhost:5173 ...

Version bump: 0.71.64 -> 0.71.65
2025-12-18 01:53:34 -05:00
Andrew Beveridge 36a60c09da Merge origin/main into feature/batch-6-two-phase-workflow
Resolved conflicts to merge Batch 6 (Two-Phase Workflow) with:
- Batch 5 (FlacFetch Audio Search) - Added SEARCHING_AUDIO states
- Instrumental review UI fixes
- Worker logging context improvements

Key merge resolutions:
- job.py: Combined state transitions for SEARCHING_AUDIO and AWAITING_INSTRUMENTAL_SELECTION
- remote_cli.py: Added prep_only/keep_brand_code to both URL and file submission paths
- render_video_worker.py: Added job_logging_context while preserving prep_only transition logic
- file_upload.py: Re-added all finalise-only endpoints and models

All 180 tests pass.
2025-12-14 13:15:10 -05:00
Andrew Beveridge ba4534fdbd feat: Batch 6 - Two-Phase Workflow (prep-only, finalise-only, keep-brand-code)
Implements the two-phase workflow for remote CLI, allowing users to:
- Run prep-only mode to stop after review phase and download all outputs
- Run finalise-only mode to upload local prep outputs and continue with finalization
- Preserve existing brand codes when re-running finalisation

Backend changes:
- Added PREP_COMPLETE status to JobStatus enum
- Added prep_only, finalise_only, keep_brand_code fields to Job/JobCreate models
- Updated state transitions to allow new workflow paths
- New endpoint /api/jobs/create-finalise-only for uploading prep outputs
- New endpoint /api/jobs/{job_id}/finalise-uploads-complete
- Modified render_video_worker to transition to PREP_COMPLETE for prep-only jobs
- Modified video_worker to use preserved brand code when set

Remote CLI changes:
- Added --prep-only flag support (passes prep_only=true to backend)
- Added --finalise-only flow (detects local prep folder, uploads files)
- Added --keep-brand-code support (extracts from folder name)
- Added PREP_COMPLETE status handling in job monitor

Tests:
- Added comprehensive tests for new Job model fields and PREP_COMPLETE status
- Added tests for finalise-only endpoints and Pydantic models
- All 151 tests pass

Version bumped to 0.71.27
2025-12-11 02:38:22 -05:00
46 changed files with 6673 additions and 1094 deletions

No files matched your search

@@ -0,0 +1,217 @@
---
name: MidiCo-Style Synchronizer
overview: Build a new MidiCo-style Synchronizer component for efficient re-syncing of lyrics timing, particularly for fixing progressive drift in the second half of songs. This involves refactoring the "Edit All" entry point and creating a high-performance timeline-based sync interface.
todos:
- id: mode-selection-modal
content: Create ModeSelectionModal for 'Replace All' vs 'Re-sync Existing' choice
status: completed
- id: timeline-canvas
content: Build canvas-based TimelineCanvas component for high-performance word block rendering
status: completed
- id: upcoming-words-bar
content: Create UpcomingWordsBar component showing next unsynced words in fixed position
status: completed
- id: sync-controls
content: Build SyncControls with Start Sync, Clear Sync, Unsync from Cursor, Delete Selected buttons
status: completed
- id: lyrics-synchronizer
content: Create main LyricsSynchronizer component integrating all sub-components
status: completed
- id: unsync-from-cursor
content: Add 'unsync from cursor' functionality to useManualSync hook
status: completed
- id: integrate-modal
content: Integrate new synchronizer into ReplaceAllLyricsModal with mode selection
status: completed
- id: word-selection
content: Implement single-click and drag-select for word blocks
status: completed
- id: zoom-slider
content: Add smooth zoom slider with 4.5s-24s range
status: completed
- id: performance-test
content: Test performance with long songs and optimize as needed
status: completed
---
# MidiCo-Style Synchronizer Implementation
## Current State Analysis
The existing implementation in [`ReplaceAllLyricsModal.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/ReplaceAllLyricsModal.tsx) has two phases:
1. **Paste Phase** - User pastes new lyrics text
2. **Sync Phase** - Manual sync using spacebar with timeline view
Key existing components:
- [`useManualSync.ts`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/hooks/useManualSync.ts) - Spacebar handling, tap/hold detection
- [`EditTimelineSection.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/EditTimelineSection.tsx) - Timeline controls with zoom/scroll
- [`TimelineEditor.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/TimelineEditor.tsx) - Timeline rendering
**Problems with current implementation:**
1. Forces full lyrics replacement - no option to keep existing sync data
2. Performance issues (laggy, slow rendering)
3. Poor UX for seeing upcoming words while syncing
4. No "unsync from cursor" feature for fixing drift
5. No two-level word block layout for overlapping segments
## Architecture Overview
```mermaid
flowchart TB
subgraph Entry [Entry Point]
EditAll[Edit All Button]
EditAll --> ChoiceModal[Mode Selection Modal]
ChoiceModal -->|Replace All| PastePhase[Paste New Lyrics]
ChoiceModal -->|Re-sync Existing| SyncView[Synchronizer View]
PastePhase --> SyncView
end
subgraph Sync [Synchronizer Component]
SyncView --> Timeline[Timeline Canvas]
SyncView --> Controls[Control Buttons]
SyncView --> UpcomingWords[Upcoming Words Display]
Timeline --> WordBlocks[Word Blocks Renderer]
Timeline --> Playhead[Playhead Marker]
Timeline --> TimeBar[Time Bar]
end
subgraph Audio [Audio Integration]
AudioPlayer[Audio Player]
AudioPlayer <--> SyncView
end
```
## Implementation Plan
### Phase 1: Entry Point Refactor
Modify the "Edit All" button flow to show a choice modal:
```typescript
// New ModeSelectionModal component
interface ModeSelectionModalProps {
open: boolean
onClose: () => void
onSelectReplace: () => void // Goes to paste phase
onSelectResync: () => void // Goes directly to sync view
}
```
- **Replace All Lyrics**: Current flow (paste box -> sync)
- **Re-sync Existing Lyrics**: Load current segments directly into sync view
### Phase 2: New Synchronizer Component
Create a new high-performance `LyricsSynchronizer.tsx` component inspired by MidiCo:
#### Core Features
1. **Timeline Canvas** (HTML5 Canvas for performance)
- Time bar at top (MM:SS format with decisecond ticks)
- Playhead with arrow indicator + vertical line
- Red word blocks with text labels
- Two-level word layout for overlapping segments
- Zoom slider (4.5s to 24s visible range)
- Horizontal scroll with smooth panning
2. **Upcoming Words Display** (dual location)
- Fixed position above timeline: next words as red/white blocks
- On timeline: words appear to right of playhead during sync
3. **Control Buttons**
- **Start Sync** - Begin sync mode from first unsynced word
- **Clear Sync** - Reset all word timings to null
- **Edit Lyrics** - Modal to edit lyrics text (warns about timing loss)
- **Unsync from Cursor** - Critical: reset all words after playhead position
- **Split/Edit Word** - Edit single selected word
- **Delete Selected** - Remove selected word blocks
4. **Word Selection**
- Single click: select one word (white border)
- Click-drag on background: selection rectangle for multi-select
- Click on time bar: set playhead position
5. **Spacebar Sync Logic** (reuse/enhance `useManualSync`)
- Tap: set start time, default 500ms duration
- Hold: set both start and end times
- Handle previous word end time calculation
### Phase 3: Performance Optimization
The current `TimelineEditor` uses DOM elements which causes lag. New approach:
1. **Canvas-based rendering** for word blocks and timeline
- Single canvas element, no DOM nodes per word
- RequestAnimationFrame for smooth playhead updates
- Only re-render changed regions
2. **Virtual scrolling** for very long songs
- Only render visible word blocks
- Cache off-screen block positions
3. **Debounced state updates**
- Batch word timing updates
- Throttle zoom/scroll handlers
### Phase 4: Apply/Cancel Flow
Changes are held in local state until user clicks Apply:
```typescript
interface SynchronizerState {
words: Word[] // Working copy of words
segments: LyricsSegment[] // Working copy of segments
isDirty: boolean // Has unsaved changes
}
```
- **Apply**: Commit changes to parent component
- **Cancel**: Discard changes, close modal
## File Changes
### New Files
- `src/components/LyricsSynchronizer.tsx` - Main synchronizer component
- `src/components/LyricsSynchronizer/TimelineCanvas.tsx` - Canvas-based timeline
- `src/components/LyricsSynchronizer/UpcomingWordsBar.tsx` - Fixed upcoming words display
- `src/components/LyricsSynchronizer/SyncControls.tsx` - Control buttons
- `src/components/ModeSelectionModal.tsx` - Entry point choice modal
### Modified Files
- [`ReplaceAllLyricsModal.tsx`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/components/ReplaceAllLyricsModal.tsx) - Add mode selection, integrate new synchronizer
- [`useManualSync.ts`](lyrics_transcriber_temp/lyrics_transcriber/frontend/src/hooks/useManualSync.ts) - Add "unsync from cursor" functionality
## Key UX Details from MidiCo Reference
Per the documentation at [`MIDICO_SYNCHRONIZER_REFERENCE.md`](lyrics_transcriber_temp/docs/MIDICO_SYNCHRONIZER_REFERENCE.md):
1. **Zoom range**: 4.5s (max zoom) to 24s (min zoom) visible
2. **Timeline click behavior**:
- Time bar click = set playhead (no playback change)
- Background click = nothing (prevent accidental deselection)
- Word block click = select
3. **Two-level word blocks**: Adjacent/overlapping words display on alternating vertical levels
4. **Playhead**: Small arrow in time bar + white vertical line spanning timeline height
5. **Selection rectangle**: Click-drag on background creates selection box
## Testing Strategy
1. Manual testing with the provided URL: `http://localhost:8000/?baseApiUrl=...`
2. Test sync accuracy with songs that have known drift issues
3. Performance testing with 400+ word songs
4. Test "unsync from cursor" workflow for fixing drift
+476
View File
@@ -131,6 +131,10 @@ class CreateJobWithUploadUrlsRequest(BaseModel):
# Existing instrumental configuration (Batch 3)
existing_instrumental: bool = Field(False, description="Whether an existing instrumental file is being uploaded")
# Two-phase workflow configuration (Batch 6)
prep_only: bool = Field(False, description="Stop after review phase, don't run finalisation")
keep_brand_code: Optional[str] = Field(None, description="Preserve existing brand code instead of generating new one")
class SignedUploadUrl(BaseModel):
@@ -638,6 +642,56 @@ VALID_FILE_TYPES = {
'existing_instrumental': ALLOWED_AUDIO_EXTENSIONS, # Batch 3: user-provided instrumental
}
# Valid file types for finalise-only mode (Batch 6)
ALLOWED_VIDEO_EXTENSIONS = {'.mkv', '.mov', '.mp4'}
FINALISE_ONLY_FILE_TYPES = {
'audio': ALLOWED_AUDIO_EXTENSIONS, # Original audio
'with_vocals': ALLOWED_VIDEO_EXTENSIONS, # Karaoke video from prep
'title_screen': ALLOWED_VIDEO_EXTENSIONS, # Title screen video
'end_screen': ALLOWED_VIDEO_EXTENSIONS, # End screen video
'title_jpg': ALLOWED_IMAGE_EXTENSIONS, # Title screen JPG
'title_png': ALLOWED_IMAGE_EXTENSIONS, # Title screen PNG
'end_jpg': ALLOWED_IMAGE_EXTENSIONS, # End screen JPG
'end_png': ALLOWED_IMAGE_EXTENSIONS, # End screen PNG
'instrumental_clean': ALLOWED_AUDIO_EXTENSIONS, # Clean instrumental
'instrumental_backing': ALLOWED_AUDIO_EXTENSIONS, # Instrumental with backing vocals
'lrc': {'.lrc'}, # LRC lyrics
'ass': {'.ass'}, # ASS subtitles
'corrections': {'.json'}, # Corrections JSON
'style_params': {'.json'}, # Style params
}
# Pydantic models for finalise-only flow (Batch 6)
class FinaliseOnlyFileRequest(BaseModel):
"""Information about a prep output file to be uploaded for finalise-only mode."""
filename: str = Field(..., description="Original filename with extension")
content_type: str = Field(..., description="MIME type of the file")
file_type: str = Field(..., description="Type of prep file: 'with_vocals', 'title_screen', 'end_screen', 'instrumental_clean', 'instrumental_backing', 'lrc', etc.")
class CreateFinaliseOnlyJobRequest(BaseModel):
"""Request to create a finalise-only job with prep output files."""
artist: str = Field(..., description="Artist name")
title: str = Field(..., description="Song title")
files: List[FinaliseOnlyFileRequest] = Field(..., description="List of prep output files to upload")
# Processing options
enable_cdg: bool = Field(True, description="Generate CDG+MP3 package")
enable_txt: bool = Field(True, description="Generate TXT+MP3 package")
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
keep_brand_code: Optional[str] = Field(None, description="Preserve existing brand code from folder name")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
# Distribution options
dropbox_path: Optional[str] = Field(None, description="Dropbox folder path for organized output")
gdrive_folder_id: Optional[str] = Field(None, description="Google Drive folder ID for public share uploads")
async def _validate_audio_durations(
storage: StorageService,
@@ -1243,3 +1297,425 @@ async def create_job_from_url(
except Exception as e:
logger.error(f"Error creating job from URL: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e)) from e
# ============================================================================
# Finalise-Only Upload Flow (Batch 6)
# ============================================================================
def _get_gcs_path_for_finalise_file(job_id: str, file_type: str, filename: str) -> str:
"""Generate the GCS path for a finalise-only file based on its type."""
ext = Path(filename).suffix.lower()
# Map file types to appropriate GCS paths
if file_type == 'audio':
return f"uploads/{job_id}/audio/{filename}"
elif file_type == 'with_vocals':
return f"jobs/{job_id}/videos/with_vocals{ext}"
elif file_type == 'title_screen':
return f"jobs/{job_id}/screens/title{ext}"
elif file_type == 'end_screen':
return f"jobs/{job_id}/screens/end{ext}"
elif file_type == 'title_jpg':
return f"jobs/{job_id}/screens/title.jpg"
elif file_type == 'title_png':
return f"jobs/{job_id}/screens/title.png"
elif file_type == 'end_jpg':
return f"jobs/{job_id}/screens/end.jpg"
elif file_type == 'end_png':
return f"jobs/{job_id}/screens/end.png"
elif file_type == 'instrumental_clean':
return f"jobs/{job_id}/stems/instrumental_clean{ext}"
elif file_type == 'instrumental_backing':
return f"jobs/{job_id}/stems/instrumental_with_backing{ext}"
elif file_type == 'lrc':
return f"jobs/{job_id}/lyrics/karaoke.lrc"
elif file_type == 'ass':
return f"jobs/{job_id}/lyrics/karaoke.ass"
elif file_type == 'corrections':
return f"jobs/{job_id}/lyrics/corrections.json"
elif file_type == 'style_params':
return f"uploads/{job_id}/style/style_params.json"
else:
return f"uploads/{job_id}/other/{filename}"
@router.post("/jobs/create-finalise-only")
async def create_finalise_only_job(
request: Request,
body: CreateFinaliseOnlyJobRequest,
):
"""
Create a finalise-only job for continuing from a local prep phase.
This endpoint is used when:
1. User previously ran --prep-only which created local prep outputs
2. User optionally made manual edits to stems, lyrics, etc.
3. User now wants cloud to handle finalisation (encoding, distribution)
Required files:
- with_vocals: The karaoke video from prep (*.mkv or *.mov)
- title_screen: Title screen video
- end_screen: End screen video
- instrumental_clean or instrumental_backing: At least one instrumental
The endpoint returns signed URLs for uploading all the prep files.
"""
try:
# Validate files list
if not body.files:
raise HTTPException(status_code=400, detail="At least one file is required")
# Validate file types
file_types = {f.file_type for f in body.files}
# Check required files for finalise-only
if 'with_vocals' not in file_types:
raise HTTPException(
status_code=400,
detail="with_vocals video is required for finalise-only mode"
)
if 'title_screen' not in file_types:
raise HTTPException(
status_code=400,
detail="title_screen video is required for finalise-only mode"
)
if 'end_screen' not in file_types:
raise HTTPException(
status_code=400,
detail="end_screen video is required for finalise-only mode"
)
if 'instrumental_clean' not in file_types and 'instrumental_backing' not in file_types:
raise HTTPException(
status_code=400,
detail="At least one instrumental (clean or backing) is required"
)
# Validate file extensions
for file_info in body.files:
if file_info.file_type not in FINALISE_ONLY_FILE_TYPES:
raise HTTPException(
status_code=400,
detail=f"Invalid file_type: '{file_info.file_type}'. Valid types: {list(FINALISE_ONLY_FILE_TYPES.keys())}"
)
ext = Path(file_info.filename).suffix.lower()
allowed_extensions = FINALISE_ONLY_FILE_TYPES[file_info.file_type]
if ext not in allowed_extensions:
raise HTTPException(
status_code=400,
detail=f"Invalid file extension '{ext}' for file_type '{file_info.file_type}'. Allowed: {allowed_extensions}"
)
# Apply default distribution settings
settings = get_settings()
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
# Validate distribution credentials if services are requested
invalid_services = []
credential_manager = get_credential_manager()
if body.enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
if effective_dropbox_path:
result = credential_manager.check_dropbox_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"dropbox ({result.message})")
if effective_gdrive_folder_id:
result = credential_manager.check_gdrive_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"gdrive ({result.message})")
if invalid_services:
raise HTTPException(
status_code=400,
detail={
"error": "credentials_invalid",
"message": f"The following distribution services need re-authorization: {', '.join(invalid_services)}",
"invalid_services": invalid_services,
"auth_url": "/api/auth/status"
}
)
# Extract request metadata
request_metadata = extract_request_metadata(request, created_from="finalise_only_upload")
# Create job with finalise_only=True
job_create = JobCreate(
artist=body.artist,
title=body.title,
filename="finalise_only", # No single audio file - using prep outputs
enable_cdg=body.enable_cdg,
enable_txt=body.enable_txt,
brand_prefix=body.brand_prefix,
enable_youtube_upload=body.enable_youtube_upload,
youtube_description=body.youtube_description,
discord_webhook_url=effective_discord_webhook_url,
dropbox_path=effective_dropbox_path,
gdrive_folder_id=effective_gdrive_folder_id,
finalise_only=True,
keep_brand_code=body.keep_brand_code,
request_metadata=request_metadata,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
logger.info(f"Created finalise-only job {job_id} for {body.artist} - {body.title}")
# Generate signed upload URLs for each file
upload_urls = []
for file_info in body.files:
gcs_path = _get_gcs_path_for_finalise_file(job_id, file_info.file_type, file_info.filename)
# Generate signed upload URL (valid for 60 minutes)
signed_url = storage_service.generate_signed_upload_url(
gcs_path,
content_type=file_info.content_type,
expiration_minutes=60
)
upload_urls.append(SignedUploadUrl(
file_type=file_info.file_type,
gcs_path=gcs_path,
upload_url=signed_url,
content_type=file_info.content_type
))
logger.info(f"Generated signed upload URL for {file_info.file_type}: {gcs_path}")
return CreateJobWithUploadUrlsResponse(
status="success",
job_id=job_id,
message="Finalise-only job created. Upload files to the provided URLs, then call /api/jobs/{job_id}/finalise-uploads-complete",
upload_urls=upload_urls,
server_version=VERSION
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Error creating finalise-only job: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e)) from e
@router.post("/jobs/{job_id}/finalise-uploads-complete")
async def mark_finalise_uploads_complete(
job_id: str,
background_tasks: BackgroundTasks,
body: UploadsCompleteRequest,
):
"""
Mark finalise-only file uploads as complete and start video generation.
This is called after uploading prep outputs for a finalise-only job.
The job will transition directly to AWAITING_INSTRUMENTAL_SELECTION.
"""
try:
# Get job and verify it exists
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Verify job is in pending state and is finalise-only
if job.status != JobStatus.PENDING:
raise HTTPException(
status_code=400,
detail=f"Job {job_id} is not in pending state (current: {job.status}). Cannot complete uploads."
)
if not job.finalise_only:
raise HTTPException(
status_code=400,
detail=f"Job {job_id} is not a finalise-only job. Use /api/jobs/{{job_id}}/uploads-complete instead."
)
# Validate required files were uploaded
required_files = {'with_vocals', 'title_screen', 'end_screen'}
uploaded_set = set(body.uploaded_files)
missing_files = required_files - uploaded_set
if missing_files:
raise HTTPException(
status_code=400,
detail=f"Missing required files: {missing_files}"
)
# Require at least one instrumental
if 'instrumental_clean' not in uploaded_set and 'instrumental_backing' not in uploaded_set:
raise HTTPException(
status_code=400,
detail="At least one instrumental (clean or backing) is required"
)
# Build file_urls structure from uploaded files
file_urls: Dict[str, Dict[str, str]] = {
'videos': {},
'screens': {},
'stems': {},
'lyrics': {},
}
for file_type in body.uploaded_files:
if file_type not in FINALISE_ONLY_FILE_TYPES:
logger.warning(f"Unknown file_type in uploaded_files: {file_type}")
continue
# Determine the GCS path - we need to find the actual file
if file_type == 'audio':
prefix = f"uploads/{job_id}/audio/"
elif file_type == 'with_vocals':
prefix = f"jobs/{job_id}/videos/with_vocals"
elif file_type == 'title_screen':
prefix = f"jobs/{job_id}/screens/title"
elif file_type == 'end_screen':
prefix = f"jobs/{job_id}/screens/end"
elif file_type == 'title_jpg':
prefix = f"jobs/{job_id}/screens/title.jpg"
elif file_type == 'title_png':
prefix = f"jobs/{job_id}/screens/title.png"
elif file_type == 'end_jpg':
prefix = f"jobs/{job_id}/screens/end.jpg"
elif file_type == 'end_png':
prefix = f"jobs/{job_id}/screens/end.png"
elif file_type == 'instrumental_clean':
prefix = f"jobs/{job_id}/stems/instrumental_clean"
elif file_type == 'instrumental_backing':
prefix = f"jobs/{job_id}/stems/instrumental_with_backing"
elif file_type == 'lrc':
prefix = f"jobs/{job_id}/lyrics/karaoke.lrc"
elif file_type == 'ass':
prefix = f"jobs/{job_id}/lyrics/karaoke.ass"
elif file_type == 'corrections':
prefix = f"jobs/{job_id}/lyrics/corrections"
elif file_type == 'style_params':
prefix = f"uploads/{job_id}/style/style_params"
else:
continue
# List files with this prefix to find the actual uploaded file
files = storage_service.list_files(prefix)
if not files:
raise HTTPException(
status_code=400,
detail=f"File for '{file_type}' was not uploaded to GCS. Expected prefix: {prefix}"
)
gcs_path = files[0]
# Map to file_urls structure
if file_type == 'audio':
pass # Handled separately
elif file_type == 'with_vocals':
file_urls['videos']['with_vocals'] = gcs_path
elif file_type == 'title_screen':
file_urls['screens']['title'] = gcs_path
elif file_type == 'end_screen':
file_urls['screens']['end'] = gcs_path
elif file_type == 'title_jpg':
file_urls['screens']['title_jpg'] = gcs_path
elif file_type == 'title_png':
file_urls['screens']['title_png'] = gcs_path
elif file_type == 'end_jpg':
file_urls['screens']['end_jpg'] = gcs_path
elif file_type == 'end_png':
file_urls['screens']['end_png'] = gcs_path
elif file_type == 'instrumental_clean':
file_urls['stems']['instrumental_clean'] = gcs_path
elif file_type == 'instrumental_backing':
file_urls['stems']['instrumental_with_backing'] = gcs_path
elif file_type == 'lrc':
file_urls['lyrics']['lrc'] = gcs_path
elif file_type == 'ass':
file_urls['lyrics']['ass'] = gcs_path
elif file_type == 'corrections':
file_urls['lyrics']['corrections'] = gcs_path
# Get audio path if uploaded
audio_gcs_path = None
if 'audio' in body.uploaded_files:
audio_files = storage_service.list_files(f"uploads/{job_id}/audio/")
if audio_files:
audio_gcs_path = audio_files[0]
# Update job with file paths
update_data = {
'file_urls': file_urls,
'finalise_only': True,
}
if audio_gcs_path:
update_data['input_media_gcs_path'] = audio_gcs_path
# Mark parallel processing as complete (we're skipping it)
update_data['state_data'] = {
'audio_complete': True,
'lyrics_complete': True,
'finalise_only': True,
}
job_manager.update_job(job_id, update_data)
logger.info(f"Validated finalise-only uploads for job {job_id}: {body.uploaded_files}")
# Transition directly to AWAITING_INSTRUMENTAL_SELECTION
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Prep files uploaded - select your instrumental"
)
# Get distribution services info for response
credential_manager = get_credential_manager()
distribution_services: Dict[str, Any] = {}
updated_job = job_manager.get_job(job_id)
if updated_job.dropbox_path:
dropbox_result = credential_manager.check_dropbox_credentials()
distribution_services["dropbox"] = {
"enabled": True,
"path": updated_job.dropbox_path,
"credentials_valid": dropbox_result.status == CredentialStatus.VALID,
}
if updated_job.gdrive_folder_id:
gdrive_result = credential_manager.check_gdrive_credentials()
distribution_services["gdrive"] = {
"enabled": True,
"folder_id": updated_job.gdrive_folder_id,
"credentials_valid": gdrive_result.status == CredentialStatus.VALID,
}
if updated_job.enable_youtube_upload:
youtube_result = credential_manager.check_youtube_credentials()
distribution_services["youtube"] = {
"enabled": True,
"credentials_valid": youtube_result.status == CredentialStatus.VALID,
}
if updated_job.discord_webhook_url:
distribution_services["discord"] = {
"enabled": True,
}
return {
"status": "success",
"job_id": job_id,
"message": "Finalise-only uploads validated. Select instrumental to continue.",
"files_validated": body.uploaded_files,
"server_version": VERSION,
"distribution_services": distribution_services,
}
except HTTPException:
raise
except Exception as e:
logger.error(f"Error completing finalise-only uploads for job {job_id}: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e)) from e
+18 -5
View File
@@ -73,6 +73,7 @@ class JobStatus(str, Enum):
# Terminal states
COMPLETE = "complete" # All processing finished successfully
PREP_COMPLETE = "prep_complete" # Prep-only job completed (stops after review)
FAILED = "failed" # Unrecoverable error occurred
CANCELLED = "cancelled" # User cancelled the job
@@ -86,8 +87,8 @@ class JobStatus(str, Enum):
# Valid state transitions
STATE_TRANSITIONS = {
# PENDING can go to DOWNLOADING (file upload) or SEARCHING_AUDIO (artist+title search)
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
# PENDING can go to DOWNLOADING (file upload), SEARCHING_AUDIO (artist+title search), or AWAITING_INSTRUMENTAL_SELECTION (finalise-only)
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED, JobStatus.CANCELLED],
# Audio search flow (for artist+title search mode)
JobStatus.SEARCHING_AUDIO: [JobStatus.AWAITING_AUDIO_SELECTION, JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED],
@@ -115,10 +116,10 @@ STATE_TRANSITIONS = {
# AWAITING_REVIEW can go directly to REVIEW_COMPLETE (quick review) or to IN_REVIEW (editing)
JobStatus.AWAITING_REVIEW: [JobStatus.IN_REVIEW, JobStatus.REVIEW_COMPLETE, JobStatus.FAILED, JobStatus.CANCELLED],
JobStatus.IN_REVIEW: [JobStatus.REVIEW_COMPLETE, JobStatus.AWAITING_REVIEW, JobStatus.FAILED],
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.FAILED],
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.PREP_COMPLETE, JobStatus.FAILED], # PREP_COMPLETE for prep-only jobs
# Video rendering (post-review)
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED],
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.PREP_COMPLETE, JobStatus.FAILED],
# Instrumental selection flow
JobStatus.AWAITING_INSTRUMENTAL_SELECTION: [JobStatus.INSTRUMENTAL_SELECTED, JobStatus.FAILED, JobStatus.CANCELLED],
@@ -133,9 +134,11 @@ STATE_TRANSITIONS = {
JobStatus.UPLOADING: [JobStatus.NOTIFYING, JobStatus.COMPLETE, JobStatus.FAILED],
JobStatus.NOTIFYING: [JobStatus.COMPLETE, JobStatus.FAILED],
# Terminal states - COMPLETE and CANCELLED have no transitions
# Terminal states - COMPLETE, PREP_COMPLETE, and CANCELLED have no transitions
# FAILED allows retry transitions to resume from checkpoints
# PREP_COMPLETE allows finalise-only continuation
JobStatus.COMPLETE: [],
JobStatus.PREP_COMPLETE: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED], # Finalise-only continues from here
JobStatus.FAILED: [
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
JobStatus.REVIEW_COMPLETE, # Retry from render stage
@@ -243,6 +246,11 @@ class Job(BaseModel):
audio_search_title: Optional[str] = None # Title used for audio search
auto_download: bool = False # Auto-select best audio source (skip selection)
# Two-phase workflow configuration (Batch 6)
prep_only: bool = False # Stop after review, don't run finalisation
finalise_only: bool = False # Skip prep, run only finalisation (requires uploaded prep outputs)
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Processing state
track_output_dir: Optional[str] = None # Local output directory (temp)
audio_hash: Optional[str] = None # Hash for deduplication
@@ -424,6 +432,11 @@ class JobCreate(BaseModel):
audio_search_title: Optional[str] = None # Title used for audio search
auto_download: bool = False # Auto-select best audio source (skip selection)
# Two-phase workflow configuration (Batch 6)
prep_only: bool = False # Stop after review, don't run finalisation
finalise_only: bool = False # Skip prep, run only finalisation
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Request metadata (set by API endpoint from request headers)
request_metadata: Dict[str, Any] = Field(default_factory=dict)
"""
+4
View File
@@ -55,6 +55,10 @@ class JobManager:
youtube_description=job_create.youtube_description,
webhook_url=job_create.webhook_url,
user_email=job_create.user_email,
# Two-phase workflow (Batch 6)
prep_only=job_create.prep_only,
finalise_only=job_create.finalise_only,
keep_brand_code=job_create.keep_brand_code,
# Request metadata (for tracking and filtering)
request_metadata=job_create.request_metadata,
)
+222
View File
@@ -963,6 +963,228 @@ class TestDurationValidation:
assert inspect.iscoroutinefunction(_validate_audio_durations)
class TestTwoPhaseWorkflowModels:
"""Test Pydantic models for two-phase workflow (Batch 6)."""
def test_create_job_with_upload_urls_request_has_prep_only(self):
"""Test that CreateJobWithUploadUrlsRequest has prep_only field."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
],
prep_only=True
)
assert request.prep_only is True
def test_create_job_with_upload_urls_request_prep_only_default_false(self):
"""Test that prep_only defaults to False."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
]
)
assert request.prep_only is False
def test_create_job_with_upload_urls_request_has_keep_brand_code(self):
"""Test that CreateJobWithUploadUrlsRequest has keep_brand_code field."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
],
keep_brand_code="NOMAD-1234"
)
assert request.keep_brand_code == "NOMAD-1234"
def test_create_job_with_upload_urls_request_keep_brand_code_default_none(self):
"""Test that keep_brand_code defaults to None."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
]
)
assert request.keep_brand_code is None
class TestFinaliseOnlyFileTypes:
"""Test finalise-only file types (Batch 6)."""
def test_finalise_only_file_types_exists(self):
"""Test that FINALISE_ONLY_FILE_TYPES is defined."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert FINALISE_ONLY_FILE_TYPES is not None
assert isinstance(FINALISE_ONLY_FILE_TYPES, dict)
def test_finalise_only_file_types_has_with_vocals(self):
"""Test that with_vocals is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'with_vocals' in FINALISE_ONLY_FILE_TYPES
assert '.mkv' in FINALISE_ONLY_FILE_TYPES['with_vocals']
assert '.mov' in FINALISE_ONLY_FILE_TYPES['with_vocals']
def test_finalise_only_file_types_has_title_screen(self):
"""Test that title_screen is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'title_screen' in FINALISE_ONLY_FILE_TYPES
def test_finalise_only_file_types_has_end_screen(self):
"""Test that end_screen is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'end_screen' in FINALISE_ONLY_FILE_TYPES
def test_finalise_only_file_types_has_instrumentals(self):
"""Test that instrumental types are valid finalise-only file types."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'instrumental_clean' in FINALISE_ONLY_FILE_TYPES
assert 'instrumental_backing' in FINALISE_ONLY_FILE_TYPES
def test_finalise_only_file_types_has_lrc(self):
"""Test that lrc is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'lrc' in FINALISE_ONLY_FILE_TYPES
assert '.lrc' in FINALISE_ONLY_FILE_TYPES['lrc']
class TestFinaliseOnlyModels:
"""Test Pydantic models for finalise-only flow (Batch 6)."""
def test_finalise_only_file_request_model(self):
"""Test FinaliseOnlyFileRequest model."""
from backend.api.routes.file_upload import FinaliseOnlyFileRequest
file_req = FinaliseOnlyFileRequest(
filename="with_vocals.mkv",
content_type="video/mkv",
file_type="with_vocals"
)
assert file_req.filename == "with_vocals.mkv"
assert file_req.content_type == "video/mkv"
assert file_req.file_type == "with_vocals"
def test_create_finalise_only_job_request_model(self):
"""Test CreateFinaliseOnlyJobRequest model."""
from backend.api.routes.file_upload import CreateFinaliseOnlyJobRequest, FinaliseOnlyFileRequest
request = CreateFinaliseOnlyJobRequest(
artist="Test Artist",
title="Test Song",
files=[
FinaliseOnlyFileRequest(filename="with_vocals.mkv", content_type="video/mkv", file_type="with_vocals"),
FinaliseOnlyFileRequest(filename="title.mov", content_type="video/quicktime", file_type="title_screen"),
FinaliseOnlyFileRequest(filename="end.mov", content_type="video/quicktime", file_type="end_screen"),
FinaliseOnlyFileRequest(filename="instrumental.flac", content_type="audio/flac", file_type="instrumental_clean"),
],
enable_cdg=True,
enable_txt=True,
brand_prefix="NOMAD",
keep_brand_code="NOMAD-1234"
)
assert request.artist == "Test Artist"
assert request.title == "Test Song"
assert len(request.files) == 4
assert request.keep_brand_code == "NOMAD-1234"
def test_create_finalise_only_job_request_optional_fields(self):
"""Test CreateFinaliseOnlyJobRequest optional fields default correctly."""
from backend.api.routes.file_upload import CreateFinaliseOnlyJobRequest, FinaliseOnlyFileRequest
request = CreateFinaliseOnlyJobRequest(
artist="Artist",
title="Title",
files=[
FinaliseOnlyFileRequest(filename="with_vocals.mkv", content_type="video/mkv", file_type="with_vocals"),
]
)
assert request.enable_cdg is True
assert request.enable_txt is True
assert request.brand_prefix is None
assert request.keep_brand_code is None
assert request.enable_youtube_upload is False
assert request.dropbox_path is None
assert request.gdrive_folder_id is None
class TestFinaliseOnlyGCSPaths:
"""Test GCS path generation for finalise-only files (Batch 6)."""
def test_with_vocals_gcs_path(self):
"""Test GCS path generation for with_vocals file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "with_vocals", "video.mkv")
assert path == "jobs/test123/videos/with_vocals.mkv"
def test_title_screen_gcs_path(self):
"""Test GCS path generation for title_screen file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "title_screen", "title.mov")
assert path == "jobs/test123/screens/title.mov"
def test_end_screen_gcs_path(self):
"""Test GCS path generation for end_screen file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "end_screen", "end.mov")
assert path == "jobs/test123/screens/end.mov"
def test_instrumental_clean_gcs_path(self):
"""Test GCS path generation for instrumental_clean file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "instrumental_clean", "clean.flac")
assert path == "jobs/test123/stems/instrumental_clean.flac"
def test_instrumental_backing_gcs_path(self):
"""Test GCS path generation for instrumental_backing file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "instrumental_backing", "backing.flac")
assert path == "jobs/test123/stems/instrumental_with_backing.flac"
def test_lrc_gcs_path(self):
"""Test GCS path generation for lrc file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "lrc", "karaoke.lrc")
assert path == "jobs/test123/lyrics/karaoke.lrc"
def test_title_jpg_gcs_path(self):
"""Test GCS path generation for title_jpg file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "title_jpg", "title.jpg")
assert path == "jobs/test123/screens/title.jpg"
# ============================================================================
# Batch 4: YouTube URL Input Tests
# ============================================================================
+243
View File
@@ -670,6 +670,249 @@ class TestExistingInstrumentalModelFields:
assert job.existing_instrumental_gcs_path == "uploads/test123/audio/my_instrumental.flac"
class TestTwoPhaseWorkflowFields:
"""Test two-phase workflow fields in Job and JobCreate models (Batch 6)."""
def test_job_has_prep_only_field(self):
"""Test that Job model has prep_only field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
prep_only=True
)
assert hasattr(job, 'prep_only')
assert job.prep_only is True
def test_job_prep_only_defaults_to_false(self):
"""Test that prep_only defaults to False."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.prep_only is False
def test_job_has_finalise_only_field(self):
"""Test that Job model has finalise_only field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
finalise_only=True
)
assert hasattr(job, 'finalise_only')
assert job.finalise_only is True
def test_job_finalise_only_defaults_to_false(self):
"""Test that finalise_only defaults to False."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.finalise_only is False
def test_job_has_keep_brand_code_field(self):
"""Test that Job model has keep_brand_code field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
keep_brand_code="NOMAD-1234"
)
assert hasattr(job, 'keep_brand_code')
assert job.keep_brand_code == "NOMAD-1234"
def test_job_keep_brand_code_defaults_to_none(self):
"""Test that keep_brand_code defaults to None."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.keep_brand_code is None
def test_job_create_has_two_phase_workflow_fields(self):
"""Test that JobCreate model has two-phase workflow fields."""
job_create = JobCreate(
artist="Artist",
title="Title",
prep_only=True,
finalise_only=False,
keep_brand_code="BRAND-0001"
)
assert job_create.prep_only is True
assert job_create.finalise_only is False
assert job_create.keep_brand_code == "BRAND-0001"
def test_two_phase_workflow_roundtrip_serialization(self):
"""Test that two-phase workflow fields survive serialization/deserialization."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
prep_only=True,
finalise_only=False,
keep_brand_code="NOMAD-5678"
)
# Serialize to dict (simulates writing to Firestore)
job_dict = job.model_dump()
# Convert to JSON and back (simulates Firestore storage)
json_str = json.dumps(job_dict, default=str)
restored_dict = json.loads(json_str)
# Deserialize back to Job (simulates reading from Firestore)
restored_job = Job(**restored_dict)
assert restored_job.prep_only is True
assert restored_job.finalise_only is False
assert restored_job.keep_brand_code == "NOMAD-5678"
class TestPrepCompleteStatus:
"""Test PREP_COMPLETE status and state transitions (Batch 6)."""
def test_prep_complete_status_exists(self):
"""Test that PREP_COMPLETE status exists in JobStatus enum."""
assert hasattr(JobStatus, 'PREP_COMPLETE')
assert JobStatus.PREP_COMPLETE.value == "prep_complete"
def test_job_can_have_prep_complete_status(self):
"""Test that Job can be created with PREP_COMPLETE status."""
job = Job(
job_id="test123",
status=JobStatus.PREP_COMPLETE,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.status == JobStatus.PREP_COMPLETE
def test_state_transitions_review_complete_to_prep_complete(self):
"""Test that REVIEW_COMPLETE can transition to PREP_COMPLETE."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.REVIEW_COMPLETE, [])
assert JobStatus.PREP_COMPLETE in valid_transitions
def test_state_transitions_prep_complete_to_awaiting_instrumental(self):
"""Test that PREP_COMPLETE can transition to AWAITING_INSTRUMENTAL_SELECTION."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PREP_COMPLETE, [])
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
def test_state_transitions_pending_to_awaiting_instrumental(self):
"""Test that PENDING can transition to AWAITING_INSTRUMENTAL_SELECTION (for finalise-only)."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PENDING, [])
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
def test_critical_statuses_include_prep_complete(self):
"""Test that PREP_COMPLETE is in the list of job statuses."""
all_statuses = [status.value for status in JobStatus]
assert "prep_complete" in all_statuses
class TestJobWithFullTwoPhaseConfig:
"""Integration tests for two-phase workflow job configuration."""
def test_prep_only_job_full_config(self):
"""Test creating a prep-only job with full configuration."""
job = Job(
job_id="prep123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
prep_only=True,
enable_cdg=True,
enable_txt=True,
brand_prefix="NOMAD",
)
assert job.prep_only is True
assert job.finalise_only is False
assert job.brand_prefix == "NOMAD"
def test_finalise_only_job_full_config(self):
"""Test creating a finalise-only job with full configuration."""
job = Job(
job_id="final123",
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
finalise_only=True,
keep_brand_code="NOMAD-1234",
enable_youtube_upload=True,
dropbox_path="/Karaoke/Tracks",
state_data={
"audio_complete": True,
"lyrics_complete": True,
"finalise_only": True,
}
)
assert job.finalise_only is True
assert job.keep_brand_code == "NOMAD-1234"
assert job.state_data["finalise_only"] is True
def test_prep_complete_job_with_file_urls(self):
"""Test a job that has reached PREP_COMPLETE status with all prep outputs."""
job = Job(
job_id="prepcomp123",
status=JobStatus.PREP_COMPLETE,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="ABBA",
title="Waterloo",
prep_only=True,
file_urls={
"stems": {
"instrumental_clean": "gs://bucket/jobs/prepcomp123/stems/instrumental_clean.flac",
"instrumental_with_backing": "gs://bucket/jobs/prepcomp123/stems/instrumental_with_backing.flac",
},
"videos": {
"with_vocals": "gs://bucket/jobs/prepcomp123/videos/with_vocals.mkv",
},
"screens": {
"title": "gs://bucket/jobs/prepcomp123/screens/title.mov",
"end": "gs://bucket/jobs/prepcomp123/screens/end.mov",
},
"lyrics": {
"lrc": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.lrc",
"ass": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.ass",
},
}
)
assert job.status == JobStatus.PREP_COMPLETE
assert job.prep_only is True
assert "with_vocals" in job.file_urls["videos"]
assert "title" in job.file_urls["screens"]
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+16 -3
View File
@@ -447,11 +447,24 @@ async def upload_lyrics_results(
corrections_data = json.load(f)
# Store metadata in state_data
job_manager.update_state_data(job_id, 'lyrics_metadata', {
# Include countdown padding info for instrumental synchronization
lyrics_metadata = {
'segment_count': len(corrections_data.get('corrected_segments', [])),
'has_corrections': True,
'ready_for_review': True
})
'ready_for_review': True,
}
# Add countdown padding info from transcription result
# This is needed by video_worker to pad instrumentals for sync
if transcription_result.get("countdown_padding_added"):
lyrics_metadata['has_countdown_padding'] = True
lyrics_metadata['countdown_padding_seconds'] = transcription_result.get("countdown_padding_seconds", 3.0)
logger.info(f"Job {job_id}: Countdown padding detected: {lyrics_metadata['countdown_padding_seconds']}s")
else:
lyrics_metadata['has_countdown_padding'] = False
lyrics_metadata['countdown_padding_seconds'] = 0.0
job_manager.update_state_data(job_id, 'lyrics_metadata', lyrics_metadata)
except Exception as e:
logger.warning(f"Job {job_id}: Could not parse corrections JSON: {e}")
else:
+21 -10
View File
@@ -270,16 +270,27 @@ async def process_render_video(job_id: str) -> bool:
job_log.info("Analyzing backing vocals for instrumental selection...")
await _analyze_backing_vocals(job_id, job_manager, storage, job_log)
# 12. Transition to awaiting instrumental selection
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Video rendered - select your instrumental"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
# 12. Transition based on prep_only flag
if getattr(job, 'prep_only', False):
# Prep-only mode: stop here and mark as prep complete
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.PREP_COMPLETE,
progress=100,
message="Prep phase complete - download outputs to continue locally"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE (PREP ONLY) ===")
logger.info(f"Job {job_id}: Prep-only video render complete")
else:
# Normal mode: proceed to instrumental selection
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Video rendered - select your instrumental"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
return True
except Exception as e:
+18
View File
@@ -162,6 +162,14 @@ async def generate_video(job_id: str) -> bool:
message="Encoding videos (15-20 min)"
)
# Get countdown padding info from lyrics metadata
# This ensures instrumental is padded to match vocals if countdown was added
lyrics_metadata = job.state_data.get('lyrics_metadata', {})
countdown_padding_seconds = None
if lyrics_metadata.get('has_countdown_padding'):
countdown_padding_seconds = lyrics_metadata.get('countdown_padding_seconds', 3.0)
job_log.info(f"Countdown padding detected: {countdown_padding_seconds}s - instrumental will be padded if needed")
# Log finalization parameters
job_log.info("Creating KaraokeFinalise with parameters:")
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
@@ -169,6 +177,7 @@ async def generate_video(job_id: str) -> bool:
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
job_log.info(f" instrumental source: {instrumental_source}")
job_log.info(f" countdown_padding_seconds: {countdown_padding_seconds}")
if existing_instrumental_path:
job_log.info(f" using user-provided instrumental (selection was: {instrumental_selection})")
@@ -209,6 +218,8 @@ async def generate_video(job_id: str) -> bool:
non_interactive=True,
server_side_mode=True,
selected_instrumental_file=instrumental_file,
# Audio synchronization - ensure instrumental matches vocal padding
countdown_padding_seconds=countdown_padding_seconds,
)
# Call process() - this does ALL the work:
@@ -324,6 +335,13 @@ async def _handle_native_distribution(
brand_code = result.get('brand_code')
base_name = f"{job.artist} - {job.title}"
# Check if we should preserve existing brand code (Batch 6: --keep-brand-code)
keep_brand_code = getattr(job, 'keep_brand_code', None)
if keep_brand_code:
brand_code = keep_brand_code
result['brand_code'] = brand_code
job_log.info(f"Using preserved brand code: {brand_code}")
# Upload to Dropbox using native SDK
dropbox_path = getattr(job, 'dropbox_path', None)
brand_prefix = getattr(job, 'brand_prefix', None)
@@ -421,19 +421,27 @@ Bump the patch version in `pyproject.toml` before committing.
---
### Batch 6: Two-Phase Workflow (HIGH)
### Batch 6: Two-Phase Workflow (HIGH) ✅ COMPLETE
**Parameters:** W1 `--prep-only`, W2 `--finalise-only`, F15 `--keep-brand-code`
**Scope:**
- **prep-only:** Remote CLI submits with `prep_only=true`. Backend runs through review, then stops. CLI downloads all outputs and exits.
- **finalise-only:** Remote CLI checks local folder for prep outputs, uploads ALL files, sends `finalise_only=true`. Backend runs finalisation only.
- **keep-brand-code:** Remote CLI extracts brand code from local folder name, sends to backend to preserve.
**Status:** ✅ Complete (2025-12-11)
**Files to modify:**
- `karaoke_gen/utils/remote_cli.py` - prep-only flow, finalise-only upload flow, brand code extraction
- `backend/api/routes/file_upload.py` - accept prep_only, finalise_only flags
- `backend/api/routes/` - new endpoint for finalise-only upload (multiple files)
- `backend/workers/orchestrator.py` or job state machine - conditional worker execution
**Implementation Summary:**
- **prep-only:** Added `prep_only` field to Job model. When set, backend transitions to new `PREP_COMPLETE` status after review and video rendering (with_vocals.mkv). CLI downloads all prep outputs.
- **finalise-only:** Added new endpoint `/api/jobs/create-finalise-only` for uploading prep outputs. Remote CLI detects files in prep folder, uploads them via signed URLs, and monitors finalisation.
- **keep-brand-code:** Remote CLI extracts brand code from folder name (e.g., "NOMAD-1234 - Artist - Title"), sends to backend. Video worker uses preserved brand code for distribution.
**Files modified:**
- `backend/models/job.py` - Added `PREP_COMPLETE` status, `prep_only`, `finalise_only`, `keep_brand_code` fields, updated state transitions
- `backend/api/routes/file_upload.py` - Added `prep_only`, `keep_brand_code` to job creation, new finalise-only endpoints
- `backend/workers/render_video_worker.py` - Conditional transition to PREP_COMPLETE for prep-only jobs
- `backend/workers/video_worker.py` - Use preserved brand code when set
- `backend/services/job_manager.py` - Pass new fields through job creation
- `karaoke_gen/utils/remote_cli.py` - Full implementation of prep-only, finalise-only, and keep-brand-code flows
**Tests added:**
- `backend/tests/test_models.py` - Tests for new fields and PREP_COMPLETE status
- `backend/tests/test_file_upload.py` - Tests for finalise-only models and endpoints
**Complexity:** High (significant workflow changes, new upload flow)
@@ -932,7 +940,7 @@ DEFAULT_DISCORD_WEBHOOK_URL=https://discord.com/...
### Medium-Term: Workflow Features
1. **Two-Phase Workflow** (Batch 6)
1. **Two-Phase Workflow** (Batch 6) ✅ COMPLETE
- `--prep-only` and `--finalise-only` modes
3. **Style Override** (Batch 7)
+105 -6
View File
@@ -7,11 +7,18 @@ using flacfetch, replacing the previous direct yt-dlp usage.
import logging
import os
import signal
import sys
import tempfile
import threading
from abc import ABC, abstractmethod
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
from dataclasses import dataclass
from typing import List, Optional
# Global flag to track if user requested cancellation via Ctrl+C
_interrupt_requested = False
@dataclass
class AudioSearchResult:
@@ -58,6 +65,19 @@ class DownloadError(AudioFetcherError):
pass
class UserCancelledError(AudioFetcherError):
"""Raised when user explicitly cancels the operation (e.g., enters 0 or Ctrl+C)."""
pass
def _check_interrupt():
"""Check if interrupt was requested and raise UserCancelledError if so."""
global _interrupt_requested
if _interrupt_requested:
raise UserCancelledError("Operation cancelled by user")
class AudioFetcher(ABC):
"""Abstract base class for audio fetching implementations."""
@@ -336,6 +356,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
Raises:
NoResultsError: If no results are found
DownloadError: If download fails
UserCancelledError: If user cancels (Ctrl+C or enters 0)
"""
from flacfetch.core.models import TrackQuery
@@ -343,7 +364,9 @@ class FlacFetchAudioFetcher(AudioFetcher):
query = TrackQuery(artist=artist, title=title)
self.logger.info(f"Searching for: {artist} - {title}")
results = manager.search(query)
# Run search in a thread so we can handle Ctrl+C
results = self._interruptible_search(manager, query)
if not results:
raise NoResultsError(f"No results found for: {artist} - {title}")
@@ -358,6 +381,8 @@ class FlacFetchAudioFetcher(AudioFetcher):
# Interactive mode: present options to user
selected = self._interactive_select(results, artist, title)
# Note: _interactive_select now raises UserCancelledError instead of returning None
# This check is kept as a safety net
if selected is None:
raise NoResultsError(f"No result selected for: {artist} - {title}")
@@ -395,9 +420,74 @@ class FlacFetchAudioFetcher(AudioFetcher):
quality=quality_str,
)
except (UserCancelledError, KeyboardInterrupt):
# Let cancellation exceptions propagate without wrapping
raise
except Exception as e:
raise DownloadError(f"Failed to download {artist} - {title}: {e}") from e
def _interruptible_search(self, manager, query) -> list:
"""
Run search in a way that can be interrupted by Ctrl+C.
The flacfetch search is a blocking network operation that doesn't
respond to SIGINT while running. This method runs it in a background
thread and periodically checks for interrupts.
Args:
manager: The FetchManager instance
query: The TrackQuery to search for
Returns:
List of search results
Raises:
UserCancelledError: If user presses Ctrl+C during search
"""
global _interrupt_requested
_interrupt_requested = False
result_container = {"results": None, "error": None}
def do_search():
try:
result_container["results"] = manager.search(query)
except Exception as e:
result_container["error"] = e
# Set up signal handler for immediate response to Ctrl+C
original_handler = signal.getsignal(signal.SIGINT)
def interrupt_handler(signum, frame):
global _interrupt_requested
_interrupt_requested = True
# Print immediately so user knows it was received
print("\nCancelling... please wait", file=sys.stderr)
signal.signal(signal.SIGINT, interrupt_handler)
try:
# Start search in background thread
search_thread = threading.Thread(target=do_search, daemon=True)
search_thread.start()
# Wait for completion with periodic interrupt checks
while search_thread.is_alive():
search_thread.join(timeout=0.1) # Check every 100ms
if _interrupt_requested:
# Don't wait for thread - it's a daemon and will be killed
raise UserCancelledError("Search cancelled by user (Ctrl+C)")
# Check for errors from the search
if result_container["error"] is not None:
raise result_container["error"]
return result_container["results"]
finally:
# Restore original signal handler
signal.signal(signal.SIGINT, original_handler)
_interrupt_requested = False
def _interactive_select(self, results: list, artist: str, title: str) -> object:
"""
Present search results to the user for interactive selection.
@@ -411,19 +501,25 @@ class FlacFetchAudioFetcher(AudioFetcher):
Returns:
The selected Release object
Raises:
UserCancelledError: If user cancels selection
"""
try:
# Use flacfetch's built-in CLIHandler for rich display
from flacfetch.interface.cli import CLIHandler
handler = CLIHandler(target_artist=artist)
return handler.select_release(results)
result = handler.select_release(results)
if result is None:
# User selected 0 to cancel
raise UserCancelledError("Selection cancelled by user")
return result
except ImportError:
# Fallback to basic display if CLIHandler not available
return self._basic_interactive_select(results, artist, title)
except (KeyboardInterrupt, EOFError):
print("\nCancelled")
return None
raise UserCancelledError("Selection cancelled by user (Ctrl+C)")
except (AttributeError, TypeError):
# Fallback if results aren't proper Release objects (e.g., in tests)
return self._basic_interactive_select(results, artist, title)
@@ -439,6 +535,9 @@ class FlacFetchAudioFetcher(AudioFetcher):
Returns:
The selected Release object
Raises:
UserCancelledError: If user cancels selection
"""
print(f"\n{'=' * 60}")
print(f"Search Results for: {artist} - {title}")
@@ -482,7 +581,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
if choice == "0":
self.logger.info("User cancelled selection")
return None
raise UserCancelledError("Selection cancelled by user")
choice_num = int(choice)
if 1 <= choice_num <= len(results):
@@ -496,7 +595,7 @@ class FlacFetchAudioFetcher(AudioFetcher):
print("Please enter a valid number")
except KeyboardInterrupt:
print("\nCancelled")
return None
raise UserCancelledError("Selection cancelled by user (Ctrl+C)")
def create_audio_fetcher(
+11 -3
View File
@@ -771,8 +771,16 @@ class AudioProcessor:
padded_result["other_stems"] = separation_result.get("other_stems", {})
padded_result["backing_vocals"] = separation_result.get("backing_vocals", {})
self.logger.info(
f"✓ Countdown padding applied to {len(padded_result['combined_instrumentals']) + 1} instrumental file(s)"
)
# Count actual padded files (don't assume clean instrumental was padded)
padded_count = (1 if padded_result["clean_instrumental"].get("instrumental") else 0) + len(padded_result["combined_instrumentals"])
if padded_count > 0:
self.logger.info(
f"✓ Countdown padding applied to {padded_count} instrumental file(s)"
)
else:
self.logger.warning(
"No instrumental files found to pad. Check that audio separation completed successfully."
)
return padded_result
@@ -47,6 +47,7 @@ class KaraokeFinalise:
user_youtube_credentials=None, # Add support for pre-stored credentials
server_side_mode=False, # New parameter for server-side deployment
selected_instrumental_file=None, # Add support for pre-selected instrumental file
countdown_padding_seconds=None, # Padding applied to vocals; instrumental must match
):
self.log_level = log_level
self.log_formatter = log_formatter
@@ -108,6 +109,7 @@ class KaraokeFinalise:
self.user_youtube_credentials = user_youtube_credentials
self.server_side_mode = server_side_mode
self.selected_instrumental_file = selected_instrumental_file
self.countdown_padding_seconds = countdown_padding_seconds
self.suffixes = {
"title_mov": " (Title).mov",
@@ -424,6 +426,15 @@ class KaraokeFinalise:
# Check if any videos were found
if "items" in response and len(response["items"]) > 0:
for item in response["items"]:
# YouTube search API sometimes returns results from other channels even with channelId filter
# Verify the video actually belongs to our channel
result_channel_id = item["snippet"]["channelId"]
if result_channel_id != channel_id:
self.logger.debug(
f"Skipping video from different channel: {item['snippet']['title']} (channel: {result_channel_id})"
)
continue
found_title = item["snippet"]["title"]
# In server-side mode, require an exact match to avoid false positives.
@@ -723,6 +734,32 @@ class KaraokeFinalise:
artist, title = base_name.split(" - ", 1)
return base_name, artist, title
def _pad_audio_file(self, input_audio, output_audio, padding_seconds):
"""
Pad an audio file by prepending silence at the beginning.
Uses the same ffmpeg approach as LyricsTranscriber's CountdownProcessor
to ensure consistent padding behavior.
Args:
input_audio: Path to input audio file
output_audio: Path for the padded output file
padding_seconds: Amount of silence to prepend (in seconds)
"""
self.logger.info(f"Padding audio file with {padding_seconds}s of silence")
# Use ffmpeg to prepend silence - this matches the approach in audio_processor.py
# adelay filter adds delay in milliseconds
delay_ms = int(padding_seconds * 1000)
ffmpeg_command = (
f'{self.ffmpeg_base_command} -i "{input_audio}" '
f'-af "adelay={delay_ms}|{delay_ms}" '
f'"{output_audio}"'
)
self.execute_command(ffmpeg_command, f"Padding audio with {padding_seconds}s silence")
def execute_command(self, command, description):
"""Execute a shell command and log the output. For general commands (rclone, etc.)"""
self.logger.info(f"{description}")
@@ -767,11 +804,32 @@ class KaraokeFinalise:
def remux_with_instrumental(self, with_vocals_file, instrumental_audio, output_file):
"""Remux the video with instrumental audio to create karaoke version"""
# Safety net: If countdown padding was applied to vocals, ensure instrumental is padded too
actual_instrumental = instrumental_audio
if self.countdown_padding_seconds and self.countdown_padding_seconds > 0:
# Check if the instrumental file is already padded (has "(Padded)" in name)
if "(Padded)" not in instrumental_audio:
self.logger.warning(
f"Countdown padding ({self.countdown_padding_seconds}s) was applied to vocals, "
f"but instrumental doesn't appear to be padded. Creating padded version..."
)
# Create a padded version of the instrumental
base, ext = os.path.splitext(instrumental_audio)
padded_instrumental = f"{base} (Padded){ext}"
if not os.path.exists(padded_instrumental):
self._pad_audio_file(instrumental_audio, padded_instrumental, self.countdown_padding_seconds)
self.logger.info(f"Created padded instrumental: {padded_instrumental}")
actual_instrumental = padded_instrumental
else:
self.logger.info(f"Using already-padded instrumental: {instrumental_audio}")
# This operation is primarily I/O bound (remuxing), so hardware acceleration doesn't provide significant benefit
# Keep the existing approach but use the new execute method
ffmpeg_command = (
f'{self.ffmpeg_base_command} -an -i "{with_vocals_file}" '
f'-vn -i "{instrumental_audio}" -c:v copy -c:a pcm_s16le "{output_file}"'
f'-vn -i "{actual_instrumental}" -c:v copy -c:a pcm_s16le "{output_file}"'
)
self.execute_command(ffmpeg_command, "Remuxing video with instrumental audio")
+111 -1
View File
@@ -29,7 +29,7 @@ from .audio_processor import AudioProcessor
from .lyrics_processor import LyricsProcessor
from .video_generator import VideoGenerator
from .video_background_processor import VideoBackgroundProcessor
from .audio_fetcher import create_audio_fetcher, AudioFetcherError, NoResultsError
from .audio_fetcher import create_audio_fetcher, AudioFetcherError, NoResultsError, UserCancelledError
class KaraokePrep:
@@ -259,6 +259,101 @@ class KaraokePrep:
self.artist = metadata_result["artist"]
self.title = metadata_result["title"]
def _scan_directory_for_instrumentals(self, track_output_dir, artist_title):
"""
Scan the directory for existing instrumental files and build a separated_audio structure.
This is used when transcription was skipped (existing files found) but we need to
pad instrumentals due to countdown padding.
Args:
track_output_dir: The track output directory to scan
artist_title: The "{artist} - {title}" string for matching files
Returns:
Dictionary with separated_audio structure containing found instrumental paths
"""
self.logger.info(f"Scanning directory for existing instrumentals: {track_output_dir}")
separated_audio = {
"clean_instrumental": {},
"backing_vocals": {},
"other_stems": {},
"combined_instrumentals": {},
}
# Search patterns for instrumental files
# Files are named like: "{artist} - {title} (Instrumental {model}).flac"
# Or with backing vocals: "{artist} - {title} (Instrumental +BV {model}).flac"
# Look for files in the track output directory
search_dir = track_output_dir
# Find all instrumental files (not padded ones - we want the originals)
instrumental_pattern = os.path.join(search_dir, f"{artist_title} (Instrumental*.flac")
instrumental_files = glob.glob(instrumental_pattern)
# Also check for wav files
instrumental_pattern_wav = os.path.join(search_dir, f"{artist_title} (Instrumental*.wav")
instrumental_files.extend(glob.glob(instrumental_pattern_wav))
self.logger.debug(f"Found {len(instrumental_files)} instrumental files")
for filepath in instrumental_files:
filename = os.path.basename(filepath)
# Skip already padded files
if "(Padded)" in filename:
self.logger.debug(f"Skipping already padded file: {filename}")
continue
# Determine if it's a combined instrumental (+BV) or clean instrumental
if "+BV" in filename or "+bv" in filename.lower():
# Combined instrumental with backing vocals
# Extract model name from filename
# Pattern: "(Instrumental +BV {model}).flac"
model_match = re.search(r'\(Instrumental \+BV ([^)]+)\)', filename)
if model_match:
model_name = model_match.group(1).strip()
separated_audio["combined_instrumentals"][model_name] = filepath
self.logger.info(f"Found combined instrumental: {filename}")
else:
# Clean instrumental (no backing vocals)
# Pattern: "(Instrumental {model}).flac"
model_match = re.search(r'\(Instrumental ([^)]+)\)', filename)
if model_match:
# Use as clean instrumental if we don't have one yet
if not separated_audio["clean_instrumental"].get("instrumental"):
separated_audio["clean_instrumental"]["instrumental"] = filepath
self.logger.info(f"Found clean instrumental: {filename}")
else:
# Additional clean instrumentals go to combined_instrumentals for padding
model_name = model_match.group(1).strip()
separated_audio["combined_instrumentals"][model_name] = filepath
self.logger.info(f"Found additional instrumental: {filename}")
# Also look for backing vocals files
backing_vocals_pattern = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.flac")
backing_vocals_files = glob.glob(backing_vocals_pattern)
backing_vocals_pattern_wav = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.wav")
backing_vocals_files.extend(glob.glob(backing_vocals_pattern_wav))
for filepath in backing_vocals_files:
filename = os.path.basename(filepath)
model_match = re.search(r'\(Backing Vocals ([^)]+)\)', filename)
if model_match:
model_name = model_match.group(1).strip()
if model_name not in separated_audio["backing_vocals"]:
separated_audio["backing_vocals"][model_name] = {"backing_vocals": filepath}
self.logger.info(f"Found backing vocals: {filename}")
# Log summary
clean_count = 1 if separated_audio["clean_instrumental"].get("instrumental") else 0
combined_count = len(separated_audio["combined_instrumentals"])
self.logger.info(f"Directory scan complete: {clean_count} clean instrumental, {combined_count} combined instrumentals")
return separated_audio
async def prep_single_track(self):
# Add signal handler at the start
loop = asyncio.get_running_loop()
@@ -422,6 +517,9 @@ class KaraokePrep:
# No still image for audio-only downloads
processed_track["input_still_image"] = None
except UserCancelledError:
# User cancelled - propagate up to CLI for graceful exit
raise
except NoResultsError as e:
self.logger.error(f"No audio found: {e}")
return None
@@ -764,6 +862,18 @@ class KaraokePrep:
f"Applying {padding_seconds}s padding to all instrumental files to sync with vocal countdown"
)
# If separated_audio is empty (e.g., transcription was skipped but existing files have countdown),
# scan the directory for existing instrumental files
has_instrumentals = (
processed_track["separated_audio"].get("clean_instrumental", {}).get("instrumental") or
processed_track["separated_audio"].get("combined_instrumentals")
)
if not has_instrumentals:
self.logger.info("No instrumentals in separated_audio, scanning directory for existing files...")
processed_track["separated_audio"] = self._scan_directory_for_instrumentals(
track_output_dir, artist_title
)
# Apply padding using AudioProcessor
padded_separation_result = self.audio_processor.apply_countdown_padding_to_instrumentals(
separation_result=processed_track["separated_audio"],
+81 -4
View File
@@ -11,6 +11,9 @@ from .utils import sanitize_filename
# Placeholder class or functions for lyrics processing
class LyricsProcessor:
# Standard countdown padding duration used by LyricsTranscriber
COUNTDOWN_PADDING_SECONDS = 3.0
def __init__(
self, logger, style_params_json, lyrics_file, skip_transcription, skip_transcription_review, render_video, subtitle_offset_ms
):
@@ -22,6 +25,60 @@ class LyricsProcessor:
self.render_video = render_video
self.subtitle_offset_ms = subtitle_offset_ms
def _detect_countdown_padding_from_lrc(self, lrc_filepath):
"""
Detect if countdown padding was applied by checking the first lyric timestamp in the LRC file.
LRC format timestamps look like: [mm:ss.xx] or [mm:ss.xxx]
If the first lyric timestamp is >= 3.0 seconds, countdown padding was likely applied.
Args:
lrc_filepath: Path to the LRC file
Returns:
Tuple of (countdown_padding_added: bool, countdown_padding_seconds: float)
"""
try:
with open(lrc_filepath, 'r', encoding='utf-8') as f:
content = f.read()
# Find all timestamp patterns in the LRC file
# LRC timestamps: [mm:ss.xx] or [mm:ss.xxx]
timestamp_pattern = r'\[(\d{1,2}):(\d{2})\.(\d{2,3})\]'
matches = re.findall(timestamp_pattern, content)
if not matches:
self.logger.debug("No timestamps found in LRC file")
return (False, 0.0)
# Find the first non-metadata timestamp (metadata like [ar:Artist] doesn't have decimal)
# We already filtered for decimal timestamps in our pattern
first_timestamp = matches[0]
minutes = int(first_timestamp[0])
seconds = int(first_timestamp[1])
# Handle both .xx and .xxx formats
centiseconds = first_timestamp[2]
if len(centiseconds) == 2:
milliseconds = int(centiseconds) * 10
else:
milliseconds = int(centiseconds)
first_lyric_time = minutes * 60 + seconds + milliseconds / 1000.0
self.logger.debug(f"First lyric timestamp in LRC: {first_lyric_time:.3f}s")
# If first lyric is at or after 3 seconds, countdown padding was applied
# Use a small buffer (2.5s) to account for songs that naturally start a bit late
if first_lyric_time >= 2.5:
self.logger.info(f"Detected countdown padding from LRC: first lyric at {first_lyric_time:.2f}s")
return (True, self.COUNTDOWN_PADDING_SECONDS)
return (False, 0.0)
except Exception as e:
self.logger.warning(f"Failed to detect countdown padding from LRC file: {e}")
return (False, 0.0)
def find_best_split_point(self, line):
"""
Find the best split point in a line based on the specified criteria.
@@ -138,23 +195,43 @@ class LyricsProcessor:
lyrics_video_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (With Vocals).mkv")
lyrics_lrc_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (Karaoke).lrc")
# If files exist in parent directory, return early
# If files exist in parent directory, return early (but detect countdown padding first)
if os.path.exists(parent_video_path) and os.path.exists(parent_lrc_path):
self.logger.info(f"Found existing video and LRC files in parent directory, skipping transcription")
self.logger.info("Found existing video and LRC files in parent directory, skipping transcription")
# Detect countdown padding from existing LRC file
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
if countdown_padding_added:
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
return {
"lrc_filepath": parent_lrc_path,
"ass_filepath": parent_video_path,
"countdown_padding_added": countdown_padding_added,
"countdown_padding_seconds": countdown_padding_seconds,
"padded_audio_filepath": None, # Original padded audio may not exist
}
# If files exist in lyrics directory, copy to parent and return
# If files exist in lyrics directory, copy to parent and return (but detect countdown padding first)
if os.path.exists(lyrics_video_path) and os.path.exists(lyrics_lrc_path):
self.logger.info(f"Found existing video and LRC files in lyrics directory, copying to parent")
self.logger.info("Found existing video and LRC files in lyrics directory, copying to parent")
os.makedirs(track_output_dir, exist_ok=True)
shutil.copy2(lyrics_video_path, parent_video_path)
shutil.copy2(lyrics_lrc_path, parent_lrc_path)
# Detect countdown padding from existing LRC file
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
if countdown_padding_added:
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
return {
"lrc_filepath": parent_lrc_path,
"ass_filepath": parent_video_path,
"countdown_padding_added": countdown_padding_added,
"countdown_padding_seconds": countdown_padding_seconds,
"padded_audio_filepath": None, # Original padded audio may not exist
}
# Create lyrics directory if it doesn't exist
+4 -2
View File
@@ -352,8 +352,10 @@ def create_parser(prog: str = "karaoke-gen") -> argparse.ArgumentParser:
)
remote_group.add_argument(
"--review-ui-url",
default=os.environ.get('REVIEW_UI_URL', 'https://lyrics.nomadkaraoke.com'),
help="Lyrics review UI URL (default: https://lyrics.nomadkaraoke.com)",
default=os.environ.get('REVIEW_UI_URL', os.environ.get('LYRICS_REVIEW_UI_URL', 'local')),
help="Lyrics review UI URL. Use 'local' for bundled frontend (default), "
"'http://localhost:5173' for Vite dev server, or 'https://lyrics.nomadkaraoke.com' for hosted. "
"(env: REVIEW_UI_URL or LYRICS_REVIEW_UI_URL)",
)
remote_group.add_argument(
"--poll-interval",
+50 -2
View File
@@ -18,6 +18,7 @@ import glob
import pyperclip
from karaoke_gen import KaraokePrep
from karaoke_gen.karaoke_finalise import KaraokeFinalise
from karaoke_gen.audio_fetcher import UserCancelledError
from karaoke_gen.instrumental_review import (
AudioAnalyzer,
WaveformGenerator,
@@ -265,6 +266,13 @@ def run_instrumental_review(track: dict, logger: logging.Logger) -> str | None:
else:
logger.warning("Custom instrumental not found, falling back to clean")
return clean_instrumental_path
elif selection == "uploaded":
uploaded_path = server.get_uploaded_instrumental_path()
if uploaded_path and os.path.exists(uploaded_path):
return uploaded_path
else:
logger.warning("Uploaded instrumental not found, falling back to clean")
return clean_instrumental_path
else:
logger.warning(f"Unknown selection: {selection}, falling back to numeric selection")
return None
@@ -294,6 +302,11 @@ async def async_main():
parser = create_parser(prog="karaoke-gen")
args = parser.parse_args()
# Set review UI URL environment variable for the lyrics transcriber review server
# This allows development against a local frontend dev server (e.g., http://localhost:5173)
if hasattr(args, 'review_ui_url') and args.review_ui_url:
os.environ['LYRICS_REVIEW_UI_URL'] = args.review_ui_url
# Process style overrides
try:
style_overrides = process_style_overrides(args.style_override, logger)
@@ -385,7 +398,21 @@ async def async_main():
kprep.input_media = input_audio_wav
# Run KaraokePrep
tracks = await kprep.process()
try:
tracks = await kprep.process()
except UserCancelledError:
logger.info("Operation cancelled by user")
return
except KeyboardInterrupt:
logger.info("Operation cancelled by user (Ctrl+C)")
return
# Filter out None tracks (can happen if prep failed for some tracks)
tracks = [t for t in tracks if t is not None] if tracks else []
if not tracks:
logger.warning("No tracks to process")
return
# Load CDG styles if CDG generation is enabled
cdg_styles = None
@@ -704,7 +731,21 @@ async def async_main():
kprep = kprep_coroutine
# Create final tracks data structure
tracks = await kprep.process()
try:
tracks = await kprep.process()
except UserCancelledError:
logger.info("Operation cancelled by user")
return
except KeyboardInterrupt:
logger.info("Operation cancelled by user (Ctrl+C)")
return
# Filter out None tracks (can happen if prep failed for some tracks)
tracks = [t for t in tracks if t is not None] if tracks else []
if not tracks:
logger.warning("No tracks to process")
return
# If prep-only mode, we're done
if args.prep_only:
@@ -779,6 +820,12 @@ async def async_main():
logger.info(f"Selected instrumental file: {selected_instrumental_file}")
# Get countdown padding info from track (if vocals were padded, instrumental must match)
countdown_padding_seconds = None
if track.get("countdown_padding_added", False):
countdown_padding_seconds = track.get("countdown_padding_seconds", 3.0)
logger.info(f"Countdown padding detected: {countdown_padding_seconds}s (will be applied to instrumental if needed)")
# Load CDG styles if CDG generation is enabled
cdg_styles = None
if args.enable_cdg:
@@ -823,6 +870,7 @@ async def async_main():
keep_brand_code=getattr(args, 'keep_brand_code', False),
non_interactive=args.yes,
selected_instrumental_file=selected_instrumental_file,
countdown_padding_seconds=countdown_padding_seconds,
)
try:
+411 -8
View File
@@ -67,6 +67,7 @@ class JobStatus(str, Enum):
UPLOADING = "uploading"
NOTIFYING = "notifying"
COMPLETE = "complete"
PREP_COMPLETE = "prep_complete" # Batch 6: Prep-only jobs stop here
FAILED = "failed"
CANCELLED = "cancelled"
ERROR = "error"
@@ -272,6 +273,9 @@ class RemoteKaraokeClient:
clean_instrumental_model: Optional[str] = None,
backing_vocals_models: Optional[list] = None,
other_stems_models: Optional[list] = None,
# Two-phase workflow (Batch 6)
prep_only: bool = False,
keep_brand_code: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a new karaoke generation job from a YouTube/online URL.
@@ -342,6 +346,11 @@ class RemoteKaraokeClient:
create_request['backing_vocals_models'] = backing_vocals_models
if other_stems_models:
create_request['other_stems_models'] = other_stems_models
# Two-phase workflow (Batch 6)
if prep_only:
create_request['prep_only'] = prep_only
if keep_brand_code:
create_request['keep_brand_code'] = keep_brand_code
self.logger.info(f"Creating URL-based job at {self.config.service_url}/api/jobs/create-from-url")
@@ -397,6 +406,9 @@ class RemoteKaraokeClient:
other_stems_models: Optional[list] = None,
# Existing instrumental (Batch 3)
existing_instrumental: Optional[str] = None,
# Two-phase workflow (Batch 6)
prep_only: bool = False,
keep_brand_code: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a new karaoke generation job with optional style configuration.
@@ -541,6 +553,11 @@ class RemoteKaraokeClient:
create_request['backing_vocals_models'] = backing_vocals_models
if other_stems_models:
create_request['other_stems_models'] = other_stems_models
# Two-phase workflow (Batch 6)
if prep_only:
create_request['prep_only'] = prep_only
if keep_brand_code:
create_request['keep_brand_code'] = keep_brand_code
response = self._request('POST', '/api/jobs/create-with-upload-urls', json=create_request)
@@ -628,6 +645,255 @@ class RemoteKaraokeClient:
return result
def submit_finalise_only_job(
self,
prep_folder: str,
artist: str,
title: str,
enable_cdg: bool = True,
enable_txt: bool = True,
brand_prefix: Optional[str] = None,
keep_brand_code: Optional[str] = None,
discord_webhook_url: Optional[str] = None,
youtube_description: Optional[str] = None,
enable_youtube_upload: bool = False,
dropbox_path: Optional[str] = None,
gdrive_folder_id: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a finalise-only job with prep output files.
This is used when the user previously ran --prep-only and now wants
to continue with the finalisation phase using cloud resources.
Args:
prep_folder: Path to the prep output folder containing stems, screens, etc.
artist: Artist name
title: Song title
enable_cdg: Generate CDG+MP3 package
enable_txt: Generate TXT+MP3 package
brand_prefix: Brand code prefix (e.g., "NOMAD")
keep_brand_code: Preserve existing brand code from folder name
discord_webhook_url: Discord webhook for notifications
youtube_description: YouTube video description
enable_youtube_upload: Enable YouTube upload
dropbox_path: Dropbox folder path for organized output
gdrive_folder_id: Google Drive folder ID for public share
"""
prep_path = Path(prep_folder)
if not prep_path.exists() or not prep_path.is_dir():
raise FileNotFoundError(f"Prep folder not found: {prep_folder}")
# Detect files in prep folder
files_info = []
local_files = {} # file_type -> local_path
base_name = f"{artist} - {title}"
# Required files - with_vocals video
for ext in ['.mkv', '.mov', '.mp4']:
with_vocals_path = prep_path / f"{base_name} (With Vocals){ext}"
if with_vocals_path.exists():
files_info.append({
'filename': with_vocals_path.name,
'content_type': f'video/{ext[1:]}',
'file_type': 'with_vocals'
})
local_files['with_vocals'] = str(with_vocals_path)
break
if 'with_vocals' not in local_files:
raise FileNotFoundError(f"with_vocals video not found in {prep_folder}")
# Title screen
for ext in ['.mov', '.mkv', '.mp4']:
title_path = prep_path / f"{base_name} (Title){ext}"
if title_path.exists():
files_info.append({
'filename': title_path.name,
'content_type': f'video/{ext[1:]}',
'file_type': 'title_screen'
})
local_files['title_screen'] = str(title_path)
break
if 'title_screen' not in local_files:
raise FileNotFoundError(f"title_screen video not found in {prep_folder}")
# End screen
for ext in ['.mov', '.mkv', '.mp4']:
end_path = prep_path / f"{base_name} (End){ext}"
if end_path.exists():
files_info.append({
'filename': end_path.name,
'content_type': f'video/{ext[1:]}',
'file_type': 'end_screen'
})
local_files['end_screen'] = str(end_path)
break
if 'end_screen' not in local_files:
raise FileNotFoundError(f"end_screen video not found in {prep_folder}")
# Instrumentals (at least one required)
stems_dir = prep_path / 'stems'
if stems_dir.exists():
for stem_file in stems_dir.iterdir():
if 'Instrumental' in stem_file.name and stem_file.suffix.lower() == '.flac':
if '+BV' not in stem_file.name:
if 'instrumental_clean' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_clean'
})
local_files['instrumental_clean'] = str(stem_file)
elif '+BV' in stem_file.name:
if 'instrumental_backing' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_backing'
})
local_files['instrumental_backing'] = str(stem_file)
# Also check root for instrumental files
for stem_file in prep_path.iterdir():
if 'Instrumental' in stem_file.name and stem_file.suffix.lower() == '.flac':
if '+BV' not in stem_file.name and 'instrumental_clean' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_clean'
})
local_files['instrumental_clean'] = str(stem_file)
elif '+BV' in stem_file.name and 'instrumental_backing' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_backing'
})
local_files['instrumental_backing'] = str(stem_file)
if 'instrumental_clean' not in local_files and 'instrumental_backing' not in local_files:
raise FileNotFoundError(f"No instrumental file found in {prep_folder}")
# Optional files - LRC
lrc_path = prep_path / f"{base_name} (Karaoke).lrc"
if lrc_path.exists():
files_info.append({
'filename': lrc_path.name,
'content_type': 'text/plain',
'file_type': 'lrc'
})
local_files['lrc'] = str(lrc_path)
# Optional - Title/End JPG/PNG
for img_type, file_type in [('Title', 'title'), ('End', 'end')]:
for ext in ['.jpg', '.png']:
img_path = prep_path / f"{base_name} ({img_type}){ext}"
if img_path.exists():
files_info.append({
'filename': img_path.name,
'content_type': f'image/{ext[1:]}',
'file_type': f'{file_type}_{ext[1:]}'
})
local_files[f'{file_type}_{ext[1:]}'] = str(img_path)
self.logger.info(f"Found {len(files_info)} files in prep folder")
for file_type in local_files:
self.logger.info(f" {file_type}: {Path(local_files[file_type]).name}")
# Create finalise-only job
create_request = {
'artist': artist,
'title': title,
'files': files_info,
'enable_cdg': enable_cdg,
'enable_txt': enable_txt,
}
if brand_prefix:
create_request['brand_prefix'] = brand_prefix
if keep_brand_code:
create_request['keep_brand_code'] = keep_brand_code
if discord_webhook_url:
create_request['discord_webhook_url'] = discord_webhook_url
if youtube_description:
create_request['youtube_description'] = youtube_description
if enable_youtube_upload:
create_request['enable_youtube_upload'] = enable_youtube_upload
if dropbox_path:
create_request['dropbox_path'] = dropbox_path
if gdrive_folder_id:
create_request['gdrive_folder_id'] = gdrive_folder_id
self.logger.info(f"Creating finalise-only job at {self.config.service_url}/api/jobs/create-finalise-only")
response = self._request('POST', '/api/jobs/create-finalise-only', json=create_request)
if response.status_code != 200:
try:
error_detail = response.json()
except Exception:
error_detail = response.text
raise RuntimeError(f"Error creating finalise-only job: {error_detail}")
create_result = response.json()
if create_result.get('status') != 'success':
raise RuntimeError(f"Error creating finalise-only job: {create_result}")
job_id = create_result['job_id']
upload_urls = create_result['upload_urls']
self.logger.info(f"Job {job_id} created. Uploading {len(upload_urls)} files directly to storage...")
# Upload each file
uploaded_files = []
for url_info in upload_urls:
file_type = url_info['file_type']
signed_url = url_info['upload_url']
content_type = url_info['content_type']
local_path = local_files.get(file_type)
if not local_path:
self.logger.warning(f"No local file found for file_type: {file_type}")
continue
file_size = os.path.getsize(local_path)
file_size_mb = file_size / (1024 * 1024)
self.logger.info(f" Uploading {file_type} ({file_size_mb:.1f} MB)...")
success = self._upload_file_to_signed_url(signed_url, local_path, content_type)
if not success:
raise RuntimeError(f"Failed to upload {file_type} to storage")
uploaded_files.append(file_type)
self.logger.info(f" ✓ Uploaded {file_type}")
# Mark uploads complete
self.logger.info(f"Notifying backend that uploads are complete...")
complete_request = {
'uploaded_files': uploaded_files
}
response = self._request('POST', f'/api/jobs/{job_id}/finalise-uploads-complete', json=complete_request)
if response.status_code != 200:
try:
error_detail = response.json()
except Exception:
error_detail = response.text
raise RuntimeError(f"Error completing finalise-only uploads: {error_detail}")
result = response.json()
if result.get('status') != 'success':
raise RuntimeError(f"Error completing finalise-only uploads: {result}")
return result
def get_job(self, job_id: str) -> Dict[str, Any]:
"""Get job status and details."""
response = self._request('GET', f'/api/jobs/{job_id}')
@@ -1054,6 +1320,7 @@ class JobMonitor:
'uploading': 'Uploading to distribution services',
'notifying': 'Sending notifications',
'complete': 'All processing complete',
'prep_complete': 'Prep phase complete - ready for local finalisation',
'failed': 'Job failed',
'cancelled': 'Job cancelled',
}
@@ -1809,6 +2076,24 @@ class JobMonitor:
self.download_outputs(job_id, job_data)
return 0
elif status == 'prep_complete':
self.logger.info("")
self.logger.info("=" * 60)
self.logger.info("PREP PHASE COMPLETE!")
self.logger.info("=" * 60)
self.logger.info(f"Track: {artist} - {title}")
self.logger.info("")
self.logger.info("Downloading all prep outputs...")
self.download_outputs(job_id, job_data)
self.logger.info("")
self.logger.info("To continue with finalisation, run:")
# Use shlex.quote for proper shell escaping of artist/title
import shlex
escaped_artist = shlex.quote(artist)
escaped_title = shlex.quote(title)
self.logger.info(f" karaoke-gen-remote --finalise-only ./<output_folder> {escaped_artist} {escaped_title}")
return 0
elif status in ['failed', 'error']:
self.logger.info("")
self.logger.error("=" * 60)
@@ -1879,9 +2164,6 @@ def main():
"""Main entry point for the remote CLI."""
# Set up logging - same format as gen_cli.py
logger = logging.getLogger(__name__)
# Prevent log propagation to root logger to avoid duplicate logs
# when external packages (like lyrics_converter) configure root logger handlers
logger.propagate = False
log_handler = logging.StreamHandler()
log_formatter = logging.Formatter(
fmt="%(asctime)s.%(msecs)03d - %(levelname)s - %(module)s - %(message)s",
@@ -2200,10 +2482,112 @@ def main():
logger.error(f"Error deleting job: {e}")
return 1
# Warn about unsupported features
# Handle finalise-only mode (Batch 6)
if args.finalise_only:
logger.error("--finalise-only is not supported in remote mode")
return 1
logger.info("=" * 60)
logger.info("Karaoke Generator (Remote) - Finalise Only Mode")
logger.info("=" * 60)
# For finalise-only, we expect the current directory to be the prep output folder
# OR a folder path as the first argument
prep_folder = "."
artist_arg_idx = 0
if args.args:
# Check if first argument is a directory
if os.path.isdir(args.args[0]):
prep_folder = args.args[0]
artist_arg_idx = 1
# Get artist and title from arguments
if len(args.args) > artist_arg_idx + 1:
artist = args.args[artist_arg_idx]
title = args.args[artist_arg_idx + 1]
elif len(args.args) > artist_arg_idx:
logger.error("Finalise-only mode requires both Artist and Title")
return 1
else:
# Try to extract from folder name
folder_name = os.path.basename(os.path.abspath(prep_folder))
parts = folder_name.split(" - ", 2)
if len(parts) >= 2:
# Format: "BRAND-XXXX - Artist - Title" or "Artist - Title"
if "-" in parts[0] and parts[0].split("-")[1].isdigit():
# Has brand code
artist = parts[1] if len(parts) > 2 else "Unknown"
title = parts[2] if len(parts) > 2 else parts[1]
else:
artist = parts[0]
title = parts[1]
logger.info(f"Extracted from folder name: {artist} - {title}")
else:
logger.error("Could not extract Artist and Title from folder name")
logger.error("Please provide: karaoke-gen-remote --finalise-only <folder> \"Artist\" \"Title\"")
return 1
else:
logger.error("Finalise-only mode requires folder path and/or Artist and Title")
return 1
# Extract brand code from folder name if --keep-brand-code is set
keep_brand_code = None
if getattr(args, 'keep_brand_code', False):
folder_name = os.path.basename(os.path.abspath(prep_folder))
parts = folder_name.split(" - ", 1)
if parts and "-" in parts[0]:
# Check if it's a brand code format (e.g., "NOMAD-1234")
potential_brand = parts[0]
brand_parts = potential_brand.split("-")
if len(brand_parts) == 2 and brand_parts[1].isdigit():
keep_brand_code = potential_brand
logger.info(f"Preserving brand code: {keep_brand_code}")
logger.info(f"Prep folder: {os.path.abspath(prep_folder)}")
logger.info(f"Artist: {artist}")
logger.info(f"Title: {title}")
if keep_brand_code:
logger.info(f"Brand Code: {keep_brand_code} (preserved)")
logger.info("")
# Read youtube description from file if provided
youtube_description = None
if args.youtube_description_file and os.path.isfile(args.youtube_description_file):
try:
with open(args.youtube_description_file, 'r') as f:
youtube_description = f.read()
except Exception as e:
logger.warning(f"Failed to read YouTube description file: {e}")
try:
result = client.submit_finalise_only_job(
prep_folder=prep_folder,
artist=artist,
title=title,
enable_cdg=args.enable_cdg,
enable_txt=args.enable_txt,
brand_prefix=args.brand_prefix,
keep_brand_code=keep_brand_code,
discord_webhook_url=args.discord_webhook_url,
youtube_description=youtube_description,
enable_youtube_upload=getattr(args, 'enable_youtube_upload', False),
dropbox_path=getattr(args, 'dropbox_path', None),
gdrive_folder_id=getattr(args, 'gdrive_folder_id', None),
)
job_id = result.get('job_id')
logger.info(f"Finalise-only job submitted: {job_id}")
logger.info("")
# Monitor job
return monitor.monitor(job_id)
except FileNotFoundError as e:
logger.error(str(e))
return 1
except RuntimeError as e:
logger.error(str(e))
return 1
except Exception as e:
logger.error(f"Error: {e}")
return 1
if args.edit_lyrics:
logger.error("--edit-lyrics is not yet supported in remote mode")
@@ -2215,8 +2599,7 @@ def main():
# Warn about features that are not yet supported in remote mode
ignored_features = []
if args.prep_only:
ignored_features.append("--prep-only")
# Note: --prep-only is now supported in remote mode (Batch 6)
if args.skip_separation:
ignored_features.append("--skip-separation")
if args.skip_transcription:
@@ -2421,6 +2804,8 @@ def main():
logger.info(f"Other Stems Models: {args.other_stems_models}")
if getattr(args, 'existing_instrumental', None):
logger.info(f"Existing Instrumental: {args.existing_instrumental}")
if getattr(args, 'prep_only', False):
logger.info(f"Mode: prep-only (will stop after review)")
logger.info(f"Service URL: {config.service_url}")
logger.info(f"Review UI: {config.review_ui_url}")
if config.non_interactive:
@@ -2437,6 +2822,18 @@ def main():
except Exception as e:
logger.warning(f"Failed to read YouTube description file: {e}")
# Extract brand code from current directory if --keep-brand-code is set
keep_brand_code_value = None
if getattr(args, 'keep_brand_code', False):
cwd_name = os.path.basename(os.getcwd())
parts = cwd_name.split(" - ", 1)
if parts and "-" in parts[0]:
potential_brand = parts[0]
brand_parts = potential_brand.split("-")
if len(brand_parts) == 2 and brand_parts[1].isdigit():
keep_brand_code_value = potential_brand
logger.info(f"Preserving brand code: {keep_brand_code_value}")
try:
# Submit job - different endpoint for URL vs file
if is_url_input:
@@ -2469,6 +2866,9 @@ def main():
clean_instrumental_model=getattr(args, 'clean_instrumental_model', None),
backing_vocals_models=getattr(args, 'backing_vocals_models', None),
other_stems_models=getattr(args, 'other_stems_models', None),
# Two-phase workflow (Batch 6)
prep_only=getattr(args, 'prep_only', False),
keep_brand_code=keep_brand_code_value,
)
else:
# File-based job submission
@@ -2498,6 +2898,9 @@ def main():
other_stems_models=getattr(args, 'other_stems_models', None),
# Existing instrumental (Batch 3)
existing_instrumental=getattr(args, 'existing_instrumental', None),
# Two-phase workflow (Batch 6)
prep_only=getattr(args, 'prep_only', False),
keep_brand_code=keep_brand_code_value,
)
job_id = result.get('job_id')
style_assets = result.get('style_assets_uploaded', [])
@@ -0,0 +1,336 @@
# MidiCo Synchronizer Reference
This document provides a detailed reference for replicating the MidiCo Synchronizer user experience in our Nomad Karaoke lyrics review UI. The MidiCo Synchronizer is a native macOS application that provides a polished, intuitive interface for manually syncing lyrics to audio.
## Problem Statement
Sometimes transcribed lyrics are correct and start well-synced, but progressively drift out of sync towards the second half of the song. We need an efficient way to fix timing for multiple segments without having to re-sync everything from scratch.
## Overview of MidiCo Interface
MidiCo consists of two windows:
1. **Main Audio Player Window** - Controls audio playback
2. **Synchronizer Window** - Timeline-based lyrics sync interface
![Empty Timeline Before Sync](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Empty-Timeline-Before-Any-Words-Synced.png)
---
## 1. Audio Player Integration
The main MidiCo window provides an audio player that stays synchronized with the Synchronizer:
- **Play/Pause/Stop buttons** - Standard playback controls
- **Rewind/Fast-forward buttons** - Skip through the track
- **Progress slider** - Shows current position (00:00 format) and remaining time (-03:45 format)
- **Volume slider** - Control audio volume
- **Tempo/Pitch sliders** - Adjust playback speed and pitch (useful for difficult sections)
**Key behavior:** The playhead position is always synced between the audio player and the Synchronizer timeline.
---
## 2. Synchronizer Primary View
The Synchronizer window contains the main timeline interface for syncing lyrics.
### 2.1 Top Control Buttons
Four main control buttons at the top:
| Button | Function |
|--------|----------|
| **Voice no.1** | Select which vocal track to sync (for multi-voice karaoke) |
| **Start Sync** | Begin sync mode - starts audio, shows upcoming words, listens for spacebar |
| **Clear Sync** | Delete all sync data, reset to unsynced state |
| **Edit Lyrics** | Open modal to edit the lyrics text directly |
| **Import Lrc** | Import timing data from an LRC file |
### 2.2 Timeline View
The main rectangular timeline area is the core of the interface.
![During Sync Showing Synced And Upcoming Words](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Synced-And-Upcoming-Words.png)
#### Time Bar (Top Strip)
- Thin bar at the top showing time in `MM:SS` format
- Small tick marks for decisecond precision
- Clicking anywhere on the time bar sets the playhead position (without changing playback state)
#### Playhead Marker
- Small arrow/triangle indicator in the time bar
- White vertical line spanning the full height of the timeline
- Shows current playback position
- Synced with the audio player position
#### Timeline Background
- Light gray gradient background
- **Clicking on the background deselects any selected words**
- **Click-drag creates a selection rectangle** for selecting multiple word blocks
#### Zoom Control
- Horizontal slider below the timeline labeled "Zoom"
- Smooth continuous zoom (approximately 50 notches)
- **Most zoomed in:** ~4.5 seconds visible edge-to-edge
- **Most zoomed out:** ~24 seconds visible edge-to-edge
- Smooth, responsive zooming is critical for fast songs
![Max Zoom Level](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Max-Zoom-Level.png)
#### Horizontal Scrolling
- Timeline scrolls horizontally to show different parts of the song
- Small horizontal scrollbar at the bottom of the timeline area
- During sync mode, the timeline auto-scrolls to follow the playhead
### 2.3 Word Blocks
Synced words appear as **red rectangular blocks** on the timeline:
- Each block shows the word text above the red bar
- If word text is too long to fit the width of the red bar, the alignment with the red blocks becomes "best effort". Words don't overlap, they're all readable above the red bars for that segment
- Block width corresponds to the word's duration
- Block horizontal position corresponds to the word's start time
#### Two-Level Word Layout (Alternating Segments)
Segments alternate between two vertical positions on the timeline. Each consecutive segment (in time order) is placed on the opposite level from the previous one:
- **Level 0 (top)**: 1st, 3rd, 5th, 7th... segments
- **Level 1 (bottom)**: 2nd, 4th, 6th, 8th... segments
![Two Level Word Blocks](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Two-Levels-Word-Blocks.png)
This alternating layout is critical because:
1. **Text never overlaps between consecutive segments** - Since each segment is on a different level, the text labels above the red blocks from one segment won't collide with text from the next segment, even if the text extends past its block width.
2. **All words remain readable** - Text is allowed to extend horizontally beyond the red block boundaries. Because consecutive segments are always on different vertical levels, this extended text doesn't overlap with adjacent segments.
![Non-Overlapping Segments](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Timeline-With-Non-Overlapping-Segments.png)
In the example above, notice how:
- "Monica Mountains" (level 0) has text that extends beyond its block
- "Where she then proceeded to pump her vehicle in and out of turns" (level 1) is on a different vertical level
- "Sometimes dropping down to fifty miles per hour" (level 0) returns to the top level
- "Only to immediately gun it back up to 90 again" (level 1) is back on the bottom level
This simple alternating pattern ensures that even long segment text remains fully readable without truncation.
**Edge case:** Overlap can still occur in rare situations - if one segment's text is extremely long AND the segment two positions later (on the same level) is very short/early. But this is uncommon in practice.
#### Word Block Selection
- **Single click** on a word block: Selects that block (shown with white 1px border)
- **Click-drag on background**: Creates selection rectangle to select multiple blocks
![Click Drag Word Selection](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Click-Drag-Word-Selection.png)
#### Word Block Manipulation (Resize & Drag)
Selected word blocks can be manipulated directly on the timeline:
##### Resizing Word Blocks (Change Duration)
When a word block is selected and you hover over it, a **small white dot/handle** appears just inside the right edge of the block. This resize handle allows you to change the word's duration:
1. Select a word block (single click)
2. Hover over the selected block to reveal the resize handle (white dot on right edge)
3. Click and drag the handle left/right to shorten/lengthen the word's duration
4. The word's end time changes; start time remains fixed
This is essential for fine-tuning word durations after initial sync, especially for:
- Words that were synced too long or too short
- Adjusting timing to better match the audio
- Creating smoother transitions between words
##### Dragging Word Blocks (Change Position)
Selected word blocks can be dragged horizontally to change their position in time:
**Single word:**
1. Select a word block
2. Click and drag anywhere on the block (not on the resize handle)
3. The entire word moves earlier/later in time (both start and end times shift)
**Multiple words:**
1. Drag-select multiple word blocks (or Ctrl/Cmd+click to add to selection)
2. Click and drag any of the selected blocks
3. **All selected blocks move together**, maintaining their relative timing
This is particularly useful for:
- Shifting an entire segment earlier or later when it's consistently off
- Adjusting groups of words that were synced correctly relative to each other but offset from the audio
- Fine-tuning timing without having to re-sync
**Example workflow:** If a segment of 10 words all come in 0.2 seconds too late:
1. Drag-select all 10 word blocks
2. Drag them 0.2 seconds earlier
3. All words maintain their relative timing but are now properly aligned with the audio
### 2.4 Upcoming Words Display
During sync mode, unsynced words appear in **two locations**:
![Start Sync Showing Upcoming Words](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Start-Sync-Showing-Upcoming-Words.png)
1. **Fixed position above time bar** (left side): Shows upcoming words as red/white blocks in a horizontal row. The next word to sync is highlighted in red, subsequent words in white.
2. **On the timeline, right of playhead**: Same words appear starting from the playhead position and extending to the right, with a pale yellow background and black text, moving along with the playhead during playback.
This dual display makes it easy to:
- See what's coming next (fixed position - easy to read while focused on timing)
- See where words will land on the timeline relative to already-synced words
---
## 3. Action Buttons (Icon Toolbar)
A row of small icon-only buttons appears above the timeline. Key buttons we need:
### Essential Buttons
| Icon | Tooltip | Function |
|------|---------|----------|
| **Rectangle with Pencil** | Split, spell or edit selected word | Opens popup to edit a single word. Useful for typos. Entering multiple space-separated words splits the selected word while preserving timing. |
| **No entry symbol** | Unsynchronize from cursor position | All word blocks **after** the current playhead position are reset to unsynced state. Critical for fixing drift in later parts of a song. |
| **Trash** | Delete selected | Deletes selected word(s) entirely. Useful for removing backing vocals or nonsense words. |
### Typical Workflow for Deleting Unwanted Words
1. Tap spacebar to sync the unwanted words (so they appear on timeline)
2. Stop playback
3. Click-drag to select the unwanted word blocks
4. Click trash icon to delete
5. Click Start Sync to resume syncing from where you left off
---
## 4. Sync Mode Operation
### Starting Sync
1. Click **Start Sync** button
2. Audio begins playing from current playhead position
3. Upcoming unsynced words appear in both display locations
4. System begins listening for spacebar input
### Spacebar Timing
Two input modes:
**Tap (press and release quickly):**
- Sets the **start time** of the next unsynced word to the current playhead position
- The word's **end time** is set when the following word is synced
- If gap between words > 1 second, previous word gets default 1 second duration
**Press-hold:**
- Hold spacebar down when word starts
- Release when word ends
- Sets both start and end times explicitly
### Stopping Sync
- Click the **Stop** button (replaces Play when in sync mode)
- Or click **Start Sync** again to toggle off
- Sync progress is preserved
---
## 5. Edit Lyrics Modal
![Lyrics Editor Modal](midico-sync-screenshots/MidiCo-Synchronizer-2025-12-18-Lyrics-Editor-Modal.png)
Accessed via the **Edit Lyrics** button. Provides:
- **Left panel**: Plain text editor with all lyrics, newlines separate segments
- **Right panel**: Preview of how lyrics will appear (karaoke style rendering)
- **Language selector**: For hyphenation support
- **Hyphenate button**: Auto-hyphenate words for syllable-level sync
- **Reset button**: Revert to original lyrics
- **Cancel/OK buttons**: Discard or apply changes
### Important Behavior
Editing lyrics when there are already synced timestamps **usually messes up the syncs**. Users typically need to Clear Sync and start over after editing lyrics.
---
## 6. Apply Button
The **Apply** button in the bottom-right corner is critical:
- Changes made in the Synchronizer are **not saved** until Apply is clicked
- This allows experimenting without fear of losing work
- If sync goes badly wrong, user can close the Synchronizer without applying changes
- Provides a safe "sandbox" for sync work
---
## 7. Performance Requirements
The entire interface must be:
- **Smooth and responsive** - No lag or stutter during playback
- **Low latency** - Spacebar input must register with sub-100ms precision
- **Efficient** - No unnecessary computation during sync (user is focused on timing)
- **Native-app feel** - As expected from a macOS application
Since we're dealing with sub-second time precision for synced lyrics, performance is critical.
---
## 8. Implementation Notes for Our App
### Entry Point: "Edit All" Button
The current "Edit All" button should show a choice:
1. **Replace all lyrics** - Shows the existing paste box flow (for completely new lyrics)
2. **Re-sync existing lyrics** - Goes directly to the new Synchronizer view with all existing words/segments plotted on the timeline
This preserves existing sync data while allowing re-sync of later portions.
### Key Features to Implement
1. **Timeline with word blocks showing existing sync data**
2. **Playhead synced with audio player**
3. **Zoom slider (4.5s to 24s range)**
4. **Horizontal scrolling**
5. **Two-level word block layout**
6. **Start Sync mode with spacebar capture**
7. **Upcoming words display (fixed + timeline)**
8. **"Unsynchronize from cursor" - critical for fixing drift**
9. **Word selection (single click + drag-select)**
10. **Delete selected words**
11. **Edit single word popup**
12. **Apply/Cancel for safe experimentation**
### Fixing Timing Drift (Primary Use Case)
The main workflow for fixing drift:
1. Open Synchronizer (with existing sync data loaded)
2. Play audio to find where sync starts drifting
3. Position playhead at last correctly-synced word
4. Click "Unsynchronize from cursor position"
5. Click "Start Sync" to resume syncing from that point
6. Tap spacebar to re-sync the remaining words
7. Click "Apply" when done
This workflow allows keeping good sync data while fixing only the problematic portion.
---
## Screenshots Reference
All screenshots are located in: `docs/midico-sync-screenshots/`
| Filename | Description |
|----------|-------------|
| `MidiCo-Synchronizer-2025-12-18-Empty-Timeline-Before-Any-Words-Synced.png` | Initial state with empty timeline |
| `MidiCo-Synchronizer-2025-12-18-Start-Sync-Showing-Upcoming-Words.png` | Sync mode started, showing upcoming words in both locations |
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Synced-And-Upcoming-Words.png` | Mid-sync showing both synced blocks and upcoming words |
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Two-Levels-Word-Blocks.png` | Demonstration of two-level word block layout |
| `MidiCo-Synchronizer-2025-12-18-During-Sync-Showing-Max-Zoom-Level.png` | Maximum zoom level (~4.5 seconds visible) |
| `MidiCo-Synchronizer-2025-12-18-Click-Drag-Word-Selection.png` | Selection rectangle for multi-word selection |
| `MidiCo-Synchronizer-2025-12-18-Lyrics-Editor-Modal.png` | Edit Lyrics modal dialog |
| `MidiCo-Synchronizer-2025-12-18-Timeline-With-Non-Overlapping-Segments.png` | Shows alternating segment levels with long text that doesn't overlap |
Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

@@ -2,7 +2,7 @@
"name": "lyrics-transcriber-frontend",
"private": true,
"homepage": "https://nomadkaraoke.github.io/lyrics-transcriber-frontend",
"version": "0.80.0",
"version": "0.82.0",
"type": "module",
"scripts": {
"dev": "vite",
@@ -1326,6 +1326,7 @@ export default function LyricsAnalyzer({ data: initialData, onFileLoad, apiClien
onPlaySegment={handlePlaySegment}
currentTime={currentAudioTime}
setModalSpacebarHandler={handleSetModalSpacebarHandler}
existingSegments={data.corrected_segments}
/>
{pendingAnnotation && (
@@ -0,0 +1,185 @@
import { memo } from 'react'
import {
Box,
Button,
Divider,
Stack
} from '@mui/material'
import PlayCircleOutlineIcon from '@mui/icons-material/PlayCircleOutline'
import StopIcon from '@mui/icons-material/Stop'
import PauseIcon from '@mui/icons-material/Pause'
import PlayArrowIcon from '@mui/icons-material/PlayArrow'
import ClearAllIcon from '@mui/icons-material/ClearAll'
import EditNoteIcon from '@mui/icons-material/EditNote'
import BlockIcon from '@mui/icons-material/Block'
import EditIcon from '@mui/icons-material/Edit'
import DeleteIcon from '@mui/icons-material/Delete'
interface SyncControlsProps {
// Main buttons
isManualSyncing: boolean
isPaused: boolean
onStartSync: () => void
onPauseSync: () => void
onResumeSync: () => void
onClearSync: () => void
onEditLyrics: () => void
// Playback controls
onPlay: () => void
onStop: () => void
isPlaying: boolean
// Word action buttons
hasSelectedWords: boolean
selectedWordCount: number
onUnsyncFromCursor: () => void
onEditSelectedWord: () => void
onDeleteSelected: () => void
// Additional state
canUnsyncFromCursor: boolean
}
const SyncControls = memo(function SyncControls({
isManualSyncing,
isPaused,
onStartSync,
onPauseSync,
onResumeSync,
onClearSync,
onEditLyrics,
onPlay,
onStop,
isPlaying,
hasSelectedWords,
selectedWordCount,
onUnsyncFromCursor,
onEditSelectedWord,
onDeleteSelected,
canUnsyncFromCursor
}: SyncControlsProps) {
return (
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 1.5 }}>
{/* Row 1: Playback & Sync Mode Controls */}
<Stack direction="row" spacing={1} alignItems="center" flexWrap="wrap">
{/* Playback controls */}
<Button
variant="outlined"
color="primary"
onClick={onPlay}
startIcon={<PlayArrowIcon />}
size="small"
disabled={isPlaying}
>
Play
</Button>
<Button
variant="outlined"
color="error"
onClick={onStop}
startIcon={<StopIcon />}
size="small"
disabled={!isPlaying}
>
Stop
</Button>
<Divider orientation="vertical" flexItem sx={{ mx: 0.5 }} />
{/* Sync Mode controls */}
{isManualSyncing ? (
<>
<Button
variant="contained"
color="error"
onClick={onStartSync}
startIcon={<StopIcon />}
size="small"
>
Stop Sync
</Button>
<Button
variant="outlined"
color={isPaused ? 'success' : 'warning'}
onClick={isPaused ? onResumeSync : onPauseSync}
startIcon={isPaused ? <PlayArrowIcon /> : <PauseIcon />}
size="small"
>
{isPaused ? 'Resume' : 'Pause'}
</Button>
</>
) : (
<Button
variant="contained"
color="primary"
onClick={onStartSync}
startIcon={<PlayCircleOutlineIcon />}
size="small"
>
Start Sync
</Button>
)}
</Stack>
{/* Row 2: Editing & Word Actions */}
<Stack direction="row" spacing={1} alignItems="center" flexWrap="wrap">
<Button
variant="outlined"
color="warning"
onClick={onClearSync}
startIcon={<ClearAllIcon />}
size="small"
disabled={isManualSyncing && !isPaused}
>
Clear Sync
</Button>
<Button
variant="outlined"
onClick={onEditLyrics}
startIcon={<EditNoteIcon />}
size="small"
disabled={isManualSyncing && !isPaused}
>
Edit Lyrics
</Button>
<Divider orientation="vertical" flexItem sx={{ mx: 0.5 }} />
<Button
variant="outlined"
onClick={onUnsyncFromCursor}
startIcon={<BlockIcon />}
size="small"
disabled={!canUnsyncFromCursor || (isManualSyncing && !isPaused)}
>
Unsync from Cursor
</Button>
<Button
variant="outlined"
onClick={onEditSelectedWord}
startIcon={<EditIcon />}
size="small"
disabled={!hasSelectedWords || selectedWordCount !== 1}
>
Edit Word
</Button>
<Button
variant="outlined"
color="error"
onClick={onDeleteSelected}
startIcon={<DeleteIcon />}
size="small"
disabled={!hasSelectedWords || (isManualSyncing && !isPaused)}
>
Delete{hasSelectedWords && selectedWordCount > 0 ? ` (${selectedWordCount})` : ''}
</Button>
</Stack>
</Box>
)
})
export default SyncControls
@@ -0,0 +1,704 @@
import { useRef, useEffect, useCallback, useState, memo } from 'react'
import { Box, IconButton, Tooltip } from '@mui/material'
import ArrowBackIcon from '@mui/icons-material/ArrowBack'
import ArrowForwardIcon from '@mui/icons-material/ArrowForward'
import { Word, LyricsSegment } from '../../types'
interface TimelineCanvasProps {
words: Word[]
segments: LyricsSegment[]
visibleStartTime: number
visibleEndTime: number
currentTime: number
selectedWordIds: Set<string>
onWordClick: (wordId: string, event: React.MouseEvent) => void
onBackgroundClick: () => void
onTimeBarClick: (time: number) => void
onSelectionComplete: (wordIds: string[]) => void
onWordTimingChange: (wordId: string, newStartTime: number, newEndTime: number) => void
onWordsMove: (updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }>) => void
syncWordIndex: number
isManualSyncing: boolean
onScrollChange: (newStartTime: number) => void
audioDuration: number
zoomSeconds: number
height?: number
}
// Constants for rendering
const TIME_BAR_HEIGHT = 28
const WORD_BLOCK_HEIGHT = 24
const WORD_LEVEL_SPACING = 50
const CANVAS_PADDING = 8
const TEXT_ABOVE_BLOCK = 14
const RESIZE_HANDLE_SIZE = 8
const RESIZE_HANDLE_HITAREA = 12
const PLAYHEAD_COLOR = '#ffffff'
const WORD_BLOCK_COLOR = '#d32f2f'
const WORD_BLOCK_SELECTED_COLOR = '#b71c1c'
const WORD_BLOCK_CURRENT_COLOR = '#f44336'
const WORD_TEXT_CURRENT_COLOR = '#d32f2f'
const UPCOMING_WORD_BG = '#fff9c4'
const UPCOMING_WORD_TEXT = '#000000'
const TIME_BAR_BG = '#f5f5f5'
const TIME_BAR_TEXT = '#666666'
const TIMELINE_BG = '#e0e0e0'
// Drag modes
type DragMode = 'none' | 'selection' | 'resize' | 'move'
// Build a map of word ID to segment index
function buildWordToSegmentMap(segments: LyricsSegment[]): Map<string, number> {
const map = new Map<string, number>()
segments.forEach((segment, idx) => {
segment.words.forEach(word => {
map.set(word.id, idx)
})
})
return map
}
// Calculate which vertical level a word should be on
function calculateWordLevels(words: Word[], segments: LyricsSegment[]): Map<string, number> {
const levels = new Map<string, number>()
const wordToSegment = buildWordToSegmentMap(segments)
const segmentsWithTiming = segments
.map((segment, idx) => {
const timedWords = segment.words.filter(w => w.start_time !== null)
const minStart = timedWords.length > 0
? Math.min(...timedWords.map(w => w.start_time!))
: Infinity
return { idx, minStart }
})
.filter(s => s.minStart !== Infinity)
.sort((a, b) => a.minStart - b.minStart)
const segmentLevels = new Map<number, number>()
segmentsWithTiming.forEach(({ idx }, orderIndex) => {
segmentLevels.set(idx, orderIndex % 2)
})
for (const word of words) {
const segmentIdx = wordToSegment.get(word.id)
if (segmentIdx !== undefined && segmentLevels.has(segmentIdx)) {
levels.set(word.id, segmentLevels.get(segmentIdx)!)
} else {
levels.set(word.id, 0)
}
}
return levels
}
function formatTime(seconds: number): string {
const mins = Math.floor(seconds / 60)
const secs = Math.floor(seconds % 60)
return `${mins.toString().padStart(2, '0')}:${secs.toString().padStart(2, '0')}`
}
const TimelineCanvas = memo(function TimelineCanvas({
words,
segments,
visibleStartTime,
visibleEndTime,
currentTime,
selectedWordIds,
onWordClick,
onBackgroundClick,
onTimeBarClick,
onSelectionComplete,
onWordTimingChange,
onWordsMove,
syncWordIndex,
isManualSyncing,
onScrollChange,
audioDuration,
zoomSeconds,
height = 200
}: TimelineCanvasProps) {
const canvasRef = useRef<HTMLCanvasElement>(null)
const containerRef = useRef<HTMLDivElement>(null)
const [canvasWidth, setCanvasWidth] = useState(800)
const animationFrameRef = useRef<number>()
const wordLevelsRef = useRef<Map<string, number>>(new Map())
// Drag state
const [dragMode, setDragMode] = useState<DragMode>('none')
const dragStartRef = useRef<{ x: number; y: number; time: number } | null>(null)
const dragWordIdRef = useRef<string | null>(null)
const dragOriginalTimesRef = useRef<Map<string, { start: number; end: number }>>(new Map())
// Selection rectangle
const [selectionRect, setSelectionRect] = useState<{
startX: number; startY: number; endX: number; endY: number
} | null>(null)
// Hover state for showing resize handle
const [hoveredWordId, setHoveredWordId] = useState<string | null>(null)
const [cursorStyle, setCursorStyle] = useState<string>('default')
// Update canvas width on resize
useEffect(() => {
const updateWidth = () => {
if (containerRef.current) {
setCanvasWidth(containerRef.current.clientWidth)
}
}
updateWidth()
const resizeObserver = new ResizeObserver(updateWidth)
if (containerRef.current) {
resizeObserver.observe(containerRef.current)
}
return () => resizeObserver.disconnect()
}, [])
// Calculate word levels when words or segments change
useEffect(() => {
wordLevelsRef.current = calculateWordLevels(words, segments)
}, [words, segments])
// Convert time to x position
const timeToX = useCallback((time: number): number => {
const duration = visibleEndTime - visibleStartTime
if (duration <= 0) return 0
return CANVAS_PADDING + ((time - visibleStartTime) / duration) * (canvasWidth - CANVAS_PADDING * 2)
}, [visibleStartTime, visibleEndTime, canvasWidth])
// Convert x position to time
const xToTime = useCallback((x: number): number => {
const duration = visibleEndTime - visibleStartTime
return visibleStartTime + ((x - CANVAS_PADDING) / (canvasWidth - CANVAS_PADDING * 2)) * duration
}, [visibleStartTime, visibleEndTime, canvasWidth])
// Get word bounds
const getWordBounds = useCallback((word: Word) => {
if (word.start_time === null || word.end_time === null) return null
const level = wordLevelsRef.current.get(word.id) || 0
const startX = timeToX(word.start_time)
const endX = timeToX(word.end_time)
const blockWidth = Math.max(endX - startX, 4)
const y = TIME_BAR_HEIGHT + CANVAS_PADDING + TEXT_ABOVE_BLOCK + level * WORD_LEVEL_SPACING
return { startX, endX, blockWidth, y, level }
}, [timeToX])
// Check if position is near resize handle
const isNearResizeHandlePos = useCallback((word: Word, x: number, y: number): boolean => {
const bounds = getWordBounds(word)
if (!bounds) return false
const handleX = bounds.startX + bounds.blockWidth - RESIZE_HANDLE_SIZE / 2
const handleY = bounds.y + WORD_BLOCK_HEIGHT / 2
return Math.abs(x - handleX) < RESIZE_HANDLE_HITAREA / 2 &&
Math.abs(y - handleY) < RESIZE_HANDLE_HITAREA / 2
}, [getWordBounds])
// Find word at position
const findWordAtPosition = useCallback((x: number, y: number): Word | null => {
for (const word of words) {
const bounds = getWordBounds(word)
if (!bounds) continue
if (x >= bounds.startX && x <= bounds.startX + bounds.blockWidth &&
y >= bounds.y && y <= bounds.y + WORD_BLOCK_HEIGHT) {
return word
}
}
return null
}, [words, getWordBounds])
// Find words in selection rectangle
const findWordsInRect = useCallback((rect: { startX: number; startY: number; endX: number; endY: number }): string[] => {
const rectLeft = Math.min(rect.startX, rect.endX)
const rectRight = Math.max(rect.startX, rect.endX)
const rectTop = Math.min(rect.startY, rect.endY)
const rectBottom = Math.max(rect.startY, rect.endY)
const selectedIds: string[] = []
for (const word of words) {
const bounds = getWordBounds(word)
if (!bounds) continue
if (bounds.startX + bounds.blockWidth >= rectLeft && bounds.startX <= rectRight &&
bounds.y + WORD_BLOCK_HEIGHT >= rectTop && bounds.y <= rectBottom) {
selectedIds.push(word.id)
}
}
return selectedIds
}, [words, getWordBounds])
// Draw the timeline
const draw = useCallback(() => {
const canvas = canvasRef.current
if (!canvas) return
const ctx = canvas.getContext('2d')
if (!ctx) return
const dpr = window.devicePixelRatio || 1
canvas.width = canvasWidth * dpr
canvas.height = height * dpr
ctx.scale(dpr, dpr)
// Clear canvas
ctx.fillStyle = TIMELINE_BG
ctx.fillRect(0, 0, canvasWidth, height)
// Draw time bar background
ctx.fillStyle = TIME_BAR_BG
ctx.fillRect(0, 0, canvasWidth, TIME_BAR_HEIGHT)
// Draw time markers
const duration = visibleEndTime - visibleStartTime
const secondsPerTick = duration > 15 ? 2 : duration > 8 ? 1 : 0.5
const startSecond = Math.ceil(visibleStartTime / secondsPerTick) * secondsPerTick
ctx.fillStyle = TIME_BAR_TEXT
ctx.font = '11px system-ui, -apple-system, sans-serif'
ctx.textAlign = 'center'
for (let t = startSecond; t <= visibleEndTime; t += secondsPerTick) {
const x = timeToX(t)
ctx.beginPath()
ctx.strokeStyle = '#999999'
ctx.lineWidth = 1
ctx.moveTo(x, TIME_BAR_HEIGHT - 6)
ctx.lineTo(x, TIME_BAR_HEIGHT)
ctx.stroke()
if (t % 1 === 0) {
ctx.fillText(formatTime(t), x, TIME_BAR_HEIGHT - 10)
}
}
ctx.beginPath()
ctx.strokeStyle = '#cccccc'
ctx.lineWidth = 1
ctx.moveTo(0, TIME_BAR_HEIGHT)
ctx.lineTo(canvasWidth, TIME_BAR_HEIGHT)
ctx.stroke()
const wordToSegment = buildWordToSegmentMap(segments)
const syncedWords = words.filter(w => w.start_time !== null && w.end_time !== null)
const currentWordId = syncedWords.find(w =>
currentTime >= w.start_time! && currentTime <= w.end_time!
)?.id || null
// First pass: draw all blocks
for (const word of syncedWords) {
const bounds = getWordBounds(word)
if (!bounds) continue
const isSelected = selectedWordIds.has(word.id)
const isCurrent = word.id === currentWordId
const isHovered = word.id === hoveredWordId
// Draw word block background
if (isSelected) {
ctx.fillStyle = WORD_BLOCK_SELECTED_COLOR
} else if (isCurrent) {
ctx.fillStyle = WORD_BLOCK_CURRENT_COLOR
} else {
ctx.fillStyle = WORD_BLOCK_COLOR
}
ctx.fillRect(bounds.startX, bounds.y, bounds.blockWidth, WORD_BLOCK_HEIGHT)
// Draw selection border
if (isSelected) {
ctx.strokeStyle = '#ffffff'
ctx.lineWidth = 2
ctx.strokeRect(bounds.startX, bounds.y, bounds.blockWidth, WORD_BLOCK_HEIGHT)
// Draw resize handle (white dot on right edge) for selected words when hovered
if (isHovered || selectedWordIds.size === 1) {
const handleX = bounds.startX + bounds.blockWidth - RESIZE_HANDLE_SIZE / 2
const handleY = bounds.y + WORD_BLOCK_HEIGHT / 2
ctx.beginPath()
ctx.fillStyle = '#ffffff'
ctx.arc(handleX, handleY, RESIZE_HANDLE_SIZE / 2, 0, Math.PI * 2)
ctx.fill()
ctx.strokeStyle = '#666666'
ctx.lineWidth = 1
ctx.stroke()
}
}
}
// Second pass: draw text
const wordsBySegment = new Map<number, Word[]>()
for (const word of syncedWords) {
const segIdx = wordToSegment.get(word.id)
if (segIdx !== undefined) {
if (!wordsBySegment.has(segIdx)) {
wordsBySegment.set(segIdx, [])
}
wordsBySegment.get(segIdx)!.push(word)
}
}
ctx.font = '11px system-ui, -apple-system, sans-serif'
ctx.textAlign = 'left'
for (const [, segmentWords] of wordsBySegment) {
const sortedWords = [...segmentWords].sort((a, b) =>
(a.start_time || 0) - (b.start_time || 0)
)
if (sortedWords.length === 0) continue
const level = wordLevelsRef.current.get(sortedWords[0].id) || 0
const textY = TIME_BAR_HEIGHT + CANVAS_PADDING + TEXT_ABOVE_BLOCK + level * WORD_LEVEL_SPACING - 3
let rightmostTextEnd = -Infinity
for (const word of sortedWords) {
const blockStartX = timeToX(word.start_time!)
const textWidth = ctx.measureText(word.text).width
const textStartX = Math.max(blockStartX, rightmostTextEnd + 3)
if (textStartX < canvasWidth - 10) {
const isCurrent = word.id === currentWordId
ctx.fillStyle = isCurrent ? WORD_TEXT_CURRENT_COLOR : '#333333'
ctx.fillText(word.text, textStartX, textY)
rightmostTextEnd = textStartX + textWidth
}
}
}
// Draw upcoming words during sync
if (isManualSyncing && syncWordIndex >= 0) {
const upcomingWords = words.slice(syncWordIndex).filter(w => w.start_time === null)
const playheadX = timeToX(currentTime)
let offsetX = playheadX + 10
ctx.font = '11px system-ui, -apple-system, sans-serif'
for (let i = 0; i < Math.min(upcomingWords.length, 12); i++) {
const word = upcomingWords[i]
const textWidth = ctx.measureText(word.text).width + 10
ctx.fillStyle = UPCOMING_WORD_BG
ctx.fillRect(offsetX, TIME_BAR_HEIGHT + CANVAS_PADDING + WORD_LEVEL_SPACING + 60, textWidth, 20)
ctx.fillStyle = UPCOMING_WORD_TEXT
ctx.textAlign = 'left'
ctx.fillText(word.text, offsetX + 5, TIME_BAR_HEIGHT + CANVAS_PADDING + WORD_LEVEL_SPACING + 74)
offsetX += textWidth + 3
if (offsetX > canvasWidth - 20) break
}
}
// Draw playhead
if (currentTime >= visibleStartTime && currentTime <= visibleEndTime) {
const playheadX = timeToX(currentTime)
ctx.beginPath()
ctx.fillStyle = PLAYHEAD_COLOR
ctx.strokeStyle = '#333333'
ctx.lineWidth = 1
ctx.moveTo(playheadX - 6, 2)
ctx.lineTo(playheadX + 6, 2)
ctx.lineTo(playheadX, TIME_BAR_HEIGHT - 4)
ctx.closePath()
ctx.fill()
ctx.stroke()
ctx.beginPath()
ctx.strokeStyle = PLAYHEAD_COLOR
ctx.lineWidth = 2
ctx.moveTo(playheadX, TIME_BAR_HEIGHT)
ctx.lineTo(playheadX, height)
ctx.stroke()
ctx.beginPath()
ctx.strokeStyle = 'rgba(0,0,0,0.4)'
ctx.lineWidth = 1
ctx.moveTo(playheadX + 1, TIME_BAR_HEIGHT)
ctx.lineTo(playheadX + 1, height)
ctx.stroke()
}
// Draw selection rectangle
if (selectionRect) {
ctx.fillStyle = 'rgba(25, 118, 210, 0.2)'
ctx.strokeStyle = 'rgba(25, 118, 210, 0.8)'
ctx.lineWidth = 1
const rectX = Math.min(selectionRect.startX, selectionRect.endX)
const rectY = Math.min(selectionRect.startY, selectionRect.endY)
const rectW = Math.abs(selectionRect.endX - selectionRect.startX)
const rectH = Math.abs(selectionRect.endY - selectionRect.startY)
ctx.fillRect(rectX, rectY, rectW, rectH)
ctx.strokeRect(rectX, rectY, rectW, rectH)
}
}, [
canvasWidth, height, visibleStartTime, visibleEndTime, currentTime,
words, segments, selectedWordIds, selectionRect, hoveredWordId,
syncWordIndex, isManualSyncing, timeToX, getWordBounds
])
// Animation frame
useEffect(() => {
const animate = () => {
draw()
animationFrameRef.current = requestAnimationFrame(animate)
}
animate()
return () => {
if (animationFrameRef.current) {
cancelAnimationFrame(animationFrameRef.current)
}
}
}, [draw])
// Mouse handlers
const handleMouseDown = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const rect = canvasRef.current?.getBoundingClientRect()
if (!rect) return
const x = e.clientX - rect.left
const y = e.clientY - rect.top
const time = xToTime(x)
// Time bar click
if (y < TIME_BAR_HEIGHT) {
onTimeBarClick(Math.max(0, time))
return
}
const clickedWord = findWordAtPosition(x, y)
if (clickedWord && selectedWordIds.has(clickedWord.id)) {
// Check if clicking on resize handle
if (isNearResizeHandlePos(clickedWord, x, y)) {
// Start resize
setDragMode('resize')
dragStartRef.current = { x, y, time }
dragWordIdRef.current = clickedWord.id
dragOriginalTimesRef.current = new Map([[clickedWord.id, {
start: clickedWord.start_time!,
end: clickedWord.end_time!
}]])
return
}
// Start move (for all selected words)
setDragMode('move')
dragStartRef.current = { x, y, time }
dragWordIdRef.current = clickedWord.id
// Store original times for all selected words
const originalTimes = new Map<string, { start: number; end: number }>()
for (const wordId of selectedWordIds) {
const word = words.find(w => w.id === wordId)
if (word && word.start_time !== null && word.end_time !== null) {
originalTimes.set(wordId, { start: word.start_time, end: word.end_time })
}
}
dragOriginalTimesRef.current = originalTimes
return
}
if (clickedWord) {
// Click on unselected word - select it
onWordClick(clickedWord.id, e)
return
}
// Background - start selection
setDragMode('selection')
dragStartRef.current = { x, y, time }
setSelectionRect({ startX: x, startY: y, endX: x, endY: y })
}, [xToTime, onTimeBarClick, findWordAtPosition, selectedWordIds, isNearResizeHandlePos, onWordClick, words])
const handleMouseMove = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const rect = canvasRef.current?.getBoundingClientRect()
if (!rect) return
const x = e.clientX - rect.left
const y = e.clientY - rect.top
const time = xToTime(x)
// Update hover state and cursor
if (dragMode === 'none') {
const hoveredWord = findWordAtPosition(x, y)
setHoveredWordId(hoveredWord?.id || null)
if (hoveredWord && selectedWordIds.has(hoveredWord.id)) {
const nearHandle = isNearResizeHandlePos(hoveredWord, x, y)
setCursorStyle(nearHandle ? 'ew-resize' : 'grab')
} else if (hoveredWord) {
setCursorStyle('pointer')
} else if (y < TIME_BAR_HEIGHT) {
setCursorStyle('pointer')
} else {
setCursorStyle('default')
}
}
if (!dragStartRef.current) return
if (dragMode === 'selection') {
setSelectionRect({
startX: dragStartRef.current.x,
startY: dragStartRef.current.y,
endX: x,
endY: y
})
} else if (dragMode === 'resize' && dragWordIdRef.current) {
// Resize the word
const originalTimes = dragOriginalTimesRef.current.get(dragWordIdRef.current)
if (originalTimes) {
const deltaTime = time - dragStartRef.current.time
const newEndTime = Math.max(originalTimes.start + 0.05, originalTimes.end + deltaTime)
onWordTimingChange(dragWordIdRef.current, originalTimes.start, newEndTime)
}
setCursorStyle('ew-resize')
} else if (dragMode === 'move') {
// Move all selected words
const deltaTime = time - dragStartRef.current.time
const updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }> = []
for (const [wordId, originalTimes] of dragOriginalTimesRef.current) {
// Ensure end time is always after start time (at least 0.05s duration)
const newStartTime = Math.max(0, originalTimes.start + deltaTime)
const newEndTime = Math.max(newStartTime + 0.05, originalTimes.end + deltaTime)
updates.push({
wordId,
newStartTime,
newEndTime
})
}
if (updates.length > 0) {
onWordsMove(updates)
}
setCursorStyle('grabbing')
}
}, [dragMode, xToTime, findWordAtPosition, selectedWordIds, isNearResizeHandlePos, onWordTimingChange, onWordsMove])
const handleMouseUp = useCallback((e: React.MouseEvent<HTMLCanvasElement>) => {
const rect = canvasRef.current?.getBoundingClientRect()
if (dragMode === 'selection' && dragStartRef.current && rect) {
const endX = e.clientX - rect.left
const endY = e.clientY - rect.top
const dragDistance = Math.sqrt(
Math.pow(endX - dragStartRef.current.x, 2) +
Math.pow(endY - dragStartRef.current.y, 2)
)
if (dragDistance < 5) {
onBackgroundClick()
} else {
const finalRect = {
startX: dragStartRef.current.x,
startY: dragStartRef.current.y,
endX,
endY
}
const selectedIds = findWordsInRect(finalRect)
if (selectedIds.length > 0) {
onSelectionComplete(selectedIds)
}
}
}
// Reset drag state
setDragMode('none')
dragStartRef.current = null
dragWordIdRef.current = null
dragOriginalTimesRef.current = new Map()
setSelectionRect(null)
setCursorStyle('default')
}, [dragMode, onBackgroundClick, findWordsInRect, onSelectionComplete])
// Wheel handler
const handleWheel = useCallback((e: React.WheelEvent<HTMLCanvasElement>) => {
const delta = e.deltaX !== 0 ? e.deltaX : e.deltaY
const scrollAmount = (delta / 100) * (zoomSeconds / 4)
let newStart = Math.max(0, Math.min(audioDuration - zoomSeconds, visibleStartTime + scrollAmount))
if (newStart !== visibleStartTime) {
onScrollChange(newStart)
}
}, [visibleStartTime, zoomSeconds, audioDuration, onScrollChange])
const handleScrollLeft = useCallback(() => {
const newStart = Math.max(0, visibleStartTime - zoomSeconds * 0.25)
onScrollChange(newStart)
}, [visibleStartTime, zoomSeconds, onScrollChange])
const handleScrollRight = useCallback(() => {
const newStart = Math.min(audioDuration - zoomSeconds, visibleStartTime + zoomSeconds * 0.25)
onScrollChange(Math.max(0, newStart))
}, [visibleStartTime, zoomSeconds, audioDuration, onScrollChange])
return (
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 0.5 }}>
<Box sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Tooltip title="Scroll Left">
<IconButton
size="small"
onClick={handleScrollLeft}
disabled={visibleStartTime <= 0}
>
<ArrowBackIcon fontSize="small" />
</IconButton>
</Tooltip>
<Box
ref={containerRef}
sx={{
flexGrow: 1,
height,
cursor: cursorStyle,
borderRadius: 1,
overflow: 'hidden'
}}
>
<canvas
ref={canvasRef}
style={{
width: '100%',
height: '100%',
display: 'block',
cursor: cursorStyle
}}
onMouseDown={handleMouseDown}
onMouseMove={handleMouseMove}
onMouseUp={handleMouseUp}
onMouseLeave={handleMouseUp}
onWheel={handleWheel}
/>
</Box>
<Tooltip title="Scroll Right">
<IconButton
size="small"
onClick={handleScrollRight}
disabled={visibleStartTime >= audioDuration - zoomSeconds}
>
<ArrowForwardIcon fontSize="small" />
</IconButton>
</Tooltip>
</Box>
</Box>
)
})
export default TimelineCanvas
@@ -0,0 +1,80 @@
import { memo, useMemo } from 'react'
import { Box, Typography } from '@mui/material'
import { Word } from '../../types'
interface UpcomingWordsBarProps {
words: Word[]
syncWordIndex: number
isManualSyncing: boolean
maxWordsToShow?: number
}
const UpcomingWordsBar = memo(function UpcomingWordsBar({
words,
syncWordIndex,
isManualSyncing,
maxWordsToShow = 20
}: UpcomingWordsBarProps) {
// Get upcoming unsynced words
const upcomingWords = useMemo(() => {
if (!isManualSyncing || syncWordIndex < 0) return []
return words
.slice(syncWordIndex)
.filter(w => w.start_time === null)
.slice(0, maxWordsToShow)
}, [words, syncWordIndex, isManualSyncing, maxWordsToShow])
const totalRemaining = useMemo(() => {
if (!isManualSyncing || syncWordIndex < 0) return 0
return words.slice(syncWordIndex).filter(w => w.start_time === null).length
}, [words, syncWordIndex, isManualSyncing])
// Show empty placeholder when no upcoming words
if (upcomingWords.length === 0) {
return null
}
return (
<Box sx={{
height: 44,
bgcolor: 'grey.100',
borderRadius: 1,
display: 'flex',
alignItems: 'center',
px: 1,
gap: 0.5,
overflow: 'hidden',
boxSizing: 'border-box'
}}>
{upcomingWords.map((word, index) => (
<Box
key={word.id}
sx={{
px: 1,
py: 0.5,
borderRadius: 0.5,
bgcolor: index === 0 ? 'error.main' : 'grey.300',
color: index === 0 ? 'white' : 'text.primary',
fontWeight: index === 0 ? 'bold' : 'normal',
fontSize: '13px',
fontFamily: 'system-ui, -apple-system, sans-serif',
whiteSpace: 'nowrap',
border: index === 0 ? '2px solid' : 'none',
borderColor: 'error.dark'
}}
>
{word.text}
</Box>
))}
{totalRemaining > maxWordsToShow && (
<Typography variant="caption" color="text.secondary" sx={{ ml: 1 }}>
+{totalRemaining - maxWordsToShow} more
</Typography>
)}
</Box>
)
})
export default UpcomingWordsBar
@@ -0,0 +1,905 @@
import { useState, useCallback, useEffect, useRef, useMemo, memo } from 'react'
import {
Box,
Typography,
Slider,
Dialog,
DialogTitle,
DialogContent,
DialogActions,
TextField,
Button,
Paper,
Alert
} from '@mui/material'
import ZoomInIcon from '@mui/icons-material/ZoomIn'
import ZoomOutIcon from '@mui/icons-material/ZoomOut'
import { Word, LyricsSegment } from '../../types'
import TimelineCanvas from './TimelineCanvas'
import UpcomingWordsBar from './UpcomingWordsBar'
import SyncControls from './SyncControls'
// Augment window type for audio functions
declare global {
interface Window {
getAudioDuration?: () => number
toggleAudioPlayback?: () => void
isAudioPlaying?: boolean
}
}
interface LyricsSynchronizerProps {
segments: LyricsSegment[]
currentTime: number
onPlaySegment?: (startTime: number) => void
onSave: (segments: LyricsSegment[]) => void
onCancel: () => void
setModalSpacebarHandler: (handler: (() => (e: KeyboardEvent) => void) | undefined) => void
}
// Constants for zoom
const MIN_ZOOM_SECONDS = 4.5 // Most zoomed in - 4.5 seconds visible
const MAX_ZOOM_SECONDS = 24 // Most zoomed out - 24 seconds visible
const ZOOM_STEPS = 50 // Number of zoom levels
// Get all words from segments
function getAllWords(segments: LyricsSegment[]): Word[] {
return segments.flatMap(s => s.words)
}
// Deep clone segments
function cloneSegments(segments: LyricsSegment[]): LyricsSegment[] {
return JSON.parse(JSON.stringify(segments))
}
const LyricsSynchronizer = memo(function LyricsSynchronizer({
segments: initialSegments,
currentTime,
onPlaySegment,
onSave,
onCancel,
setModalSpacebarHandler
}: LyricsSynchronizerProps) {
// Working copy of segments
const [workingSegments, setWorkingSegments] = useState<LyricsSegment[]>(() =>
cloneSegments(initialSegments)
)
// Get all words flattened
const allWords = useMemo(() => getAllWords(workingSegments), [workingSegments])
// Audio duration
const audioDuration = useMemo(() => {
if (typeof window.getAudioDuration === 'function') {
const duration = window.getAudioDuration()
return duration > 0 ? duration : 300
}
return 300 // 5 minute fallback
}, [])
// Zoom state (value is the visible time window in seconds)
const [zoomSeconds, setZoomSeconds] = useState(12) // Default ~12 seconds visible
// Visible time range
const [visibleStartTime, setVisibleStartTime] = useState(0)
const visibleEndTime = useMemo(() =>
Math.min(visibleStartTime + zoomSeconds, audioDuration),
[visibleStartTime, zoomSeconds, audioDuration]
)
// Manual sync state
const [isManualSyncing, setIsManualSyncing] = useState(false)
const [isPaused, setIsPaused] = useState(false)
const [syncWordIndex, setSyncWordIndex] = useState(-1)
const [isSpacebarPressed, setIsSpacebarPressed] = useState(false)
const wordStartTimeRef = useRef<number | null>(null)
const spacebarPressTimeRef = useRef<number | null>(null)
const currentTimeRef = useRef(currentTime)
// Selection state
const [selectedWordIds, setSelectedWordIds] = useState<Set<string>>(new Set())
// Edit lyrics modal state
const [showEditLyricsModal, setShowEditLyricsModal] = useState(false)
const [editLyricsText, setEditLyricsText] = useState('')
// Edit word modal state
const [showEditWordModal, setShowEditWordModal] = useState(false)
const [editWordText, setEditWordText] = useState('')
const [editWordId, setEditWordId] = useState<string | null>(null)
// Keep currentTimeRef up to date
useEffect(() => {
currentTimeRef.current = currentTime
}, [currentTime])
// Auto-scroll to follow playhead during sync
useEffect(() => {
if (isManualSyncing && !isPaused && currentTime > 0) {
// If playhead is near the end of visible area, scroll forward
if (currentTime > visibleEndTime - (zoomSeconds * 0.1)) {
const newStart = Math.max(0, currentTime - zoomSeconds * 0.1)
setVisibleStartTime(newStart)
}
// If playhead is before visible area, scroll back
else if (currentTime < visibleStartTime) {
setVisibleStartTime(Math.max(0, currentTime - 1))
}
}
}, [currentTime, isManualSyncing, isPaused, visibleStartTime, visibleEndTime, zoomSeconds])
// Handle zoom slider change
const handleZoomChange = useCallback((_: Event, value: number | number[]) => {
const zoomValue = value as number
// Map slider value (0-50) to zoom range (4.5-24 seconds)
const newZoomSeconds = MIN_ZOOM_SECONDS + (zoomValue / ZOOM_STEPS) * (MAX_ZOOM_SECONDS - MIN_ZOOM_SECONDS)
setZoomSeconds(newZoomSeconds)
}, [])
// Get slider value from zoom seconds
const sliderValue = useMemo(() => {
return ((zoomSeconds - MIN_ZOOM_SECONDS) / (MAX_ZOOM_SECONDS - MIN_ZOOM_SECONDS)) * ZOOM_STEPS
}, [zoomSeconds])
// Handle scroll change from timeline
const handleScrollChange = useCallback((newStartTime: number) => {
setVisibleStartTime(newStartTime)
}, [])
// Update words in segments
const updateWords = useCallback((newWords: Word[]) => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
// Create a map of word id to word for quick lookup
const wordMap = new Map(newWords.map(w => [w.id, w]))
// Update each segment's words
for (const segment of newSegments) {
segment.words = segment.words.map(w => wordMap.get(w.id) || w)
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
} else {
segment.start_time = null
segment.end_time = null
}
}
return newSegments
})
}, [])
// Check if audio is playing
const [isPlaying, setIsPlaying] = useState(false)
// Update isPlaying state periodically
useEffect(() => {
const checkPlaying = () => {
setIsPlaying(typeof window.isAudioPlaying === 'boolean' ? window.isAudioPlaying : false)
}
checkPlaying()
const interval = setInterval(checkPlaying, 100)
return () => clearInterval(interval)
}, [])
// Play audio from current position
const handlePlayAudio = useCallback(() => {
if (onPlaySegment) {
onPlaySegment(currentTimeRef.current)
}
}, [onPlaySegment])
// Stop audio playback - also exits sync mode
const handleStopAudio = useCallback(() => {
if (typeof window.toggleAudioPlayback === 'function' && window.isAudioPlaying) {
window.toggleAudioPlayback()
}
// Also exit sync mode when stopping
if (isManualSyncing) {
setIsManualSyncing(false)
setIsPaused(false)
setIsSpacebarPressed(false)
}
}, [isManualSyncing])
// Start manual sync
const handleStartSync = useCallback(() => {
if (isManualSyncing) {
// Stop sync
setIsManualSyncing(false)
setIsPaused(false)
setSyncWordIndex(-1)
setIsSpacebarPressed(false)
// Stop audio
handleStopAudio()
return
}
// Find first unsynced word
const firstUnsyncedIndex = allWords.findIndex(w =>
w.start_time === null || w.end_time === null
)
const startIndex = firstUnsyncedIndex !== -1 ? firstUnsyncedIndex : 0
setIsManualSyncing(true)
setIsPaused(false)
setSyncWordIndex(startIndex)
setIsSpacebarPressed(false)
// Start playback
if (onPlaySegment) {
onPlaySegment(Math.max(0, currentTimeRef.current - 1))
}
}, [isManualSyncing, allWords, onPlaySegment, handleStopAudio])
// Pause sync
const handlePauseSync = useCallback(() => {
setIsPaused(true)
handleStopAudio()
}, [handleStopAudio])
// Resume sync
const handleResumeSync = useCallback(() => {
setIsPaused(false)
// Find first unsynced word from current position
const firstUnsyncedIndex = allWords.findIndex(w =>
w.start_time === null || w.end_time === null
)
if (firstUnsyncedIndex !== -1 && firstUnsyncedIndex !== syncWordIndex) {
setSyncWordIndex(firstUnsyncedIndex)
}
// Resume playback
if (onPlaySegment) {
onPlaySegment(currentTimeRef.current)
}
}, [allWords, syncWordIndex, onPlaySegment])
// Clear all sync data
const handleClearSync = useCallback(() => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
for (const word of segment.words) {
word.start_time = null
word.end_time = null
}
segment.start_time = null
segment.end_time = null
}
return newSegments
})
setSyncWordIndex(-1)
}, [])
// Unsync from cursor position
const handleUnsyncFromCursor = useCallback(() => {
const cursorTime = currentTimeRef.current
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
for (const word of segment.words) {
// Reset words that start after cursor position
if (word.start_time !== null && word.start_time > cursorTime) {
word.start_time = null
word.end_time = null
}
}
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
} else {
segment.start_time = null
segment.end_time = null
}
}
return newSegments
})
}, [])
// Check if there are words after cursor that can be unsynced
const canUnsyncFromCursor = useMemo(() => {
const cursorTime = currentTimeRef.current
return allWords.some(w =>
w.start_time !== null && w.start_time > cursorTime
)
}, [allWords, currentTime])
// Open edit lyrics modal
const handleEditLyrics = useCallback(() => {
const text = workingSegments.map(s => s.text).join('\n')
setEditLyricsText(text)
setShowEditLyricsModal(true)
}, [workingSegments])
// Save edited lyrics (warning: resets timing)
const handleSaveEditedLyrics = useCallback(() => {
const lines = editLyricsText.split('\n').filter(l => l.trim())
const newSegments: LyricsSegment[] = lines.map((line, idx) => {
const words = line.trim().split(/\s+/).map((text, wIdx) => ({
id: `word-${idx}-${wIdx}-${Date.now()}`,
text,
start_time: null,
end_time: null,
confidence: 1.0
}))
return {
id: `segment-${idx}-${Date.now()}`,
text: line.trim(),
words,
start_time: null,
end_time: null
}
})
setWorkingSegments(newSegments)
setShowEditLyricsModal(false)
setSyncWordIndex(-1)
}, [editLyricsText])
// Open edit word modal
const handleEditSelectedWord = useCallback(() => {
if (selectedWordIds.size !== 1) return
const wordId = Array.from(selectedWordIds)[0]
const word = allWords.find(w => w.id === wordId)
if (word) {
setEditWordId(wordId)
setEditWordText(word.text)
setShowEditWordModal(true)
}
}, [selectedWordIds, allWords])
// Save edited word
const handleSaveEditedWord = useCallback(() => {
if (!editWordId) return
const newText = editWordText.trim()
if (!newText) return
// Check if we're splitting into multiple words
const newWords = newText.split(/\s+/)
if (newWords.length === 1) {
// Simple rename
const updatedWords = allWords.map(w =>
w.id === editWordId ? { ...w, text: newWords[0] } : w
)
updateWords(updatedWords)
} else {
// Split word - preserve timing for first word, null for rest
const originalWord = allWords.find(w => w.id === editWordId)
if (!originalWord) return
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
const wordIndex = segment.words.findIndex(w => w.id === editWordId)
if (wordIndex !== -1) {
const newWordObjects = newWords.map((text, idx) => ({
id: idx === 0 ? editWordId : `${editWordId}-split-${idx}`,
text,
start_time: idx === 0 ? originalWord.start_time : null,
end_time: idx === 0 ? originalWord.end_time : null,
confidence: 1.0
}))
segment.words.splice(wordIndex, 1, ...newWordObjects)
segment.text = segment.words.map(w => w.text).join(' ')
break
}
}
return newSegments
})
}
setShowEditWordModal(false)
setEditWordId(null)
setEditWordText('')
setSelectedWordIds(new Set())
}, [editWordId, editWordText, allWords, updateWords])
// Delete selected words
const handleDeleteSelected = useCallback(() => {
if (selectedWordIds.size === 0) return
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
segment.words = segment.words.filter(w => !selectedWordIds.has(w.id))
segment.text = segment.words.map(w => w.text).join(' ')
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
} else {
segment.start_time = null
segment.end_time = null
}
}
// Remove empty segments
return newSegments.filter(s => s.words.length > 0)
})
setSelectedWordIds(new Set())
}, [selectedWordIds])
// Handle word click (selection)
const handleWordClick = useCallback((wordId: string, event: React.MouseEvent) => {
if (event.shiftKey || event.ctrlKey || event.metaKey) {
// Add to selection
setSelectedWordIds(prev => {
const newSet = new Set(prev)
if (newSet.has(wordId)) {
newSet.delete(wordId)
} else {
newSet.add(wordId)
}
return newSet
})
} else {
// Single selection
setSelectedWordIds(new Set([wordId]))
}
}, [])
// Handle background click (deselect)
const handleBackgroundClick = useCallback(() => {
setSelectedWordIds(new Set())
}, [])
// Handle single word timing change (from resize)
const handleWordTimingChange = useCallback((wordId: string, newStartTime: number, newEndTime: number) => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
for (const segment of newSegments) {
const word = segment.words.find(w => w.id === wordId)
if (word) {
word.start_time = Math.max(0, newStartTime)
word.end_time = Math.max(word.start_time + 0.05, newEndTime)
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
}
break
}
}
return newSegments
})
}, [])
// Handle moving multiple words (from drag)
const handleWordsMove = useCallback((updates: Array<{ wordId: string; newStartTime: number; newEndTime: number }>) => {
setWorkingSegments(prevSegments => {
const newSegments = cloneSegments(prevSegments)
// Create a map for quick lookup
const updateMap = new Map(updates.map(u => [u.wordId, u]))
for (const segment of newSegments) {
for (const word of segment.words) {
const update = updateMap.get(word.id)
if (update) {
word.start_time = update.newStartTime
word.end_time = update.newEndTime
}
}
// Recalculate segment timing
const timedWords = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (timedWords.length > 0) {
segment.start_time = Math.min(...timedWords.map(w => w.start_time!))
segment.end_time = Math.max(...timedWords.map(w => w.end_time!))
}
}
return newSegments
})
}, [])
// Handle time bar click (seek to position without playing)
const handleTimeBarClick = useCallback((time: number) => {
// Scroll the timeline to show this time centered
const newStart = Math.max(0, time - zoomSeconds / 2)
setVisibleStartTime(Math.min(newStart, Math.max(0, audioDuration - zoomSeconds)))
// Seek to the position (this will briefly start playback)
if (onPlaySegment) {
onPlaySegment(time)
// Immediately stop playback after seeking
setTimeout(() => {
if (typeof window.toggleAudioPlayback === 'function' && window.isAudioPlaying) {
window.toggleAudioPlayback()
}
}, 50)
}
}, [zoomSeconds, audioDuration, onPlaySegment])
// Handle selection complete from drag
const handleSelectionComplete = useCallback((wordIds: string[]) => {
setSelectedWordIds(new Set(wordIds))
}, [])
// Handle spacebar for manual sync
const handleKeyDown = useCallback((e: KeyboardEvent) => {
if (e.code !== 'Space') return
if (!isManualSyncing || isPaused) return
if (syncWordIndex < 0 || syncWordIndex >= allWords.length) return
e.preventDefault()
e.stopPropagation()
if (isSpacebarPressed) return
setIsSpacebarPressed(true)
wordStartTimeRef.current = currentTimeRef.current
spacebarPressTimeRef.current = Date.now()
// Set start time for current word
const newWords = [...allWords]
const currentWord = newWords[syncWordIndex]
currentWord.start_time = currentTimeRef.current
// Handle previous word's end time
if (syncWordIndex > 0) {
const prevWord = newWords[syncWordIndex - 1]
if (prevWord.start_time !== null && prevWord.end_time === null) {
const gap = currentTimeRef.current - prevWord.start_time
if (gap > 1.0) {
prevWord.end_time = prevWord.start_time + 0.5
} else {
prevWord.end_time = currentTimeRef.current - 0.005
}
}
}
updateWords(newWords)
}, [isManualSyncing, isPaused, syncWordIndex, allWords, isSpacebarPressed, updateWords])
const handleKeyUp = useCallback((e: KeyboardEvent) => {
if (e.code !== 'Space') return
if (!isManualSyncing || isPaused) return
if (!isSpacebarPressed) return
e.preventDefault()
e.stopPropagation()
setIsSpacebarPressed(false)
const pressDuration = spacebarPressTimeRef.current
? Date.now() - spacebarPressTimeRef.current
: 0
const isTap = pressDuration < 200 // 200ms threshold
const newWords = [...allWords]
const currentWord = newWords[syncWordIndex]
if (isTap) {
// Short tap: default 500ms duration
currentWord.end_time = (wordStartTimeRef.current || currentTimeRef.current) + 0.5
} else {
// Hold: use actual timing
currentWord.end_time = currentTimeRef.current
}
updateWords(newWords)
// Move to next word
if (syncWordIndex < allWords.length - 1) {
setSyncWordIndex(syncWordIndex + 1)
} else {
// All words synced
setIsManualSyncing(false)
setSyncWordIndex(-1)
handleStopAudio()
}
wordStartTimeRef.current = null
spacebarPressTimeRef.current = null
}, [isManualSyncing, isPaused, isSpacebarPressed, syncWordIndex, allWords, updateWords, handleStopAudio])
// Combined spacebar handler
const handleSpacebar = useCallback((e: KeyboardEvent) => {
if (e.type === 'keydown') {
handleKeyDown(e)
} else if (e.type === 'keyup') {
handleKeyUp(e)
}
}, [handleKeyDown, handleKeyUp])
// Keep ref for handler
const spacebarHandlerRef = useRef(handleSpacebar)
spacebarHandlerRef.current = handleSpacebar
// Set up spacebar handler
useEffect(() => {
const handler = (e: KeyboardEvent) => {
if (e.code === 'Space') {
e.preventDefault()
e.stopPropagation()
spacebarHandlerRef.current(e)
}
}
setModalSpacebarHandler(() => handler)
return () => {
setModalSpacebarHandler(undefined)
}
}, [setModalSpacebarHandler])
// Handle save
const handleSave = useCallback(() => {
onSave(workingSegments)
}, [workingSegments, onSave])
// Progress stats
const stats = useMemo(() => {
const total = allWords.length
const synced = allWords.filter(w =>
w.start_time !== null && w.end_time !== null
).length
return { total, synced, remaining: total - synced }
}, [allWords])
// Get instruction text based on state
const getInstructionText = useCallback(() => {
if (isManualSyncing) {
if (isSpacebarPressed) {
// Always include secondary text to prevent layout shift
return { primary: '⏱️ Holding... release when word ends', secondary: 'Release spacebar when the word finishes' }
}
if (stats.remaining === 0) {
return { primary: '✅ All words synced!', secondary: 'Click "Stop Sync" then "Apply" to save' }
}
return { primary: '👆 Press SPACEBAR when you hear each word', secondary: 'Tap for short words, hold for longer words' }
}
if (stats.synced === 0) {
return { primary: 'Click "Start Sync" to begin timing words', secondary: 'Audio will play and you\'ll tap spacebar for each word' }
}
if (stats.remaining > 0) {
return { primary: `${stats.remaining} words remaining to sync`, secondary: 'Click "Start Sync" to continue, or "Unsync from Cursor" to re-sync from a point' }
}
return { primary: '✅ All words synced!', secondary: 'Click "Apply" to save changes, or make adjustments first' }
}, [isManualSyncing, isSpacebarPressed, stats.synced, stats.remaining])
const instruction = getInstructionText()
return (
<Box sx={{ display: 'flex', flexDirection: 'column', height: '100%', gap: 1 }}>
{/* Stats bar - fixed height */}
<Box sx={{ display: 'flex', justifyContent: 'flex-end', alignItems: 'center', height: 24 }}>
<Typography variant="body2" color="text.secondary">
{stats.synced} / {stats.total} words synced
{stats.remaining > 0 && ` (${stats.remaining} remaining)`}
</Typography>
</Box>
{/* Instruction banner - fixed height to prevent layout shifts */}
<Box
sx={{
height: 56,
flexShrink: 0
}}
>
<Paper
sx={{
p: 1.5,
height: '100%',
bgcolor: isManualSyncing ? 'info.main' : 'grey.100',
color: isManualSyncing ? 'info.contrastText' : 'text.primary',
display: 'flex',
flexDirection: 'column',
justifyContent: 'center',
overflow: 'hidden',
boxSizing: 'border-box'
}}
>
<Typography variant="body2" sx={{ fontWeight: 500, lineHeight: 1.3 }}>
{instruction.primary}
</Typography>
<Typography variant="caption" sx={{ opacity: 0.85, display: 'block', lineHeight: 1.3 }}>
{instruction.secondary}
</Typography>
</Paper>
</Box>
{/* Controls - fixed height section */}
<Box sx={{ height: 88, flexShrink: 0 }}>
<SyncControls
isManualSyncing={isManualSyncing}
isPaused={isPaused}
onStartSync={handleStartSync}
onPauseSync={handlePauseSync}
onResumeSync={handleResumeSync}
onClearSync={handleClearSync}
onEditLyrics={handleEditLyrics}
onPlay={handlePlayAudio}
onStop={handleStopAudio}
isPlaying={isPlaying}
hasSelectedWords={selectedWordIds.size > 0}
selectedWordCount={selectedWordIds.size}
onUnsyncFromCursor={handleUnsyncFromCursor}
onEditSelectedWord={handleEditSelectedWord}
onDeleteSelected={handleDeleteSelected}
canUnsyncFromCursor={canUnsyncFromCursor}
/>
</Box>
{/* Upcoming words bar - fixed height, immediately above timeline */}
<Box sx={{ height: 44, flexShrink: 0 }}>
{isManualSyncing && (
<UpcomingWordsBar
words={allWords}
syncWordIndex={syncWordIndex}
isManualSyncing={isManualSyncing}
/>
)}
</Box>
{/* Timeline canvas */}
<Box sx={{ flexGrow: 1, minHeight: 200 }}>
<TimelineCanvas
words={allWords}
segments={workingSegments}
visibleStartTime={visibleStartTime}
visibleEndTime={visibleEndTime}
currentTime={currentTime}
selectedWordIds={selectedWordIds}
onWordClick={handleWordClick}
onBackgroundClick={handleBackgroundClick}
onTimeBarClick={handleTimeBarClick}
onSelectionComplete={handleSelectionComplete}
onWordTimingChange={handleWordTimingChange}
onWordsMove={handleWordsMove}
syncWordIndex={syncWordIndex}
isManualSyncing={isManualSyncing}
onScrollChange={handleScrollChange}
audioDuration={audioDuration}
zoomSeconds={zoomSeconds}
height={200}
/>
</Box>
{/* Zoom slider */}
<Box sx={{ display: 'flex', alignItems: 'center', gap: 2, px: 2 }}>
<ZoomInIcon color="action" fontSize="small" />
<Slider
value={sliderValue}
onChange={handleZoomChange}
min={0}
max={ZOOM_STEPS}
step={1}
sx={{ flexGrow: 1 }}
disabled={isManualSyncing && !isPaused}
/>
<ZoomOutIcon color="action" fontSize="small" />
<Typography variant="caption" color="text.secondary" sx={{ minWidth: 60 }}>
{zoomSeconds.toFixed(1)}s view
</Typography>
</Box>
{/* Action buttons */}
<Box sx={{ display: 'flex', justifyContent: 'flex-end', gap: 2, pt: 2, borderTop: 1, borderColor: 'divider' }}>
<Button onClick={onCancel} color="inherit">
Cancel
</Button>
<Button
onClick={handleSave}
variant="contained"
color="primary"
disabled={isManualSyncing && !isPaused}
>
Apply
</Button>
</Box>
{/* Edit Lyrics Modal */}
<Dialog
open={showEditLyricsModal}
onClose={() => setShowEditLyricsModal(false)}
maxWidth="md"
fullWidth
>
<DialogTitle>Edit Lyrics</DialogTitle>
<DialogContent>
<Alert severity="warning" sx={{ mb: 2 }}>
Editing lyrics will reset all timing data. You will need to re-sync the entire song.
</Alert>
<TextField
multiline
rows={15}
fullWidth
value={editLyricsText}
onChange={(e) => setEditLyricsText(e.target.value)}
placeholder="Enter lyrics, one line per segment..."
/>
</DialogContent>
<DialogActions>
<Button onClick={() => setShowEditLyricsModal(false)}>Cancel</Button>
<Button onClick={handleSaveEditedLyrics} variant="contained" color="warning">
Save & Reset Timing
</Button>
</DialogActions>
</Dialog>
{/* Edit Word Modal */}
<Dialog
open={showEditWordModal}
onClose={() => setShowEditWordModal(false)}
maxWidth="xs"
fullWidth
>
<DialogTitle>Edit Word</DialogTitle>
<DialogContent>
<Typography variant="body2" color="text.secondary" sx={{ mb: 2 }}>
Edit the word text. Enter multiple words separated by spaces to split.
</Typography>
<TextField
fullWidth
value={editWordText}
onChange={(e) => setEditWordText(e.target.value)}
autoFocus
onKeyDown={(e) => {
if (e.key === 'Enter') {
handleSaveEditedWord()
}
}}
/>
</DialogContent>
<DialogActions>
<Button onClick={() => setShowEditWordModal(false)}>Cancel</Button>
<Button onClick={handleSaveEditedWord} variant="contained">
Save
</Button>
</DialogActions>
</Dialog>
</Box>
)
})
export default LyricsSynchronizer
@@ -0,0 +1,127 @@
import {
Dialog,
DialogTitle,
DialogContent,
DialogActions,
IconButton,
Box,
Button,
Typography,
Paper
} from '@mui/material'
import CloseIcon from '@mui/icons-material/Close'
import ContentPasteIcon from '@mui/icons-material/ContentPaste'
import SyncIcon from '@mui/icons-material/Sync'
interface ModeSelectionModalProps {
open: boolean
onClose: () => void
onSelectReplace: () => void // Goes to paste phase for replacing all lyrics
onSelectResync: () => void // Goes directly to sync view with existing lyrics
hasExistingLyrics: boolean // Whether there are existing lyrics to re-sync
}
export default function ModeSelectionModal({
open,
onClose,
onSelectReplace,
onSelectResync,
hasExistingLyrics
}: ModeSelectionModalProps) {
return (
<Dialog
open={open}
onClose={onClose}
maxWidth="sm"
fullWidth
>
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Box sx={{ flex: 1 }}>
Edit All Lyrics
</Box>
<IconButton onClick={onClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
<DialogContent dividers>
<Typography variant="body1" sx={{ mb: 3 }}>
Choose how you want to edit the lyrics:
</Typography>
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 2 }}>
{/* Re-sync Existing option - only show if there are existing lyrics */}
{hasExistingLyrics && (
<Paper
sx={{
p: 2,
cursor: 'pointer',
border: 2,
borderColor: 'primary.main',
'&:hover': {
bgcolor: 'action.hover',
borderColor: 'primary.dark'
}
}}
onClick={onSelectResync}
>
<Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}>
<SyncIcon color="primary" sx={{ fontSize: 40, mt: 0.5 }} />
<Box>
<Typography variant="h6" color="primary">
Re-sync Existing Lyrics
</Typography>
<Typography variant="body2" color="text.secondary">
Keep the current lyrics text and fix timing issues.
Use this when lyrics are correct but timing has drifted,
especially in the second half of the song.
</Typography>
<Typography variant="caption" color="success.main" sx={{ mt: 1, display: 'block' }}>
Recommended for fixing timing drift
</Typography>
</Box>
</Box>
</Paper>
)}
{/* Replace All option */}
<Paper
sx={{
p: 2,
cursor: 'pointer',
border: 1,
borderColor: 'divider',
'&:hover': {
bgcolor: 'action.hover',
borderColor: 'text.secondary'
}
}}
onClick={onSelectReplace}
>
<Box sx={{ display: 'flex', alignItems: 'flex-start', gap: 2 }}>
<ContentPasteIcon sx={{ fontSize: 40, mt: 0.5, color: 'text.secondary' }} />
<Box>
<Typography variant="h6">
Replace All Lyrics
</Typography>
<Typography variant="body2" color="text.secondary">
Paste completely new lyrics from clipboard and manually
sync timing for all words from scratch.
</Typography>
<Typography variant="caption" color="warning.main" sx={{ mt: 1, display: 'block' }}>
All existing timing data will be lost
</Typography>
</Box>
</Box>
</Paper>
</Box>
</DialogContent>
<DialogActions>
<Button onClick={onClose} color="inherit">
Cancel
</Button>
</DialogActions>
</Dialog>
)
}
@@ -7,19 +7,17 @@ import {
Box,
Button,
Typography,
TextField,
Paper,
Divider
TextField
} from '@mui/material'
import CloseIcon from '@mui/icons-material/Close'
import ContentPasteIcon from '@mui/icons-material/ContentPaste'
import AutoFixHighIcon from '@mui/icons-material/AutoFixHigh'
import ArrowBackIcon from '@mui/icons-material/ArrowBack'
import { LyricsSegment, Word } from '../types'
import { useState, useEffect, useCallback, useMemo, useRef, memo } from 'react'
import { useState, useEffect, useCallback, useMemo } from 'react'
import { nanoid } from 'nanoid'
import useManualSync from '../hooks/useManualSync'
import EditTimelineSection from './EditTimelineSection'
import EditActionBar from './EditActionBar'
import ModeSelectionModal from './ModeSelectionModal'
import LyricsSynchronizer from './LyricsSynchronizer'
// Augment window type for audio functions
declare global {
@@ -30,6 +28,8 @@ declare global {
}
}
type ModalMode = 'selection' | 'replace' | 'resync'
interface ReplaceAllLyricsModalProps {
open: boolean
onClose: () => void
@@ -37,6 +37,7 @@ interface ReplaceAllLyricsModalProps {
onPlaySegment?: (startTime: number) => void
currentTime?: number
setModalSpacebarHandler: (handler: (() => (e: KeyboardEvent) => void) | undefined) => void
existingSegments?: LyricsSegment[] // Current segments for re-sync mode
}
export default function ReplaceAllLyricsModal({
@@ -45,22 +46,21 @@ export default function ReplaceAllLyricsModal({
onSave,
onPlaySegment,
currentTime = 0,
setModalSpacebarHandler
setModalSpacebarHandler,
existingSegments = []
}: ReplaceAllLyricsModalProps) {
const [mode, setMode] = useState<ModalMode>('selection')
const [inputText, setInputText] = useState('')
const [isReplaced, setIsReplaced] = useState(false)
const [globalSegment, setGlobalSegment] = useState<LyricsSegment | null>(null)
const [originalSegments, setOriginalSegments] = useState<LyricsSegment[]>([])
const [currentSegments, setCurrentSegments] = useState<LyricsSegment[]>([])
const [newSegments, setNewSegments] = useState<LyricsSegment[]>([])
// Get the real audio duration, with fallback
const getAudioDuration = useCallback(() => {
if (window.getAudioDuration) {
const duration = window.getAudioDuration()
return duration > 0 ? duration : 600 // Use real duration or 10 min fallback
// Reset state when modal opens
useEffect(() => {
if (open) {
setMode('selection')
setInputText('')
setNewSegments([])
}
return 600 // 10 minute fallback if audio not loaded
}, [])
}, [open])
// Parse the input text to get line and word counts
const parseInfo = useMemo(() => {
@@ -79,61 +79,31 @@ export default function ReplaceAllLyricsModal({
if (!inputText.trim()) return
const lines = inputText.trim().split('\n').filter(line => line.trim().length > 0)
const newSegments: LyricsSegment[] = []
const allWords: Word[] = []
const segments: LyricsSegment[] = []
lines.forEach((line) => {
const words = line.trim().split(/\s+/).filter(word => word.length > 0)
const segmentWords: Word[] = []
const segmentWords: Word[] = words.map((wordText) => ({
id: nanoid(),
text: wordText,
start_time: null,
end_time: null,
confidence: 1.0,
created_during_correction: true
}))
words.forEach((wordText) => {
const word: Word = {
id: nanoid(),
text: wordText,
start_time: null,
end_time: null,
confidence: 1.0,
created_during_correction: true
}
segmentWords.push(word)
allWords.push(word)
})
const segment: LyricsSegment = {
segments.push({
id: nanoid(),
text: line.trim(),
words: segmentWords,
start_time: null,
end_time: null
}
newSegments.push(segment)
})
})
// Create a global segment with all words for manual sync
// Set a very large end time to ensure manual sync doesn't stop prematurely
const audioDuration = getAudioDuration()
const endTime = Math.max(audioDuration, 3600) // At least 1 hour to prevent auto-stop
console.log('ReplaceAllLyricsModal - Creating global segment', {
audioDuration,
endTime,
wordCount: allWords.length
})
const globalSegment: LyricsSegment = {
id: 'global-replacement',
text: allWords.map(w => w.text).join(' '),
words: allWords,
start_time: 0,
end_time: endTime
}
setCurrentSegments(newSegments)
setOriginalSegments(JSON.parse(JSON.stringify(newSegments)))
setGlobalSegment(globalSegment)
setIsReplaced(true)
}, [inputText, getAudioDuration])
setNewSegments(segments)
setMode('resync') // Go to synchronizer with the new segments
}, [inputText])
// Handle paste from clipboard
const handlePasteFromClipboard = useCallback(async () => {
@@ -146,311 +116,90 @@ export default function ReplaceAllLyricsModal({
}
}, [])
// Update segment when words change during manual sync
const updateSegment = useCallback((newWords: Word[]) => {
if (!globalSegment) return
const validStartTimes = newWords.map(w => w.start_time).filter((t): t is number => t !== null)
const validEndTimes = newWords.map(w => w.end_time).filter((t): t is number => t !== null)
const segmentStartTime = validStartTimes.length > 0 ? Math.min(...validStartTimes) : null
const segmentEndTime = validEndTimes.length > 0 ? Math.max(...validEndTimes) : null
const updatedGlobalSegment = {
...globalSegment,
words: newWords,
text: newWords.map(w => w.text).join(' '),
start_time: segmentStartTime,
end_time: segmentEndTime
}
// Batch state updates to prevent multiple re-renders
setGlobalSegment(updatedGlobalSegment)
// Update individual segment timing as words get synced - but only calculate when needed
const updatedSegments = currentSegments.map(segment => {
// Find words that belong to this segment and have been timed
const segmentWordsWithTiming = segment.words.map(segmentWord => {
const globalWord = newWords.find(w => w.id === segmentWord.id)
return globalWord || segmentWord
})
// Calculate segment timing if all words have timing
const wordsWithTiming = segmentWordsWithTiming.filter(w =>
w.start_time !== null && w.end_time !== null
)
if (wordsWithTiming.length === segmentWordsWithTiming.length && wordsWithTiming.length > 0) {
// All words in this segment have timing - update segment timing
const segmentStart = Math.min(...wordsWithTiming.map(w => w.start_time!))
const segmentEnd = Math.max(...wordsWithTiming.map(w => w.end_time!))
return {
...segment,
words: segmentWordsWithTiming,
start_time: segmentStart,
end_time: segmentEnd
}
} else {
// Some words still don't have timing - just update the words
return {
...segment,
words: segmentWordsWithTiming
}
}
})
setCurrentSegments(updatedSegments)
}, [globalSegment, currentSegments])
// Use the manual sync hook
const {
isManualSyncing,
isPaused,
syncWordIndex,
startManualSync,
pauseManualSync,
resumeManualSync,
cleanupManualSync,
handleSpacebar,
isSpacebarPressed
} = useManualSync({
editedSegment: globalSegment,
currentTime,
onPlaySegment,
updateSegment
})
// Handle manual word updates (drag/resize in timeline)
const handleWordUpdate = useCallback((wordIndex: number, updates: Partial<Word>) => {
if (!globalSegment) return
// Only allow manual adjustments when manual sync is paused or not active
if (isManualSyncing && !isPaused) {
console.log('ReplaceAllLyricsModal - Ignoring word update during active manual sync')
return
}
console.log('ReplaceAllLyricsModal - Manual word update', {
wordIndex,
wordText: globalSegment.words[wordIndex]?.text,
updates,
isManualSyncing,
isPaused
})
// Update the word in the global segment
const newWords = [...globalSegment.words]
newWords[wordIndex] = {
...newWords[wordIndex],
...updates
}
// Update the global segment through the existing updateSegment function
updateSegment(newWords)
}, [globalSegment, updateSegment, isManualSyncing, isPaused])
// Handle un-syncing a word (right-click context menu)
const handleUnsyncWord = useCallback((wordIndex: number) => {
if (!globalSegment) return
console.log('ReplaceAllLyricsModal - Un-syncing word', {
wordIndex,
wordText: globalSegment.words[wordIndex]?.text
})
// Update the word to remove timing
const newWords = [...globalSegment.words]
newWords[wordIndex] = {
...newWords[wordIndex],
start_time: null,
end_time: null
}
// Update the global segment through the existing updateSegment function
updateSegment(newWords)
}, [globalSegment, updateSegment])
// Handle modal close
const handleClose = useCallback(() => {
cleanupManualSync()
setMode('selection')
setInputText('')
setIsReplaced(false)
setGlobalSegment(null)
setOriginalSegments([])
setCurrentSegments([])
setNewSegments([])
onClose()
}, [onClose, cleanupManualSync])
}, [onClose])
// Handle save
const handleSave = useCallback(() => {
if (!globalSegment || !currentSegments.length) return
// Distribute the timed words back to their original segments
const finalSegments: LyricsSegment[] = []
let wordIndex = 0
currentSegments.forEach((segment) => {
const originalWordCount = segment.words.length
const segmentWords = globalSegment.words.slice(wordIndex, wordIndex + originalWordCount)
wordIndex += originalWordCount
if (segmentWords.length > 0) {
// Recalculate segment start and end times
const validStartTimes = segmentWords.map(w => w.start_time).filter((t): t is number => t !== null)
const validEndTimes = segmentWords.map(w => w.end_time).filter((t): t is number => t !== null)
const segmentStartTime = validStartTimes.length > 0 ? Math.min(...validStartTimes) : null
const segmentEndTime = validEndTimes.length > 0 ? Math.max(...validEndTimes) : null
finalSegments.push({
...segment,
words: segmentWords,
text: segmentWords.map(w => w.text).join(' '),
start_time: segmentStartTime,
end_time: segmentEndTime
})
}
})
console.log('ReplaceAllLyricsModal - Saving new segments:', {
originalSegmentCount: currentSegments.length,
finalSegmentCount: finalSegments.length,
totalWords: finalSegments.reduce((count, seg) => count + seg.words.length, 0)
})
onSave(finalSegments)
// Handle save from synchronizer
const handleSave = useCallback((segments: LyricsSegment[]) => {
onSave(segments)
handleClose()
}, [globalSegment, currentSegments, onSave, handleClose])
}, [onSave, handleClose])
// Handle reset
const handleReset = useCallback(() => {
if (!originalSegments.length) return
// Handle mode selection
const handleSelectReplace = useCallback(() => {
setMode('replace')
}, [])
console.log('ReplaceAllLyricsModal - Resetting to original state')
// Reset all words to have null timing (ready for fresh manual sync)
const resetWords = originalSegments.flatMap(segment =>
segment.words.map(word => ({
...word,
start_time: null,
end_time: null
}))
)
const audioDuration = getAudioDuration()
const resetGlobalSegment: LyricsSegment = {
id: 'global-replacement',
text: resetWords.map(w => w.text).join(' '),
words: resetWords,
start_time: 0,
end_time: Math.max(audioDuration, 3600) // At least 1 hour to prevent auto-stop
}
const handleSelectResync = useCallback(() => {
setMode('resync')
}, [])
// Also reset the current segments to have null timing
const resetCurrentSegments = originalSegments.map(segment => ({
...segment,
words: segment.words.map(word => ({
...word,
start_time: null,
end_time: null
})),
start_time: null,
end_time: null
}))
// Handle back to selection
const handleBackToSelection = useCallback(() => {
setMode('selection')
setInputText('')
setNewSegments([])
}, [])
setGlobalSegment(resetGlobalSegment)
setCurrentSegments(resetCurrentSegments)
}, [originalSegments, getAudioDuration])
// Determine which segments to use for synchronizer
const segmentsForSync = mode === 'resync' && newSegments.length > 0
? newSegments
: existingSegments
// Keep a ref to the current spacebar handler to avoid closure issues
const spacebarHandlerRef = useRef(handleSpacebar)
spacebarHandlerRef.current = handleSpacebar
// Update the spacebar handler when modal state changes
useEffect(() => {
if (open && isReplaced) {
console.log('ReplaceAllLyricsModal - Setting up spacebar handler')
const handleKeyEvent = (e: KeyboardEvent) => {
if (e.code === 'Space') {
console.log('ReplaceAllLyricsModal - Spacebar captured in modal')
e.preventDefault()
e.stopPropagation()
// Use the ref to get the current handler
spacebarHandlerRef.current(e)
}
}
setModalSpacebarHandler(() => handleKeyEvent)
return () => {
if (!open) {
console.log('ReplaceAllLyricsModal - Clearing spacebar handler')
setModalSpacebarHandler(undefined)
}
}
} else if (open) {
// Clear handler when not in replaced state
setModalSpacebarHandler(undefined)
}
}, [open, isReplaced, setModalSpacebarHandler])
// Memoize timeline range to prevent recalculation
const timeRange = useMemo(() => {
const audioDuration = getAudioDuration()
// Always use full song duration for replace-all mode
return { start: 0, end: audioDuration }
}, [getAudioDuration])
// Memoize the segment progress props to prevent unnecessary re-renders
const segmentProgressProps = useMemo(() => ({
currentSegments,
globalSegment,
syncWordIndex
}), [currentSegments, globalSegment, syncWordIndex])
// Check if we have existing lyrics
const hasExistingLyrics = existingSegments.length > 0 &&
existingSegments.some(s => s.words.length > 0)
return (
<Dialog
open={open}
onClose={handleClose}
maxWidth={false}
fullWidth={true}
onKeyDown={(e) => {
if (e.key === 'Enter' && !e.shiftKey && isReplaced) {
e.preventDefault()
handleSave()
}
}}
PaperProps={{
sx: {
height: '90vh',
margin: '5vh 2vh',
maxWidth: 'calc(100vw - 4vh)',
width: 'calc(100vw - 4vh)'
}
}}
>
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<Box sx={{ flex: 1 }}>
Replace All Lyrics
</Box>
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
<>
{/* Mode Selection Modal */}
<ModeSelectionModal
open={open && mode === 'selection'}
onClose={handleClose}
onSelectReplace={handleSelectReplace}
onSelectResync={handleSelectResync}
hasExistingLyrics={hasExistingLyrics}
/>
<DialogContent
dividers
sx={{
display: 'flex',
flexDirection: 'column',
flexGrow: 1,
overflow: 'hidden'
{/* Replace All Lyrics Modal (Paste Phase) */}
<Dialog
open={open && mode === 'replace'}
onClose={handleClose}
maxWidth="md"
fullWidth
PaperProps={{
sx: {
height: '80vh',
maxHeight: '80vh'
}
}}
>
{!isReplaced ? (
// Step 1: Input new lyrics
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<IconButton onClick={handleBackToSelection} size="small">
<ArrowBackIcon />
</IconButton>
<Box sx={{ flex: 1 }}>
Replace All Lyrics
</Box>
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
<DialogContent
dividers
sx={{
display: 'flex',
flexDirection: 'column',
flexGrow: 1,
overflow: 'hidden'
}}
>
<Box sx={{ display: 'flex', flexDirection: 'column', gap: 2, height: '100%' }}>
<Typography variant="h6" gutterBottom>
Paste your new lyrics below:
@@ -492,197 +241,96 @@ export default function ReplaceAllLyricsModal({
}
}}
/>
<Box sx={{ display: 'flex', justifyContent: 'flex-end', gap: 2 }}>
<Button
variant="contained"
onClick={processLyrics}
disabled={!inputText.trim()}
startIcon={<AutoFixHighIcon />}
>
Replace All Lyrics
</Button>
</Box>
</Box>
) : (
// Step 2: Manual sync interface
<Box sx={{ display: 'flex', flexDirection: 'column', height: '100%', gap: 2 }}>
<Paper sx={{ p: 2, bgcolor: 'background.paper' }}>
<Typography variant="h6" gutterBottom>
Lyrics Replaced Successfully
</Typography>
<Typography variant="body2" color="text.secondary">
Created {currentSegments.length} segments with {globalSegment?.words.length} words total.
Use Manual Sync to set timing for all words.
</Typography>
</Paper>
</DialogContent>
<Divider />
<DialogActions>
<Button onClick={handleClose} color="inherit">
Cancel
</Button>
<Button
variant="contained"
onClick={processLyrics}
disabled={!inputText.trim()}
startIcon={<AutoFixHighIcon />}
>
Continue to Sync
</Button>
</DialogActions>
</Dialog>
{globalSegment && (
<Box sx={{ display: 'flex', gap: 2, flexGrow: 1, minHeight: 0 }}>
{/* Timeline Section */}
<Box sx={{ flex: 2, display: 'flex', flexDirection: 'column', minHeight: 0 }}>
<EditTimelineSection
words={globalSegment.words}
startTime={timeRange.start}
endTime={timeRange.end}
originalStartTime={0}
originalEndTime={getAudioDuration()}
currentStartTime={globalSegment.start_time}
currentEndTime={globalSegment.end_time}
currentTime={currentTime}
isManualSyncing={isManualSyncing}
syncWordIndex={syncWordIndex}
isSpacebarPressed={isSpacebarPressed}
onWordUpdate={handleWordUpdate}
onUnsyncWord={handleUnsyncWord}
onPlaySegment={onPlaySegment}
onStopAudio={() => {
// Stop audio playback using global function
if (window.toggleAudioPlayback && window.isAudioPlaying) {
window.toggleAudioPlayback()
}
}}
startManualSync={startManualSync}
pauseManualSync={pauseManualSync}
resumeManualSync={resumeManualSync}
isPaused={isPaused}
isGlobal={true}
defaultZoomLevel={10} // Show 10 seconds by default
isReplaceAllMode={true} // Prevent zoom changes during sync
/>
</Box>
{/* Segment Progress Section */}
<SegmentProgressPanel
currentSegments={segmentProgressProps.currentSegments}
globalSegment={segmentProgressProps.globalSegment}
syncWordIndex={segmentProgressProps.syncWordIndex}
/>
</Box>
)}
</Box>
)}
</DialogContent>
<DialogActions>
{isReplaced && (
<EditActionBar
onReset={handleReset}
onClose={handleClose}
onSave={handleSave}
editedSegment={globalSegment}
isGlobal={true}
/>
)}
</DialogActions>
</Dialog>
)
}
// Memoized Segment Progress Item to prevent unnecessary re-renders
const SegmentProgressItem = memo(({
segment,
index,
isActive
}: {
segment: LyricsSegment
index: number
isActive: boolean
}) => {
const wordsWithTiming = segment.words.filter(w =>
w.start_time !== null && w.end_time !== null
).length
const totalWords = segment.words.length
const isComplete = wordsWithTiming === totalWords
return (
<Paper
key={segment.id}
ref={isActive ? (el) => {
// Auto-scroll to active segment
if (el) {
el.scrollIntoView({
behavior: 'smooth',
block: 'center'
})
}
} : undefined}
sx={{
p: 1,
mb: 1,
bgcolor: isActive ? 'primary.light' :
isComplete ? 'success.light' : 'background.paper',
border: isActive ? 2 : 1,
borderColor: isActive ? 'primary.main' : 'divider'
}}
>
<Typography
variant="body2"
sx={{
fontWeight: isActive ? 'bold' : 'normal',
mb: 0.5
{/* Synchronizer Modal */}
<Dialog
open={open && mode === 'resync'}
onClose={handleClose}
maxWidth={false}
fullWidth
PaperProps={{
sx: {
height: '90vh',
margin: '5vh 2vw',
maxWidth: 'calc(100vw - 4vw)',
width: 'calc(100vw - 4vw)'
}
}}
>
Segment {index + 1}: {segment.text.slice(0, 50)}
{segment.text.length > 50 ? '...' : ''}
</Typography>
<Typography variant="caption" color="text.secondary">
{wordsWithTiming}/{totalWords} words synced
{isComplete && segment.start_time !== null && segment.end_time !== null && (
<>
<br />
{segment.start_time.toFixed(2)}s - {segment.end_time.toFixed(2)}s
</>
)}
</Typography>
</Paper>
)
})
<DialogTitle sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
<IconButton onClick={handleBackToSelection} size="small">
<ArrowBackIcon />
</IconButton>
<Box sx={{ flex: 1 }}>
{newSegments.length > 0 ? 'Sync New Lyrics' : 'Re-sync Existing Lyrics'}
</Box>
<IconButton onClick={handleClose} sx={{ ml: 'auto' }}>
<CloseIcon />
</IconButton>
</DialogTitle>
// Memoized Segment Progress Panel
const SegmentProgressPanel = memo(({
currentSegments,
globalSegment,
syncWordIndex
}: {
currentSegments: LyricsSegment[]
globalSegment: LyricsSegment | null
syncWordIndex: number
}) => {
return (
<Box sx={{ flex: 1, display: 'flex', flexDirection: 'column', minHeight: 0 }}>
<Typography variant="h6" gutterBottom>
Segment Progress
</Typography>
<Box sx={{
overflow: 'auto',
flexGrow: 1,
border: 1,
borderColor: 'divider',
borderRadius: 1,
p: 1
}}>
{currentSegments.map((segment, index) => {
const isActive = Boolean(
globalSegment &&
syncWordIndex >= 0 &&
syncWordIndex < globalSegment.words.length &&
globalSegment.words[syncWordIndex] &&
segment.words.some(w => w.id === globalSegment.words[syncWordIndex].id)
)
return (
<SegmentProgressItem
key={segment.id}
segment={segment}
index={index}
isActive={isActive}
<DialogContent
dividers
sx={{
display: 'flex',
flexDirection: 'column',
flexGrow: 1,
overflow: 'hidden',
p: 2
}}
>
{segmentsForSync.length > 0 ? (
<LyricsSynchronizer
segments={segmentsForSync}
currentTime={currentTime}
onPlaySegment={onPlaySegment}
onSave={handleSave}
onCancel={handleClose}
setModalSpacebarHandler={setModalSpacebarHandler}
/>
)
})}
</Box>
</Box>
) : (
<Box sx={{
display: 'flex',
flexDirection: 'column',
alignItems: 'center',
justifyContent: 'center',
height: '100%',
gap: 2
}}>
<Typography variant="h6" color="text.secondary">
No lyrics to sync
</Typography>
<Typography variant="body2" color="text.secondary">
Go back and paste new lyrics, or close this modal.
</Typography>
<Button
variant="outlined"
onClick={handleBackToSelection}
startIcon={<ArrowBackIcon />}
>
Back to Selection
</Button>
</Box>
)}
</DialogContent>
</Dialog>
</>
)
})
}
@@ -1 +1 @@
{"root":["./src/app.tsx","./src/api.ts","./src/main.tsx","./src/theme.ts","./src/types.ts","./src/validation.ts","./src/vite-env.d.ts","./src/components/aifeedbackmodal.tsx","./src/components/addlyricsmodal.tsx","./src/components/agenticcorrectionmetrics.tsx","./src/components/audioplayer.tsx","./src/components/correctedwordwithactions.tsx","./src/components/correctionannotationmodal.tsx","./src/components/correctiondetailcard.tsx","./src/components/correctionmetrics.tsx","./src/components/durationtimelineview.tsx","./src/components/editactionbar.tsx","./src/components/editmodal.tsx","./src/components/edittimelinesection.tsx","./src/components/editwordlist.tsx","./src/components/fileupload.tsx","./src/components/findreplacemodal.tsx","./src/components/header.tsx","./src/components/lyricsanalyzer.tsx","./src/components/metricsdashboard.tsx","./src/components/modeselector.tsx","./src/components/modelselector.tsx","./src/components/previewvideosection.tsx","./src/components/referenceview.tsx","./src/components/replacealllyricsmodal.tsx","./src/components/reviewchangesmodal.tsx","./src/components/segmentdetailsmodal.tsx","./src/components/timelineeditor.tsx","./src/components/timingoffsetmodal.tsx","./src/components/transcriptionview.tsx","./src/components/worddivider.tsx","./src/components/shared/constants.ts","./src/components/shared/styles.ts","./src/components/shared/types.ts","./src/components/shared/components/highlightedtext.tsx","./src/components/shared/components/sourceselector.tsx","./src/components/shared/components/word.tsx","./src/components/shared/hooks/usewordclick.ts","./src/components/shared/utils/keyboardhandlers.ts","./src/components/shared/utils/localstorage.ts","./src/components/shared/utils/referencelinecalculator.ts","./src/components/shared/utils/segmentoperations.ts","./src/components/shared/utils/timingutils.ts","./src/components/shared/utils/wordutils.ts","./src/hooks/usemanualsync.ts","./src/types/global.d.ts"],"version":"5.6.3"}
{"root":["./src/app.tsx","./src/api.ts","./src/main.tsx","./src/theme.ts","./src/types.ts","./src/validation.ts","./src/vite-env.d.ts","./src/components/aifeedbackmodal.tsx","./src/components/addlyricsmodal.tsx","./src/components/agenticcorrectionmetrics.tsx","./src/components/audioplayer.tsx","./src/components/correctedwordwithactions.tsx","./src/components/correctionannotationmodal.tsx","./src/components/correctiondetailcard.tsx","./src/components/correctionmetrics.tsx","./src/components/durationtimelineview.tsx","./src/components/editactionbar.tsx","./src/components/editmodal.tsx","./src/components/edittimelinesection.tsx","./src/components/editwordlist.tsx","./src/components/fileupload.tsx","./src/components/findreplacemodal.tsx","./src/components/header.tsx","./src/components/lyricsanalyzer.tsx","./src/components/metricsdashboard.tsx","./src/components/modeselectionmodal.tsx","./src/components/modeselector.tsx","./src/components/modelselector.tsx","./src/components/previewvideosection.tsx","./src/components/referenceview.tsx","./src/components/replacealllyricsmodal.tsx","./src/components/reviewchangesmodal.tsx","./src/components/segmentdetailsmodal.tsx","./src/components/timelineeditor.tsx","./src/components/timingoffsetmodal.tsx","./src/components/transcriptionview.tsx","./src/components/worddivider.tsx","./src/components/lyricssynchronizer/synccontrols.tsx","./src/components/lyricssynchronizer/timelinecanvas.tsx","./src/components/lyricssynchronizer/upcomingwordsbar.tsx","./src/components/lyricssynchronizer/index.tsx","./src/components/shared/constants.ts","./src/components/shared/styles.ts","./src/components/shared/types.ts","./src/components/shared/components/highlightedtext.tsx","./src/components/shared/components/sourceselector.tsx","./src/components/shared/components/word.tsx","./src/components/shared/hooks/usewordclick.ts","./src/components/shared/utils/keyboardhandlers.ts","./src/components/shared/utils/localstorage.ts","./src/components/shared/utils/referencelinecalculator.ts","./src/components/shared/utils/segmentoperations.ts","./src/components/shared/utils/timingutils.ts","./src/components/shared/utils/wordutils.ts","./src/hooks/usemanualsync.ts","./src/types/global.d.ts"],"version":"5.6.3"}
@@ -0,0 +1 @@
{"version":3,"file":"index-COYImAcx.js","sources":["../../node_modules/react/cjs/react.production.min.js","../../node_modules/react/index.js","../../node_modules/react/cjs/react-jsx-runtime.production.min.js","../../node_modules/react/jsx-runtime.js","../../node_modules/scheduler/cjs/scheduler.production.min.js","../../node_modules/scheduler/index.js","../../node_modules/react-dom/cjs/react-dom.production.min.js","../../node_modules/react-dom/index.js","../../node_modules/react-dom/client.js","../../node_modules/@mui/utils/esm/formatMuiErrorMessage/formatMuiErrorMessage.js","../../node_modules/@mui/material/styles/identifier.js","../../node_modules/@babel/runtime/helpers/esm/extends.js","../../node_modules/@emotion/sheet/dist/emotion-sheet.esm.js","../../node_modules/stylis/src/Enum.js","../../node_modules/stylis/src/Utility.js","../../node_modules/stylis/src/Tokenizer.js","../../node_modules/stylis/src/Parser.js","../../node_modules/stylis/src/Serializer.js","../../node_modules/stylis/src/Middleware.js","../../node_modules/@emotion/memoize/dist/emotion-memoize.esm.js","../../node_modules/@emotion/cache/dist/emotion-cache.browser.esm.js","../../node_modules/react-is/cjs/react-is.production.min.js","../../node_modules/react-is/index.js","../../node_modules/hoist-non-react-statics/dist/hoist-non-react-statics.cjs.js","../../node_modules/@emotion/utils/dist/emotion-utils.browser.esm.js","../../node_modules/@emotion/hash/dist/emotion-hash.esm.js","../../node_modules/@emotion/unitless/dist/emotion-unitless.esm.js","../../node_modules/@emotion/serialize/dist/emotion-serialize.esm.js","../../node_modules/@emotion/use-insertion-effect-with-fallbacks/dist/emotion-use-insertion-effect-with-fallbacks.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-element-f0de968e.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-react.browser.esm.js","../../node_modules/@emotion/is-prop-valid/dist/emotion-is-prop-valid.esm.js","../../node_modules/@emotion/styled/base/dist/emotion-styled-base.browser.esm.js","../../node_modules/@emotion/styled/dist/emotion-styled.browser.esm.js","../../node_modules/@mui/styled-engine/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/styled-engine/index.js","../../node_modules/@mui/utils/node_modules/react-is/cjs/react-is.production.js","../../node_modules/@mui/utils/node_modules/react-is/index.js","../../node_modules/@mui/utils/esm/deepmerge/deepmerge.js","../../node_modules/@mui/system/esm/createBreakpoints/createBreakpoints.js","../../node_modules/@mui/system/esm/cssContainerQueries/cssContainerQueries.js","../../node_modules/@mui/system/esm/createTheme/shape.js","../../node_modules/@mui/system/esm/merge/merge.js","../../node_modules/@mui/system/esm/breakpoints/breakpoints.js","../../node_modules/@mui/utils/esm/capitalize/capitalize.js","../../node_modules/@mui/system/esm/style/style.js","../../node_modules/@mui/system/esm/memoize/memoize.js","../../node_modules/@mui/system/esm/spacing/spacing.js","../../node_modules/@mui/system/esm/createTheme/createSpacing.js","../../node_modules/@mui/system/esm/compose/compose.js","../../node_modules/@mui/system/esm/borders/borders.js","../../node_modules/@mui/system/esm/cssGrid/cssGrid.js","../../node_modules/@mui/system/esm/palette/palette.js","../../node_modules/@mui/system/esm/sizing/sizing.js","../../node_modules/@mui/system/esm/styleFunctionSx/defaultSxConfig.js","../../node_modules/@mui/system/esm/styleFunctionSx/styleFunctionSx.js","../../node_modules/@mui/system/esm/createTheme/applyStyles.js","../../node_modules/@mui/system/esm/createTheme/createTheme.js","../../node_modules/@mui/system/esm/useThemeWithoutDefault/useThemeWithoutDefault.js","../../node_modules/@mui/system/esm/useTheme/useTheme.js","../../node_modules/@mui/system/esm/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/system/esm/styleFunctionSx/extendSxProp.js","../../node_modules/@mui/utils/esm/ClassNameGenerator/ClassNameGenerator.js","../../node_modules/clsx/dist/clsx.mjs","../../node_modules/@mui/system/esm/createBox/createBox.js","../../node_modules/@mui/utils/esm/generateUtilityClass/generateUtilityClass.js","../../node_modules/@mui/utils/esm/generateUtilityClasses/generateUtilityClasses.js","../../node_modules/@mui/system/esm/preprocessStyles.js","../../node_modules/@mui/system/esm/createStyled/createStyled.js","../../node_modules/@mui/system/esm/styled/styled.js","../../node_modules/@mui/utils/esm/resolveProps/resolveProps.js","../../node_modules/@mui/system/esm/useThemeProps/getThemeProps.js","../../node_modules/@mui/system/esm/useThemeProps/useThemeProps.js","../../node_modules/@mui/utils/esm/useEnhancedEffect/useEnhancedEffect.js","../../node_modules/@mui/system/esm/useMediaQuery/useMediaQuery.js","../../node_modules/@mui/utils/esm/clamp/clamp.js","../../node_modules/@mui/system/esm/colorManipulator/colorManipulator.js","../../node_modules/@mui/utils/esm/createChainedFunction/createChainedFunction.js","../../node_modules/@mui/utils/esm/debLine truncated
@@ -1 +0,0 @@
{"version":3,"file":"index-DdJTDWH3.js","sources":["../../node_modules/react/cjs/react.production.min.js","../../node_modules/react/index.js","../../node_modules/react/cjs/react-jsx-runtime.production.min.js","../../node_modules/react/jsx-runtime.js","../../node_modules/scheduler/cjs/scheduler.production.min.js","../../node_modules/scheduler/index.js","../../node_modules/react-dom/cjs/react-dom.production.min.js","../../node_modules/react-dom/index.js","../../node_modules/react-dom/client.js","../../node_modules/@mui/utils/esm/formatMuiErrorMessage/formatMuiErrorMessage.js","../../node_modules/@mui/material/styles/identifier.js","../../node_modules/@babel/runtime/helpers/esm/extends.js","../../node_modules/@emotion/sheet/dist/emotion-sheet.esm.js","../../node_modules/stylis/src/Enum.js","../../node_modules/stylis/src/Utility.js","../../node_modules/stylis/src/Tokenizer.js","../../node_modules/stylis/src/Parser.js","../../node_modules/stylis/src/Serializer.js","../../node_modules/stylis/src/Middleware.js","../../node_modules/@emotion/memoize/dist/emotion-memoize.esm.js","../../node_modules/@emotion/cache/dist/emotion-cache.browser.esm.js","../../node_modules/react-is/cjs/react-is.production.min.js","../../node_modules/react-is/index.js","../../node_modules/hoist-non-react-statics/dist/hoist-non-react-statics.cjs.js","../../node_modules/@emotion/utils/dist/emotion-utils.browser.esm.js","../../node_modules/@emotion/hash/dist/emotion-hash.esm.js","../../node_modules/@emotion/unitless/dist/emotion-unitless.esm.js","../../node_modules/@emotion/serialize/dist/emotion-serialize.esm.js","../../node_modules/@emotion/use-insertion-effect-with-fallbacks/dist/emotion-use-insertion-effect-with-fallbacks.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-element-f0de968e.browser.esm.js","../../node_modules/@emotion/react/dist/emotion-react.browser.esm.js","../../node_modules/@emotion/is-prop-valid/dist/emotion-is-prop-valid.esm.js","../../node_modules/@emotion/styled/base/dist/emotion-styled-base.browser.esm.js","../../node_modules/@emotion/styled/dist/emotion-styled.browser.esm.js","../../node_modules/@mui/styled-engine/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/styled-engine/index.js","../../node_modules/@mui/utils/node_modules/react-is/cjs/react-is.production.js","../../node_modules/@mui/utils/node_modules/react-is/index.js","../../node_modules/@mui/utils/esm/deepmerge/deepmerge.js","../../node_modules/@mui/system/esm/createBreakpoints/createBreakpoints.js","../../node_modules/@mui/system/esm/cssContainerQueries/cssContainerQueries.js","../../node_modules/@mui/system/esm/createTheme/shape.js","../../node_modules/@mui/system/esm/merge/merge.js","../../node_modules/@mui/system/esm/breakpoints/breakpoints.js","../../node_modules/@mui/utils/esm/capitalize/capitalize.js","../../node_modules/@mui/system/esm/style/style.js","../../node_modules/@mui/system/esm/memoize/memoize.js","../../node_modules/@mui/system/esm/spacing/spacing.js","../../node_modules/@mui/system/esm/createTheme/createSpacing.js","../../node_modules/@mui/system/esm/compose/compose.js","../../node_modules/@mui/system/esm/borders/borders.js","../../node_modules/@mui/system/esm/cssGrid/cssGrid.js","../../node_modules/@mui/system/esm/palette/palette.js","../../node_modules/@mui/system/esm/sizing/sizing.js","../../node_modules/@mui/system/esm/styleFunctionSx/defaultSxConfig.js","../../node_modules/@mui/system/esm/styleFunctionSx/styleFunctionSx.js","../../node_modules/@mui/system/esm/createTheme/applyStyles.js","../../node_modules/@mui/system/esm/createTheme/createTheme.js","../../node_modules/@mui/system/esm/useThemeWithoutDefault/useThemeWithoutDefault.js","../../node_modules/@mui/system/esm/useTheme/useTheme.js","../../node_modules/@mui/system/esm/GlobalStyles/GlobalStyles.js","../../node_modules/@mui/system/esm/styleFunctionSx/extendSxProp.js","../../node_modules/@mui/utils/esm/ClassNameGenerator/ClassNameGenerator.js","../../node_modules/clsx/dist/clsx.mjs","../../node_modules/@mui/system/esm/createBox/createBox.js","../../node_modules/@mui/utils/esm/generateUtilityClass/generateUtilityClass.js","../../node_modules/@mui/utils/esm/generateUtilityClasses/generateUtilityClasses.js","../../node_modules/@mui/system/esm/preprocessStyles.js","../../node_modules/@mui/system/esm/createStyled/createStyled.js","../../node_modules/@mui/system/esm/styled/styled.js","../../node_modules/@mui/utils/esm/resolveProps/resolveProps.js","../../node_modules/@mui/system/esm/useThemeProps/getThemeProps.js","../../node_modules/@mui/system/esm/useThemeProps/useThemeProps.js","../../node_modules/@mui/utils/esm/useEnhancedEffect/useEnhancedEffect.js","../../node_modules/@mui/system/esm/useMediaQuery/useMediaQuery.js","../../node_modules/@mui/utils/esm/clamp/clamp.js","../../node_modules/@mui/system/esm/colorManipulator/colorManipulator.js","../../node_modules/@mui/utils/esm/createChainedFunction/createChainedFunction.js","../../node_modules/@mui/utils/esm/debLine truncated
@@ -10,7 +10,7 @@
<link rel="icon" type="image/png" sizes="512x512" href="/android-chrome-512x512.png" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Nomad Karaoke: Lyrics Review</title>
<script type="module" crossorigin src="/assets/index-DdJTDWH3.js"></script>
<script type="module" crossorigin src="/assets/index-COYImAcx.js"></script>
</head>
<body>
<div id="root"></div>
@@ -629,14 +629,14 @@ class ReviewServer:
else ""
)
# Use hosted review UI by default, can be overridden with LYRICS_REVIEW_UI_URL env var
# Set to "local" to use the bundled local frontend instead
review_ui_url = os.environ.get("LYRICS_REVIEW_UI_URL", "https://lyrics.nomadkaraoke.com")
# Use bundled local frontend by default for local karaoke-gen runs
# Can override with LYRICS_REVIEW_UI_URL env var (e.g., http://localhost:5173 for Vite dev)
review_ui_url = os.environ.get("LYRICS_REVIEW_UI_URL", "local")
if review_ui_url.lower() == "local":
# Use the bundled local frontend
# Use the bundled local frontend served by this FastAPI server
browser_url = f"http://localhost:8000?baseApiUrl={encoded_api_url}{audio_hash_param}"
else:
# Use the hosted/external review UI
# Use an external review UI (dev server or hosted)
browser_url = f"{review_ui_url}?baseApiUrl={encoded_api_url}{audio_hash_param}"
self.logger.info(f"Opening review UI: {browser_url}")
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.71.64"
version = "0.72.0"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"
+16 -6
View File
@@ -17,6 +17,7 @@ from karaoke_gen.audio_fetcher import (
AudioFetcherError,
NoResultsError,
DownloadError,
UserCancelledError,
FlacFetchAudioFetcher,
create_audio_fetcher,
)
@@ -120,6 +121,15 @@ class TestExceptions:
raise DownloadError("Download failed")
assert str(exc_info.value) == "Download failed"
def test_user_cancelled_error(self):
"""Test UserCancelledError inherits from AudioFetcherError."""
with pytest.raises(AudioFetcherError):
raise UserCancelledError("User cancelled")
with pytest.raises(UserCancelledError) as exc_info:
raise UserCancelledError("User cancelled")
assert str(exc_info.value) == "User cancelled"
class TestFlacFetchAudioFetcher:
"""Tests for FlacFetchAudioFetcher implementation."""
@@ -353,7 +363,7 @@ class TestFlacFetchAudioFetcher:
@patch('builtins.input', return_value='0')
def test_interactive_select_cancel(self, mock_input, fetcher):
"""Test interactive selection returns None when user cancels."""
"""Test interactive selection raises UserCancelledError when user cancels."""
# Use correct flacfetch Release attribute names for fallback display
mock_result = MagicMock()
mock_result.source_name = "YouTube"
@@ -364,8 +374,8 @@ class TestFlacFetchAudioFetcher:
mock_result.seeders = None
mock_result.target_file = None
result = fetcher._interactive_select([mock_result], "Artist", "Test")
assert result is None
with pytest.raises(UserCancelledError):
fetcher._interactive_select([mock_result], "Artist", "Test")
@patch('builtins.input', side_effect=['invalid', '1'])
def test_interactive_select_invalid_then_valid(self, mock_input, fetcher):
@@ -402,7 +412,7 @@ class TestFlacFetchAudioFetcher:
@patch('builtins.input', side_effect=KeyboardInterrupt)
def test_interactive_select_keyboard_interrupt(self, mock_input, fetcher, capsys):
"""Test interactive selection handles keyboard interrupt."""
"""Test interactive selection raises UserCancelledError on keyboard interrupt."""
mock_result = MagicMock()
mock_result.source_name = "YouTube"
mock_result.title = "Test"
@@ -412,8 +422,8 @@ class TestFlacFetchAudioFetcher:
mock_result.seeders = None
mock_result.target_file = None
result = fetcher._interactive_select([mock_result], "Artist", "Test")
assert result is None
with pytest.raises(UserCancelledError):
fetcher._interactive_select([mock_result], "Artist", "Test")
captured = capsys.readouterr()
assert "Cancelled" in captured.out
+260
View File
@@ -435,6 +435,266 @@ class TestExistingInstrumentalPadding:
assert '3.0' in call_args or 3.0 in call_args
class TestDetectCountdownPaddingFromLRC:
"""Tests for detecting countdown padding from existing LRC files."""
def test_detect_padding_from_lrc_with_padding(self, basic_karaoke_gen, temp_dir):
"""Test that countdown padding is detected when first lyric is after 3 seconds."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at 3 seconds (padding was applied)
lrc_content = """[ti:Test Song]
[ar:Test Artist]
[00:03.50]First line of lyrics
[00:06.20]Second line of lyrics
[00:10.00]Third line of lyrics
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is True
assert padding_seconds == 3.0
def test_detect_padding_from_lrc_without_padding(self, basic_karaoke_gen, temp_dir):
"""Test that no padding is detected when first lyric is before 3 seconds."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at 0.5 seconds (no padding)
lrc_content = """[ti:Test Song]
[ar:Test Artist]
[00:00.50]First line of lyrics
[00:03.20]Second line of lyrics
[00:06.00]Third line of lyrics
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_edge_case(self, basic_karaoke_gen, temp_dir):
"""Test detection at the 2.5 second boundary."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at exactly 2.5 seconds (boundary)
lrc_content = """[00:02.50]First line at boundary
[00:05.00]Second line
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
# 2.5 is the boundary - should be detected as padding
assert padding_added is True
assert padding_seconds == 3.0
def test_detect_padding_from_lrc_just_below_boundary(self, basic_karaoke_gen, temp_dir):
"""Test detection just below the 2.5 second boundary."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at 2.4 seconds (just below boundary)
lrc_content = """[00:02.40]First line just below boundary
[00:05.00]Second line
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
# 2.4 is below boundary - should NOT be detected as padding
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_empty_file(self, basic_karaoke_gen, temp_dir):
"""Test handling of empty LRC file."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
with open(lrc_file, 'w') as f:
f.write('')
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_no_timestamps(self, basic_karaoke_gen, temp_dir):
"""Test handling of LRC file with no valid timestamps."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
lrc_content = """[ti:Test Song]
[ar:Test Artist]
Just text without timestamps
More text
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_missing_file(self, basic_karaoke_gen, temp_dir):
"""Test handling of missing LRC file."""
lrc_file = os.path.join(temp_dir, 'nonexistent.lrc')
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_three_digit_milliseconds(self, basic_karaoke_gen, temp_dir):
"""Test parsing LRC files with three-digit milliseconds."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Some LRC files use [mm:ss.xxx] format
lrc_content = """[00:03.500]First line at 3.5 seconds
[00:06.200]Second line
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is True
assert padding_seconds == 3.0
class TestExistingFilesWithPaddingDetection:
"""Tests for transcribe_lyrics returning padding info from existing files."""
def test_transcribe_lyrics_existing_files_returns_padding_info(self, basic_karaoke_gen, temp_dir):
"""Test that transcribe_lyrics returns padding info when using existing files."""
track_output_dir = temp_dir
artist = "Test Artist"
title = "Test Title"
# Create existing files
video_path = os.path.join(track_output_dir, f"{artist} - {title} (With Vocals).mkv")
lrc_path = os.path.join(track_output_dir, f"{artist} - {title} (Karaoke).lrc")
# Create the video file
with open(video_path, 'w') as f:
f.write('mock video')
# Create LRC file with countdown padding (first lyric at 3.5s)
lrc_content = """[00:03.50]First line with padding
[00:06.00]Second line
"""
with open(lrc_path, 'w') as f:
f.write(lrc_content)
# Call transcribe_lyrics - it should detect existing files and return padding info
result = basic_karaoke_gen.lyrics_processor.transcribe_lyrics(
input_audio_wav=None, # Not needed when existing files found
artist=artist,
title=title,
track_output_dir=track_output_dir,
)
# Should return padding info from LRC detection
assert result['countdown_padding_added'] is True
assert result['countdown_padding_seconds'] == 3.0
assert result['padded_audio_filepath'] is None # Original padded audio not available
def test_transcribe_lyrics_existing_files_no_padding(self, basic_karaoke_gen, temp_dir):
"""Test existing files case when there's no countdown padding."""
track_output_dir = temp_dir
artist = "Test Artist"
title = "Test Title"
# Create existing files
video_path = os.path.join(track_output_dir, f"{artist} - {title} (With Vocals).mkv")
lrc_path = os.path.join(track_output_dir, f"{artist} - {title} (Karaoke).lrc")
# Create the video file
with open(video_path, 'w') as f:
f.write('mock video')
# Create LRC file WITHOUT countdown padding (first lyric at 0.5s)
lrc_content = """[00:00.50]First line starts immediately
[00:03.00]Second line
"""
with open(lrc_path, 'w') as f:
f.write(lrc_content)
result = basic_karaoke_gen.lyrics_processor.transcribe_lyrics(
input_audio_wav=None,
artist=artist,
title=title,
track_output_dir=track_output_dir,
)
# Should return no padding info
assert result['countdown_padding_added'] is False
assert result['countdown_padding_seconds'] == 0.0
class TestScanDirectoryForInstrumentals:
"""Tests for scanning directory for existing instrumentals."""
def test_scan_directory_finds_clean_instrumental(self, basic_karaoke_gen, temp_dir):
"""Test that _scan_directory_for_instrumentals finds clean instrumentals."""
artist_title = "Test Artist - Test Song"
# Create a clean instrumental file
instrumental_file = os.path.join(temp_dir, f"{artist_title} (Instrumental Clean).flac")
with open(instrumental_file, 'w') as f:
f.write('mock instrumental')
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
assert result['clean_instrumental']['instrumental'] == instrumental_file
def test_scan_directory_finds_combined_instrumentals(self, basic_karaoke_gen, temp_dir):
"""Test that _scan_directory_for_instrumentals finds combined instrumentals."""
artist_title = "Test Artist - Test Song"
# Create combined instrumental files
bv_file = os.path.join(temp_dir, f"{artist_title} (Instrumental +BV HTDemucs).flac")
with open(bv_file, 'w') as f:
f.write('mock instrumental with backing')
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
assert 'HTDemucs' in result['combined_instrumentals']
assert result['combined_instrumentals']['HTDemucs'] == bv_file
def test_scan_directory_skips_padded_files(self, basic_karaoke_gen, temp_dir):
"""Test that _scan_directory_for_instrumentals skips already-padded files."""
artist_title = "Test Artist - Test Song"
# Create both original and padded files
original_file = os.path.join(temp_dir, f"{artist_title} (Instrumental Clean).flac")
padded_file = os.path.join(temp_dir, f"{artist_title} (Instrumental Clean) (Padded).flac")
with open(original_file, 'w') as f:
f.write('mock original')
with open(padded_file, 'w') as f:
f.write('mock padded')
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
# Should return original, not padded
assert result['clean_instrumental']['instrumental'] == original_file
assert 'Padded' not in result['clean_instrumental']['instrumental']
def test_scan_directory_empty_directory(self, basic_karaoke_gen, temp_dir):
"""Test _scan_directory_for_instrumentals on empty directory."""
artist_title = "Test Artist - Test Song"
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
# Should return empty structure
assert result['clean_instrumental'].get('instrumental') is None
assert len(result['combined_instrumentals']) == 0
if __name__ == '__main__':
pytest.main([__file__, '-v'])
@@ -299,7 +299,7 @@ def test_check_if_video_title_exists_found_confirm(mock_input, mock_fuzz_ratio,
found_video_id = "existing_video_id"
found_video_title = f"{ARTIST} - {TITLE} (Karaoke) Official"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 85 # High similarity
@@ -322,7 +322,7 @@ def test_check_if_video_title_exists_found_reject(mock_input, mock_fuzz_ratio, m
found_video_id = "existing_video_id"
found_video_title = f"{ARTIST} - {TITLE} (Karaoke) Official"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 85
@@ -344,7 +344,7 @@ def test_check_if_video_title_exists_low_similarity(mock_fuzz_ratio, mock_auth,
found_video_id = "existing_video_id"
found_video_title = "Completely Different Title"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 30 # Low similarity
@@ -367,7 +367,7 @@ def test_check_if_video_title_exists_non_interactive(mock_input, mock_fuzz_ratio
found_video_id = "existing_video_id"
found_video_title = f"{ARTIST} - {TITLE} (Karaoke) Official"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 85
@@ -392,6 +392,63 @@ def test_check_if_video_title_exists_not_found(mock_auth, finaliser_for_yt, mock
assert finaliser_for_yt.youtube_url is None # Check initial state remains None
assert finaliser_for_yt.skip_notifications is False
@patch.object(KaraokeFinalise, 'authenticate_youtube')
@patch('thefuzz.fuzz.ratio')
@patch('builtins.input')
def test_check_if_video_title_exists_filters_other_channels(mock_input, mock_fuzz_ratio, mock_auth, finaliser_for_yt, mock_youtube_service):
"""Test that videos from other channels are filtered out even if returned by search API.
The YouTube search API sometimes returns results from other channels even when
channelId parameter is specified. This test verifies we filter those out.
"""
mock_auth.return_value = mock_youtube_service
youtube_title = f"{ARTIST} - {TITLE} (Karaoke)"
# Simulate YouTube API bug: returns videos from other channels despite channelId filter
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [
# Video from another channel - should be skipped
{"id": {"videoId": "other_channel_video_1"}, "snippet": {"title": f"{ARTIST} - {TITLE}", "channelId": "UC_other_channel_1"}},
{"id": {"videoId": "other_channel_video_2"}, "snippet": {"title": f"{ARTIST} - {TITLE} (Official)", "channelId": "UC_other_channel_2"}},
# Video from user's channel - should be matched
{"id": {"videoId": "my_channel_video"}, "snippet": {"title": f"{ARTIST} - {TITLE} (Karaoke)", "channelId": "UC_test_channel_id"}},
]
}
mock_fuzz_ratio.return_value = 95
mock_input.return_value = 'y'
exists = finaliser_for_yt.check_if_video_title_exists_on_youtube_channel(youtube_title)
assert exists is True
# Should match the video from our channel, not the ones from other channels
assert finaliser_for_yt.youtube_video_id == "my_channel_video"
# fuzz.ratio should only be called for videos from our channel (1 video)
assert mock_fuzz_ratio.call_count == 1
@patch.object(KaraokeFinalise, 'authenticate_youtube')
@patch('thefuzz.fuzz.ratio')
def test_check_if_video_title_exists_all_from_other_channels(mock_fuzz_ratio, mock_auth, finaliser_for_yt, mock_youtube_service):
"""Test that if all returned videos are from other channels, no match is found."""
mock_auth.return_value = mock_youtube_service
youtube_title = f"{ARTIST} - {TITLE} (Karaoke)"
# All results are from other channels
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [
{"id": {"videoId": "other_channel_video_1"}, "snippet": {"title": f"{ARTIST} - {TITLE}", "channelId": "UC_other_channel_1"}},
{"id": {"videoId": "other_channel_video_2"}, "snippet": {"title": f"{ARTIST} - {TITLE} (Official)", "channelId": "UC_other_channel_2"}},
]
}
exists = finaliser_for_yt.check_if_video_title_exists_on_youtube_channel(youtube_title)
assert exists is False
# fuzz.ratio should never be called since all videos are filtered out
mock_fuzz_ratio.assert_not_called()
# Should log debug messages about skipping videos from other channels
assert finaliser_for_yt.logger.debug.call_count >= 2
# --- Delete / Upload Tests ---
@patch.object(KaraokeFinalise, 'authenticate_youtube')
+5 -2
View File
@@ -74,6 +74,7 @@ class TestJobStatus:
assert JobStatus.UPLOADING == "uploading"
assert JobStatus.NOTIFYING == "notifying"
assert JobStatus.COMPLETE == "complete"
assert JobStatus.PREP_COMPLETE == "prep_complete" # Batch 6: Prep-only jobs stop here
assert JobStatus.FAILED == "failed"
assert JobStatus.CANCELLED == "cancelled"
assert JobStatus.ERROR == "error"
@@ -963,10 +964,10 @@ class TestMain:
@patch('karaoke_gen.utils.remote_cli.create_parser')
@patch('karaoke_gen.utils.remote_cli.get_auth_token')
@patch('karaoke_gen.utils.remote_cli.check_prerequisites')
def test_main_finalise_only_not_supported(
def test_main_finalise_only_requires_folder_and_args(
self, mock_check, mock_auth, mock_create_parser
):
"""Test main rejects --finalise-only in remote mode."""
"""Test main --finalise-only requires folder and artist/title args (Batch 6)."""
mock_parser = MagicMock()
mock_args = MagicMock()
mock_args.service_url = "https://api.example.com"
@@ -984,12 +985,14 @@ class TestMain:
mock_args.filter_client_id = None
mock_args.environment = ""
mock_args.client_id = ""
mock_args.args = [] # No args provided - should fail
mock_parser.parse_args.return_value = mock_args
mock_create_parser.return_value = mock_parser
mock_auth.return_value = "token"
result = main()
# Should fail because no folder/artist/title provided
assert result == 1
@patch('karaoke_gen.utils.remote_cli.create_parser')