Compare commits

...
Author SHA1 Message Date
Andrew Beveridge f99d6c1fb4 style: remove extraneous f-string prefixes
Address CodeRabbit nitpick comments - remove f-string prefixes from
strings without placeholders.
2025-12-19 18:42:03 -05:00
Andrew Beveridge f8a45bfbd2 fix: countdown padding synchronization for instrumental audio
When LyricsTranscriber adds 3-second countdown padding to vocals,
the instrumental audio must be padded to match. This was broken when
transcription was skipped due to existing files.

Changes:
- Add _detect_countdown_padding_from_lrc() to detect padding from existing LRC
- Update early returns in transcribe_lyrics() to include padding info
- Add _scan_directory_for_instrumentals() to find existing instrumentals
- Fix counting bug in apply_countdown_padding_to_instrumentals
- Add countdown_padding_seconds safety net to KaraokeFinalise
- Store padding metadata in lyrics_worker job state_data
- Pass padding info to video_worker for instrumental sync
- Add comprehensive unit tests for new functionality
2025-12-19 18:35:37 -05:00
Andrew Beveridge ☄️ f524796f7f Merge pull request #14 from nomadkaraoke/feature/batch-6-two-phase-workflow
feat: Batch 6 - Two-Phase Workflow (prep-only, finalise-only, keep-brand-code)
2025-12-19 00:07:15 -05:00
Andrew Beveridge ☄️ 8372ed211a Merge pull request #34 from nomadkaraoke/fix/youtube-duplicate-checker-channel-filter
Fix YouTube duplicate checker returning videos from other channels
2025-12-19 00:06:11 -05:00
Andrew Beveridge 78068ea1d8 Fix YouTube duplicate checker returning videos from other channels
The YouTube Data API v3 search endpoint's channelId parameter doesn't
always filter correctly and sometimes returns videos from other channels.

This fix verifies each search result's snippet.channelId matches the
user's channel ID before considering it as a potential match, preventing
false positives from unrelated videos with similar titles.

Added tests for the channel filtering behavior.
2025-12-18 23:28:23 -05:00
Andrew Beveridge 84d7c12f13 fix: Address CodeRabbit review comments for Batch 6 PR
- Add PREP_COMPLETE to RENDERING_VIDEO state transitions (job.py)
- Simplify instrumental detection logic for consistency (remote_cli.py)
- Add shell escaping (shlex.quote) for continuation command display (remote_cli.py)
- Update test for --finalise-only now that it's implemented (test_remote_cli.py)
- Add test for PREP_COMPLETE status (test_remote_cli.py)
- Bump version to 0.71.69
2025-12-18 23:22:05 -05:00
Andrew Beveridge 7a4ed9661c Merge origin/main into feature/batch-6-two-phase-workflow 2025-12-18 22:42:49 -05:00
Andrew Beveridge ☄️ dd62c32026 Merge pull request #33 from nomadkaraoke/fix/uploaded-instrumental-selection
Fix: Handle 'uploaded' selection in instrumental review CLI
2025-12-18 21:35:28 -05:00
Andrew Beveridge c130fa8a07 Merge remote-tracking branch 'origin/main' into fix/uploaded-instrumental-selection 2025-12-18 20:24:14 -05:00
Andrew Beveridge ☄️ e0d1fa7536 Merge pull request #32 from nomadkaraoke/replicate-midico-synchronizer
feat: Add MidiCo-style lyrics synchronizer for re-syncing existing lyrics
2025-12-18 20:23:47 -05:00
Andrew Beveridge 29ffbd4ae2 fix: handle 'uploaded' selection in instrumental review CLI
The instrumental review UI supports uploading custom instrumental files,
but gen_cli.py was missing the handler for the 'uploaded' selection type.

When a user uploaded a custom instrumental and selected it, the CLI would
log 'Unknown selection: uploaded' and crash with an unhelpful error message.

This fix adds proper handling for the 'uploaded' case, retrieving the
uploaded file path from the server just like the 'custom' case does.
2025-12-18 20:21:00 -05:00
Andrew Beveridge 36a60c09da Merge origin/main into feature/batch-6-two-phase-workflow
Resolved conflicts to merge Batch 6 (Two-Phase Workflow) with:
- Batch 5 (FlacFetch Audio Search) - Added SEARCHING_AUDIO states
- Instrumental review UI fixes
- Worker logging context improvements

Key merge resolutions:
- job.py: Combined state transitions for SEARCHING_AUDIO and AWAITING_INSTRUMENTAL_SELECTION
- remote_cli.py: Added prep_only/keep_brand_code to both URL and file submission paths
- render_video_worker.py: Added job_logging_context while preserving prep_only transition logic
- file_upload.py: Re-added all finalise-only endpoints and models

All 180 tests pass.
2025-12-14 13:15:10 -05:00
Andrew Beveridge ba4534fdbd feat: Batch 6 - Two-Phase Workflow (prep-only, finalise-only, keep-brand-code)
Implements the two-phase workflow for remote CLI, allowing users to:
- Run prep-only mode to stop after review phase and download all outputs
- Run finalise-only mode to upload local prep outputs and continue with finalization
- Preserve existing brand codes when re-running finalisation

Backend changes:
- Added PREP_COMPLETE status to JobStatus enum
- Added prep_only, finalise_only, keep_brand_code fields to Job/JobCreate models
- Updated state transitions to allow new workflow paths
- New endpoint /api/jobs/create-finalise-only for uploading prep outputs
- New endpoint /api/jobs/{job_id}/finalise-uploads-complete
- Modified render_video_worker to transition to PREP_COMPLETE for prep-only jobs
- Modified video_worker to use preserved brand code when set

Remote CLI changes:
- Added --prep-only flag support (passes prep_only=true to backend)
- Added --finalise-only flow (detects local prep folder, uploads files)
- Added --keep-brand-code support (extracts from folder name)
- Added PREP_COMPLETE status handling in job monitor

Tests:
- Added comprehensive tests for new Job model fields and PREP_COMPLETE status
- Added tests for finalise-only endpoints and Pydantic models
- All 151 tests pass

Version bumped to 0.71.27
2025-12-11 02:38:22 -05:00
19 changed files with 2047 additions and 52 deletions

No files matched your search

+476
View File
@@ -131,6 +131,10 @@ class CreateJobWithUploadUrlsRequest(BaseModel):
# Existing instrumental configuration (Batch 3)
existing_instrumental: bool = Field(False, description="Whether an existing instrumental file is being uploaded")
# Two-phase workflow configuration (Batch 6)
prep_only: bool = Field(False, description="Stop after review phase, don't run finalisation")
keep_brand_code: Optional[str] = Field(None, description="Preserve existing brand code instead of generating new one")
class SignedUploadUrl(BaseModel):
@@ -638,6 +642,56 @@ VALID_FILE_TYPES = {
'existing_instrumental': ALLOWED_AUDIO_EXTENSIONS, # Batch 3: user-provided instrumental
}
# Valid file types for finalise-only mode (Batch 6)
ALLOWED_VIDEO_EXTENSIONS = {'.mkv', '.mov', '.mp4'}
FINALISE_ONLY_FILE_TYPES = {
'audio': ALLOWED_AUDIO_EXTENSIONS, # Original audio
'with_vocals': ALLOWED_VIDEO_EXTENSIONS, # Karaoke video from prep
'title_screen': ALLOWED_VIDEO_EXTENSIONS, # Title screen video
'end_screen': ALLOWED_VIDEO_EXTENSIONS, # End screen video
'title_jpg': ALLOWED_IMAGE_EXTENSIONS, # Title screen JPG
'title_png': ALLOWED_IMAGE_EXTENSIONS, # Title screen PNG
'end_jpg': ALLOWED_IMAGE_EXTENSIONS, # End screen JPG
'end_png': ALLOWED_IMAGE_EXTENSIONS, # End screen PNG
'instrumental_clean': ALLOWED_AUDIO_EXTENSIONS, # Clean instrumental
'instrumental_backing': ALLOWED_AUDIO_EXTENSIONS, # Instrumental with backing vocals
'lrc': {'.lrc'}, # LRC lyrics
'ass': {'.ass'}, # ASS subtitles
'corrections': {'.json'}, # Corrections JSON
'style_params': {'.json'}, # Style params
}
# Pydantic models for finalise-only flow (Batch 6)
class FinaliseOnlyFileRequest(BaseModel):
"""Information about a prep output file to be uploaded for finalise-only mode."""
filename: str = Field(..., description="Original filename with extension")
content_type: str = Field(..., description="MIME type of the file")
file_type: str = Field(..., description="Type of prep file: 'with_vocals', 'title_screen', 'end_screen', 'instrumental_clean', 'instrumental_backing', 'lrc', etc.")
class CreateFinaliseOnlyJobRequest(BaseModel):
"""Request to create a finalise-only job with prep output files."""
artist: str = Field(..., description="Artist name")
title: str = Field(..., description="Song title")
files: List[FinaliseOnlyFileRequest] = Field(..., description="List of prep output files to upload")
# Processing options
enable_cdg: bool = Field(True, description="Generate CDG+MP3 package")
enable_txt: bool = Field(True, description="Generate TXT+MP3 package")
# Finalisation options
brand_prefix: Optional[str] = Field(None, description="Brand code prefix (e.g., NOMAD)")
keep_brand_code: Optional[str] = Field(None, description="Preserve existing brand code from folder name")
enable_youtube_upload: bool = Field(False, description="Upload to YouTube")
youtube_description: Optional[str] = Field(None, description="YouTube video description text")
discord_webhook_url: Optional[str] = Field(None, description="Discord webhook URL for notifications")
# Distribution options
dropbox_path: Optional[str] = Field(None, description="Dropbox folder path for organized output")
gdrive_folder_id: Optional[str] = Field(None, description="Google Drive folder ID for public share uploads")
async def _validate_audio_durations(
storage: StorageService,
@@ -1243,3 +1297,425 @@ async def create_job_from_url(
except Exception as e:
logger.error(f"Error creating job from URL: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e)) from e
# ============================================================================
# Finalise-Only Upload Flow (Batch 6)
# ============================================================================
def _get_gcs_path_for_finalise_file(job_id: str, file_type: str, filename: str) -> str:
"""Generate the GCS path for a finalise-only file based on its type."""
ext = Path(filename).suffix.lower()
# Map file types to appropriate GCS paths
if file_type == 'audio':
return f"uploads/{job_id}/audio/{filename}"
elif file_type == 'with_vocals':
return f"jobs/{job_id}/videos/with_vocals{ext}"
elif file_type == 'title_screen':
return f"jobs/{job_id}/screens/title{ext}"
elif file_type == 'end_screen':
return f"jobs/{job_id}/screens/end{ext}"
elif file_type == 'title_jpg':
return f"jobs/{job_id}/screens/title.jpg"
elif file_type == 'title_png':
return f"jobs/{job_id}/screens/title.png"
elif file_type == 'end_jpg':
return f"jobs/{job_id}/screens/end.jpg"
elif file_type == 'end_png':
return f"jobs/{job_id}/screens/end.png"
elif file_type == 'instrumental_clean':
return f"jobs/{job_id}/stems/instrumental_clean{ext}"
elif file_type == 'instrumental_backing':
return f"jobs/{job_id}/stems/instrumental_with_backing{ext}"
elif file_type == 'lrc':
return f"jobs/{job_id}/lyrics/karaoke.lrc"
elif file_type == 'ass':
return f"jobs/{job_id}/lyrics/karaoke.ass"
elif file_type == 'corrections':
return f"jobs/{job_id}/lyrics/corrections.json"
elif file_type == 'style_params':
return f"uploads/{job_id}/style/style_params.json"
else:
return f"uploads/{job_id}/other/{filename}"
@router.post("/jobs/create-finalise-only")
async def create_finalise_only_job(
request: Request,
body: CreateFinaliseOnlyJobRequest,
):
"""
Create a finalise-only job for continuing from a local prep phase.
This endpoint is used when:
1. User previously ran --prep-only which created local prep outputs
2. User optionally made manual edits to stems, lyrics, etc.
3. User now wants cloud to handle finalisation (encoding, distribution)
Required files:
- with_vocals: The karaoke video from prep (*.mkv or *.mov)
- title_screen: Title screen video
- end_screen: End screen video
- instrumental_clean or instrumental_backing: At least one instrumental
The endpoint returns signed URLs for uploading all the prep files.
"""
try:
# Validate files list
if not body.files:
raise HTTPException(status_code=400, detail="At least one file is required")
# Validate file types
file_types = {f.file_type for f in body.files}
# Check required files for finalise-only
if 'with_vocals' not in file_types:
raise HTTPException(
status_code=400,
detail="with_vocals video is required for finalise-only mode"
)
if 'title_screen' not in file_types:
raise HTTPException(
status_code=400,
detail="title_screen video is required for finalise-only mode"
)
if 'end_screen' not in file_types:
raise HTTPException(
status_code=400,
detail="end_screen video is required for finalise-only mode"
)
if 'instrumental_clean' not in file_types and 'instrumental_backing' not in file_types:
raise HTTPException(
status_code=400,
detail="At least one instrumental (clean or backing) is required"
)
# Validate file extensions
for file_info in body.files:
if file_info.file_type not in FINALISE_ONLY_FILE_TYPES:
raise HTTPException(
status_code=400,
detail=f"Invalid file_type: '{file_info.file_type}'. Valid types: {list(FINALISE_ONLY_FILE_TYPES.keys())}"
)
ext = Path(file_info.filename).suffix.lower()
allowed_extensions = FINALISE_ONLY_FILE_TYPES[file_info.file_type]
if ext not in allowed_extensions:
raise HTTPException(
status_code=400,
detail=f"Invalid file extension '{ext}' for file_type '{file_info.file_type}'. Allowed: {allowed_extensions}"
)
# Apply default distribution settings
settings = get_settings()
effective_dropbox_path = body.dropbox_path or settings.default_dropbox_path
effective_gdrive_folder_id = body.gdrive_folder_id or settings.default_gdrive_folder_id
effective_discord_webhook_url = body.discord_webhook_url or settings.default_discord_webhook_url
# Validate distribution credentials if services are requested
invalid_services = []
credential_manager = get_credential_manager()
if body.enable_youtube_upload:
result = credential_manager.check_youtube_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"youtube ({result.message})")
if effective_dropbox_path:
result = credential_manager.check_dropbox_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"dropbox ({result.message})")
if effective_gdrive_folder_id:
result = credential_manager.check_gdrive_credentials()
if result.status != CredentialStatus.VALID:
invalid_services.append(f"gdrive ({result.message})")
if invalid_services:
raise HTTPException(
status_code=400,
detail={
"error": "credentials_invalid",
"message": f"The following distribution services need re-authorization: {', '.join(invalid_services)}",
"invalid_services": invalid_services,
"auth_url": "/api/auth/status"
}
)
# Extract request metadata
request_metadata = extract_request_metadata(request, created_from="finalise_only_upload")
# Create job with finalise_only=True
job_create = JobCreate(
artist=body.artist,
title=body.title,
filename="finalise_only", # No single audio file - using prep outputs
enable_cdg=body.enable_cdg,
enable_txt=body.enable_txt,
brand_prefix=body.brand_prefix,
enable_youtube_upload=body.enable_youtube_upload,
youtube_description=body.youtube_description,
discord_webhook_url=effective_discord_webhook_url,
dropbox_path=effective_dropbox_path,
gdrive_folder_id=effective_gdrive_folder_id,
finalise_only=True,
keep_brand_code=body.keep_brand_code,
request_metadata=request_metadata,
)
job = job_manager.create_job(job_create)
job_id = job.job_id
logger.info(f"Created finalise-only job {job_id} for {body.artist} - {body.title}")
# Generate signed upload URLs for each file
upload_urls = []
for file_info in body.files:
gcs_path = _get_gcs_path_for_finalise_file(job_id, file_info.file_type, file_info.filename)
# Generate signed upload URL (valid for 60 minutes)
signed_url = storage_service.generate_signed_upload_url(
gcs_path,
content_type=file_info.content_type,
expiration_minutes=60
)
upload_urls.append(SignedUploadUrl(
file_type=file_info.file_type,
gcs_path=gcs_path,
upload_url=signed_url,
content_type=file_info.content_type
))
logger.info(f"Generated signed upload URL for {file_info.file_type}: {gcs_path}")
return CreateJobWithUploadUrlsResponse(
status="success",
job_id=job_id,
message="Finalise-only job created. Upload files to the provided URLs, then call /api/jobs/{job_id}/finalise-uploads-complete",
upload_urls=upload_urls,
server_version=VERSION
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Error creating finalise-only job: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e)) from e
@router.post("/jobs/{job_id}/finalise-uploads-complete")
async def mark_finalise_uploads_complete(
job_id: str,
background_tasks: BackgroundTasks,
body: UploadsCompleteRequest,
):
"""
Mark finalise-only file uploads as complete and start video generation.
This is called after uploading prep outputs for a finalise-only job.
The job will transition directly to AWAITING_INSTRUMENTAL_SELECTION.
"""
try:
# Get job and verify it exists
job = job_manager.get_job(job_id)
if not job:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
# Verify job is in pending state and is finalise-only
if job.status != JobStatus.PENDING:
raise HTTPException(
status_code=400,
detail=f"Job {job_id} is not in pending state (current: {job.status}). Cannot complete uploads."
)
if not job.finalise_only:
raise HTTPException(
status_code=400,
detail=f"Job {job_id} is not a finalise-only job. Use /api/jobs/{{job_id}}/uploads-complete instead."
)
# Validate required files were uploaded
required_files = {'with_vocals', 'title_screen', 'end_screen'}
uploaded_set = set(body.uploaded_files)
missing_files = required_files - uploaded_set
if missing_files:
raise HTTPException(
status_code=400,
detail=f"Missing required files: {missing_files}"
)
# Require at least one instrumental
if 'instrumental_clean' not in uploaded_set and 'instrumental_backing' not in uploaded_set:
raise HTTPException(
status_code=400,
detail="At least one instrumental (clean or backing) is required"
)
# Build file_urls structure from uploaded files
file_urls: Dict[str, Dict[str, str]] = {
'videos': {},
'screens': {},
'stems': {},
'lyrics': {},
}
for file_type in body.uploaded_files:
if file_type not in FINALISE_ONLY_FILE_TYPES:
logger.warning(f"Unknown file_type in uploaded_files: {file_type}")
continue
# Determine the GCS path - we need to find the actual file
if file_type == 'audio':
prefix = f"uploads/{job_id}/audio/"
elif file_type == 'with_vocals':
prefix = f"jobs/{job_id}/videos/with_vocals"
elif file_type == 'title_screen':
prefix = f"jobs/{job_id}/screens/title"
elif file_type == 'end_screen':
prefix = f"jobs/{job_id}/screens/end"
elif file_type == 'title_jpg':
prefix = f"jobs/{job_id}/screens/title.jpg"
elif file_type == 'title_png':
prefix = f"jobs/{job_id}/screens/title.png"
elif file_type == 'end_jpg':
prefix = f"jobs/{job_id}/screens/end.jpg"
elif file_type == 'end_png':
prefix = f"jobs/{job_id}/screens/end.png"
elif file_type == 'instrumental_clean':
prefix = f"jobs/{job_id}/stems/instrumental_clean"
elif file_type == 'instrumental_backing':
prefix = f"jobs/{job_id}/stems/instrumental_with_backing"
elif file_type == 'lrc':
prefix = f"jobs/{job_id}/lyrics/karaoke.lrc"
elif file_type == 'ass':
prefix = f"jobs/{job_id}/lyrics/karaoke.ass"
elif file_type == 'corrections':
prefix = f"jobs/{job_id}/lyrics/corrections"
elif file_type == 'style_params':
prefix = f"uploads/{job_id}/style/style_params"
else:
continue
# List files with this prefix to find the actual uploaded file
files = storage_service.list_files(prefix)
if not files:
raise HTTPException(
status_code=400,
detail=f"File for '{file_type}' was not uploaded to GCS. Expected prefix: {prefix}"
)
gcs_path = files[0]
# Map to file_urls structure
if file_type == 'audio':
pass # Handled separately
elif file_type == 'with_vocals':
file_urls['videos']['with_vocals'] = gcs_path
elif file_type == 'title_screen':
file_urls['screens']['title'] = gcs_path
elif file_type == 'end_screen':
file_urls['screens']['end'] = gcs_path
elif file_type == 'title_jpg':
file_urls['screens']['title_jpg'] = gcs_path
elif file_type == 'title_png':
file_urls['screens']['title_png'] = gcs_path
elif file_type == 'end_jpg':
file_urls['screens']['end_jpg'] = gcs_path
elif file_type == 'end_png':
file_urls['screens']['end_png'] = gcs_path
elif file_type == 'instrumental_clean':
file_urls['stems']['instrumental_clean'] = gcs_path
elif file_type == 'instrumental_backing':
file_urls['stems']['instrumental_with_backing'] = gcs_path
elif file_type == 'lrc':
file_urls['lyrics']['lrc'] = gcs_path
elif file_type == 'ass':
file_urls['lyrics']['ass'] = gcs_path
elif file_type == 'corrections':
file_urls['lyrics']['corrections'] = gcs_path
# Get audio path if uploaded
audio_gcs_path = None
if 'audio' in body.uploaded_files:
audio_files = storage_service.list_files(f"uploads/{job_id}/audio/")
if audio_files:
audio_gcs_path = audio_files[0]
# Update job with file paths
update_data = {
'file_urls': file_urls,
'finalise_only': True,
}
if audio_gcs_path:
update_data['input_media_gcs_path'] = audio_gcs_path
# Mark parallel processing as complete (we're skipping it)
update_data['state_data'] = {
'audio_complete': True,
'lyrics_complete': True,
'finalise_only': True,
}
job_manager.update_job(job_id, update_data)
logger.info(f"Validated finalise-only uploads for job {job_id}: {body.uploaded_files}")
# Transition directly to AWAITING_INSTRUMENTAL_SELECTION
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Prep files uploaded - select your instrumental"
)
# Get distribution services info for response
credential_manager = get_credential_manager()
distribution_services: Dict[str, Any] = {}
updated_job = job_manager.get_job(job_id)
if updated_job.dropbox_path:
dropbox_result = credential_manager.check_dropbox_credentials()
distribution_services["dropbox"] = {
"enabled": True,
"path": updated_job.dropbox_path,
"credentials_valid": dropbox_result.status == CredentialStatus.VALID,
}
if updated_job.gdrive_folder_id:
gdrive_result = credential_manager.check_gdrive_credentials()
distribution_services["gdrive"] = {
"enabled": True,
"folder_id": updated_job.gdrive_folder_id,
"credentials_valid": gdrive_result.status == CredentialStatus.VALID,
}
if updated_job.enable_youtube_upload:
youtube_result = credential_manager.check_youtube_credentials()
distribution_services["youtube"] = {
"enabled": True,
"credentials_valid": youtube_result.status == CredentialStatus.VALID,
}
if updated_job.discord_webhook_url:
distribution_services["discord"] = {
"enabled": True,
}
return {
"status": "success",
"job_id": job_id,
"message": "Finalise-only uploads validated. Select instrumental to continue.",
"files_validated": body.uploaded_files,
"server_version": VERSION,
"distribution_services": distribution_services,
}
except HTTPException:
raise
except Exception as e:
logger.error(f"Error completing finalise-only uploads for job {job_id}: {e}", exc_info=True)
raise HTTPException(status_code=500, detail=str(e)) from e
+18 -5
View File
@@ -73,6 +73,7 @@ class JobStatus(str, Enum):
# Terminal states
COMPLETE = "complete" # All processing finished successfully
PREP_COMPLETE = "prep_complete" # Prep-only job completed (stops after review)
FAILED = "failed" # Unrecoverable error occurred
CANCELLED = "cancelled" # User cancelled the job
@@ -86,8 +87,8 @@ class JobStatus(str, Enum):
# Valid state transitions
STATE_TRANSITIONS = {
# PENDING can go to DOWNLOADING (file upload) or SEARCHING_AUDIO (artist+title search)
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.FAILED, JobStatus.CANCELLED],
# PENDING can go to DOWNLOADING (file upload), SEARCHING_AUDIO (artist+title search), or AWAITING_INSTRUMENTAL_SELECTION (finalise-only)
JobStatus.PENDING: [JobStatus.DOWNLOADING, JobStatus.SEARCHING_AUDIO, JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED, JobStatus.CANCELLED],
# Audio search flow (for artist+title search mode)
JobStatus.SEARCHING_AUDIO: [JobStatus.AWAITING_AUDIO_SELECTION, JobStatus.DOWNLOADING_AUDIO, JobStatus.FAILED],
@@ -115,10 +116,10 @@ STATE_TRANSITIONS = {
# AWAITING_REVIEW can go directly to REVIEW_COMPLETE (quick review) or to IN_REVIEW (editing)
JobStatus.AWAITING_REVIEW: [JobStatus.IN_REVIEW, JobStatus.REVIEW_COMPLETE, JobStatus.FAILED, JobStatus.CANCELLED],
JobStatus.IN_REVIEW: [JobStatus.REVIEW_COMPLETE, JobStatus.AWAITING_REVIEW, JobStatus.FAILED],
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.FAILED],
JobStatus.REVIEW_COMPLETE: [JobStatus.RENDERING_VIDEO, JobStatus.PREP_COMPLETE, JobStatus.FAILED], # PREP_COMPLETE for prep-only jobs
# Video rendering (post-review)
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED],
JobStatus.RENDERING_VIDEO: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.PREP_COMPLETE, JobStatus.FAILED],
# Instrumental selection flow
JobStatus.AWAITING_INSTRUMENTAL_SELECTION: [JobStatus.INSTRUMENTAL_SELECTED, JobStatus.FAILED, JobStatus.CANCELLED],
@@ -133,9 +134,11 @@ STATE_TRANSITIONS = {
JobStatus.UPLOADING: [JobStatus.NOTIFYING, JobStatus.COMPLETE, JobStatus.FAILED],
JobStatus.NOTIFYING: [JobStatus.COMPLETE, JobStatus.FAILED],
# Terminal states - COMPLETE and CANCELLED have no transitions
# Terminal states - COMPLETE, PREP_COMPLETE, and CANCELLED have no transitions
# FAILED allows retry transitions to resume from checkpoints
# PREP_COMPLETE allows finalise-only continuation
JobStatus.COMPLETE: [],
JobStatus.PREP_COMPLETE: [JobStatus.AWAITING_INSTRUMENTAL_SELECTION, JobStatus.FAILED], # Finalise-only continues from here
JobStatus.FAILED: [
JobStatus.INSTRUMENTAL_SELECTED, # Retry from video generation
JobStatus.REVIEW_COMPLETE, # Retry from render stage
@@ -243,6 +246,11 @@ class Job(BaseModel):
audio_search_title: Optional[str] = None # Title used for audio search
auto_download: bool = False # Auto-select best audio source (skip selection)
# Two-phase workflow configuration (Batch 6)
prep_only: bool = False # Stop after review, don't run finalisation
finalise_only: bool = False # Skip prep, run only finalisation (requires uploaded prep outputs)
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Processing state
track_output_dir: Optional[str] = None # Local output directory (temp)
audio_hash: Optional[str] = None # Hash for deduplication
@@ -424,6 +432,11 @@ class JobCreate(BaseModel):
audio_search_title: Optional[str] = None # Title used for audio search
auto_download: bool = False # Auto-select best audio source (skip selection)
# Two-phase workflow configuration (Batch 6)
prep_only: bool = False # Stop after review, don't run finalisation
finalise_only: bool = False # Skip prep, run only finalisation
keep_brand_code: Optional[str] = None # Preserve existing brand code instead of generating new one
# Request metadata (set by API endpoint from request headers)
request_metadata: Dict[str, Any] = Field(default_factory=dict)
"""
+4
View File
@@ -55,6 +55,10 @@ class JobManager:
youtube_description=job_create.youtube_description,
webhook_url=job_create.webhook_url,
user_email=job_create.user_email,
# Two-phase workflow (Batch 6)
prep_only=job_create.prep_only,
finalise_only=job_create.finalise_only,
keep_brand_code=job_create.keep_brand_code,
# Request metadata (for tracking and filtering)
request_metadata=job_create.request_metadata,
)
+222
View File
@@ -963,6 +963,228 @@ class TestDurationValidation:
assert inspect.iscoroutinefunction(_validate_audio_durations)
class TestTwoPhaseWorkflowModels:
"""Test Pydantic models for two-phase workflow (Batch 6)."""
def test_create_job_with_upload_urls_request_has_prep_only(self):
"""Test that CreateJobWithUploadUrlsRequest has prep_only field."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
],
prep_only=True
)
assert request.prep_only is True
def test_create_job_with_upload_urls_request_prep_only_default_false(self):
"""Test that prep_only defaults to False."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
]
)
assert request.prep_only is False
def test_create_job_with_upload_urls_request_has_keep_brand_code(self):
"""Test that CreateJobWithUploadUrlsRequest has keep_brand_code field."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
],
keep_brand_code="NOMAD-1234"
)
assert request.keep_brand_code == "NOMAD-1234"
def test_create_job_with_upload_urls_request_keep_brand_code_default_none(self):
"""Test that keep_brand_code defaults to None."""
from backend.api.routes.file_upload import CreateJobWithUploadUrlsRequest, FileUploadRequest
request = CreateJobWithUploadUrlsRequest(
artist="Test Artist",
title="Test Song",
files=[
FileUploadRequest(filename="test.flac", content_type="audio/flac", file_type="audio")
]
)
assert request.keep_brand_code is None
class TestFinaliseOnlyFileTypes:
"""Test finalise-only file types (Batch 6)."""
def test_finalise_only_file_types_exists(self):
"""Test that FINALISE_ONLY_FILE_TYPES is defined."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert FINALISE_ONLY_FILE_TYPES is not None
assert isinstance(FINALISE_ONLY_FILE_TYPES, dict)
def test_finalise_only_file_types_has_with_vocals(self):
"""Test that with_vocals is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'with_vocals' in FINALISE_ONLY_FILE_TYPES
assert '.mkv' in FINALISE_ONLY_FILE_TYPES['with_vocals']
assert '.mov' in FINALISE_ONLY_FILE_TYPES['with_vocals']
def test_finalise_only_file_types_has_title_screen(self):
"""Test that title_screen is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'title_screen' in FINALISE_ONLY_FILE_TYPES
def test_finalise_only_file_types_has_end_screen(self):
"""Test that end_screen is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'end_screen' in FINALISE_ONLY_FILE_TYPES
def test_finalise_only_file_types_has_instrumentals(self):
"""Test that instrumental types are valid finalise-only file types."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'instrumental_clean' in FINALISE_ONLY_FILE_TYPES
assert 'instrumental_backing' in FINALISE_ONLY_FILE_TYPES
def test_finalise_only_file_types_has_lrc(self):
"""Test that lrc is a valid finalise-only file type."""
from backend.api.routes.file_upload import FINALISE_ONLY_FILE_TYPES
assert 'lrc' in FINALISE_ONLY_FILE_TYPES
assert '.lrc' in FINALISE_ONLY_FILE_TYPES['lrc']
class TestFinaliseOnlyModels:
"""Test Pydantic models for finalise-only flow (Batch 6)."""
def test_finalise_only_file_request_model(self):
"""Test FinaliseOnlyFileRequest model."""
from backend.api.routes.file_upload import FinaliseOnlyFileRequest
file_req = FinaliseOnlyFileRequest(
filename="with_vocals.mkv",
content_type="video/mkv",
file_type="with_vocals"
)
assert file_req.filename == "with_vocals.mkv"
assert file_req.content_type == "video/mkv"
assert file_req.file_type == "with_vocals"
def test_create_finalise_only_job_request_model(self):
"""Test CreateFinaliseOnlyJobRequest model."""
from backend.api.routes.file_upload import CreateFinaliseOnlyJobRequest, FinaliseOnlyFileRequest
request = CreateFinaliseOnlyJobRequest(
artist="Test Artist",
title="Test Song",
files=[
FinaliseOnlyFileRequest(filename="with_vocals.mkv", content_type="video/mkv", file_type="with_vocals"),
FinaliseOnlyFileRequest(filename="title.mov", content_type="video/quicktime", file_type="title_screen"),
FinaliseOnlyFileRequest(filename="end.mov", content_type="video/quicktime", file_type="end_screen"),
FinaliseOnlyFileRequest(filename="instrumental.flac", content_type="audio/flac", file_type="instrumental_clean"),
],
enable_cdg=True,
enable_txt=True,
brand_prefix="NOMAD",
keep_brand_code="NOMAD-1234"
)
assert request.artist == "Test Artist"
assert request.title == "Test Song"
assert len(request.files) == 4
assert request.keep_brand_code == "NOMAD-1234"
def test_create_finalise_only_job_request_optional_fields(self):
"""Test CreateFinaliseOnlyJobRequest optional fields default correctly."""
from backend.api.routes.file_upload import CreateFinaliseOnlyJobRequest, FinaliseOnlyFileRequest
request = CreateFinaliseOnlyJobRequest(
artist="Artist",
title="Title",
files=[
FinaliseOnlyFileRequest(filename="with_vocals.mkv", content_type="video/mkv", file_type="with_vocals"),
]
)
assert request.enable_cdg is True
assert request.enable_txt is True
assert request.brand_prefix is None
assert request.keep_brand_code is None
assert request.enable_youtube_upload is False
assert request.dropbox_path is None
assert request.gdrive_folder_id is None
class TestFinaliseOnlyGCSPaths:
"""Test GCS path generation for finalise-only files (Batch 6)."""
def test_with_vocals_gcs_path(self):
"""Test GCS path generation for with_vocals file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "with_vocals", "video.mkv")
assert path == "jobs/test123/videos/with_vocals.mkv"
def test_title_screen_gcs_path(self):
"""Test GCS path generation for title_screen file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "title_screen", "title.mov")
assert path == "jobs/test123/screens/title.mov"
def test_end_screen_gcs_path(self):
"""Test GCS path generation for end_screen file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "end_screen", "end.mov")
assert path == "jobs/test123/screens/end.mov"
def test_instrumental_clean_gcs_path(self):
"""Test GCS path generation for instrumental_clean file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "instrumental_clean", "clean.flac")
assert path == "jobs/test123/stems/instrumental_clean.flac"
def test_instrumental_backing_gcs_path(self):
"""Test GCS path generation for instrumental_backing file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "instrumental_backing", "backing.flac")
assert path == "jobs/test123/stems/instrumental_with_backing.flac"
def test_lrc_gcs_path(self):
"""Test GCS path generation for lrc file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "lrc", "karaoke.lrc")
assert path == "jobs/test123/lyrics/karaoke.lrc"
def test_title_jpg_gcs_path(self):
"""Test GCS path generation for title_jpg file."""
from backend.api.routes.file_upload import _get_gcs_path_for_finalise_file
path = _get_gcs_path_for_finalise_file("test123", "title_jpg", "title.jpg")
assert path == "jobs/test123/screens/title.jpg"
# ============================================================================
# Batch 4: YouTube URL Input Tests
# ============================================================================
+243
View File
@@ -670,6 +670,249 @@ class TestExistingInstrumentalModelFields:
assert job.existing_instrumental_gcs_path == "uploads/test123/audio/my_instrumental.flac"
class TestTwoPhaseWorkflowFields:
"""Test two-phase workflow fields in Job and JobCreate models (Batch 6)."""
def test_job_has_prep_only_field(self):
"""Test that Job model has prep_only field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
prep_only=True
)
assert hasattr(job, 'prep_only')
assert job.prep_only is True
def test_job_prep_only_defaults_to_false(self):
"""Test that prep_only defaults to False."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.prep_only is False
def test_job_has_finalise_only_field(self):
"""Test that Job model has finalise_only field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
finalise_only=True
)
assert hasattr(job, 'finalise_only')
assert job.finalise_only is True
def test_job_finalise_only_defaults_to_false(self):
"""Test that finalise_only defaults to False."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.finalise_only is False
def test_job_has_keep_brand_code_field(self):
"""Test that Job model has keep_brand_code field."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
keep_brand_code="NOMAD-1234"
)
assert hasattr(job, 'keep_brand_code')
assert job.keep_brand_code == "NOMAD-1234"
def test_job_keep_brand_code_defaults_to_none(self):
"""Test that keep_brand_code defaults to None."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.keep_brand_code is None
def test_job_create_has_two_phase_workflow_fields(self):
"""Test that JobCreate model has two-phase workflow fields."""
job_create = JobCreate(
artist="Artist",
title="Title",
prep_only=True,
finalise_only=False,
keep_brand_code="BRAND-0001"
)
assert job_create.prep_only is True
assert job_create.finalise_only is False
assert job_create.keep_brand_code == "BRAND-0001"
def test_two_phase_workflow_roundtrip_serialization(self):
"""Test that two-phase workflow fields survive serialization/deserialization."""
job = Job(
job_id="test123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
prep_only=True,
finalise_only=False,
keep_brand_code="NOMAD-5678"
)
# Serialize to dict (simulates writing to Firestore)
job_dict = job.model_dump()
# Convert to JSON and back (simulates Firestore storage)
json_str = json.dumps(job_dict, default=str)
restored_dict = json.loads(json_str)
# Deserialize back to Job (simulates reading from Firestore)
restored_job = Job(**restored_dict)
assert restored_job.prep_only is True
assert restored_job.finalise_only is False
assert restored_job.keep_brand_code == "NOMAD-5678"
class TestPrepCompleteStatus:
"""Test PREP_COMPLETE status and state transitions (Batch 6)."""
def test_prep_complete_status_exists(self):
"""Test that PREP_COMPLETE status exists in JobStatus enum."""
assert hasattr(JobStatus, 'PREP_COMPLETE')
assert JobStatus.PREP_COMPLETE.value == "prep_complete"
def test_job_can_have_prep_complete_status(self):
"""Test that Job can be created with PREP_COMPLETE status."""
job = Job(
job_id="test123",
status=JobStatus.PREP_COMPLETE,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC)
)
assert job.status == JobStatus.PREP_COMPLETE
def test_state_transitions_review_complete_to_prep_complete(self):
"""Test that REVIEW_COMPLETE can transition to PREP_COMPLETE."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.REVIEW_COMPLETE, [])
assert JobStatus.PREP_COMPLETE in valid_transitions
def test_state_transitions_prep_complete_to_awaiting_instrumental(self):
"""Test that PREP_COMPLETE can transition to AWAITING_INSTRUMENTAL_SELECTION."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PREP_COMPLETE, [])
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
def test_state_transitions_pending_to_awaiting_instrumental(self):
"""Test that PENDING can transition to AWAITING_INSTRUMENTAL_SELECTION (for finalise-only)."""
from backend.models.job import STATE_TRANSITIONS
valid_transitions = STATE_TRANSITIONS.get(JobStatus.PENDING, [])
assert JobStatus.AWAITING_INSTRUMENTAL_SELECTION in valid_transitions
def test_critical_statuses_include_prep_complete(self):
"""Test that PREP_COMPLETE is in the list of job statuses."""
all_statuses = [status.value for status in JobStatus]
assert "prep_complete" in all_statuses
class TestJobWithFullTwoPhaseConfig:
"""Integration tests for two-phase workflow job configuration."""
def test_prep_only_job_full_config(self):
"""Test creating a prep-only job with full configuration."""
job = Job(
job_id="prep123",
status=JobStatus.PENDING,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
prep_only=True,
enable_cdg=True,
enable_txt=True,
brand_prefix="NOMAD",
)
assert job.prep_only is True
assert job.finalise_only is False
assert job.brand_prefix == "NOMAD"
def test_finalise_only_job_full_config(self):
"""Test creating a finalise-only job with full configuration."""
job = Job(
job_id="final123",
status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="Test Artist",
title="Test Song",
finalise_only=True,
keep_brand_code="NOMAD-1234",
enable_youtube_upload=True,
dropbox_path="/Karaoke/Tracks",
state_data={
"audio_complete": True,
"lyrics_complete": True,
"finalise_only": True,
}
)
assert job.finalise_only is True
assert job.keep_brand_code == "NOMAD-1234"
assert job.state_data["finalise_only"] is True
def test_prep_complete_job_with_file_urls(self):
"""Test a job that has reached PREP_COMPLETE status with all prep outputs."""
job = Job(
job_id="prepcomp123",
status=JobStatus.PREP_COMPLETE,
created_at=datetime.now(UTC),
updated_at=datetime.now(UTC),
artist="ABBA",
title="Waterloo",
prep_only=True,
file_urls={
"stems": {
"instrumental_clean": "gs://bucket/jobs/prepcomp123/stems/instrumental_clean.flac",
"instrumental_with_backing": "gs://bucket/jobs/prepcomp123/stems/instrumental_with_backing.flac",
},
"videos": {
"with_vocals": "gs://bucket/jobs/prepcomp123/videos/with_vocals.mkv",
},
"screens": {
"title": "gs://bucket/jobs/prepcomp123/screens/title.mov",
"end": "gs://bucket/jobs/prepcomp123/screens/end.mov",
},
"lyrics": {
"lrc": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.lrc",
"ass": "gs://bucket/jobs/prepcomp123/lyrics/karaoke.ass",
},
}
)
assert job.status == JobStatus.PREP_COMPLETE
assert job.prep_only is True
assert "with_vocals" in job.file_urls["videos"]
assert "title" in job.file_urls["screens"]
if __name__ == "__main__":
pytest.main([__file__, "-v"])
+16 -3
View File
@@ -447,11 +447,24 @@ async def upload_lyrics_results(
corrections_data = json.load(f)
# Store metadata in state_data
job_manager.update_state_data(job_id, 'lyrics_metadata', {
# Include countdown padding info for instrumental synchronization
lyrics_metadata = {
'segment_count': len(corrections_data.get('corrected_segments', [])),
'has_corrections': True,
'ready_for_review': True
})
'ready_for_review': True,
}
# Add countdown padding info from transcription result
# This is needed by video_worker to pad instrumentals for sync
if transcription_result.get("countdown_padding_added"):
lyrics_metadata['has_countdown_padding'] = True
lyrics_metadata['countdown_padding_seconds'] = transcription_result.get("countdown_padding_seconds", 3.0)
logger.info(f"Job {job_id}: Countdown padding detected: {lyrics_metadata['countdown_padding_seconds']}s")
else:
lyrics_metadata['has_countdown_padding'] = False
lyrics_metadata['countdown_padding_seconds'] = 0.0
job_manager.update_state_data(job_id, 'lyrics_metadata', lyrics_metadata)
except Exception as e:
logger.warning(f"Job {job_id}: Could not parse corrections JSON: {e}")
else:
+21 -10
View File
@@ -270,16 +270,27 @@ async def process_render_video(job_id: str) -> bool:
job_log.info("Analyzing backing vocals for instrumental selection...")
await _analyze_backing_vocals(job_id, job_manager, storage, job_log)
# 12. Transition to awaiting instrumental selection
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Video rendered - select your instrumental"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
# 12. Transition based on prep_only flag
if getattr(job, 'prep_only', False):
# Prep-only mode: stop here and mark as prep complete
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.PREP_COMPLETE,
progress=100,
message="Prep phase complete - download outputs to continue locally"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE (PREP ONLY) ===")
logger.info(f"Job {job_id}: Prep-only video render complete")
else:
# Normal mode: proceed to instrumental selection
job_manager.transition_to_state(
job_id=job_id,
new_status=JobStatus.AWAITING_INSTRUMENTAL_SELECTION,
progress=80,
message="Video rendered - select your instrumental"
)
job_log.info("=== RENDER VIDEO WORKER COMPLETE ===")
logger.info(f"Job {job_id}: Video render complete, awaiting instrumental selection")
return True
except Exception as e:
+18
View File
@@ -162,6 +162,14 @@ async def generate_video(job_id: str) -> bool:
message="Encoding videos (15-20 min)"
)
# Get countdown padding info from lyrics metadata
# This ensures instrumental is padded to match vocals if countdown was added
lyrics_metadata = job.state_data.get('lyrics_metadata', {})
countdown_padding_seconds = None
if lyrics_metadata.get('has_countdown_padding'):
countdown_padding_seconds = lyrics_metadata.get('countdown_padding_seconds', 3.0)
job_log.info(f"Countdown padding detected: {countdown_padding_seconds}s - instrumental will be padded if needed")
# Log finalization parameters
job_log.info("Creating KaraokeFinalise with parameters:")
job_log.info(f" enable_cdg: {getattr(job, 'enable_cdg', False)}")
@@ -169,6 +177,7 @@ async def generate_video(job_id: str) -> bool:
job_log.info(f" brand_prefix: {getattr(job, 'brand_prefix', None)}")
job_log.info(f" discord_webhook: {'configured' if getattr(job, 'discord_webhook_url', None) else 'not configured'}")
job_log.info(f" instrumental source: {instrumental_source}")
job_log.info(f" countdown_padding_seconds: {countdown_padding_seconds}")
if existing_instrumental_path:
job_log.info(f" using user-provided instrumental (selection was: {instrumental_selection})")
@@ -209,6 +218,8 @@ async def generate_video(job_id: str) -> bool:
non_interactive=True,
server_side_mode=True,
selected_instrumental_file=instrumental_file,
# Audio synchronization - ensure instrumental matches vocal padding
countdown_padding_seconds=countdown_padding_seconds,
)
# Call process() - this does ALL the work:
@@ -324,6 +335,13 @@ async def _handle_native_distribution(
brand_code = result.get('brand_code')
base_name = f"{job.artist} - {job.title}"
# Check if we should preserve existing brand code (Batch 6: --keep-brand-code)
keep_brand_code = getattr(job, 'keep_brand_code', None)
if keep_brand_code:
brand_code = keep_brand_code
result['brand_code'] = brand_code
job_log.info(f"Using preserved brand code: {brand_code}")
# Upload to Dropbox using native SDK
dropbox_path = getattr(job, 'dropbox_path', None)
brand_prefix = getattr(job, 'brand_prefix', None)
@@ -421,19 +421,27 @@ Bump the patch version in `pyproject.toml` before committing.
---
### Batch 6: Two-Phase Workflow (HIGH)
### Batch 6: Two-Phase Workflow (HIGH) ✅ COMPLETE
**Parameters:** W1 `--prep-only`, W2 `--finalise-only`, F15 `--keep-brand-code`
**Scope:**
- **prep-only:** Remote CLI submits with `prep_only=true`. Backend runs through review, then stops. CLI downloads all outputs and exits.
- **finalise-only:** Remote CLI checks local folder for prep outputs, uploads ALL files, sends `finalise_only=true`. Backend runs finalisation only.
- **keep-brand-code:** Remote CLI extracts brand code from local folder name, sends to backend to preserve.
**Status:** ✅ Complete (2025-12-11)
**Files to modify:**
- `karaoke_gen/utils/remote_cli.py` - prep-only flow, finalise-only upload flow, brand code extraction
- `backend/api/routes/file_upload.py` - accept prep_only, finalise_only flags
- `backend/api/routes/` - new endpoint for finalise-only upload (multiple files)
- `backend/workers/orchestrator.py` or job state machine - conditional worker execution
**Implementation Summary:**
- **prep-only:** Added `prep_only` field to Job model. When set, backend transitions to new `PREP_COMPLETE` status after review and video rendering (with_vocals.mkv). CLI downloads all prep outputs.
- **finalise-only:** Added new endpoint `/api/jobs/create-finalise-only` for uploading prep outputs. Remote CLI detects files in prep folder, uploads them via signed URLs, and monitors finalisation.
- **keep-brand-code:** Remote CLI extracts brand code from folder name (e.g., "NOMAD-1234 - Artist - Title"), sends to backend. Video worker uses preserved brand code for distribution.
**Files modified:**
- `backend/models/job.py` - Added `PREP_COMPLETE` status, `prep_only`, `finalise_only`, `keep_brand_code` fields, updated state transitions
- `backend/api/routes/file_upload.py` - Added `prep_only`, `keep_brand_code` to job creation, new finalise-only endpoints
- `backend/workers/render_video_worker.py` - Conditional transition to PREP_COMPLETE for prep-only jobs
- `backend/workers/video_worker.py` - Use preserved brand code when set
- `backend/services/job_manager.py` - Pass new fields through job creation
- `karaoke_gen/utils/remote_cli.py` - Full implementation of prep-only, finalise-only, and keep-brand-code flows
**Tests added:**
- `backend/tests/test_models.py` - Tests for new fields and PREP_COMPLETE status
- `backend/tests/test_file_upload.py` - Tests for finalise-only models and endpoints
**Complexity:** High (significant workflow changes, new upload flow)
@@ -932,7 +940,7 @@ DEFAULT_DISCORD_WEBHOOK_URL=https://discord.com/...
### Medium-Term: Workflow Features
1. **Two-Phase Workflow** (Batch 6)
1. **Two-Phase Workflow** (Batch 6) ✅ COMPLETE
- `--prep-only` and `--finalise-only` modes
3. **Style Override** (Batch 7)
+11 -3
View File
@@ -771,8 +771,16 @@ class AudioProcessor:
padded_result["other_stems"] = separation_result.get("other_stems", {})
padded_result["backing_vocals"] = separation_result.get("backing_vocals", {})
self.logger.info(
f"✓ Countdown padding applied to {len(padded_result['combined_instrumentals']) + 1} instrumental file(s)"
)
# Count actual padded files (don't assume clean instrumental was padded)
padded_count = (1 if padded_result["clean_instrumental"].get("instrumental") else 0) + len(padded_result["combined_instrumentals"])
if padded_count > 0:
self.logger.info(
f"✓ Countdown padding applied to {padded_count} instrumental file(s)"
)
else:
self.logger.warning(
"No instrumental files found to pad. Check that audio separation completed successfully."
)
return padded_result
@@ -47,6 +47,7 @@ class KaraokeFinalise:
user_youtube_credentials=None, # Add support for pre-stored credentials
server_side_mode=False, # New parameter for server-side deployment
selected_instrumental_file=None, # Add support for pre-selected instrumental file
countdown_padding_seconds=None, # Padding applied to vocals; instrumental must match
):
self.log_level = log_level
self.log_formatter = log_formatter
@@ -108,6 +109,7 @@ class KaraokeFinalise:
self.user_youtube_credentials = user_youtube_credentials
self.server_side_mode = server_side_mode
self.selected_instrumental_file = selected_instrumental_file
self.countdown_padding_seconds = countdown_padding_seconds
self.suffixes = {
"title_mov": " (Title).mov",
@@ -424,6 +426,15 @@ class KaraokeFinalise:
# Check if any videos were found
if "items" in response and len(response["items"]) > 0:
for item in response["items"]:
# YouTube search API sometimes returns results from other channels even with channelId filter
# Verify the video actually belongs to our channel
result_channel_id = item["snippet"]["channelId"]
if result_channel_id != channel_id:
self.logger.debug(
f"Skipping video from different channel: {item['snippet']['title']} (channel: {result_channel_id})"
)
continue
found_title = item["snippet"]["title"]
# In server-side mode, require an exact match to avoid false positives.
@@ -723,6 +734,32 @@ class KaraokeFinalise:
artist, title = base_name.split(" - ", 1)
return base_name, artist, title
def _pad_audio_file(self, input_audio, output_audio, padding_seconds):
"""
Pad an audio file by prepending silence at the beginning.
Uses the same ffmpeg approach as LyricsTranscriber's CountdownProcessor
to ensure consistent padding behavior.
Args:
input_audio: Path to input audio file
output_audio: Path for the padded output file
padding_seconds: Amount of silence to prepend (in seconds)
"""
self.logger.info(f"Padding audio file with {padding_seconds}s of silence")
# Use ffmpeg to prepend silence - this matches the approach in audio_processor.py
# adelay filter adds delay in milliseconds
delay_ms = int(padding_seconds * 1000)
ffmpeg_command = (
f'{self.ffmpeg_base_command} -i "{input_audio}" '
f'-af "adelay={delay_ms}|{delay_ms}" '
f'"{output_audio}"'
)
self.execute_command(ffmpeg_command, f"Padding audio with {padding_seconds}s silence")
def execute_command(self, command, description):
"""Execute a shell command and log the output. For general commands (rclone, etc.)"""
self.logger.info(f"{description}")
@@ -767,11 +804,32 @@ class KaraokeFinalise:
def remux_with_instrumental(self, with_vocals_file, instrumental_audio, output_file):
"""Remux the video with instrumental audio to create karaoke version"""
# Safety net: If countdown padding was applied to vocals, ensure instrumental is padded too
actual_instrumental = instrumental_audio
if self.countdown_padding_seconds and self.countdown_padding_seconds > 0:
# Check if the instrumental file is already padded (has "(Padded)" in name)
if "(Padded)" not in instrumental_audio:
self.logger.warning(
f"Countdown padding ({self.countdown_padding_seconds}s) was applied to vocals, "
f"but instrumental doesn't appear to be padded. Creating padded version..."
)
# Create a padded version of the instrumental
base, ext = os.path.splitext(instrumental_audio)
padded_instrumental = f"{base} (Padded){ext}"
if not os.path.exists(padded_instrumental):
self._pad_audio_file(instrumental_audio, padded_instrumental, self.countdown_padding_seconds)
self.logger.info(f"Created padded instrumental: {padded_instrumental}")
actual_instrumental = padded_instrumental
else:
self.logger.info(f"Using already-padded instrumental: {instrumental_audio}")
# This operation is primarily I/O bound (remuxing), so hardware acceleration doesn't provide significant benefit
# Keep the existing approach but use the new execute method
ffmpeg_command = (
f'{self.ffmpeg_base_command} -an -i "{with_vocals_file}" '
f'-vn -i "{instrumental_audio}" -c:v copy -c:a pcm_s16le "{output_file}"'
f'-vn -i "{actual_instrumental}" -c:v copy -c:a pcm_s16le "{output_file}"'
)
self.execute_command(ffmpeg_command, "Remuxing video with instrumental audio")
+107
View File
@@ -259,6 +259,101 @@ class KaraokePrep:
self.artist = metadata_result["artist"]
self.title = metadata_result["title"]
def _scan_directory_for_instrumentals(self, track_output_dir, artist_title):
"""
Scan the directory for existing instrumental files and build a separated_audio structure.
This is used when transcription was skipped (existing files found) but we need to
pad instrumentals due to countdown padding.
Args:
track_output_dir: The track output directory to scan
artist_title: The "{artist} - {title}" string for matching files
Returns:
Dictionary with separated_audio structure containing found instrumental paths
"""
self.logger.info(f"Scanning directory for existing instrumentals: {track_output_dir}")
separated_audio = {
"clean_instrumental": {},
"backing_vocals": {},
"other_stems": {},
"combined_instrumentals": {},
}
# Search patterns for instrumental files
# Files are named like: "{artist} - {title} (Instrumental {model}).flac"
# Or with backing vocals: "{artist} - {title} (Instrumental +BV {model}).flac"
# Look for files in the track output directory
search_dir = track_output_dir
# Find all instrumental files (not padded ones - we want the originals)
instrumental_pattern = os.path.join(search_dir, f"{artist_title} (Instrumental*.flac")
instrumental_files = glob.glob(instrumental_pattern)
# Also check for wav files
instrumental_pattern_wav = os.path.join(search_dir, f"{artist_title} (Instrumental*.wav")
instrumental_files.extend(glob.glob(instrumental_pattern_wav))
self.logger.debug(f"Found {len(instrumental_files)} instrumental files")
for filepath in instrumental_files:
filename = os.path.basename(filepath)
# Skip already padded files
if "(Padded)" in filename:
self.logger.debug(f"Skipping already padded file: {filename}")
continue
# Determine if it's a combined instrumental (+BV) or clean instrumental
if "+BV" in filename or "+bv" in filename.lower():
# Combined instrumental with backing vocals
# Extract model name from filename
# Pattern: "(Instrumental +BV {model}).flac"
model_match = re.search(r'\(Instrumental \+BV ([^)]+)\)', filename)
if model_match:
model_name = model_match.group(1).strip()
separated_audio["combined_instrumentals"][model_name] = filepath
self.logger.info(f"Found combined instrumental: {filename}")
else:
# Clean instrumental (no backing vocals)
# Pattern: "(Instrumental {model}).flac"
model_match = re.search(r'\(Instrumental ([^)]+)\)', filename)
if model_match:
# Use as clean instrumental if we don't have one yet
if not separated_audio["clean_instrumental"].get("instrumental"):
separated_audio["clean_instrumental"]["instrumental"] = filepath
self.logger.info(f"Found clean instrumental: {filename}")
else:
# Additional clean instrumentals go to combined_instrumentals for padding
model_name = model_match.group(1).strip()
separated_audio["combined_instrumentals"][model_name] = filepath
self.logger.info(f"Found additional instrumental: {filename}")
# Also look for backing vocals files
backing_vocals_pattern = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.flac")
backing_vocals_files = glob.glob(backing_vocals_pattern)
backing_vocals_pattern_wav = os.path.join(search_dir, f"{artist_title} (Backing Vocals*.wav")
backing_vocals_files.extend(glob.glob(backing_vocals_pattern_wav))
for filepath in backing_vocals_files:
filename = os.path.basename(filepath)
model_match = re.search(r'\(Backing Vocals ([^)]+)\)', filename)
if model_match:
model_name = model_match.group(1).strip()
if model_name not in separated_audio["backing_vocals"]:
separated_audio["backing_vocals"][model_name] = {"backing_vocals": filepath}
self.logger.info(f"Found backing vocals: {filename}")
# Log summary
clean_count = 1 if separated_audio["clean_instrumental"].get("instrumental") else 0
combined_count = len(separated_audio["combined_instrumentals"])
self.logger.info(f"Directory scan complete: {clean_count} clean instrumental, {combined_count} combined instrumentals")
return separated_audio
async def prep_single_track(self):
# Add signal handler at the start
loop = asyncio.get_running_loop()
@@ -767,6 +862,18 @@ class KaraokePrep:
f"Applying {padding_seconds}s padding to all instrumental files to sync with vocal countdown"
)
# If separated_audio is empty (e.g., transcription was skipped but existing files have countdown),
# scan the directory for existing instrumental files
has_instrumentals = (
processed_track["separated_audio"].get("clean_instrumental", {}).get("instrumental") or
processed_track["separated_audio"].get("combined_instrumentals")
)
if not has_instrumentals:
self.logger.info("No instrumentals in separated_audio, scanning directory for existing files...")
processed_track["separated_audio"] = self._scan_directory_for_instrumentals(
track_output_dir, artist_title
)
# Apply padding using AudioProcessor
padded_separation_result = self.audio_processor.apply_countdown_padding_to_instrumentals(
separation_result=processed_track["separated_audio"],
+81 -4
View File
@@ -11,6 +11,9 @@ from .utils import sanitize_filename
# Placeholder class or functions for lyrics processing
class LyricsProcessor:
# Standard countdown padding duration used by LyricsTranscriber
COUNTDOWN_PADDING_SECONDS = 3.0
def __init__(
self, logger, style_params_json, lyrics_file, skip_transcription, skip_transcription_review, render_video, subtitle_offset_ms
):
@@ -22,6 +25,60 @@ class LyricsProcessor:
self.render_video = render_video
self.subtitle_offset_ms = subtitle_offset_ms
def _detect_countdown_padding_from_lrc(self, lrc_filepath):
"""
Detect if countdown padding was applied by checking the first lyric timestamp in the LRC file.
LRC format timestamps look like: [mm:ss.xx] or [mm:ss.xxx]
If the first lyric timestamp is >= 3.0 seconds, countdown padding was likely applied.
Args:
lrc_filepath: Path to the LRC file
Returns:
Tuple of (countdown_padding_added: bool, countdown_padding_seconds: float)
"""
try:
with open(lrc_filepath, 'r', encoding='utf-8') as f:
content = f.read()
# Find all timestamp patterns in the LRC file
# LRC timestamps: [mm:ss.xx] or [mm:ss.xxx]
timestamp_pattern = r'\[(\d{1,2}):(\d{2})\.(\d{2,3})\]'
matches = re.findall(timestamp_pattern, content)
if not matches:
self.logger.debug("No timestamps found in LRC file")
return (False, 0.0)
# Find the first non-metadata timestamp (metadata like [ar:Artist] doesn't have decimal)
# We already filtered for decimal timestamps in our pattern
first_timestamp = matches[0]
minutes = int(first_timestamp[0])
seconds = int(first_timestamp[1])
# Handle both .xx and .xxx formats
centiseconds = first_timestamp[2]
if len(centiseconds) == 2:
milliseconds = int(centiseconds) * 10
else:
milliseconds = int(centiseconds)
first_lyric_time = minutes * 60 + seconds + milliseconds / 1000.0
self.logger.debug(f"First lyric timestamp in LRC: {first_lyric_time:.3f}s")
# If first lyric is at or after 3 seconds, countdown padding was applied
# Use a small buffer (2.5s) to account for songs that naturally start a bit late
if first_lyric_time >= 2.5:
self.logger.info(f"Detected countdown padding from LRC: first lyric at {first_lyric_time:.2f}s")
return (True, self.COUNTDOWN_PADDING_SECONDS)
return (False, 0.0)
except Exception as e:
self.logger.warning(f"Failed to detect countdown padding from LRC file: {e}")
return (False, 0.0)
def find_best_split_point(self, line):
"""
Find the best split point in a line based on the specified criteria.
@@ -138,23 +195,43 @@ class LyricsProcessor:
lyrics_video_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (With Vocals).mkv")
lyrics_lrc_path = os.path.join(lyrics_dir, f"{sanitized_artist} - {sanitized_title} (Karaoke).lrc")
# If files exist in parent directory, return early
# If files exist in parent directory, return early (but detect countdown padding first)
if os.path.exists(parent_video_path) and os.path.exists(parent_lrc_path):
self.logger.info(f"Found existing video and LRC files in parent directory, skipping transcription")
self.logger.info("Found existing video and LRC files in parent directory, skipping transcription")
# Detect countdown padding from existing LRC file
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
if countdown_padding_added:
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
return {
"lrc_filepath": parent_lrc_path,
"ass_filepath": parent_video_path,
"countdown_padding_added": countdown_padding_added,
"countdown_padding_seconds": countdown_padding_seconds,
"padded_audio_filepath": None, # Original padded audio may not exist
}
# If files exist in lyrics directory, copy to parent and return
# If files exist in lyrics directory, copy to parent and return (but detect countdown padding first)
if os.path.exists(lyrics_video_path) and os.path.exists(lyrics_lrc_path):
self.logger.info(f"Found existing video and LRC files in lyrics directory, copying to parent")
self.logger.info("Found existing video and LRC files in lyrics directory, copying to parent")
os.makedirs(track_output_dir, exist_ok=True)
shutil.copy2(lyrics_video_path, parent_video_path)
shutil.copy2(lyrics_lrc_path, parent_lrc_path)
# Detect countdown padding from existing LRC file
countdown_padding_added, countdown_padding_seconds = self._detect_countdown_padding_from_lrc(parent_lrc_path)
if countdown_padding_added:
self.logger.info(f"Existing files have countdown padding: {countdown_padding_seconds}s")
return {
"lrc_filepath": parent_lrc_path,
"ass_filepath": parent_video_path,
"countdown_padding_added": countdown_padding_added,
"countdown_padding_seconds": countdown_padding_seconds,
"padded_audio_filepath": None, # Original padded audio may not exist
}
# Create lyrics directory if it doesn't exist
+14
View File
@@ -266,6 +266,13 @@ def run_instrumental_review(track: dict, logger: logging.Logger) -> str | None:
else:
logger.warning("Custom instrumental not found, falling back to clean")
return clean_instrumental_path
elif selection == "uploaded":
uploaded_path = server.get_uploaded_instrumental_path()
if uploaded_path and os.path.exists(uploaded_path):
return uploaded_path
else:
logger.warning("Uploaded instrumental not found, falling back to clean")
return clean_instrumental_path
else:
logger.warning(f"Unknown selection: {selection}, falling back to numeric selection")
return None
@@ -813,6 +820,12 @@ async def async_main():
logger.info(f"Selected instrumental file: {selected_instrumental_file}")
# Get countdown padding info from track (if vocals were padded, instrumental must match)
countdown_padding_seconds = None
if track.get("countdown_padding_added", False):
countdown_padding_seconds = track.get("countdown_padding_seconds", 3.0)
logger.info(f"Countdown padding detected: {countdown_padding_seconds}s (will be applied to instrumental if needed)")
# Load CDG styles if CDG generation is enabled
cdg_styles = None
if args.enable_cdg:
@@ -857,6 +870,7 @@ async def async_main():
keep_brand_code=getattr(args, 'keep_brand_code', False),
non_interactive=args.yes,
selected_instrumental_file=selected_instrumental_file,
countdown_padding_seconds=countdown_padding_seconds,
)
try:
+411 -8
View File
@@ -67,6 +67,7 @@ class JobStatus(str, Enum):
UPLOADING = "uploading"
NOTIFYING = "notifying"
COMPLETE = "complete"
PREP_COMPLETE = "prep_complete" # Batch 6: Prep-only jobs stop here
FAILED = "failed"
CANCELLED = "cancelled"
ERROR = "error"
@@ -272,6 +273,9 @@ class RemoteKaraokeClient:
clean_instrumental_model: Optional[str] = None,
backing_vocals_models: Optional[list] = None,
other_stems_models: Optional[list] = None,
# Two-phase workflow (Batch 6)
prep_only: bool = False,
keep_brand_code: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a new karaoke generation job from a YouTube/online URL.
@@ -342,6 +346,11 @@ class RemoteKaraokeClient:
create_request['backing_vocals_models'] = backing_vocals_models
if other_stems_models:
create_request['other_stems_models'] = other_stems_models
# Two-phase workflow (Batch 6)
if prep_only:
create_request['prep_only'] = prep_only
if keep_brand_code:
create_request['keep_brand_code'] = keep_brand_code
self.logger.info(f"Creating URL-based job at {self.config.service_url}/api/jobs/create-from-url")
@@ -397,6 +406,9 @@ class RemoteKaraokeClient:
other_stems_models: Optional[list] = None,
# Existing instrumental (Batch 3)
existing_instrumental: Optional[str] = None,
# Two-phase workflow (Batch 6)
prep_only: bool = False,
keep_brand_code: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a new karaoke generation job with optional style configuration.
@@ -541,6 +553,11 @@ class RemoteKaraokeClient:
create_request['backing_vocals_models'] = backing_vocals_models
if other_stems_models:
create_request['other_stems_models'] = other_stems_models
# Two-phase workflow (Batch 6)
if prep_only:
create_request['prep_only'] = prep_only
if keep_brand_code:
create_request['keep_brand_code'] = keep_brand_code
response = self._request('POST', '/api/jobs/create-with-upload-urls', json=create_request)
@@ -628,6 +645,255 @@ class RemoteKaraokeClient:
return result
def submit_finalise_only_job(
self,
prep_folder: str,
artist: str,
title: str,
enable_cdg: bool = True,
enable_txt: bool = True,
brand_prefix: Optional[str] = None,
keep_brand_code: Optional[str] = None,
discord_webhook_url: Optional[str] = None,
youtube_description: Optional[str] = None,
enable_youtube_upload: bool = False,
dropbox_path: Optional[str] = None,
gdrive_folder_id: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit a finalise-only job with prep output files.
This is used when the user previously ran --prep-only and now wants
to continue with the finalisation phase using cloud resources.
Args:
prep_folder: Path to the prep output folder containing stems, screens, etc.
artist: Artist name
title: Song title
enable_cdg: Generate CDG+MP3 package
enable_txt: Generate TXT+MP3 package
brand_prefix: Brand code prefix (e.g., "NOMAD")
keep_brand_code: Preserve existing brand code from folder name
discord_webhook_url: Discord webhook for notifications
youtube_description: YouTube video description
enable_youtube_upload: Enable YouTube upload
dropbox_path: Dropbox folder path for organized output
gdrive_folder_id: Google Drive folder ID for public share
"""
prep_path = Path(prep_folder)
if not prep_path.exists() or not prep_path.is_dir():
raise FileNotFoundError(f"Prep folder not found: {prep_folder}")
# Detect files in prep folder
files_info = []
local_files = {} # file_type -> local_path
base_name = f"{artist} - {title}"
# Required files - with_vocals video
for ext in ['.mkv', '.mov', '.mp4']:
with_vocals_path = prep_path / f"{base_name} (With Vocals){ext}"
if with_vocals_path.exists():
files_info.append({
'filename': with_vocals_path.name,
'content_type': f'video/{ext[1:]}',
'file_type': 'with_vocals'
})
local_files['with_vocals'] = str(with_vocals_path)
break
if 'with_vocals' not in local_files:
raise FileNotFoundError(f"with_vocals video not found in {prep_folder}")
# Title screen
for ext in ['.mov', '.mkv', '.mp4']:
title_path = prep_path / f"{base_name} (Title){ext}"
if title_path.exists():
files_info.append({
'filename': title_path.name,
'content_type': f'video/{ext[1:]}',
'file_type': 'title_screen'
})
local_files['title_screen'] = str(title_path)
break
if 'title_screen' not in local_files:
raise FileNotFoundError(f"title_screen video not found in {prep_folder}")
# End screen
for ext in ['.mov', '.mkv', '.mp4']:
end_path = prep_path / f"{base_name} (End){ext}"
if end_path.exists():
files_info.append({
'filename': end_path.name,
'content_type': f'video/{ext[1:]}',
'file_type': 'end_screen'
})
local_files['end_screen'] = str(end_path)
break
if 'end_screen' not in local_files:
raise FileNotFoundError(f"end_screen video not found in {prep_folder}")
# Instrumentals (at least one required)
stems_dir = prep_path / 'stems'
if stems_dir.exists():
for stem_file in stems_dir.iterdir():
if 'Instrumental' in stem_file.name and stem_file.suffix.lower() == '.flac':
if '+BV' not in stem_file.name:
if 'instrumental_clean' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_clean'
})
local_files['instrumental_clean'] = str(stem_file)
elif '+BV' in stem_file.name:
if 'instrumental_backing' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_backing'
})
local_files['instrumental_backing'] = str(stem_file)
# Also check root for instrumental files
for stem_file in prep_path.iterdir():
if 'Instrumental' in stem_file.name and stem_file.suffix.lower() == '.flac':
if '+BV' not in stem_file.name and 'instrumental_clean' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_clean'
})
local_files['instrumental_clean'] = str(stem_file)
elif '+BV' in stem_file.name and 'instrumental_backing' not in local_files:
files_info.append({
'filename': stem_file.name,
'content_type': 'audio/flac',
'file_type': 'instrumental_backing'
})
local_files['instrumental_backing'] = str(stem_file)
if 'instrumental_clean' not in local_files and 'instrumental_backing' not in local_files:
raise FileNotFoundError(f"No instrumental file found in {prep_folder}")
# Optional files - LRC
lrc_path = prep_path / f"{base_name} (Karaoke).lrc"
if lrc_path.exists():
files_info.append({
'filename': lrc_path.name,
'content_type': 'text/plain',
'file_type': 'lrc'
})
local_files['lrc'] = str(lrc_path)
# Optional - Title/End JPG/PNG
for img_type, file_type in [('Title', 'title'), ('End', 'end')]:
for ext in ['.jpg', '.png']:
img_path = prep_path / f"{base_name} ({img_type}){ext}"
if img_path.exists():
files_info.append({
'filename': img_path.name,
'content_type': f'image/{ext[1:]}',
'file_type': f'{file_type}_{ext[1:]}'
})
local_files[f'{file_type}_{ext[1:]}'] = str(img_path)
self.logger.info(f"Found {len(files_info)} files in prep folder")
for file_type in local_files:
self.logger.info(f" {file_type}: {Path(local_files[file_type]).name}")
# Create finalise-only job
create_request = {
'artist': artist,
'title': title,
'files': files_info,
'enable_cdg': enable_cdg,
'enable_txt': enable_txt,
}
if brand_prefix:
create_request['brand_prefix'] = brand_prefix
if keep_brand_code:
create_request['keep_brand_code'] = keep_brand_code
if discord_webhook_url:
create_request['discord_webhook_url'] = discord_webhook_url
if youtube_description:
create_request['youtube_description'] = youtube_description
if enable_youtube_upload:
create_request['enable_youtube_upload'] = enable_youtube_upload
if dropbox_path:
create_request['dropbox_path'] = dropbox_path
if gdrive_folder_id:
create_request['gdrive_folder_id'] = gdrive_folder_id
self.logger.info(f"Creating finalise-only job at {self.config.service_url}/api/jobs/create-finalise-only")
response = self._request('POST', '/api/jobs/create-finalise-only', json=create_request)
if response.status_code != 200:
try:
error_detail = response.json()
except Exception:
error_detail = response.text
raise RuntimeError(f"Error creating finalise-only job: {error_detail}")
create_result = response.json()
if create_result.get('status') != 'success':
raise RuntimeError(f"Error creating finalise-only job: {create_result}")
job_id = create_result['job_id']
upload_urls = create_result['upload_urls']
self.logger.info(f"Job {job_id} created. Uploading {len(upload_urls)} files directly to storage...")
# Upload each file
uploaded_files = []
for url_info in upload_urls:
file_type = url_info['file_type']
signed_url = url_info['upload_url']
content_type = url_info['content_type']
local_path = local_files.get(file_type)
if not local_path:
self.logger.warning(f"No local file found for file_type: {file_type}")
continue
file_size = os.path.getsize(local_path)
file_size_mb = file_size / (1024 * 1024)
self.logger.info(f" Uploading {file_type} ({file_size_mb:.1f} MB)...")
success = self._upload_file_to_signed_url(signed_url, local_path, content_type)
if not success:
raise RuntimeError(f"Failed to upload {file_type} to storage")
uploaded_files.append(file_type)
self.logger.info(f" ✓ Uploaded {file_type}")
# Mark uploads complete
self.logger.info(f"Notifying backend that uploads are complete...")
complete_request = {
'uploaded_files': uploaded_files
}
response = self._request('POST', f'/api/jobs/{job_id}/finalise-uploads-complete', json=complete_request)
if response.status_code != 200:
try:
error_detail = response.json()
except Exception:
error_detail = response.text
raise RuntimeError(f"Error completing finalise-only uploads: {error_detail}")
result = response.json()
if result.get('status') != 'success':
raise RuntimeError(f"Error completing finalise-only uploads: {result}")
return result
def get_job(self, job_id: str) -> Dict[str, Any]:
"""Get job status and details."""
response = self._request('GET', f'/api/jobs/{job_id}')
@@ -1054,6 +1320,7 @@ class JobMonitor:
'uploading': 'Uploading to distribution services',
'notifying': 'Sending notifications',
'complete': 'All processing complete',
'prep_complete': 'Prep phase complete - ready for local finalisation',
'failed': 'Job failed',
'cancelled': 'Job cancelled',
}
@@ -1809,6 +2076,24 @@ class JobMonitor:
self.download_outputs(job_id, job_data)
return 0
elif status == 'prep_complete':
self.logger.info("")
self.logger.info("=" * 60)
self.logger.info("PREP PHASE COMPLETE!")
self.logger.info("=" * 60)
self.logger.info(f"Track: {artist} - {title}")
self.logger.info("")
self.logger.info("Downloading all prep outputs...")
self.download_outputs(job_id, job_data)
self.logger.info("")
self.logger.info("To continue with finalisation, run:")
# Use shlex.quote for proper shell escaping of artist/title
import shlex
escaped_artist = shlex.quote(artist)
escaped_title = shlex.quote(title)
self.logger.info(f" karaoke-gen-remote --finalise-only ./<output_folder> {escaped_artist} {escaped_title}")
return 0
elif status in ['failed', 'error']:
self.logger.info("")
self.logger.error("=" * 60)
@@ -1879,9 +2164,6 @@ def main():
"""Main entry point for the remote CLI."""
# Set up logging - same format as gen_cli.py
logger = logging.getLogger(__name__)
# Prevent log propagation to root logger to avoid duplicate logs
# when external packages (like lyrics_converter) configure root logger handlers
logger.propagate = False
log_handler = logging.StreamHandler()
log_formatter = logging.Formatter(
fmt="%(asctime)s.%(msecs)03d - %(levelname)s - %(module)s - %(message)s",
@@ -2200,10 +2482,112 @@ def main():
logger.error(f"Error deleting job: {e}")
return 1
# Warn about unsupported features
# Handle finalise-only mode (Batch 6)
if args.finalise_only:
logger.error("--finalise-only is not supported in remote mode")
return 1
logger.info("=" * 60)
logger.info("Karaoke Generator (Remote) - Finalise Only Mode")
logger.info("=" * 60)
# For finalise-only, we expect the current directory to be the prep output folder
# OR a folder path as the first argument
prep_folder = "."
artist_arg_idx = 0
if args.args:
# Check if first argument is a directory
if os.path.isdir(args.args[0]):
prep_folder = args.args[0]
artist_arg_idx = 1
# Get artist and title from arguments
if len(args.args) > artist_arg_idx + 1:
artist = args.args[artist_arg_idx]
title = args.args[artist_arg_idx + 1]
elif len(args.args) > artist_arg_idx:
logger.error("Finalise-only mode requires both Artist and Title")
return 1
else:
# Try to extract from folder name
folder_name = os.path.basename(os.path.abspath(prep_folder))
parts = folder_name.split(" - ", 2)
if len(parts) >= 2:
# Format: "BRAND-XXXX - Artist - Title" or "Artist - Title"
if "-" in parts[0] and parts[0].split("-")[1].isdigit():
# Has brand code
artist = parts[1] if len(parts) > 2 else "Unknown"
title = parts[2] if len(parts) > 2 else parts[1]
else:
artist = parts[0]
title = parts[1]
logger.info(f"Extracted from folder name: {artist} - {title}")
else:
logger.error("Could not extract Artist and Title from folder name")
logger.error("Please provide: karaoke-gen-remote --finalise-only <folder> \"Artist\" \"Title\"")
return 1
else:
logger.error("Finalise-only mode requires folder path and/or Artist and Title")
return 1
# Extract brand code from folder name if --keep-brand-code is set
keep_brand_code = None
if getattr(args, 'keep_brand_code', False):
folder_name = os.path.basename(os.path.abspath(prep_folder))
parts = folder_name.split(" - ", 1)
if parts and "-" in parts[0]:
# Check if it's a brand code format (e.g., "NOMAD-1234")
potential_brand = parts[0]
brand_parts = potential_brand.split("-")
if len(brand_parts) == 2 and brand_parts[1].isdigit():
keep_brand_code = potential_brand
logger.info(f"Preserving brand code: {keep_brand_code}")
logger.info(f"Prep folder: {os.path.abspath(prep_folder)}")
logger.info(f"Artist: {artist}")
logger.info(f"Title: {title}")
if keep_brand_code:
logger.info(f"Brand Code: {keep_brand_code} (preserved)")
logger.info("")
# Read youtube description from file if provided
youtube_description = None
if args.youtube_description_file and os.path.isfile(args.youtube_description_file):
try:
with open(args.youtube_description_file, 'r') as f:
youtube_description = f.read()
except Exception as e:
logger.warning(f"Failed to read YouTube description file: {e}")
try:
result = client.submit_finalise_only_job(
prep_folder=prep_folder,
artist=artist,
title=title,
enable_cdg=args.enable_cdg,
enable_txt=args.enable_txt,
brand_prefix=args.brand_prefix,
keep_brand_code=keep_brand_code,
discord_webhook_url=args.discord_webhook_url,
youtube_description=youtube_description,
enable_youtube_upload=getattr(args, 'enable_youtube_upload', False),
dropbox_path=getattr(args, 'dropbox_path', None),
gdrive_folder_id=getattr(args, 'gdrive_folder_id', None),
)
job_id = result.get('job_id')
logger.info(f"Finalise-only job submitted: {job_id}")
logger.info("")
# Monitor job
return monitor.monitor(job_id)
except FileNotFoundError as e:
logger.error(str(e))
return 1
except RuntimeError as e:
logger.error(str(e))
return 1
except Exception as e:
logger.error(f"Error: {e}")
return 1
if args.edit_lyrics:
logger.error("--edit-lyrics is not yet supported in remote mode")
@@ -2215,8 +2599,7 @@ def main():
# Warn about features that are not yet supported in remote mode
ignored_features = []
if args.prep_only:
ignored_features.append("--prep-only")
# Note: --prep-only is now supported in remote mode (Batch 6)
if args.skip_separation:
ignored_features.append("--skip-separation")
if args.skip_transcription:
@@ -2421,6 +2804,8 @@ def main():
logger.info(f"Other Stems Models: {args.other_stems_models}")
if getattr(args, 'existing_instrumental', None):
logger.info(f"Existing Instrumental: {args.existing_instrumental}")
if getattr(args, 'prep_only', False):
logger.info(f"Mode: prep-only (will stop after review)")
logger.info(f"Service URL: {config.service_url}")
logger.info(f"Review UI: {config.review_ui_url}")
if config.non_interactive:
@@ -2437,6 +2822,18 @@ def main():
except Exception as e:
logger.warning(f"Failed to read YouTube description file: {e}")
# Extract brand code from current directory if --keep-brand-code is set
keep_brand_code_value = None
if getattr(args, 'keep_brand_code', False):
cwd_name = os.path.basename(os.getcwd())
parts = cwd_name.split(" - ", 1)
if parts and "-" in parts[0]:
potential_brand = parts[0]
brand_parts = potential_brand.split("-")
if len(brand_parts) == 2 and brand_parts[1].isdigit():
keep_brand_code_value = potential_brand
logger.info(f"Preserving brand code: {keep_brand_code_value}")
try:
# Submit job - different endpoint for URL vs file
if is_url_input:
@@ -2469,6 +2866,9 @@ def main():
clean_instrumental_model=getattr(args, 'clean_instrumental_model', None),
backing_vocals_models=getattr(args, 'backing_vocals_models', None),
other_stems_models=getattr(args, 'other_stems_models', None),
# Two-phase workflow (Batch 6)
prep_only=getattr(args, 'prep_only', False),
keep_brand_code=keep_brand_code_value,
)
else:
# File-based job submission
@@ -2498,6 +2898,9 @@ def main():
other_stems_models=getattr(args, 'other_stems_models', None),
# Existing instrumental (Batch 3)
existing_instrumental=getattr(args, 'existing_instrumental', None),
# Two-phase workflow (Batch 6)
prep_only=getattr(args, 'prep_only', False),
keep_brand_code=keep_brand_code_value,
)
job_id = result.get('job_id')
style_assets = result.get('style_assets_uploaded', [])
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "karaoke-gen"
version = "0.71.67"
version = "0.72.0"
description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens."
authors = ["Andrew Beveridge <andrew@beveridge.uk>"]
license = "MIT"
+260
View File
@@ -435,6 +435,266 @@ class TestExistingInstrumentalPadding:
assert '3.0' in call_args or 3.0 in call_args
class TestDetectCountdownPaddingFromLRC:
"""Tests for detecting countdown padding from existing LRC files."""
def test_detect_padding_from_lrc_with_padding(self, basic_karaoke_gen, temp_dir):
"""Test that countdown padding is detected when first lyric is after 3 seconds."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at 3 seconds (padding was applied)
lrc_content = """[ti:Test Song]
[ar:Test Artist]
[00:03.50]First line of lyrics
[00:06.20]Second line of lyrics
[00:10.00]Third line of lyrics
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is True
assert padding_seconds == 3.0
def test_detect_padding_from_lrc_without_padding(self, basic_karaoke_gen, temp_dir):
"""Test that no padding is detected when first lyric is before 3 seconds."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at 0.5 seconds (no padding)
lrc_content = """[ti:Test Song]
[ar:Test Artist]
[00:00.50]First line of lyrics
[00:03.20]Second line of lyrics
[00:06.00]Third line of lyrics
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_edge_case(self, basic_karaoke_gen, temp_dir):
"""Test detection at the 2.5 second boundary."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at exactly 2.5 seconds (boundary)
lrc_content = """[00:02.50]First line at boundary
[00:05.00]Second line
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
# 2.5 is the boundary - should be detected as padding
assert padding_added is True
assert padding_seconds == 3.0
def test_detect_padding_from_lrc_just_below_boundary(self, basic_karaoke_gen, temp_dir):
"""Test detection just below the 2.5 second boundary."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Create an LRC file with first lyric at 2.4 seconds (just below boundary)
lrc_content = """[00:02.40]First line just below boundary
[00:05.00]Second line
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
# 2.4 is below boundary - should NOT be detected as padding
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_empty_file(self, basic_karaoke_gen, temp_dir):
"""Test handling of empty LRC file."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
with open(lrc_file, 'w') as f:
f.write('')
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_no_timestamps(self, basic_karaoke_gen, temp_dir):
"""Test handling of LRC file with no valid timestamps."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
lrc_content = """[ti:Test Song]
[ar:Test Artist]
Just text without timestamps
More text
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_missing_file(self, basic_karaoke_gen, temp_dir):
"""Test handling of missing LRC file."""
lrc_file = os.path.join(temp_dir, 'nonexistent.lrc')
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is False
assert padding_seconds == 0.0
def test_detect_padding_from_lrc_three_digit_milliseconds(self, basic_karaoke_gen, temp_dir):
"""Test parsing LRC files with three-digit milliseconds."""
lrc_file = os.path.join(temp_dir, 'test.lrc')
# Some LRC files use [mm:ss.xxx] format
lrc_content = """[00:03.500]First line at 3.5 seconds
[00:06.200]Second line
"""
with open(lrc_file, 'w') as f:
f.write(lrc_content)
padding_added, padding_seconds = basic_karaoke_gen.lyrics_processor._detect_countdown_padding_from_lrc(lrc_file)
assert padding_added is True
assert padding_seconds == 3.0
class TestExistingFilesWithPaddingDetection:
"""Tests for transcribe_lyrics returning padding info from existing files."""
def test_transcribe_lyrics_existing_files_returns_padding_info(self, basic_karaoke_gen, temp_dir):
"""Test that transcribe_lyrics returns padding info when using existing files."""
track_output_dir = temp_dir
artist = "Test Artist"
title = "Test Title"
# Create existing files
video_path = os.path.join(track_output_dir, f"{artist} - {title} (With Vocals).mkv")
lrc_path = os.path.join(track_output_dir, f"{artist} - {title} (Karaoke).lrc")
# Create the video file
with open(video_path, 'w') as f:
f.write('mock video')
# Create LRC file with countdown padding (first lyric at 3.5s)
lrc_content = """[00:03.50]First line with padding
[00:06.00]Second line
"""
with open(lrc_path, 'w') as f:
f.write(lrc_content)
# Call transcribe_lyrics - it should detect existing files and return padding info
result = basic_karaoke_gen.lyrics_processor.transcribe_lyrics(
input_audio_wav=None, # Not needed when existing files found
artist=artist,
title=title,
track_output_dir=track_output_dir,
)
# Should return padding info from LRC detection
assert result['countdown_padding_added'] is True
assert result['countdown_padding_seconds'] == 3.0
assert result['padded_audio_filepath'] is None # Original padded audio not available
def test_transcribe_lyrics_existing_files_no_padding(self, basic_karaoke_gen, temp_dir):
"""Test existing files case when there's no countdown padding."""
track_output_dir = temp_dir
artist = "Test Artist"
title = "Test Title"
# Create existing files
video_path = os.path.join(track_output_dir, f"{artist} - {title} (With Vocals).mkv")
lrc_path = os.path.join(track_output_dir, f"{artist} - {title} (Karaoke).lrc")
# Create the video file
with open(video_path, 'w') as f:
f.write('mock video')
# Create LRC file WITHOUT countdown padding (first lyric at 0.5s)
lrc_content = """[00:00.50]First line starts immediately
[00:03.00]Second line
"""
with open(lrc_path, 'w') as f:
f.write(lrc_content)
result = basic_karaoke_gen.lyrics_processor.transcribe_lyrics(
input_audio_wav=None,
artist=artist,
title=title,
track_output_dir=track_output_dir,
)
# Should return no padding info
assert result['countdown_padding_added'] is False
assert result['countdown_padding_seconds'] == 0.0
class TestScanDirectoryForInstrumentals:
"""Tests for scanning directory for existing instrumentals."""
def test_scan_directory_finds_clean_instrumental(self, basic_karaoke_gen, temp_dir):
"""Test that _scan_directory_for_instrumentals finds clean instrumentals."""
artist_title = "Test Artist - Test Song"
# Create a clean instrumental file
instrumental_file = os.path.join(temp_dir, f"{artist_title} (Instrumental Clean).flac")
with open(instrumental_file, 'w') as f:
f.write('mock instrumental')
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
assert result['clean_instrumental']['instrumental'] == instrumental_file
def test_scan_directory_finds_combined_instrumentals(self, basic_karaoke_gen, temp_dir):
"""Test that _scan_directory_for_instrumentals finds combined instrumentals."""
artist_title = "Test Artist - Test Song"
# Create combined instrumental files
bv_file = os.path.join(temp_dir, f"{artist_title} (Instrumental +BV HTDemucs).flac")
with open(bv_file, 'w') as f:
f.write('mock instrumental with backing')
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
assert 'HTDemucs' in result['combined_instrumentals']
assert result['combined_instrumentals']['HTDemucs'] == bv_file
def test_scan_directory_skips_padded_files(self, basic_karaoke_gen, temp_dir):
"""Test that _scan_directory_for_instrumentals skips already-padded files."""
artist_title = "Test Artist - Test Song"
# Create both original and padded files
original_file = os.path.join(temp_dir, f"{artist_title} (Instrumental Clean).flac")
padded_file = os.path.join(temp_dir, f"{artist_title} (Instrumental Clean) (Padded).flac")
with open(original_file, 'w') as f:
f.write('mock original')
with open(padded_file, 'w') as f:
f.write('mock padded')
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
# Should return original, not padded
assert result['clean_instrumental']['instrumental'] == original_file
assert 'Padded' not in result['clean_instrumental']['instrumental']
def test_scan_directory_empty_directory(self, basic_karaoke_gen, temp_dir):
"""Test _scan_directory_for_instrumentals on empty directory."""
artist_title = "Test Artist - Test Song"
result = basic_karaoke_gen._scan_directory_for_instrumentals(temp_dir, artist_title)
# Should return empty structure
assert result['clean_instrumental'].get('instrumental') is None
assert len(result['combined_instrumentals']) == 0
if __name__ == '__main__':
pytest.main([__file__, '-v'])
@@ -299,7 +299,7 @@ def test_check_if_video_title_exists_found_confirm(mock_input, mock_fuzz_ratio,
found_video_id = "existing_video_id"
found_video_title = f"{ARTIST} - {TITLE} (Karaoke) Official"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 85 # High similarity
@@ -322,7 +322,7 @@ def test_check_if_video_title_exists_found_reject(mock_input, mock_fuzz_ratio, m
found_video_id = "existing_video_id"
found_video_title = f"{ARTIST} - {TITLE} (Karaoke) Official"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 85
@@ -344,7 +344,7 @@ def test_check_if_video_title_exists_low_similarity(mock_fuzz_ratio, mock_auth,
found_video_id = "existing_video_id"
found_video_title = "Completely Different Title"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 30 # Low similarity
@@ -367,7 +367,7 @@ def test_check_if_video_title_exists_non_interactive(mock_input, mock_fuzz_ratio
found_video_id = "existing_video_id"
found_video_title = f"{ARTIST} - {TITLE} (Karaoke) Official"
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title}}]
"items": [{"id": {"videoId": found_video_id}, "snippet": {"title": found_video_title, "channelId": "UC_test_channel_id"}}]
}
mock_fuzz_ratio.return_value = 85
@@ -392,6 +392,63 @@ def test_check_if_video_title_exists_not_found(mock_auth, finaliser_for_yt, mock
assert finaliser_for_yt.youtube_url is None # Check initial state remains None
assert finaliser_for_yt.skip_notifications is False
@patch.object(KaraokeFinalise, 'authenticate_youtube')
@patch('thefuzz.fuzz.ratio')
@patch('builtins.input')
def test_check_if_video_title_exists_filters_other_channels(mock_input, mock_fuzz_ratio, mock_auth, finaliser_for_yt, mock_youtube_service):
"""Test that videos from other channels are filtered out even if returned by search API.
The YouTube search API sometimes returns results from other channels even when
channelId parameter is specified. This test verifies we filter those out.
"""
mock_auth.return_value = mock_youtube_service
youtube_title = f"{ARTIST} - {TITLE} (Karaoke)"
# Simulate YouTube API bug: returns videos from other channels despite channelId filter
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [
# Video from another channel - should be skipped
{"id": {"videoId": "other_channel_video_1"}, "snippet": {"title": f"{ARTIST} - {TITLE}", "channelId": "UC_other_channel_1"}},
{"id": {"videoId": "other_channel_video_2"}, "snippet": {"title": f"{ARTIST} - {TITLE} (Official)", "channelId": "UC_other_channel_2"}},
# Video from user's channel - should be matched
{"id": {"videoId": "my_channel_video"}, "snippet": {"title": f"{ARTIST} - {TITLE} (Karaoke)", "channelId": "UC_test_channel_id"}},
]
}
mock_fuzz_ratio.return_value = 95
mock_input.return_value = 'y'
exists = finaliser_for_yt.check_if_video_title_exists_on_youtube_channel(youtube_title)
assert exists is True
# Should match the video from our channel, not the ones from other channels
assert finaliser_for_yt.youtube_video_id == "my_channel_video"
# fuzz.ratio should only be called for videos from our channel (1 video)
assert mock_fuzz_ratio.call_count == 1
@patch.object(KaraokeFinalise, 'authenticate_youtube')
@patch('thefuzz.fuzz.ratio')
def test_check_if_video_title_exists_all_from_other_channels(mock_fuzz_ratio, mock_auth, finaliser_for_yt, mock_youtube_service):
"""Test that if all returned videos are from other channels, no match is found."""
mock_auth.return_value = mock_youtube_service
youtube_title = f"{ARTIST} - {TITLE} (Karaoke)"
# All results are from other channels
mock_youtube_service.search.return_value.list.return_value.execute.return_value = {
"items": [
{"id": {"videoId": "other_channel_video_1"}, "snippet": {"title": f"{ARTIST} - {TITLE}", "channelId": "UC_other_channel_1"}},
{"id": {"videoId": "other_channel_video_2"}, "snippet": {"title": f"{ARTIST} - {TITLE} (Official)", "channelId": "UC_other_channel_2"}},
]
}
exists = finaliser_for_yt.check_if_video_title_exists_on_youtube_channel(youtube_title)
assert exists is False
# fuzz.ratio should never be called since all videos are filtered out
mock_fuzz_ratio.assert_not_called()
# Should log debug messages about skipping videos from other channels
assert finaliser_for_yt.logger.debug.call_count >= 2
# --- Delete / Upload Tests ---
@patch.object(KaraokeFinalise, 'authenticate_youtube')
+5 -2
View File
@@ -74,6 +74,7 @@ class TestJobStatus:
assert JobStatus.UPLOADING == "uploading"
assert JobStatus.NOTIFYING == "notifying"
assert JobStatus.COMPLETE == "complete"
assert JobStatus.PREP_COMPLETE == "prep_complete" # Batch 6: Prep-only jobs stop here
assert JobStatus.FAILED == "failed"
assert JobStatus.CANCELLED == "cancelled"
assert JobStatus.ERROR == "error"
@@ -963,10 +964,10 @@ class TestMain:
@patch('karaoke_gen.utils.remote_cli.create_parser')
@patch('karaoke_gen.utils.remote_cli.get_auth_token')
@patch('karaoke_gen.utils.remote_cli.check_prerequisites')
def test_main_finalise_only_not_supported(
def test_main_finalise_only_requires_folder_and_args(
self, mock_check, mock_auth, mock_create_parser
):
"""Test main rejects --finalise-only in remote mode."""
"""Test main --finalise-only requires folder and artist/title args (Batch 6)."""
mock_parser = MagicMock()
mock_args = MagicMock()
mock_args.service_url = "https://api.example.com"
@@ -984,12 +985,14 @@ class TestMain:
mock_args.filter_client_id = None
mock_args.environment = ""
mock_args.client_id = ""
mock_args.args = [] # No args provided - should fail
mock_parser.parse_args.return_value = mock_args
mock_create_parser.return_value = mock_parser
mock_auth.return_value = "token"
result = main()
# Should fail because no folder/artist/title provided
assert result == 1
@patch('karaoke_gen.utils.remote_cli.create_parser')