diff --git a/.claude/plans/fix-youtube-remote-download.plan.md b/.claude/plans/fix-youtube-remote-download.plan.md new file mode 100644 index 00000000..dc0c4609 --- /dev/null +++ b/.claude/plans/fix-youtube-remote-download.plan.md @@ -0,0 +1,166 @@ +# Plan: Fix YouTube Remote Download Missing GCS Path + +**Created:** 2026-01-20 +**Branch:** feat/sess-20260120-1756-fix-youtube-remote-download +**Status:** Complete + +## Overview + +When remote flacfetch is enabled and a user selects a YouTube audio source, the backend calls `download_by_id()` without the `gcs_path` parameter. This causes flacfetch to download the file locally on the VM instead of uploading to GCS. The backend then tries to open the VM's local path, which doesn't exist on Cloud Run, causing the job to fail. + +This is the same class of bug that was fixed on 2025-12-22 for RED/OPS torrent sources (see `docs/archive/2025-12-phase1-testing-gaps.md`), but the fix only covered torrent sources, not YouTube. + +## Root Cause + +In `backend/api/routes/audio_search.py` lines 354-363: + +```python +# Use download_by_id if we have source_id and remote is enabled +if source_id and source_name and is_remote_enabled: + logger.info(f"Using download_by_id for local download: {source_name} ID={source_id}") + + result = audio_search_service.download_by_id( + source_name=source_name, + source_id=source_id, + output_dir=temp_dir, # <-- This is a local temp dir, but flacfetch runs on VM! + target_file=target_file, + download_url=download_url, + # <-- MISSING: gcs_path parameter + ) +``` + +When `is_remote_enabled=True`, `download_by_id()` routes to the remote flacfetch VM. Without `gcs_path`, the VM downloads locally and returns a local path like `/var/lib/transmission-daemon/downloads/...` which doesn't exist on Cloud Run. + +## Requirements + +- [x] YouTube downloads with remote flacfetch should upload directly to GCS +- [x] Backend should receive a `gs://` path from flacfetch +- [x] No changes to flacfetch itself (it already supports `upload_to_gcs` and `gcs_destination`) +- [x] Maintain backwards compatibility (local-only deployments should still work) +- [x] Add test coverage for this code path + +## Technical Approach + +**Option A (chosen):** Pass `gcs_path` to `download_by_id()` for ALL remote downloads, not just torrent sources. + +The current code structure has two branches: +1. `if is_torrent_source and is_remote_enabled:` - handles RED/OPS with GCS upload ✅ +2. `else:` branch - handles YouTube, but incorrectly assumes "local download" even when remote is enabled + +The fix refactors to check `is_remote_enabled` first, then handle torrent vs YouTube within that: +- If remote enabled: always pass `gcs_path` to let flacfetch upload directly +- If remote disabled: download locally and upload to GCS from Cloud Run + +## Implementation Steps + +1. [x] Refactor the `else` branch in `_download_and_start_processing()` to check for remote-enabled YouTube downloads +2. [x] Pass `gcs_path` when calling `download_by_id()` for remote YouTube downloads +3. [x] Add test case for YouTube with remote enabled to verify `gcs_path` is passed +4. [x] Update logging to clarify the download path being used +5. [x] Bump version in `pyproject.toml` +6. [x] Run tests to verify no regressions + +## Files to Create/Modify + +| File | Action | Description | +|------|--------|-------------| +| `backend/api/routes/audio_search.py` | Modify | Add `gcs_path` parameter for remote YouTube downloads | +| `backend/tests/test_audio_search.py` | Modify | Add test case for YouTube remote download with GCS path | +| `pyproject.toml` | Modify | Bump version | + +## Detailed Code Changes + +### audio_search.py (lines 349-393) + +Current structure: +```python +else: + # Local download (YouTube or fallback) <-- WRONG: not always local! + temp_dir = tempfile.mkdtemp(...) + + if source_id and source_name and is_remote_enabled: + result = audio_search_service.download_by_id( + ... + output_dir=temp_dir, # <-- BUG: unused for remote + # Missing gcs_path! + ) +``` + +Fixed structure: +```python +else: + # Non-torrent download (YouTube or fallback) + + if is_remote_enabled and source_id and source_name: + # Remote download via flacfetch VM - upload directly to GCS + gcs_destination = f"uploads/{job_id}/audio/" + logger.info(f"Using remote download_by_id for {source_name} ID={source_id} with GCS upload to: {gcs_destination}") + + result = audio_search_service.download_by_id( + source_name=source_name, + source_id=source_id, + output_dir="", # Not used for remote + target_file=target_file, + download_url=download_url, + gcs_path=gcs_destination, # <-- FIX: Add this! + ) + + # Handle GCS path response (same logic as torrent branch) + if result.filepath.startswith("gs://"): + parts = result.filepath.replace("gs://", "").split("/", 1) + if len(parts) == 2: + audio_gcs_path = parts[1] + else: + audio_gcs_path = result.filepath + filename = os.path.basename(result.filepath) + else: + # Fallback: upload manually + filename = os.path.basename(result.filepath) + audio_gcs_path = f"uploads/{job_id}/audio/{filename}" + logger.warning(f"Remote download returned local path: {result.filepath}, uploading manually") + with open(result.filepath, 'rb') as f: + storage_service.upload_fileobj(f, audio_gcs_path, content_type='audio/flac') + + logger.info(f"Remote download complete, GCS path: {audio_gcs_path}") + else: + # Local download (remote disabled or no source_id) + temp_dir = tempfile.mkdtemp(prefix=f"audio_download_{job_id}_") + # ... existing local download logic ... +``` + +## Testing Strategy + +### Unit Tests +Add to `test_audio_search.py` in `TestAudioSearchApiRouteDownload`: + +```python +def test_youtube_remote_download_passes_gcs_path(self, mock_job_manager, mock_storage_service): + """ + Test that YouTube downloads via remote flacfetch include GCS path. + + This is the same bug pattern as the torrent download bug fixed in 2025-12: + when remote is enabled, we must pass gcs_path so flacfetch uploads to GCS. + """ + # Setup: YouTube source with remote enabled + # Verify: download_by_id called with gcs_path parameter +``` + +### Manual Testing (post-deploy) +1. Create a job and select a YouTube audio source +2. Verify job completes successfully +3. Check GCS path is properly set in job data + +## Open Questions + +None - the fix is straightforward and follows the pattern established for torrent sources. + +## Rollback Plan + +This is a minor code change. If issues occur: +1. Revert the PR +2. The failure mode returns to the current state (YouTube downloads fail when remote is enabled) + +## Related Documentation + +- `docs/archive/2025-12-phase1-testing-gaps.md` - Original bug fix for torrent sources +- `docs/LESSONS-LEARNED.md` - "Fix Both Sides of Dual Code Paths" pattern diff --git a/backend/api/routes/audio_search.py b/backend/api/routes/audio_search.py index ae8daa5d..32a94ef1 100644 --- a/backend/api/routes/audio_search.py +++ b/backend/api/routes/audio_search.py @@ -13,6 +13,7 @@ The flow is: """ import asyncio import logging +import mimetypes import os import tempfile from typing import Optional, List, Dict, Any @@ -342,26 +343,52 @@ async def _download_and_start_processing( audio_gcs_path = f"uploads/{job_id}/audio/{filename}" logger.warning(f"Remote download returned local path: {result.filepath}, uploading manually") + content_type, _ = mimetypes.guess_type(result.filepath) with open(result.filepath, 'rb') as f: - storage_service.upload_fileobj(f, audio_gcs_path, content_type='audio/flac') + storage_service.upload_fileobj(f, audio_gcs_path, content_type=content_type or 'application/octet-stream') + + logger.info(f"Remote download complete, GCS path: {audio_gcs_path}") + elif is_remote_enabled and source_id and source_name: + # Non-torrent remote download (e.g., YouTube via flacfetch VM) + # Must pass gcs_path so flacfetch uploads directly to GCS + gcs_destination = f"uploads/{job_id}/audio/" + logger.info(f"Using remote download_by_id for {source_name} ID={source_id} with GCS upload to: {gcs_destination}") + + result = audio_search_service.download_by_id( + source_name=source_name, + source_id=source_id, + output_dir="", # Not used for remote + target_file=target_file, + download_url=download_url, + gcs_path=gcs_destination, + ) + + # Handle GCS path response (same logic as torrent branch) + if result.filepath.startswith("gs://"): + # Extract the path portion after the bucket name + # Format: gs://bucket/uploads/job_id/audio/filename.flac + parts = result.filepath.replace("gs://", "").split("/", 1) + if len(parts) == 2: + audio_gcs_path = parts[1] + else: + audio_gcs_path = result.filepath + filename = os.path.basename(result.filepath) + else: + # Fallback: treat as local path (shouldn't happen for remote) + filename = os.path.basename(result.filepath) + audio_gcs_path = f"uploads/{job_id}/audio/{filename}" + + logger.warning(f"Remote download returned local path: {result.filepath}, uploading manually") + content_type, _ = mimetypes.guess_type(result.filepath) + with open(result.filepath, 'rb') as f: + storage_service.upload_fileobj(f, audio_gcs_path, content_type=content_type or 'application/octet-stream') logger.info(f"Remote download complete, GCS path: {audio_gcs_path}") else: - # Local download (YouTube or fallback) + # Local download (remote disabled or no source_id) temp_dir = tempfile.mkdtemp(prefix=f"audio_download_{job_id}_") - # Use download_by_id if we have source_id and remote is enabled - if source_id and source_name and is_remote_enabled: - logger.info(f"Using download_by_id for local download: {source_name} ID={source_id}") - - result = audio_search_service.download_by_id( - source_name=source_name, - source_id=source_id, - output_dir=temp_dir, - target_file=target_file, - download_url=download_url, - ) - elif source_name == 'YouTube' and download_url: + if source_name == 'YouTube' and download_url: # For YouTube without remote, download directly using URL from state_data # This avoids relying on in-memory cache which doesn't persist across instances logger.info(f"Downloading YouTube audio directly from URL: {download_url}") @@ -396,11 +423,13 @@ async def _download_and_start_processing( filename = os.path.basename(result.filepath) audio_gcs_path = f"uploads/{job_id}/audio/{filename}" + # Detect content type from file extension (YouTube downloads are typically .webm/.opus, torrent downloads are .flac) + content_type, _ = mimetypes.guess_type(result.filepath) with open(result.filepath, 'rb') as f: storage_service.upload_fileobj( f, audio_gcs_path, - content_type='audio/flac' # flacfetch typically returns FLAC + content_type=content_type or 'application/octet-stream' ) logger.info(f"Uploaded audio to GCS: {audio_gcs_path}") diff --git a/backend/tests/test_audio_search.py b/backend/tests/test_audio_search.py index 70267237..259f1e82 100644 --- a/backend/tests/test_audio_search.py +++ b/backend/tests/test_audio_search.py @@ -1219,67 +1219,129 @@ class TestAudioSearchApiRouteDownload: assert 'gcs_path' in call_kwargs assert call_kwargs['gcs_path'] == "uploads/job123/audio/" - def test_local_youtube_download_does_not_pass_gcs_path(self, mock_job_manager, mock_storage_service): - """Test that YouTube downloads don't use gcs_path (download locally, upload manually).""" + def test_youtube_remote_download_passes_gcs_path(self, mock_job_manager, mock_storage_service): + """ + Test that YouTube downloads via remote flacfetch include GCS path. + + This is the same bug pattern as the torrent download bug fixed in 2025-12: + when remote is enabled, we must pass gcs_path so flacfetch uploads to GCS. + Without gcs_path, flacfetch downloads locally on the VM and returns a local + path that Cloud Run can't access. + """ + from backend.api.routes.audio_search import _download_and_start_processing + import asyncio + + # Setup mock job with YouTube search result that has source_id + mock_job = Mock() + mock_job.state_data = { + 'audio_search_results': [{ + 'title': 'Demons', + 'artist': 'Kenny Chesney', + 'provider': 'YouTube', + 'quality': 'Opus 128kbps', + 'source_id': 'qmTjyGH9ro4', # YouTube video ID + 'url': 'https://www.youtube.com/watch?v=qmTjyGH9ro4', + }] + } + mock_job.audio_search_artist = 'Kenny Chesney' + mock_job.audio_search_title = 'Demons' + mock_job_manager.get_job.return_value = mock_job + + # Create mock audio search service with remote enabled + mock_audio_service = Mock() + mock_audio_service.is_remote_enabled.return_value = True + + # Mock download_by_id to return GCS path (as remote flacfetch should) + mock_download_result = Mock() + mock_download_result.filepath = "gs://bucket/uploads/job456/audio/Kenny Chesney - Demons.webm" + mock_audio_service.download_by_id.return_value = mock_download_result + + # Create mock background tasks + mock_bg_tasks = Mock() + + # Run the async function + loop = asyncio.new_event_loop() + try: + result = loop.run_until_complete( + _download_and_start_processing( + job_id="job456", + selection_index=0, + audio_search_service=mock_audio_service, + background_tasks=mock_bg_tasks, + ) + ) + finally: + loop.close() + + # CRITICAL: Verify download_by_id was called with gcs_path for remote YouTube source + mock_audio_service.download_by_id.assert_called_once() + call_kwargs = mock_audio_service.download_by_id.call_args.kwargs + + # The gcs_path should be set for remote YouTube downloads + assert 'gcs_path' in call_kwargs + assert call_kwargs['gcs_path'] == "uploads/job456/audio/" + + # Storage service should NOT be called to upload (flacfetch already uploaded) + mock_storage_service.upload_fileobj.assert_not_called() + + def test_youtube_local_download_when_remote_disabled(self, mock_job_manager, mock_storage_service): + """Test that YouTube downloads locally when remote is disabled.""" from backend.api.routes.audio_search import _download_and_start_processing import asyncio import tempfile - + # Setup mock job with YouTube search result mock_job = Mock() mock_job.state_data = { 'audio_search_results': [{ 'title': 'Unwanted', 'artist': 'Avril Lavigne', - 'provider': 'YouTube', # NOT a torrent source + 'provider': 'YouTube', 'quality': 'Opus 128kbps', + 'url': 'https://www.youtube.com/watch?v=abc123', }] } mock_job.audio_search_artist = 'Avril Lavigne' mock_job.audio_search_title = 'Unwanted' mock_job_manager.get_job.return_value = mock_job - - # Create mock audio search service with remote client (but YouTube doesn't use it) + + # Create mock audio search service with remote DISABLED mock_audio_service = Mock() - mock_audio_service.is_remote_enabled.return_value = True - + mock_audio_service.is_remote_enabled.return_value = False + # Create a temp file to simulate downloaded file temp_file = tempfile.NamedTemporaryFile(suffix='.opus', delete=False) temp_file.write(b'fake audio data') temp_file.close() - + try: - # Mock download to return local path - mock_download_result = Mock() - mock_download_result.filepath = temp_file.name - mock_audio_service.download.return_value = mock_download_result - - # Create mock background tasks - mock_bg_tasks = Mock() - - # Run the async function - loop = asyncio.new_event_loop() - try: - result = loop.run_until_complete( - _download_and_start_processing( - job_id="job456", - selection_index=0, - audio_search_service=mock_audio_service, - background_tasks=mock_bg_tasks, + # Mock download_from_url to return temp path + # Note: download_from_url is imported inside the function, so patch at source + with patch('backend.workers.audio_worker.download_from_url') as mock_download_from_url: + mock_download_from_url.return_value = temp_file.name + + # Create mock background tasks + mock_bg_tasks = Mock() + + # Run the async function + loop = asyncio.new_event_loop() + try: + result = loop.run_until_complete( + _download_and_start_processing( + job_id="job789", + selection_index=0, + audio_search_service=mock_audio_service, + background_tasks=mock_bg_tasks, + ) ) - ) - finally: - loop.close() - - # YouTube should NOT have gcs_path in kwargs - mock_audio_service.download.assert_called_once() - call_kwargs = mock_audio_service.download.call_args.kwargs - - # For YouTube, gcs_path should NOT be set (or should be None) - assert call_kwargs.get('gcs_path') is None - - # Instead, storage service should be called to upload - mock_storage_service.upload_fileobj.assert_called_once() + finally: + loop.close() + + # download_from_url should be called for local YouTube download + mock_download_from_url.assert_called_once() + + # Storage service SHOULD be called to upload (local download) + mock_storage_service.upload_fileobj.assert_called_once() finally: import os try: diff --git a/docs/LESSONS-LEARNED.md b/docs/LESSONS-LEARNED.md index 6dbbd9fb..4aa19274 100644 --- a/docs/LESSONS-LEARNED.md +++ b/docs/LESSONS-LEARNED.md @@ -42,6 +42,8 @@ When multiple code paths create jobs (file upload, audio search, webhooks), use ### Fix Both Sides of Dual Code Paths When fixing a bug in a system with multiple code paths (e.g., legacy vs orchestrator, local vs cloud), verify ALL paths are fixed. PR #271 fixed the GCE worker to READ `instrumental_selection` but only checked the legacy path which was already SENDING it. The orchestrator path (production default) wasn't sending it. **Pattern**: If a component receives config from multiple callers, check ALL callers when fixing the receiving side. Write integration tests that cover each path. +**Example:** The `gcs_path` parameter bug for remote flacfetch downloads was fixed for RED/OPS torrent sources in December 2025, but the same bug existed for YouTube sources. The fix only addressed one branch of the conditional, leaving YouTube downloads broken when remote was enabled. Always search for ALL code paths that might need the same fix. + ### Defense in Depth Enforce critical requirements at multiple layers (e.g., reject at creation in JobManager + safety net at processing time). diff --git a/pyproject.toml b/pyproject.toml index 1c35afd4..010be828 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "karaoke-gen" -version = "0.108.8" +version = "0.108.9" description = "Generate karaoke videos with synchronized lyrics. Handles the entire process from downloading audio and lyrics to creating the final video with title screens." authors = ["Andrew Beveridge "] license = "MIT"