Files
karaoke-gen/backend/services/audio_transcoding_service.py
Andrew Beveridge ☄️andClaude Opus 5.5 455fb11a14 feat(upload): unified browser upload flow, bring-your-own instrumental, stranded-upload cleanup (v0.243.0) (#1058)
## Summary
Two invisible upload failures on the same song led to this:
- **Job 8729f19a**: large audio upload (signed-URL flow) abandoned
mid-upload. The only UI signal was the "Creating..." button text and the
card already said "Setting up", so the job sat `pending` forever with
the credit spent.
- **Job 2d0c1ae0**: 44 MB instrumental upload on the review screen was a
multipart POST through Cloud Run (32 MiB cap) and failed silently. Only
the CORS preflight was logged.

## Changes
**Shared upload layer (frontend)**: all user uploads now go through one
path:
- `lib/upload.ts`: signed-URL PUT with progress, sequential multi-file
upload with aggregate progress, client-side duration check.
- `hooks/useUploadTask.ts`: progress state plus a `beforeunload` guard.
- `components/upload/UploadProgressModal.tsx`: a modal the user can't
close, showing progress, speed, ETA, "file N of M" and a keep-tab-open
warning.
- Used by guided job creation, tenant single job and instrumental
review. Tenant bulk keeps its per-row table and uses the shared guard.

**Guided flow**
- Always uses the signed-URL path, which also fixes
`requires_audio_edit` being silently dropped for uploads of 25MB or
more.
- Private upload jobs can attach their own instrumental
(`existing_instrumental`). This skips separation and instrumental
review. The duration is checked before upload, and the backing-vocals
option is hidden.

**Instrumental review**
- New `instrumental-upload-url` and `instrumental-upload-complete`
endpoints: direct-to-GCS upload, then a finalize step.
- One shared processing helper with the multipart route: duration check,
FLAC conversion, `stems.custom_instrumental`.
- Returns a signed `audio_url`. The old post-upload preview URL had no
backend route.
- Inline error plus toast for mismatches and failures.

**Backend robustness**
- `state_data.awaiting_upload`: the dashboard shows "Waiting for upload"
instead of "Setting up".
- The hourly stale-review run also cancels and refunds non-tenant jobs
still awaiting upload after 2h with nothing uploaded.
- `uploads-complete` now checks ownership (previously any signed-in user
could finalize any pending job). On an instrumental duration mismatch it
cancels and refunds the job instead of stranding it.

## Testing
- `make test`: backend 1839 + 4788 + 150 emulator passed; frontend Jest
1391 passed across 117 suites.
- New tests:
- backend: stale-upload processor; signed instrumental endpoints (15
cases); `requires_audio_edit` persistence; uploads-complete ownership
and cancel-on-mismatch
- frontend: upload lib, `useUploadTask`, modal (speed/ETA, can't be
dismissed), `OwnInstrumentalField`, `TenantJobFlow`, API flows
- Manual check against the real frontend (local dev, prod read-only;
create, upload and finalize endpoints mocked, uploads sent to a
throttled local PUT server, so no prod data was changed):
  - guided flow with and without an instrumental
- instrumental review upload on 2d0c1ae0, including a mismatch caught
before upload
- Escape and outside clicks don't close the modal; navigating away
triggers the `beforeunload` prompt

Plan: `docs/archive/2026-09-26-unified-upload-flow-plan.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 20:25:00 -04:00

266 lines
11 KiB
Python
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""
Audio transcoding service for review UI optimization.
Transcodes FLAC audio files to OGG Opus for efficient browser playback.
Original FLAC files are never modified — transcoded versions are cached
in GCS at jobs/{job_id}/review-audio/{filename}.ogg.
Eager transcoding: called from screens_worker before AWAITING_REVIEW
so all files are cached before the user opens the review UI.
Falls back to original FLAC signed URLs if transcoded version is missing.
"""
import asyncio
import hashlib
import logging
import os
import subprocess
import tempfile
from pathlib import Path
from typing import Optional
from backend.services.storage_service import SignedUrlTimeout, StorageService
from backend.utils.audio_filenames import audio_content_type, local_audio_filename
logger = logging.getLogger(__name__)
class AudioTranscodingService:
"""
Transcodes audio files to OGG Opus and caches them in GCS.
Used by the review UI to serve ~3 MB OGG files instead of 35+ MB FLACs.
"""
def __init__(self, storage_service: Optional[StorageService] = None):
self.storage = storage_service or StorageService()
def _get_cache_path(self, source_gcs_path: str) -> str:
"""
Derive the cache path for a transcoded file.
Examples:
jobs/abc123/input/song.flac -> jobs/abc123/review-audio/song.ogg
jobs/abc123/stems/instrumental_clean.flac -> jobs/abc123/review-audio/instrumental_clean.ogg
"""
parts = Path(source_gcs_path).parts
# Find "jobs/{job_id}" prefix
try:
jobs_idx = list(parts).index("jobs")
job_id = parts[jobs_idx + 1]
except (ValueError, IndexError):
# Fallback: use a hash of the full path to avoid collisions
job_id = "_" + hashlib.sha256(source_gcs_path.encode()).hexdigest()[:12]
filename = Path(source_gcs_path).stem + ".ogg"
return f"jobs/{job_id}/review-audio/{filename}"
def get_transcoded_cache_path(self, source_gcs_path: str) -> str:
"""Public accessor for the transcoded-OGG cache path of a source file."""
return self._get_cache_path(source_gcs_path)
def _transcode_and_upload(self, source_gcs_path: str, cache_path: str) -> str:
"""
Download FLAC from GCS, transcode to OGG Opus, upload to cache path.
Returns the cache GCS path on success.
Raises on failure.
"""
with tempfile.TemporaryDirectory() as temp_dir:
local_input = os.path.join(temp_dir, local_audio_filename(source_gcs_path, "input"))
local_output = os.path.join(temp_dir, "output.ogg")
# Download source
self.storage.download_file(source_gcs_path, local_input)
# Transcode: FLAC -> OGG Opus 128kbps
cmd = [
"ffmpeg", "-hide_banner", "-loglevel", "error",
"-y",
"-i", local_input,
"-c:a", "libopus",
"-b:a", "128k",
"-vn",
local_output,
]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=120)
if result.returncode != 0:
raise RuntimeError(
f"ffmpeg transcoding failed (exit {result.returncode}): {result.stderr}"
)
# Upload transcoded file
self.storage.upload_file(local_output, cache_path)
return cache_path
def invalidate_cache(self, source_gcs_path: str) -> None:
"""Drop the cached OGG for a source that was overwritten in place (cache is
keyed only on the source path, so a stale transcode would otherwise be served)."""
try:
self.storage.delete_file(self._get_cache_path(source_gcs_path), ignore_missing=True)
except Exception as e:
logger.warning(f"Could not invalidate review-audio cache for {source_gcs_path}: {e}")
def transcode_if_needed(self, source_gcs_path: str) -> str:
"""
Return the GCS path of the transcoded OGG file, transcoding if not cached.
Returns the cache path (always an OGG file in review-audio/).
Raises if transcoding fails.
"""
cache_path = self._get_cache_path(source_gcs_path)
if self.storage.file_exists(cache_path):
logger.debug(f"Cache hit: {cache_path}")
return cache_path
logger.info(f"Cache miss, transcoding: {source_gcs_path} -> {cache_path}")
return self._transcode_and_upload(source_gcs_path, cache_path)
def get_review_audio_url(
self, source_gcs_path: str, expiration_minutes: int = 120
) -> str:
"""
Get a signed URL for the transcoded audio, falling back to original FLAC.
"""
try:
cache_path = self.transcode_if_needed(source_gcs_path)
return self.storage.generate_signed_url(cache_path, expiration_minutes)
except SignedUrlTimeout:
# The signing backend (IAM signBlob) is stalling — every fallback below
# would sign too and stall identically. Bail immediately so the caller
# degrades fast (null audio_url) instead of burning 2-3× the timeout.
raise
except Exception as e:
# Source may be deleted but cache from earlier transcode persists
try:
cache_path = self._get_cache_path(source_gcs_path)
if self.storage.file_exists(cache_path):
logger.info(f"Source gone but cache exists, serving: {cache_path}")
return self.storage.generate_signed_url(cache_path, expiration_minutes)
except Exception:
pass # Fall through to FLAC fallback below
logger.warning(
f"Transcoding failed for {source_gcs_path}, falling back to FLAC: {e}"
)
return self.storage.generate_signed_url(source_gcs_path, expiration_minutes)
async def get_review_audio_url_async(
self, source_gcs_path: str, expiration_minutes: int = 120
) -> str:
"""Async wrapper around get_review_audio_url via asyncio.to_thread."""
return await asyncio.to_thread(
self.get_review_audio_url, source_gcs_path, expiration_minutes
)
def get_review_audio_bytes(self, source_gcs_path: str) -> tuple[bytes, str]:
"""Return (bytes, content_type) of the transcoded review audio.
Used to proxy the audio through the API instead of 302-redirecting to a
signed GCS URL. The waveform visualizer fetch()es + decodeAudioData()s
this, which is a cross-origin read; a redirect to storage.googleapis.com
makes the browser send `Origin: null` on the redirected request, which
the bucket CORS rejects. Proxying keeps it same-origin to the API (which
sets the right CORS headers). The transcoded OGG is small (~3 MB), well
under Cloud Run's 32 MB response cap.
Falls back to the original (FLAC) source if transcoding fails, mirroring
get_review_audio_url, so a transient ffmpeg/cache error doesn't blank the
waveform while the source still exists. (A very long FLAC could exceed
Cloud Run's 32 MB cap on that rare fallback path — the frontend then just
hides the waveform, which is acceptable degradation.)
"""
try:
cache_path = self.transcode_if_needed(source_gcs_path)
return self.storage.download_bytes(cache_path), "audio/ogg"
except Exception as e:
logger.warning(f"Transcoding failed for {source_gcs_path}, serving source audio: {e}")
return self.storage.download_bytes(source_gcs_path), audio_content_type(source_gcs_path)
async def get_review_audio_bytes_async(self, source_gcs_path: str) -> tuple[bytes, str]:
"""Async wrapper around get_review_audio_bytes via asyncio.to_thread."""
return await asyncio.to_thread(self.get_review_audio_bytes, source_gcs_path)
# Stems the review UI plays (instrumental options + the backing-vocals waveform).
REVIEW_STEM_KEYS = ("instrumental_clean", "instrumental_with_backing", "backing_vocals")
def transcode_review_stems(self, job, force: bool = True) -> list[str]:
"""
Transcode the review-playable stems right after audio separation uploads them.
screens_worker's eager pass runs on LYRICS completion, which often precedes
separation — so without this the stems' OGGs were only made on demand when the
reviewer first played them (a 7-15 s ffmpeg run blocking the preview modal's
instrumental audio). ``force`` (default) re-transcodes even if a cached OGG
exists, since freshly uploaded stems supersede any OGG from an earlier run.
Stems are transcoded in parallel; failures are logged, not raised.
Returns list of cache paths that were written.
"""
from concurrent.futures import ThreadPoolExecutor
stems = (job.file_urls or {}).get("stems", {})
sources = [(k, stems[k]) for k in self.REVIEW_STEM_KEYS if stems.get(k)]
if not sources:
return []
def _one(item):
label, source_path = item
try:
if force:
return self._transcode_and_upload(source_path, self._get_cache_path(source_path))
return self.transcode_if_needed(source_path)
except Exception as e:
logger.warning(f"[{job.job_id}] Failed to transcode review stem {label}: {e}")
return None
with ThreadPoolExecutor(max_workers=len(sources)) as pool:
results = [r for r in pool.map(_one, sources) if r]
logger.info(f"[{job.job_id}] Review stems transcoded: {len(results)}/{len(sources)}")
return results
def prepare_review_audio_for_job(self, job) -> list[str]:
"""
Transcode all review audio files for a job (eager transcoding).
Called from screens_worker before AWAITING_REVIEW transition
and from the admin backfill endpoint.
Returns list of cache paths that were transcoded.
"""
job_id = job.job_id
transcoded = []
# Collect all audio paths to transcode
paths_to_transcode = []
# Main input audio
if job.input_media_gcs_path:
paths_to_transcode.append(("input", job.input_media_gcs_path))
# Stems
stems = job.file_urls.get("stems", {})
for stem_key in ["instrumental_clean", "instrumental_with_backing", "backing_vocals"]:
path = stems.get(stem_key)
if path:
paths_to_transcode.append((stem_key, path))
for label, source_path in paths_to_transcode:
try:
cache_path = self.transcode_if_needed(source_path)
transcoded.append(cache_path)
logger.info(f"[{job_id}] Transcoded {label}: {cache_path}")
except Exception as e:
# Non-fatal: review UI will fall back to FLAC
logger.warning(
f"[{job_id}] Failed to transcode {label} ({source_path}): {e}"
)
logger.info(
f"[{job_id}] Review audio preparation complete: "
f"{len(transcoded)}/{len(paths_to_transcode)} files transcoded"
)
return transcoded