Files
Andrew Beveridge ☄️andClaude Opus 5.5 5a9d0024a5 fix(quick-version): ffmpeg 4.4 never-ending render + fast ONNX model (v0.246.1) (#1074)
## Summary
First production make-it quick version (job `405eed71`, Rihanna "Push Up
On Me") failed with `render timed out`:
- **Render never ended**: the GPU image ships ffmpeg **4.4** (Ubuntu
22.04). With the guide-vocals `amix` in the filtergraph plus the
infinite lavfi colour + looped PNG inputs, it ignores `-shortest` —
reproduced in `ubuntu:22.04`: a 20 s song produced 540 s+ of video until
killed. ffmpeg 8 (local dev) stops fine, which is why local tests
passed. Fix: every input and the output are bounded with `-t <duration>`
(colour source `d=`). Verified in `ubuntu:22.04`: 3.7 s render, exactly
20.0 s audio+video.
- **Quick separation too slow**: the baked roformer took 14 s load + 124
s for a 3:15 song on the L4. Switched to fastgen's
`UVR-MDX-NET-Inst_HQ_4.onnx`. It's downloaded on load until the next GPU
base rebuild bakes it (`download_models.py` `EXTRA_MODELS`), with a
fallback to the baked roformer if the load fails.

Prod timeline observed for that job (for the <4 min goal): job created
05:22:12 → **audio-download-job container took 3m46s to start**
(existing issue, 1m40–4m across recent executions; separate follow-up) →
audio in GCS 05:26:46 → GPU worker started 05:27:10 → quick separation
done 05:29:49 → render hung.

## Testing
- `backend/tests/test_quick_version.py`: 40 pass (new: every stream
bounded to the duration; model fallback; both-fail raises).
- Docker `ubuntu:22.04` ffmpeg 4.4.2 reproduction before/after (see
above).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Quick audio separation now uses a faster model by default, with an
alternate model available if the primary model cannot be loaded.
* **Bug Fixes**
* Quick video rendering is now limited to the song’s duration, helping
prevent renders from continuing beyond the audio or failing to finish.
* Model loading failures are handled more reliably: if both the primary
and alternate models fail to load, the error is reported.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 01:51:13 -04:00

55 lines
1.9 KiB
Python

#!/usr/bin/env python3
"""
Download ensemble preset models at Docker build time.
Bakes models into the image so they're available immediately at runtime,
avoiding cold-start model download latency in Cloud Run Jobs.
Usage: python backend/scripts/download_models.py /models
"""
import json
import sys
import logging
import importlib.resources as resources
from audio_separator.separator import Separator
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
# Presets to bake into the GPU image
PRESETS_TO_DOWNLOAD = ["instrumental_clean", "karaoke"]
# Individual models baked in addition to the presets.
# UVR-MDX-NET-Inst_HQ_4.onnx: the kjbox make-it "quick version" separation
# (backend/services/quick_version/service.py).
EXTRA_MODELS = ["UVR-MDX-NET-Inst_HQ_4.onnx"]
def download_preset_models(model_dir: str) -> None:
"""Download all models for the configured presets individually.
Ensemble presets resolve to multiple model filenames. We must download
each model individually via load_model(model_filename) — calling
load_model() with no args on an ensemble preset doesn't trigger downloads.
"""
with resources.open_text("audio_separator", "ensemble_presets.json") as f:
presets = json.load(f)["presets"]
models_to_download = set()
for preset_name in PRESETS_TO_DOWNLOAD:
models_to_download.update(presets[preset_name]["models"])
models_to_download.update(EXTRA_MODELS)
logger.info(f"Downloading {len(models_to_download)} models for presets {PRESETS_TO_DOWNLOAD} + {EXTRA_MODELS}")
for model in sorted(models_to_download):
logger.info(f" Downloading: {model}")
sep = Separator(model_file_dir=model_dir)
sep.load_model(model)
logger.info(f" Done: {model}")
if __name__ == "__main__":
model_dir = sys.argv[1] if len(sys.argv) > 1 else "/models"
download_preset_models(model_dir)
logger.info(f"All preset models downloaded to {model_dir}")