Files
Andrew Beveridge ☄️andClaude Opus 5.5 c138c9d1ac feat(review): vocal waveforms for own-instrumental jobs (v0.244.1) (#1062)
## Summary
Jobs with a user-supplied instrumental (`existing_instrumental`) skip
GPU separation. That left the lyrics review **Waveforms** mode with no
vocals stem, so `/vocals-peaks` returned 404 and no strips were drawn.
The waveform only needs a peak envelope, so a CPU approximation is
enough.

- **`backend/services/derived_vocals.py`**: decodes the mix and the
instrumental to mono at 22.05 kHz (ffmpeg). It aligns them by FFT
cross-correlation (numpy, ±2s search), fits a least-squares gain, and
subtracts, leaving roughly the vocals. It writes a FLAC and logs offset,
gain and residual ratio. A few seconds of CPU.
- **Lyrics worker**: for existing-instrumental jobs, runs this in a
thread alongside transcription. It uploads
`jobs/{id}/stems/vocals_derived.flac` and registers
`stems.vocals_derived`, and it's awaited before advancing to screens so
the screens step pre-computes peaks as usual. Best-effort: failures log
and the job continues.
- **`vocals_stem_path`**: adds `vocals_derived` as the last-resort key.
It's waveform-only: deliverable stems, the original-vocals guide and the
auto-approval timing gate don't read it.

## Testing
`backend/tests/test_derived_vocals.py` covers:
- offset recovery for positive, negative and large lags
- gain and vocals recovery on synthetic signals
- an unrelated instrumental leaving the mix intact
- a real ffmpeg round trip (24-bit stereo WAVs → mono 22.05 kHz FLAC)
- stem precedence (never overrides real separated vocals)
- worker upload and registration, plus the non-fatal failure path

The full backend unit suite passes in an isolated venv (4812 passed).
The only exceptions are 6 tests needing the spaCy `en_core_web_sm`
model, which isn't installed in that venv; CI has it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 23:04:15 -04:00

24 lines
1.0 KiB
Python

"""Stem-selection helpers shared by review routes and workers."""
from typing import Optional
def vocals_stem_path(job) -> Optional[str]:
"""The full-vocal-mix stem for the review waveform, or None if absent.
Stem keys vary by separation model. We want the full vocal mix
(lead + backing) so every word can be lined up against the waveform:
- "vocals" : full vocal from a 2-stem split (rare)
- "vocals_clean" : full vocal from the primary vocal/instrumental
split (present on essentially all cloud jobs)
- "lead_vocals" : fallback (misses backing lines)
- "vocals_derived": mix minus the user's own instrumental, for jobs that
skipped separation (backend/services/derived_vocals.py).
Waveform-only — never a deliverable stem.
"""
stems = job.file_urls.get("stems", {}) if job.file_urls else {}
for key in ("vocals", "vocals_clean", "lead_vocals", "vocals_derived"):
if stems.get(key):
return stems[key]
return None