mirror of
https://github.com/nomadkaraoke/karaoke-gen.git
synced 2026-11-20 13:07:57 +09:00
## Summary
Jobs with a user-supplied instrumental (`existing_instrumental`) skip
GPU separation. That left the lyrics review **Waveforms** mode with no
vocals stem, so `/vocals-peaks` returned 404 and no strips were drawn.
The waveform only needs a peak envelope, so a CPU approximation is
enough.
- **`backend/services/derived_vocals.py`**: decodes the mix and the
instrumental to mono at 22.05 kHz (ffmpeg). It aligns them by FFT
cross-correlation (numpy, ±2s search), fits a least-squares gain, and
subtracts, leaving roughly the vocals. It writes a FLAC and logs offset,
gain and residual ratio. A few seconds of CPU.
- **Lyrics worker**: for existing-instrumental jobs, runs this in a
thread alongside transcription. It uploads
`jobs/{id}/stems/vocals_derived.flac` and registers
`stems.vocals_derived`, and it's awaited before advancing to screens so
the screens step pre-computes peaks as usual. Best-effort: failures log
and the job continues.
- **`vocals_stem_path`**: adds `vocals_derived` as the last-resort key.
It's waveform-only: deliverable stems, the original-vocals guide and the
auto-approval timing gate don't read it.
## Testing
`backend/tests/test_derived_vocals.py` covers:
- offset recovery for positive, negative and large lags
- gain and vocals recovery on synthetic signals
- an unrelated instrumental leaving the mix intact
- a real ffmpeg round trip (24-bit stereo WAVs → mono 22.05 kHz FLAC)
- stem precedence (never overrides real separated vocals)
- worker upload and registration, plus the non-fatal failure path
The full backend unit suite passes in an isolated venv (4812 passed).
The only exceptions are 6 tests needing the spaCy `en_core_web_sm`
model, which isn't installed in that venv; CI has it.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
24 lines
1.0 KiB
Python
24 lines
1.0 KiB
Python
"""Stem-selection helpers shared by review routes and workers."""
|
|
|
|
from typing import Optional
|
|
|
|
|
|
def vocals_stem_path(job) -> Optional[str]:
|
|
"""The full-vocal-mix stem for the review waveform, or None if absent.
|
|
|
|
Stem keys vary by separation model. We want the full vocal mix
|
|
(lead + backing) so every word can be lined up against the waveform:
|
|
- "vocals" : full vocal from a 2-stem split (rare)
|
|
- "vocals_clean" : full vocal from the primary vocal/instrumental
|
|
split (present on essentially all cloud jobs)
|
|
- "lead_vocals" : fallback (misses backing lines)
|
|
- "vocals_derived": mix minus the user's own instrumental, for jobs that
|
|
skipped separation (backend/services/derived_vocals.py).
|
|
Waveform-only — never a deliverable stem.
|
|
"""
|
|
stems = job.file_urls.get("stems", {}) if job.file_urls else {}
|
|
for key in ("vocals", "vocals_clean", "lead_vocals", "vocals_derived"):
|
|
if stems.get(key):
|
|
return stems[key]
|
|
return None
|