46 KiB
StemEngine — Session Summaries
2026-03-06
Diarization fix — whisperx.DiarizationPipeline not exposed
Problem: whisperx.DiarizationPipeline raised AttributeError because this version of
WhisperX uses a lazy-loader __init__.py that only re-exports a handful of functions
(load_model, align, assign_word_speakers, etc.) and never added DiarizationPipeline
to the list.
Fix: Import directly from the submodule:
from whisperx.diarize import DiarizationPipeline as _DiarizationPipeline
Diarization fix — use_auth_token renamed to token
Problem: After fixing the import, DiarizationPipeline.__init__() raised
TypeError: unexpected keyword argument 'use_auth_token'. The installed version renamed the
parameter to match the huggingface_hub convention change.
Fix: Changed constructor call to _DiarizationPipeline(token=hf_token, device=device).
Also noted: this version defaults to pyannote/speaker-diarization-community-1, a community
model that does not require accepting HuggingFace terms of use.
Diarization fix — speaker labels not propagated from words to segments
Problem: whisperx.assign_word_speakers in some versions assigns "speaker" only to
individual word dicts, not to the parent segment dict. seg.get("speaker") returned None
so |sp:SPEAKER_XX| tags were never emitted in LRC/ELRC output.
Fix (already applied in prior session): After assign_word_speakers, iterate segments
and propagate the most-frequent word-level speaker to each segment that lacks one:
for _seg in wx_result.get("segments", []):
if _seg.get("speaker"):
continue
_spk_counts = {}
for _w in _seg.get("words", []):
_s = _w.get("speaker")
if _s:
_spk_counts[_s] = _spk_counts.get(_s, 0) + 1
if _spk_counts:
_seg["speaker"] = max(_spk_counts, key=_spk_counts.get)
Timing fix — unaligned words all sharing seg["start"]
Problem: When WhisperX cannot align individual words (returns start=None), all such
words in a segment received seg["start"] as their timestamp. In the frontend
wordFillPct() function, consecutive words with identical timestamps produce dur = 0
→ NaN fill percentage → broken karaoke highlight animation.
Fix: Distribute unaligned words evenly across [seg_start, seg_end] by index:
if _ws is None:
_ws = _seg_start + (_wi / _n_words) * (_seg_end - _seg_start)
_we = _seg_start + ((_wi + 1) / _n_words) * (_seg_end - _seg_start)
CUDA OOM — retry with reduced settings
Problem: Loading large, large-v2, or large-v3 models on cards with limited VRAM
raised RuntimeError: CUDA out of memory, failing the job entirely.
Fix: Wrapped load_model + transcribe in a 2-attempt retry loop. On first OOM:
- Clear CUDA cache (
gc.collect()+torch.cuda.empty_cache()) - Reduce
batch_sizeto 4 - Switch
compute_typeto"int8" - Update job progress to inform the user
- Retry
On second OOM or any non-OOM error, re-raise. Also added _clear_cuda_cache() at job start
to flush leftover allocations from previous jobs.
Added user-friendly error message on OOM failure suggesting smaller model or Eco preset.
WSL2 display stability — VRAM cap + CPU niceness + inter-phase yields
Problem: On WSL2, WhisperX saturates the shared GPU, starving the Windows Display Manager (DWM). This causes TDR timeouts, screen flicker, and display driver restarts.
Fixes added to lyric_transcriber_service.py:
-
_apply_cuda_limits()called at job start:os.nice(10)— lowers the job thread's CPU scheduling prioritytorch.cuda.set_per_process_memory_fraction(fraction)— hard-caps VRAM usage, reserving headroom for DWM. Default 80%, configurable viaSTEMENGINE_GPU_MEMORY_FRACTIONenv var.
-
time.sleep(0.15)inserted after each model-unload+cache-clear between the three heavy GPU phases (Whisper → alignment → diarization), giving the GPU scheduler a window to service DWM between compute bursts.
System-level recommendation (Windows host, not in code):
- Create
%UserProfile%\.wslconfig:[wsl2] memory=8GB processors=6 - Set
STEMENGINE_GPU_MEMORY_FRACTION=0.75in Docker compose or.envfor tighter cards.
2026-03-06 (continued)
Syllables checkbox + hide Phonemization in Auto Pipeline
Added lsyncAlign checkbox ("Syllables") to Auto Pipeline options (Settings.tsx, AutoPipelineDialog.tsx).
Removed Phonemization from both ProjectDetail.tsx (Lyrics Sync tab) and Settings.tsx (Auto Pipeline step editor) — state kept for API compatibility.
Explicit mode — Whisper profanity uncensor
Added explicit: bool to LyricTranscribeRequest, run_lyric_transcription(), and frontend (ProjectDetail.tsx, AutoPipelineDialog.tsx, client.ts).
_uncensor_segments() in lyric_transcriber_service.py applies regex post-processing after transcription to restore censored words (f*** → fuck, etc.). Initial-prompt primer approach was discarded — it caused Whisper to reproduce the prompt text verbatim at the start of the transcription.
Hip-hop vocal style
Added vocal_style: str = "general" parameter throughout the stack. When "hiphop":
- VAD thresholds:
vad_onset=0.25,vad_offset=0.15,no_speech_threshold=0.35,hallucination_silence_threshold=1.0 - Monkeypatches
Silero.__call__in try/finally to injectmin_speech_duration_ms=100, min_silence_duration_ms=150(these params are commented out in whisperx's wrapper, so they must be injected at call time).
GlobalAudioBar fixes + X button
- Bar not appearing:
isActiveRefis a ref so changing it doesn't trigger effects. Fixed by callingpublishMeta(...)immediately afterisActiveRef.current = truein bothDynamicMixPlayerandLyricsLivePreviewtogglePlaybranches. - Audio continuing after navigation:
unregisterPlaybackonly removes the registry reference, not the audio element. Fixed by callingstopPlayback()beforeunregisterPlayback()in unmount cleanup of both players. - X button: Added
stopCurrent()export toaudioRegistry.ts(calls registered stop callback + clears_current). Added X (lucide-react) button toGlobalAudioBar.tsx.
Karaoke fill/pop mode + Open Foundry fonts
LyricsLivePreview.tsx:
- Added
karaokeMode: 'fill' | 'pop'toStyleState(default'fill'). KaraokeWordnow acceptsisActive?: booleanandmode?: 'fill' | 'pop':fill: existing left-to-right gradient sweep.pop: active word bright + 15% larger; already-sung words stay bright; future words dim. Line accumulates highlighted words as sung — no gradient, no fill.
- All three
KaraokeWordcall sites (karaoke, glow, blur styles) passisActiveandmode. - Added Fill/Pop toggle buttons to
StylePicker. - Replaced
FONT_OPTIONSwith 10 Open Foundry fonts (Archivo, EB Garamond, Inter, Libre Baskerville, Oswald, Poppins, Roboto, Work Sans, Young Serif + System fallback).
package.json: Added @fontsource/* packages for all 9 named fonts — self-hosted, no Google CDN calls. Installed automatically by start.sh at container start.
main.tsx: Imports each @fontsource/* package so Vite bundles the font files.
Font dropdown dark background + bold toggle
- Dropdown white background fix: Added
colorScheme: 'dark'on the<select>andbackground: '#1e293b', color: '#e2e8f0'on each<option>so the native OS dropdown renders dark. - Bold toggle: Added
fontBold: booleantoStyleState(defaultfalse). B button inStylePickernext to the font selector. Applied asfontWeight: 700 / 400on the lyrics container so it cascades to all rendered text.
Pop mode rendering bug fix
Problem: In pop mode, WebkitTextFillColor: 'transparent' set by a previous fill-mode render persisted on the DOM element (React doesn't always clear vendor-prefixed style properties when they're removed from the style object). This made text invisible while the gradient background showed as a colored rectangle.
Fix: In the KaraokeWord pop-mode branch, explicitly reset all gradient/clip properties:
backgroundImage: 'none',
WebkitBackgroundClip: 'border-box',
WebkitTextFillColor: 'currentColor', // key: restores text color rendering
backgroundClip: 'border-box',
WebkitTextFillColor: 'currentColor' makes the text use the CSS color property again, which is what pop mode sets per-word.
Pop mode — RAF loop still writing gradient to DOM
Problem: Even after the React CSS resets in KaraokeWord, a fill bar kept appearing in pop mode. The real cause: a requestAnimationFrame loop writes backgroundImage directly to activeWordSpanRef.current.style every frame (to avoid 60fps React re-renders). This direct DOM write completely bypassed React's style reconciliation and overwrote the pop-mode resets on every tick.
Fix:
- Added
karaokeModeRef = useRef(karaokeMode)alongside the other RAF refs. - Updated it inline each render:
karaokeModeRef.current = karaokeMode(outside any effect, so it's always current without restarting the loop). - Gated the gradient write in the RAF loop:
if (activeWordSpanRef.current && lastLineIdx >= 0 && karaokeModeRef.current !== 'pop').
Pop mode — done words should stay bright
Corrected pop mode behavior: sung (done) words stay bright, not dim. Only future words are dim. Active word is bright + 15% larger.
Changed color: isActive ? bright : dim → color: (isActive || done) ? bright : dim in KaraokeWord.
Pop mode — active word shifts down
Problem: Active word is fontSize * 1.15; base uses verticalAlign: 'top', so the larger word's top aligns with others but extends further down.
Fix: Override to verticalAlign: 'middle' in the pop-mode branch so all words (normal and enlarged) center on the same axis.
Album card font not affected by font option
Both the small windowed overlay and the fullscreen album panel used hardcoded fontFamily / fontWeight values, so the user's font selection had no effect.
Fix: Added fontFamily to both text containers (windowed overlay <div> and fullscreen panel <div>). The subtitle in both also respects fontWeight: fontBold ? 700 : 400. Artist name stays at 700 (it's always a heading).
Also changed the windowed overlay from flexDirection: 'column' (art above, text below) to flexDirection: 'row', alignItems: 'center' so text sits beside and vertically centered on the art.
TypeScript error 7026 / 2875 / 2322 (pre-existing, not build-blocking)
These TS language server errors ("no interface JSX.IntrinsicElements exists", "react/jsx-runtime not found") appear on whichever line was last edited. They are a pre-existing tsconfig / language server cache issue — not introduced by any of the above changes. Vite uses esbuild for transpilation and does not type-check, so vite build is unaffected. Restart the TS server (VS Code: Ctrl+Shift+P → "TypeScript: Restart TS Server") or let the container run npm install to resolve.
Styling options — stroke, shadow, scale, pop size
Added to StyleState: strokeWidth, strokeColor, shadowBlur, shadowX, shadowY, shadowColor, scaleX, scaleY, popScale.
StylePicker gained two new rows:
- Effects row: stroke color swatch + width slider (0–4), shadow color swatch + blur/X/Y sliders.
- Scale row: W (horizontal) and H (vertical) scale sliders (0.5–2×). A Pop slider (15–25%) appears conditionally when
karaokeMode === 'pop'.
Lyrics container (outer div): textShadow and WebkitTextStroke applied from state; as any cast used because the file uses named React imports (no React namespace for React.CSSProperties).
Inner lyrics div: transform updated to translateY(…) scaleX(…) scaleY(…).
KaraokeWord: accepts popScale?: number (default 15); pop branch uses fontSize * (1 + popScale / 100) for the active word size.
All three KaraokeWord call sites (karaoke / glow / blur renderers) updated to pass popScale={popScale}.
Stroke fix — proper outer border, not hollow outline; show/hide toggles; shadow opacity; Reset
Problem: -webkit-text-stroke strokes symmetrically inward and outward, hollowing the glyph fill. In gradient (fill) mode, -webkit-text-fill-color: transparent made it worse — just an outline with no visible fill.
Fix: Replaced -webkit-text-stroke entirely with a disc-shaped text-shadow simulation. makeStrokeShadow(width, color) generates zero-blur shadows at every integer offset within a radius, producing a filled outer ring. Works correctly with WebkitTextFillColor: transparent (gradient mode) and normal color fills (pop mode).
Added hexToRgba(hex, opacity) to apply shadow opacity to the drop shadow.
New StyleState fields: strokeEnabled: boolean, shadowEnabled: boolean, shadowOpacity: number (0–1).
DEFAULT_STYLE constant extracted so defaults are a single source of truth used by both useState and the new resetStyle handler.
StylePicker UI changes:
- Stroke and Shadow labels are now toggle buttons (click to enable/disable); controls collapse when disabled.
- Shadow controls gained an Opacity slider (0–1) next to the color swatch.
- Reset button added in the font row (right side) — clears localStorage and restores
DEFAULT_STYLE.
StylePicker prop: onReset: () => void added. resetStyle callback in main component removes the localStorage key and calls setStyle(DEFAULT_STYLE).
Stroke rendering order — abandoned text-shadow approach; switched to WebkitTextStroke + paint-order
Attempt 1 (disc text-shadow): Generated a disc of zero-blur text-shadow entries. Inner shadows covered the gradient interior, making text look hollow/broken.
Attempt 2 (ring text-shadow): Limited shadows to the outermost pixel band. Still jagged/bubbly in practice — each shadow entry is a full pixel-offset copy of the glyph shape; many overlapping copies create rough anti-aliasing artefacts visible as a bubbly halo.
Final fix: Removed makeStrokeShadow entirely. Now use:
WebkitTextStroke: '${strokeWidth}px ${strokeColor}'on the container (cascades to all child text)paintOrder: 'stroke fill'on the container (cascades): renders stroke first, then fill on top — only the outer half of the stroke extends beyond the glyph edge; the inner half is covered by the opaque fill
For pop mode (solid fill): only the outer stroke ring is visible. Clean outer border. ✓
For fill/gradient mode (WebkitTextFillColor: transparent): fill is transparent, so inner half of stroke is slightly visible at the glyph edge. For thin strokes this is a minor tint at the very edge; outer border is clean and anti-aliased. Acceptable trade-off vs. jagged shadow approach. ✓
Drop shadow remains via textShadow on the container, unchanged.
Stroke/shadow layered DOM approach in KaraokeWord
Problem: WebkitTextStroke + paint-order: stroke fill still broke gradient text. CSS paints background (where the gradient lives via background-clip:text) before text-shadow/WebkitTextStroke, so any stroke technique on the same element appears on top of the gradient, not behind it. No single-element CSS solution can place the stroke behind a background-clip:text gradient.
Fix: Layered DOM structure inside KaraokeWord. When stroke or shadow is active:
<span style="position:relative"> ← wrapper (layout)
<span aria-hidden style="position:absolute"> ← behind-layer: stroke + shadow
<span ref={elRef}> ← fill-layer: gradient/pop text
- Behind-layer: renders
WebkitTextStroke+paintOrder: stroke fill+textShadow. With opaque fill color, only the outer half of the stroke extends beyond the glyph → clean outer ring. - Fill-layer: resets
WebkitTextStroke: '0 transparent'andtextShadow: 'none'to cancel container inheritance, then renders gradient or solid-color text on top.
Result: shadow (back) → stroke → base text (front), all properly layered.
WordEffects interface added; wordEffects object constructed from destructured style state; passed to all 3 KaraokeWord call sites as effects={wordEffects}.
hexToRgba moved above KaraokeWord so behindLayer can call it.
Superseded — stroke/shadow are now handled per-line via the behind-layer pattern for all modes. See "Universal behind-layer stroke/shadow" below.
Layer z-index fix + sluggishness fix
z-index bug: position: absolute elements paint on top of in-flow elements by default regardless of DOM order. The behind-layer (position: absolute) was rendering above the fill span. Fixed by adding zIndex: 0 to the behind-layer and position: relative; zIndex: 1 to the fill/pop front span via frontLayerBase. CSS z-index only applies to positioned elements, so the fill span needed position: relative to participate in z-index stacking.
Sluggishness: The RAF loop triggers React state updates (~60 fps), causing re-renders of the lyrics line. Each re-render was constructing a new wordEffects object, propagating as a changed prop to every KaraokeWord in the line. Fixed by wrapping wordEffects in useMemo with the style fields as dependencies — the object reference is now stable during playback and only changes when the user adjusts a style control. useMemo added to the React import.
Album art overlay text exempt from stroke/shadow
The windowed overlay (div wrapping artist + title <p> tags) and the fullscreen album panel (div with artist/title) both inherit WebkitTextStroke and textShadow from the outer lyrics container. Fixed by adding WebkitTextStroke: 'initial' and textShadow: 'none' to the style prop of each wrapper div (as any cast to avoid TS CSSProperties namespace error).
Pop mode syllable transition — font-size → transform: scale()
Problem: Pop mode used fontSize: activeFontSize (switching between base and scaled size) on the wrapper, with transition: 'font-size 0.08s ease'. Font-size changes trigger layout reflow on every transition frame, causing jank. 80ms was also too short, making the snap feel jarring.
Fix:
- Replaced
fontSizetoggling withtransform: scale(1 + popScale/100)on the wrapper span — GPU-composited, zero layout reflow. fontSizeis now alwaysfontSize(base size); the wrapper transform handles the visual growth.- Transition:
transform 0.15s ease-out, color 0.15s ease-out— 150ms feels smooth without lag;ease-outsnaps into the active state and eases out of it naturally. willChange: 'transform'added on pop-mode wrappers to hint the browser to promote them to a compositor layer.- Behind-layer
fontSizealso stays at base size (scales with the wrapper via inherited transform).
Stroke rounded joins — prevent "fang" artefacts
-webkit-text-stroke defaults to miter joins at sharp glyph corners, producing spike/fang artefacts. Fixed by adding strokeLinejoin: 'round' and strokeLinecap: 'round' (SVG stroke properties, honoured by Chromium/WebKit on HTML text with -webkit-text-stroke) to behind-layer spans. Superseded by universal behind-layer approach — see below.
Pop mode — Snap slider (transition duration)
Added popSnap: number (ms) to StyleState and DEFAULT_STYLE (default: 150ms). A Snap slider (50–500ms, step 25ms) appears next to the Pop slider when karaokeMode === 'pop'. The value is passed as snapMs prop to KaraokeWord (default 150) and used in the wrapStyle transition: `transform ${snapMs}ms ease-out, color ${snapMs}ms ease-out`. Range rationale: 50ms feels near-instant/snappy; 500ms is a slow, deliberate fade.
BG color scope fix — fullscreen root no longer inherits bgColor
The fullscreen root <div> had style={isFullscreen ? { backgroundColor: effectiveBg } : undefined}. This painted the entire fullscreen container (including toolbar, controls, album panel) with the user's BG color. The BG color is already applied correctly on the inner lyrics area div (backgroundColor: effectiveBg at line ~1804). Removed the root-level style entirely (style={undefined}).
Karaoke scroll jitter — root cause and fix
Root cause (windowed rendering): renderKaraokeStyle, renderGlowStyle, and renderBlurStyle each used a window = 3 slice (lines.slice(activeLine-3, activeLine+4)). Because the active line was always at index 3 within this window, its offsetTop was effectively constant (~3 × lineHeight) regardless of which line was active. The translateY formula containerHeight/2 − offsetTop − lineHeight/2 therefore produced the same value on every line transition — no scroll animation occurred and lines appeared to "jump" into place.
Fix: All three windowed render functions now iterate over all lines (lines.map(...)). Lines with dist > 5 from the active line get opacity: 0; lines with dist > 4 in blur mode skip the expensive blur() filter. The inner scroll div still uses overflow: hidden on its container, so only nearby lines are visible. With all lines in the DOM, offsetTop grows monotonically as activeLine advances, and the CSS transition: transform 0.35s cubic-bezier(0.33, 1, 0.68, 1) on the scroll div animates a true smooth vertical scroll.
Secondary fix — translateY bypasses React state: translateY was moved from useState to a direct DOM write in useLayoutEffect via scrollDivRef.current.style.transform. This eliminates the two-render sequence on line changes (content update → state update → second render) that could cause the transition to be interrupted. translateYRef tracks the current value for use by handleRecenter and the scaleX/scaleY deps in the same effect.
Glow slider — configurable glow radius
Added glowSize: number to StyleState and DEFAULT_STYLE (default: 8px). A Glow slider (2–24px, step 1px) appears in StylePicker when visualStyle === 'glow'. Required passing visualStyle as a new optional prop to StylePicker. The glow filter in renderGlowStyle uses glowSize for the inner shadow and glowSize * 2 for the wider outer shadow: `drop-shadow(0 0 ${glowSize}px ${color}cc) drop-shadow(0 0 ${glowSize*2}px ${color}66)`.
Smooth size transition between active and adjacent lines
Problem: The active line used a different fontSize multiplier (e.g. fontSize * 1.05) than inactive lines (fontSize * 0.85), causing an instant size jump on each line transition with no animation.
Why font-size transition wasn't used: CSS font-size transitions cause layout reflow on every frame, which would desync the scroll position (the useLayoutEffect measures offsetTop based on the final layout, but the font-size is still mid-animation, so lines aren't at their measured positions yet).
Fix: All three render functions (renderKaraokeStyle, renderGlowStyle, renderBlurStyle) now use a uniform base fontSize for all line divs in layout. Visual size is controlled via transform: scale() per line based on distance from active:
dist = 0(active):scale(1.0)dist = 1:scale(0.88)dist = 2:scale(0.82)dist ≥ 3:scale(0.78), opacity fades to 0 past dist 5
transformOrigin is set to match textAlign (left center / center center / right center). All lines get transition: 'transform 0.35s ease-out, opacity 0.35s ease-out' matching the scroll transition duration so both animate in sync. Since transform doesn't affect layout, offsetTop is stable and the scroll calculation remains correct. KaraokeWord now receives base fontSize (the wrapper div's scale handles visual enlargement).
Universal behind-layer stroke/shadow for all line types
Problem: Stroke and shadow were applied via WebkitTextStroke / textShadow on the outer lyrics container div, inherited by all descendant text. This caused:
- Fangs on non-active karaoke/glow/blur lines: plain
<span>elements inheritedpaintOrder: stroke fillfrom the container, producing miter-join spikes at sharp letter corners. - Stroke inside gradient fill: for wordless active lines using
WebkitTextFillColor: transparent, the inherited stroke bled through the transparent fill area.
Previously only KaraokeWord (active words) used the behind-layer pattern. Non-active lines and wordless active lines inherited stroke/shadow from the container.
Fix: WebkitTextStroke, paintOrder, strokeLinejoin, strokeLinecap, and textShadow are removed from the container entirely. Every line in every mode now renders stroke and shadow via its own behind-layer:
Outer div — opacity, transform/scale, font size (NO stroke/shadow/padding)
behindLayer div — position: absolute, zIndex: 0, aria-hidden
strokeColor fill, WebkitTextStroke, paintOrder: stroke fill
strokeLinejoin: round, textShadow
frontLayer div — position: relative, zIndex: 1, padding
WebkitTextStroke: '0 transparent', textShadow: none
actual text color / gradient fill
Three shared constants computed once per render:
const lineStrokeVal = strokeEnabled && strokeWidth > 0 ? `${strokeWidth}px ${strokeColor}` : undefined
const lineShadowVal = shadowEnabled ? `${shadowX}px ${shadowY}px ${shadowBlur}px ${hexToRgba(...)}` : undefined
const lineNeedsLayer = !!(lineStrokeVal || lineShadowVal)
When lineNeedsLayer is false the behindLayer is skipped entirely (single-div fast path).
Coverage: renderScrollStyle, renderFadeStyle, renderKaraokeStyle (non-active + active wordless), renderGlowStyle (non-active + active wordless), renderBlurStyle (non-active + active wordless). Active word-by-word lines continue to use KaraokeWord's existing behind-layer.
Result: Fill is always the top layer; stroke appears only outside the letterform; shadow is always behind both stroke and fill — in every mode, for every line.
2026-03-09
WhisperX hallucination fix — "Thank you", emoji, and Japanese subscription prompts appended to transcriptions
Problem: WhisperX transcriptions occasionally appended hallucinated text at the end of or within a transcription — e.g. "Thank you", emoji, or "チャンネル登録よろしくお願いします" ("Please subscribe to my channel"). Reproduced on 為に by Sasuke Haraguchi.
Root cause — two compounding factors:
-
_fill_transcript_gapsre-transcribed trailing silence. The function (lyric_transcriber_service.py:1144) scanned for audio regions > 2 s with no detected speech and re-ran WhisperX on each slice. The trailing silence after the last lyric was included in this scan. With no real speech, WhisperX hallucinated YouTube-trained filler content (subscription prompts, sign-offs, "Thank you"). Whisper was trained heavily on YouTube video audio, and these are the completion patterns it learned for silence/ambient noise, especially when Japanese is the detected language. -
no_speech_threshold: 0.45was too low. Whisper's stock default is 0.60. The lower value made WhisperX more aggressively classify silent or ambient-noise segments as "probably speech" rather than discarding them, which fed more silent audio to the decoder and increased hallucination frequency.
Fixes applied to backend/services/lyric_transcriber_service.py:
-
_fill_transcript_gaps: Removed the trailing-gap check. Mid-song gaps are still re-transcribed (catches lyrics in quiet passages), but the audio tail after the last segment is no longer sent back through the model. Comment added explaining the rationale. -
no_speech_threshold: Raised from0.45→0.50. Comment updated to document that 0.45 was too aggressive and caused silence to be treated as speech.
MDX-Net fix — module 'numpy' has no attribute 'stft'
Problem: mdxnet_separator.py called np.stft() and np.istft() in _stft / _istft. These functions do not exist in NumPy in any version — they were never part of the NumPy API. The code was broken from the start and had never been exercised.
Fix: Replaced both functions with librosa.stft / librosa.istft, which share the same n_fft / hop_length parameter names. librosa is already a dependency (used in _load_audio).
Shape contract preserved: _spec_to_batch expects (channels, freq, time, 2) where the last dimension is [real, imag]. librosa returns a complex (freq, time) array, so _stft now packs it with np.stack([spec.real, spec.imag], axis=-1), and _istft reconstructs the complex array with spec[ch,:,:,0] + 1j * spec[ch,:,:,1] before calling librosa.istft.
MDX-Net fix — [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: input … index: 1
Problem: After the librosa fix, ONNX inference still failed with an input-dimension error. Three compounding bugs:
-
N_FFT = 2048was wrong.librosa.stftwithn_fft=2048produces1025frequency bins. The ONNX models expect input withDIM_F = 3072bins (plus the Nyquist bin = 3073 total). The correctn_fftfor3073bins is6144. -
_spec_to_batchproduced(batch, 2, dim_f, dim_t)instead of(batch, 4, dim_f, dim_t). UVR MDX-Net ONNX models expect 4 input channels:[L_real, L_imag, R_real, R_imag]— i.e. stereo(2) × complex(2) merged into one dimension. The old code kept stereo and complex as separate dimensions, producing the wrong channel count of 2. -
_batch_to_specinverted the wrong shape — inverse of the broken forward pass; was not the correct inverse of the corrected forward pass. -
Length mismatch in
_demix. After ISTFT,target_audiocan be slightly shorter or longer than the originalmixdue to STFT/ISTFT padding behaviour.mix - target_audioraised an operand-shape error.
Fixes applied to backend/services/mdxnet_separator.py:
-
N_FFT: Changed2048→6144. This produces 3073 freq bins; the top bin is cropped to 3072 before ONNX and zero-padded back after. -
_spec_to_batchrewritten:Input: (2, freq, time, 2) = (stereo, freq, time, complex) → transpose to (stereo, complex, freq, time) = (2, 2, freq, time) → reshape to merge: (4, freq, time) → crop freq to dim_f, pad time to multiple of dim_t → chunk time: (4, dim_f, t_chunks, dim_t) → transpose: (t_chunks, 4, dim_f, dim_t) ← correct ONNX input shape -
_batch_to_specrewritten as the exact inverse:Input: (t_chunks, 4, dim_f, dim_t) → reassemble time: (4, dim_f, time) → remove time pad, restore freq (zero-pad back to n_fft//2+1) → reshape: (2, 2, freq, time) [stereo, complex] → transpose: (2, freq, time, 2) ← original spec format -
_demixlength guard added: after ISTFT,target_audiois trimmed or zero-padded to matchmix.shape[1]before subtraction.
Second-pass fix (after inspecting actual model): Inspecting UVR_MDXNET_KARA.onnx with onnxruntime revealed the true input shape is (batch, 4, **2048**, **256**) — not (batch, 4, 3072, 8). The hardcoded DIM_F=3072 and DIM_T=8 were simply wrong; different model files have different dimensions. Fix: removed all module-level DIM_F, DIM_T, N_FFT constants. _get_session() now reads session.get_inputs()[0].shape and stores self.dim_f = shape[2], self.dim_t = shape[3], self.n_fft = self.dim_f * 2. All calls to _stft, _spec_to_batch, _batch_to_spec, _istft pass these instance values explicitly. _batch_to_spec now reads dim_f/dim_t from the metadata dict rather than globals.
Third-pass fix — name 'N_FFT' is not defined: After removing the module-level constants, the three function signatures (_stft, _istft, _spec_to_batch) still used N_FFT, DIM_F, DIM_T as default parameter values, causing a NameError at import time. Fixed by replacing the defaults with inline literals (n_fft=4096, dim_f=2048, dim_t=256) matching the KARA model's actual dimensions. These defaults are never used in practice — MDXNetSeparator._demix always passes explicit values derived from the session.
MelBand Karaoke fix — 404 on model download
Problem: mel_band_karaoke separation failed with 404 Client Error during model download. The repo becruily/mel-band-roformer-karaoke still exists on HuggingFace, but the filenames changed: the actual files are mel_band_roformer_karaoke_becruily.ckpt (1.72 GB) and config_karaoke_becruily.yaml, not model.ckpt / config.yaml.
Two locations had the wrong filenames:
-
backend/routers/models.py—MODEL_DOWNLOAD_INFO["mel_band_karaoke"]:fileslist updated from["model.ckpt", "config.yaml"]to["mel_band_roformer_karaoke_becruily.ckpt", "config_karaoke_becruily.yaml"]. Also correctedsize_mbfrom170→1760to reflect the actual 1.72 GB checkpoint. -
backend/services/mel_band_specialized_separator.py—MelBandKaraokeSeparator.__init__:model_pathandconfig_pathupdated to use the correct filenames so that_ensure_model's existence check andsnapshot_downloadboth reference the right paths.
MelBand Karaoke fix — Missing key(s) in state_dict: static_alpha
Problem: After the download fix, model loading raised Error(s) in loading state_dict for MelBandRoformer: Missing key(s) in state_dict: "layers.0.0.layers.0.0.static_alpha", .... The installed bs_roformer library is newer than the checkpoint — a static_alpha learnable scalar was added to the RoFormer attention layers in a later release. The checkpoint was trained before this parameter existed, so it is absent from the saved weights.
Fix: Changed model.load_state_dict(state_dict) → model.load_state_dict(state_dict, strict=False) in MelBandSeparatorBase.get_model(). Missing keys keep their initialized values (static_alpha defaults to 1.0, a neutral multiplier). A logging.warning is emitted listing how many keys were missing, so any future version-mismatch is visible in logs without crashing the job.
MelBand Karaoke OOM fix — VRAM cap and OOM retry wired into separation path
Problem: mel_band_specialized_separator.py was not wired into StemEngine's global power/GPU settings. Two gaps:
-
No VRAM cap on separation.
_apply_cuda_limits()(which callstorch.cuda.set_per_process_memory_fraction(0.80)) was only ever called bylyric_transcriber_service.py. If separation ran before transcription (or transcription never ran in the session), the allocator had no VRAM ceiling and could OOM. -
No OOM retry in
_run_mel_band_karaoke. On anout of memoryexception the function just re-raised, letting the top-level handler report it as a CUDA error with no recovery.
Fixes applied to backend/services/separator_service.py:
-
VRAM cap: Added
from .lyric_transcriber_service import _apply_cuda_limits; _apply_cuda_limits()at the top ofrun_separation(), beforetorch.cuda.empty_cache(). Now separation and transcription share the same VRAM ceiling (default 80%, overridable viaSTEMENGINE_GPU_MEMORY_FRACTION). -
OOM retry in
_run_mel_band_karaoke: Wrappedsep.separate_into_parts(...)in a 2-attempt loop. On the firstout of memoryerror: clears CUDA cache, updates job progress ("GPU OOM — retrying with batch_size=1…"), and retries withbatch_size=1. If the second attempt also fails, the exception propagates normally to the top-level CUDA error handler.
MelBand Karaoke OOM fix (second pass) — revert VRAM cap; add pre-load flush
Problem: After wiring _apply_cuda_limits() into run_separation(), users saw OOM even with 4 GB physically free on the GPU. Root cause: set_per_process_memory_fraction(0.80) caps the process at 9.6 GB. PyTorch's allocator holds a reserved cache from prior jobs inside that budget; within the remaining headroom, only 1.95 GB was truly free — not enough for the 1.04 GiB allocation the separator needed. The GPU monitor showed 4 GB free because it counts PyTorch's cache as "available" while CUDA counts it as "in use."
Fix:
-
run_separation()now flushes cache BEFORE applying the VRAM cap:gc.collect() + torch.cuda.empty_cache()runs first to release stale allocations from prior jobs, then_apply_cuda_limits()is called. The order matters: if stale cached blocks are counted against the cap before they're released, the cap appears tighter than it actually is. On this WSL2 system ~2 GB is permanently reserved for Windows (DWM); the 80% cap on 12 GB = 9.6 GB leaves that headroom for Windows processes. Flushing first ensures the separator starts with a clean budget within the cap. -
Added
gc.collect() + torch.cuda.empty_cache()flush in_run_mel_band_karaokeimmediately before constructing the separator (which loads the model checkpoint). This second flush is necessary because allocations can accumulate between the start ofrun_separation()and when the model actually loads.
BSRoFormer wired into power settings (parity with MelBand Karaoke)
Audit of _run_bsroformer, _run_bsroformer_chorus, _run_bsroformer_sw revealed none had the pre-load VRAM flush or OOM retry that _run_mel_band_karaoke received. Applied the same pattern to all three:
- Pre-load flush:
gc.collect() + torch.cuda.empty_cache()added before the separator object is constructed in each runner. - OOM retry with batch_size=1: Added to
_run_bsroformerand_run_bsroformer_chorus(2-attempt loop, same as MelBand)._run_bsroformer_swgets flush only — it delegates to theaudio-separatorlibrary which doesn't expose abatch_sizeparameter. strict=Falseonload_state_dict: Applied tobs_roformer_separator.pyandbs_roformer_chorus_separator.pywith the same warning pattern asmel_band_specialized_separator.py— guards against future checkpoint/library version mismatches causing hard crashes.
2026-03-09 — Power settings wired into all remaining 10 separator runners
Extended the "power settings" pattern (pre-load VRAM flush + OOM retry + strict=False) to every remaining runner and separator that was missing it.
separator_service.py — pre-load gc+empty_cache flush added before the separator object is constructed in:
_run_mdxnet_run_mel_band_roformer(also gets OOM retry)_run_scnet(also gets OOM retry)_run_medleyvox_run_apollo_run_aspiration_run_bandit_run_ensemble_run_sepacap_run_sepreformer
OOM retry loops added:
_run_mel_band_roformer: 2-attempt loop; on first OOM, gc+cache flush then retry withbatch_size_override=1._run_scnet: 2-attempt loop; on first OOM, gc+cache flush then retry withchunk_size_override = (original or 262144) // 2.
strict=False on load_state_dict with warning applied to:
mel_band_roformer_separator.py(line 219): protects against missing keys (e.g.static_alpha) when library is newer than checkpoint.scnet_separator.py(line 249): same guard.
2026-03-09 — Fix post-cancel VRAM leak and wrong job status after mid-run cancellation
Root cause diagnosis:
- Cancellation is cooperative (flag-based). Once
separate_into_parts()starts GPU compute, it runs to completion even after cancel — the flag is only checked between major stages. This is why the GPU keeps working after cancel; it's a fundamental limitation of thread-based execution (Demucs avoids this by using a subprocess that can be killed). - When a separator's internal
should_cancel()check fires mid-run, it raisesRuntimeError("Cancelled by user"). This propagated torun_separation's top-levelexcept Exceptionblock, where it didn't match"CUDA", so: (a) no VRAM cleanup was done, and (b) the job was stored with status=ERROR instead of CANCELLED. The model's VRAM wasn't freed until Python's GC got around to it — which could be after the next job's model load begins → OOM despite apparent free VRAM.
Fixes in separator_service.py run_separation():
- Cancellation branch in
except: Addedif jm.should_cancel(job_id) or "Cancelled by user" in err_stras the first condition — sets status=CANCELLED (not ERROR) and skips the error message path. finallyblock: Added after theexceptblock — always runsgc.collect() + torch.cuda.empty_cache()regardless of how the job exits (success, error, or cancel). Ensures VRAM is returned to the CUDA driver before the next job starts.
2026-03-09 — Suppress noisy third-party FutureWarnings and demote missing-keys log to DEBUG
main.py: Added warnings.filterwarnings("ignore", ...) at startup to suppress FutureWarnings from rotary_embedding_torch (torch.cuda.amp.autocast deprecated) and contextlib (torch.backends.cuda.sdp_kernel deprecated). Both are upstream issues in third-party packages; the warnings are harmless.
All 5 separator files (mel_band_specialized_separator.py, mel_band_roformer_separator.py, scnet_separator.py, bs_roformer_separator.py, bs_roformer_chorus_separator.py): Demoted the "Checkpoint is missing N key(s)" log from WARNING to DEBUG. With up to 498 missing keys this was very noisy; the condition is expected and harmless (library newer than checkpoint, missing params use default init).
2026-03-09 — GPU power reduction: nvidia-smi power limit + TF32 + float16 autocast
Goal: Reduce GPU power draw (and fan noise) during inference.
lyric_transcriber_service.py _apply_cuda_limits() — three additions:
nvidia-smi --power-limit: Caps hardware TDP. Default 100 W (min for this GPU); override withSTEMENGINE_GPU_POWER_LIMITenv var. Set to 0 to disable. Silently ignored if the driver rejects (e.g. insufficient privilege). Called once per job at limits-apply time.- TF32 flags:
torch.backends.cuda.matmul.allow_tf32 = True,torch.backends.cudnn.allow_tf32 = True,torch.set_float32_matmul_precision('high')— enables TF32 for matmuls on Ampere+ GPUs; imperceptible quality difference for audio, reduces compute intensity. - Both are no-ops if CUDA is unavailable or the call fails.
float16 autocast added to 4 separator inference loops (enabled only when device != cpu):
bs_roformer_separator.py:torch.inference_mode()+torch.amp.autocast('cuda', dtype=torch.float16, enabled=(device != 'cpu'))mel_band_roformer_separator.py: samemel_band_specialized_separator.py: samescnet_separator.py:torch.no_grad()+ autocast (enabled=(device.type == 'cuda')since device istorch.device)
Autocast runs the model forward pass in float16 (halving memory bandwidth) while keeping the float32 accumulation buffers (result, counter) unaffected — PyTorch widens float16 outputs back to float32 on the += assignment automatically.
2026-03-09 — Move GPU power limit to container startup (start.sh + SYS_ADMIN cap)
Problem: nvidia-smi --power-limit requires SYS_ADMIN capability. The previous implementation called it on every job from _apply_cuda_limits(), which lacked the capability and failed silently.
Changes:
start.sh: Addednvidia-smi --power-limit=${STEMENGINE_GPU_POWER_LIMIT:-100}near the top. Runs once at startup as root before uvicorn. Warns (does not abort) if it fails.stemengine.yml: Addedcap_add: [SYS_ADMIN]so the container has the driver-level permission to write the power limit via NVML. Also addedSTEMENGINE_GPU_POWER_LIMIT=${STEMENGINE_GPU_POWER_LIMIT:-100}to the environment block (default 100 W; set to 0 to skip).lyric_transcriber_service.py_apply_cuda_limits(): Removed the per-job subprocess call; TF32 flags and VRAM fraction cap remain.
2026-03-09 — Harden SYS_ADMIN: move privilege to init container, main container unprivileged
Problem: SYS_ADMIN in the main container persists for its entire lifetime. Any RCE or path traversal in the app could exploit it. The start.sh volume mount (writable by default) was also a vector for injecting privileged commands.
Solution — init container pattern:
stemengine-gpu-initservice added tostemengine.yml: usesnvidia/cuda:12.1.0-base-ubuntu22.04, hascap_add: SYS_ADMIN, runsnvidia-smi --power-limitinline in its entrypoint, then exits (restart: "no"). The command is hardcoded in the compose file, not sourced from any mounted file.stemengineservice:cap_add: SYS_ADMINremoved entirely. Addeddepends_on: stemengine-gpu-init: condition: service_completed_successfullyso the main container waits for the init to finish before starting.start.shmount changed to read-only (:ro) — the container can no longer overwrite the startup script.start.sh: GPU power limit block removed (now handled by init container).
2026-03-09 — Fix stem rename filename format; add rescan-stems endpoint
Bug fix — rename_stem_endpoint (routers/projects.py):
The rename endpoint built the new filename as f"{new_name}{ext}", producing vocals.mp3 instead of Artist - Title (vocals).mp3. Fixed to use _stem_filename(project["name"], new_name, old_path.suffix) — the same helper used during separation. The DB stem key stays as the short name (e.g. "vocals"); only the filename on disk and in the files list changes.
New endpoint — POST /api/projects/{project_id}/rescan-stems:
Recursively walks ProjectDir/Stems/ and registers any audio files not already in the separation job's files list. Stem name is inferred from the filename:
Artist - Title (vocals).mp3→ stem key "vocals" (parenthesised suffix pattern)vocals.mp3→ stem key "vocals" (fallback: use filename stem directly) Also strips a leading project-name prefix if present. Returns{"added": N, "stems": [...]}. Idempotent — already-tracked paths are skipped.
2026-03-09
Fix: ELRC not generated + Syllables not appearing when Word Timestamps disabled
Root cause: lyricTranscribeProject in frontend/src/api/client.ts computed defaultFormats based on wordTs (from localStorage stemengine-whisper-word-ts). When the user had Word Timestamps disabled (wordTs=false), defaultFormats=['lrc','txt'] — no 'elrc' — regardless of lsyncAlign. Since syllable data lives in the ELRC file, both issues (no ELRC + no visible syllables) were caused by the same missing format.
Fix: Added lsyncAlign variable and effectiveWordTs = wordTs || lsyncAlign. When lsyncAlign=true, effectiveWordTs is forced true, ensuring 'elrc' is always included in defaultFormats and word_timestamps: true is sent to the backend.
Also reverted an earlier incorrect change that added 'sylrc' to formats when lsyncAlign=true (user confirmed sylrc is not needed; phonemeAlign is no longer used).