2026-03-16 16:35:08 +09:00
2026-03-09 17:55:38 +09:00
2026-03-03 14:52:19 +09:00
2026-03-17 11:39:17 +09:00
2026-03-16 16:35:08 +09:00
2026-03-03 14:52:19 +09:00
2026-03-17 11:39:17 +09:00
2026-03-13 17:55:07 +09:00
2026-03-17 11:39:17 +09:00
2026-03-13 15:32:26 +09:00

StemEngine - Karaoke Anything?

This is a vibe-coded project based on the work of people who actually know what they're doing. But now I can theoretically karaoke anything and so can you.


Features

  • Stem Separation — 40+ models across 10 model families
  • MIDI Transcription — basic-pitch with polyphonic and melody modes
  • GPU Safety Limits — shared VRAM reserve for separation, lyric transcription, basic-pitch, and Omnizart sidecar
  • ABC Notation — convert MIDI to ABC notation for sheet music
  • Lyrics Sync — forced-alignment to produce ELRC files with word-level timestamps and a live animated preview
  • Project Management — persistent project tabs, per-stem reprocessing, stem renaming
  • Mix Presets — Instrumental, Static (remove elements), Dynamic (per-stem volume)
  • Subsonic Integration — browse and import from a Subsonic-compatible server
  • Model Downloader — download model weights on-demand from the Settings UI

Separation Models

Family Models
Demucs v4, v4 Fine-tuned, v3 Extra Quantized, v3
BS-RoFormer 4-stem, 5-stem (Guitar), 5-stem (Piano), 6-stem, Chorus, Anvuew, Karaoke Anvuew, SW 6-stem
MelBand RoFormer 4-stem, Vocals, Vocals Anvuew, Dereverb, Dereverb Anvuew, Dereverb Room, Karaoke, Karaoke Gabox, Deux Becruily
MDX-Net Vocals FT, Inst HQ 1, Inst HQ 2, Karaoke, Dereverb, Drum Sep
SCNet XL (2-stem), XL IHF, Large (4-stem), Large Starrytong
MedleyVox Base, 55/66/88/100 Epoch variants
Apollo Vocals, EDM, EDM Big
Aspiration Full, Light
Bandit v2 Multi-Language, English
Ensemble Vocal Best Quality, 4-Stem, 6-Stem Full

Model weights are downloaded on first use and cached in /opt/.


Requirements

  • Docker with the NVIDIA Container Toolkit (for GPU acceleration)
  • NVIDIA GPU with CUDA 12.4 support
  • Docker Compose

CPU-only mode is available via CPU_SEPARATION=true but separation will be significantly slower.


Quick Start

  1. Copy .env.example to .env and fill in the required values (see Configuration below).

  2. Build and start:

docker compose up -d --build
  1. Open http://localhost:8000 in your browser.

Configuration

All configuration is via environment variables. The docker-compose.yml accepts these via a .env file in the same directory.

Required

Variable Description
SEFOLDER Host path for persistent runtime data (uploads, jobs, DB, LRC sync)
ENGINEFOLDER Host path for model weight caches
PROJECTS Host path for separated project output
SECRET_KEY Random hex key for encrypting Subsonic credentials at rest. Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"

Optional

Variable Default Description
CPU_SEPARATION false Set to true to force CPU inference
MAX_UPLOAD_MB 500 Maximum upload file size in MB
STEMENGINE_GPU_VRAM_RESERVE_MB 2048 Keeps this many MB free on the GPU (WSL2-safe headroom)
OMNIZART_GPU_VRAM_RESERVE_MB STEMENGINE_GPU_VRAM_RESERVE_MB Optional sidecar override for Omnizart TensorFlow GPU cap
TZ UTC Timezone
NAVSERVER — Subsonic server URL (can also be set in the UI)
NAVUSER — Subsonic username
NAVPASS — Subsonic password

Model Directories

These are set automatically in docker-compose.yml and point inside the /opt/ volume. Override only if you need a custom layout.

BS_ROFORMER_MODEL_DIR, MEL_BAND_MODEL_DIR, MDXNET_MODEL_DIR, SCNET_MODEL_DIR, MEDLEYVOX_MODEL_DIR, APOLLO_MODEL_DIR, ASPIRATION_MODEL_DIR, BANDIT_MODEL_DIR


Project Folder Structure

Each separation job creates a folder under PROJECTS_DIR:

Artist - Title/
  Artist - Title.mp3        # Original source audio (copy)
  Artist - Title.elrc       # ELRC lyrics with word-level timestamps (if aligned)
  Artist - Title.lrc        # Plain LRC (if aligned)
  Stems/
    vocals.mp3
    drums.mp3
    bass.mp3
    other.mp3
  Instrumental/             # Only when Instrumental preset selected
    Artist - Title - Instrumental.mp3
  Static/                   # Only when Static preset selected
    Artist - Title - No Vocals.mp3
  Dynamic/                  # Only when Dynamic preset selected
    Artist - Title - Dynamic Mix.mp3

Tech Stack

  • Frontend — React 18, TypeScript, Vite, TailwindCSS
  • Backend — FastAPI, Python 3.11, SQLAlchemy
  • Separation — Demucs, BS-RoFormer, MelBand RoFormer, MDX-Net (onnxruntime-gpu), SCNet, MedleyVox, Apollo, Aspiration, Bandit
  • Transcription — basic-pitch (ONNX, CUDAExecutionProvider)
  • Lyrics Sync — wav2vec2-based forced alignment via custom lsync module
  • Runtime — NVIDIA CUDA 12.4 + cuDNN 9, PyTorch 2.6 (cu124 wheels)
  • Container — nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04

Auto Pipeline + Lyric Engine Notes (2026-03)

  • Auto Pipeline now supports:
    • per-instrument refinement mapping (instrument_models) at separation stage
    • lyric engine selection (WhisperX or self-hosted Parakeet) in lyric steps
  • Saved auto-pipeline configs are migrated on read:
    • legacy whisperModel is upgraded to engine-aware engine + model
    • missing instrumentModels defaults to empty mapping

Parakeet constraints

Parakeet uses alternate handling and does not currently expose WhisperX-only controls:

  • diarize
  • lsync_align
  • two_pass
  • explicit
  • translate task

If Parakeet is selected and unavailable, verify sidecar health and profile wiring:

  • backend must have PARAKEET_ENABLED=true
  • backend PARAKEET_SIDECAR_URL must point to the sidecar service
  • sidecar GET /health must return 200

GPU Notes

The container requires the NVIDIA Container Toolkit. On the host:

# Verify GPU is visible inside the container
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

PyTorch wheels are self-contained (cu124) and do not require a matching CUDA toolkit on the host — only the driver.

STEMENGINE_GPU_VRAM_RESERVE_MB is applied in backend GPU code paths (separation, lyric transcription, and basic-pitch startup) to preserve display headroom on WSL2.

Omnizart sidecar now applies a TensorFlow memory cap using OMNIZART_GPU_VRAM_RESERVE_MB (or STEMENGINE_GPU_VRAM_RESERVE_MB fallback), so sidecar transcription follows the same reserve policy.

S
Description
No description provided
Readme
33 MiB
0 Stars 1 Watchers 0 Forks
Languages
Jupyter Notebook 90.9%
TypeScript 4.7%
Python 3.7%
JavaScript 0.6%