Commit
First push
This commit is contained in:
commit
c4a020c191
146 files changed
+40796
No files matched your search
@@ -0,0 +1,142 @@
|
||||
# StemEngine - Karaoke Anything?
|
||||
|
||||
> This is a **vibe-coded** project based on the work of people who actually know what they're doing. But now I can theoretically karaoke anything and so can you.
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **Stem Separation** — 40+ models across 10 model families
|
||||
- **MIDI Transcription** — basic-pitch with polyphonic and melody modes
|
||||
- **ABC Notation** — convert MIDI to ABC notation for sheet music
|
||||
- **Lyrics Sync** — forced-alignment to produce ELRC files with word-level timestamps and a live animated preview
|
||||
- **Project Management** — persistent project tabs, per-stem reprocessing, stem renaming
|
||||
- **Mix Presets** — Instrumental, Static (remove elements), Dynamic (per-stem volume)
|
||||
- **Subsonic Integration** — browse and import from a Subsonic-compatible server
|
||||
- **Model Downloader** — download model weights on-demand from the Settings UI
|
||||
|
||||
---
|
||||
|
||||
## Separation Models
|
||||
|
||||
| Family | Models |
|
||||
|---|---|
|
||||
| **Demucs** | v4, v4 Fine-tuned, v3 Extra Quantized, v3 |
|
||||
| **BS-RoFormer** | 4-stem, 5-stem (Guitar), 5-stem (Piano), 6-stem, Chorus, Anvuew, Karaoke Anvuew, SW 6-stem |
|
||||
| **MelBand RoFormer** | 4-stem, Vocals, Vocals Anvuew, Dereverb, Dereverb Anvuew, Dereverb Room, Karaoke, Karaoke Gabox, Deux Becruily |
|
||||
| **MDX-Net** | Vocals FT, Inst HQ 1, Inst HQ 2, Karaoke, Dereverb, Drum Sep |
|
||||
| **SCNet** | XL (2-stem), XL IHF, Large (4-stem), Large Starrytong |
|
||||
| **MedleyVox** | Base, 55/66/88/100 Epoch variants |
|
||||
| **Apollo** | Vocals, EDM, EDM Big |
|
||||
| **Aspiration** | Full, Light |
|
||||
| **Bandit v2** | Multi-Language, English |
|
||||
| **Ensemble** | Vocal Best Quality, 4-Stem, 6-Stem Full |
|
||||
|
||||
Model weights are downloaded on first use and cached in `/opt/`.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
- Docker with the NVIDIA Container Toolkit (for GPU acceleration)
|
||||
- NVIDIA GPU with CUDA 12.4 support
|
||||
- Docker Compose
|
||||
|
||||
CPU-only mode is available via `CPU_SEPARATION=true` but separation will be significantly slower.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Copy `.env.example` to `.env` and fill in the required values (see Configuration below).
|
||||
|
||||
2. Build and start:
|
||||
|
||||
```bash
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
3. Open `http://localhost:8000` in your browser.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
All configuration is via environment variables. The `docker-compose.yml` accepts these via a `.env` file in the same directory.
|
||||
|
||||
### Required
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `SEFOLDER` | Host path for persistent runtime data (uploads, jobs, DB, LRC sync) |
|
||||
| `ENGINEFOLDER` | Host path for model weight caches |
|
||||
| `PROJECTS` | Host path for separated project output |
|
||||
| `SECRET_KEY` | Random hex key for encrypting Subsonic credentials at rest. Generate with: `python3 -c "import secrets; print(secrets.token_hex(32))"` |
|
||||
|
||||
### Optional
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `CPU_SEPARATION` | `false` | Set to `true` to force CPU inference |
|
||||
| `MAX_UPLOAD_MB` | `500` | Maximum upload file size in MB |
|
||||
| `TZ` | `UTC` | Timezone |
|
||||
| `NAVSERVER` | — | Subsonic server URL (can also be set in the UI) |
|
||||
| `NAVUSER` | — | Subsonic username |
|
||||
| `NAVPASS` | — | Subsonic password |
|
||||
|
||||
### Model Directories
|
||||
|
||||
These are set automatically in `docker-compose.yml` and point inside the `/opt/` volume. Override only if you need a custom layout.
|
||||
|
||||
`BS_ROFORMER_MODEL_DIR`, `MEL_BAND_MODEL_DIR`, `MDXNET_MODEL_DIR`, `SCNET_MODEL_DIR`, `MEDLEYVOX_MODEL_DIR`, `APOLLO_MODEL_DIR`, `ASPIRATION_MODEL_DIR`, `BANDIT_MODEL_DIR`
|
||||
|
||||
---
|
||||
|
||||
## Project Folder Structure
|
||||
|
||||
Each separation job creates a folder under `PROJECTS_DIR`:
|
||||
|
||||
```
|
||||
Artist - Title/
|
||||
Artist - Title.mp3 # Original source audio (copy)
|
||||
Artist - Title.elrc # ELRC lyrics with word-level timestamps (if aligned)
|
||||
Artist - Title.lrc # Plain LRC (if aligned)
|
||||
Stems/
|
||||
vocals.mp3
|
||||
drums.mp3
|
||||
bass.mp3
|
||||
other.mp3
|
||||
Instrumental/ # Only when Instrumental preset selected
|
||||
Artist - Title - Instrumental.mp3
|
||||
Static/ # Only when Static preset selected
|
||||
Artist - Title - No Vocals.mp3
|
||||
Dynamic/ # Only when Dynamic preset selected
|
||||
Artist - Title - Dynamic Mix.mp3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Frontend** — React 18, TypeScript, Vite, TailwindCSS
|
||||
- **Backend** — FastAPI, Python 3.11, SQLAlchemy
|
||||
- **Separation** — Demucs, BS-RoFormer, MelBand RoFormer, MDX-Net (onnxruntime-gpu), SCNet, MedleyVox, Apollo, Aspiration, Bandit
|
||||
- **Transcription** — basic-pitch (ONNX, CUDAExecutionProvider)
|
||||
- **Lyrics Sync** — wav2vec2-based forced alignment via custom `lsync` module
|
||||
- **Runtime** — NVIDIA CUDA 12.4 + cuDNN 9, PyTorch 2.6 (cu124 wheels)
|
||||
- **Container** — `nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04`
|
||||
|
||||
---
|
||||
|
||||
## GPU Notes
|
||||
|
||||
The container requires the NVIDIA Container Toolkit. On the host:
|
||||
|
||||
```bash
|
||||
# Verify GPU is visible inside the container
|
||||
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi
|
||||
```
|
||||
|
||||
PyTorch wheels are self-contained (cu124) and do not require a matching CUDA toolkit on the host — only the driver.
|
||||
|
||||
`onnxruntime-gpu` is used for MDX-Net and basic-pitch ONNX inference; it auto-selects `CUDAExecutionProvider` when a GPU is available.
|
||||
Reference in new issue
Block a user