6.6 KiB
GPU Class Tuning & Recommendations (Phase 5)
This guide provides hardware-specific tuning recommendations for StemEngine separation across consumer, professional, and server-class GPUs.
Overview
StemEngine's chunking system allows dynamic tuning based on GPU memory class. Three strategies work in concert:
- Eco-params guardrail — Automatic parameter scaling for RoFormer family on small VRAM
- Long-track chunking — Time-domain splitting with configurable chunk/overlap sizes
- OOM retry ladder — Automatic fallback with reduced batch/segment sizes
GPU Classes & Recommendations
Entry-level GPUs (≤ 4 GB VRAM)
Examples: RTX 3050 (4GB), RTX 2060 (6GB), GTX 1660 (6GB)
Default Behavior:
- Eco-params forced on all MelBand separators
- Chunking mode:
auto(activates at 480s) - Chunk duration: 120 seconds (vs 180s default)
- Demucs segment: auto-scales to 4s for long tracks
Configuration:
LONG_TRACK_CHUNKING_MODE=auto
LONG_TRACK_CHUNK_DURATION_SECS=120 # Smaller chunks
LONG_TRACK_CHUNK_OVERLAP_SECS=5 # Smaller overlap
DEMUCS_SEGMENT_OVERRIDE=4 # Aggressive tuning
Expected Performance:
- Throughput: 0.5–1.0× real-time (10-min track → 10–20 min runtime)
- Max concurrent jobs: 1
- Supported models: All except BS-RoFormer 6-stem, MDX-Net large
- Failure modes: OOM on 4-stem + long track; fallback to Demucs recommended
Mid-range GPUs (6–8 GB VRAM)
Examples: RTX 3070 (8GB), RTX 4070 (12GB), RTX A5000 (24GB)
Default Behavior:
- MelBand Karaoke 4-stem: no eco-params, good quality
- Chunking mode:
auto(activates at 480s) - Chunk duration: 180 seconds
- Demucs segment: auto-scales to 8s for medium tracks
Configuration:
LONG_TRACK_CHUNKING_MODE=auto
LONG_TRACK_CHUNK_DURATION_SECS=180
LONG_TRACK_CHUNK_OVERLAP_SECS=10
DEMUCS_SEGMENT_OVERRIDE=null # Use auto-scaling
Expected Performance:
- Throughput: 1.5–2.5× real-time
- Max concurrent jobs: 2
- Supported models: All standard models
- Ideal workloads: Mixed 2–8 min tracks; good production baseline
High-end GPUs (≥ 12 GB VRAM)
Examples: RTX 3090 (24GB), RTX 4090 (24GB), A100 (40–80GB)
Default Behavior:
- No eco-params; full quality for all separators
- Chunking mode:
off(not needed for typical tracks) - Can run full 10–30 min tracks without chunking
- Demucs segment: default 12s (standard Demucs)
Configuration:
LONG_TRACK_CHUNKING_MODE=off # Disable for lowest latency
LONG_TRACK_DURATION_SECS=1200 # Increase threshold
DEMUCS_SEGMENT_OVERRIDE=null # Standard Demucs
CPU_SEPARATION=false # GPU-only mode
Expected Performance:
- Throughput: 3–5× real-time or better
- Max concurrent jobs: 2–4 (depending on model)
- Supported models: All models simultaneously
- Ideal workloads: Batch processing, live streaming
Model-Specific Tuning
MelBand Karaoke + Long Tracks
RTX 4090 baseline (no chunking needed):
input: 10-minute 44.1 kHz stereo
runtime: ~90–120 seconds
VRAM peak: 16–18 GB
RTX 3070 with chunking:
input: 10-minute 44.1 kHz stereo
config: 180s chunks, 10s overlap
runtime: ~8–10 minutes (OLA overhead)
VRAM peak: ~5 GB per chunk
RTX 3050 with aggressive tuning:
input: 10-minute 44.1 kHz stereo
config: 120s chunks, 5s overlap + eco-params
runtime: ~15–20 minutes
VRAM peak: ~3.5 GB
fallback: Demucs recommended for reliability
Demucs Segment Tuning
Auto-scaling formula (Phase 3):
if duration < 120s: segment = 12 # Default (best quality)
elif duration < 480s: segment = 8 # Quality-biased
else: segment = 4 # VRAM-biased
To override (e.g., for high-end GPU):
DEMUCS_SEGMENT_OVERRIDE=12 # Always use quality mode
BS-RoFormer 6-stem
Only runs on RTX 3090+ or with aggressive chunking:
RTX 4090 (recommended):
LONG_TRACK_CHUNKING_MODE=off
input: 6-minute maximum recommended
runtime: ~1–2 minutes
RTX 3070 (chunked):
LONG_TRACK_CHUNKING_MODE=auto
LONG_TRACK_CHUNK_DURATION_SECS=120
input: 10-minute maximum
runtime: ~10–15 minutes
VRAM peak: ~5 GB per chunk
Tuning Workflow
Step 1: Identify Your GPU
nvidia-smi --query-gpu=name,memory.total --format=csv,noheader
Step 2: Select Base Profile
Match your GPU to the class above; copy the recommended env vars.
Step 3: Test a Benchmark Track
Run a 10-minute test with your most common model:
# Monitor VRAM and runtime
nvidia-smi -l 1 # Refresh every 1 second
# Check job status at /api/jobs/{id}
Step 4: Tune Based on Results
| Symptom | Adjustment |
|---|---|
| OOM during separation | Reduce LONG_TRACK_CHUNK_DURATION_SECS by 30s, or enable chunking with always mode |
| VRAM unused (< 50% peak) | Increase chunk duration or disable chunking for faster latency |
| Seam artifacts at chunk boundaries | Increase LONG_TRACK_CHUNK_OVERLAP_SECS from 10s to 15s |
| Demucs slow on long tracks | Reduce DEMUCS_SEGMENT_OVERRIDE from 8 to 4 (or use auto-scaling) |
Step 5: Validate with Multiple Tracks
Test a variety: short (2 min), medium (5 min), long (15 min). Verify no regressions.
Advanced: Server Deployment
Multi-GPU Setup (A100 × 2)
# Each GPU runs independently; use load balancer to distribute jobs
CUDA_VISIBLE_DEVICES=0 # GPU 0
LONG_TRACK_CHUNKING_MODE=off
MAX_PARALLEL_JOBS=2
# Separate container
CUDA_VISIBLE_DEVICES=1 # GPU 1
LONG_TRACK_CHUNKING_MODE=off
MAX_PARALLEL_JOBS=2
High-Throughput CPU-Only Fallback
CPU_SEPARATION=true
# Slower but runs on any server; useful for backup/low-priority queue
Monitoring & Diagnostics
Enable per-stage diagnostics for MelBand:
MELBAND_STAGE_DIAGNOSTICS=true
This logs:
- Pre-separation VRAM allocator state
- Checkpoint load time
- Inference time per stage
- Post-separation VRAM release
Useful for identifying bottlenecks before OOM occurs.
Future Tuning (Phase 5+)
Planned improvements:
- Per-model VRAM profile: Learned from profiling, used to pre-allocate optimally
- Dynamic chunk-size selection: Based on available VRAM at job submit time
- Overlap-add quality metric: Measure seam artifacts; auto-adjust overlap if needed
- GPU utilization dashboard: Real-time VRAM/compute occupancy per job
Quick Reference
| GPU | Max Track Length (no chunking) | Recommended Chunk Size | Typical Throughput |
|---|---|---|---|
| RTX 3050 (4GB) | 2 min | 120s | 0.5× real-time |
| RTX 3070 (8GB) | 5 min | 180s | 1.5× real-time |
| RTX 4090 (24GB) | 30+ min | off | 3–5× real-time |