mirror of
https://github.com/RevLayDust/RLD-UVR-Opt.git
synced 2026-11-20 13:07:58 +09:00
docs(contributing): establish modular contribution guides and developer documentation
This commit is contained in:
1 parent
b7d71dcee0
commit
7daa15523f
8 files changed
+1031
-37
No files matched your search
@@ -21,7 +21,6 @@ output/*
|
||||
uvr_run.log
|
||||
data.pkl
|
||||
error.log
|
||||
CONTRIBUTING.md
|
||||
rubberband.exe
|
||||
ffmpeg.exe
|
||||
sndfile.dll
|
||||
|
||||
+79
-36
@@ -1,55 +1,98 @@
|
||||
# Contributing to RLD UVR-Opt
|
||||
|
||||
Thank you for your interest in contributing to **RLD UVR-Opt**! We welcome bug fixes, performance optimizations, and documentation improvements.
|
||||
Thank you for your interest in contributing to **RLD UVR-Opt**! We welcome bug fixes, performance optimizations, hardware verification, and documentation improvements.
|
||||
|
||||
This project is a performance-optimized fork of [Ultimate Vocal Remover GUI](https://github.com/Anjok07/ultimatevocalremovergui) (v5.6) focusing on low-latency ONNX Runtime inference, hardware-accelerated Execution Providers (CUDA and TensorRT-RTX), and modular environment architecture for Windows x64.
|
||||
|
||||
---
|
||||
|
||||
## Repository Structure
|
||||
## TL;DR 5-Step Contribution Flow
|
||||
|
||||
Contributions should be organized according to the existing module boundaries:
|
||||
1. **Fork & Clone**: Fork the repository on GitHub and clone your fork locally:
|
||||
```powershell
|
||||
git clone https://github.com/<your-username>/RLD-UVR-Opt.git
|
||||
cd "RLD-UVR-Opt"
|
||||
```
|
||||
|
||||
| Component | Path | Description |
|
||||
| :--- | :--- | :--- |
|
||||
| **CLI & Benchmarking** | `uvr_cli/` | Headless execution interface (`cli.py`), runner bridge (`runner.py`), model resolver (`model_resolver.py`), telemetry monitor (`telemetry.py`), and benchmark engine (`engine.py`). |
|
||||
| **Core Inference Engine** | `separate.py` | Core model inference routines, in-memory model cache (`MODEL_CACHE`), precision policies (`SmartPrecisionPolicy`), and memory lifecycle handlers. |
|
||||
| **Automated Tests** | `tests/` | Unit tests for CLI parsing, headless resolution, in-memory caching, progress bars, and batch line endings. |
|
||||
| **Installer Scripts** | `install/` | Windows batch installer (`install_packages.bat`) and runtime dependency specifications (`requirements.txt`). |
|
||||
2. **Set Up the Environment**: Run the modular PowerShell installer to configure a virtual environment:
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\install\Install-UVR.ps1 -Mode A
|
||||
```
|
||||
*(Select `A` for NVIDIA CUDA, `B` for TensorRT-RTX, or `C` for CPU-only. See [docs/development/setup.md](docs/development/setup.md) for manual setup.)*
|
||||
|
||||
3. **Create a Feature Branch**: Branch from the default branch with a descriptive prefix:
|
||||
```powershell
|
||||
git checkout -b feat/my-optimization
|
||||
# Or fix/my-bugfix, docs/my-documentation
|
||||
```
|
||||
|
||||
4. **Verify Locally**: Run test suites and verify line endings before committing:
|
||||
```powershell
|
||||
# Run all Python regression tests
|
||||
.\venv\Scripts\python.exe -m unittest discover tests -v
|
||||
|
||||
# Run PowerShell installer tests (if touching install scripts)
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -Command "Invoke-Pester -Path ./install/tests"
|
||||
```
|
||||
|
||||
5. **Submit a Pull Request**: Push your branch to GitHub and open a Pull Request using our [PR Template](.github/PULL_REQUEST_TEMPLATE.md).
|
||||
|
||||
---
|
||||
|
||||
## Development Guidelines
|
||||
## Before You Begin
|
||||
|
||||
1. **CLI Changes**:
|
||||
- New CLI arguments, commands, or telemetry metrics belong inside `uvr_cli/`.
|
||||
- Update `uvr_cli/README.md` whenever adding or changing command-line flags.
|
||||
|
||||
2. **Core Inference Changes**:
|
||||
- Changes to model separation logic or memory handling belong in `separate.py`.
|
||||
- Ensure in-memory caching semantics and model eviction behaviors are preserved.
|
||||
|
||||
3. **Writing Tests**:
|
||||
- Any modifications to the CLI or model cache must include corresponding unit tests in `tests/test_uvr_cli.py` or `tests/test_in_memory_model_cache.py`.
|
||||
|
||||
4. **Git Hygiene & Clean Pull Requests**:
|
||||
- Do **NOT** commit private audio tracks, raw model weights (`*.onnx`, `*.ckpt`, `*.pth`), stem files, or unredacted system logs.
|
||||
- The `.gitignore` is already configured to exclude model checkpoints and output directories.
|
||||
- **Check Existing Issues**: Search [GitHub Issues](https://github.com/RevLayDust/RLD-UVR-Opt/issues) to avoid duplicate work.
|
||||
- **Discuss Major Changes First**: If you plan to introduce an architectural change, rewrite chunking math, add dependencies, or alter CLI arguments, open an issue labeled `RFC` or `Feature Request` first.
|
||||
- **Do Not Commit Large Files**: Never commit raw model checkpoints (`*.onnx`, `*.pth`, `*.ckpt`), audio tracks, or temporary stems. The [.gitignore](.gitignore) file excludes model weights and output directories.
|
||||
|
||||
---
|
||||
|
||||
## Running Verification Tests
|
||||
## Detailed Development Guides
|
||||
|
||||
Before submitting changes, run the test suite to ensure everything passes:
|
||||
To keep this entry point concise, detailed documentation is organized in `docs/development/`:
|
||||
|
||||
```powershell
|
||||
# Run the complete test suite
|
||||
.\venv\Scripts\python.exe -m unittest discover tests -v
|
||||
| Guide | Description |
|
||||
| :--- | :--- |
|
||||
| [**Environment Setup**](docs/development/setup.md) | System prerequisites, installation modes (A, B, C), verification steps, and common troubleshooting. |
|
||||
| [**Architecture Overview**](docs/development/architecture.md) | Module tree, Execution Provider fallback ladder, inference engine decoupling, and data flow. |
|
||||
| [**Coding Standards**](docs/development/coding-standards.md) | Python & PowerShell rules, lazy imports, context locking avoidance, and commit conventions. |
|
||||
| [**Testing Guide**](docs/development/testing.md) | Running unit tests, smoke tests, Pester suites, and testing without physical GPUs. |
|
||||
| [**Benchmarking Methodology**](docs/development/benchmarking.md) | Mandatory performance profiling, telemetry capture, audio integrity verification, and report formats. |
|
||||
| [**Installer Development**](docs/development/installer.md) | The 5-phase installer engine (`Detect` -> `Plan` -> `Verify` -> `Install` -> `Validate`), `-DryRun`, and Pester fixtures. |
|
||||
|
||||
# Or run targeted test files
|
||||
.\venv\Scripts\python.exe -m unittest tests.test_uvr_cli tests.test_in_memory_model_cache -v
|
||||
```
|
||||
---
|
||||
|
||||
You can also run an end-to-end synthetic audio smoke test to verify the CLI inference pipeline:
|
||||
## Pull Request Checklist
|
||||
|
||||
When opening a Pull Request, ensure:
|
||||
|
||||
- [ ] All tests pass via `.\venv\Scripts\python.exe -m unittest discover tests`.
|
||||
- [ ] Installer tests pass via `Invoke-Pester -Path ./install/tests` if scripts in `install/` were modified.
|
||||
- [ ] No extraneous model weights, temporary stems, or private WAV files are committed.
|
||||
- [ ] Batch files maintain Windows CRLF line endings (tested via `tests.test_windows_batch_files`).
|
||||
- [ ] If changing inference logic or EP selection, verified benchmark results using `uvr_cli bench` are attached.
|
||||
- [ ] Code follows [Coding Standards](docs/development/coding-standards.md) (lazy imports, no Tkinter dependencies in computation modules).
|
||||
- [ ] Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/) (e.g., `feat:`, `fix:`, `docs:`, `refactor:`).
|
||||
|
||||
---
|
||||
|
||||
## Reporting Bugs and Hardware Benchmarks
|
||||
|
||||
- **Bug Reports**: Open an issue using the [Bug Report Template](.github/ISSUE_TEMPLATE/bug_report.yml). Attach your `install/install_record.json` and error logs.
|
||||
- **Hardware Benchmarks**: We actively collect performance telemetry across diverse GPU and CPU architectures. Please submit results using the [Performance Report Template](.github/ISSUE_TEMPLATE/performance_report.yml).
|
||||
|
||||
---
|
||||
|
||||
## License & Attribution
|
||||
|
||||
- **Core Application & GUI**: Licensed under the [MIT License](LICENSE). Upstream Ultimate Vocal Remover GUI is developed by Anjok07 and the UVR community (Copyright © 2022 Ultimate Vocal Remover). Modifications, TensorRT-RTX backend, and installer pipelines are contributed by RevLayDust and project contributors (Copyright © 2026 RevLayDust).
|
||||
- **Headless CLI & Benchmark Engine (`uvr_cli/`)**: The CLI-only execution interface and benchmarking engine in `uvr_cli/` is licensed under the [GNU Affero General Public License v3.0 or later](uvr_cli/__init__.py) (`SPDX-License-Identifier: AGPL-3.0-or-later`, Copyright © 2026 RevLayDust). Any new contributions inside `uvr_cli/` must comply with AGPL v3.
|
||||
- **Third-Party Components**: Model architectures (MDX-Net, Demucs, VR Arch), weights, and execution provider binaries retain their respective upstream licenses.
|
||||
|
||||
---
|
||||
|
||||
## Community & Questions
|
||||
|
||||
- Have a question or proposal? Open a discussion on [GitHub Discussions](https://github.com/RevLayDust/RLD-UVR-Opt/discussions) or create an issue with the `question` label.
|
||||
- Please review and abide by our [Code of Conduct](CODE_OF_CONDUCT.md) in all project interactions.
|
||||
|
||||
```powershell
|
||||
.\venv\Scripts\python.exe -m uvr_cli smoke-test
|
||||
```
|
||||
@@ -0,0 +1,142 @@
|
||||
# Architecture & System Design
|
||||
|
||||
This document describes the codebase architecture, module responsibilities, execution provider lifecycle, and component boundaries of **RLD UVR-Opt**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Directory Tree & Responsibilities
|
||||
|
||||
```text
|
||||
RLD-UVR-Opt/
|
||||
├── UVR.py # GUI presentation entry point (Tkinter layout, menus, event loop)
|
||||
├── separate.py # Core inference engine (MDX-Net, VR, Demucs processing & caching)
|
||||
├── __version__.py # Application VERSION and BUILD_ID metadata
|
||||
├── run_uvr.bat # Launcher batch script (activates venv and runs UVR.py via uv run)
|
||||
├── engine/
|
||||
│ └── tensorrt_rtx/ # TensorRT-RTX Execution Provider backend
|
||||
│ ├── __init__.py # Package export and provider constants
|
||||
│ ├── environment.py # GPU architecture check (CC >= 8.0) and DLL registration
|
||||
│ └── session.py # Session factory with automatic graceful fallback ladder
|
||||
├── uvr_cli/ # Headless command-line execution & benchmarking interface
|
||||
│ ├── cli.py # Argument parser and command dispatcher
|
||||
│ ├── runner.py # Headless model runner and audio loader bridge
|
||||
│ ├── model_resolver.py # Model metadata discovery and parameter resolution
|
||||
│ ├── engine.py # Multi-round benchmark coordinator
|
||||
│ ├── telemetry.py # Hardware telemetry sampler (pynvml, psutil, PyTorch VRAM)
|
||||
│ └── exporter.py # CSV and JSON result serializer
|
||||
├── gui_data/ # GUI configuration, constants, themes, and model parameters
|
||||
│ └── constants.py # Execution Provider names, choices, and UI strings
|
||||
├── install/ # 5-phase modular installer pipeline
|
||||
│ ├── Install-UVR.ps1 # PowerShell orchestrator
|
||||
│ ├── install_packages.bat # Thin launcher for Install-UVR.ps1
|
||||
│ ├── lib/ # Pipeline phases (Config, Detect, Plan, Verify, Install, Validate, Record)
|
||||
│ ├── reqs/ # Stack requirements (requirements.txt, requirements_trt_rtx.txt, ...)
|
||||
│ ├── tests/ # Pester unit tests for installer planning and verification gates
|
||||
│ └── wheels/ # Pre-compiled Windows x64 binary wheels (diffq)
|
||||
├── tests/ # Automated unit tests for CLI, cache, TRT, and batch files
|
||||
└── benchmark_results/ # Performance datasets, CSV summaries, and per-round session logs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. High-Level Architecture Diagram
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Presentation ["Presentation Layer"]
|
||||
GUI["UVR.py (Tkinter GUI)"]
|
||||
CLI["uvr_cli (Headless CLI)"]
|
||||
end
|
||||
|
||||
subgraph Orchestration ["Orchestration & Resolution"]
|
||||
MR["uvr_cli.model_resolver"]
|
||||
GD["gui_data.constants"]
|
||||
end
|
||||
|
||||
subgraph CoreEngine ["Core Inference Engine"]
|
||||
SEP["separate.py (SeAudio / ModelData)"]
|
||||
CACHE["In-Memory ModelCache"]
|
||||
PREC["SmartPrecisionPolicy (FP32 / FP16)"]
|
||||
end
|
||||
|
||||
subgraph EPBackend ["Execution Provider Backend"]
|
||||
TRT["engine.tensorrt_rtx (TRT-RTX Factory)"]
|
||||
CUDA["CUDAExecutionProvider"]
|
||||
CPU["CPUExecutionProvider"]
|
||||
end
|
||||
|
||||
GUI --> GD
|
||||
GUI --> SEP
|
||||
CLI --> MR
|
||||
CLI --> SEP
|
||||
|
||||
SEP --> CACHE
|
||||
SEP --> PREC
|
||||
SEP --> TRT
|
||||
|
||||
TRT -->|"Compute Capability >= 8.0"| NVTRT["NvTensorRTRTXExecutionProvider"]
|
||||
TRT -->|"Fallback or Turing CC 7.5"| CUDA
|
||||
TRT -->|"CPU Selection or Fallback"| CPU
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Execution Provider Selection & Fallback Ladder
|
||||
|
||||
RLD UVR-Opt supports three Execution Providers (EP) in ONNX Runtime:
|
||||
1. **`NvTensorRTRTXExecutionProvider`** (`trt_rtx`): Accelerated TensorRT runtime for RTX 30/40/50-series (Ampere+ / CC $\ge 8.0$).
|
||||
2. **`CUDAExecutionProvider`** (`cuda`): Standard CUDA provider for NVIDIA GPUs (Turing RTX 20+ / CC $\ge 7.5$).
|
||||
3. **`CPUExecutionProvider`** (`cpu`): Universal CPU execution using OpenMP/MKL thread pools.
|
||||
|
||||
### Provider Resolution Ladder
|
||||
|
||||
When TensorRT-RTX is requested, the session factory ([engine/tensorrt_rtx/session.py](../../engine/tensorrt_rtx/session.py)) executes an explicit fallback chain:
|
||||
|
||||
```text
|
||||
[Request: TensorRT-RTX]
|
||||
│
|
||||
├──► 1. Check Hardware (Compute Capability >= 8.0)
|
||||
│ └── If False: Fall back to CUDAExecutionProvider immediately.
|
||||
│
|
||||
├──► 2. Standalone EP ABI Plugin (onnxruntime-ep-nv-tensorrt-rtx)
|
||||
│ └── Query ort.get_ep_devices() -> build SessionOptions with TRT config.
|
||||
│
|
||||
├──► 3. Built-in NvTensorRTRTXExecutionProvider
|
||||
│ └── If plugin device absent, attempt built-in provider symbol.
|
||||
│
|
||||
├──► 4. Stable CUDA Fallback
|
||||
│ └── If TensorRT engine creation fails, fall back to CUDAExecutionProvider.
|
||||
│
|
||||
└──► 5. Safe CPU Fallback
|
||||
└── If GPU runtime is missing, evaluate on CPUExecutionProvider.
|
||||
```
|
||||
|
||||
### No Silent Fallback Rule
|
||||
A common pitfall in ONNX Runtime is silent fallback: when a requested Execution Provider fails to load internal dependencies, ONNX Runtime may quietly fall back to CPU without throwing an exception.
|
||||
|
||||
In RLD UVR-Opt:
|
||||
- All session factories inspect `session.get_providers()` immediately after instantiation.
|
||||
- Active providers are logged explicitly to the terminal/logging pipeline:
|
||||
```python
|
||||
active_provider = session.get_providers()[0]
|
||||
print(f"[BACKEND] Active execution provider: {active_provider}")
|
||||
```
|
||||
- If a fallback occurs (e.g. from TensorRT-RTX to CUDA), a warning is logged so telemetry and benchmark recordings never record a fallback as an accelerated run.
|
||||
|
||||
---
|
||||
|
||||
## 4. Component Boundaries & Pipeline Decoupling
|
||||
|
||||
To ensure testability and headless execution:
|
||||
|
||||
1. **Decoupled Computation**:
|
||||
- `separate.py`, `engine/tensorrt_rtx/`, and `uvr_cli/` must never import Tkinter or reference GUI widgets (`root`, `ttk.Entry`, etc.).
|
||||
- Progress reporting uses standard callbacks: `write_progress(value: float, message: str)`.
|
||||
|
||||
2. **In-Memory Model Caching**:
|
||||
- `separate.py` maintains an in-memory resident cache (`ModelCache`) to avoid reloading heavy ONNX graphs when running repeated chunks or identical models across tracks.
|
||||
- Cache keys consist of `(model_path, precision, device)`. If precision or device changes, the cache evicts previous handles safely to prevent VRAM fragmentation.
|
||||
|
||||
3. **In-Memory FP16 Downcasting**:
|
||||
- When Performance (FP16) mode is enabled on FP32 models, `onnxconverter_common.convert_float_to_float16` converts graphs entirely in system memory.
|
||||
- Converted bytes are passed directly to `ort.InferenceSession` as an in-memory byte buffer without writing temporary `.fp16.onnx` files to disk.
|
||||
@@ -0,0 +1,253 @@
|
||||
# Benchmarking & Performance Profiling
|
||||
|
||||
This document specifies the official methodology, metric definitions, and reporting requirements for performance benchmarks in **RLD UVR-Opt**.
|
||||
|
||||
---
|
||||
|
||||
## 1. When Benchmarking is Mandatory
|
||||
|
||||
Benchmarking is strictly required for any Pull Request that:
|
||||
1. Modifies inference routines in `separate.py` (chunking, windowing, STFT/iSTFT).
|
||||
2. Alters Execution Provider registration or session options in `engine/tensorrt_rtx/`.
|
||||
3. Changes precision downcasting policies (`SmartPrecisionPolicy`, in-memory FP16).
|
||||
4. Modifies buffer allocation or headless runner bridges in `uvr_cli/`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Metric Definitions & Formulas
|
||||
|
||||
To ensure reproducible comparisons across contributors and hardware:
|
||||
|
||||
| Metric | Identifier | Definition & Formula | Unit |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Inference Time** | `total_time_sec` | Total wall-clock time from tensor chunk submission until final stems are written. | Seconds ($s$) |
|
||||
| **Real-Time Factor** | `speed_factor_rt` | Processing speed relative to audio playback speed: $\text{Speed Factor} = \frac{\text{Audio Duration (s)}}{\text{Inference Time (s)}}$ | Multiplier ($x\text{ RT}$) |
|
||||
| **Throughput** | `throughput` | Audio duration processed per unit of time: $\text{Throughput} = \frac{\text{Audio Duration}}{\text{Total Time}}$ | Audio sec/sec |
|
||||
| **Chunk Latency (p50 / p95)** | `chunk_latency_ms` | Per-chunk inference latency evaluated at median (50th percentile) and tail (95th percentile). | Milliseconds ($\text{ms}$) |
|
||||
| **Active Tensor VRAM** | `peak_torch_vram_mb` | Peak memory explicitly allocated to active PyTorch tensors: `torch.cuda.max_memory_allocated()`. | Megabytes ($\text{MB}$) |
|
||||
| **Peak Device VRAM** | `peak_device_vram_mb` | Total physical GPU VRAM consumed across CUDA contexts and ORT memory arenas, measured via NVML `nvmlDeviceGetMemoryInfo()`. | Megabytes ($\text{MB}$) |
|
||||
| **Peak Process RAM** | `peak_process_ram_mb` | Maximum resident set size (RSS) of the application process: `psutil.Process().memory_info().rss`. | Megabytes ($\text{MB}$) |
|
||||
|
||||
### Statistical Reporting Standards
|
||||
When reporting steady-state metrics, contributors must provide:
|
||||
- **Sample Size ($n$)**: Minimum 9 measured repetitions `[PROPOSED]`.
|
||||
- **Central Tendency**: Median or Mean.
|
||||
- **Dispersion**: Standard Deviation ($s$) or Interquartile Range ($\text{IQR} = Q_3 - Q_1$).
|
||||
- **Single-run submissions are strictly rejected.**
|
||||
|
||||
---
|
||||
|
||||
## 3. Engine Build vs. Steady-State Measurement
|
||||
|
||||
Accelerated Execution Providers (especially TensorRT) introduce significant latency on the first inference pass due to kernel timing calibration, graph optimization, and engine compilation:
|
||||
|
||||
1. **Cold Run (Overhead)**:
|
||||
- The initial pass where ONNX Runtime compiles the execution graph and writes plan caches to `models/MDX_Net_Models/trt_cache/`.
|
||||
- **Must be reported separately** as the cold/warmup run latency and must **never** be averaged into steady-state numbers.
|
||||
2. **Warm Run (Steady-State)**:
|
||||
- Measured rounds after the engine has been compiled and cached in resident memory.
|
||||
- At least **2 warmup rounds** `[PROPOSED]` with dummy tensors or test audio must be completed before recording measurement rounds to eliminate cold-start spikes.
|
||||
|
||||
---
|
||||
|
||||
## 4. Controlled Variables & Benchmark Profile
|
||||
|
||||
All standardized benchmarks must be conducted using the following baseline profile:
|
||||
|
||||
| Parameter | Standard Value | Description |
|
||||
| :--- | :--- | :--- |
|
||||
| **Test Model** | `UVR-MDX-NET-Inst_HQ_4.onnx` | MDX-Net architecture FP32 model. |
|
||||
| **Segment Size** | `4000` | Standard sample window length. |
|
||||
| **Overlap** | `0.75` | Standard high-quality overlap factor. |
|
||||
| **Batch Size** | `1` | Single stream inference. |
|
||||
| **Precision** | `Ultra Quality (FP32)` | Baseline precision mode. Explicitly pass `--precision fp32` in CLI. |
|
||||
| **Test Audio** | Standard stereo WAV | Minimum 30–60 seconds duration (e.g. 48.34s test sample). Free-licensed or synthetic only. |
|
||||
|
||||
### Test Audio Licensing Rules
|
||||
- Audio samples must come from freely licensed sources (CC0, Creative Commons Attribution, or public domain) or synthetic waveforms (`np.sin`).
|
||||
- **Never commit copyrighted audio files, commercial music tracks, or raw stems to the repository.**
|
||||
|
||||
Contributors must also attach their [install/install_record.json](../../install/install_record.json) or include OS version, GPU model, and NVIDIA display driver version.
|
||||
|
||||
---
|
||||
|
||||
## 5. Audio Integrity & Quality Tolerances
|
||||
|
||||
Performance optimization must **not** degrade audio separation quality. When submitting optimization PRs:
|
||||
|
||||
1. **Numerical Residual Check**:
|
||||
- Compare output stem arrays against the baseline FP32 output:
|
||||
$$\Delta_{\max} = \max |A_{\text{opt}} - A_{\text{base}}|$$
|
||||
- In pure FP32 code optimization, $\Delta_{\max} \le 10^{-5}$.
|
||||
2. **Signal Degradation Tolerance (FP16 / Quantization)**:
|
||||
- For lower-precision paths, compute Signal-to-Distortion Ratio (SDR) or Signal-to-Noise Ratio (SNR):
|
||||
$$\text{SNR} = 10 \log_{10} \left( \frac{\sum A_{\text{base}}^2}{\sum (A_{\text{base}} - A_{\text{opt}})^2} \right)$$
|
||||
- Minimum acceptable threshold: **$\text{SNR} \ge 40\text{ dB}$** `[PROPOSED]`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Standard Execution Procedure
|
||||
|
||||
Follow this step-by-step procedure when running benchmark rounds:
|
||||
|
||||
### Step 1: Prepare Environment
|
||||
- Close all background applications (web browsers, media players, IDEs).
|
||||
- Verify Windows Power Plan is set to **High Performance**.
|
||||
- Check that the GPU is at baseline temperature ($< 50^\circ\text{C}$) to avoid thermal throttling.
|
||||
|
||||
### Step 2: Warm-up Phase
|
||||
Execute at least 2 warmup passes on each target provider to compile caches:
|
||||
```powershell
|
||||
.\venv\Scripts\python.exe -m uvr_cli bench -a "sample.wav" -m "UVR-MDX-NET-Inst_HQ_4.onnx" --precision fp32 --device cuda:0 --warmup 2 --rounds 1
|
||||
.\venv\Scripts\python.exe -m uvr_cli bench -a "sample.wav" -m "UVR-MDX-NET-Inst_HQ_4.onnx" --precision fp32 --device trt_rtx:0 --warmup 2 --rounds 1
|
||||
```
|
||||
|
||||
### Step 3: Interleaved Measurement Rounds (Mitigating Thermal Drift)
|
||||
To avoid thermal bias where the second engine tested runs hotter than the first, execute alternating rounds across configurations:
|
||||
```powershell
|
||||
# Interleaved benchmark execution pattern:
|
||||
# Alternate between Baseline and Target across rounds:
|
||||
for ($r = 1; $r -le 9; $r++) {
|
||||
Write-Host "--- Executing Measurement Round $r ---"
|
||||
# 1. Baseline Run (CUDA EP)
|
||||
.\venv\Scripts\python.exe -m uvr_cli bench -a "sample.wav" -m "UVR-MDX-NET-Inst_HQ_4.onnx" --precision fp32 --device cuda:0 --rounds 1 --warmup 0
|
||||
# 2. Optimized Run (TensorRT-RTX EP)
|
||||
.\venv\Scripts\python.exe -m uvr_cli bench -a "sample.wav" -m "UVR-MDX-NET-Inst_HQ_4.onnx" --precision fp32 --device trt_rtx:0 --rounds 1 --warmup 0
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Outlier Rejection Policy
|
||||
- If an individual measurement run suffers an operating system spike (e.g. Windows background indexing, GPU interrupt), it may be excluded if it deviates by more than $1.5 \times \text{IQR}$ from the median.
|
||||
- **All excluded outlier runs must be explicitly listed and justified in the PR submission table.**
|
||||
|
||||
Telemetry and per-round metrics are automatically saved to `benchmark_results/summary.csv` and timestamped session JSON files.
|
||||
|
||||
---
|
||||
|
||||
## 7. Submission Templates
|
||||
|
||||
### Markdown Table Template for PRs
|
||||
|
||||
Copy and fill this table in your Pull Request description:
|
||||
|
||||
```markdown
|
||||
### Benchmark Results
|
||||
|
||||
**Test Hardware**: [e.g., NVIDIA GeForce RTX 4070 (Driver 572.16) | AMD Ryzen 7 7800X3D | Windows 11]
|
||||
**Audio Sample**: [e.g., sample.wav (48.34s stereo, 44.1kHz, CC0)]
|
||||
**Model**: `UVR-MDX-NET-Inst_HQ_4.onnx` (Overlap 0.75, Segment 4000, FP32)
|
||||
**Sample Count**: n = 9 measured rounds (after 2 warmups, interleaved order)
|
||||
|
||||
| Execution Provider | Warmup Time | Steady-State Time (Median / Mean ± Std) | Chunk Latency (p50 / p95) | Real-Time Factor | Peak Device VRAM | Peak Process RAM |
|
||||
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
|
||||
| **CUDA EP (Baseline)** | 0.00 s | 0.00 s / 0.00 s ± 0.00 s | 0.0 ms / 0.0 ms | 0.00x RT | 0,000 MB | 0,000 MB |
|
||||
| **Your Optimization** | 0.00 s | 0.00 s / 0.00 s ± 0.00 s | 0.0 ms / 0.0 ms | 0.00x RT | 0,000 MB | 0,000 MB |
|
||||
| **Delta / Speedup** | — | **~0.00x Faster** | **-00.0% Latency** | **+00.0% Throughput** | **-00.0% VRAM** | **-00.0% RAM** |
|
||||
|
||||
*Session IDs: `session_<timestamp>_<id>` (Optimized) vs `session_<timestamp>_<id>` (Baseline).*
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Machine-Readable JSON Schema (Draft 2020-12)
|
||||
|
||||
Benchmark results can be validated against the formal schema below:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "RLDUVRBenchmarkSession",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"session_id",
|
||||
"timestamp_utc",
|
||||
"model_info",
|
||||
"summary"
|
||||
],
|
||||
"properties": {
|
||||
"session_id": { "type": "string" },
|
||||
"timestamp_utc": { "type": "string" },
|
||||
"model_info": {
|
||||
"type": "object",
|
||||
"required": ["model_name", "backend", "device", "precision", "overlap", "segment_size"],
|
||||
"properties": {
|
||||
"model_name": { "type": "string" },
|
||||
"backend": { "type": "string" },
|
||||
"device": { "type": "string" },
|
||||
"precision": { "type": "string" },
|
||||
"overlap": { "type": "number" },
|
||||
"segment_size": { "type": "integer" }
|
||||
}
|
||||
},
|
||||
"summary": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"warmup_rounds",
|
||||
"rounds_measured",
|
||||
"avg_total_time_sec",
|
||||
"avg_speed_factor_rt",
|
||||
"peak_device_vram_mb"
|
||||
],
|
||||
"properties": {
|
||||
"warmup_rounds": { "type": "integer" },
|
||||
"rounds_measured": { "type": "integer" },
|
||||
"avg_total_time_sec": { "type": "number" },
|
||||
"median_total_time_sec": { "type": "number" },
|
||||
"avg_speed_factor_rt": { "type": "number" },
|
||||
"median_speed_factor_rt": { "type": "number" },
|
||||
"min_total_time_sec": { "type": "number" },
|
||||
"max_total_time_sec": { "type": "number" },
|
||||
"peak_device_vram_mb": { "type": "number" },
|
||||
"peak_torch_allocated_vram_mb": { "type": "number" },
|
||||
"chunk_latency_p50_ms": { "type": "number" },
|
||||
"chunk_latency_p95_ms": { "type": "number" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Valid Session JSON Example (Placeholders Only)
|
||||
|
||||
*Note: All values below are illustrative placeholders per project contribution standards.*
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "session_<YYYYMMDD_HHMMSS>_<hex>",
|
||||
"timestamp_utc": "2026-01-01T00:00:00Z",
|
||||
"model_info": {
|
||||
"model_name": "UVR-MDX-NET-Inst_HQ_4.onnx",
|
||||
"backend": "MDX-Net",
|
||||
"device": "trt_rtx:0",
|
||||
"precision": "Ultra Quality (FP32)",
|
||||
"overlap": 0.75,
|
||||
"segment_size": 4000
|
||||
},
|
||||
"summary": {
|
||||
"warmup_rounds": 2,
|
||||
"rounds_measured": 9,
|
||||
"avg_total_time_sec": 0.00,
|
||||
"median_total_time_sec": 0.00,
|
||||
"avg_speed_factor_rt": 0.00,
|
||||
"median_speed_factor_rt": 0.00,
|
||||
"min_total_time_sec": 0.00,
|
||||
"max_total_time_sec": 0.00,
|
||||
"peak_device_vram_mb": 0.0,
|
||||
"peak_torch_allocated_vram_mb": 0.0,
|
||||
"chunk_latency_p50_ms": 0.0,
|
||||
"chunk_latency_p95_ms": 0.0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Common Benchmarking Pitfalls
|
||||
|
||||
1. **Averaging the Cold Run**: Counting the initial compilation pass into the average latency drastically inflates steady-state timing.
|
||||
2. **Thermal Throttling**: Running 20 consecutive passes on an overheated GPU will skew measurements downward. Ensure adequate cooling pauses and use interleaved testing.
|
||||
3. **Silent Provider Fallbacks**: Always confirm `Active execution provider` in terminal logs to ensure your run actually evaluated on the GPU.
|
||||
4. **Different Test Audio Lengths**: Comparing speed factor across tracks of differing audio lengths or sample rates introduces bias.
|
||||
5. **Missing Explicit Precision Flag**: Forgetting `--precision fp32` in the CLI causes `uvr_cli bench` to run in its default FP16 mode instead of the required FP32 baseline.
|
||||
@@ -0,0 +1,156 @@
|
||||
# Coding Standards & Architecture Constraints
|
||||
|
||||
This document defines the coding standards, project invariants, and architectural anti-patterns enforced in **RLD UVR-Opt**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Technical Invariants & Anti-Patterns
|
||||
|
||||
### A. Deferred & Lazy Imports
|
||||
Heavy machine learning dependencies (`torch`, `onnxruntime`, `tensorrt_rtx`) must **not** be imported at module root level during application startup. Importing heavy DLLs during root load slows down CLI command dispatch (`uvr_cli --help`, `list-models`) and causes unnecessary cold-start latency.
|
||||
|
||||
#### Bad (Root-Level Eager Import)
|
||||
```python
|
||||
# BAD: Imported when module is evaluated, slowing CLI launch
|
||||
import torch
|
||||
import onnxruntime as ort
|
||||
|
||||
def process_audio(audio_path: str) -> None:
|
||||
session = ort.InferenceSession("model.onnx")
|
||||
...
|
||||
```
|
||||
|
||||
#### Good (Deferred Local Import)
|
||||
```python
|
||||
# GOOD: Imports are deferred until the inference function is actually called
|
||||
def process_audio(audio_path: str) -> None:
|
||||
import torch
|
||||
import onnxruntime as ort
|
||||
from engine.tensorrt_rtx.session import create_trt_rtx_session
|
||||
|
||||
session = create_trt_rtx_session("model.onnx")
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### B. Context Lock Prevention
|
||||
Initializing CUDA context or calling `torch.cuda.init()` in the main thread before Execution Provider resolution locks GPU device context. This can cause ONNX Runtime providers (especially TensorRT) to fail or run into resource contention.
|
||||
|
||||
#### Bad (Premature CUDA Initialization)
|
||||
```python
|
||||
# BAD: Eagerly queries device before user or CLI arguments select EP
|
||||
device = "cuda:0" if torch.cuda.is_available() else "cpu"
|
||||
torch.cuda.init()
|
||||
```
|
||||
|
||||
#### Good (Deferred Device Binding)
|
||||
```python
|
||||
# GOOD: Defer device context initialization until model session is instantiated
|
||||
def resolve_inference_device(device_arg: str) -> str:
|
||||
# Validate string argument without initializing CUDA driver context
|
||||
if device_arg.startswith("cuda") or device_arg.startswith("trt_rtx"):
|
||||
return device_arg
|
||||
return "cpu"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### C. Pure Pipeline Decoupling
|
||||
Inference routines in `separate.py`, `engine/tensorrt_rtx/`, and `uvr_cli/` must remain completely decoupled from GUI presentation widgets (Tkinter, `ttk`).
|
||||
|
||||
- **Forbidden**: Passing `tk.Tk`, `ttk.Progressbar`, or referencing GUI global state inside model runner functions.
|
||||
- **Allowed**: Standard function signatures accepting primitive arguments (`str`, `Path`, `float`, `int`, `dict`) and reporting progress through callable callbacks:
|
||||
```python
|
||||
def run_separation(
|
||||
audio_path: Path,
|
||||
model_path: Path,
|
||||
progress_callback: Optional[Callable[[float, str], None]] = None,
|
||||
) -> Tuple[Dict[str, np.ndarray], bool]:
|
||||
...
|
||||
if progress_callback:
|
||||
progress_callback(0.5, "Processing chunk 2/4...")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### D. PowerShell Invariants & Error Handling
|
||||
All scripts in [install/](../../install/) must maintain strict compatibility with **Windows PowerShell 5.1**:
|
||||
|
||||
1. **Strict Mode**: Every `.ps1` file must begin with:
|
||||
```powershell
|
||||
Set-StrictMode -Version Latest
|
||||
```
|
||||
2. **Native Process Invocation**: Every external command (`python.exe`, `uv.exe`, `git.exe`) must verify exit codes:
|
||||
```powershell
|
||||
# Use the shared Invoke-Native helper or check $LASTEXITCODE explicitly
|
||||
$res = Invoke-Native -FilePath 'uv.exe' -ArgumentList @('pip', 'install', '-r', $reqFile)
|
||||
if (-not $res.Success) {
|
||||
throw "Command failed with exit code $($res.ExitCode): $($res.StdErr)"
|
||||
}
|
||||
```
|
||||
3. **No PS7 Syntactic Sugar**: Do not use PowerShell 7+ features:
|
||||
- Ternary operator (`?:`) → use `if (...) { } else { }`.
|
||||
- Null-coalescing (`??`) → use `if ($null -eq $x) { $x = ... }`.
|
||||
- Pipeline chaining (`&&`, `||`) → use explicit conditional statements.
|
||||
4. **Encoding**: Save PowerShell scripts as UTF-8 without BOM or ASCII.
|
||||
|
||||
---
|
||||
|
||||
### E. Installer Architectural Invariants
|
||||
1. **Zero Side-Effects in Early Phases**:
|
||||
- `Detect.ps1` (Phase 1), `Plan.ps1` (Phase 2), and `Verify.ps1` (Phase 3) must be strictly read-only.
|
||||
- `-DryRun` must exit at Phase 3 with zero modifications to filesystem, registry, or environment.
|
||||
2. **Single Source of Truth**:
|
||||
- All version pins (PyTorch, ONNX Runtime, plugins, driver thresholds) must reside exclusively in [install/lib/Config.ps1](../../install/lib/Config.ps1).
|
||||
3. **Additive Record Schema**:
|
||||
- Changes to [install/install_record.json](../../install/install_record.json) must be purely additive to preserve backward compatibility with `install_record.py inspect-stack`.
|
||||
4. **No Silent EP Fallbacks**:
|
||||
- Inference sessions must explicitly inspect `session.get_providers()` and log the active provider.
|
||||
|
||||
---
|
||||
|
||||
## 2. Python Code Standards
|
||||
|
||||
- **Formatting**: Adhere to PEP 8 with 4-space indentation.
|
||||
- **Type Annotations**: Provide type hints for public module functions and classes.
|
||||
- **Logging vs Print**: In library code (`engine/`, `uvr_cli/`), use structured logging or verbose flags rather than raw `print()` statements.
|
||||
- **Error Handling**: Catch specific exceptions rather than bare `except:`. Provide descriptive error messages.
|
||||
|
||||
---
|
||||
|
||||
## 3. Git Commit & Branch Conventions
|
||||
|
||||
### Commit Messages
|
||||
We follow the [Conventional Commits](https://www.conventionalcommits.org/) specification:
|
||||
|
||||
```text
|
||||
<type>(<scope>): <short description in imperative mood>
|
||||
|
||||
[optional body explaining context and rationale]
|
||||
|
||||
[optional footer(s)]
|
||||
```
|
||||
|
||||
#### Allowed Types:
|
||||
- `feat`: A new feature (e.g., new Execution Provider, CLI flag).
|
||||
- `fix`: A bug fix (e.g., resolver error, memory leak).
|
||||
- `docs`: Documentation updates.
|
||||
- `refactor`: Code restructuring without functional changes.
|
||||
- `perf`: A code change that improves performance.
|
||||
- `test`: Adding or correcting tests.
|
||||
- `chore`: Maintenance tasks (release metadata, config updates).
|
||||
|
||||
#### Example:
|
||||
```text
|
||||
feat(engine): add Ampere architecture guard for TensorRT-RTX
|
||||
|
||||
- Check compute capability >= 8.0 via pynvml
|
||||
- Fall back gracefully to CUDAExecutionProvider on Turing GPUs
|
||||
- Add unit test coverage in tests/test_tensorrt_rtx.py
|
||||
```
|
||||
|
||||
### Branch Naming
|
||||
- Features: `feat/<feature-name>`
|
||||
- Bugfixes: `fix/<bug-name>`
|
||||
- Documentation: `docs/<doc-name>`
|
||||
@@ -0,0 +1,129 @@
|
||||
# Installer Architecture & Development
|
||||
|
||||
This guide documents the architecture, lifecycle phases, and testing practices for the **RLD UVR-Opt** PowerShell installer pipeline located in [install/](../../install/).
|
||||
|
||||
---
|
||||
|
||||
## 1. Five-Phase Pipeline Architecture
|
||||
|
||||
The installer replaces monolithic batch files with a deterministic, modular pipeline implemented in **Windows PowerShell 5.1**:
|
||||
|
||||
```text
|
||||
[Input Flags / Prompts]
|
||||
│
|
||||
▼
|
||||
Phase 1: Detect (lib/Detect.ps1) - Read-only hardware & package inspection
|
||||
│
|
||||
▼
|
||||
Phase 2: Plan (lib/Plan.ps1) - Pure action mapping (Keep, Install, Replace, Remove)
|
||||
│
|
||||
▼
|
||||
Phase 3: Verify (lib/Verify.ps1) - Read-only gate; checks invariants & -DryRun
|
||||
│
|
||||
▼
|
||||
Phase 4: Install (lib/Install.ps1) - Safe mutation with snapshot rollback & uv execution
|
||||
│
|
||||
▼
|
||||
Phase 5: Validate (lib/Validate.ps1) - Live runtime validation with in-memory ONNX model
|
||||
│
|
||||
▼
|
||||
Post: Record (lib/Record.ps1) - Atomic persistence to install_record.json
|
||||
```
|
||||
|
||||
### Module Responsibilities:
|
||||
|
||||
| File | Phase | Responsibility |
|
||||
| :--- | :--- | :--- |
|
||||
| [**`Config.ps1`**](../../install/lib/Config.ps1) | Config | **Single Source of Truth**: Package pins (PyTorch, ONNX Runtime, plugins), URLs, driver threshold ($\ge 570$), CC threshold ($\ge 8.0$), and disk space ($5\text{ GB}$). |
|
||||
| [**`Common.ps1`**](../../install/lib/Common.ps1) | Common | Shared logging (`Write-InstallerLog`), deadlock-free native process runner (`Invoke-Native`), file I/O, and interactive prompts. |
|
||||
| [**`Detect.ps1`**](../../install/lib/Detect.ps1) | Phase 1 | Inspects OS bitness, GPU controllers via CIM, `nvidia-smi`, Python venv, and `.dist-info` package state (without importing torch). |
|
||||
| [**`Plan.ps1`**](../../install/lib/Plan.ps1) | Phase 2 | Pure function `New-InstallPlan`: maps current state to desired target profile (A, B, C) and marks destructive operations. |
|
||||
| [**`Verify.ps1`**](../../install/lib/Verify.ps1) | Phase 3 | Verifies plan invariants: architecture suitability, minimum driver version, disk space, and active process locks. Handles `-DryRun`. |
|
||||
| [**`Install.ps1`**](../../install/lib/Install.ps1) | Phase 4 | Captures rollback snapshot, cleans variant conflicts, installs wheels and filtered requirements via Astral `uv`. |
|
||||
| [**`Validate.ps1`**](../../install/lib/Validate.ps1) | Phase 5 | Deep runtime check inside venv: verifies GPU tensor allocation and creates live ONNX Runtime sessions using embedded IR 8 test models. |
|
||||
| [**`Record.ps1`**](../../install/lib/Record.ps1) | Record | Atomically writes [install_record.json](../../install/install_record.json) while delegating base schema to `install_record.py write`. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Running Non-Destructive Dry Runs
|
||||
|
||||
Contributors can simulate any installation or migration scenario on their local machine without touching the filesystem or installing packages:
|
||||
|
||||
```powershell
|
||||
# Simulate Option A (CUDA)
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\install\Install-UVR.ps1 -Mode A -DryRun
|
||||
|
||||
# Simulate Option B (TensorRT-RTX)
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\install\Install-UVR.ps1 -Mode B -DryRun
|
||||
|
||||
# Simulate Option C (CPU)
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\install\Install-UVR.ps1 -Mode C -DryRun
|
||||
```
|
||||
|
||||
A `-DryRun` prints the generated plan table and verification status, then exits with code 0 without performing changes.
|
||||
|
||||
---
|
||||
|
||||
## 3. Single Source of Truth Rule
|
||||
|
||||
All version pins, index URLs, and hardware thresholds **must reside exclusively** in [install/lib/Config.ps1](../../install/lib/Config.ps1). Never hardcode package versions or driver versions inside `Detect.ps1`, `Plan.ps1`, or `Install.ps1`.
|
||||
|
||||
### Example: Adding or Updating a Dependency Pin
|
||||
To bump PyTorch or ONNX Runtime, update the `Get-InstallerConfig` hashtable in [install/lib/Config.ps1](../../install/lib/Config.ps1):
|
||||
|
||||
```powershell
|
||||
Torch = [PSCustomObject]@{
|
||||
Version = '2.11.0'
|
||||
CudaIndexUrl = 'https://download.pytorch.org/whl/cu128'
|
||||
CpuIndexUrl = 'https://download.pytorch.org/whl/cpu'
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Writing Pester Tests for Installer Scenarios
|
||||
|
||||
Installer logic is thoroughly covered by Pester test fixtures located in [install/tests/](../../install/tests/).
|
||||
|
||||
### Adding a New Planning Scenario
|
||||
To test plan generation for a new package state, add an `It` block to [install/tests/Plan.Tests.ps1](../../install/tests/Plan.Tests.ps1) using `New-TestFacts`:
|
||||
|
||||
```powershell
|
||||
It 'Scenario: Venv with custom package condition' {
|
||||
$facts = New-TestFacts -TorchVariant 'CUDA' -TorchVersion '2.11.0' -HasOrtGpu $true
|
||||
$plan = New-InstallPlan -Facts $facts -Choice 'B'
|
||||
|
||||
$plan.Profile | Should Be 'B'
|
||||
$plan.Actions | Where-Object { $_.Component -eq 'TRT-RTX-Plugin' } | Should Not BeNullOrEmpty
|
||||
}
|
||||
```
|
||||
|
||||
### Adding a New Verification Gate
|
||||
To test a hardware gate or safety block, add an `It` block to [install/tests/Verify.Tests.ps1](../../install/tests/Verify.Tests.ps1):
|
||||
|
||||
```powershell
|
||||
It 'Scenario: System with insufficient memory' {
|
||||
$facts = New-TestFacts -FreeDiskSpaceBytes (2GB) # Less than 5GB required
|
||||
$plan = New-InstallPlan -Facts $facts -Choice 'A'
|
||||
$verdict = Test-InstallPlan -Facts $facts -Choice 'A' -Plan $plan
|
||||
|
||||
$verdict.Status | Should Be 'Blocked'
|
||||
}
|
||||
```
|
||||
|
||||
Execute the test suite to verify:
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -Command "Invoke-Pester -Path ./install/tests"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. The `install_record.py` Contract
|
||||
|
||||
To maintain full backward compatibility with legacy tooling and the GUI:
|
||||
1. `install_record.py inspect-stack`:
|
||||
- Prints `TORCH_VARIANT|TORCH_VERSION|ORT_VARIANT|ORT_VERSION` used by external inspectors.
|
||||
2. `install_record.py write`:
|
||||
- Writes the base schema fields (`schema_version`, `timestamp_utc`, `selected_option`, `install_mode`).
|
||||
3. **Additive Extensions**:
|
||||
- [install/lib/Record.ps1](../../install/lib/Record.ps1) injects additional telemetry fields (`effective_option`, `detected`, `validation`) atomically without breaking base consumers.
|
||||
@@ -0,0 +1,164 @@
|
||||
# Environment Setup & Installation Guide
|
||||
|
||||
This guide describes how to configure a reproducible development environment for **RLD UVR-Opt**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
| Requirement | Specification | Notes |
|
||||
| :--- | :--- | :--- |
|
||||
| **Operating System** | Windows 10 / 11 64-bit (x64 / AMD64) | Primary target platform for installer, CUDA, and TensorRT-RTX. *(Linux/macOS contributors can test CPU-only headless CLI).* |
|
||||
| **Python** | Python 3.11.x (64-bit) | Python 3.11.9 recommended. 32-bit and ARM64 architectures are unsupported. |
|
||||
| **Shell** | Windows PowerShell 5.1 | Built into Windows 10/11. PowerShell 7 is supported but not required. |
|
||||
| **Git** | Git 2.30+ | Required for version control and submodule handling. |
|
||||
| **Privileges** | Standard User | Administrator privileges are only required if Visual C++ Redistributable needs system-level installation. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Cloning the Repository
|
||||
|
||||
Clone your fork of the repository into a directory path without non-ASCII characters:
|
||||
|
||||
```powershell
|
||||
git clone https://github.com/<your-username>/RLD-UVR-Opt.git
|
||||
cd "RLD-UVR-Opt"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Automated Setup via PowerShell Installer
|
||||
|
||||
The repository includes a modern 5-phase installer pipeline ([install/Install-UVR.ps1](../../install/Install-UVR.ps1)). Run the script selecting the mode matching your hardware:
|
||||
|
||||
### Option A: NVIDIA GPU (CUDA Execution Provider)
|
||||
Recommended for all NVIDIA GPUs (RTX 20-series / Turing or newer, including GTX 16-series):
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\install\Install-UVR.ps1 -Mode A
|
||||
```
|
||||
- **Installs**: PyTorch (CUDA wheel), `onnxruntime-gpu`, and dependencies as specified in [install/reqs/requirements.txt](../../install/reqs/requirements.txt) and defined in [install/lib/Config.ps1](../../install/lib/Config.ps1).
|
||||
|
||||
### Option B: NVIDIA GPU (TensorRT-RTX Execution Provider, Experimental)
|
||||
Designed for NVIDIA RTX 30-series (Ampere), RTX 40-series (Ada Lovelace), and newer (Compute Capability $\ge$ 8.0):
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\install\Install-UVR.ps1 -Mode B
|
||||
```
|
||||
- **Installs**: PyTorch (CUDA wheel), `onnxruntime-gpu`, TensorRT-RTX execution provider plugin, and dependencies as specified in [install/reqs/requirements_trt_rtx.txt](../../install/reqs/requirements_trt_rtx.txt) and defined in [install/lib/Config.ps1](../../install/lib/Config.ps1). CUDA EP remains active as an automatic fallback.
|
||||
|
||||
### Option C: CPU Only (Fallback)
|
||||
For non-NVIDIA GPUs (AMD, Intel) or systems without a dedicated GPU:
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\install\Install-UVR.ps1 -Mode C
|
||||
```
|
||||
- **Installs**: PyTorch (CPU wheel), `onnxruntime` (CPU), and dependencies as specified in [install/reqs/requirements_cpu.txt](../../install/reqs/requirements_cpu.txt) and defined in [install/lib/Config.ps1](../../install/lib/Config.ps1).
|
||||
|
||||
### Installer Pipeline Behavior
|
||||
When running, the installer executes the following deterministic phases:
|
||||
1. **Detect**: Inspects hardware via CIM controllers and `nvidia-smi`, validates Python 64-bit, and inspects `.dist-info` packages.
|
||||
2. **Plan**: Formulates an action plan (`Keep`, `Install`, `Replace`, `Remove`) without modifying files.
|
||||
3. **Verify**: Read-only verification gate checking driver version ($\ge 570$), disk space ($\ge 5\text{ GB}$), network connectivity, and running process locks.
|
||||
4. **Install**: Creates or reuses `.venv`, records rollback snapshots, removes variant conflicts, and installs packages via Astral `uv`.
|
||||
5. **Validate**: Submits an in-memory test ONNX model to the active Execution Provider, validates tensor allocation, and writes [install/install_record.json](../../install/install_record.json).
|
||||
|
||||
---
|
||||
|
||||
## 4. Manual Setup (Alternative with Astral uv)
|
||||
|
||||
If you prefer configuring your virtual environment manually without running the orchestrator:
|
||||
|
||||
```powershell
|
||||
# 1. Install or update Astral uv
|
||||
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
|
||||
|
||||
# 2. Create the virtual environment
|
||||
uv venv --python 3.11 venv
|
||||
.\venv\Scripts\activate.ps1
|
||||
|
||||
# 3. Install PyTorch stack using definitions from install/lib/Config.ps1
|
||||
# Query configuration from the single source of truth:
|
||||
$cfg = & powershell -NoProfile -Command ". .\install\lib\Config.ps1; Get-InstallerConfig"
|
||||
|
||||
# For GPU (Options A and B):
|
||||
uv pip install $cfg.Torch.Packages --index-url $cfg.Torch.Cu128IndexUrl
|
||||
|
||||
# For CPU (Option C):
|
||||
uv pip install $cfg.Torch.Packages --index-url $cfg.Torch.CpuIndexUrl
|
||||
|
||||
# 4. Pre-install local binary wheels (provided in install/wheels/)
|
||||
uv pip install (Get-ChildItem .\install\wheels\*.whl).FullName --no-deps
|
||||
|
||||
# 5. Install requirements based on target mode
|
||||
# Option A (CUDA):
|
||||
uv pip install -r .\install\reqs\requirements.txt
|
||||
# Option B (TensorRT-RTX):
|
||||
uv pip install -r .\install\reqs\requirements_trt_rtx.txt
|
||||
# Option C (CPU):
|
||||
uv pip install -r .\install\reqs\requirements_cpu.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. External Media Binaries
|
||||
|
||||
The application requires `ffmpeg.exe`, `rubberband.exe`, and `sndfile.dll`.
|
||||
|
||||
- **FFmpeg**: Place `ffmpeg.exe` in the application root directory or ensure it is accessible in your system `PATH`.
|
||||
- **Rubber Band & Libsndfile**: Ensure `rubberband.exe` and `sndfile.dll` are located in the repository root (both are provided in the repo root).
|
||||
|
||||
Verify external binary availability:
|
||||
```powershell
|
||||
.\ffmpeg.exe -version
|
||||
.\rubberband.exe --version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Verifying the Setup
|
||||
|
||||
Run these verification steps before writing code:
|
||||
|
||||
1. **Run Unit Tests**:
|
||||
```powershell
|
||||
.\venv\Scripts\python.exe -m unittest discover tests -v
|
||||
```
|
||||
*Expected result: All tests passing (`OK`).*
|
||||
|
||||
2. **Run Headless Smoke Test**:
|
||||
```powershell
|
||||
# Test inference with synthetic sine wave audio
|
||||
.\venv\Scripts\python.exe -m uvr_cli smoke-test --device cpu
|
||||
# Or --device cuda:0 / --device trt_rtx:0 if GPU hardware is present
|
||||
```
|
||||
*Expected result: `[OK] Smoke test passed.`*
|
||||
|
||||
3. **Launch the Graphical Interface**:
|
||||
```powershell
|
||||
.\run_uvr.bat
|
||||
```
|
||||
*Expected result: UVR GUI window opens.*
|
||||
|
||||
---
|
||||
|
||||
## 7. Troubleshooting Real Failures
|
||||
|
||||
### 1. PowerShell Script Execution Disabled
|
||||
- **Symptom**: `File Install-UVR.ps1 cannot be loaded because running scripts is disabled on this system.`
|
||||
- **Fix**: Launch PowerShell and set execution policy for the process:
|
||||
```powershell
|
||||
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
|
||||
```
|
||||
|
||||
### 2. Missing Visual C++ Runtime (VCRedist)
|
||||
- **Symptom**: `ImportError: DLL load failed while importing onnxruntime_pybind11_state` or `sndfile.dll`.
|
||||
- **Fix**: Install the latest official Microsoft Visual C++ 2015-2022 Redistributable (x64) from [aka.ms/vs/17/release/vc_redist.x64.exe](https://aka.ms/vs/17/release/vc_redist.x64.exe).
|
||||
|
||||
### 3. File Lock During Installation / Package Update
|
||||
- **Symptom**: `PermissionError: [WinError 5] Access is denied: ... python.exe` or `DLL in use`.
|
||||
- **Fix**: Close any open instances of Ultimate Vocal Remover GUI or background `python.exe` processes holding files in `venv/`:
|
||||
```powershell
|
||||
Get-Process | Where-Object { $_.Path -like "*\venv\*" } | Stop-Process -Force
|
||||
```
|
||||
|
||||
### 4. GPU Architecture Mismatch on Option B
|
||||
- **Symptom**: Installer reports `Adjusted` from Option B to Option A.
|
||||
- **Explanation**: TensorRT-RTX Execution Provider requires Compute Capability $\ge 8.0$ (Ampere / RTX 30-series or newer). Turing GPUs (RTX 20-series, GTX 16-series, CC 7.5) are automatically redirected to Option A (CUDA EP).
|
||||
@@ -0,0 +1,108 @@
|
||||
# Testing Guide
|
||||
|
||||
This guide details how to execute and write automated tests for **RLD UVR-Opt**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Test Suite Overview
|
||||
|
||||
The repository maintains automated test suites covering Python inference/CLI components and PowerShell installer pipelines:
|
||||
|
||||
| Suite | Runner | Location | Purpose |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Python Regression Tests** | `unittest` | [tests/](../../tests/) | Tests CLI argument parsing, model resolution, in-memory cache eviction, TRT guards, and batch file CRLF line endings. |
|
||||
| **Installer Tests** | `Pester` | [install/tests/](../../install/tests/) | Tests 15 installer scenarios covering plan generation, hardware gates, driver checks, and atomic state recording. |
|
||||
| **Headless Smoke Test** | `uvr_cli` | [uvr_cli/](../../uvr_cli/) | Performs end-to-end inference verification on procedurally generated synthetic audio. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Running Python Unit Tests
|
||||
|
||||
### Run Full Test Suite
|
||||
To run all 41 repository unit tests:
|
||||
```powershell
|
||||
.\venv\Scripts\python.exe -m unittest discover tests -v
|
||||
```
|
||||
|
||||
### Run Targeted Test Modules
|
||||
You can execute individual test modules during focused development:
|
||||
|
||||
```powershell
|
||||
# Test CLI commands and flag parsing
|
||||
.\venv\Scripts\python.exe -m unittest tests.test_uvr_cli -v
|
||||
|
||||
# Test in-memory resident model cache and FP16 lifecycle
|
||||
.\venv\Scripts\python.exe -m unittest tests.test_in_memory_model_cache -v
|
||||
|
||||
# Test TensorRT-RTX provider guards and Ampere check
|
||||
.\venv\Scripts\python.exe -m unittest tests.test_tensorrt_rtx -v
|
||||
|
||||
# Test batch file line endings (verifies CRLF formatting)
|
||||
.\venv\Scripts\python.exe -m unittest tests.test_windows_batch_files -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Running PowerShell Installer Tests (Pester)
|
||||
|
||||
The installer logic is verified through [Pester 3.4.0+](https://github.com/pester/Pester) (bundled with Windows 10/11):
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -Command "Invoke-Pester -Path ./install/tests"
|
||||
```
|
||||
|
||||
### What the Pester Suite Covers:
|
||||
- **Phase 2 (Plan Generation)**:
|
||||
- Migrating CPU venv to GPU (Option A) replaces PyTorch CPU with CUDA wheels.
|
||||
- Upgrading Option A to Option B keeps PyTorch and installs TRT-RTX plugin.
|
||||
- Resolving conflicting `onnxruntime` vs `onnxruntime-gpu` packages.
|
||||
- Rerunning identical options produces zero destructive changes (`Keep`).
|
||||
- **Phase 3 (Verification & Gate)**:
|
||||
- No GPU or AMD GPU detected → adjusts selection to Option C.
|
||||
- NVIDIA Turing (CC 7.5) selecting Option B → adjusts to Option A.
|
||||
- NVIDIA driver version too old ($< 570$) → blocks installation.
|
||||
- Insufficient disk space ($< 5\text{ GB}$) or active file locks → blocks installation.
|
||||
- **Phase 5 (Atomic Record)**:
|
||||
- Verifies atomic writing of `install_record.json` without file locking or syntax errors.
|
||||
|
||||
---
|
||||
|
||||
## 4. Running the Headless Smoke Test
|
||||
|
||||
The smoke test verifies end-to-end model inference without requiring existing music files. It generates a short 440 Hz synthetic stereo sine wave in memory and separates it:
|
||||
|
||||
```powershell
|
||||
# Run smoke test on CPU
|
||||
.\venv\Scripts\python.exe -m uvr_cli smoke-test --device cpu
|
||||
|
||||
# Run smoke test on GPU (CUDA)
|
||||
.\venv\Scripts\python.exe -m uvr_cli smoke-test --device cuda:0
|
||||
|
||||
# Run smoke test on TensorRT-RTX
|
||||
.\venv\Scripts\python.exe -m uvr_cli smoke-test --device trt_rtx:0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Testing Without Physical GPUs (Headless & Mocking)
|
||||
|
||||
Many contributors or automated CI environments do not possess physical NVIDIA RTX GPUs:
|
||||
|
||||
- **Mocking GPU Hardware**:
|
||||
In [tests/test_tensorrt_rtx.py](../../tests/test_tensorrt_rtx.py), GPU architecture checks and ONNX Runtime provider sessions are mocked via `unittest.mock.patch`:
|
||||
```python
|
||||
with patch("torch.cuda.is_available", return_value=True), \
|
||||
patch("torch.cuda.get_device_capability", return_value=(7, 5)):
|
||||
# Simulates Turing GPU (CC 7.5) to test fallback
|
||||
providers = get_supported_execution_providers(device_id=0)
|
||||
self.assertNotIn("NvTensorRTRTXExecutionProvider", providers)
|
||||
```
|
||||
- **PRs Touching GPU Paths**:
|
||||
While mocked tests allow validating control flow without a GPU, any PR that modifies raw CUDA kernels, TensorRT engine options, or memory layout must include verified test logs or benchmark telemetry from a physical GPU machine.
|
||||
|
||||
---
|
||||
|
||||
## 6. Prohibited Test Artifacts
|
||||
|
||||
- **No Model Weights in Git**: Never commit binary model weights (`*.onnx`, `*.pth`, `*.ckpt`, `*.safetensors`). Use synthetic models (e.g. tiny base64 ONNX graphs) for unit testing.
|
||||
- **No Copyrighted Audio**: Never commit commercial or copyrighted music files to `input/` or `tests/`. Unit tests should synthesize audio arrays in memory (`np.sin(...)`).
|
||||
Reference in new issue
Block a user