Files
stemengine_old/docs/API_REFERENCE.md
T
2026-03-26 15:14:41 +09:00

16 KiB
Raw Blame History

StemEngine API Reference (Phase 2)

This is a practical API map for contributors. It focuses on endpoint groups, request shape, and common response patterns.

Base conventions

  • Base URL (default): http://localhost:8000
  • Most long-running operations return a job_id and run asynchronously.
  • Poll status via GET /api/jobs/{job_id}.
  • Download job outputs via GET /api/jobs/{job_id}/download/{filename} when status is done.

Common async pattern

  1. Start operation (for example separation/transcription/alignment).
  2. Receive { "job_id": "..." }.
  3. Poll job until status is done, error, or cancelled.
  4. Read files and result from job payload.

System and operations

GET /api/system/health

Lightweight liveness probe.

GET /api/system/readiness

Readiness checks for DB and writable directories.

GET /api/system/executors

Queue and worker state from the job manager.

GET /api/system/gpu-info

GPU availability + VRAM figures + transcription CUDA health flag.

POST /api/system/gpu-reset

Admin-protected reset path for GPU recovery and cache flush.


Jobs

GET /api/jobs

List jobs.

GET /api/jobs/{job_id}

Get one job payload.

DELETE /api/jobs/{job_id}

Request cancellation.

GET /api/jobs/{job_id}/download/{filename}

Download one output file from a completed job.


Separation

GET /api/separate/models

Returns available separation models.

GET /api/separate/formats

Returns output formats.

GET /api/separate/presets

Returns preset types and static options.

GET /api/separate/per-instrument-models

Returns models usable for per-stem reprocessing.

POST /api/separate

Multipart upload separation request.

Form fields:

  • file (required)
  • model
  • output_format
  • preset_type
  • static_option
  • dynamic_levels (JSON string)
  • project_name
  • instrument_models (JSON string)
  • sep_power_preset

Optional runtime overrides (Phase 4):

  • chunking_mode_override (auto | always | off)
  • chunk_duration_secs_override (int, >= 1)
  • chunk_overlap_secs_override (int, >= 0)
  • roformer_batch_size_override (int, >= 1)
  • roformer_num_overlap_override (int, >= 1)
  • demucs_segment_override (int, >= 1)
  • demucs_overlap_override (float, 0.0–0.95)
  • scnet_chunk_size_override (int, >= 1)
  • scnet_num_overlap_override (int, >= 1)

Global power-policy overrides (Phase 2):

  • global_power_tier_override (eco | standard | performance)
  • global_segment_secs_override (float, 0.1–3600.0)
  • global_overlap_override (float, 0.0–16.0)
  • global_batch_size_override (int, >= 1)

Example response:

{
  "job_id": "f52d...",
  "preset_type": "stems_only",
  "project_name": "Artist - Title"
}

POST /api/separate/existing

Re-run separation on an existing project source.

Supports the same optional runtime overrides as POST /api/separate.

POST /api/separate/from-path

Run separation from a server-side file path (restricted to imports directory).

Supports the same optional runtime overrides as POST /api/separate.

Runtime diagnostics in completed job result

For separation jobs, result now includes:

  • diagnostics
    • track_duration_secs
    • family
    • chunking (supported, mode, threshold_secs, active, chunk_duration_secs, chunk_overlap_secs)
    • demucs_tuning (segment_override, overlap_override, effective_segment_secs)
    • effective_power_policy (tier, segment_secs, overlap, batch_size, source fields, adapter_targets)
  • request_overrides
    • echo of the explicitly provided request-level override fields

UI advanced controls mapping

Recommended advanced UI panel fields map directly to the override form fields:

  • "Chunking mode" → chunking_mode_override
  • "Chunk length (s)" → chunk_duration_secs_override
  • "Chunk overlap (s)" → chunk_overlap_secs_override
  • "RoFormer batch size" → roformer_batch_size_override
  • "RoFormer overlaps" → roformer_num_overlap_override
  • "Demucs segment" → demucs_segment_override
  • "Demucs overlap" → demucs_overlap_override
  • "SCNet chunk size" → scnet_chunk_size_override
  • "SCNet overlaps" → scnet_num_overlap_override
  • "Global power tier" → global_power_tier_override
  • "Global segment seconds" → global_segment_secs_override
  • "Global overlap" → global_overlap_override
  • "Global batch size" → global_batch_size_override

WebUI status (implemented)

The Separator page now exposes a Global power overrides panel and forwards these fields on both upload and imported-file separation paths.

Auto Pipeline separation submission also forwards the same stored global override values when present.

Implementation safeguards (post-review)

  • Apollo/Aspiration/Bandit chunk-adapter path now forces WAV intermediate chunk outputs before overlap-add reconstruction.
    • This avoids decoder variance when reading per-chunk stems and keeps reconstruction lossless before final output encoding.
  • Auto Pipeline forwarding now applies backend-aligned range checks before sending stored global override values.
  • Operational caveat: file-based chunk adapters currently decode the full source track once in host RAM before chunk slicing. GPU peak remains bounded by chunk policy, but host memory usage scales with input duration.

Additional edge-case hardening (double-check pass)

  • File-chunk adapter input/stem reads now use a robust decode path:
    • primary read via soundfile
    • automatic fallback decode via ffmpeg to temporary WAV when needed
  • Chunk stem sample-rate mismatches are now resampled to the input sample rate before overlap-add reconstruction (instead of failing the whole job).

Observed runtime example (real test)

Tested endpoint: POST /api/separate/from-path

Request profile:

  • model: mel_band_karaoke
  • output format: flac
  • preset: stems_only
  • source: one imported Subsonic file

Observed runtime behavior:

  • Job lifecycle:
    1. Reading metadata...
    2. Copying source file to project...
    3. Separating lead/backing vocals...
    4. Separation complete.
  • Completed successfully (status=done) with FLAC outputs.
  • Produced stems:
    • lead (.flac)
    • backing_instrumental (.flac)
  • Diagnostics showed:
    • family = roformer
    • chunking mode auto, but active=false for this track duration (duration 307.733s < threshold 480s)
    • effective policy tier standard with no explicit request overrides.

Under-the-hood implications for this run:

  • Because chunking was not activated, the run used normal MelBand inference path (not long-track chunked path).
  • FLAC output was preserved end-to-end in final stem export.

Low-heat validation (A/B rerun after MelBand hop-length fix)

Purpose:

  • verify that MelBand global segment mapping now respects each model's configured hop_length (from YAML) instead of assuming 1024.

Test profile (same source/model/format for both runs):

  • endpoint: POST /api/separate/from-path
  • model: mel_band_karaoke
  • output: flac
  • source: imported Our Lady Peace ... Car Crash.flac (307.733s)
  • low-heat controls:
    • global_power_tier_override=eco
    • global_batch_size_override=1
    • global_overlap_override=2
    • chunking_mode_override=always
    • chunk_duration_secs_override=120
    • chunk_overlap_secs_override=10

Run A (target dim_t≈256 for Karaoke hop_length=441):

  • request override: global_segment_secs_override=2.55
  • job id: 2524851b-b8f2-4f37-acf4-322afc36ea4b
  • status: done
  • elapsed (job timestamps): 176.084s

Run B (larger segment):

  • request override: global_segment_secs_override=5.0
  • job id: bcb6fe54-abf3-4c74-8f34-14dcb4810f6e
  • status: done
  • elapsed (job timestamps): 171.732s

Observed comparison:

  • both runs completed with expected FLAC outputs (lead, backing_instrumental).
  • larger segment (5.0s) was slightly faster in this sample (~2.5%).
  • backend log summaries showed fewer demix windows for the larger segment, consistent with expected dim_t behavior.

Interpretation:

  • the mapping bug is fixed (segment sizing now follows model YAML hop length).
  • lowering segment size alone does not guarantee lower thermals; it can increase window count and scheduling overhead.
  • for practical low-heat behavior, keep the full bundle (eco tier + batch 1 + lower overlap + chunking), then tune segment size empirically per model/GPU.

Global power-policy extension (design + implementation status)

Status: Phase 2 implemented (global override wiring) + Phase 3 adapter coverage expanded.

Goal: provide one cross-family override layer that includes segment size for all model families, while preserving current family-specific controls.

Implemented optional fields:

  • global_power_tier_override (eco | standard | performance)
  • global_segment_secs_override (float/int, > 0)
  • global_overlap_override (normalized overlap policy; adapted per family)
  • global_batch_size_override (int, >= 1)

Implemented behavior:

  • Demucs: map global segment directly to segment.
  • MelBand family (karaoke/vocals/dereverb/deux + MelBand 4-stem): map global segment seconds to MelBand dim_t using each model's configured stft_hop_length from its YAML.
  • MDX-Net: map global segment to chunk duration policy used by chunked path.
  • MedleyVox: map global segment to internal streaming chunk seconds.
  • Apollo / Aspiration / Bandit: map global segment to a file-based chunk adapter (time-domain chunking + overlap-add reconstruction) in service runtime.
  • SCNet / sample-based families: convert global segment to sample-based chunk fields.
  • SepACap / SepReformer: map global segment to chunk_samples.
  • Long-track chunking duration fallback: when chunk_duration_secs_override is not provided, global_segment_secs_override is used as the chunk-duration hint.
  • MELBAND_DIM_T_OVERRIDE remains available as a static fallback when no request/policy-derived MelBand segment mapping is provided.

Backward compatibility:

  • Existing family-specific override fields remain valid and take precedence.
  • Global fields are applied first, then family-specific fields refine/override.

MIDI transcription

GET /api/transcribe/modes

Returns available modes and availability flags.

POST /api/transcribe

Multipart upload request:

  • file
  • mode

POST /api/transcribe/from-project

Form request:

  • project_name
  • mode

Selects the best non-drum stem by preference order.


Lyric transcription (ASR)

GET /api/lyric-transcribe/models

Returns engines/models/defaults and runtime availability.

GET /api/lyric-transcribe/languages

Returns supported language choices.

GET /api/lyric-transcribe/formats

Returns output formats (lrc, elrc, json, txt, srt, vtt).

POST /api/lyric-transcribe

Multipart upload request.

Form fields:

  • file
  • engine (whisperx | parakeet)
  • model
  • language
  • task (transcribe | translate, with engine constraints)
  • output_formats (comma-separated)
  • initial_prompt
  • temperature

POST /api/lyric-transcribe/from-project

Form fields include:

  • project_name
  • stem_name (optional; vocals auto-detection if absent)
  • transcription options above

POST /api/lyric-transcribe/with-diarization

Multipart upload + diarization options.

POST /api/lyric-transcribe/project/{project_id}

JSON body version for project-stem lyric transcription.

Example JSON body:

{
  "stem_name": "vocals",
  "engine": "whisperx",
  "model": "large-v2",
  "task": "transcribe",
  "output_formats": ["elrc", "json", "txt"],
  "temperature": 0.0,
  "condition_on_previous_text": false,
  "word_timestamps": true
}

LRC / ELRC alignment

GET /api/lrc/sources

Returns source modes: auto, tag, file, text.

POST /api/lrc/extract

Extracts lyrics from uploaded audio metadata/source.

POST /api/lrc/align

Multipart request:

  • audio
  • lyrics_source
  • lyrics_text (required when source=text)
  • output_filename
  • style_options (JSON string)

POST /api/lrc/align-with-file

Multipart request with audio + lyrics_file.

POST /api/lrc/align-existing

Align using existing project vocals stem.


Projects

GET /api/projects

List projects.

GET /api/projects/{project_id}

Get project detail with linked jobs.

GET /api/projects/{project_id}/cover-art

Returns embedded art bytes if available.

DELETE /api/projects/{project_id}

Delete project and related assets.

POST /api/projects/{project_id}/transcribe

JSON body:

{
  "stem_name": "other",
  "mode": "polyphonic"
}

POST /api/projects/{project_id}/abc

JSON body:

{
  "stem_name": "other",
  "song_title": "My Song"
}

POST /api/projects/{project_id}/mix

JSON body:

{
  "preset_type": "dynamic",
  "dynamic_levels": {"vocals": 0.8, "drums": 1.1}
}

POST /api/projects/{project_id}/lrc

JSON body supports lyrics source, style, timing precision, and power preset.

POST /api/projects/{project_id}/lyric-transcribe

JSON body supports engine, model, alignment options, diarization, and existing lyrics alignment paths.

POST /api/projects/{project_id}/reprocess

Reprocess a single stem with a selected model.

POST /api/projects/{project_id}/rename-stem

Rename tracked stem file.

POST /api/projects/{project_id}/rescan-stems

Rescan stem directory and register untracked files.

Maintenance (admin-protected)

  • GET /api/projects/cleanup/verify
  • POST /api/projects/cleanup/write-metadata
  • POST /api/projects/cleanup/rename-stems
  • POST /api/projects/cleanup/purge-orphaned
  • POST /api/projects/cleanup/clean-lrc-files

Pipeline (end-to-end)

POST /api/pipeline

Multipart request combining separation + transcription + ABC path.

Form fields:

  • file
  • sep_model
  • sep_format
  • transcribe_mode
  • song_title
  • instrument_models (JSON string)
  • lyric_engine
  • lyric_model

Response:

{ "job_id": "..." }

ABC conversion

POST /api/abc-convert

Upload one or more MIDI files and convert to ABC.

POST /api/abc-convert/from-job

Reuse MIDI files produced by an existing completed job.


Model download management

GET /api/models/list

Returns model catalog and download status.

POST /api/models/download

Admin-protected start/resume download.

Request:

{ "model_id": "bs_roformer_4stem" }

POST /api/models/pause

POST /api/models/resume

POST /api/models/cancel

Admin-protected download control.

GET /api/models/progress/{model_id}

Get one model progress.

GET /api/models/progress

Get all active progress entries.

DELETE /api/models/{model_id}

Admin-protected model deletion.


Settings

GET /api/settings

Returns persisted settings with sensitive values redacted (*_set flags).

PATCH /api/settings

Partial update. Common keys include:

  • hf_token
  • whisper_model
  • sep_power
  • whisper_word_ts
  • whisper_condition_prev
  • whisper_phoneme_align
  • auto_steps
  • auto_model
  • auto_format
  • auto_presets

Example patch:

{
  "whisper_model": "large-v2",
  "sep_power": "standard",
  "whisper_word_ts": true
}

DELETE /api/settings/hf-token

Delete stored token; env-var fallback can still apply.


Subsonic integration

GET /api/subsonic/config

Returns configured state (no password echoed).

POST /api/subsonic/config

Validates and stores connection config.

DELETE /api/subsonic/config

Removes stored config.

GET /api/subsonic/search

Query params support text + title/artist/album filtering.

POST /api/subsonic/import

Imports one Subsonic song into local imports.

POST /api/subsonic/import-ytdl

Enqueues yt-dlp import job.

GET /api/subsonic/imports

Lists local imports.

DELETE /api/subsonic/imports/{filename}

Deletes one imported file.


Errors and status codes (common)

  • 400 validation/unsupported option
  • 403 security/path constraints
  • 404 missing job/project/file/model
  • 503 optional runtime unavailable (for example Omnizart/Parakeet/WhisperX)

For production-grade clients, treat all non-2xx as recoverable with user-readable error surfacing and retry logic where appropriate.