Copilot Phase 4 implementation.

This commit is contained in:
translator committed 2026-03-13 17:55:07 +09:00
1 parent de77114724
commit 07ca3d7d43
28 files changed
+2022 -1224

No files matched your search

+206
View File
@@ -1245,3 +1245,209 @@ The `/api/projects` clean-DB test, the `HF_TOKEN` env-fallback settings test,
and the `/api/separate` validation coverage are now complete.
At this point, the originally identified Phase 1A smoke-test gap is closed.
---
## 2026-03-13 progress update (service split continuation)
Continued the backend modularization pass and validated syntax integrity.
### Changes completed
- split pipeline stages into dedicated module:
- `backend/services/pipeline_stages.py`
- `backend/services/pipeline_service.py` now orchestrates by stage helpers
- split project service by concern and replaced monolith with compatibility facade:
- `backend/services/project_service.py`
- `backend/services/project_service_common.py`
- `backend/services/project_assets_service.py`
- `backend/services/project_cleanup_service.py`
- `backend/services/project_reprocess_service.py`
- created separation helper package and moved shared registry/layout/download/preset logic:
- `backend/services/separation/__init__.py`
- `backend/services/separation/model_registry.py`
- `backend/services/separation/model_downloads.py`
- `backend/services/separation/output_layout.py`
- `backend/services/separation/power_presets.py`
- updated `backend/services/separator_service.py` to consume shared separation helpers while retaining compatibility behavior
### Validation run
- compiled backend service and router modules successfully:
- `python3 -m compileall backend/services backend/routers`
No compile errors remained after repair/cleanup.
---
## 2026-03-13 Phase 4 runtime tests (image validation)
Executed runtime tests against the freshly built `stemengine:latest` image in a temporary container with `ADMIN_TOKEN` enabled.
### Test container launch
```bash
docker run -d --name se_phase4_test -p 18000:8000 \
-e MEDIA_DIR=/tmp/media \
-e UPLOAD_DIR=/tmp/media/uploads \
-e JOBS_DIR=/tmp/media/jobs \
-e PROJECTS_DIR=/tmp/media/projects \
-e DB_PATH=/tmp/media/db/stemengine.db \
-e LRC_SYNC_DIR=/tmp/media/lrc_sync \
-e ADMIN_TOKEN=test-admin-token \
stemengine:latest \
bash -lc 'python3.11 -m uvicorn backend.main:app --host 0.0.0.0 --port 8000'
```
### Runtime endpoint results
- `GET /api/system/health` → `200`
- `{"status":"ok","service":"stemengine","environment":"development"}`
- `GET /api/system/readiness` → `200`
- readiness checks passed for DB + writable media directories
- `GET /api/system/executors` → `200`
- executor queue/worker state returned (`general`, `gpu`, `transcription`)
### Admin-token enforcement checks (high-power endpoints)
- `POST /api/system/gpu-reset`
- without token → `403`
- with `X-Admin-Token: test-admin-token` → `200`
- `GET /api/projects/cleanup/verify`
- without token → `403`
- with token → `200`
- `POST /api/models/download`
- without token → `403`
- with token → `200`
### Phase 4 status
Phase 4 runtime hardening changes are verified in-container for:
- liveness/readiness
- executor visibility
- admin-token protection of high-power operational routes
---
## 2026-03-13 Phase 5 implementation (data model cleanup)
Implemented a first normalized data-model step for project stem/job linkage while keeping API behavior backward compatible.
### What changed
#### 1) Added normalized project stem-job link table
- file updated: `backend/db.py`
- new table/model: `project_stem_job_links` (`ProjectStemJobLink`)
- columns: `project_id`, `stage`, `stem_name`, `job_id`, timestamps
- indexes on `project_id`, `stage`, `job_id`
- unique key on `(project_id, stage, stem_name)`
#### 2) Added safe migration + backfill
- file updated: `backend/db.py`
- `init_db()` now ensures table/index creation via idempotent SQL
- added `_backfill_project_stem_job_links()`
- migrates existing legacy JSON maps (`trans_jobs`, `abc_jobs`, `lrc_jobs`) into normalized rows
- uses insert-or-ignore semantics so reruns are safe
#### 3) Repository writes now maintain normalized rows
- file updated: `backend/repositories/projects.py`
- `set_transcription_job()`, `set_abc_job()`, `set_lrc_job()` continue writing legacy JSON for compatibility
- same methods now also upsert into `project_stem_job_links`
- project deletion now removes normalized link rows for that project
#### 4) Repository reads now prefer normalized data
- file updated: `backend/repositories/projects.py`
- `get()`, `get_by_sep_job()`, `list_all()`, `list_persisted()`, `find_or_create()` now compose project dicts via normalized link rows first
- fallback to JSON columns remains in place when normalized rows are absent
### Tests added
- new file: `backend/tests/test_phase5_normalization.py`
- verifies `set_transcription_job()` writes normalized link row
- verifies project reads prefer normalized link rows over stale JSON
### Validation
- compile check:
- `python3 -m compileall backend` ✅
- in-image unit tests (with backend mount):
- `python3.11 -m unittest backend.tests.test_phase5_normalization -v` ✅
- result: `Ran 2 tests ... OK`
---
## 2026-03-13 progress update (Phase 4 implemented)
Implemented the Phase 4 operational-hardening slice directly in backend runtime/router code.
### Implemented items
1. startup/runtime operational safety
- added env-driven operational knobs in `backend/config.py`:
- `APP_ENV`
- `CORS_ORIGINS`
- `ADMIN_TOKEN`
2. tightened CORS in non-dev environments
- updated `backend/main.py` CORS wiring:
- explicit `CORS_ORIGINS` is honored when set
- wildcard CORS remains only for dev/local/test-style environments
- non-dev with no configured origins defaults to no allowed origins
3. startup health/readiness checks
- extended `backend/routers/system.py` with:
- `GET /api/system/health` (liveness)
- `GET /api/system/readiness` (DB probe + writable-dir checks)
4. queue-depth / executor-state visibility
- added `jm.get_executor_state()` in `backend/job_manager.py`
- exposed via `GET /api/system/executors`
5. structured job log correlation by job id
- enhanced `backend/job_manager.py` queue wrappers to emit correlated logs:
- queued / started / finished / failed entries include `[job_id]`
- covers general queue, GPU queue, and transcription queue
6. high-power endpoint review and protection
- added `backend/security.py` with `require_admin_token` dependency
- protected high-power endpoints when `ADMIN_TOKEN` is configured:
- `POST /api/system/gpu-reset`
- mutating `POST/DELETE` model-management endpoints under `/api/models/*`
- cleanup maintenance endpoints under `/api/projects/cleanup/*`
- dev behavior is preserved when `ADMIN_TOKEN` is unset
7. regression coverage for operational hardening
- added `backend/tests/test_operational_hardening.py` covering:
- health/readiness/executor visibility endpoints
- admin-token protection for system/models/projects cleanup routes
- updated `backend/tests/support.py` env bookkeeping for `ADMIN_TOKEN`
### Validation
- `python3 -m compileall backend` passed on host environment.
- backend unit tests requiring FastAPI runtime dependencies could not be executed on host (`ModuleNotFoundError: fastapi`).
- containerized test command could not be executed because no `stemengine` container is currently running in this host context.
### Resource-safe runtime test procedure (post-crash)
When host resources are tight and VS Code becomes unstable, do **not** start the default dev stack (frontend build + watch + backend reload). Use runtime-test mode:
1. bring down `stemengine` if running
2. start with runtime-test env overrides:
- `STEMENGINE_MODE=runtime-test`
- `FRONTEND_BUILD=0`
- `FRONTEND_WATCH=0`
- `UVICORN_RELOAD=0`
3. keep compose limits bounded (defaults now wired in compose):
- `SE_MEM_LIMIT=8g`
- `SE_CPUS=4.0`
- `SE_PIDS_LIMIT=1024`
4. run only the targeted Phase 4 tests (avoid full suite unless required)
5. after tests complete, stop the stack before returning to editor-heavy tasks
This mode is now supported directly by `start.sh` and `stemengine.yml` without code edits.
+12
View File
@@ -1,6 +1,18 @@
import os
from pathlib import Path
def _csv_env(name: str) -> list[str]:
value = os.environ.get(name, "")
if not value:
return []
return [item.strip() for item in value.split(",") if item.strip()]
APP_ENV = os.environ.get("APP_ENV", "development").strip().lower()
ADMIN_TOKEN = os.environ.get("ADMIN_TOKEN", "").strip()
CORS_ORIGINS = _csv_env("CORS_ORIGINS")
# Base paths — each can be overridden independently via its own env var,
# or all inherit from MEDIA_DIR when left unset.
MEDIA_DIR = Path(os.environ.get("MEDIA_DIR", "/app/media"))
+82 -1
View File
@@ -65,6 +65,24 @@ class ProjectRecord(Base):
mix_job_id = Column(String, nullable=True)
class ProjectStemJobLink(Base):
"""Normalized mapping of project + stem + stage -> job_id.
This reduces reliance on JSON blobs (`trans_jobs`, `abc_jobs`, `lrc_jobs`) for
operational queries while preserving backward compatibility.
"""
__tablename__ = "project_stem_job_links"
id = Column(String, primary_key=True, default=lambda: str(uuid4()))
project_id = Column(String, nullable=False, index=True)
stage = Column(String, nullable=False, index=True) # transcribe | abc | lrc
stem_name = Column(String, nullable=False)
job_id = Column(String, nullable=False, index=True)
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
SessionLocal = sessionmaker(bind=engine, expire_on_commit=False)
_init_lock = threading.Lock()
@@ -89,6 +107,19 @@ def init_db(logger=None) -> None:
"ALTER TABLE projects ADD COLUMN abc_jobs JSON",
"ALTER TABLE projects ADD COLUMN lrc_jobs JSON",
"ALTER TABLE projects ADD COLUMN tab_label TEXT DEFAULT ''",
"CREATE TABLE IF NOT EXISTS project_stem_job_links ("
"id TEXT PRIMARY KEY, "
"project_id TEXT NOT NULL, "
"stage TEXT NOT NULL, "
"stem_name TEXT NOT NULL, "
"job_id TEXT NOT NULL, "
"created_at DATETIME, "
"updated_at DATETIME"
")",
"CREATE INDEX IF NOT EXISTS ix_project_stem_job_links_project_id ON project_stem_job_links (project_id)",
"CREATE INDEX IF NOT EXISTS ix_project_stem_job_links_stage ON project_stem_job_links (stage)",
"CREATE INDEX IF NOT EXISTS ix_project_stem_job_links_job_id ON project_stem_job_links (job_id)",
"CREATE UNIQUE INDEX IF NOT EXISTS uq_project_stage_stem ON project_stem_job_links (project_id, stage, stem_name)",
)
for migration_sql in migrations:
@@ -102,4 +133,54 @@ def init_db(logger=None) -> None:
if logger:
logger.debug("Skipped DB migration %s: %s", migration_sql, exc)
_initialized = True
_backfill_project_stem_job_links(logger)
_initialized = True
def _backfill_project_stem_job_links(logger=None) -> None:
"""Backfill normalized link rows from legacy JSON columns once.
Safe to run repeatedly due to unique constraint and insert-or-ignore behavior.
"""
stage_map = (
("transcribe", "trans_jobs"),
("abc", "abc_jobs"),
("lrc", "lrc_jobs"),
)
inserted = 0
with SessionLocal() as session:
projects = session.query(ProjectRecord).all()
for project in projects:
for stage, field_name in stage_map:
job_map = getattr(project, field_name) or {}
if not isinstance(job_map, dict):
continue
for stem_name, job_id in job_map.items():
if not stem_name or not job_id:
continue
stmt = text(
"INSERT OR IGNORE INTO project_stem_job_links "
"(id, project_id, stage, stem_name, job_id, created_at, updated_at) "
"VALUES (:id, :project_id, :stage, :stem_name, :job_id, :created_at, :updated_at)"
)
now = datetime.utcnow()
result = session.execute(
stmt,
{
"id": str(uuid4()),
"project_id": project.id,
"stage": stage,
"stem_name": str(stem_name),
"job_id": str(job_id),
"created_at": now,
"updated_at": now,
},
)
if result.rowcount:
inserted += 1
session.commit()
if logger and inserted:
logger.info("Backfilled %d project stem-job link rows", inserted)
+82 -4
View File
@@ -1,16 +1,18 @@
"""
Job manager: SQLite-backed job state + ThreadPoolExecutor for background tasks.
"""
import re
import logging
import threading
from concurrent.futures import ThreadPoolExecutor
from enum import Enum
from typing import Callable, Optional
from .db import init_db
from .repositories import jobs as jobs_repo
from .repositories import projects as projects_repo
_log = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Cancellation flags: job_id → threading.Event (set = cancel requested)
# ---------------------------------------------------------------------------
@@ -63,6 +65,7 @@ def create_job(job_type: str) -> str:
job_id = jobs_repo.create(job_type)
with _flags_lock:
_cancel_flags[job_id] = threading.Event()
_log.info("[%s] created job_type=%s", job_id, job_type)
return job_id
@@ -112,7 +115,7 @@ def should_cancel(job_id: str) -> bool:
def submit(fn: Callable, *args, **kwargs):
"""Submit a CPU-bound background task (multi-worker)."""
_general_executor.submit(fn, *args, **kwargs)
_submit_with_logging(_general_executor, "general", fn, *args, **kwargs)
def submit_gpu(fn: Callable, *args, **kwargs):
@@ -121,7 +124,7 @@ def submit_gpu(fn: Callable, *args, **kwargs):
Use for stem separation (Demucs / BS-RoFormer) and MIDI transcription.
Tasks run strictly one-at-a-time in the order they were submitted.
"""
_gpu_executor.submit(fn, *args, **kwargs)
_submit_with_logging(_gpu_executor, "gpu", fn, *args, **kwargs)
def submit_transcription(job_id: str, fn: Callable, *args, **kwargs):
@@ -137,6 +140,8 @@ def submit_transcription(job_id: str, fn: Callable, *args, **kwargs):
_tx_queue_depth += 1
position = _tx_queue_depth
_log.info("[%s] queued on transcription queue position=%d", job_id, position)
if position > 1:
update_job(job_id, progress=f"Queued (position {position} of {position})…")
@@ -145,10 +150,15 @@ def submit_transcription(job_id: str, fn: Callable, *args, **kwargs):
# Recalculate position at start time (earlier jobs may have finished).
with _tx_queue_lock:
pos = _tx_queue_depth
_log.info("[%s] transcription task started queue_depth=%d", job_id, pos)
if pos > 1:
update_job(job_id, progress=f"Queued — waiting for {pos - 1} job(s) to finish…")
try:
fn(*args, **kwargs)
_log.info("[%s] transcription task finished", job_id)
except Exception:
_log.exception("[%s] transcription task failed", job_id)
raise
finally:
with _tx_queue_lock:
_tx_queue_depth -= 1
@@ -190,3 +200,71 @@ def set_project_abc_job(project_id: str, stem_name: str, job_id: str):
def set_project_lrc_job(project_id: str, stem_name: str, job_id: str):
"""Store a per-stem LRC alignment job_id in the project's lrc_jobs dict."""
projects_repo.set_lrc_job(project_id, stem_name, job_id)
def get_executor_state() -> dict:
"""Return queue/worker visibility for operational dashboards."""
with _tx_queue_lock:
tx_depth = _tx_queue_depth
return {
"general": _executor_state(_general_executor),
"gpu": _executor_state(_gpu_executor),
"transcription": {
**_executor_state(_transcription_executor),
"queue_depth_including_running": tx_depth,
},
}
def _extract_job_id(args: tuple, kwargs: dict) -> str | None:
if args and isinstance(args[0], str):
return args[0]
candidate = kwargs.get("job_id")
if isinstance(candidate, str):
return candidate
return None
def _submit_with_logging(executor: ThreadPoolExecutor, queue_name: str, fn: Callable, *args, **kwargs):
job_id = _extract_job_id(args, kwargs)
fn_name = getattr(fn, "__name__", fn.__class__.__name__)
if job_id:
_log.info("[%s] queued on %s queue task=%s", job_id, queue_name, fn_name)
def _wrapper():
if job_id:
_log.info("[%s] started on %s queue task=%s", job_id, queue_name, fn_name)
try:
fn(*args, **kwargs)
if job_id:
_log.info("[%s] finished on %s queue task=%s", job_id, queue_name, fn_name)
except Exception:
if job_id:
_log.exception("[%s] failed on %s queue task=%s", job_id, queue_name, fn_name)
raise
executor.submit(_wrapper)
def _executor_state(executor: ThreadPoolExecutor) -> dict:
queued = None
workers = None
alive_workers = None
try:
queued = executor._work_queue.qsize()
except Exception:
pass
try:
workers = len(executor._threads)
alive_workers = sum(1 for t in executor._threads if t.is_alive())
except Exception:
pass
return {
"queued": queued,
"workers": workers,
"alive_workers": alive_workers,
}
+9 -2
View File
@@ -38,7 +38,7 @@ from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse
from .routers import separate, transcribe, abc_convert, pipeline, jobs, lrc, subsonic, projects, models, lyric_transcribe, system, settings
from .config import SUBSONIC_CONFIG_PATH
from .config import APP_ENV, CORS_ORIGINS, SUBSONIC_CONFIG_PATH
from .db import init_db
_log = logging.getLogger(__name__)
@@ -116,9 +116,16 @@ def _mount_static(app: FastAPI) -> None:
def create_app() -> FastAPI:
app = FastAPI(title="StemEngine", version="1.0.0", lifespan=lifespan)
if CORS_ORIGINS:
allow_origins = CORS_ORIGINS
elif APP_ENV in {"dev", "development", "local", "test"}:
allow_origins = ["*"]
else:
allow_origins = []
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_origins=allow_origins,
allow_methods=["*"],
allow_headers=["*"],
)
+71 -6
View File
@@ -5,8 +5,9 @@ from typing import Optional
from uuid import uuid4
from sqlalchemy.orm.attributes import flag_modified
from sqlalchemy import text
from ..db import ProjectRecord, SessionLocal, init_db
from ..db import ProjectRecord, ProjectStemJobLink, SessionLocal, init_db
from . import jobs as jobs_repo
@@ -27,19 +28,19 @@ def find_or_create(name: str, sep_job_id: str, tab_label: str = '') -> dict:
)
session.add(record)
session.commit()
return to_dict(record)
return _to_dict_with_links(session, record)
def get(project_id: str) -> Optional[dict]:
with _session() as session:
record = session.get(ProjectRecord, project_id)
return to_dict(record) if record else None
return _to_dict_with_links(session, record) if record else None
def get_by_sep_job(sep_job_id: str) -> Optional[dict]:
with _session() as session:
record = session.query(ProjectRecord).filter_by(sep_job_id=sep_job_id).first()
return to_dict(record) if record else None
return _to_dict_with_links(session, record) if record else None
def update(project_id: str, **kwargs) -> bool:
@@ -58,6 +59,7 @@ def delete(project_id: str) -> bool:
record = session.get(ProjectRecord, project_id)
if record is None:
return False
session.query(ProjectStemJobLink).filter_by(project_id=project_id).delete()
session.delete(record)
session.commit()
return True
@@ -67,7 +69,7 @@ def list_persisted() -> list[dict]:
"""Return only existing project records without auto-materializing new ones."""
with _session() as session:
records = session.query(ProjectRecord).order_by(ProjectRecord.created_at.desc()).all()
return [to_dict(record) for record in records]
return [_to_dict_with_links(session, record) for record in records]
def list_all() -> list[dict]:
@@ -90,7 +92,7 @@ def list_all() -> list[dict]:
session.commit()
records = session.query(ProjectRecord).order_by(ProjectRecord.created_at.desc()).all()
return [to_dict(record) for record in records]
return [_to_dict_with_links(session, record) for record in records]
def set_transcription_job(project_id: str, stem_name: str, job_id: str) -> bool:
@@ -114,10 +116,73 @@ def _set_json_job_map(project_id: str, field_name: str, stem_name: str, job_id:
jobs[stem_name] = job_id
setattr(record, field_name, jobs)
flag_modified(record, field_name)
stage = {
"trans_jobs": "transcribe",
"abc_jobs": "abc",
"lrc_jobs": "lrc",
}[field_name]
_upsert_stem_job_link(session, project_id, stage, stem_name, job_id)
session.commit()
return True
def _upsert_stem_job_link(session, project_id: str, stage: str, stem_name: str, job_id: str) -> None:
now_expr = "CURRENT_TIMESTAMP"
session.execute(
text(
"INSERT INTO project_stem_job_links (id, project_id, stage, stem_name, job_id, created_at, updated_at) "
"VALUES (:id, :project_id, :stage, :stem_name, :job_id, " + now_expr + ", " + now_expr + ") "
"ON CONFLICT(project_id, stage, stem_name) DO UPDATE SET "
"job_id=excluded.job_id, updated_at=" + now_expr
),
{
"id": str(uuid4()),
"project_id": project_id,
"stage": stage,
"stem_name": stem_name,
"job_id": job_id,
},
)
def _load_stage_map(session, project_id: str, stage: str) -> dict:
links = (
session.query(ProjectStemJobLink)
.filter_by(project_id=project_id, stage=stage)
.all()
)
return {link.stem_name: link.job_id for link in links}
def _to_dict_with_links(session, record: ProjectRecord | None) -> Optional[dict]:
if record is None:
return None
trans_jobs = _load_stage_map(session, record.id, "transcribe")
abc_jobs = _load_stage_map(session, record.id, "abc")
lrc_jobs = _load_stage_map(session, record.id, "lrc")
if not trans_jobs:
trans_jobs = record.trans_jobs or {}
if not abc_jobs:
abc_jobs = record.abc_jobs or {}
if not lrc_jobs:
lrc_jobs = record.lrc_jobs or {}
return {
"id": record.id,
"name": record.name,
"tab_label": record.tab_label or '',
"created_at": record.created_at.isoformat() if record.created_at else None,
"sep_job_id": record.sep_job_id,
"trans_jobs": trans_jobs,
"abc_jobs": abc_jobs,
"lrc_jobs": lrc_jobs,
"mix_job_id": record.mix_job_id,
}
def to_dict(record: ProjectRecord | None) -> Optional[dict]:
if record is None:
return None
+11 -6
View File
@@ -11,7 +11,7 @@ from enum import Enum
from pathlib import Path
from typing import Dict, List, Optional
from fastapi import APIRouter, HTTPException, BackgroundTasks
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
from pydantic import BaseModel
from ..config import (
@@ -27,6 +27,7 @@ from ..config import (
SEPACAP_MODEL_DIR,
SEPREFORMER_MODEL_DIR,
)
from ..security import require_admin_token
router = APIRouter(prefix="/api/models", tags=["models"])
@@ -558,7 +559,11 @@ def list_models():
@router.post("/download", response_model=ModelDownloadResponse)
def download_model(body: ModelDownloadRequest, background_tasks: BackgroundTasks):
def download_model(
body: ModelDownloadRequest,
background_tasks: BackgroundTasks,
_: None = Depends(require_admin_token),
):
"""Start downloading a model in the background."""
model_id = body.model_id
@@ -595,7 +600,7 @@ def download_model(body: ModelDownloadRequest, background_tasks: BackgroundTasks
@router.post("/pause")
def pause_download(body: ModelDownloadRequest):
def pause_download(body: ModelDownloadRequest, _: None = Depends(require_admin_token)):
"""Pause an ongoing download."""
model_id = body.model_id
@@ -613,7 +618,7 @@ def pause_download(body: ModelDownloadRequest):
@router.post("/resume")
def resume_download(body: ModelDownloadRequest):
def resume_download(body: ModelDownloadRequest, _: None = Depends(require_admin_token)):
"""Resume a paused download."""
model_id = body.model_id
@@ -631,7 +636,7 @@ def resume_download(body: ModelDownloadRequest):
@router.post("/cancel")
def cancel_download(body: ModelDownloadRequest):
def cancel_download(body: ModelDownloadRequest, _: None = Depends(require_admin_token)):
"""Cancel an ongoing or paused download."""
model_id = body.model_id
@@ -676,7 +681,7 @@ def get_all_progress():
@router.delete("/{model_id}")
def delete_model(model_id: str):
def delete_model(model_id: str, _: None = Depends(require_admin_token)):
"""Delete a downloaded model to free up space."""
if model_id not in MODEL_DOWNLOAD_INFO:
raise HTTPException(status_code=404, detail=f"Unknown model: {model_id}")
+7 -6
View File
@@ -7,7 +7,7 @@ It links the separation job to downstream jobs (transcription, ABC, mix).
from pathlib import Path
from typing import Optional
from fastapi import APIRouter, HTTPException, Response
from fastapi import APIRouter, Depends, HTTPException, Response
from pydantic import BaseModel
from ..config import DEFAULT_TRANSCRIBE_MODE
@@ -33,6 +33,7 @@ from ..services.project_service import (
write_all_project_metadata_snapshots,
)
from ..services.lyric_transcriber_service import DEFAULT_WHISPER_MODEL
from ..security import require_admin_token
router = APIRouter(prefix="/api/projects", tags=["projects"])
@@ -298,7 +299,7 @@ def rescan_stems(project_id: str):
# ---------------------------------------------------------------------------
@router.get("/cleanup/verify")
def cleanup_verify():
def cleanup_verify(_: None = Depends(require_admin_token)):
"""
Check every project's stem files are actually on disk.
Returns a list of projects with missing files — no mutations performed.
@@ -307,7 +308,7 @@ def cleanup_verify():
@router.post("/cleanup/write-metadata")
def cleanup_write_metadata():
def cleanup_write_metadata(_: None = Depends(require_admin_token)):
"""
Write/refresh project.json for every project in the database.
Uses current DB state so trans_jobs, lrc_jobs etc. are included.
@@ -316,7 +317,7 @@ def cleanup_write_metadata():
@router.post("/cleanup/rename-stems")
def cleanup_rename_stems():
def cleanup_rename_stems(_: None = Depends(require_admin_token)):
"""
Rename all stem files that still use the old 'filename - Vocals.ext' format
to the new 'Artist - Title (vocals).ext' format.
@@ -326,7 +327,7 @@ def cleanup_rename_stems():
@router.post("/cleanup/purge-orphaned")
def cleanup_purge_orphaned():
def cleanup_purge_orphaned(_: None = Depends(require_admin_token)):
"""
Delete separation job rows (job_type='separate') that have no matching
ProjectRecord. Also deletes jobs of any type whose parent project is gone.
@@ -335,7 +336,7 @@ def cleanup_purge_orphaned():
@router.post("/cleanup/clean-lrc-files")
def cleanup_clean_lrc_files():
def cleanup_clean_lrc_files(_: None = Depends(require_admin_token)):
"""
Scan every LRC/ELRC output file in all lrc jobs and strip section-marker
lines (e.g. [01:30.00][Chorus]). Returns counts of files and lines affected.
+63 -2
View File
@@ -1,11 +1,41 @@
"""System information endpoints (GPU hardware)."""
from fastapi import APIRouter
from pathlib import Path
from fastapi import APIRouter, Depends
from .. import job_manager as jm
from ..config import APP_ENV, DB_PATH, JOBS_DIR, MEDIA_DIR, PROJECTS_DIR, UPLOAD_DIR
from ..db import engine
from ..security import require_admin_token
router = APIRouter(prefix="/api/system", tags=["system"])
@router.get("/health")
def health():
"""Lightweight liveness probe."""
return {"status": "ok", "service": "stemengine", "environment": APP_ENV}
@router.get("/readiness")
def readiness():
"""Readiness probe for DB connectivity and writable core directories."""
checks = {
"db": _db_ready(),
"directories": _dirs_ready(),
}
ok = checks["db"]["ok"] and checks["directories"]["ok"]
return {"status": "ready" if ok else "not_ready", "ok": ok, "checks": checks}
@router.get("/executors")
def executors_state():
"""Queue-depth / worker visibility for background executors."""
return jm.get_executor_state()
@router.post("/gpu-reset")
def gpu_reset():
def gpu_reset(_: None = Depends(require_admin_token)):
"""
Re-enable GPU acceleration for all services without restarting the container.
@@ -73,3 +103,34 @@ def gpu_info():
}
except Exception as exc: # pragma: no cover
return {"cuda_available": False, "cuda_ok": False, "name": None, "total_mb": 0, "free_mb": 0, "error": str(exc)}
def _db_ready() -> dict:
try:
with engine.connect() as conn:
conn.exec_driver_sql("SELECT 1")
return {"ok": True, "path": str(DB_PATH)}
except Exception as exc:
return {"ok": False, "path": str(DB_PATH), "error": str(exc)}
def _dirs_ready() -> dict:
required: list[Path] = [MEDIA_DIR, UPLOAD_DIR, JOBS_DIR, PROJECTS_DIR]
details = {}
ok = True
for directory in required:
exists = directory.exists()
writable = exists and directory.is_dir()
if writable:
try:
probe = directory / ".write_probe"
probe.write_text("ok", encoding="utf-8")
probe.unlink(missing_ok=True)
except Exception:
writable = False
details[str(directory)] = {"exists": exists, "writable": writable}
if not (exists and writable):
ok = False
return {"ok": ok, "details": details}
+17
View File
@@ -0,0 +1,17 @@
"""Security helpers for high-power operational endpoints."""
from __future__ import annotations
from fastapi import Header, HTTPException
from .config import ADMIN_TOKEN
def require_admin_token(x_admin_token: str | None = Header(default=None, alias="X-Admin-Token")) -> None:
"""Require the configured admin token when ADMIN_TOKEN is set.
In development (no ADMIN_TOKEN configured), this is a no-op.
"""
if not ADMIN_TOKEN:
return
if x_admin_token != ADMIN_TOKEN:
raise HTTPException(status_code=403, detail="Forbidden")
+19 -241
View File
@@ -1,21 +1,13 @@
"""
Full pipeline: separate → transcribe → abc-convert.
Supports both Demucs (CLI) and BS-RoFormer (Python API) for the separation stage.
Uses project-based folder structure for organized output.
"""
"""Pipeline orchestrator composed from explicit separation/transcription/ABC stages."""
import logging
import sys
import subprocess
import time
import shutil
from pathlib import Path
from .. import job_manager as jm
from ..config import PROJECTS_DIR, JOBS_DIR, CPU_SEPARATION, TRANSCRIBE_PRESETS
from .metadata_util import copy_metadata, read_metadata
from .project_namer import project_name, make_project_dirs
from .separator_service import BSR_PREFIX, BSR_STEM_MODE
from . import abc_generator
from ..config import PROJECTS_DIR
from .metadata_util import read_metadata
from .pipeline_stages import run_abc_stage, run_separation_stage, run_transcription_stage
from .project_namer import make_project_dirs, project_name
log = logging.getLogger(__name__)
@@ -65,24 +57,8 @@ def run_pipeline(
all_files = []
# Normalise mp3_320 → mp3 with explicit bitrate
mp3_bitrate = '320k' if sep_format == 'mp3_320' else '256k'
normalised_format = 'mp3' if sep_format == 'mp3_320' else sep_format
# ------------------------------------------------------------------
# Stage 1: Separation
# ------------------------------------------------------------------
jm.update_job(job_id, progress="[1/3] Separating stems...")
try:
if sep_model.startswith(BSR_PREFIX):
stem_files = _separate_bsroformer(
job_id, input_path, sep_model, normalised_format, stems_dir, mp3_bitrate
)
else:
stem_files = _separate_demucs(
job_id, input_path, sep_model, sep_format, stems_dir
)
stem_files = run_separation_stage(job_id, input_path, sep_model, sep_format, stems_dir)
except Exception as exc:
log.debug("[%s] pipeline separation failed — %s: %s", job_id, type(exc).__name__, exc)
jm.update_job(job_id, status=jm.JobStatus.ERROR, error=str(exc), progress="Separation failed.")
@@ -91,73 +67,19 @@ def run_pipeline(
if stem_files is None:
return # cancelled inside sub-function
# Record stems with proper metadata
for stem_name, path in sorted(stem_files.items()):
copy_metadata(input_path, path, stem_label=stem_name.capitalize())
all_files.append({
"name": Path(path).name,
"path": path,
"stage": "separation",
"stem": stem_name
})
midi_files, stage_files = run_transcription_stage(
job_id,
proj_name,
input_path,
stem_files,
transcribe_mode,
transcriptions_dir,
)
all_files.extend(stage_files)
if not midi_files and jm.should_cancel(job_id):
return
# ------------------------------------------------------------------
# Stage 2: Transcription (skip drum stems)
# ------------------------------------------------------------------
from basic_pitch.inference import predict
from basic_pitch import ICASSP_2022_MODEL_PATH
jm.update_job(job_id, progress="[2/3] Transcribing stems to MIDI...")
midi_files = []
transcribe_paths = [
(name, path) for name, path in stem_files.items()
if 'drums' not in name.lower()
]
params = TRANSCRIBE_PRESETS.get(transcribe_mode, TRANSCRIBE_PRESETS["polyphonic"])
for stem_name, stem_path in transcribe_paths:
if jm.should_cancel(job_id):
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return
jm.update_job(job_id, progress=f"[2/3] Transcribing: {stem_name}...")
try:
_, midi_data, _ = predict(stem_path, ICASSP_2022_MODEL_PATH, **params)
output_name = f"{proj_name} - {stem_name.capitalize()} ({transcribe_mode}).mid"
dest = transcriptions_dir / output_name
midi_data.write(str(dest))
all_files.append({
"name": dest.name,
"path": str(dest),
"stage": "transcription"
})
midi_files.append(str(dest))
except Exception as exc:
jm.update_job(job_id, progress=f"[2/3] Error on {stem_name}: {exc}")
# ------------------------------------------------------------------
# Stage 3: ABC conversion
# ------------------------------------------------------------------
if midi_files:
jm.update_job(job_id, progress="[3/3] Converting MIDI to ABC notation...")
try:
abc_results = abc_generator.convert_midi_to_abc(midi_files, song_title or proj_name)
if abc_results:
for part_name, abc_text in abc_results.items():
safe = part_name.replace("/", "_").replace("\\", "_")
dest = transcriptions_dir / f"{safe}.abc"
dest.write_text(abc_text, encoding="utf-8")
all_files.append({
"name": dest.name,
"path": str(dest),
"stage": "abc"
})
except Exception as exc:
jm.update_job(job_id, progress=f"[3/3] ABC conversion error: {exc}")
run_abc_stage(job_id, midi_files, song_title or proj_name, all_files)
log.debug("[%s] pipeline complete — project=%s stems=%d files=%d",
job_id, proj_name, len(stem_files), len(all_files))
@@ -173,147 +95,3 @@ def run_pipeline(
}
)
jm.find_or_create_project(proj_name, job_id)
# ---------------------------------------------------------------------------
# Separation helpers
# ---------------------------------------------------------------------------
def _separate_demucs(
job_id: str, input_path: str, model: str, output_format: str, stems_dir: Path
) -> dict | None:
"""
Run Demucs CLI; returns dict mapping stem names to file paths, or None if cancelled.
"""
ext = output_format.lower()
fmt_flag = []
if ext == "mp3_320":
fmt_flag = ["--mp3", "--mp3-bitrate", "320"]
ext = "mp3"
elif ext == "mp3":
fmt_flag = ["--mp3", "--mp3-bitrate", "256"]
elif ext == "flac":
fmt_flag = ["--flac"]
# Use temp directory for demucs output
temp_out = JOBS_DIR / job_id / "demucs_out"
temp_out.mkdir(parents=True, exist_ok=True)
cmd = [
sys.executable, "-m", "demucs",
"--name", model,
"--out", str(temp_out),
*fmt_flag,
input_path,
]
try:
process = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
)
while True:
if jm.should_cancel(job_id):
process.terminate()
process.wait()
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return None
line = process.stdout.readline()
if line:
jm.update_job(job_id, progress=f"[1/3] {line.rstrip()}")
elif process.poll() is not None:
break
else:
time.sleep(0.1)
if process.returncode != 0:
raise RuntimeError(f"Demucs failed with code {process.returncode}")
input_stem = Path(input_path).stem
stem_src_dir = temp_out / model / input_stem
stem_files = {}
if stem_src_dir.exists():
for f in sorted(stem_src_dir.iterdir()):
if f.is_file():
stem_name = f.stem.lower()
for prefix in [input_stem.lower(), 'htdemucs', 'mdx']:
stem_name = stem_name.replace(prefix, '').strip('_- ')
if not stem_name:
stem_name = f.stem
dest = stems_dir / f"{input_stem} - {stem_name.capitalize()}{f.suffix}"
shutil.move(str(f), str(dest))
stem_files[stem_name] = str(dest)
try:
shutil.rmtree(temp_out, ignore_errors=True)
except:
pass
return stem_files
except Exception:
try:
shutil.rmtree(temp_out, ignore_errors=True)
except:
pass
raise
def _separate_bsroformer(
job_id: str, input_path: str, model: str, output_format: str, stems_dir: Path,
mp3_bitrate: str = '256k'
) -> dict | None:
"""
Run BS-RoFormer in-process; returns dict mapping stem names to file paths, or None if cancelled.
"""
from .bs_roformer_separator import BSRoformerSeparator
stem_mode = BSR_STEM_MODE.get(model, "4stem")
jm.update_job(job_id, progress=f"[1/3] Loading BS-RoFormer ({stem_mode})...")
if jm.should_cancel(job_id):
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return None
sep = BSRoformerSeparator(
cpu_separation=CPU_SEPARATION,
output_format=output_format,
mp3_bitrate=mp3_bitrate,
stem_mode=stem_mode,
)
jm.update_job(job_id, progress="[1/3] Separating with BS-RoFormer...")
temp_out = JOBS_DIR / job_id / "bsr_out"
temp_out.mkdir(parents=True, exist_ok=True)
try:
stem_paths = sep.separate_into_parts(input_path, str(temp_out))
if jm.should_cancel(job_id):
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return None
input_stem = Path(input_path).stem
stem_files = {}
for stem_name, path in sorted(stem_paths.items()):
src = Path(path)
dest = stems_dir / f"{input_stem} - {stem_name.capitalize()}{src.suffix}"
shutil.move(str(src), str(dest))
stem_files[stem_name.lower()] = str(dest)
try:
shutil.rmtree(temp_out, ignore_errors=True)
except:
pass
return stem_files
except Exception:
try:
shutil.rmtree(temp_out, ignore_errors=True)
except:
pass
raise
+248
View File
@@ -0,0 +1,248 @@
from __future__ import annotations
import shutil
import subprocess
import sys
import time
from pathlib import Path
from .. import job_manager as jm
from ..config import CPU_SEPARATION, JOBS_DIR, TRANSCRIBE_PRESETS
from . import abc_generator
from .metadata_util import copy_metadata
from .separation.model_registry import BSR_PREFIX, BSR_STEM_MODE
def run_separation_stage(
job_id: str,
input_path: str,
sep_model: str,
sep_format: str,
stems_dir: Path,
) -> dict | None:
jm.update_job(job_id, progress="[1/3] Separating stems...")
mp3_bitrate = "320k" if sep_format == "mp3_320" else "256k"
normalised_format = "mp3" if sep_format == "mp3_320" else sep_format
if sep_model.startswith(BSR_PREFIX):
return _separate_bsroformer(
job_id,
input_path,
sep_model,
normalised_format,
stems_dir,
mp3_bitrate,
)
return _separate_demucs(job_id, input_path, sep_model, sep_format, stems_dir)
def run_transcription_stage(
job_id: str,
proj_name: str,
input_path: str,
stem_files: dict,
transcribe_mode: str,
transcriptions_dir: Path,
) -> tuple[list[str], list[dict]]:
from basic_pitch.inference import predict
from basic_pitch import ICASSP_2022_MODEL_PATH
all_files = []
midi_files = []
for stem_name, path in sorted(stem_files.items()):
copy_metadata(input_path, path, stem_label=stem_name.capitalize())
all_files.append(
{
"name": Path(path).name,
"path": path,
"stage": "separation",
"stem": stem_name,
}
)
jm.update_job(job_id, progress="[2/3] Transcribing stems to MIDI...")
transcribe_paths = [
(name, path) for name, path in stem_files.items() if "drums" not in name.lower()
]
params = TRANSCRIBE_PRESETS.get(transcribe_mode, TRANSCRIBE_PRESETS["polyphonic"])
for stem_name, stem_path in transcribe_paths:
if jm.should_cancel(job_id):
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return [], all_files
jm.update_job(job_id, progress=f"[2/3] Transcribing: {stem_name}...")
try:
_, midi_data, _ = predict(stem_path, ICASSP_2022_MODEL_PATH, **params)
output_name = f"{proj_name} - {stem_name.capitalize()} ({transcribe_mode}).mid"
dest = transcriptions_dir / output_name
midi_data.write(str(dest))
all_files.append(
{
"name": dest.name,
"path": str(dest),
"stage": "transcription",
}
)
midi_files.append(str(dest))
except Exception as exc:
jm.update_job(job_id, progress=f"[2/3] Error on {stem_name}: {exc}")
return midi_files, all_files
def run_abc_stage(job_id: str, midi_files: list[str], song_title: str, all_files: list[dict]) -> None:
if not midi_files:
return
jm.update_job(job_id, progress="[3/3] Converting MIDI to ABC notation...")
try:
abc_results = abc_generator.convert_midi_to_abc(midi_files, song_title)
if abc_results:
first_mid_parent = Path(midi_files[0]).parent
for part_name, abc_text in abc_results.items():
safe = part_name.replace("/", "_").replace("\\", "_")
dest = first_mid_parent / f"{safe}.abc"
dest.write_text(abc_text, encoding="utf-8")
all_files.append(
{
"name": dest.name,
"path": str(dest),
"stage": "abc",
}
)
except Exception as exc:
jm.update_job(job_id, progress=f"[3/3] ABC conversion error: {exc}")
def _separate_demucs(
job_id: str, input_path: str, model: str, output_format: str, stems_dir: Path
) -> dict | None:
ext = output_format.lower()
fmt_flag = []
if ext == "mp3_320":
fmt_flag = ["--mp3", "--mp3-bitrate", "320"]
ext = "mp3"
elif ext == "mp3":
fmt_flag = ["--mp3", "--mp3-bitrate", "256"]
elif ext == "flac":
fmt_flag = ["--flac"]
temp_out = JOBS_DIR / job_id / "demucs_out"
temp_out.mkdir(parents=True, exist_ok=True)
cmd = [
sys.executable,
"-m",
"demucs",
"--name",
model,
"--out",
str(temp_out),
*fmt_flag,
input_path,
]
try:
process = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True
)
while True:
if jm.should_cancel(job_id):
process.terminate()
process.wait()
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return None
line = process.stdout.readline()
if line:
jm.update_job(job_id, progress=f"[1/3] {line.rstrip()}")
elif process.poll() is not None:
break
else:
time.sleep(0.1)
if process.returncode != 0:
raise RuntimeError(f"Demucs failed with code {process.returncode}")
input_stem = Path(input_path).stem
stem_src_dir = temp_out / model / input_stem
stem_files = {}
if stem_src_dir.exists():
for f in sorted(stem_src_dir.iterdir()):
if f.is_file():
stem_name = f.stem.lower()
for prefix in [input_stem.lower(), "htdemucs", "mdx"]:
stem_name = stem_name.replace(prefix, "").strip("_- ")
if not stem_name:
stem_name = f.stem
dest = stems_dir / f"{input_stem} - {stem_name.capitalize()}{f.suffix}"
shutil.move(str(f), str(dest))
stem_files[stem_name] = str(dest)
shutil.rmtree(temp_out, ignore_errors=True)
return stem_files
except Exception:
shutil.rmtree(temp_out, ignore_errors=True)
raise
def _separate_bsroformer(
job_id: str,
input_path: str,
model: str,
output_format: str,
stems_dir: Path,
mp3_bitrate: str = "256k",
) -> dict | None:
from .bs_roformer_separator import BSRoformerSeparator
stem_mode = BSR_STEM_MODE.get(model, "4stem")
jm.update_job(job_id, progress=f"[1/3] Loading BS-RoFormer ({stem_mode})...")
if jm.should_cancel(job_id):
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return None
sep = BSRoformerSeparator(
cpu_separation=CPU_SEPARATION,
output_format=output_format,
mp3_bitrate=mp3_bitrate,
stem_mode=stem_mode,
)
jm.update_job(job_id, progress="[1/3] Separating with BS-RoFormer...")
temp_out = JOBS_DIR / job_id / "bsr_out"
temp_out.mkdir(parents=True, exist_ok=True)
try:
stem_paths = sep.separate_into_parts(input_path, str(temp_out))
if jm.should_cancel(job_id):
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return None
input_stem = Path(input_path).stem
stem_files = {}
for stem_name, path in sorted(stem_paths.items()):
src = Path(path)
dest = stems_dir / f"{input_stem} - {stem_name.capitalize()}{src.suffix}"
shutil.move(str(src), str(dest))
stem_files[stem_name.lower()] = str(dest)
shutil.rmtree(temp_out, ignore_errors=True)
return stem_files
except Exception:
shutil.rmtree(temp_out, ignore_errors=True)
raise
+211
View File
@@ -0,0 +1,211 @@
from __future__ import annotations
import os
import shutil
from pathlib import Path
from ..config import UPLOAD_DIR
from ..repositories import jobs as jobs_repo
from ..repositories import projects as projects_repo
from .metadata_util import _read_cover_art, read_metadata
from .project_service_common import ProjectOperationError, _infer_stem_name, project_output_dir
def list_projects() -> list[dict]:
return projects_repo.list_all()
def get_project_or_raise(project_id: str) -> dict:
project = projects_repo.get(project_id)
if project is None:
raise ProjectOperationError(404, f"Project not found: {project_id}")
return project
def collect_related_job_ids(project: dict) -> set[str]:
job_ids = {project["sep_job_id"]}
for job_id in (project.get("trans_jobs") or {}).values():
job_ids.add(job_id)
for job_id in (project.get("abc_jobs") or {}).values():
job_ids.add(job_id)
for job_id in (project.get("lrc_jobs") or {}).values():
job_ids.add(job_id)
if project.get("mix_job_id"):
job_ids.add(project["mix_job_id"])
return job_ids
def delete_project_with_related_jobs(project: dict) -> bool:
"""Delete project files plus the project record and all linked job records."""
project_dir = project_output_dir(project)
if project_dir.exists():
shutil.rmtree(project_dir, ignore_errors=True)
upload_dir = UPLOAD_DIR / project["sep_job_id"]
if upload_dir.exists():
shutil.rmtree(upload_dir, ignore_errors=True)
job_ids_to_delete = collect_related_job_ids(project)
deleted = projects_repo.delete(project["id"])
for job_id in job_ids_to_delete:
jobs_repo.delete(job_id)
return deleted
def build_project_detail(project: dict) -> dict:
"""Expand linked jobs and source metadata for a project detail response."""
trans_jobs_expanded = {
stem: jobs_repo.get(job_id)
for stem, job_id in (project.get("trans_jobs") or {}).items()
}
abc_jobs_expanded = {
stem: jobs_repo.get(job_id)
for stem, job_id in (project.get("abc_jobs") or {}).items()
}
lrc_jobs_expanded = {
stem: jobs_repo.get(job_id)
for stem, job_id in (project.get("lrc_jobs") or {}).items()
}
sep_job = jobs_repo.get(project["sep_job_id"])
mix_job = jobs_repo.get(project.get("mix_job_id")) if project.get("mix_job_id") else None
source_artist = ""
source_title = ""
source_file = next(
(file_entry for file_entry in (sep_job or {}).get("files", []) if file_entry.get("type") == "source"),
None,
)
if source_file:
src_path = source_file.get("path", "")
if src_path and Path(src_path).exists():
try:
meta = read_metadata(src_path)
source_artist = meta.get("artist") or meta.get("albumartist") or ""
source_title = meta.get("title") or ""
except Exception:
pass
return {
**project,
"sep_job": sep_job,
"trans_jobs": trans_jobs_expanded,
"abc_jobs": abc_jobs_expanded,
"lrc_jobs": lrc_jobs_expanded,
"mix_job": mix_job,
"source_artist": source_artist,
"source_title": source_title,
}
def rename_project_stem(project: dict, stem_name: str, new_name: str) -> dict:
"""Rename a tracked stem on disk and update the separation job record."""
from .separator_service import _stem_filename
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_files = list(sep_job.get("files", []))
stem_file = None
stem_idx = None
for index, file_entry in enumerate(stem_files):
if file_entry.get("stem") == stem_name or file_entry.get("name") == stem_name:
stem_file = file_entry
stem_idx = index
break
if stem_file is None or stem_idx is None:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
old_path = Path(stem_file["path"])
if not old_path.exists():
raise ProjectOperationError(404, f"Stem file not found on disk: {old_path}")
cleaned_name = new_name.strip()
if not cleaned_name:
raise ProjectOperationError(400, "New name cannot be empty")
new_filename = _stem_filename(project["name"], cleaned_name, old_path.suffix)
new_path = old_path.parent / new_filename
if new_path.exists() and new_path != old_path:
raise ProjectOperationError(409, f"A file with name '{new_filename}' already exists")
try:
os.rename(str(old_path), str(new_path))
except Exception as exc:
raise ProjectOperationError(500, f"Failed to rename stem: {exc}") from exc
stem_files[stem_idx] = {
**stem_file,
"name": new_filename,
"stem": cleaned_name,
"path": str(new_path),
}
jobs_repo.update(project["sep_job_id"], files=stem_files)
return {
"success": True,
"old_name": stem_name,
"new_name": cleaned_name,
"new_path": str(new_path),
}
def rescan_project_stems(project: dict, audio_exts: set[str]) -> dict:
"""Register untracked audio files under the project's Stems directory."""
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stems_dir = project_output_dir(project) / "Stems"
if not stems_dir.exists():
raise ProjectOperationError(404, "Stems directory not found")
files = list(sep_job.get("files", []))
tracked_paths = {file_entry["path"] for file_entry in files}
added = []
for audio_file in sorted(stems_dir.rglob("*")):
if not audio_file.is_file():
continue
if audio_file.suffix.lower() not in audio_exts:
continue
if str(audio_file) in tracked_paths:
continue
entry = {
"name": audio_file.name,
"path": str(audio_file),
"type": "stem",
"stem": _infer_stem_name(audio_file.stem, project["name"]),
}
files.append(entry)
tracked_paths.add(str(audio_file))
added.append(entry)
if added:
jobs_repo.update(project["sep_job_id"], files=files)
return {"added": len(added), "stems": [entry["stem"] for entry in added]}
def get_project_cover_art(project: dict) -> tuple[bytes, str]:
"""Return embedded cover-art bytes and mime type from a project's source file."""
sep_job_data = jobs_repo.get(project["sep_job_id"]) or {}
source_file = next(
(file_entry for file_entry in sep_job_data.get("files", []) if file_entry.get("type") == "source"),
None,
)
if not source_file:
raise ProjectOperationError(404, "No source file")
source_path = source_file.get("path", "")
if not source_path or not Path(source_path).exists():
raise ProjectOperationError(404, "Source file not found")
data, mime = _read_cover_art(source_path)
if not data:
raise ProjectOperationError(404, "No cover art")
return data, mime or "image/jpeg"
+202
View File
@@ -0,0 +1,202 @@
from __future__ import annotations
from pathlib import Path
from ..repositories import jobs as jobs_repo
from ..repositories import projects as projects_repo
from .project_metadata import write_project_metadata
from .project_service_common import project_output_dir
def verify_project_stems(projects: list[dict]) -> dict:
"""Report projects whose tracked stem files are missing on disk."""
results = []
for project in projects:
sep_job = jobs_repo.get(project["sep_job_id"]) or {}
files = sep_job.get("files", [])
missing = [
file_entry["path"]
for file_entry in files
if file_entry.get("type") == "stem" and not Path(file_entry.get("path", "")).exists()
]
if missing:
results.append(
{
"project_id": project["id"],
"name": project["name"],
"missing_files": missing,
}
)
return {"projects_with_missing_files": results, "total": len(results)}
def write_project_metadata_snapshots(projects: list[dict]) -> dict:
"""Write or refresh project metadata files for all projects with directories on disk."""
written = 0
errors = []
for project in projects:
try:
project_dir = project_output_dir(project)
if not project_dir.exists():
continue
sep_job = jobs_repo.get(project["sep_job_id"]) or {}
files = sep_job.get("files", [])
stem_files = {
file_entry["stem"]: file_entry["path"]
for file_entry in files
if file_entry.get("type") == "stem" and file_entry.get("stem")
}
source_file = next(
(file_entry["path"] for file_entry in files if file_entry.get("type") == "source"),
"",
)
result = sep_job.get("result") or {}
write_project_metadata(
project_dir,
project_id=project["id"],
name=project["name"],
model=result.get("model_label", ""),
model_label=result.get("model_label", ""),
format=result.get("format_label", ""),
format_label=result.get("format_label", ""),
source_file=source_file,
stems=stem_files,
sep_job_id=project["sep_job_id"],
trans_jobs=project.get("trans_jobs") or {},
abc_jobs=project.get("abc_jobs") or {},
lrc_jobs=project.get("lrc_jobs") or {},
mix_job_id=project.get("mix_job_id"),
created_at=project.get("created_at"),
)
written += 1
except Exception as exc:
errors.append({"project": project.get("name", project["id"]), "error": str(exc)})
return {"written": written, "errors": errors}
def cleanup_rename_project_stems(projects: list[dict]) -> dict:
"""Rename tracked stem files into the canonical stem filename format."""
from .separator_service import _stem_filename
renamed_total = 0
errors = []
for project in projects:
sep_job = jobs_repo.get(project["sep_job_id"])
if not sep_job:
continue
files = list(sep_job.get("files", []))
changed = False
for index, file_entry in enumerate(files):
if file_entry.get("type") != "stem" or not file_entry.get("stem"):
continue
old_path = Path(file_entry["path"])
if not old_path.exists():
continue
new_filename = _stem_filename(project["name"], file_entry["stem"], old_path.suffix)
new_path = old_path.parent / new_filename
if old_path == new_path:
continue
try:
old_path.rename(new_path)
files[index] = {**file_entry, "name": new_filename, "path": str(new_path)}
changed = True
renamed_total += 1
except Exception as exc:
errors.append(
{
"project": project["name"],
"file": str(old_path),
"error": str(exc),
}
)
if changed:
jobs_repo.update(project["sep_job_id"], files=files)
return {"renamed": renamed_total, "errors": errors}
def purge_orphaned_separation_jobs(all_jobs: list[dict]) -> dict:
"""Delete completed-or-stale separation jobs that are no longer attached to a project."""
projects = projects_repo.list_persisted()
project_job_ids = {project["sep_job_id"] for project in projects if project.get("sep_job_id")}
purged = 0
for job in all_jobs:
if job.get("job_type") == "separate" and job["id"] not in project_job_ids:
if jobs_repo.delete(job["id"]):
purged += 1
return {"purged": purged}
def cleanup_lrc_output_files(all_jobs: list[dict], cleaner) -> dict:
"""Strip section-marker lines from tracked LRC/ELRC output files."""
files_modified = 0
lines_removed = 0
errors = []
for job in all_jobs:
if job.get("job_type") != "lrc":
continue
for file_entry in (job.get("files") or []):
file_type = file_entry.get("type", "")
if file_type not in ("elrc", "lrc"):
continue
file_path = file_entry.get("path")
if not file_path:
continue
path = Path(file_path)
if not path.exists():
continue
try:
content = path.read_text(encoding="utf-8")
cleaned, removed = cleaner(content)
if removed > 0:
path.write_text(cleaned, encoding="utf-8")
files_modified += 1
lines_removed += removed
except Exception as exc:
errors.append({"file": file_path, "error": str(exc)})
return {
"files_modified": files_modified,
"lines_removed": lines_removed,
"errors": errors,
}
def verify_all_project_stems() -> dict:
return verify_project_stems(projects_repo.list_all())
def write_all_project_metadata_snapshots() -> dict:
return write_project_metadata_snapshots(projects_repo.list_all())
def cleanup_rename_all_project_stems() -> dict:
return cleanup_rename_project_stems(projects_repo.list_all())
def purge_orphaned_separation_jobs_from_store() -> dict:
return purge_orphaned_separation_jobs(jobs_repo.list_all())
def cleanup_all_lrc_output_files() -> dict:
from .lrc_service import clean_lrc_content
return cleanup_lrc_output_files(jobs_repo.list_all(), clean_lrc_content)
@@ -0,0 +1,296 @@
from __future__ import annotations
from pathlib import Path
from typing import Optional
from .. import job_manager as jm
from ..config import ALL_MODEL_IDS, STATIC_OPTIONS, TRANSCRIBE_MODES
from ..repositories import jobs as jobs_repo
from ..repositories import projects as projects_repo
from .project_service_common import (
ProjectOperationError,
_find_stem_path,
_infer_output_format_from_paths,
_static_label,
_stem_files_from_sep_job,
project_output_dir,
)
def enqueue_stem_reprocess(project: dict, stem_name: str, model: str, output_format: str = "mp3") -> dict:
"""Queue re-separation of a single project stem with a chosen model."""
from .separator_service import run_separation
if model not in ALL_MODEL_IDS:
raise ProjectOperationError(400, f"Unknown model: {model}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_path = _find_stem_path(_stem_files_from_sep_job(sep_job), stem_name)
if stem_path is None:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
stems_dir = Path(stem_path).parent
job_id = jm.create_job("separate")
jm.submit_gpu(
run_separation,
job_id,
stem_path,
model,
output_format,
preset_type="stems_only",
output_dir_override=stems_dir,
parent_job_id=project["sep_job_id"],
)
return {"job_id": job_id}
def enqueue_project_transcription(project: dict, stem_name: str, mode: str) -> dict:
"""Queue stem transcription for a project and persist the stem→job link."""
valid_modes = {mode_entry["id"] for mode_entry in TRANSCRIBE_MODES}
if mode not in valid_modes:
raise ProjectOperationError(400, f"Unknown transcribe mode: {mode}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_path = _find_stem_path(_stem_files_from_sep_job(sep_job), stem_name)
if stem_path is None:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
from .transcriber_service import run_transcription
job_id = jm.create_job("transcribe")
projects_repo.set_transcription_job(project["id"], stem_name, job_id)
jm.submit_gpu(run_transcription, job_id, stem_path, mode)
return {"job_id": job_id}
def enqueue_project_abc_conversion(project: dict, stem_name: str, song_title: Optional[str]) -> dict:
"""Queue ABC conversion from a completed stem transcription job."""
trans_job_id = (project.get("trans_jobs") or {}).get(stem_name)
if not trans_job_id:
raise ProjectOperationError(400, f"No transcription job for stem '{stem_name}'")
trans_job = jobs_repo.get(trans_job_id)
if trans_job is None:
raise ProjectOperationError(404, "Transcription job not found")
if trans_job.get("status") != "done":
raise ProjectOperationError(400, "Transcription job has not completed successfully")
midi_paths = [
file_entry["path"]
for file_entry in trans_job.get("files", [])
if Path(file_entry.get("path", "")).suffix.lower() == ".mid"
and Path(file_entry.get("path", "")).exists()
]
if not midi_paths:
raise ProjectOperationError(404, "No MIDI files found in transcription job")
from .abc_service import run_abc_conversion
title = (song_title or "").strip() or project["name"]
job_id = jm.create_job("abc-convert")
projects_repo.set_abc_job(project["id"], stem_name, job_id)
jm.submit(run_abc_conversion, job_id, midi_paths, title)
return {"job_id": job_id}
def enqueue_project_mix(
project: dict,
preset_type: str,
static_option: Optional[str],
dynamic_levels: Optional[dict],
output_format: Optional[str],
) -> dict:
"""Queue static or dynamic mix generation for a project."""
if preset_type not in ("static", "dynamic"):
raise ProjectOperationError(400, "preset_type must be 'static' or 'dynamic'")
valid_static_options = {option["id"] for option in STATIC_OPTIONS}
if preset_type == "static" and static_option and static_option not in valid_static_options:
raise ProjectOperationError(400, f"Unknown static_option: {static_option}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_files = _stem_files_from_sep_job(sep_job)
if not stem_files:
raise ProjectOperationError(404, "No stem files found in separation job")
resolved_output_format = output_format or _infer_output_format_from_paths(stem_files)
ext = "mp3" if resolved_output_format.startswith("mp3") else resolved_output_format
project_dir = project_output_dir(project)
if preset_type == "static":
subfolder = "Static"
suffix = _static_label(static_option)
else:
subfolder = "Dynamic"
suffix = "Dynamic Mix"
mix_dir = project_dir / subfolder
mix_dir.mkdir(parents=True, exist_ok=True)
output_path = str(mix_dir / f"{project['name']} - {suffix}.{ext}")
from .mix_service import run_mix
job_id = jm.create_job("mix")
projects_repo.update(project["id"], mix_job_id=job_id)
jm.submit(
run_mix,
job_id,
stem_files,
preset_type,
output_path,
resolved_output_format,
static_option,
{str(key): float(value) for key, value in (dynamic_levels or {}).items()},
)
return {"job_id": job_id}
def enqueue_project_lrc_alignment(
project: dict,
stem_name: str,
lyrics_source: str,
lyrics_text: Optional[str],
output_filename: Optional[str],
style_options: Optional[dict],
timing_precision: str,
lrc_power_preset: str,
) -> dict:
"""Queue lyrics alignment for a project stem and persist lrc stem linkage."""
if lyrics_source not in ("auto", "tag", "text"):
raise ProjectOperationError(400, f"Invalid lyrics_source: {lyrics_source}")
if lyrics_source == "text" and not (lyrics_text or "").strip():
raise ProjectOperationError(400, "lyrics_text required when lyrics_source='text'")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
source_file = next(
(file_entry for file_entry in sep_job.get("files", []) if file_entry.get("type") == "source"),
None,
)
if source_file is None:
raise ProjectOperationError(404, "Source audio file not found in project")
base_name = Path(source_file["name"]).stem
resolved_output_filename = (output_filename or "").strip() or f"{base_name}.elrc"
stem_path = _find_stem_path(_stem_files_from_sep_job(sep_job), stem_name)
if stem_path and Path(stem_path).exists():
audio_path = stem_path
is_vocals = True
else:
audio_path = source_file["path"]
is_vocals = False
from .lrc_service import run_lrc_alignment
project_dir = project_output_dir(project)
job_id = jm.create_job("lrc_align")
projects_repo.set_lrc_job(project["id"], stem_name, job_id)
jm.submit_transcription(
job_id,
run_lrc_alignment,
job_id,
audio_path,
lyrics_source=lyrics_source,
lyrics_content=lyrics_text,
output_filename=resolved_output_filename,
style_options=style_options,
is_vocals=is_vocals,
output_dir=str(project_dir),
timing_precision=timing_precision,
lrc_power_preset=lrc_power_preset,
)
return {"job_id": job_id}
def enqueue_project_lyric_transcription(
project: dict,
stem_name: str,
model: str,
language: Optional[str],
task: str,
output_formats: list,
initial_prompt: Optional[str],
temperature: float,
condition_on_previous_text: bool,
engine: str,
diarize: bool,
lrc_power_preset: str,
phoneme_align: bool,
lsync_align: bool,
two_pass: bool,
vocal_style: str,
explicit: bool,
lyrics_source: str,
existing_lyrics: Optional[str],
) -> dict:
"""Queue AI lyric transcription and persist project linkage for the output job."""
from .lyric_transcriber_service import WHISPER_MODELS, run_lyric_transcription
if model not in WHISPER_MODELS:
raise ProjectOperationError(400, f"Unknown model: {model}")
if task not in ("transcribe", "translate"):
raise ProjectOperationError(400, f"Invalid task: {task}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_files = _stem_files_from_sep_job(sep_job)
if stem_name not in stem_files:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
stem_path = stem_files[stem_name]
output_dir = project_output_dir(project) / "Transcriptions"
output_dir.mkdir(parents=True, exist_ok=True)
resolved_lyrics = existing_lyrics
if resolved_lyrics is None and lyrics_source != "transcribe":
from .metadata_util import extract_lyrics_from_file
lyrics_data = extract_lyrics_from_file(stem_path)
if lyrics_data.get("plain"):
resolved_lyrics = lyrics_data.get("raw") or lyrics_data["plain"]
else:
raise ProjectOperationError(
422,
f"lyrics_source='{lyrics_source}' requested but no lyrics found in file tags or companion LRC.",
)
job_id = jm.create_job("lyric-transcribe")
projects_repo.set_lrc_job(project["id"], f"{stem_name}_whisper", job_id)
jm.submit_transcription(
job_id,
run_lyric_transcription,
job_id,
stem_path,
model=model,
language=language,
task=task,
output_formats=output_formats,
initial_prompt=initial_prompt,
temperature=temperature,
condition_on_previous_text=condition_on_previous_text,
output_dir=str(output_dir),
engine=engine,
diarize=diarize,
power_preset=lrc_power_preset,
phoneme_align=phoneme_align,
lsync_align=lsync_align,
two_pass=two_pass,
existing_lyrics=resolved_lyrics,
vocal_style=vocal_style,
explicit=explicit,
)
return {"job_id": job_id, "output_dir": str(output_dir)}
+33 -760
View File
@@ -1,760 +1,33 @@
from __future__ import annotations
import os
import shutil
from pathlib import Path
from typing import Optional
from .. import job_manager as jm
from ..config import PROJECTS_DIR, STATIC_OPTIONS, TRANSCRIBE_MODES, UPLOAD_DIR
from ..repositories import jobs as jobs_repo
from ..repositories import projects as projects_repo
from .metadata_util import _read_cover_art, read_metadata
from .project_metadata import write_project_metadata
class ProjectOperationError(Exception):
def __init__(self, status_code: int, detail: str):
super().__init__(detail)
self.status_code = status_code
self.detail = detail
def list_projects() -> list[dict]:
return projects_repo.list_all()
def get_project_or_raise(project_id: str) -> dict:
project = projects_repo.get(project_id)
if project is None:
raise ProjectOperationError(404, f"Project not found: {project_id}")
return project
def project_output_dir(project: dict) -> Path:
"""Return the project's root directory under MEDIA_DIR."""
sep_job = jobs_repo.get(project["sep_job_id"]) or {}
result = sep_job.get("result") or {}
project_path = result.get("project_path")
if project_path and Path(project_path).is_dir():
return Path(project_path)
return PROJECTS_DIR / project["name"]
def delete_project_with_related_jobs(project: dict) -> bool:
"""Delete project files plus the project record and all linked job records."""
project_dir = project_output_dir(project)
if project_dir.exists():
shutil.rmtree(project_dir, ignore_errors=True)
upload_dir = UPLOAD_DIR / project["sep_job_id"]
if upload_dir.exists():
shutil.rmtree(upload_dir, ignore_errors=True)
job_ids_to_delete = collect_related_job_ids(project)
deleted = projects_repo.delete(project["id"])
for job_id in job_ids_to_delete:
jobs_repo.delete(job_id)
return deleted
def collect_related_job_ids(project: dict) -> set[str]:
job_ids = {project["sep_job_id"]}
for job_id in (project.get("trans_jobs") or {}).values():
job_ids.add(job_id)
for job_id in (project.get("abc_jobs") or {}).values():
job_ids.add(job_id)
for job_id in (project.get("lrc_jobs") or {}).values():
job_ids.add(job_id)
if project.get("mix_job_id"):
job_ids.add(project["mix_job_id"])
return job_ids
def build_project_detail(project: dict) -> dict:
"""Expand linked jobs and source metadata for a project detail response."""
trans_jobs_expanded = {
stem: jobs_repo.get(job_id)
for stem, job_id in (project.get("trans_jobs") or {}).items()
}
abc_jobs_expanded = {
stem: jobs_repo.get(job_id)
for stem, job_id in (project.get("abc_jobs") or {}).items()
}
lrc_jobs_expanded = {
stem: jobs_repo.get(job_id)
for stem, job_id in (project.get("lrc_jobs") or {}).items()
}
sep_job = jobs_repo.get(project["sep_job_id"])
mix_job = jobs_repo.get(project.get("mix_job_id")) if project.get("mix_job_id") else None
source_artist = ""
source_title = ""
source_file = next(
(file_entry for file_entry in (sep_job or {}).get("files", []) if file_entry.get("type") == "source"),
None,
)
if source_file:
src_path = source_file.get("path", "")
if src_path and Path(src_path).exists():
try:
meta = read_metadata(src_path)
source_artist = meta.get("artist") or meta.get("albumartist") or ""
source_title = meta.get("title") or ""
except Exception:
pass
return {
**project,
"sep_job": sep_job,
"trans_jobs": trans_jobs_expanded,
"abc_jobs": abc_jobs_expanded,
"lrc_jobs": lrc_jobs_expanded,
"mix_job": mix_job,
"source_artist": source_artist,
"source_title": source_title,
}
def rename_project_stem(project: dict, stem_name: str, new_name: str) -> dict:
"""Rename a tracked stem on disk and update the separation job record."""
from .separator_service import _stem_filename
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_files = list(sep_job.get("files", []))
stem_file = None
stem_idx = None
for index, file_entry in enumerate(stem_files):
if file_entry.get("stem") == stem_name or file_entry.get("name") == stem_name:
stem_file = file_entry
stem_idx = index
break
if stem_file is None or stem_idx is None:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
old_path = Path(stem_file["path"])
if not old_path.exists():
raise ProjectOperationError(404, f"Stem file not found on disk: {old_path}")
cleaned_name = new_name.strip()
if not cleaned_name:
raise ProjectOperationError(400, "New name cannot be empty")
new_filename = _stem_filename(project["name"], cleaned_name, old_path.suffix)
new_path = old_path.parent / new_filename
if new_path.exists() and new_path != old_path:
raise ProjectOperationError(409, f"A file with name '{new_filename}' already exists")
try:
os.rename(str(old_path), str(new_path))
except Exception as exc:
raise ProjectOperationError(500, f"Failed to rename stem: {exc}") from exc
stem_files[stem_idx] = {
**stem_file,
"name": new_filename,
"stem": cleaned_name,
"path": str(new_path),
}
jobs_repo.update(project["sep_job_id"], files=stem_files)
return {
"success": True,
"old_name": stem_name,
"new_name": cleaned_name,
"new_path": str(new_path),
}
def rescan_project_stems(project: dict, audio_exts: set[str]) -> dict:
"""Register untracked audio files under the project's Stems directory."""
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stems_dir = project_output_dir(project) / "Stems"
if not stems_dir.exists():
raise ProjectOperationError(404, "Stems directory not found")
files = list(sep_job.get("files", []))
tracked_paths = {file_entry["path"] for file_entry in files}
added = []
for audio_file in sorted(stems_dir.rglob("*")):
if not audio_file.is_file():
continue
if audio_file.suffix.lower() not in audio_exts:
continue
if str(audio_file) in tracked_paths:
continue
entry = {
"name": audio_file.name,
"path": str(audio_file),
"type": "stem",
"stem": _infer_stem_name(audio_file.stem, project["name"]),
}
files.append(entry)
tracked_paths.add(str(audio_file))
added.append(entry)
if added:
jobs_repo.update(project["sep_job_id"], files=files)
return {"added": len(added), "stems": [entry["stem"] for entry in added]}
def verify_project_stems(projects: list[dict]) -> dict:
"""Report projects whose tracked stem files are missing on disk."""
results = []
for project in projects:
sep_job = jobs_repo.get(project["sep_job_id"]) or {}
files = sep_job.get("files", [])
missing = [
file_entry["path"]
for file_entry in files
if file_entry.get("type") == "stem" and not Path(file_entry.get("path", "")).exists()
]
if missing:
results.append(
{
"project_id": project["id"],
"name": project["name"],
"missing_files": missing,
}
)
return {"projects_with_missing_files": results, "total": len(results)}
def write_project_metadata_snapshots(projects: list[dict]) -> dict:
"""Write or refresh project metadata files for all projects with directories on disk."""
written = 0
errors = []
for project in projects:
try:
project_dir = project_output_dir(project)
if not project_dir.exists():
continue
sep_job = jobs_repo.get(project["sep_job_id"]) or {}
files = sep_job.get("files", [])
stem_files = {
file_entry["stem"]: file_entry["path"]
for file_entry in files
if file_entry.get("type") == "stem" and file_entry.get("stem")
}
source_file = next(
(file_entry["path"] for file_entry in files if file_entry.get("type") == "source"),
"",
)
result = sep_job.get("result") or {}
write_project_metadata(
project_dir,
project_id=project["id"],
name=project["name"],
model=result.get("model_label", ""),
model_label=result.get("model_label", ""),
format=result.get("format_label", ""),
format_label=result.get("format_label", ""),
source_file=source_file,
stems=stem_files,
sep_job_id=project["sep_job_id"],
trans_jobs=project.get("trans_jobs") or {},
abc_jobs=project.get("abc_jobs") or {},
lrc_jobs=project.get("lrc_jobs") or {},
mix_job_id=project.get("mix_job_id"),
created_at=project.get("created_at"),
)
written += 1
except Exception as exc:
errors.append({"project": project.get("name", project["id"]), "error": str(exc)})
return {"written": written, "errors": errors}
def cleanup_rename_project_stems(projects: list[dict]) -> dict:
"""Rename tracked stem files into the canonical stem filename format."""
from .separator_service import _stem_filename
renamed_total = 0
errors = []
for project in projects:
sep_job = jobs_repo.get(project["sep_job_id"])
if not sep_job:
continue
files = list(sep_job.get("files", []))
changed = False
for index, file_entry in enumerate(files):
if file_entry.get("type") != "stem" or not file_entry.get("stem"):
continue
old_path = Path(file_entry["path"])
if not old_path.exists():
continue
new_filename = _stem_filename(project["name"], file_entry["stem"], old_path.suffix)
new_path = old_path.parent / new_filename
if old_path == new_path:
continue
try:
old_path.rename(new_path)
files[index] = {**file_entry, "name": new_filename, "path": str(new_path)}
changed = True
renamed_total += 1
except Exception as exc:
errors.append(
{
"project": project["name"],
"file": str(old_path),
"error": str(exc),
}
)
if changed:
jobs_repo.update(project["sep_job_id"], files=files)
return {"renamed": renamed_total, "errors": errors}
def purge_orphaned_separation_jobs(all_jobs: list[dict]) -> dict:
"""Delete completed-or-stale separation jobs that are no longer attached to a project."""
projects = projects_repo.list_persisted()
project_job_ids = {project["sep_job_id"] for project in projects if project.get("sep_job_id")}
purged = 0
for job in all_jobs:
if job.get("job_type") == "separate" and job["id"] not in project_job_ids:
if jobs_repo.delete(job["id"]):
purged += 1
return {"purged": purged}
def cleanup_lrc_output_files(all_jobs: list[dict], cleaner) -> dict:
"""Strip section-marker lines from tracked LRC/ELRC output files."""
files_modified = 0
lines_removed = 0
errors = []
for job in all_jobs:
if job.get("job_type") != "lrc":
continue
for file_entry in (job.get("files") or []):
file_type = file_entry.get("type", "")
if file_type not in ("elrc", "lrc"):
continue
file_path = file_entry.get("path")
if not file_path:
continue
path = Path(file_path)
if not path.exists():
continue
try:
content = path.read_text(encoding="utf-8")
cleaned, removed = cleaner(content)
if removed > 0:
path.write_text(cleaned, encoding="utf-8")
files_modified += 1
lines_removed += removed
except Exception as exc:
errors.append({"file": file_path, "error": str(exc)})
return {
"files_modified": files_modified,
"lines_removed": lines_removed,
"errors": errors,
}
def get_project_cover_art(project: dict) -> tuple[bytes, str]:
"""Return embedded cover-art bytes and mime type from a project's source file."""
sep_job_data = jobs_repo.get(project["sep_job_id"]) or {}
source_file = next(
(file_entry for file_entry in sep_job_data.get("files", []) if file_entry.get("type") == "source"),
None,
)
if not source_file:
raise ProjectOperationError(404, "No source file")
source_path = source_file.get("path", "")
if not source_path or not Path(source_path).exists():
raise ProjectOperationError(404, "Source file not found")
data, mime = _read_cover_art(source_path)
if not data:
raise ProjectOperationError(404, "No cover art")
return data, mime or "image/jpeg"
def verify_all_project_stems() -> dict:
return verify_project_stems(projects_repo.list_all())
def write_all_project_metadata_snapshots() -> dict:
return write_project_metadata_snapshots(projects_repo.list_all())
def cleanup_rename_all_project_stems() -> dict:
return cleanup_rename_project_stems(projects_repo.list_all())
def purge_orphaned_separation_jobs_from_store() -> dict:
return purge_orphaned_separation_jobs(jobs_repo.list_all())
def cleanup_all_lrc_output_files() -> dict:
from .lrc_service import clean_lrc_content
return cleanup_lrc_output_files(jobs_repo.list_all(), clean_lrc_content)
def enqueue_stem_reprocess(project: dict, stem_name: str, model: str, output_format: str = "mp3") -> dict:
"""Queue re-separation of a single project stem with a chosen model."""
from ..config import ALL_MODEL_IDS
from .separator_service import run_separation
if model not in ALL_MODEL_IDS:
raise ProjectOperationError(400, f"Unknown model: {model}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_path = _find_stem_path(_stem_files_from_sep_job(sep_job), stem_name)
if stem_path is None:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
stems_dir = Path(stem_path).parent
job_id = jm.create_job("separate")
jm.submit_gpu(
run_separation,
job_id,
stem_path,
model,
output_format,
preset_type="stems_only",
output_dir_override=stems_dir,
parent_job_id=project["sep_job_id"],
)
return {"job_id": job_id}
def enqueue_project_transcription(project: dict, stem_name: str, mode: str) -> dict:
"""Queue stem transcription for a project and persist the stem→job link."""
valid_modes = {mode_entry["id"] for mode_entry in TRANSCRIBE_MODES}
if mode not in valid_modes:
raise ProjectOperationError(400, f"Unknown transcribe mode: {mode}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_path = _find_stem_path(_stem_files_from_sep_job(sep_job), stem_name)
if stem_path is None:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
from .transcriber_service import run_transcription
job_id = jm.create_job("transcribe")
projects_repo.set_transcription_job(project["id"], stem_name, job_id)
jm.submit_gpu(run_transcription, job_id, stem_path, mode)
return {"job_id": job_id}
def enqueue_project_abc_conversion(project: dict, stem_name: str, song_title: Optional[str]) -> dict:
"""Queue ABC conversion from a completed stem transcription job."""
trans_job_id = (project.get("trans_jobs") or {}).get(stem_name)
if not trans_job_id:
raise ProjectOperationError(400, f"No transcription job for stem '{stem_name}'")
trans_job = jobs_repo.get(trans_job_id)
if trans_job is None:
raise ProjectOperationError(404, "Transcription job not found")
if trans_job.get("status") != "done":
raise ProjectOperationError(400, "Transcription job has not completed successfully")
midi_paths = [
file_entry["path"]
for file_entry in trans_job.get("files", [])
if Path(file_entry.get("path", "")).suffix.lower() == ".mid"
and Path(file_entry.get("path", "")).exists()
]
if not midi_paths:
raise ProjectOperationError(404, "No MIDI files found in transcription job")
from .abc_service import run_abc_conversion
title = (song_title or "").strip() or project["name"]
job_id = jm.create_job("abc-convert")
projects_repo.set_abc_job(project["id"], stem_name, job_id)
jm.submit(run_abc_conversion, job_id, midi_paths, title)
return {"job_id": job_id}
def enqueue_project_mix(
project: dict,
preset_type: str,
static_option: Optional[str],
dynamic_levels: Optional[dict],
output_format: Optional[str],
) -> dict:
"""Queue static or dynamic mix generation for a project."""
if preset_type not in ("static", "dynamic"):
raise ProjectOperationError(400, "preset_type must be 'static' or 'dynamic'")
valid_static_options = {option["id"] for option in STATIC_OPTIONS}
if preset_type == "static" and static_option and static_option not in valid_static_options:
raise ProjectOperationError(400, f"Unknown static_option: {static_option}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_files = _stem_files_from_sep_job(sep_job)
if not stem_files:
raise ProjectOperationError(404, "No stem files found in separation job")
resolved_output_format = output_format or _infer_output_format_from_paths(stem_files)
ext = "mp3" if resolved_output_format.startswith("mp3") else resolved_output_format
project_dir = project_output_dir(project)
if preset_type == "static":
subfolder = "Static"
suffix = _static_label(static_option)
else:
subfolder = "Dynamic"
suffix = "Dynamic Mix"
mix_dir = project_dir / subfolder
mix_dir.mkdir(parents=True, exist_ok=True)
output_path = str(mix_dir / f"{project['name']} - {suffix}.{ext}")
from .mix_service import run_mix
job_id = jm.create_job("mix")
projects_repo.update(project["id"], mix_job_id=job_id)
jm.submit(
run_mix,
job_id,
stem_files,
preset_type,
output_path,
resolved_output_format,
static_option,
{str(key): float(value) for key, value in (dynamic_levels or {}).items()},
)
return {"job_id": job_id}
def enqueue_project_lrc_alignment(
project: dict,
stem_name: str,
lyrics_source: str,
lyrics_text: Optional[str],
output_filename: Optional[str],
style_options: Optional[dict],
timing_precision: str,
lrc_power_preset: str,
) -> dict:
"""Queue lyrics alignment for a project stem and persist lrc stem linkage."""
if lyrics_source not in ("auto", "tag", "text"):
raise ProjectOperationError(400, f"Invalid lyrics_source: {lyrics_source}")
if lyrics_source == "text" and not (lyrics_text or "").strip():
raise ProjectOperationError(400, "lyrics_text required when lyrics_source='text'")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
source_file = next(
(file_entry for file_entry in sep_job.get("files", []) if file_entry.get("type") == "source"),
None,
)
if source_file is None:
raise ProjectOperationError(404, "Source audio file not found in project")
base_name = Path(source_file["name"]).stem
resolved_output_filename = (output_filename or "").strip() or f"{base_name}.elrc"
stem_path = _find_stem_path(_stem_files_from_sep_job(sep_job), stem_name)
if stem_path and Path(stem_path).exists():
audio_path = stem_path
is_vocals = True
else:
audio_path = source_file["path"]
is_vocals = False
from .lrc_service import run_lrc_alignment
project_dir = project_output_dir(project)
job_id = jm.create_job("lrc_align")
projects_repo.set_lrc_job(project["id"], stem_name, job_id)
jm.submit_transcription(
job_id,
run_lrc_alignment,
job_id,
audio_path,
lyrics_source=lyrics_source,
lyrics_content=lyrics_text,
output_filename=resolved_output_filename,
style_options=style_options,
is_vocals=is_vocals,
output_dir=str(project_dir),
timing_precision=timing_precision,
lrc_power_preset=lrc_power_preset,
)
return {"job_id": job_id}
def enqueue_project_lyric_transcription(
project: dict,
stem_name: str,
model: str,
language: Optional[str],
task: str,
output_formats: list,
initial_prompt: Optional[str],
temperature: float,
condition_on_previous_text: bool,
engine: str,
diarize: bool,
lrc_power_preset: str,
phoneme_align: bool,
lsync_align: bool,
two_pass: bool,
vocal_style: str,
explicit: bool,
lyrics_source: str,
existing_lyrics: Optional[str],
) -> dict:
"""Queue AI lyric transcription and persist project linkage for the output job."""
from .lyric_transcriber_service import WHISPER_MODELS, run_lyric_transcription
if model not in WHISPER_MODELS:
raise ProjectOperationError(400, f"Unknown model: {model}")
if task not in ("transcribe", "translate"):
raise ProjectOperationError(400, f"Invalid task: {task}")
sep_job = jobs_repo.get(project["sep_job_id"])
if sep_job is None:
raise ProjectOperationError(404, "Separation job not found")
stem_files = _stem_files_from_sep_job(sep_job)
if stem_name not in stem_files:
raise ProjectOperationError(404, f"Stem not found: {stem_name}")
stem_path = stem_files[stem_name]
output_dir = project_output_dir(project) / "Transcriptions"
output_dir.mkdir(parents=True, exist_ok=True)
resolved_lyrics = existing_lyrics
if resolved_lyrics is None and lyrics_source != "transcribe":
from .metadata_util import extract_lyrics_from_file
lyrics_data = extract_lyrics_from_file(stem_path)
if lyrics_data.get("plain"):
resolved_lyrics = lyrics_data.get("raw") or lyrics_data["plain"]
else:
raise ProjectOperationError(
422,
f"lyrics_source='{lyrics_source}' requested but no lyrics found in file tags or companion LRC.",
)
job_id = jm.create_job("lyric-transcribe")
projects_repo.set_lrc_job(project["id"], f"{stem_name}_whisper", job_id)
jm.submit_transcription(
job_id,
run_lyric_transcription,
job_id,
stem_path,
model=model,
language=language,
task=task,
output_formats=output_formats,
initial_prompt=initial_prompt,
temperature=temperature,
condition_on_previous_text=condition_on_previous_text,
output_dir=str(output_dir),
engine=engine,
diarize=diarize,
power_preset=lrc_power_preset,
phoneme_align=phoneme_align,
lsync_align=lsync_align,
two_pass=two_pass,
existing_lyrics=resolved_lyrics,
vocal_style=vocal_style,
explicit=explicit,
)
return {"job_id": job_id, "output_dir": str(output_dir)}
def _infer_stem_name(file_stem: str, project_name: str) -> str:
import re
match = re.search(r'\(([^)]+)\)\s*$', file_stem.strip())
if match:
return match.group(1).strip().lower()
trimmed_stem = file_stem.strip()
trimmed_project = project_name.strip()
if trimmed_stem.lower().startswith(trimmed_project.lower()):
remainder = trimmed_stem[len(trimmed_project):].strip(' -_()')
if remainder:
return remainder.lower()
return file_stem.strip().lower()
def _stem_files_from_sep_job(sep_job: dict) -> dict:
return {
file_entry["stem"]: file_entry["path"]
for file_entry in sep_job.get("files", [])
if file_entry.get("type") == "stem"
and file_entry.get("stem")
and Path(file_entry.get("path", "")).exists()
}
def _find_stem_path(stem_files: dict, stem_name: str) -> Optional[str]:
if stem_name in stem_files:
return stem_files[stem_name]
for path in stem_files.values():
path_obj = Path(path)
if path_obj.name == stem_name or path_obj.stem == stem_name:
return path
return None
def _infer_output_format_from_paths(stem_files: dict) -> str:
for path in stem_files.values():
ext = Path(path).suffix.lower().lstrip(".")
if ext in ("mp3", "flac", "wav"):
return ext
return "mp3"
def _static_label(static_option: Optional[str]) -> str:
labels = {
"no_vocals": "No Vocals",
"no_drums": "No Drums",
"no_bass": "No Bass",
"no_vocals_no_bass": "No Vocals No Bass",
"no_drums_no_bass": "No Drums No Bass",
}
return labels.get(static_option or "", "Static Mix")
"""Compatibility facade for project service modules split by concern."""
from .project_assets_service import (
build_project_detail,
collect_related_job_ids,
delete_project_with_related_jobs,
get_project_cover_art,
get_project_or_raise,
list_projects,
rename_project_stem,
rescan_project_stems,
)
from .project_cleanup_service import (
cleanup_all_lrc_output_files,
cleanup_lrc_output_files,
cleanup_rename_all_project_stems,
cleanup_rename_project_stems,
purge_orphaned_separation_jobs,
purge_orphaned_separation_jobs_from_store,
verify_all_project_stems,
verify_project_stems,
write_all_project_metadata_snapshots,
write_project_metadata_snapshots,
)
from .project_reprocess_service import (
enqueue_project_abc_conversion,
enqueue_project_lrc_alignment,
enqueue_project_lyric_transcription,
enqueue_project_mix,
enqueue_project_transcription,
enqueue_stem_reprocess,
)
from .project_service_common import ProjectOperationError, project_output_dir
@@ -0,0 +1,80 @@
from __future__ import annotations
from pathlib import Path
from typing import Optional
from ..config import PROJECTS_DIR
from ..repositories import jobs as jobs_repo
class ProjectOperationError(Exception):
def __init__(self, status_code: int, detail: str):
super().__init__(detail)
self.status_code = status_code
self.detail = detail
def project_output_dir(project: dict) -> Path:
"""Return the project's root directory under MEDIA_DIR."""
sep_job = jobs_repo.get(project["sep_job_id"]) or {}
result = sep_job.get("result") or {}
project_path = result.get("project_path")
if project_path and Path(project_path).is_dir():
return Path(project_path)
return PROJECTS_DIR / project["name"]
def _infer_stem_name(file_stem: str, project_name: str) -> str:
import re
match = re.search(r'\(([^)]+)\)\s*$', file_stem.strip())
if match:
return match.group(1).strip().lower()
trimmed_stem = file_stem.strip()
trimmed_project = project_name.strip()
if trimmed_stem.lower().startswith(trimmed_project.lower()):
remainder = trimmed_stem[len(trimmed_project):].strip(' -_()')
if remainder:
return remainder.lower()
return file_stem.strip().lower()
def _stem_files_from_sep_job(sep_job: dict) -> dict:
return {
file_entry["stem"]: file_entry["path"]
for file_entry in sep_job.get("files", [])
if file_entry.get("type") == "stem"
and file_entry.get("stem")
and Path(file_entry.get("path", "")).exists()
}
def _find_stem_path(stem_files: dict, stem_name: str) -> Optional[str]:
if stem_name in stem_files:
return stem_files[stem_name]
for path in stem_files.values():
path_obj = Path(path)
if path_obj.name == stem_name or path_obj.stem == stem_name:
return path
return None
def _infer_output_format_from_paths(stem_files: dict) -> str:
for path in stem_files.values():
ext = Path(path).suffix.lower().lstrip(".")
if ext in ("mp3", "flac", "wav"):
return ext
return "mp3"
def _static_label(static_option: Optional[str]) -> str:
labels = {
"no_vocals": "No Vocals",
"no_drums": "No Drums",
"no_bass": "No Bass",
"no_vocals_no_bass": "No Vocals No Bass",
"no_drums_no_bass": "No Drums No Bass",
}
return labels.get(static_option or "", "Static Mix")
+1
View File
@@ -0,0 +1 @@
"""Separation submodules for model registry, power presets, and runtime helpers."""
@@ -0,0 +1,65 @@
from __future__ import annotations
from pathlib import Path
from ... import job_manager as jm
def ensure_model_available(job_id: str, model_id: str) -> bool:
"""
Download model weights if they are not already present on disk.
Demucs models are skipped — they self-download through the demucs library.
Returns True if the model is ready, False if download failed (job set to ERROR).
"""
demucs_ids = {"htdemucs", "htdemucs_ft", "mdx_extra_q", "mdx"}
if model_id in demucs_ids:
return True
from ...routers.models import MODEL_DOWNLOAD_INFO, get_model_directory, is_model_downloaded
if model_id not in MODEL_DOWNLOAD_INFO:
return True
if is_model_downloaded(model_id):
return True
info = MODEL_DOWNLOAD_INFO[model_id]
hf_repo = info.get("hf_repo")
if not hf_repo:
return True
jm.update_job(job_id, progress=f"Downloading model weights for '{model_id}'...")
try:
import shutil as _shutil
import tempfile
from huggingface_hub import hf_hub_download
model_dir = get_model_directory(model_id)
model_dir.mkdir(parents=True, exist_ok=True)
for file_path in info["files"]:
dest = model_dir / Path(file_path).name
if dest.exists():
continue
jm.update_job(job_id, progress=f"Downloading {Path(file_path).name}...")
with tempfile.TemporaryDirectory() as tmp:
tmp_path = hf_hub_download(
repo_id=hf_repo,
filename=file_path,
local_dir=tmp,
local_dir_use_symlinks=False,
)
_shutil.copy2(tmp_path, str(dest))
return True
except Exception as exc:
jm.update_job(
job_id,
status=jm.JobStatus.ERROR,
error=f"Failed to download model '{model_id}': {exc}",
progress="Error: model download failed.",
)
return False
@@ -0,0 +1,55 @@
from __future__ import annotations
from ...config import (
ALL_MODELS,
APOLLO_MODELS,
ASPIRATION_MODELS,
BANDIT_MODELS,
BS_ROFORMER_SPECIALIZED_MODELS,
BS_ROFORMER_SW_MODELS,
ENSEMBLE_MODELS,
MDXNET_MODELS,
MEDLEYVOX_MODELS,
MEL_BAND_MODELS,
OUTPUT_FORMATS,
SCNET_MODELS,
SEPACAP_MODELS,
SEPREFORMER_MODELS,
)
MODEL_LABELS = {m["id"]: m["label"] for m in ALL_MODELS}
FORMAT_LABELS = {f["id"]: f["label"] for f in OUTPUT_FORMATS}
BSR_PREFIX = "bs_roformer_"
BSR_STEM_MODE = {
"bs_roformer_4stem": "4stem",
"bs_roformer_5stem_guitar": "5stem_guitar",
"bs_roformer_5stem_piano": "5stem_piano",
"bs_roformer_6stem": "6stem",
}
MBR_PREFIX = "mel_band_roformer_"
MBR_VOCALS_PREFIX = "mel_band_vocals"
MBR_DEREVERB_PREFIX = "mel_band_dereverb"
MBR_KARAOKE_PREFIX = "mel_band_karaoke"
MBR_MODELS = {m["id"] for m in MEL_BAND_MODELS}
MBR_VOCALS_MODELS = {m["id"] for m in MEL_BAND_MODELS if "vocals" in m["id"]}
MBR_DEREVERB_MODELS = {m["id"] for m in MEL_BAND_MODELS if "dereverb" in m["id"]}
MBR_KARAOKE_MODELS = {m["id"] for m in MEL_BAND_MODELS if "karaoke" in m["id"]}
MBR_DEUX_BECRUILY_MODELS = {m["id"] for m in MEL_BAND_MODELS if "deux_becruily" in m["id"]}
BSR_CHORUS_MODELS = {m["id"] for m in BS_ROFORMER_SPECIALIZED_MODELS if "chorus" in m["id"]}
BSR_ANVUEW_MODELS = {m["id"] for m in BS_ROFORMER_SPECIALIZED_MODELS if "anvuew" in m["id"] and "karaoke" not in m["id"]}
BSR_KARAOKE_ANVUEW_MODELS = {m["id"] for m in BS_ROFORMER_SPECIALIZED_MODELS if "karaoke" in m["id"] and "anvuew" in m["id"]}
MDXNET_MODEL_IDS = {m["id"] for m in MDXNET_MODELS}
SCNET_MODEL_IDS = {m["id"] for m in SCNET_MODELS}
BSR_SW_MODEL_IDS = {m["id"] for m in BS_ROFORMER_SW_MODELS}
MEDLEYVOX_MODEL_IDS = {m["id"] for m in MEDLEYVOX_MODELS}
APOLLO_MODEL_IDS = {m["id"] for m in APOLLO_MODELS}
ASPIRATION_MODEL_IDS = {m["id"] for m in ASPIRATION_MODELS}
BANDIT_MODEL_IDS = {m["id"] for m in BANDIT_MODELS}
ENSEMBLE_MODEL_IDS = {m["id"] for m in ENSEMBLE_MODELS}
SEPACAP_MODEL_IDS = {m["id"] for m in SEPACAP_MODELS}
SEPREFORMER_MODEL_IDS = {m["id"] for m in SEPREFORMER_MODELS}
@@ -0,0 +1,33 @@
from __future__ import annotations
from typing import List, Optional
def stem_filename(prefix: str, stem_name: str, suffix: str) -> str:
label = stem_name.lower().replace("_", " ")
return f"{prefix} ({label}){suffix}"
def get_muted_stems(static_option: Optional[str]) -> List[str]:
if not static_option:
return []
option_map = {
"no_vocals": ["vocals"],
"no_drums": ["drums"],
"no_bass": ["bass"],
"no_vocals_no_bass": ["vocals", "bass"],
"no_drums_no_bass": ["drums", "bass"],
}
return option_map.get(static_option, [])
def static_name(static_option: Optional[str]) -> str:
name_map = {
"no_vocals": "No Vocals",
"no_drums": "No Drums",
"no_bass": "No Bass",
"no_vocals_no_bass": "No Vocals No Bass",
"no_drums_no_bass": "No Drums No Bass",
}
return name_map.get(static_option, "Static Mix")
@@ -0,0 +1,31 @@
from __future__ import annotations
from typing import Dict
SEP_POWER_PRESETS: Dict[str, Dict] = {
"eco": {
"roformer": {"batch_size": 1, "num_overlap": 2},
"demucs": {"segment": 5, "overlap": 0.1},
"sepacap": {"chunk_samples": 24_000},
"sepreformer": {"chunk_samples": 24_000},
"scnet": {"chunk_size": 132_300, "num_overlap": 2},
},
"standard": {
"roformer": {"batch_size": None, "num_overlap": None},
"demucs": {"segment": None, "overlap": None},
"sepacap": {"chunk_samples": 48_000},
"sepreformer": {"chunk_samples": 48_000},
"scnet": {"chunk_size": None, "num_overlap": None},
},
"performance": {
"roformer": {"batch_size": 8, "num_overlap": 4},
"demucs": {"segment": 20, "overlap": 0.5},
"sepacap": {"chunk_samples": 96_000},
"sepreformer": {"chunk_samples": 96_000},
"scnet": {"chunk_size": None, "num_overlap": 4},
},
}
def get_power_opts(preset: str) -> Dict:
return SEP_POWER_PRESETS.get(preset, SEP_POWER_PRESETS["standard"])
+38 -184
View File
@@ -14,195 +14,49 @@ from pathlib import Path
from typing import Dict, List, Optional
from .. import job_manager as jm
from ..config import (
PROJECTS_DIR, JOBS_DIR, CPU_SEPARATION, PROJECT_SUBFOLDERS, ALL_MODELS, OUTPUT_FORMATS,
MEL_BAND_MODELS, BS_ROFORMER_SPECIALIZED_MODELS, MDXNET_MODELS,
SCNET_MODELS, MEDLEYVOX_MODELS, ENSEMBLE_MODELS, APOLLO_MODELS, ASPIRATION_MODELS, BANDIT_MODELS,
BS_ROFORMER_SW_MODELS, SEPACAP_MODELS, SEPREFORMER_MODELS
)
from ..config import PROJECTS_DIR, JOBS_DIR, CPU_SEPARATION, PROJECT_SUBFOLDERS
from .metadata_util import copy_metadata, read_metadata
from .project_namer import project_name, make_project_dirs
from .preset_service import make_instrumental, make_static, make_dynamic
from .project_metadata import write_project_metadata
from .separation.model_downloads import ensure_model_available
from .separation.model_registry import (
APOLLO_MODEL_IDS,
ASPIRATION_MODEL_IDS,
BANDIT_MODEL_IDS,
BSR_ANVUEW_MODELS,
BSR_CHORUS_MODELS,
BSR_KARAOKE_ANVUEW_MODELS,
BSR_PREFIX,
BSR_STEM_MODE,
BSR_SW_MODEL_IDS,
ENSEMBLE_MODEL_IDS,
FORMAT_LABELS,
MBR_DEUX_BECRUILY_MODELS,
MBR_DEREVERB_MODELS,
MBR_DEREVERB_PREFIX,
MBR_KARAOKE_MODELS,
MBR_KARAOKE_PREFIX,
MBR_MODELS,
MBR_PREFIX,
MBR_VOCALS_MODELS,
MBR_VOCALS_PREFIX,
MDXNET_MODEL_IDS,
MEDLEYVOX_MODEL_IDS,
MODEL_LABELS,
SCNET_MODEL_IDS,
SEPACAP_MODEL_IDS,
SEPREFORMER_MODEL_IDS,
)
from .separation.output_layout import stem_filename
from .separation.power_presets import get_power_opts
log = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Stem file naming helper
# ---------------------------------------------------------------------------
def _stem_filename(prefix: str, stem_name: str, suffix: str) -> str:
"""
Return the destination filename for a separated stem.
Format: "Artist - Title (vocals).mp3"
"""
label = stem_name.lower().replace('_', ' ')
return f"{prefix} ({label}){suffix}"
# Lookup maps for human-readable labels
_MODEL_LABELS = {m['id']: m['label'] for m in ALL_MODELS}
_FORMAT_LABELS = {f['id']: f['label'] for f in OUTPUT_FORMATS}
# Model ID prefix that identifies BS-RoFormer models
BSR_PREFIX = 'bs_roformer_'
# Map from our model ID suffix → BSRoformerSeparator stem_mode string
BSR_STEM_MODE = {
'bs_roformer_4stem': '4stem',
'bs_roformer_5stem_guitar': '5stem_guitar',
'bs_roformer_5stem_piano': '5stem_piano',
'bs_roformer_6stem': '6stem',
}
# Model ID prefix that identifies MelBand RoFormer models
MBR_PREFIX = 'mel_band_roformer_'
MBR_VOCALS_PREFIX = 'mel_band_vocals'
MBR_DEREVERB_PREFIX = 'mel_band_dereverb'
MBR_KARAOKE_PREFIX = 'mel_band_karaoke'
# MelBand RoFormer model IDs
MBR_MODELS = {m['id'] for m in MEL_BAND_MODELS}
# MelBand specialized model groups
MBR_VOCALS_MODELS = {m['id'] for m in MEL_BAND_MODELS if 'vocals' in m['id']}
MBR_DEREVERB_MODELS = {m['id'] for m in MEL_BAND_MODELS if 'dereverb' in m['id']}
MBR_KARAOKE_MODELS = {m['id'] for m in MEL_BAND_MODELS if 'karaoke' in m['id']}
MBR_DEUX_BECRUILY_MODELS = {m['id'] for m in MEL_BAND_MODELS if 'deux_becruily' in m['id']}
# BS-RoFormer specialized model IDs
BSR_CHORUS_MODELS = {m['id'] for m in BS_ROFORMER_SPECIALIZED_MODELS if 'chorus' in m['id']}
BSR_ANVUEW_MODELS = {m['id'] for m in BS_ROFORMER_SPECIALIZED_MODELS if 'anvuew' in m['id'] and 'karaoke' not in m['id']}
BSR_KARAOKE_ANVUEW_MODELS = {m['id'] for m in BS_ROFORMER_SPECIALIZED_MODELS if 'karaoke' in m['id'] and 'anvuew' in m['id']}
# MDX-Net model IDs
MDXNET_MODEL_IDS = {m['id'] for m in MDXNET_MODELS}
# SCNet XL model IDs
SCNET_MODEL_IDS = {m['id'] for m in SCNET_MODELS}
# BS-RoFormer SW model IDs
BSR_SW_MODEL_IDS = {m['id'] for m in BS_ROFORMER_SW_MODELS}
# MedleyVox model IDs
MEDLEYVOX_MODEL_IDS = {m['id'] for m in MEDLEYVOX_MODELS}
# Apollo model IDs
APOLLO_MODEL_IDS = {m['id'] for m in APOLLO_MODELS}
# Aspiration model IDs
ASPIRATION_MODEL_IDS = {m['id'] for m in ASPIRATION_MODELS}
# Bandit model IDs
BANDIT_MODEL_IDS = {m['id'] for m in BANDIT_MODELS}
# Ensemble model IDs
ENSEMBLE_MODEL_IDS = {m['id'] for m in ENSEMBLE_MODELS}
# SepACap model IDs
SEPACAP_MODEL_IDS = {m['id'] for m in SEPACAP_MODELS}
# SepReformer model IDs
SEPREFORMER_MODEL_IDS = {m['id'] for m in SEPREFORMER_MODELS}
# ---------------------------------------------------------------------------
# Separation power presets
# ---------------------------------------------------------------------------
# Each preset maps to:
# roformer: batch_size and num_overlap overrides (None = use model config default)
# demucs: --segment and --overlap CLI flags (None = omit flag, use demucs default)
_SEP_POWER_PRESETS: Dict[str, Dict] = {
'eco': {
'roformer': {'batch_size': 1, 'num_overlap': 2},
'demucs': {'segment': 5, 'overlap': 0.1},
'sepacap': {'chunk_samples': 24_000}, # 1 s @ 24 kHz
'sepreformer': {'chunk_samples': 24_000}, # 3 s @ 8 kHz
'scnet': {'chunk_size': 132_300, 'num_overlap': 2}, # ~3 s @ 44.1 kHz
},
'standard': {
'roformer': {'batch_size': None, 'num_overlap': None},
'demucs': {'segment': None, 'overlap': None},
'sepacap': {'chunk_samples': 48_000}, # 2 s @ 24 kHz
'sepreformer': {'chunk_samples': 48_000}, # 6 s @ 8 kHz
'scnet': {'chunk_size': None, 'num_overlap': None}, # model config defaults
},
'performance': {
'roformer': {'batch_size': 8, 'num_overlap': 4},
'demucs': {'segment': 20, 'overlap': 0.5},
'sepacap': {'chunk_samples': 96_000}, # 4 s @ 24 kHz
'sepreformer': {'chunk_samples': 96_000}, # 12 s @ 8 kHz
'scnet': {'chunk_size': None, 'num_overlap': 4}, # larger overlap for quality
},
}
def _get_power_opts(preset: str) -> Dict:
"""Resolve a power preset name to its options dict."""
return _SEP_POWER_PRESETS.get(preset, _SEP_POWER_PRESETS['standard'])
def _ensure_model_available(job_id: str, model_id: str) -> bool:
"""
Download model weights if they are not already present on disk.
Demucs models are skipped — they self-download through the demucs library.
Returns True if the model is ready, False if download failed (job set to ERROR).
"""
# Demucs downloads itself via the CLI; no action needed
demucs_ids = {'htdemucs', 'htdemucs_ft', 'mdx_extra_q', 'mdx'}
if model_id in demucs_ids:
return True
from ..routers.models import MODEL_DOWNLOAD_INFO, get_model_directory, is_model_downloaded
if model_id not in MODEL_DOWNLOAD_INFO:
return True # Not in registry — assume bundled / self-managed
if is_model_downloaded(model_id):
return True
info = MODEL_DOWNLOAD_INFO[model_id]
hf_repo = info.get("hf_repo")
if not hf_repo:
# No public download URL — the individual separator service will attempt its
# own download (snapshot_download) when it runs. Don't fail the job here.
return True
jm.update_job(job_id, progress=f"Downloading model weights for '{model_id}'...")
try:
import shutil as _shutil
import tempfile
from huggingface_hub import hf_hub_download
model_dir = get_model_directory(model_id)
model_dir.mkdir(parents=True, exist_ok=True)
for file_path in info["files"]:
dest = model_dir / Path(file_path).name
if dest.exists():
continue
jm.update_job(job_id, progress=f"Downloading {Path(file_path).name}...")
with tempfile.TemporaryDirectory() as tmp:
tmp_path = hf_hub_download(
repo_id=hf_repo,
filename=file_path,
local_dir=tmp,
local_dir_use_symlinks=False,
)
_shutil.copy2(tmp_path, str(dest))
return True
except Exception as exc:
jm.update_job(
job_id,
status=jm.JobStatus.ERROR,
error=f"Failed to download model '{model_id}': {exc}",
progress="Error: model download failed.",
)
return False
"""Compatibility wrapper around shared `stem_filename()` helper."""
return stem_filename(prefix, stem_name, suffix)
def run_separation(
@@ -240,12 +94,12 @@ def run_separation(
metadata = read_metadata(input_path)
# Ensure model weights are present — download automatically if missing
if not _ensure_model_available(job_id, model):
if not ensure_model_available(job_id, model):
return
# Resolve human-readable labels for tab naming
model_label = _MODEL_LABELS.get(model, model)
format_label = _FORMAT_LABELS.get(output_format, output_format)
model_label = MODEL_LABELS.get(model, model)
format_label = FORMAT_LABELS.get(output_format, output_format)
tab_label = f"{model_label}_{format_label}"
# Determine project name
@@ -297,7 +151,7 @@ def run_separation(
normalised_format = 'mp3' if output_format == 'mp3_320' else output_format
# Resolve power preset options
_power = _get_power_opts(sep_power_preset)
_power = get_power_opts(sep_power_preset)
_roformer_opts = _power['roformer']
_demucs_opts = _power['demucs']
+2 -2
View File
@@ -19,8 +19,8 @@
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="apple-mobile-web-app-title" content="StemEngine" />
<link rel="apple-touch-icon" href="/icon.svg" />
<script type="module" crossorigin src="/assets/index-D3f7N2-z.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-B8Si_0xu.css">
<script type="module" crossorigin src="/assets/index-D7qgrvOb.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-7HDOX98b.css">
</head>
<body>
<div id="root"></div>
+2
View File
@@ -23,6 +23,7 @@ class BackendTestCase(unittest.TestCase):
"LRC_SYNC_DIR",
"SECRET_KEY",
"HF_TOKEN",
"ADMIN_TOKEN",
]
self._saved_env = {key: os.environ.get(key) for key in self._env_keys}
@@ -34,6 +35,7 @@ class BackendTestCase(unittest.TestCase):
os.environ["LRC_SYNC_DIR"] = str(media_dir / "lrc_sync")
os.environ["SECRET_KEY"] = "test-secret-key"
os.environ.pop("HF_TOKEN", None)
os.environ.pop("ADMIN_TOKEN", None)
for key, value in self.env_overrides.items():
if value is None:
os.environ.pop(key, None)
@@ -0,0 +1,58 @@
from backend.tests.support import BackendTestCase
class OperationalHardeningTests(BackendTestCase):
def test_system_health_endpoint(self):
response = self.client.get("/api/system/health")
self.assertEqual(response.status_code, 200)
payload = response.json()
self.assertEqual(payload["status"], "ok")
self.assertEqual(payload["service"], "stemengine")
def test_system_readiness_endpoint(self):
response = self.client.get("/api/system/readiness")
self.assertEqual(response.status_code, 200)
payload = response.json()
self.assertIn(payload["status"], {"ready", "not_ready"})
self.assertIn("checks", payload)
self.assertIn("db", payload["checks"])
self.assertIn("directories", payload["checks"])
def test_executor_visibility_endpoint(self):
response = self.client.get("/api/system/executors")
self.assertEqual(response.status_code, 200)
payload = response.json()
self.assertIn("general", payload)
self.assertIn("gpu", payload)
self.assertIn("transcription", payload)
class AdminTokenProtectionTests(BackendTestCase):
env_overrides = {"ADMIN_TOKEN": "test-admin-token"}
def test_gpu_reset_requires_admin_token(self):
denied = self.client.post("/api/system/gpu-reset")
self.assertEqual(denied.status_code, 403)
allowed = self.client.post("/api/system/gpu-reset", headers={"X-Admin-Token": "test-admin-token"})
self.assertEqual(allowed.status_code, 200)
def test_cleanup_requires_admin_token(self):
denied = self.client.get("/api/projects/cleanup/verify")
self.assertEqual(denied.status_code, 403)
allowed = self.client.get("/api/projects/cleanup/verify", headers={"X-Admin-Token": "test-admin-token"})
self.assertEqual(allowed.status_code, 200)
def test_models_mutation_requires_admin_token(self):
body = {"model_id": "htdemucs"}
denied = self.client.post("/api/models/download", json=body)
self.assertEqual(denied.status_code, 403)
allowed = self.client.post(
"/api/models/download",
json=body,
headers={"X-Admin-Token": "test-admin-token"},
)
self.assertEqual(allowed.status_code, 200)
@@ -0,0 +1,45 @@
from backend.tests.support import BackendTestCase
class Phase5NormalizationTests(BackendTestCase):
def test_set_transcription_job_writes_normalized_link(self):
from backend.db import ProjectStemJobLink, SessionLocal
from backend.repositories import jobs as jobs_repo
from backend.repositories import projects as projects_repo
sep_job_id = jobs_repo.create("separate")
project = projects_repo.find_or_create("Phase5 Project", sep_job_id)
ok = projects_repo.set_transcription_job(project["id"], "vocals", "job-trans-1")
self.assertTrue(ok)
with SessionLocal() as session:
link = (
session.query(ProjectStemJobLink)
.filter_by(project_id=project["id"], stage="transcribe", stem_name="vocals")
.first()
)
self.assertIsNotNone(link)
self.assertEqual(link.job_id, "job-trans-1")
def test_project_get_prefers_normalized_maps(self):
from sqlalchemy import text
from backend.db import SessionLocal
from backend.repositories import jobs as jobs_repo
from backend.repositories import projects as projects_repo
sep_job_id = jobs_repo.create("separate")
project = projects_repo.find_or_create("Phase5 Project 2", sep_job_id)
projects_repo.set_abc_job(project["id"], "guitar", "job-abc-1")
# Simulate stale JSON map while normalized link remains current.
with SessionLocal() as session:
session.execute(
text("UPDATE projects SET abc_jobs = :payload WHERE id = :project_id"),
{"payload": "{}", "project_id": project["id"]},
)
session.commit()
loaded = projects_repo.get(project["id"])
self.assertEqual(loaded["abc_jobs"].get("guitar"), "job-abc-1")
+43 -10
View File
@@ -9,6 +9,23 @@ set -e
FRONTEND=/app/frontend
BACKEND=/app/backend
# Runtime knobs (preserve previous behavior by default)
STEMENGINE_MODE="${STEMENGINE_MODE:-dev}"
SKIP_NPM_INSTALL="${SKIP_NPM_INSTALL:-0}"
FRONTEND_BUILD="${FRONTEND_BUILD:-1}"
FRONTEND_WATCH="${FRONTEND_WATCH:-1}"
UVICORN_RELOAD="${UVICORN_RELOAD:-1}"
UVICORN_LOG_LEVEL="${UVICORN_LOG_LEVEL:-debug}"
if [[ "$STEMENGINE_MODE" == "test" || "$STEMENGINE_MODE" == "runtime-test" ]]; then
# Resource-safe defaults for runtime checks/tests.
SKIP_NPM_INSTALL="1"
FRONTEND_BUILD="0"
FRONTEND_WATCH="0"
UVICORN_RELOAD="0"
UVICORN_LOG_LEVEL="info"
fi
echo "========================================="
echo " StemEngine startup"
echo "========================================="
@@ -18,21 +35,37 @@ cd "$FRONTEND"
# Install / sync packages. --prefer-offline uses the local cache first so
# subsequent starts are fast; falls back to the registry if needed.
echo "[frontend] Syncing npm packages..."
npm install --prefer-offline --no-audit --no-fund 2>&1 | grep -v "^npm warn" || true
if [[ "$SKIP_NPM_INSTALL" == "1" ]]; then
echo "[frontend] Skipping npm install (SKIP_NPM_INSTALL=1)"
else
echo "[frontend] Syncing npm packages..."
npm install --prefer-offline --no-audit --no-fund 2>&1 | grep -v "^npm warn" || true
fi
# Initial full build so backend/static is populated before uvicorn starts.
# Subsequent hot-rebuilds are handled by --watch below.
echo "[frontend] Building..."
npx vite build 2>&1
if [[ "$FRONTEND_BUILD" == "1" ]]; then
echo "[frontend] Building..."
npx vite build 2>&1
else
echo "[frontend] Skipping initial build (FRONTEND_BUILD=0)"
fi
echo "[frontend] Starting watch mode (rebuilds on .tsx/.ts changes)..."
# CHOKIDAR_USEPOLLING=1: WSL2→Docker bind-mounts don't propagate inotify events,
# so chokidar falls back to stat-based polling to detect host-side edits.
CHOKIDAR_USEPOLLING=1 CHOKIDAR_INTERVAL=800 npx vite build --watch 2>&1 &
if [[ "$FRONTEND_WATCH" == "1" ]]; then
echo "[frontend] Starting watch mode (rebuilds on .tsx/.ts changes)..."
# CHOKIDAR_USEPOLLING=1: WSL2→Docker bind-mounts don't propagate inotify events,
# so chokidar falls back to stat-based polling to detect host-side edits.
CHOKIDAR_USEPOLLING=1 CHOKIDAR_INTERVAL=800 npx vite build --watch 2>&1 &
else
echo "[frontend] Watch mode disabled (FRONTEND_WATCH=0)"
fi
# ── Backend ───────────────────────────────────────────────────────────────────
cd /app
echo "[backend] Starting uvicorn..."
exec python3.11 -m uvicorn backend.main:app \
--host 0.0.0.0 --port 8000 --log-level debug --reload
UVICORN_ARGS=(--host 0.0.0.0 --port 8000 --log-level "$UVICORN_LOG_LEVEL")
if [[ "$UVICORN_RELOAD" == "1" ]]; then
UVICORN_ARGS+=(--reload)
fi
exec python3.11 -m uvicorn backend.main:app "${UVICORN_ARGS[@]}"