Myriad changes.
This commit is contained in:
1 parent
d1bcd7b082
commit
e11f46527b
30 files changed
+4535
-51
No files matched your search
@@ -8,3 +8,16 @@ MUON_IFRAME_URL=http://work:3011
|
||||
# Pinned Muon fork commit/tag/branch used by muon-iframe build context.
|
||||
# Examples: main, v0.1.0, 1a2b3c4d
|
||||
MUON_GIT_REF=main
|
||||
|
||||
# Optional: PSARC -> sloppak converter paths inside container.
|
||||
# Defaults are usually fine.
|
||||
SLOPPAK_SOURCE_ROOT=/app/media/uploads
|
||||
SLOPPAK_OUTPUT_ROOT=/app/media/sloppak
|
||||
|
||||
# Optional: command template used to unpack PSARC when python patcher module
|
||||
# is not present. Must include both placeholders:
|
||||
# {input} -> absolute source .psarc path
|
||||
# {output} -> absolute extraction directory path
|
||||
# Example (replace with your actual unpacker executable):
|
||||
# SLOPPAK_PSARC_UNPACK_CMD=/usr/local/bin/psarc unpack {input} {output}
|
||||
SLOPPAK_PSARC_UNPACK_CMD=
|
||||
@@ -1,5 +1,691 @@
|
||||
# 2026-03-13 Backend Fixes / Improvement Plan (Copilot)
|
||||
|
||||
## 2026-07-09 Follow-up — implemented Slices F, A, and B (MIDI editor + sloppak converter queue)
|
||||
|
||||
Status: implemented and validated (backend compile + frontend typecheck).
|
||||
|
||||
Completed:
|
||||
|
||||
- Implemented Slice F backend API and service:
|
||||
- `backend/routers/midi_editor.py`
|
||||
- `backend/services/midi_editor_service.py`
|
||||
- Session lifecycle: create/apply/save/discard.
|
||||
- Save behavior writes edited sibling MIDI artifact and preserves original source MIDI.
|
||||
- Save emits a downloadable done job (`job_type: midi-edit-save`).
|
||||
|
||||
- Implemented Slice F frontend editor:
|
||||
- `frontend/src/components/MidiLightEditor.tsx`
|
||||
- Integrated into `frontend/src/pages/ProjectDetail.tsx` per MIDI stem row.
|
||||
- Supports note edits, bend edits, quantize, save, and download of saved artifact.
|
||||
|
||||
- Implemented Slice A/B converter backend and controls:
|
||||
- `backend/routers/sloppak_converter.py`
|
||||
- `backend/services/sloppak_converter_service.py`
|
||||
- `backend/main.py` router registration
|
||||
- Endpoints for enqueue/enqueue-bulk/list/pause/resume/cancel-queued/retry-failed/clear-finished.
|
||||
- Skip reasons and dedupe behavior included.
|
||||
|
||||
- Implemented Slice A/B frontend trigger and queue panel:
|
||||
- `frontend/src/components/SloppakConverterPanel.tsx`
|
||||
- Integrated into `frontend/src/pages/ProjectDetail.tsx`
|
||||
- UI supports discrete and batch enqueue plus queue controls and download links.
|
||||
|
||||
- Added frontend client/type surface for new APIs:
|
||||
- `frontend/src/api/client.ts`
|
||||
- `frontend/src/types.ts` (`sloppak_convert`, `midi-edit-save` job types)
|
||||
|
||||
Implementation note:
|
||||
|
||||
- Current sloppak conversion path is an operational packaging baseline that produces `.sloppak` zip artifacts containing `manifest.json` + source `.psarc` payload. Advanced Rocksmith-specific extraction/normalization remains a follow-on enhancement.
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- Backend:
|
||||
- `python3 -m compileall backend/main.py backend/routers/sloppak_converter.py backend/routers/midi_editor.py backend/services/sloppak_converter_service.py backend/services/midi_editor_service.py` (pass)
|
||||
- Frontend:
|
||||
- `cd frontend && npm run -s typecheck` (pass)
|
||||
|
||||
## 2026-07-09 Follow-up — deeper Rocksmith-aware extraction/normalization added to sloppak converter
|
||||
|
||||
Status: implemented and validated.
|
||||
|
||||
Completed:
|
||||
|
||||
- Enhanced converter implementation in:
|
||||
- `backend/services/sloppak_converter_service.py`
|
||||
- Added optional PSARC unpack support:
|
||||
- primary: `patcher.unpack_psarc` (when module exists)
|
||||
- fallback: env-provided unpack command template (`SLOPPAK_PSARC_UNPACK_CMD`)
|
||||
- Added normalization pass over unpacked payload:
|
||||
- song metadata extraction (`title`, `artist`, `album`, `year`)
|
||||
- arrangement extraction/normalization (`id`, `name`, `tuning`, `capo`)
|
||||
- WEM file inventory
|
||||
- vocals XML lyrics extraction to normalized token records
|
||||
- Enriched sloppak package contents:
|
||||
- `manifest.json` now includes rocksmith unpack status + normalized metadata
|
||||
- `metadata/rocksmith-summary.json` included
|
||||
- `lyrics.json` included when extractable
|
||||
|
||||
Compatibility behavior:
|
||||
|
||||
- If no unpacker is available, conversion still succeeds with baseline source packaging and explicit unpack-failure metadata in manifest.
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- Backend compile:
|
||||
- `python3 -m compileall backend/services/sloppak_converter_service.py` (pass)
|
||||
|
||||
## 2026-07-09 Follow-up — unpacker capability endpoint + deployment docs
|
||||
|
||||
Status: implemented and validated.
|
||||
|
||||
Completed:
|
||||
|
||||
- Added converter capability endpoint:
|
||||
- `GET /api/sloppak/capabilities`
|
||||
- implemented via:
|
||||
- `backend/routers/sloppak_converter.py`
|
||||
- `backend/services/sloppak_converter_service.py` (`get_converter_capabilities`)
|
||||
- Endpoint reports active unpacker path availability and root path readiness.
|
||||
|
||||
- Added deployment/env wiring for converter:
|
||||
- `stemengine.yml`
|
||||
- `SLOPPAK_SOURCE_ROOT`
|
||||
- `SLOPPAK_OUTPUT_ROOT`
|
||||
- `SLOPPAK_PSARC_UNPACK_CMD`
|
||||
- `.env.example` entries with concrete command-template example.
|
||||
|
||||
- Added operator docs in `README.md`:
|
||||
- deep extraction unpacker resolution order
|
||||
- command template requirements for `SLOPPAK_PSARC_UNPACK_CMD`
|
||||
- `/api/sloppak/capabilities` verification guidance
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- `python3 -m compileall backend/routers/sloppak_converter.py backend/services/sloppak_converter_service.py` (pass)
|
||||
|
||||
## 2026-07-09 Follow-up — implemented Slices C, D, and E (bend policy/derivation/playback)
|
||||
|
||||
Status: implemented and validated.
|
||||
|
||||
Completed:
|
||||
|
||||
- Slice C (basic-pitch auto bend policy + metadata):
|
||||
- `backend/services/transcriber_service.py`
|
||||
- `backend/routers/transcribe.py`
|
||||
- `backend/routers/projects.py`
|
||||
- `backend/services/project_reprocess_service.py`
|
||||
- Added `bend_policy` support (`auto|force_on|force_off`) and adaptive auto behavior.
|
||||
- Added `result.bend_stats` metadata including confidence summary, stability, and bend counts.
|
||||
|
||||
- Slice D (Omnizart bend derivation + safeguards):
|
||||
- `backend/services/omnizart_service.py`
|
||||
- Added post-process bend derivation when native bends are absent.
|
||||
- Auto mode derives for `omnizart-vocal-contour`; other modes remain opt-in via `force_on`.
|
||||
- Added dense-polyphony guardrails and bounded bend range.
|
||||
- Added transparent derivation metadata in `result.bend_stats`.
|
||||
|
||||
- Slice E (MIDI player bend playback + UI controls):
|
||||
- `frontend/src/pages/ProjectDetail.tsx`
|
||||
- Added pitch-bend event parsing/scheduling during MIDI playback.
|
||||
- Added playback controls: bend on/off toggle (default on) and intensity slider (0-200%).
|
||||
- Added bend count visibility in playback status text.
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- Backend compile:
|
||||
- `python3 -m compileall backend/services/transcriber_service.py backend/services/omnizart_service.py backend/routers/transcribe.py backend/routers/projects.py backend/services/project_reprocess_service.py` (pass)
|
||||
- Frontend typecheck:
|
||||
- `cd frontend && npm run -s typecheck` (pass)
|
||||
|
||||
## 2026-07-09 Follow-up — dedicated conversions page added + plan remainder checklist
|
||||
|
||||
Status: implemented and documented.
|
||||
|
||||
Completed:
|
||||
|
||||
- Added dedicated conversions page (requested):
|
||||
- `frontend/src/pages/Conversions.tsx`
|
||||
- route constant: `frontend/src/routes.ts` (`/conversions`)
|
||||
- route wiring: `frontend/src/App.tsx`
|
||||
- sidebar navigation entry: `frontend/src/components/Sidebar.tsx`
|
||||
|
||||
- Updated implementation plan with explicit remaining-work checklist:
|
||||
- `docs/plans/2026-07-09_StemEngine_PSARC_SlopPak_And_PitchBend_Integration_Plan.md`
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- Frontend typecheck:
|
||||
- `cd frontend && npm run -s typecheck` (pass)
|
||||
|
||||
## 2026-07-09 Follow-up — completed remaining plan work (Phase 3 + Slice G closure)
|
||||
|
||||
Status: implemented and validated.
|
||||
|
||||
Completed:
|
||||
|
||||
- Phase 3 efficiency features implemented for sloppak converter:
|
||||
- queue hydration/recovery across restart:
|
||||
- `backend/services/sloppak_converter_service.py` (`ensure_queue_hydrated`)
|
||||
- startup hook in `backend/main.py`
|
||||
- threaded bulk preflight checks for large enqueue lists:
|
||||
- `backend/services/sloppak_converter_service.py` (`ThreadPoolExecutor` path in `enqueue_bulk`)
|
||||
- snapshot/update stream for low-latency UI updates:
|
||||
- `backend/routers/sloppak_converter.py`
|
||||
- `GET /api/sloppak/snapshot`
|
||||
- `GET /api/sloppak/updates` (SSE)
|
||||
- `POST /api/sloppak/hydrate`
|
||||
- frontend wiring:
|
||||
- `frontend/src/api/client.ts`
|
||||
- `frontend/src/components/SloppakConverterPanel.tsx` (EventSource subscription)
|
||||
|
||||
- Slice G evidence captured with runtime regression sample:
|
||||
- executed in running `stemengine` container using `python3.11`
|
||||
- sample artifact paths:
|
||||
- source: `/app/media/jobs/midi_regression_evidence/regression_source.mid`
|
||||
- edited: `/app/media/jobs/midi_regression_evidence/regression_source.edited.mid`
|
||||
- verified:
|
||||
- `source_unchanged_after_save=true`
|
||||
- `source_unchanged_after_discard=true`
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- Backend compile:
|
||||
- `python3 -m compileall backend/services/sloppak_converter_service.py backend/routers/sloppak_converter.py backend/main.py` (pass)
|
||||
- Frontend typecheck:
|
||||
- `cd frontend && npm run -s typecheck` (pass)
|
||||
- Runtime container smoke checks:
|
||||
- `docker exec stemengine python3.11 -c "... ensure_queue_hydrated/get_snapshot ..."` (pass)
|
||||
- `docker exec stemengine python3.11 -c "... stream_updates ..."` emitted `retry: 1500` and `event: snapshot`
|
||||
|
||||
Cross-worktree sync (`/home/translator/docker/stemengine_master`):
|
||||
|
||||
- Mirrored all touched backend/frontend source files from `appdata/stemengine` into `stemengine_master/src`.
|
||||
- Validation in mirror tree:
|
||||
- `python3 -m compileall backend/services/sloppak_converter_service.py backend/routers/sloppak_converter.py backend/main.py` (pass)
|
||||
- `cd frontend && npm ci && npm run -s typecheck` (pass)
|
||||
|
||||
## 2026-07-09 Follow-up — PSARC/sloppak + pitch bend integration plan
|
||||
|
||||
Status: analysis and implementation plan documented; implementation pending.
|
||||
|
||||
Completed:
|
||||
|
||||
- Compared StemEngine current capabilities against Slopsmith engineup plugin features for:
|
||||
- PSARC to sloppak conversion workflows (single + batch)
|
||||
- converter queue orchestration and efficiency patterns
|
||||
- note detection/pitch bend strategy touchpoints
|
||||
- Confirmed StemEngine baseline gaps:
|
||||
- no native PSARC/sloppak conversion endpoints yet
|
||||
- no explicit frontend MIDI pitch bend playback path yet
|
||||
- Authored new phased plan document:
|
||||
- `docs/plans/2026-07-09_StemEngine_PSARC_SlopPak_And_PitchBend_Integration_Plan.md`
|
||||
|
||||
Plan scope summary:
|
||||
|
||||
- New converter backend/router endpoints for discrete and batch conversion.
|
||||
- Queue controls (pause/resume/cancel/retry/clear) and dedupe/skip-reason semantics.
|
||||
- Auto pitch bend policy for basic-pitch and contour-derived bend support for Omnizart.
|
||||
- Frontend MIDI player pitch bend scheduling and playback toggle.
|
||||
- Lightweight in-app MIDI editor with edit sessions and save/discard persistence flow.
|
||||
|
||||
Scope delta (added after initial draft):
|
||||
|
||||
- Added explicit planning for a lightweight MIDI editor inside StemEngine.
|
||||
- Defined v1 edit operations (note and bend edits, velocity, coarse quantize).
|
||||
- Added session-based backend API contract for safe apply/save/discard semantics.
|
||||
- Added artifact persistence rule: keep source MIDI immutable, write edited sibling output.
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- Source references reviewed:
|
||||
- `/mnt/e/Docker/Slopsmith/engineup/slopsmith-plugin-sloppak-converter/*`
|
||||
- `/mnt/e/Docker/Slopsmith/engineup/slopsmith-plugin-midi/*`
|
||||
- `/mnt/e/Docker/Slopsmith/engineup/slopsmith-plugin-notedetect/*`
|
||||
- StemEngine touchpoints reviewed:
|
||||
- `backend/config.py`
|
||||
- `backend/routers/transcribe.py`
|
||||
- `backend/services/transcriber_service.py`
|
||||
- `backend/services/omnizart_service.py`
|
||||
- `frontend/src/pages/ProjectDetail.tsx`
|
||||
|
||||
## 2026-07-09 Follow-up — stemengine image size analysis and reduction plan
|
||||
|
||||
Status: analysis documented; implementation pending.
|
||||
|
||||
Completed:
|
||||
|
||||
- Investigated Docker size reports for `stemengine:latest`.
|
||||
- Captured evidence from:
|
||||
- `docker image ls`
|
||||
- `docker system df -v`
|
||||
- `docker image inspect`
|
||||
- `docker history`
|
||||
- Identified dominant contributors:
|
||||
- heavy Python ML dependency layers (`requirements.txt` and CUDA torch wheels)
|
||||
- large CUDA runtime/system packages
|
||||
- dev tooling (Node + build toolchain) in main runtime image
|
||||
- significant local build cache footprint
|
||||
- Added plan document:
|
||||
- `docs/plans/2026-07-09_StemEngine_Image_Size_Analysis_And_Reduction_Plan.md`
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- `docker system df -v` reported `stemengine:latest` around `30.3GB` and build cache around `18GB`.
|
||||
- `docker history stemengine:latest` showed largest install layers (`8.61GB`, `5.2GB`, `1.96GB`).
|
||||
|
||||
## 2026-07-08 Follow-up — finalized location-independent pathing to ./src (no ./src/stemengine)
|
||||
|
||||
Status: implemented and validated.
|
||||
|
||||
Completed in `/home/translator/docker/stemengine`:
|
||||
|
||||
- Normalized all workspace-level script/runtime defaults from `./src/stemengine` to `./src`:
|
||||
- `scripts/build-webui.ps1`
|
||||
- `scripts/build-backend.ps1`
|
||||
- `scripts/assemble-runtime.ps1`
|
||||
- `scripts/build-all.ps1`
|
||||
- `scripts/build-release.ps1`
|
||||
- `packaging/pyinstaller/launch_backend.py`
|
||||
- `packaging/pyinstaller/stemengine_backend.spec`
|
||||
- `.github/workflows/windows-packaging.yml` (`STEMENGINE_SOURCE_ROOT=${{ github.workspace }}/src`)
|
||||
|
||||
- Normalized dev container stack paths from `./src/stemengine` to `./src`:
|
||||
- `docker-compose.yml`
|
||||
- `docker/Dockerfile.backend.dev`
|
||||
- `docker/Dockerfile.webui.dev`
|
||||
|
||||
- Improved bootstrap scripts for layman-friendly behavior:
|
||||
- `scripts/bootstrap-source.sh`
|
||||
- `scripts/bootstrap-source.ps1`
|
||||
- now clone/update directly into `./src`
|
||||
- no hard requirement for absolute machine path
|
||||
- if requested ref (default `main`) does not exist, scripts keep current branch instead of failing.
|
||||
|
||||
- Updated docs:
|
||||
- `README.md` now documents `./src` as canonical local source root and includes simplified compose/bootstrap guidance.
|
||||
|
||||
Validation:
|
||||
|
||||
- `scripts/bootstrap-source.sh` successfully updated existing `./src` checkout and handled missing `main` ref gracefully.
|
||||
- Verified `./src/frontend/package.json` and `./src/backend/main.py` exist (source ready).
|
||||
|
||||
Notes:
|
||||
|
||||
- This closes the pathing requirement: workspace scripts are now independent of absolute `/home/translator/...` locations and operate relative to workspace `./src`.
|
||||
|
||||
## 2026-07-08 Follow-up — moved source-root defaults to ./src and added dev docker-compose with git.imibimi.xyz clone + Vite hot reload
|
||||
|
||||
Status: implemented.
|
||||
|
||||
Completed in `/home/translator/docker/stemengine`:
|
||||
|
||||
- Updated build script defaults to local source root:
|
||||
- `scripts/build-webui.ps1`
|
||||
- `scripts/build-backend.ps1`
|
||||
- `scripts/assemble-runtime.ps1`
|
||||
- `scripts/build-all.ps1`
|
||||
- `scripts/build-release.ps1`
|
||||
- New default source path: `/home/translator/docker/stemengine/src/stemengine`
|
||||
|
||||
- Added source bootstrap scripts for local checkout from git.imibimi.xyz:
|
||||
- `scripts/bootstrap-source.sh`
|
||||
- `scripts/bootstrap-source.ps1`
|
||||
|
||||
- Added dev compose stack in workspace root:
|
||||
- `docker-compose.yml`
|
||||
- backend build: `docker/Dockerfile.backend.dev`
|
||||
- webui build: `docker/Dockerfile.webui.dev`
|
||||
|
||||
Compose/dev behavior implemented:
|
||||
|
||||
- Image builds clone source from `https://git.imibimi.xyz/stemengine` (override via `STEMENGINE_GIT_URL` + `STEMENGINE_GIT_REF`).
|
||||
- Runtime bind-mount uses `./src/stemengine` so local edits are live in containers.
|
||||
- Backend open dev access with reload:
|
||||
- `uvicorn ... --reload --host 0.0.0.0 --port 8000`
|
||||
- WebUI open dev access + Vite hot reload:
|
||||
- `npm run dev -- --host 0.0.0.0 --port 5173`
|
||||
- Polling-based file watch env enabled for containerized hot reload reliability.
|
||||
|
||||
- Updated docs and ignore rules:
|
||||
- `README.md` source-root and compose quickstart sections updated
|
||||
- `.gitignore` now ignores `src/`, local node_modules, and `.env`
|
||||
|
||||
Validation:
|
||||
|
||||
- Verified script/file set exists under workspace.
|
||||
- Verified README contains updated source-root and compose/hot-reload guidance.
|
||||
|
||||
## 2026-07-08 Follow-up — true one-command release build wrapper added
|
||||
|
||||
Status: implemented.
|
||||
|
||||
Completed in `/home/translator/docker/stemengine`:
|
||||
|
||||
- Added one-command release wrapper script:
|
||||
- `scripts/build-release.ps1`
|
||||
- Runs:
|
||||
- `scripts/build-all.ps1`
|
||||
- `scripts/build-installer.ps1`
|
||||
- Passes through version/product/manufacturer/upgrade-code parameters for MSI build.
|
||||
|
||||
- Updated workspace docs:
|
||||
- `README.md`
|
||||
- Replaced ambiguous "One-shot build" wording with:
|
||||
- `One-command release build: scripts/build-release.ps1`
|
||||
- separate "Two-step equivalent" section.
|
||||
|
||||
Validation:
|
||||
|
||||
- Verified script exists via scripts directory listing.
|
||||
|
||||
Notes:
|
||||
|
||||
- This resolves the wording/UX mismatch where "one-shot" previously required two manual commands.
|
||||
|
||||
## 2026-07-08 Follow-up — strict release-tag validation + layperson release docs
|
||||
|
||||
Status: implemented.
|
||||
|
||||
Completed in this pass under `/home/translator/docker/stemengine`:
|
||||
|
||||
- Tightened CI version policy in `.github/workflows/windows-packaging.yml`:
|
||||
- tag-triggered builds are now strict and fail on malformed tags,
|
||||
- accepted format remains numeric semver-like tags (`vX.Y.Z` or `vX.Y.Z.W`),
|
||||
- non-tag runs retain fallback version `1.0.<run_number>`.
|
||||
|
||||
- Updated workspace docs:
|
||||
- `README.md` now reflects strict tag behavior.
|
||||
- Added layperson guide:
|
||||
- `docs/LAYPERSON_RELEASE_TAGS.md`
|
||||
- Guide explains in plain language what tags are valid, what happens on invalid tags, and why strict failure protects release quality.
|
||||
|
||||
Validation:
|
||||
|
||||
- Verified workflow includes strict `throw` path for malformed tag refs.
|
||||
- Verified README points to layperson doc and describes fallback only for non-tag runs.
|
||||
|
||||
Notes:
|
||||
|
||||
- This fulfills the requested "go ahead" (strict policy) and "document for laypeople" follow-up.
|
||||
|
||||
## 2026-07-08 Follow-up — release-tag-driven installer versioning enabled
|
||||
|
||||
Status: implemented.
|
||||
|
||||
Completed in this pass under `/home/translator/docker/stemengine`:
|
||||
|
||||
- Updated Windows packaging CI to resolve product version from git tags:
|
||||
- `.github/workflows/windows-packaging.yml`
|
||||
- Added trigger on push tags `v*`.
|
||||
- Added `Resolve product version` step:
|
||||
- tag `vX.Y.Z` or `X.Y.Z` -> MSI/ProductVersion `X.Y.Z`
|
||||
- invalid/non-numeric tag fallback -> `1.0.<run_number>`
|
||||
- Installer build now always uses resolved `$env:PRODUCT_VERSION`.
|
||||
- Uploaded artifact names now include resolved version.
|
||||
|
||||
- Updated local workspace documentation:
|
||||
- `README.md`
|
||||
- Added release tag versioning behavior notes.
|
||||
|
||||
Validation:
|
||||
|
||||
- Verified workflow contains:
|
||||
- tag trigger `v*`
|
||||
- resolve-version step
|
||||
- build-installer using `$env:PRODUCT_VERSION`
|
||||
- version-suffixed artifact names.
|
||||
|
||||
Notes:
|
||||
|
||||
- This implements the requested next increment after CI packaging scaffold.
|
||||
|
||||
## 2026-07-08 Follow-up — "next steps" implemented for Windows packaging workspace
|
||||
|
||||
Status: implemented (WiX harvest + ops scripts + CI workflow).
|
||||
|
||||
Completed in this pass under `/home/translator/docker/stemengine`:
|
||||
|
||||
- Wired real WiX runtime harvest and installer build path:
|
||||
- `packaging/wix/Product.wxs` now takes build-time variables (`ProductName`, `ProductManufacturer`, `ProductVersion`, `UpgradeCode`).
|
||||
- Added `packaging/wix/RuntimeFiles.wxs` with WiX v4 harvest rule:
|
||||
- `Files Include="!(bindpath.runtime)\\**"`
|
||||
- Added `scripts/build-installer.ps1` to build MSI with bindpath `runtime=build/runtime/StemEngine`.
|
||||
|
||||
- Added Windows service and firewall operation scripts:
|
||||
- `scripts/windows/install-service.ps1`
|
||||
- `scripts/windows/uninstall-service.ps1`
|
||||
- `scripts/windows/configure-firewall.ps1`
|
||||
- `scripts/windows/remove-firewall.ps1`
|
||||
- Firewall rule is private-profile scoped for LAN mode.
|
||||
|
||||
- Added CI packaging workflow:
|
||||
- `.github/workflows/windows-packaging.yml`
|
||||
- Workflow provisions Node/Python/.NET+WiX, runs build scripts, validates runtime, builds MSI, uploads runtime + installer artifacts.
|
||||
|
||||
- Updated local workspace documentation:
|
||||
- `README.md`
|
||||
- `docs/IMPLEMENTATION_TRACKER.md`
|
||||
- `docs/VALIDATION_CHECKLIST.md`
|
||||
|
||||
Validation:
|
||||
|
||||
- Verified expanded workspace file set via `find` listing.
|
||||
|
||||
Notes:
|
||||
|
||||
- This pass completes the three previously listed next steps.
|
||||
- End-to-end MSI build execution is defined in CI and intended for Windows runners.
|
||||
|
||||
## 2026-07-08 Work Log — Windows x64 executable workspace scaffold implemented under ~/docker/stemengine
|
||||
|
||||
Status: implemented (scaffold + scripts + templates + docs).
|
||||
|
||||
Completed in this pass:
|
||||
|
||||
- Implemented a new standalone packaging workspace under:
|
||||
- `/home/translator/docker/stemengine`
|
||||
|
||||
- Added scope-locked implementation scaffold for local Windows distribution:
|
||||
- backend + standard WebUI only
|
||||
- SingUI explicitly out of scope for this local installer path
|
||||
|
||||
- Added build/runtime orchestration scripts:
|
||||
- `/home/translator/docker/stemengine/scripts/build-webui.ps1`
|
||||
- `/home/translator/docker/stemengine/scripts/build-backend.ps1`
|
||||
- `/home/translator/docker/stemengine/scripts/assemble-runtime.ps1`
|
||||
- `/home/translator/docker/stemengine/scripts/build-all.ps1`
|
||||
- `/home/translator/docker/stemengine/scripts/validate-runtime.ps1`
|
||||
|
||||
- Added packaging templates:
|
||||
- PyInstaller launcher/spec:
|
||||
- `/home/translator/docker/stemengine/packaging/pyinstaller/launch_backend.py`
|
||||
- `/home/translator/docker/stemengine/packaging/pyinstaller/stemengine_backend.spec`
|
||||
- WinSW service template:
|
||||
- `/home/translator/docker/stemengine/packaging/winsw/stemengine-service.xml`
|
||||
- WiX installer template:
|
||||
- `/home/translator/docker/stemengine/packaging/wix/Product.wxs`
|
||||
|
||||
- Added runtime/docs/config support files:
|
||||
- `/home/translator/docker/stemengine/README.md`
|
||||
- `/home/translator/docker/stemengine/config/settings.template.json`
|
||||
- `/home/translator/docker/stemengine/docs/IMPLEMENTATION_TRACKER.md`
|
||||
- `/home/translator/docker/stemengine/docs/VALIDATION_CHECKLIST.md`
|
||||
|
||||
Validation:
|
||||
|
||||
- Structural verification:
|
||||
- `find /home/translator/docker/stemengine -maxdepth 4 -type f | sort`
|
||||
- Python syntax check:
|
||||
- `python3 -m compileall /home/translator/docker/stemengine/packaging/pyinstaller/launch_backend.py`
|
||||
|
||||
Notes:
|
||||
|
||||
- This pass implements the plan as an executable scaffold and delivery workspace.
|
||||
- It does not perform signed installer release publishing or production firewall automation yet (tracked as next phases in the plan).
|
||||
|
||||
## 2026-07-08 Work Log — Integrations separation device selector + Windows x64 installer plan
|
||||
|
||||
Status: implemented (frontend + docs).
|
||||
|
||||
Completed in this pass:
|
||||
|
||||
- Added a dedicated separation device option under Settings -> Integrations:
|
||||
- `frontend/src/pages/Settings.tsx`
|
||||
- New "Separation device" control allows choosing:
|
||||
- `GPU (auto when available)`
|
||||
- `CPU only`
|
||||
- Persists via existing settings API as:
|
||||
- `cpu_separation` (`true` for CPU mode, `false` for GPU-auto)
|
||||
- `hw_profile: custom` to keep profile state consistent with manual override.
|
||||
|
||||
- Created a formal implementation plan for shipping StemEngine as a Windows x64 installable executable with Jellyfin-style network-host behavior:
|
||||
- `docs/plans/2026-07-08_StemEngine_Windows_x64_Installable_Executable_Plan.md`
|
||||
- Plan includes packaging strategy, installer/service model, LAN access model, firewall/auth guardrails, release pipeline, and phase-gated validation.
|
||||
|
||||
Validation:
|
||||
|
||||
- Frontend typecheck:
|
||||
- `cd frontend && npm run -s typecheck` (no type errors reported)
|
||||
|
||||
Notes:
|
||||
|
||||
- Backend runtime behavior for separation device was already wired through `cpu_separation`; this pass adds the requested Integrations-surface control and documentation plan.
|
||||
|
||||
## 2026-07-07 Work Log — ONNX Runtime CUDA alignment + image build aliases
|
||||
|
||||
Status: implemented and runtime-validated.
|
||||
|
||||
Completed in this pass:
|
||||
|
||||
- ONNX Runtime GPU compatibility fix:
|
||||
- `backend/requirements.txt`
|
||||
- Pinned `onnxruntime-gpu==1.20.1` to avoid `libcudart.so.13` runtime linkage in the current CUDA 12.4 stack.
|
||||
|
||||
- Runtime validation in live container:
|
||||
- Confirmed previous failure on `onnxruntime-gpu 1.27.0` (`ImportError: libcudart.so.13`).
|
||||
- Applied compatibility hotfix in running container (`pip install onnxruntime-gpu==1.20.1`).
|
||||
- Verified providers now load successfully:
|
||||
- `['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']`
|
||||
|
||||
- Added discrete/all-at-once image build helpers:
|
||||
- `~/.bashrc`
|
||||
- New functions:
|
||||
- `se-img-build <image>`
|
||||
- `se-img-build-all`
|
||||
- New aliases:
|
||||
- `se-build-image` -> `se-img-build`
|
||||
- `se-build-images` -> `se-img-build-all`
|
||||
- Supported image targets:
|
||||
- `stemengine`
|
||||
- `stemengine-webui`
|
||||
- `stemengine-muon`
|
||||
- `stemengine-fluid`
|
||||
- `stemengine-tendrils`
|
||||
- `stemengine-spirit`
|
||||
- `omnizart-sidecar`
|
||||
- `parakeet-sidecar`
|
||||
|
||||
Notes:
|
||||
|
||||
- Current deployed compose at `/home/translator/docker/stemengine.yml` pulls prebuilt registry images by default.
|
||||
- The runtime hotfix is active now, but full persistence requires rebuilding/publishing the `stemengine` image with the pinned requirement and redeploying from that image.
|
||||
|
||||
### 2026-07-07 Follow-up — one-command local deploy + compose local-build override
|
||||
|
||||
Status: implemented and validated.
|
||||
|
||||
Completed in this pass:
|
||||
|
||||
- Added local-build override compose file:
|
||||
- `/home/translator/docker/stemengine.local.override.yml`
|
||||
- Services overridden to prefer local builds and never pull:
|
||||
- `stemengine`
|
||||
- `stemengine-muon`
|
||||
- `stemengine-fluid`
|
||||
- `stemengine-tendrils`
|
||||
- `stemengine-spirit`
|
||||
- `omnizart-sidecar`
|
||||
- Each service now has:
|
||||
- `pull_policy: never`
|
||||
- explicit `build.context` + `build.dockerfile`
|
||||
|
||||
- Added one-command deploy helper to `~/.bashrc`:
|
||||
- `se-deploy-stemengine-local [tag]`
|
||||
- Behavior:
|
||||
- builds `git.imibimi.xyz/translator/stemengine:latest`
|
||||
- tags image as `git.imibimi.xyz/translator/stemengine:<tag>` (timestamp default)
|
||||
- recreates only `stemengine` with no pull:
|
||||
- `docker compose -f /home/translator/docker/stemengine.yml -f /home/translator/docker/stemengine.local.override.yml up -d --no-deps --force-recreate --pull never stemengine`
|
||||
|
||||
- Added local stack shortcut helpers to `~/.bashrc`:
|
||||
- `se-up-local`
|
||||
- runs full stack bring-up using base + override compose files.
|
||||
- alias: `se-deploy-local` -> `se-deploy-stemengine-local`
|
||||
|
||||
Validation:
|
||||
|
||||
- `docker compose -f /home/translator/docker/stemengine.yml -f /home/translator/docker/stemengine.local.override.yml config` confirms:
|
||||
- local `build` blocks are applied
|
||||
- `pull_policy: never` is applied for overridden services
|
||||
- interactive shell check confirms helpers are loaded:
|
||||
- `type se-deploy-stemengine-local`
|
||||
- `alias se-deploy-local`
|
||||
- `alias se-up-local`
|
||||
|
||||
## 2026-07-07 Work Log — Demucs stuck-at-100% fix + GPU device enforcement
|
||||
|
||||
Status: implemented, deployed, and validated on live backend.
|
||||
|
||||
Completed in this pass:
|
||||
|
||||
- Fixed Demucs post-completion spin loop:
|
||||
- `backend/services/separator_service.py`
|
||||
- In `_run_demucs()`, the stdout polling loop now exits on EOF when the child process has finished.
|
||||
- This resolves the observed state where jobs remained `running` at `100%` indefinitely while `separator_worker` consumed high CPU.
|
||||
|
||||
- Enforced explicit Demucs device selection:
|
||||
- `backend/services/separator_service.py`
|
||||
- `_run_demucs()` now computes `demucs_device` and passes `--device` to the CLI.
|
||||
- Behavior:
|
||||
- `CPU_SEPARATION=true` => force `cpu`
|
||||
- otherwise => use `cuda` when CUDA is available, else fallback `cpu`
|
||||
- Job progress now includes selected device (`device=cuda` or `device=cpu`) for observability.
|
||||
|
||||
- Hardened compose GPU intent for backend service:
|
||||
- `docker/stemengine.yml`
|
||||
- Added `gpus: all` for `stemengine`.
|
||||
- Added explicit env defaults:
|
||||
- `NVIDIA_VISIBLE_DEVICES=${NVIDIA_VISIBLE_DEVICES:-all}`
|
||||
- `NVIDIA_DRIVER_CAPABILITIES=${NVIDIA_DRIVER_CAPABILITIES:-compute,utility}`
|
||||
|
||||
Operational actions performed:
|
||||
|
||||
- Recreated backend service:
|
||||
- `docker compose -f /home/translator/docker/stemengine.yml up -d --force-recreate stemengine`
|
||||
- Restarted backend after code patch:
|
||||
- `docker restart stemengine`
|
||||
- Cleared stale stuck job from queue:
|
||||
- `DELETE /api/jobs/6cafc416-fe31-489f-b5f1-135b3b9a1eb2`
|
||||
|
||||
Validation:
|
||||
|
||||
- Fresh separation request accepted:
|
||||
- `POST /api/separate/from-path` for `/app/media/imports/tUnE-yArDs - Heartbreak.flac`
|
||||
- Job: `1a2b9beb-d79e-4bdf-929d-0db5d8a7a633`
|
||||
- Live job state confirms GPU selection:
|
||||
- `GET /api/jobs/1a2b9beb-d79e-4bdf-929d-0db5d8a7a633`
|
||||
- progress includes: `Long-track tuning: ... device=cuda ...`
|
||||
- Runtime capability checks inside container:
|
||||
- `torch.cuda.is_available() == True`
|
||||
- `torch.cuda.device_count() == 1`
|
||||
- device name: `NVIDIA GeForce RTX 3060`
|
||||
|
||||
Notes:
|
||||
|
||||
- `nvidia-smi` inside container is functional; CUDA compute is available to PyTorch.
|
||||
- ONNX Runtime currently fails import with `libcudart.so.13` missing in this image; this is separate from Demucs and may affect ONNX-model GPU paths until runtime/library versions are aligned.
|
||||
|
||||
## 2026-04-29 Work Log — Per-renderer iframe audit remediation pass
|
||||
|
||||
Implemented audit follow-up items across Spirit runtime, iframe Docker guards, and docs/spec drift.
|
||||
|
||||
@@ -132,6 +132,9 @@ All configuration is via environment variables. The `docker-compose.yml` accepts
|
||||
| `NAVSERVER` | — | Subsonic server URL (can also be set in the UI) |
|
||||
| `NAVUSER` | — | Subsonic username |
|
||||
| `NAVPASS` | — | Subsonic password |
|
||||
| `SLOPPAK_SOURCE_ROOT` | `/app/media/uploads` | Converter source root for relative PSARC paths |
|
||||
| `SLOPPAK_OUTPUT_ROOT` | `/app/media/sloppak` | Converter output root for generated `.sloppak` files |
|
||||
| `SLOPPAK_PSARC_UNPACK_CMD` | empty | Optional PSARC unpack command template (`{input}` and `{output}` placeholders) used for deep Rocksmith metadata extraction when python patcher module is unavailable |
|
||||
|
||||
### Model Directories
|
||||
|
||||
@@ -146,6 +149,31 @@ These are set automatically in `docker-compose.yml` and point inside the `/opt/`
|
||||
Each separation job creates a folder under `PROJECTS_DIR`:
|
||||
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## sloppak Converter Deep Extraction
|
||||
|
||||
The PSARC -> sloppak converter can enrich packages with Rocksmith metadata (arrangements, lyrics, WEM inventory) when a PSARC unpacker is available.
|
||||
|
||||
Unpacker resolution order:
|
||||
|
||||
1. Python `patcher.unpack_psarc` module (if installed)
|
||||
2. `SLOPPAK_PSARC_UNPACK_CMD` command template from environment
|
||||
|
||||
Command template requirements:
|
||||
|
||||
- Must include `{input}` and `{output}` placeholders.
|
||||
- Example:
|
||||
|
||||
```bash
|
||||
SLOPPAK_PSARC_UNPACK_CMD="/usr/local/bin/psarc unpack {input} {output}"
|
||||
```
|
||||
|
||||
Runtime verification endpoint:
|
||||
|
||||
- `GET /api/sloppak/capabilities`
|
||||
- Reports whether a usable unpacker is currently detected.
|
||||
Artist - Title/
|
||||
Artist - Title.mp3 # Original source audio (copy)
|
||||
Artist - Title.elrc # ELRC lyrics with word-level timestamps (if aligned)
|
||||
|
||||
@@ -1,5 +1,412 @@
|
||||
# StemEngine — Session Summaries
|
||||
|
||||
## 2026-07-09
|
||||
|
||||
### Implemented: Slice F then Slices A/B (MIDI editor + sloppak converter)
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Slice F (lightweight MIDI editor) implemented end-to-end:
|
||||
- backend:
|
||||
- `backend/routers/midi_editor.py`
|
||||
- `backend/services/midi_editor_service.py`
|
||||
- frontend:
|
||||
- `frontend/src/components/MidiLightEditor.tsx`
|
||||
- integrated in `frontend/src/pages/ProjectDetail.tsx`
|
||||
- Slice A/B converter implementation added:
|
||||
- backend:
|
||||
- `backend/routers/sloppak_converter.py`
|
||||
- `backend/services/sloppak_converter_service.py`
|
||||
- router wired in `backend/main.py`
|
||||
- frontend:
|
||||
- `frontend/src/components/SloppakConverterPanel.tsx`
|
||||
- integrated in `frontend/src/pages/ProjectDetail.tsx`
|
||||
- API/type updates:
|
||||
- `frontend/src/api/client.ts`
|
||||
- `frontend/src/types.ts`
|
||||
|
||||
Behavior delivered:
|
||||
|
||||
1. Session-based MIDI edits (notes + pitch bends) with save/discard flow.
|
||||
2. Save writes a new sibling MIDI artifact; original MIDI remains unchanged.
|
||||
3. sloppak queue supports single/batch enqueue and operational controls (pause/resume/cancel-queued/retry-failed/clear-finished).
|
||||
|
||||
Validation:
|
||||
|
||||
- Backend compile pass:
|
||||
- `python3 -m compileall backend/main.py backend/routers/sloppak_converter.py backend/routers/midi_editor.py backend/services/sloppak_converter_service.py backend/services/midi_editor_service.py`
|
||||
- Frontend typecheck pass:
|
||||
- `cd frontend && npm run -s typecheck`
|
||||
|
||||
### Follow-up implemented: deeper Rocksmith-aware extraction/normalization
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Upgraded converter packaging logic in:
|
||||
- `backend/services/sloppak_converter_service.py`
|
||||
- Added optional unpack + normalization flow:
|
||||
- tries Python `patcher.unpack_psarc` first,
|
||||
- can use `SLOPPAK_PSARC_UNPACK_CMD` template fallback.
|
||||
- Added normalized extraction outputs when unpack succeeds:
|
||||
- song metadata (title/artist/album/year),
|
||||
- arrangements (id/name/tuning/capo),
|
||||
- WEM inventory,
|
||||
- vocals XML lyrics.
|
||||
- Added these into output package:
|
||||
- enriched `manifest.json` rocksmith sections,
|
||||
- `metadata/rocksmith-summary.json`,
|
||||
- optional `lyrics.json`.
|
||||
|
||||
Validation:
|
||||
|
||||
- `python3 -m compileall backend/services/sloppak_converter_service.py` (pass)
|
||||
|
||||
### Follow-up implemented: unpacker capability endpoint + env docs
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Added `GET /api/sloppak/capabilities` for runtime unpacker health/status.
|
||||
- Added converter env pass-through in `stemengine.yml`:
|
||||
- `SLOPPAK_SOURCE_ROOT`
|
||||
- `SLOPPAK_OUTPUT_ROOT`
|
||||
- `SLOPPAK_PSARC_UNPACK_CMD`
|
||||
- Updated `.env.example` with concrete command-template guidance.
|
||||
- Updated `README.md` with deep extraction setup and capability endpoint verification.
|
||||
|
||||
Validation:
|
||||
|
||||
- `python3 -m compileall backend/routers/sloppak_converter.py backend/services/sloppak_converter_service.py` (pass)
|
||||
|
||||
### Follow-up implemented: Slices C, D, and E (pitch bend pipeline)
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Slice C (basic-pitch):
|
||||
- Added bend policy routing (`auto|force_on|force_off`) through transcription endpoints.
|
||||
- Implemented adaptive auto bend behavior and persisted `result.bend_stats` diagnostics.
|
||||
- Touched:
|
||||
- `backend/services/transcriber_service.py`
|
||||
- `backend/routers/transcribe.py`
|
||||
- `backend/routers/projects.py`
|
||||
- `backend/services/project_reprocess_service.py`
|
||||
|
||||
- Slice D (Omnizart):
|
||||
- Added post-process bend derivation when native bends are missing.
|
||||
- Auto behavior targets `vocal-contour`; other modes are opt-in via `force_on`.
|
||||
- Added dense/polyphonic guardrails and derivation transparency metadata.
|
||||
- Touched:
|
||||
- `backend/services/omnizart_service.py`
|
||||
|
||||
- Slice E (Frontend playback):
|
||||
- Added pitch bend scheduling in MIDI playback.
|
||||
- Added bend playback on/off toggle (default on) and intensity slider.
|
||||
- Added bend event count visibility in playback status.
|
||||
- Touched:
|
||||
- `frontend/src/pages/ProjectDetail.tsx`
|
||||
|
||||
Validation:
|
||||
|
||||
- `python3 -m compileall backend/services/transcriber_service.py backend/services/omnizart_service.py backend/routers/transcribe.py backend/routers/projects.py backend/services/project_reprocess_service.py` (pass)
|
||||
- `cd frontend && npm run -s typecheck` (pass)
|
||||
|
||||
### Follow-up implemented: dedicated conversions page + explicit remaining-work checklist
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Added dedicated conversion page and navigation:
|
||||
- `frontend/src/pages/Conversions.tsx`
|
||||
- `frontend/src/routes.ts` (`/conversions`)
|
||||
- `frontend/src/App.tsx` route wiring
|
||||
- `frontend/src/components/Sidebar.tsx` entry link
|
||||
|
||||
- Updated plan with a clear "Remaining Work Checklist" section to distinguish done vs open items:
|
||||
- `docs/plans/2026-07-09_StemEngine_PSARC_SlopPak_And_PitchBend_Integration_Plan.md`
|
||||
|
||||
Validation:
|
||||
|
||||
- `cd frontend && npm run -s typecheck` (pass)
|
||||
|
||||
### Follow-up implemented: completed remaining plan work + mirrored to stemengine_master
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Phase 3 converter efficiency items closed:
|
||||
- restart hydration (`ensure_queue_hydrated` + startup hook)
|
||||
- threaded bulk preflight checks for large batches
|
||||
- snapshot/update stream endpoints + frontend SSE subscription
|
||||
- Slice G evidence closed with runtime regression artifacts:
|
||||
- source `/app/media/jobs/midi_regression_evidence/regression_source.mid`
|
||||
- edited `/app/media/jobs/midi_regression_evidence/regression_source.edited.mid`
|
||||
- source immutability verified after save/discard
|
||||
|
||||
Mirror sync completed to `/home/translator/docker/stemengine_master/src`:
|
||||
|
||||
- Synced touched backend/frontend files (sloppak service/router/startup, client, panel, routes, app/sidebar, conversions page).
|
||||
|
||||
Validation:
|
||||
|
||||
- appdata backend compile: pass
|
||||
- appdata frontend typecheck: pass
|
||||
- runtime container checks for hydrate/snapshot/stream: pass
|
||||
- stemengine_master backend compile: pass
|
||||
- stemengine_master frontend `npm ci` + typecheck: pass
|
||||
|
||||
### PSARC to sloppak + pitch bend feature integration plan
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Analyzed Slopsmith engineup references for converter and queue behavior:
|
||||
- `slopsmith-plugin-sloppak-converter`
|
||||
- `slopsmith-plugin-midi`
|
||||
- `slopsmith-plugin-notedetect`
|
||||
- Compared against StemEngine backend/frontend capabilities and identified implementation gaps.
|
||||
- Produced a phased architecture and delivery plan in:
|
||||
- `docs/plans/2026-07-09_StemEngine_PSARC_SlopPak_And_PitchBend_Integration_Plan.md`
|
||||
|
||||
Planned feature scope:
|
||||
|
||||
1. Discrete + batch PSARC to sloppak conversion endpoints and queue UX.
|
||||
2. Queue efficiency and resilience patterns (dedupe, reason codes, pause/resume, persistence).
|
||||
3. Automatic pitch bend policy for basic-pitch and contour-derived bends for Omnizart.
|
||||
4. MIDI player pitch bend playback support and user toggle.
|
||||
5. Lightweight MIDI editor (notes + bends) with save/discard persistence workflow.
|
||||
|
||||
Scope update:
|
||||
|
||||
- Plan expanded to include an in-app lightweight MIDI editor that saves user edits as a new MIDI artifact while preserving the original source file.
|
||||
|
||||
Status:
|
||||
|
||||
- Documentation/planning complete.
|
||||
- Code implementation intentionally deferred to the next execution phase.
|
||||
|
||||
### StemEngine image-size investigation and shrink plan
|
||||
|
||||
Completed in `/home/translator/docker/appdata/stemengine`:
|
||||
|
||||
- Investigated why `stemengine:latest` is reported around `30.3GB`.
|
||||
- Confirmed the main image is the dominant local image footprint; additional host pressure includes large Docker build cache.
|
||||
- Reviewed Dockerfile/layer history and dependency manifests to isolate high-impact contributors.
|
||||
- Documented full analysis + phased reduction strategy in:
|
||||
- `docs/plans/2026-07-09_StemEngine_Image_Size_Analysis_And_Reduction_Plan.md`
|
||||
|
||||
Key findings:
|
||||
|
||||
- Largest layers are Python dependency installation and CUDA torch wheel install.
|
||||
- Runtime image currently includes dev/watch tooling and compiler toolchain.
|
||||
- Build cache is a separate but major storage contributor.
|
||||
|
||||
Next implementation phases (planned):
|
||||
|
||||
1. Split dev/runtime Dockerfiles and remove dev-only runtime dependencies.
|
||||
2. Split dependency profiles and image variants (core/full).
|
||||
3. Rationalize CUDA/cuDNN footprint while preserving compatibility constraints.
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### Final path normalization: workspace now uses ./src only
|
||||
|
||||
Follow-up refinement completed in `/home/translator/docker/stemengine`:
|
||||
|
||||
- Removed workspace-level `./src/stemengine` assumptions and standardized on `./src`.
|
||||
- Updated scripts, compose, Dockerfiles, CI source-root env, and PyInstaller defaults to use `./src`.
|
||||
|
||||
Layman-friendly bootstrap improvement:
|
||||
|
||||
- `scripts/bootstrap-source.sh` and `scripts/bootstrap-source.ps1` now:
|
||||
- clone/update directly into `./src`,
|
||||
- fall back gracefully when requested ref (default `main`) is missing,
|
||||
- keep current branch instead of hard-failing.
|
||||
|
||||
Validation:
|
||||
|
||||
- bootstrap run updated `./src` successfully and printed graceful missing-ref fallback.
|
||||
- source readiness verified (`./src/frontend/package.json` and `./src/backend/main.py`).
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### Source-root moved to ./src and dev docker-compose stack implemented
|
||||
|
||||
Implemented in `/home/translator/docker/stemengine`:
|
||||
|
||||
- Packaging/build script defaults now target local source path:
|
||||
- `/home/translator/docker/stemengine/src/stemengine`
|
||||
|
||||
- Added source bootstrap scripts:
|
||||
- `scripts/bootstrap-source.sh`
|
||||
- `scripts/bootstrap-source.ps1`
|
||||
|
||||
- Added root-level dev stack:
|
||||
- `docker-compose.yml`
|
||||
- `docker/Dockerfile.backend.dev`
|
||||
- `docker/Dockerfile.webui.dev`
|
||||
|
||||
Dev stack behavior:
|
||||
|
||||
- image build clones from `https://git.imibimi.xyz/stemengine`,
|
||||
- runtime bind-mount uses `./src/stemengine`,
|
||||
- backend open access + reload on `0.0.0.0:8000`,
|
||||
- Vite open access + hot reload on `0.0.0.0:5173`.
|
||||
|
||||
Docs updated:
|
||||
|
||||
- `README.md` now documents new source root and docker compose quickstart.
|
||||
- `.gitignore` updated for local source/dev artifacts.
|
||||
|
||||
Validation:
|
||||
|
||||
- workspace file listing and README grep checks confirm implementation.
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### One-command release build script added
|
||||
|
||||
Implemented in `/home/translator/docker/stemengine`:
|
||||
|
||||
- Added `scripts/build-release.ps1` as a true one-command wrapper.
|
||||
- Executes runtime build (`build-all.ps1`) then MSI build (`build-installer.ps1`).
|
||||
|
||||
- Updated `README.md` wording:
|
||||
- now documents `build-release.ps1` under "One-command release build"
|
||||
- keeps old two-command flow under "Two-step equivalent"
|
||||
|
||||
Validation:
|
||||
|
||||
- scripts folder listing confirms `build-release.ps1` is present.
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### Strict tag validation enabled and layperson release guide added
|
||||
|
||||
Follow-up completed in `/home/translator/docker/stemengine`:
|
||||
|
||||
- CI behavior changed in `.github/workflows/windows-packaging.yml`:
|
||||
- tag-triggered builds now fail fast on malformed tags,
|
||||
- valid release tags remain `vX.Y.Z` (or `vX.Y.Z.W`),
|
||||
- fallback `1.0.<run_number>` remains for non-tag runs only.
|
||||
|
||||
- Documentation updates:
|
||||
- `README.md` updated to describe strict/fallback split.
|
||||
- New layperson-friendly guide added:
|
||||
- `docs/LAYPERSON_RELEASE_TAGS.md`
|
||||
|
||||
Validation:
|
||||
|
||||
- grep verification confirms strict `throw` path exists and README links to layperson guide.
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### CI installer version now derived from release tags
|
||||
|
||||
Follow-up enhancement in `/home/translator/docker/stemengine`:
|
||||
|
||||
- `.github/workflows/windows-packaging.yml` now supports release-tag versioning.
|
||||
- push tag trigger: `v*`
|
||||
- resolve step maps `vX.Y.Z` -> `X.Y.Z`
|
||||
- fallback for non-numeric tags: `1.0.<run_number>`
|
||||
- MSI build consumes resolved `$env:PRODUCT_VERSION`
|
||||
- artifact names include resolved version suffix
|
||||
|
||||
- `README.md` updated with release-tag versioning behavior.
|
||||
|
||||
Validation:
|
||||
|
||||
- workflow grep checks confirmed tag trigger, version resolver, version-fed installer build, and versioned artifact names.
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### Windows packaging next steps completed (WiX harvest + ops scripts + CI)
|
||||
|
||||
Follow-up implementation completed in `/home/translator/docker/stemengine`:
|
||||
|
||||
- WiX harvest/build wiring:
|
||||
- `packaging/wix/Product.wxs` switched to build-time variables
|
||||
- `packaging/wix/RuntimeFiles.wxs` added with `Files Include="!(bindpath.runtime)\\**"`
|
||||
- `scripts/build-installer.ps1` added to build MSI from assembled runtime payload
|
||||
|
||||
- Windows operational scripts added:
|
||||
- `scripts/windows/install-service.ps1`
|
||||
- `scripts/windows/uninstall-service.ps1`
|
||||
- `scripts/windows/configure-firewall.ps1`
|
||||
- `scripts/windows/remove-firewall.ps1`
|
||||
|
||||
- CI workflow added:
|
||||
- `.github/workflows/windows-packaging.yml`
|
||||
- Builds runtime, validates layout, builds MSI, uploads artifacts.
|
||||
|
||||
- Docs updated:
|
||||
- `README.md`
|
||||
- `docs/IMPLEMENTATION_TRACKER.md`
|
||||
- `docs/VALIDATION_CHECKLIST.md`
|
||||
|
||||
Validation:
|
||||
|
||||
- file inventory verification confirms new workflow/scripts/templates present.
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### Windows x64 executable implementation scaffold under ~/docker/stemengine
|
||||
|
||||
Implemented a concrete plan execution workspace at:
|
||||
|
||||
- `/home/translator/docker/stemengine`
|
||||
|
||||
Delivered artifacts:
|
||||
|
||||
- Build/runtime scripts:
|
||||
- `scripts/build-webui.ps1`
|
||||
- `scripts/build-backend.ps1`
|
||||
- `scripts/assemble-runtime.ps1`
|
||||
- `scripts/build-all.ps1`
|
||||
- `scripts/validate-runtime.ps1`
|
||||
- Packaging templates:
|
||||
- `packaging/pyinstaller/launch_backend.py`
|
||||
- `packaging/pyinstaller/stemengine_backend.spec`
|
||||
- `packaging/winsw/stemengine-service.xml`
|
||||
- `packaging/wix/Product.wxs`
|
||||
- Docs/config:
|
||||
- `README.md`
|
||||
- `config/settings.template.json`
|
||||
- `docs/IMPLEMENTATION_TRACKER.md`
|
||||
- `docs/VALIDATION_CHECKLIST.md`
|
||||
|
||||
Scope lock enforced in scaffold:
|
||||
|
||||
- local Windows distribution targets backend + standard WebUI only,
|
||||
- SingUI is out of scope for this packaging path.
|
||||
|
||||
Validation performed:
|
||||
|
||||
- file structure verification via `find` on workspace,
|
||||
- Python syntax check passed for `launch_backend.py`.
|
||||
|
||||
## 2026-07-08
|
||||
|
||||
### Integrations CPU/GPU separation selector + Windows x64 packaging plan
|
||||
|
||||
Implemented:
|
||||
|
||||
- Added a new "Separation device" control in Settings -> Integrations:
|
||||
- `frontend/src/pages/Settings.tsx`
|
||||
- Option values:
|
||||
- GPU (auto when available)
|
||||
- CPU only
|
||||
- Save action persists through existing settings API by writing:
|
||||
- `cpu_separation`
|
||||
- `hw_profile: custom`
|
||||
|
||||
- Added new planning document for a Jellyfin-style Windows distribution path:
|
||||
- `docs/plans/2026-07-08_StemEngine_Windows_x64_Installable_Executable_Plan.md`
|
||||
- Covers:
|
||||
- installer/runtime architecture,
|
||||
- service + LAN-access model,
|
||||
- firewall/auth/security baseline,
|
||||
- phased implementation and release gates.
|
||||
|
||||
Validation:
|
||||
|
||||
- frontend typecheck passed (`npm run -s typecheck` from `frontend/`).
|
||||
|
||||
## 2026-04-29
|
||||
|
||||
### Per-renderer iframe audit remediation (implementation + docs)
|
||||
|
||||
+14
-1
@@ -41,12 +41,13 @@ from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.responses import FileResponse
|
||||
|
||||
from . import job_manager as jm
|
||||
from .routers import separate, transcribe, abc_convert, pipeline, jobs, lrc, subsonic, projects, models, lyric_transcribe, system, settings, karaoke_sek, pixabay, room, local_library, gloss
|
||||
from .routers import separate, transcribe, abc_convert, pipeline, jobs, lrc, subsonic, projects, models, lyric_transcribe, system, settings, karaoke_sek, pixabay, room, local_library, gloss, sloppak_converter, midi_editor
|
||||
from .config import SUBSONIC_CONFIG_PATH
|
||||
from .app_settings import settings as app_settings
|
||||
from .db import init_db
|
||||
from .services.local_library_service import seed_music_library_path_from_env
|
||||
from .services.gloss_service import initialize_gloss_runtime
|
||||
from .services import sloppak_converter_service
|
||||
|
||||
_log = logging.getLogger(__name__)
|
||||
|
||||
@@ -111,6 +112,16 @@ async def lifespan(app):
|
||||
initialize_gloss_runtime()
|
||||
except Exception:
|
||||
_log.exception("Startup gloss runtime initialization failed")
|
||||
try:
|
||||
hydrate = sloppak_converter_service.ensure_queue_hydrated()
|
||||
if hydrate.get("resubmitted", 0) or hydrate.get("requeued_running", 0):
|
||||
_log.info(
|
||||
"Startup sloppak queue hydrate: resubmitted=%s requeued_running=%s",
|
||||
hydrate.get("resubmitted", 0),
|
||||
hydrate.get("requeued_running", 0),
|
||||
)
|
||||
except Exception:
|
||||
_log.exception("Startup sloppak queue hydration failed")
|
||||
for issue in app_settings.validate_startup():
|
||||
_log.warning("Startup check: %s", issue)
|
||||
yield
|
||||
@@ -134,6 +145,8 @@ def _include_routers(app: FastAPI) -> None:
|
||||
app.include_router(karaoke_sek.router)
|
||||
app.include_router(pixabay.router)
|
||||
app.include_router(room.router)
|
||||
app.include_router(sloppak_converter.router)
|
||||
app.include_router(midi_editor.router)
|
||||
|
||||
|
||||
def _mount_static(app: FastAPI) -> None:
|
||||
|
||||
@@ -40,7 +40,8 @@ hyper-connections>=0.0.10
|
||||
# MDX-Net dependencies (ONNX models from UVR)
|
||||
# onnxruntime-gpu provides CUDAExecutionProvider so both MDX-Net and basic-pitch
|
||||
# automatically run on the GPU when available (conflicts with onnxruntime CPU package).
|
||||
onnxruntime-gpu>=1.16.0
|
||||
# Pin below 1.27 because newer wheels in this stack require libcudart.so.13.
|
||||
onnxruntime-gpu==1.20.1
|
||||
onnx>=1.14.0
|
||||
|
||||
# HuggingFace CLI (for BS-RoFormer model download)
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..services import midi_editor_service as svc
|
||||
|
||||
|
||||
router = APIRouter(prefix="/api/midi/edit", tags=["midi-editor"])
|
||||
|
||||
|
||||
class CreateSessionRequest(BaseModel):
|
||||
job_id: str = Field(..., min_length=1)
|
||||
filename: Optional[str] = None
|
||||
|
||||
|
||||
class ApplyOpsRequest(BaseModel):
|
||||
ops: list[dict[str, Any]] = Field(default_factory=list)
|
||||
|
||||
|
||||
@router.post("/sessions")
|
||||
def create_session(req: CreateSessionRequest):
|
||||
try:
|
||||
return svc.create_session(req.job_id, req.filename)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.get("/sessions/{session_id}")
|
||||
def get_session(session_id: str):
|
||||
try:
|
||||
return svc.get_session(session_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.patch("/sessions/{session_id}")
|
||||
def apply_ops(session_id: str, req: ApplyOpsRequest):
|
||||
try:
|
||||
return svc.apply_ops(session_id, req.ops)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/sessions/{session_id}/save")
|
||||
def save_session(session_id: str):
|
||||
try:
|
||||
return svc.save_session(session_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.delete("/sessions/{session_id}")
|
||||
def discard_session(session_id: str):
|
||||
try:
|
||||
return svc.discard_session(session_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
@@ -73,6 +73,7 @@ def _validate_mp3_bitrate(value: Optional[str]) -> Optional[str]:
|
||||
class TranscribeRequest(BaseModel):
|
||||
stem_name: str # which stem to transcribe (key from sep job)
|
||||
mode: str = DEFAULT_TRANSCRIBE_MODE
|
||||
bend_policy: str = "auto" # auto | force_on | force_off
|
||||
|
||||
|
||||
class AbcRequest(BaseModel):
|
||||
@@ -182,7 +183,7 @@ def transcribe_project(project_id: str, body: TranscribeRequest):
|
||||
"""Transcribe a specific stem from this project to MIDI."""
|
||||
project = _get_project_or_404(project_id)
|
||||
try:
|
||||
return enqueue_project_transcription(project, body.stem_name, body.mode)
|
||||
return enqueue_project_transcription(project, body.stem_name, body.mode, body.bend_policy)
|
||||
except ProjectOperationError as exc:
|
||||
raise HTTPException(status_code=exc.status_code, detail=exc.detail) from exc
|
||||
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from ..services import sloppak_converter_service as svc
|
||||
|
||||
|
||||
router = APIRouter(prefix="/api/sloppak", tags=["sloppak"])
|
||||
|
||||
|
||||
class EnqueueRequest(BaseModel):
|
||||
filename: str = Field(..., min_length=1)
|
||||
reconvert: bool = False
|
||||
|
||||
|
||||
class EnqueueBulkRequest(BaseModel):
|
||||
filenames: list[str] = Field(default_factory=list)
|
||||
reconvert: bool = False
|
||||
|
||||
|
||||
@router.get("/jobs")
|
||||
def list_jobs():
|
||||
return {
|
||||
"paused": svc.is_paused(),
|
||||
"jobs": svc.list_conversion_jobs(),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/snapshot")
|
||||
def snapshot():
|
||||
return svc.get_snapshot()
|
||||
|
||||
|
||||
@router.get("/updates")
|
||||
def updates():
|
||||
return StreamingResponse(
|
||||
svc.stream_updates(),
|
||||
media_type="text/event-stream",
|
||||
headers={"Cache-Control": "no-cache", "Connection": "keep-alive"},
|
||||
)
|
||||
|
||||
|
||||
@router.post("/hydrate")
|
||||
def hydrate():
|
||||
return svc.ensure_queue_hydrated()
|
||||
|
||||
|
||||
@router.get("/capabilities")
|
||||
def capabilities():
|
||||
return svc.get_converter_capabilities()
|
||||
|
||||
|
||||
@router.post("/enqueue")
|
||||
def enqueue(req: EnqueueRequest):
|
||||
return svc.enqueue_conversion(req.filename, reconvert=req.reconvert)
|
||||
|
||||
|
||||
@router.post("/enqueue-bulk")
|
||||
def enqueue_bulk(req: EnqueueBulkRequest):
|
||||
return svc.enqueue_bulk(req.filenames, reconvert=req.reconvert)
|
||||
|
||||
|
||||
@router.post("/pause")
|
||||
def pause_queue():
|
||||
return svc.pause_queue()
|
||||
|
||||
|
||||
@router.post("/resume")
|
||||
def resume_queue():
|
||||
return svc.resume_queue()
|
||||
|
||||
|
||||
@router.post("/jobs/cancel-queued")
|
||||
def cancel_queued():
|
||||
return svc.cancel_queued()
|
||||
|
||||
|
||||
@router.post("/jobs/retry-failed")
|
||||
def retry_failed():
|
||||
return svc.retry_failed()
|
||||
|
||||
|
||||
@router.post("/jobs/clear-finished")
|
||||
def clear_finished():
|
||||
return svc.clear_finished()
|
||||
@@ -12,6 +12,7 @@ from ..services.upload_io import save_upload_file
|
||||
router = APIRouter(prefix="/api/transcribe", tags=["transcribe"])
|
||||
|
||||
VALID_MODES = {m["id"] for m in TRANSCRIBE_MODES}
|
||||
_VALID_BEND_POLICIES = {"auto", "force_on", "force_off"}
|
||||
|
||||
# Preferred stem order for transcription (drums are skipped; vocals last since
|
||||
# Omnizart's "music" mode is most useful on pitched instruments).
|
||||
@@ -70,6 +71,7 @@ def get_modes():
|
||||
async def transcribe(
|
||||
file: UploadFile = File(...),
|
||||
mode: str = Form(DEFAULT_TRANSCRIBE_MODE),
|
||||
bend_policy: str = Form("auto"),
|
||||
_admin_guard: None = Depends(require_admin_token),
|
||||
):
|
||||
if mode not in VALID_MODES:
|
||||
@@ -77,13 +79,16 @@ async def transcribe(
|
||||
if mode.startswith("omnizart-") and not OMNIZART_ENABLED:
|
||||
raise HTTPException(status_code=503, detail="Omnizart is not available on this server (OMNIZART_ENABLED=false).")
|
||||
|
||||
if bend_policy not in _VALID_BEND_POLICIES:
|
||||
raise HTTPException(status_code=400, detail=f"Unknown bend_policy: {bend_policy}")
|
||||
|
||||
job_id = jm.create_job("transcribe")
|
||||
upload_path = UPLOAD_DIR / job_id / _safe_audio_upload_filename(file.filename)
|
||||
upload_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
await save_upload_file(file, upload_path)
|
||||
|
||||
jm.submit_gpu(run_transcription, job_id, str(upload_path), mode)
|
||||
jm.submit_gpu(run_transcription, job_id, str(upload_path), mode, bend_policy)
|
||||
return {"job_id": job_id}
|
||||
|
||||
|
||||
@@ -91,6 +96,7 @@ async def transcribe(
|
||||
async def transcribe_from_project(
|
||||
project_name: str = Form(...),
|
||||
mode: str = Form(DEFAULT_TRANSCRIBE_MODE),
|
||||
bend_policy: str = Form("auto"),
|
||||
_admin_guard: None = Depends(require_admin_token),
|
||||
):
|
||||
"""
|
||||
@@ -101,6 +107,8 @@ async def transcribe_from_project(
|
||||
raise HTTPException(status_code=400, detail=f"Unknown mode: {mode}")
|
||||
if mode.startswith("omnizart-") and not OMNIZART_ENABLED:
|
||||
raise HTTPException(status_code=503, detail="Omnizart is not available on this server (OMNIZART_ENABLED=false).")
|
||||
if bend_policy not in _VALID_BEND_POLICIES:
|
||||
raise HTTPException(status_code=400, detail=f"Unknown bend_policy: {bend_policy}")
|
||||
|
||||
safe_project_name = _safe_project_name(project_name)
|
||||
stems_dir = PROJECTS_DIR / safe_project_name / "Stems"
|
||||
@@ -133,5 +141,5 @@ async def transcribe_from_project(
|
||||
raise HTTPException(status_code=404, detail="No suitable (non-drum) stem found in project")
|
||||
|
||||
job_id = jm.create_job("transcribe")
|
||||
jm.submit_gpu(run_transcription, job_id, str(stem_file), mode)
|
||||
jm.submit_gpu(run_transcription, job_id, str(stem_file), mode, bend_policy)
|
||||
return {"job_id": job_id, "stem": stem_file.name}
|
||||
@@ -0,0 +1,311 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
|
||||
import pretty_midi
|
||||
|
||||
from .. import job_manager as jm
|
||||
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_SESSIONS_LOCK = threading.Lock()
|
||||
_SESSIONS: dict[str, dict[str, Any]] = {}
|
||||
|
||||
|
||||
def _now_iso() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def _clamp(value: float, lo: float, hi: float) -> float:
|
||||
return max(lo, min(hi, value))
|
||||
|
||||
|
||||
def _resolve_midi_path(job_id: str, filename: str | None) -> Path:
|
||||
job = jm.get_job(job_id)
|
||||
if not job:
|
||||
raise ValueError("job not found")
|
||||
|
||||
files = [f for f in (job.get("files") or []) if str(f.get("name", "")).lower().endswith(".mid")]
|
||||
if not files:
|
||||
raise ValueError("no midi file found on job")
|
||||
|
||||
chosen = None
|
||||
if filename:
|
||||
for item in files:
|
||||
if item.get("name") == filename:
|
||||
chosen = item
|
||||
break
|
||||
if not chosen:
|
||||
raise ValueError("midi filename not found on job")
|
||||
else:
|
||||
chosen = files[0]
|
||||
|
||||
path = Path(str(chosen.get("path", "")))
|
||||
if not path.exists() or not path.is_file():
|
||||
raise ValueError("midi file is missing from disk")
|
||||
return path
|
||||
|
||||
|
||||
def create_session(job_id: str, filename: str | None = None) -> dict[str, Any]:
|
||||
source_path = _resolve_midi_path(job_id, filename)
|
||||
midi = pretty_midi.PrettyMIDI(str(source_path))
|
||||
|
||||
tracks: list[dict[str, Any]] = []
|
||||
for idx, instrument in enumerate(midi.instruments):
|
||||
notes = [
|
||||
{
|
||||
"id": uuid4().hex[:10],
|
||||
"pitch": int(note.pitch),
|
||||
"start": float(note.start),
|
||||
"duration": float(max(0.001, note.end - note.start)),
|
||||
"velocity": int(note.velocity),
|
||||
}
|
||||
for note in instrument.notes
|
||||
]
|
||||
bends = [
|
||||
{
|
||||
"id": uuid4().hex[:10],
|
||||
"time": float(bend.time),
|
||||
"value": int(bend.pitch),
|
||||
}
|
||||
for bend in instrument.pitch_bends
|
||||
]
|
||||
tracks.append(
|
||||
{
|
||||
"index": idx,
|
||||
"name": instrument.name or f"Track {idx + 1}",
|
||||
"program": int(instrument.program),
|
||||
"is_drum": bool(instrument.is_drum),
|
||||
"notes": notes,
|
||||
"bends": bends,
|
||||
}
|
||||
)
|
||||
|
||||
session_id = uuid4().hex
|
||||
payload = {
|
||||
"id": session_id,
|
||||
"created_at": _now_iso(),
|
||||
"source_job_id": job_id,
|
||||
"source_filename": source_path.name,
|
||||
"source_path": str(source_path),
|
||||
"tracks": tracks,
|
||||
"editor_version": "stemengine-midi-editor-v1",
|
||||
}
|
||||
|
||||
with _SESSIONS_LOCK:
|
||||
_SESSIONS[session_id] = payload
|
||||
return payload
|
||||
|
||||
|
||||
def get_session(session_id: str) -> dict[str, Any]:
|
||||
with _SESSIONS_LOCK:
|
||||
session = _SESSIONS.get(session_id)
|
||||
if not session:
|
||||
raise ValueError("session not found")
|
||||
return session
|
||||
|
||||
|
||||
def discard_session(session_id: str) -> dict[str, Any]:
|
||||
with _SESSIONS_LOCK:
|
||||
existed = _SESSIONS.pop(session_id, None) is not None
|
||||
if not existed:
|
||||
raise ValueError("session not found")
|
||||
return {"discarded": True}
|
||||
|
||||
|
||||
def apply_ops(session_id: str, ops: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
with _SESSIONS_LOCK:
|
||||
session = _SESSIONS.get(session_id)
|
||||
if not session:
|
||||
raise ValueError("session not found")
|
||||
|
||||
for op in ops:
|
||||
_apply_single_op(session, op)
|
||||
|
||||
return session
|
||||
|
||||
|
||||
def _get_track(session: dict[str, Any], track_index: int) -> dict[str, Any]:
|
||||
tracks: list[dict[str, Any]] = session["tracks"]
|
||||
if track_index < 0 or track_index >= len(tracks):
|
||||
raise ValueError("track_index out of range")
|
||||
return tracks[track_index]
|
||||
|
||||
|
||||
def _apply_single_op(session: dict[str, Any], op: dict[str, Any]) -> None:
|
||||
op_name = str(op.get("op", "")).strip()
|
||||
if not op_name:
|
||||
raise ValueError("op is required")
|
||||
|
||||
track_index = int(op.get("track_index", 0))
|
||||
track = _get_track(session, track_index)
|
||||
|
||||
if op_name == "add_note":
|
||||
start = float(op.get("start", 0.0))
|
||||
duration = float(op.get("duration", 0.5))
|
||||
note = {
|
||||
"id": uuid4().hex[:10],
|
||||
"pitch": int(_clamp(float(op.get("pitch", 60)), 0, 127)),
|
||||
"start": _clamp(start, 0.0, 86400.0),
|
||||
"duration": _clamp(duration, 0.001, 60.0),
|
||||
"velocity": int(_clamp(float(op.get("velocity", 100)), 1, 127)),
|
||||
}
|
||||
track["notes"].append(note)
|
||||
track["notes"].sort(key=lambda x: (x["start"], x["pitch"]))
|
||||
return
|
||||
|
||||
if op_name == "update_note":
|
||||
note_id = str(op.get("note_id", ""))
|
||||
note = next((n for n in track["notes"] if n["id"] == note_id), None)
|
||||
if not note:
|
||||
raise ValueError("note not found")
|
||||
if "pitch" in op:
|
||||
note["pitch"] = int(_clamp(float(op["pitch"]), 0, 127))
|
||||
if "start" in op:
|
||||
note["start"] = _clamp(float(op["start"]), 0.0, 86400.0)
|
||||
if "duration" in op:
|
||||
note["duration"] = _clamp(float(op["duration"]), 0.001, 60.0)
|
||||
if "velocity" in op:
|
||||
note["velocity"] = int(_clamp(float(op["velocity"]), 1, 127))
|
||||
track["notes"].sort(key=lambda x: (x["start"], x["pitch"]))
|
||||
return
|
||||
|
||||
if op_name == "delete_note":
|
||||
note_id = str(op.get("note_id", ""))
|
||||
track["notes"] = [n for n in track["notes"] if n["id"] != note_id]
|
||||
return
|
||||
|
||||
if op_name == "quantize_notes":
|
||||
step = float(op.get("step", 0.25))
|
||||
if step <= 0:
|
||||
raise ValueError("quantize step must be > 0")
|
||||
for note in track["notes"]:
|
||||
note["start"] = round(note["start"] / step) * step
|
||||
note["duration"] = max(step, round(note["duration"] / step) * step)
|
||||
track["notes"].sort(key=lambda x: (x["start"], x["pitch"]))
|
||||
return
|
||||
|
||||
if op_name == "add_bend":
|
||||
bend = {
|
||||
"id": uuid4().hex[:10],
|
||||
"time": _clamp(float(op.get("time", 0.0)), 0.0, 86400.0),
|
||||
"value": int(_clamp(float(op.get("value", 0)), -8192, 8191)),
|
||||
}
|
||||
track["bends"].append(bend)
|
||||
track["bends"].sort(key=lambda x: x["time"])
|
||||
return
|
||||
|
||||
if op_name == "update_bend":
|
||||
bend_id = str(op.get("bend_id", ""))
|
||||
bend = next((b for b in track["bends"] if b["id"] == bend_id), None)
|
||||
if not bend:
|
||||
raise ValueError("bend not found")
|
||||
if "time" in op:
|
||||
bend["time"] = _clamp(float(op["time"]), 0.0, 86400.0)
|
||||
if "value" in op:
|
||||
bend["value"] = int(_clamp(float(op["value"]), -8192, 8191))
|
||||
track["bends"].sort(key=lambda x: x["time"])
|
||||
return
|
||||
|
||||
if op_name == "delete_bend":
|
||||
bend_id = str(op.get("bend_id", ""))
|
||||
track["bends"] = [b for b in track["bends"] if b["id"] != bend_id]
|
||||
return
|
||||
|
||||
raise ValueError(f"unsupported op: {op_name}")
|
||||
|
||||
|
||||
def save_session(session_id: str) -> dict[str, Any]:
|
||||
with _SESSIONS_LOCK:
|
||||
session = _SESSIONS.get(session_id)
|
||||
if not session:
|
||||
raise ValueError("session not found")
|
||||
|
||||
source_path = Path(session["source_path"])
|
||||
if not source_path.exists() or not source_path.is_file():
|
||||
raise ValueError("source midi file is missing")
|
||||
|
||||
midi = pretty_midi.PrettyMIDI(str(source_path))
|
||||
tracks: list[dict[str, Any]] = session["tracks"]
|
||||
|
||||
while len(midi.instruments) < len(tracks):
|
||||
midi.instruments.append(pretty_midi.Instrument(program=0, is_drum=False, name="New Track"))
|
||||
|
||||
for idx, track in enumerate(tracks):
|
||||
instrument = midi.instruments[idx]
|
||||
instrument.program = int(track.get("program", 0))
|
||||
instrument.is_drum = bool(track.get("is_drum", False))
|
||||
instrument.name = str(track.get("name", instrument.name or f"Track {idx + 1}"))
|
||||
|
||||
notes = []
|
||||
for item in track.get("notes", []):
|
||||
start = float(item["start"])
|
||||
duration = float(item["duration"])
|
||||
notes.append(
|
||||
pretty_midi.Note(
|
||||
velocity=int(item["velocity"]),
|
||||
pitch=int(item["pitch"]),
|
||||
start=max(0.0, start),
|
||||
end=max(0.001, start + duration),
|
||||
)
|
||||
)
|
||||
notes.sort(key=lambda x: (x.start, x.pitch))
|
||||
|
||||
bends = []
|
||||
for item in track.get("bends", []):
|
||||
bends.append(
|
||||
pretty_midi.PitchBend(
|
||||
pitch=int(item["value"]),
|
||||
time=max(0.0, float(item["time"])),
|
||||
)
|
||||
)
|
||||
bends.sort(key=lambda x: x.time)
|
||||
|
||||
instrument.notes = notes
|
||||
instrument.pitch_bends = bends
|
||||
|
||||
edited_name = f"{source_path.stem}.edited.mid"
|
||||
edited_path = source_path.with_name(edited_name)
|
||||
suffix = 2
|
||||
while edited_path.exists():
|
||||
edited_path = source_path.with_name(f"{source_path.stem}.edited.{suffix}.mid")
|
||||
suffix += 1
|
||||
|
||||
midi.write(str(edited_path))
|
||||
|
||||
summary = {
|
||||
"track_count": len(tracks),
|
||||
"note_count": sum(len(t.get("notes", [])) for t in tracks),
|
||||
"bend_count": sum(len(t.get("bends", [])) for t in tracks),
|
||||
}
|
||||
|
||||
save_job_id = jm.create_job("midi-edit-save")
|
||||
jm.update_job(
|
||||
save_job_id,
|
||||
status=jm.JobStatus.DONE,
|
||||
progress="MIDI edits saved.",
|
||||
files=[{"name": edited_path.name, "path": str(edited_path), "type": "midi"}],
|
||||
result={
|
||||
"source_midi_path": str(source_path),
|
||||
"edited_midi_path": str(edited_path),
|
||||
"midi_edit_summary": summary,
|
||||
"midi_editor_version": session.get("editor_version", "stemengine-midi-editor-v1"),
|
||||
"saved_at": _now_iso(),
|
||||
},
|
||||
)
|
||||
|
||||
session["last_saved_path"] = str(edited_path)
|
||||
session["last_saved_job_id"] = save_job_id
|
||||
|
||||
return {
|
||||
"job_id": save_job_id,
|
||||
"filename": edited_path.name,
|
||||
"path": str(edited_path),
|
||||
"summary": summary,
|
||||
}
|
||||
@@ -36,6 +36,8 @@ import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import pretty_midi
|
||||
|
||||
from .. import job_manager as jm
|
||||
from ..config import JOBS_DIR, OMNIZART_SIDECAR_URL
|
||||
|
||||
@@ -56,7 +58,7 @@ _POLL_INTERVAL: float = 5.0
|
||||
# Public entry point
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def run_omnizart_transcription(job_id: str, input_path: str, mode_id: str) -> None:
|
||||
def run_omnizart_transcription(job_id: str, input_path: str, mode_id: str, bend_policy: str = "auto") -> None:
|
||||
"""
|
||||
Background task: transcribe *input_path* to MIDI using Omnizart.
|
||||
|
||||
@@ -64,16 +66,16 @@ def run_omnizart_transcription(job_id: str, input_path: str, mode_id: str) -> No
|
||||
Routes to sidecar HTTP or local subprocess depending on OMNIZART_SIDECAR_URL.
|
||||
"""
|
||||
if OMNIZART_SIDECAR_URL:
|
||||
_run_via_sidecar(job_id, input_path, mode_id)
|
||||
_run_via_sidecar(job_id, input_path, mode_id, bend_policy)
|
||||
else:
|
||||
_run_via_subprocess(job_id, input_path, mode_id)
|
||||
_run_via_subprocess(job_id, input_path, mode_id, bend_policy)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sidecar HTTP path (Phase 3 / production)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _run_via_sidecar(job_id: str, input_path: str, mode_id: str) -> None:
|
||||
def _run_via_sidecar(job_id: str, input_path: str, mode_id: str, bend_policy: str) -> None:
|
||||
"""
|
||||
POST the audio file to the omnizart-sidecar HTTP service and save the
|
||||
returned MIDI bytes to the jobs directory.
|
||||
@@ -122,6 +124,7 @@ def _run_via_sidecar(job_id: str, input_path: str, mode_id: str) -> None:
|
||||
|
||||
dest = output_dir / mid_name
|
||||
dest.write_bytes(response.content)
|
||||
bend_stats = _postprocess_omnizart_bends(dest, mode, bend_policy)
|
||||
|
||||
log.debug("[%s] omnizart sidecar complete — output=%s", job_id, dest.name)
|
||||
jm.update_job(
|
||||
@@ -129,6 +132,7 @@ def _run_via_sidecar(job_id: str, input_path: str, mode_id: str) -> None:
|
||||
status=jm.JobStatus.DONE,
|
||||
progress="Transcription complete.",
|
||||
files=[{"name": dest.name, "path": str(dest)}],
|
||||
result={"bend_stats": bend_stats},
|
||||
)
|
||||
|
||||
except Exception as exc:
|
||||
@@ -140,7 +144,7 @@ def _run_via_sidecar(job_id: str, input_path: str, mode_id: str) -> None:
|
||||
# Local subprocess path (development / Option B)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _run_via_subprocess(job_id: str, input_path: str, mode_id: str) -> None:
|
||||
def _run_via_subprocess(job_id: str, input_path: str, mode_id: str, bend_policy: str) -> None:
|
||||
"""
|
||||
Convert the input to WAV, invoke `omnizart {mode} transcribe ...` as a
|
||||
subprocess, capture the output .mid file and store it in the jobs directory.
|
||||
@@ -208,12 +212,15 @@ def _run_via_subprocess(job_id: str, input_path: str, mode_id: str) -> None:
|
||||
dest = output_dir / mid_name
|
||||
dest.write_bytes(expected_output.read_bytes())
|
||||
|
||||
bend_stats = _postprocess_omnizart_bends(dest, mode, bend_policy)
|
||||
|
||||
log.debug("[%s] omnizart subprocess complete — output=%s", job_id, dest.name)
|
||||
jm.update_job(
|
||||
job_id,
|
||||
status=jm.JobStatus.DONE,
|
||||
progress="Transcription complete.",
|
||||
files=[{"name": dest.name, "path": str(dest)}],
|
||||
result={"bend_stats": bend_stats},
|
||||
)
|
||||
|
||||
except Exception as exc:
|
||||
@@ -253,3 +260,135 @@ def _convert_to_wav(src: Path, dst: Path) -> None:
|
||||
|
||||
def _fail(job_id: str, message: str) -> None:
|
||||
jm.update_job(job_id, status=jm.JobStatus.ERROR, error=message, progress="Error.")
|
||||
|
||||
|
||||
def _postprocess_omnizart_bends(mid_path: Path, mode: str, bend_policy: str) -> dict:
|
||||
midi = pretty_midi.PrettyMIDI(str(mid_path))
|
||||
native_bend_count = sum(len(inst.pitch_bends) for inst in midi.instruments)
|
||||
|
||||
stats = {
|
||||
"engine": "omnizart",
|
||||
"mode": mode,
|
||||
"bend_policy": bend_policy,
|
||||
"native_bend_count": int(native_bend_count),
|
||||
"derived_bend_count": 0,
|
||||
"bend_event_count": int(native_bend_count),
|
||||
"derivation_applied": False,
|
||||
"derivation_reason": "native_bends_present" if native_bend_count > 0 else "not_requested",
|
||||
}
|
||||
|
||||
if native_bend_count > 0:
|
||||
return stats
|
||||
|
||||
derive = False
|
||||
if bend_policy == "force_on":
|
||||
derive = True
|
||||
stats["derivation_reason"] = "forced_on"
|
||||
elif bend_policy == "force_off":
|
||||
derive = False
|
||||
stats["derivation_reason"] = "forced_off"
|
||||
else:
|
||||
derive = mode == "vocal-contour"
|
||||
stats["derivation_reason"] = "auto_vocal_contour" if derive else "auto_opt_in_required"
|
||||
|
||||
if not derive:
|
||||
return stats
|
||||
|
||||
if _is_polyphonic_dense(midi):
|
||||
stats["derivation_reason"] = "guard_dense_polyphonic"
|
||||
return stats
|
||||
|
||||
added = 0
|
||||
for instrument in midi.instruments:
|
||||
if instrument.is_drum:
|
||||
continue
|
||||
if instrument.pitch_bends:
|
||||
continue
|
||||
derived = _derive_bends_from_notes(instrument.notes, mode)
|
||||
if derived:
|
||||
instrument.pitch_bends = derived
|
||||
added += len(derived)
|
||||
|
||||
if added > 0:
|
||||
midi.write(str(mid_path))
|
||||
|
||||
stats["derived_bend_count"] = int(added)
|
||||
stats["bend_event_count"] = int(native_bend_count + added)
|
||||
stats["derivation_applied"] = added > 0
|
||||
if added == 0 and stats["derivation_reason"] in {"forced_on", "auto_vocal_contour"}:
|
||||
stats["derivation_reason"] = "no_eligible_note_transitions"
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
def _is_polyphonic_dense(midi: pretty_midi.PrettyMIDI) -> bool:
|
||||
events: list[tuple[float, int]] = []
|
||||
for instrument in midi.instruments:
|
||||
if instrument.is_drum:
|
||||
continue
|
||||
for note in instrument.notes:
|
||||
events.append((float(note.start), 1))
|
||||
events.append((float(note.end), -1))
|
||||
|
||||
if not events:
|
||||
return False
|
||||
|
||||
events.sort(key=lambda item: (item[0], -item[1]))
|
||||
active = 0
|
||||
max_active = 0
|
||||
dense_time = 0.0
|
||||
prev_t = events[0][0]
|
||||
|
||||
for t, delta in events:
|
||||
dt = max(0.0, t - prev_t)
|
||||
if active >= 3:
|
||||
dense_time += dt
|
||||
active += delta
|
||||
max_active = max(max_active, active)
|
||||
prev_t = t
|
||||
|
||||
total_time = max(0.001, events[-1][0] - events[0][0])
|
||||
dense_ratio = dense_time / total_time
|
||||
return max_active >= 4 or dense_ratio > 0.35
|
||||
|
||||
|
||||
def _derive_bends_from_notes(notes: list[pretty_midi.Note], mode: str) -> list[pretty_midi.PitchBend]:
|
||||
if len(notes) < 2:
|
||||
return []
|
||||
|
||||
sorted_notes = sorted(notes, key=lambda n: (float(n.start), float(n.pitch)))
|
||||
bend_range_semitones = 2.0
|
||||
max_delta_semitones = 6.0 if mode == "vocal-contour" else 2.5
|
||||
bends: list[pretty_midi.PitchBend] = [pretty_midi.PitchBend(pitch=0, time=max(0.0, float(sorted_notes[0].start)))]
|
||||
|
||||
for idx in range(1, len(sorted_notes)):
|
||||
prev_note = sorted_notes[idx - 1]
|
||||
note = sorted_notes[idx]
|
||||
gap = float(note.start) - float(prev_note.end)
|
||||
if gap > 0.20:
|
||||
continue
|
||||
|
||||
semitone_delta = float(note.pitch) - float(prev_note.pitch)
|
||||
if abs(semitone_delta) < 0.25:
|
||||
continue
|
||||
|
||||
semitone_delta = max(-max_delta_semitones, min(max_delta_semitones, semitone_delta))
|
||||
midi_value = int(max(-8192, min(8191, round((semitone_delta / bend_range_semitones) * 8192))))
|
||||
if midi_value == 0:
|
||||
continue
|
||||
|
||||
transition_time = max(float(prev_note.start), float(prev_note.end) - 0.035)
|
||||
settle_time = min(float(note.end), float(note.start) + 0.060)
|
||||
bends.append(pretty_midi.PitchBend(pitch=midi_value, time=max(0.0, transition_time)))
|
||||
bends.append(pretty_midi.PitchBend(pitch=0, time=max(0.0, settle_time)))
|
||||
|
||||
tail = max(float(sorted_notes[-1].end), 0.0)
|
||||
bends.append(pretty_midi.PitchBend(pitch=0, time=tail))
|
||||
|
||||
# Keep only one bend per timestamp (last event wins) and sorted ordering.
|
||||
merged: dict[float, int] = {}
|
||||
for bend in bends:
|
||||
key = round(float(bend.time), 4)
|
||||
merged[key] = int(max(-8192, min(8191, bend.pitch)))
|
||||
|
||||
return [pretty_midi.PitchBend(pitch=value, time=time) for time, value in sorted(merged.items())]
|
||||
@@ -70,7 +70,7 @@ def enqueue_stem_reprocess(
|
||||
return {"job_id": job_id}
|
||||
|
||||
|
||||
def enqueue_project_transcription(project: dict, stem_name: str, mode: str) -> dict:
|
||||
def enqueue_project_transcription(project: dict, stem_name: str, mode: str, bend_policy: str = "auto") -> dict:
|
||||
"""Queue stem transcription for a project and persist the stem→job link."""
|
||||
valid_modes = {mode_entry["id"] for mode_entry in TRANSCRIBE_MODES}
|
||||
if mode not in valid_modes:
|
||||
@@ -88,7 +88,7 @@ def enqueue_project_transcription(project: dict, stem_name: str, mode: str) -> d
|
||||
|
||||
job_id = jm.create_job("transcribe")
|
||||
projects_repo.set_transcription_job(project["id"], stem_name, job_id)
|
||||
jm.submit_gpu(run_transcription, job_id, stem_path, mode)
|
||||
jm.submit_gpu(run_transcription, job_id, stem_path, mode, bend_policy)
|
||||
return {"job_id": job_id}
|
||||
|
||||
|
||||
|
||||
@@ -1493,6 +1493,16 @@ def _run_demucs(
|
||||
elif ext == 'flac':
|
||||
fmt_flag = ['--flac']
|
||||
|
||||
demucs_device = 'cpu'
|
||||
if not CPU_SEPARATION:
|
||||
try:
|
||||
import torch
|
||||
|
||||
if torch.cuda.is_available() and torch.cuda.device_count() > 0:
|
||||
demucs_device = 'cuda'
|
||||
except Exception:
|
||||
demucs_device = 'cpu'
|
||||
|
||||
# Determine segment size: check env override first, then auto-scale by duration
|
||||
_dur = get_audio_duration_secs(input_path)
|
||||
_use_long_track_tuning = should_chunk(_dur, LONG_TRACK_CHUNKING_MODE, LONG_TRACK_DURATION_SECS)
|
||||
@@ -1526,6 +1536,7 @@ def _run_demucs(
|
||||
cmd = [
|
||||
sys.executable, '-m', 'demucs',
|
||||
'--name', model,
|
||||
'--device', demucs_device,
|
||||
'--out', str(temp_out),
|
||||
*fmt_flag,
|
||||
*power_flags,
|
||||
@@ -1537,10 +1548,12 @@ def _run_demucs(
|
||||
jm.update_job(
|
||||
job_id,
|
||||
progress=(
|
||||
f"Long-track tuning: segment={_segment}s, overlap={_overlap or 0.25}x "
|
||||
f"Long-track tuning: segment={_segment}s, overlap={_overlap or 0.25}x, device={demucs_device} "
|
||||
f"(duration={_dur:.0f}s)..."
|
||||
),
|
||||
)
|
||||
else:
|
||||
jm.update_job(job_id, progress=f"Demucs device: {demucs_device}...")
|
||||
|
||||
process = subprocess.Popen(
|
||||
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True,
|
||||
@@ -1563,6 +1576,9 @@ def _run_demucs(
|
||||
line = process.stdout.readline()
|
||||
if line:
|
||||
jm.update_job(job_id, progress=line.rstrip())
|
||||
elif process.poll() is not None:
|
||||
# EOF with a finished subprocess: stop polling.
|
||||
break
|
||||
elif process.poll() is not None:
|
||||
break
|
||||
|
||||
|
||||
@@ -0,0 +1,751 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shlex
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import zipfile
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
from .. import job_manager as jm
|
||||
from ..config import MEDIA_DIR
|
||||
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_PAUSE_EVENT = threading.Event()
|
||||
_PAUSE_EVENT.set()
|
||||
|
||||
_STATE_LOCK = threading.Lock()
|
||||
_QUEUE_HYDRATED = False
|
||||
_ACTIVE_SUBMISSIONS: set[str] = set()
|
||||
|
||||
_SOURCE_ROOT = Path(os.environ.get("SLOPPAK_SOURCE_ROOT", str(MEDIA_DIR / "uploads"))).resolve()
|
||||
_OUTPUT_ROOT = Path(os.environ.get("SLOPPAK_OUTPUT_ROOT", str(MEDIA_DIR / "sloppak"))).resolve()
|
||||
|
||||
|
||||
def _now_iso() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def _normalize_rel_psarc(filename: str) -> str:
|
||||
rel = (filename or "").strip().replace("\\", "/")
|
||||
if not rel:
|
||||
raise ValueError("filename is required")
|
||||
rel_path = Path(rel)
|
||||
if rel_path.is_absolute() or ".." in rel_path.parts:
|
||||
raise ValueError("filename must be a relative path under source root")
|
||||
if rel_path.suffix.lower() != ".psarc":
|
||||
raise ValueError("filename must end with .psarc")
|
||||
return rel_path.as_posix()
|
||||
|
||||
|
||||
def _resolve_source_file(rel_psarc: str) -> Path:
|
||||
source = (_SOURCE_ROOT / rel_psarc).resolve()
|
||||
source.relative_to(_SOURCE_ROOT)
|
||||
return source
|
||||
|
||||
|
||||
def _output_path_for(rel_psarc: str) -> Path:
|
||||
rel = Path(rel_psarc)
|
||||
return (_OUTPUT_ROOT / rel).with_suffix(".sloppak")
|
||||
|
||||
|
||||
def _job_source_filename(job: dict) -> str:
|
||||
result = job.get("result") if isinstance(job.get("result"), dict) else {}
|
||||
return str(result.get("source_filename", "")).strip()
|
||||
|
||||
|
||||
def list_conversion_jobs() -> list[dict]:
|
||||
jobs = [j for j in jm.list_all_jobs() if j.get("job_type") == "sloppak_convert"]
|
||||
jobs.sort(key=lambda x: x.get("created_at") or "", reverse=True)
|
||||
return jobs
|
||||
|
||||
|
||||
def get_snapshot() -> dict[str, Any]:
|
||||
ensure_queue_hydrated()
|
||||
return {
|
||||
"paused": is_paused(),
|
||||
"jobs": list_conversion_jobs(),
|
||||
"capabilities": get_converter_capabilities(),
|
||||
}
|
||||
|
||||
|
||||
def stream_updates(*, poll_interval_sec: float = 1.0):
|
||||
ensure_queue_hydrated()
|
||||
last_payload = ""
|
||||
# Retry hint helps reconnect logic recover quickly after restarts/network blips.
|
||||
yield "retry: 1500\n\n"
|
||||
while True:
|
||||
payload = json.dumps(get_snapshot(), ensure_ascii=True, sort_keys=True)
|
||||
if payload != last_payload:
|
||||
last_payload = payload
|
||||
yield f"event: snapshot\ndata: {payload}\n\n"
|
||||
time.sleep(max(0.2, float(poll_interval_sec)))
|
||||
|
||||
|
||||
def is_paused() -> bool:
|
||||
return not _PAUSE_EVENT.is_set()
|
||||
|
||||
|
||||
def get_converter_capabilities() -> dict[str, Any]:
|
||||
ensure_queue_hydrated()
|
||||
unpack = _get_unpacker_capability()
|
||||
return {
|
||||
"source_root": str(_SOURCE_ROOT),
|
||||
"output_root": str(_OUTPUT_ROOT),
|
||||
"source_root_exists": _SOURCE_ROOT.exists(),
|
||||
"output_root_exists": _OUTPUT_ROOT.exists(),
|
||||
"unpacker": unpack,
|
||||
"ready": bool(unpack.get("available")),
|
||||
}
|
||||
|
||||
|
||||
def enqueue_conversion(filename: str, *, reconvert: bool = False) -> dict:
|
||||
ensure_queue_hydrated()
|
||||
try:
|
||||
rel_psarc = _normalize_rel_psarc(filename)
|
||||
except ValueError as exc:
|
||||
return {"skipped": True, "filename": filename, "reason": str(exc)}
|
||||
|
||||
try:
|
||||
source_file = _resolve_source_file(rel_psarc)
|
||||
except Exception:
|
||||
return {"skipped": True, "filename": rel_psarc, "reason": "not_found"}
|
||||
|
||||
if not source_file.is_file():
|
||||
return {"skipped": True, "filename": rel_psarc, "reason": "not_found"}
|
||||
|
||||
output_path = _output_path_for(rel_psarc)
|
||||
if output_path.exists() and not reconvert:
|
||||
return {
|
||||
"skipped": True,
|
||||
"filename": rel_psarc,
|
||||
"reason": "already_converted",
|
||||
"output_path": str(output_path),
|
||||
}
|
||||
|
||||
for existing in list_conversion_jobs():
|
||||
if _job_source_filename(existing) != rel_psarc:
|
||||
continue
|
||||
status = existing.get("status")
|
||||
if status == "queued":
|
||||
return {
|
||||
"skipped": True,
|
||||
"filename": rel_psarc,
|
||||
"reason": "already_queued",
|
||||
"job_id": existing.get("id"),
|
||||
}
|
||||
if status == "running":
|
||||
return {
|
||||
"skipped": True,
|
||||
"filename": rel_psarc,
|
||||
"reason": "already_running",
|
||||
"job_id": existing.get("id"),
|
||||
}
|
||||
|
||||
job_id = jm.create_job("sloppak_convert")
|
||||
jm.update_job(
|
||||
job_id,
|
||||
progress="Queued for conversion.",
|
||||
result={
|
||||
"source_filename": rel_psarc,
|
||||
"output_path": str(output_path),
|
||||
"conversion_kind": "psarc_to_sloppak",
|
||||
"queued_at": _now_iso(),
|
||||
},
|
||||
)
|
||||
_submit_conversion(job_id, rel_psarc)
|
||||
return {"ok": True, "filename": rel_psarc, "job_id": job_id}
|
||||
|
||||
|
||||
def enqueue_bulk(filenames: list[str], *, reconvert: bool = False) -> dict:
|
||||
ensure_queue_hydrated()
|
||||
enqueued: list[dict] = []
|
||||
skipped: list[dict] = []
|
||||
existing_jobs = list_conversion_jobs()
|
||||
existing_status_by_source: dict[str, str] = {}
|
||||
for job in existing_jobs:
|
||||
src = _job_source_filename(job)
|
||||
status = str(job.get("status") or "")
|
||||
if src and status in {"queued", "running"}:
|
||||
existing_status_by_source[src] = status
|
||||
|
||||
use_threaded_preflight = len(filenames) >= 40
|
||||
if use_threaded_preflight:
|
||||
max_workers = max(2, min(8, (os.cpu_count() or 4)))
|
||||
with ThreadPoolExecutor(max_workers=max_workers) as pool:
|
||||
preflight_results = list(
|
||||
pool.map(
|
||||
lambda item: _preflight_filename(item, reconvert=reconvert, existing_status=existing_status_by_source),
|
||||
filenames,
|
||||
)
|
||||
)
|
||||
else:
|
||||
preflight_results = [
|
||||
_preflight_filename(item, reconvert=reconvert, existing_status=existing_status_by_source)
|
||||
for item in filenames
|
||||
]
|
||||
|
||||
for preflight in preflight_results:
|
||||
if preflight.get("ok"):
|
||||
rel_psarc = str(preflight["filename"])
|
||||
res = enqueue_conversion(rel_psarc, reconvert=reconvert)
|
||||
if res.get("ok"):
|
||||
enqueued.append(res)
|
||||
else:
|
||||
skipped.append(res)
|
||||
else:
|
||||
skipped.append(preflight)
|
||||
|
||||
return {
|
||||
"enqueued": enqueued,
|
||||
"skipped": skipped,
|
||||
"summary": {
|
||||
"requested": len(filenames),
|
||||
"enqueued": len(enqueued),
|
||||
"skipped": len(skipped),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def pause_queue() -> dict:
|
||||
ensure_queue_hydrated()
|
||||
_PAUSE_EVENT.clear()
|
||||
return {"paused": True}
|
||||
|
||||
|
||||
def resume_queue() -> dict:
|
||||
ensure_queue_hydrated()
|
||||
_PAUSE_EVENT.set()
|
||||
return {"paused": False}
|
||||
|
||||
|
||||
def cancel_queued() -> dict:
|
||||
ensure_queue_hydrated()
|
||||
count = 0
|
||||
for job in list_conversion_jobs():
|
||||
if job.get("status") == "queued":
|
||||
jm.cancel_job(job["id"])
|
||||
count += 1
|
||||
return {"cancelled": count}
|
||||
|
||||
|
||||
def clear_finished() -> dict:
|
||||
ensure_queue_hydrated()
|
||||
deleted = 0
|
||||
for job in list_conversion_jobs():
|
||||
if job.get("status") in {"done", "error", "cancelled"}:
|
||||
if jm.delete_job(job["id"]):
|
||||
deleted += 1
|
||||
return {"deleted": deleted}
|
||||
|
||||
|
||||
def retry_failed() -> dict:
|
||||
ensure_queue_hydrated()
|
||||
failed_jobs = [j for j in list_conversion_jobs() if j.get("status") == "error"]
|
||||
retried = 0
|
||||
skipped = 0
|
||||
for job in failed_jobs:
|
||||
source_filename = _job_source_filename(job)
|
||||
if not source_filename:
|
||||
skipped += 1
|
||||
continue
|
||||
response = enqueue_conversion(source_filename, reconvert=True)
|
||||
if response.get("ok"):
|
||||
retried += 1
|
||||
else:
|
||||
skipped += 1
|
||||
return {"retried": retried, "skipped": skipped}
|
||||
|
||||
|
||||
def _run_convert_job(job_id: str, rel_psarc: str) -> None:
|
||||
try:
|
||||
while not _PAUSE_EVENT.is_set():
|
||||
if jm.should_cancel(job_id):
|
||||
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
|
||||
return
|
||||
jm.update_job(job_id, progress="Paused.")
|
||||
time.sleep(0.25)
|
||||
|
||||
jm.update_job(job_id, status=jm.JobStatus.RUNNING, progress="Preparing conversion...")
|
||||
source_file = _resolve_source_file(rel_psarc)
|
||||
if not source_file.is_file():
|
||||
jm.update_job(
|
||||
job_id,
|
||||
status=jm.JobStatus.ERROR,
|
||||
progress="Error.",
|
||||
error=f"Source file not found: {rel_psarc}",
|
||||
)
|
||||
return
|
||||
|
||||
if jm.should_cancel(job_id):
|
||||
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
|
||||
return
|
||||
|
||||
output_path = _output_path_for(rel_psarc)
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
_build_sloppak_archive(job_id, source_file, output_path, rel_psarc)
|
||||
|
||||
result = {
|
||||
"source_filename": rel_psarc,
|
||||
"output_path": str(output_path),
|
||||
"conversion_kind": "psarc_to_sloppak",
|
||||
"finished_at": _now_iso(),
|
||||
"output_size": output_path.stat().st_size,
|
||||
}
|
||||
jm.update_job(
|
||||
job_id,
|
||||
status=jm.JobStatus.DONE,
|
||||
progress="Conversion complete.",
|
||||
files=[{"name": output_path.name, "path": str(output_path), "type": "sloppak"}],
|
||||
result=result,
|
||||
)
|
||||
except Exception as exc:
|
||||
log.exception("sloppak conversion failed for job %s", job_id)
|
||||
jm.update_job(job_id, status=jm.JobStatus.ERROR, progress="Error.", error=str(exc))
|
||||
|
||||
|
||||
def ensure_queue_hydrated() -> dict[str, int]:
|
||||
global _QUEUE_HYDRATED
|
||||
with _STATE_LOCK:
|
||||
if _QUEUE_HYDRATED:
|
||||
return {"resubmitted": 0, "requeued_running": 0}
|
||||
|
||||
jobs = list_conversion_jobs()
|
||||
resubmitted = 0
|
||||
requeued_running = 0
|
||||
for job in jobs:
|
||||
status = str(job.get("status") or "")
|
||||
if status not in {"queued", "running"}:
|
||||
continue
|
||||
|
||||
rel_psarc = _job_source_filename(job)
|
||||
job_id = str(job.get("id") or "")
|
||||
if not rel_psarc or not job_id:
|
||||
continue
|
||||
|
||||
if status == "running":
|
||||
requeued_running += 1
|
||||
jm.update_job(
|
||||
job_id,
|
||||
status=jm.JobStatus.QUEUED,
|
||||
progress="Recovered after restart; re-queued.",
|
||||
)
|
||||
|
||||
if _submit_conversion(job_id, rel_psarc):
|
||||
resubmitted += 1
|
||||
|
||||
with _STATE_LOCK:
|
||||
_QUEUE_HYDRATED = True
|
||||
if resubmitted or requeued_running:
|
||||
log.info(
|
||||
"sloppak queue hydrated: resubmitted=%s requeued_running=%s",
|
||||
resubmitted,
|
||||
requeued_running,
|
||||
)
|
||||
return {"resubmitted": resubmitted, "requeued_running": requeued_running}
|
||||
|
||||
|
||||
def _submit_conversion(job_id: str, rel_psarc: str) -> bool:
|
||||
with _STATE_LOCK:
|
||||
if job_id in _ACTIVE_SUBMISSIONS:
|
||||
return False
|
||||
_ACTIVE_SUBMISSIONS.add(job_id)
|
||||
|
||||
def _runner() -> None:
|
||||
try:
|
||||
_run_convert_job(job_id, rel_psarc)
|
||||
finally:
|
||||
with _STATE_LOCK:
|
||||
_ACTIVE_SUBMISSIONS.discard(job_id)
|
||||
|
||||
jm.submit(_runner)
|
||||
return True
|
||||
|
||||
|
||||
def _preflight_filename(
|
||||
filename: str,
|
||||
*,
|
||||
reconvert: bool,
|
||||
existing_status: dict[str, str],
|
||||
) -> dict[str, Any]:
|
||||
try:
|
||||
rel_psarc = _normalize_rel_psarc(filename)
|
||||
except ValueError as exc:
|
||||
return {"skipped": True, "filename": filename, "reason": str(exc)}
|
||||
|
||||
status = existing_status.get(rel_psarc)
|
||||
if status == "queued":
|
||||
return {"skipped": True, "filename": rel_psarc, "reason": "already_queued"}
|
||||
if status == "running":
|
||||
return {"skipped": True, "filename": rel_psarc, "reason": "already_running"}
|
||||
|
||||
try:
|
||||
source_file = _resolve_source_file(rel_psarc)
|
||||
except Exception:
|
||||
return {"skipped": True, "filename": rel_psarc, "reason": "not_found"}
|
||||
|
||||
if not source_file.is_file():
|
||||
return {"skipped": True, "filename": rel_psarc, "reason": "not_found"}
|
||||
|
||||
output_path = _output_path_for(rel_psarc)
|
||||
if output_path.exists() and not reconvert:
|
||||
return {
|
||||
"skipped": True,
|
||||
"filename": rel_psarc,
|
||||
"reason": "already_converted",
|
||||
"output_path": str(output_path),
|
||||
}
|
||||
|
||||
return {"ok": True, "filename": rel_psarc}
|
||||
|
||||
|
||||
def _build_sloppak_archive(job_id: str, source_file: Path, output_path: Path, rel_psarc: str) -> None:
|
||||
jm.update_job(job_id, progress="Analyzing Rocksmith payload...")
|
||||
rocksmith = _extract_rocksmith_context(source_file)
|
||||
|
||||
manifest = {
|
||||
"format": "sloppak",
|
||||
"version": 1,
|
||||
"metadata": {
|
||||
"title": rocksmith.get("song", {}).get("title") or Path(rel_psarc).stem,
|
||||
"artist": rocksmith.get("song", {}).get("artist") or "",
|
||||
"album": rocksmith.get("song", {}).get("album") or "",
|
||||
"year": rocksmith.get("song", {}).get("year") or 0,
|
||||
"source_file": rel_psarc,
|
||||
"converted_at": _now_iso(),
|
||||
"converter": "stemengine.slice_ab",
|
||||
},
|
||||
"contents": {
|
||||
"source_psarc": f"source/{Path(rel_psarc).name}",
|
||||
},
|
||||
"rocksmith": {
|
||||
"unpack": {
|
||||
"succeeded": bool(rocksmith.get("unpack_succeeded")),
|
||||
"method": rocksmith.get("unpack_method"),
|
||||
"error": rocksmith.get("unpack_error"),
|
||||
},
|
||||
"arrangements": rocksmith.get("arrangements", []),
|
||||
"audio": {
|
||||
"wem_count": len(rocksmith.get("wem_files", [])),
|
||||
"wem_files": rocksmith.get("wem_files", []),
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="stemengine_sloppak_") as tmp:
|
||||
tmp_dir = Path(tmp)
|
||||
src_dir = tmp_dir / "source"
|
||||
meta_dir = tmp_dir / "metadata"
|
||||
src_dir.mkdir(parents=True, exist_ok=True)
|
||||
meta_dir.mkdir(parents=True, exist_ok=True)
|
||||
staged_psarc = src_dir / Path(rel_psarc).name
|
||||
staged_psarc.write_bytes(source_file.read_bytes())
|
||||
|
||||
lyrics = rocksmith.get("lyrics", [])
|
||||
if lyrics:
|
||||
lyrics_path = tmp_dir / "lyrics.json"
|
||||
lyrics_path.write_text(json.dumps(lyrics, ensure_ascii=True, indent=2), encoding="utf-8")
|
||||
manifest["contents"]["lyrics"] = "lyrics.json"
|
||||
manifest["metadata"]["lyrics_source"] = "xml-vocals"
|
||||
|
||||
summary_path = meta_dir / "rocksmith-summary.json"
|
||||
summary_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"song": rocksmith.get("song", {}),
|
||||
"arrangements": rocksmith.get("arrangements", []),
|
||||
"wem_files": rocksmith.get("wem_files", []),
|
||||
},
|
||||
ensure_ascii=True,
|
||||
indent=2,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
manifest["contents"]["metadata"] = {
|
||||
"summary": "metadata/rocksmith-summary.json",
|
||||
}
|
||||
|
||||
manifest_path = tmp_dir / "manifest.json"
|
||||
manifest_path.write_text(json.dumps(manifest, ensure_ascii=True, indent=2), encoding="utf-8")
|
||||
|
||||
tmp_output = output_path.with_suffix(output_path.suffix + ".tmp")
|
||||
with zipfile.ZipFile(tmp_output, "w", compression=zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.write(manifest_path, "manifest.json")
|
||||
zf.write(staged_psarc, f"source/{staged_psarc.name}")
|
||||
if lyrics:
|
||||
zf.write(tmp_dir / "lyrics.json", "lyrics.json")
|
||||
zf.write(summary_path, "metadata/rocksmith-summary.json")
|
||||
tmp_output.replace(output_path)
|
||||
|
||||
|
||||
def _extract_rocksmith_context(source_file: Path) -> dict[str, Any]:
|
||||
with tempfile.TemporaryDirectory(prefix="stemengine_psarc_extract_") as tmp:
|
||||
extract_dir = Path(tmp)
|
||||
unpack_succeeded = _try_unpack_psarc(source_file, extract_dir)
|
||||
|
||||
context: dict[str, Any] = {
|
||||
"unpack_succeeded": unpack_succeeded.get("ok", False),
|
||||
"unpack_method": unpack_succeeded.get("method"),
|
||||
"unpack_error": unpack_succeeded.get("error"),
|
||||
"song": {
|
||||
"title": "",
|
||||
"artist": "",
|
||||
"album": "",
|
||||
"year": 0,
|
||||
},
|
||||
"arrangements": [],
|
||||
"lyrics": [],
|
||||
"wem_files": [],
|
||||
}
|
||||
|
||||
if not unpack_succeeded.get("ok"):
|
||||
return context
|
||||
|
||||
context["song"], context["arrangements"] = _extract_song_and_arrangements(extract_dir)
|
||||
context["lyrics"] = _extract_lyrics(extract_dir)
|
||||
context["wem_files"] = sorted(
|
||||
[p.relative_to(extract_dir).as_posix() for p in extract_dir.rglob("*.wem")]
|
||||
)
|
||||
return context
|
||||
|
||||
|
||||
def _try_unpack_psarc(source_file: Path, extract_dir: Path) -> dict[str, Any]:
|
||||
# Preferred path when available: Slopsmith-compatible Python unpacker.
|
||||
try:
|
||||
from patcher import unpack_psarc as patcher_unpack_psarc # type: ignore[import-not-found]
|
||||
|
||||
patcher_unpack_psarc(str(source_file), str(extract_dir))
|
||||
return {"ok": True, "method": "python-patcher"}
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Optional CLI path: user can provide explicit unpack command template.
|
||||
# Template supports {input} and {output} placeholders.
|
||||
# Example: SLOPPAK_PSARC_UNPACK_CMD="/usr/local/bin/psarc unpack {input} {output}"
|
||||
raw_cmd = (os.environ.get("SLOPPAK_PSARC_UNPACK_CMD") or "").strip()
|
||||
if not raw_cmd:
|
||||
return {
|
||||
"ok": False,
|
||||
"method": None,
|
||||
"error": "No PSARC unpacker available (patcher module or SLOPPAK_PSARC_UNPACK_CMD).",
|
||||
}
|
||||
|
||||
try:
|
||||
rendered = raw_cmd.format(input=str(source_file), output=str(extract_dir))
|
||||
args = shlex.split(rendered)
|
||||
run = subprocess.run(args, capture_output=True, text=True)
|
||||
if run.returncode != 0:
|
||||
return {
|
||||
"ok": False,
|
||||
"method": "env-command",
|
||||
"error": (run.stderr or run.stdout or "psarc unpack command failed").strip()[:500],
|
||||
}
|
||||
return {"ok": True, "method": "env-command"}
|
||||
except Exception as exc:
|
||||
return {"ok": False, "method": "env-command", "error": str(exc)}
|
||||
|
||||
|
||||
def _get_unpacker_capability() -> dict[str, Any]:
|
||||
try:
|
||||
from patcher import unpack_psarc as _unused # type: ignore[import-not-found]
|
||||
|
||||
return {
|
||||
"available": True,
|
||||
"method": "python-patcher",
|
||||
"detail": "Using patcher.unpack_psarc",
|
||||
}
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
raw_cmd = (os.environ.get("SLOPPAK_PSARC_UNPACK_CMD") or "").strip()
|
||||
if not raw_cmd:
|
||||
return {
|
||||
"available": False,
|
||||
"method": None,
|
||||
"detail": "Set SLOPPAK_PSARC_UNPACK_CMD or install patcher module.",
|
||||
}
|
||||
|
||||
try:
|
||||
rendered = raw_cmd.format(input="/path/to/source.psarc", output="/path/to/extract")
|
||||
args = shlex.split(rendered)
|
||||
if not args:
|
||||
return {
|
||||
"available": False,
|
||||
"method": "env-command",
|
||||
"detail": "SLOPPAK_PSARC_UNPACK_CMD is empty after template rendering.",
|
||||
}
|
||||
|
||||
binary = args[0]
|
||||
binary_exists = os.path.isabs(binary) and Path(binary).exists() or bool(shutil.which(binary))
|
||||
return {
|
||||
"available": bool(binary_exists),
|
||||
"method": "env-command",
|
||||
"detail": "Using SLOPPAK_PSARC_UNPACK_CMD template.",
|
||||
"binary": binary,
|
||||
"binary_found": bool(binary_exists),
|
||||
}
|
||||
except Exception as exc:
|
||||
return {
|
||||
"available": False,
|
||||
"method": "env-command",
|
||||
"detail": f"Invalid SLOPPAK_PSARC_UNPACK_CMD: {exc}",
|
||||
}
|
||||
|
||||
|
||||
def _extract_song_and_arrangements(extract_dir: Path) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
song: dict[str, Any] = {
|
||||
"title": "",
|
||||
"artist": "",
|
||||
"album": "",
|
||||
"year": 0,
|
||||
}
|
||||
arrangements: list[dict[str, Any]] = []
|
||||
|
||||
json_files = sorted(extract_dir.rglob("*.json"))
|
||||
for file_path in json_files:
|
||||
if file_path.stat().st_size > 5_000_000:
|
||||
continue
|
||||
payload = _read_json(file_path)
|
||||
if not isinstance(payload, dict):
|
||||
continue
|
||||
|
||||
_merge_song_fields(song, payload)
|
||||
_append_arrangements(arrangements, payload, file_path)
|
||||
|
||||
_dedupe_arrangements(arrangements)
|
||||
return song, arrangements
|
||||
|
||||
|
||||
def _read_json(path: Path) -> Any:
|
||||
try:
|
||||
return json.loads(path.read_text(encoding="utf-8", errors="replace"))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _merge_song_fields(song: dict[str, Any], payload: dict[str, Any]) -> None:
|
||||
def pick(*keys: str) -> str:
|
||||
for key in keys:
|
||||
value = payload.get(key)
|
||||
if isinstance(value, str) and value.strip():
|
||||
return value.strip()
|
||||
return ""
|
||||
|
||||
title = pick("title", "Title", "SongName")
|
||||
artist = pick("artist", "ArtistName", "artistName")
|
||||
album = pick("album", "AlbumName", "albumName")
|
||||
year_raw = payload.get("SongYear") or payload.get("year") or payload.get("Year")
|
||||
|
||||
if title and not song["title"]:
|
||||
song["title"] = title
|
||||
if artist and not song["artist"]:
|
||||
song["artist"] = artist
|
||||
if album and not song["album"]:
|
||||
song["album"] = album
|
||||
if not song["year"]:
|
||||
try:
|
||||
year = int(year_raw)
|
||||
if 1900 <= year <= 2200:
|
||||
song["year"] = year
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _append_arrangements(arrangements: list[dict[str, Any]], payload: dict[str, Any], path: Path) -> None:
|
||||
candidates: list[dict[str, Any]] = []
|
||||
if isinstance(payload.get("Arrangements"), list):
|
||||
for item in payload["Arrangements"]:
|
||||
if isinstance(item, dict):
|
||||
candidates.append(item)
|
||||
if payload.get("ArrangementProperties") or payload.get("Tuning"):
|
||||
candidates.append(payload)
|
||||
|
||||
for item in candidates:
|
||||
name = _first_text(item, "name", "Name", "ArrangementName", "arrangementName")
|
||||
if not name:
|
||||
base = path.stem
|
||||
if base:
|
||||
name = base
|
||||
tuning = item.get("Tuning") or item.get("tuning") or []
|
||||
if not isinstance(tuning, list):
|
||||
tuning = []
|
||||
capo = item.get("Capo")
|
||||
if capo is None:
|
||||
capo = item.get("capo")
|
||||
try:
|
||||
capo_value = int(capo) if capo is not None else 0
|
||||
except Exception:
|
||||
capo_value = 0
|
||||
|
||||
normalized_tuning: list[int] = []
|
||||
for value in tuning:
|
||||
try:
|
||||
normalized_tuning.append(int(value))
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
arrangements.append(
|
||||
{
|
||||
"id": _first_text(item, "PersistentID", "persistentID") or name.lower().replace(" ", "_"),
|
||||
"name": name,
|
||||
"tuning": normalized_tuning[:6],
|
||||
"capo": capo_value,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _first_text(payload: dict[str, Any], *keys: str) -> str:
|
||||
for key in keys:
|
||||
value = payload.get(key)
|
||||
if isinstance(value, str) and value.strip():
|
||||
return value.strip()
|
||||
return ""
|
||||
|
||||
|
||||
def _dedupe_arrangements(arrangements: list[dict[str, Any]]) -> None:
|
||||
seen: set[tuple[str, str]] = set()
|
||||
deduped: list[dict[str, Any]] = []
|
||||
for item in arrangements:
|
||||
key = (str(item.get("id", "")), str(item.get("name", "")))
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
deduped.append(item)
|
||||
arrangements.clear()
|
||||
arrangements.extend(deduped)
|
||||
|
||||
|
||||
def _extract_lyrics(extract_dir: Path) -> list[dict[str, Any]]:
|
||||
lyrics: list[dict[str, Any]] = []
|
||||
for xml_path in sorted(extract_dir.rglob("*.xml")):
|
||||
try:
|
||||
root = ET.parse(xml_path).getroot()
|
||||
except Exception:
|
||||
continue
|
||||
if root.tag != "vocals":
|
||||
continue
|
||||
for vocal in root.findall("vocal"):
|
||||
try:
|
||||
t = float(vocal.get("time", "0"))
|
||||
except Exception:
|
||||
t = 0.0
|
||||
try:
|
||||
d = float(vocal.get("length", "0"))
|
||||
except Exception:
|
||||
d = 0.0
|
||||
word = (vocal.get("lyric") or "").strip()
|
||||
if not word:
|
||||
continue
|
||||
lyrics.append({"t": round(t, 3), "d": round(d, 3), "w": word})
|
||||
return lyrics
|
||||
@@ -6,7 +6,9 @@ Routes to the appropriate engine based on the mode prefix:
|
||||
'omnizart-*' → Omnizart CLI or sidecar HTTP (requires OMNIZART_ENABLED=true)
|
||||
"""
|
||||
import logging
|
||||
from statistics import mean
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .. import job_manager as jm
|
||||
from ..config import JOBS_DIR, TRANSCRIBE_PRESETS, OMNIZART_ENABLED
|
||||
@@ -14,7 +16,10 @@ from ..config import JOBS_DIR, TRANSCRIBE_PRESETS, OMNIZART_ENABLED
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def run_transcription(job_id: str, input_path: str, mode: str) -> None:
|
||||
_BEND_POLICY_VALUES = {"auto", "force_on", "force_off"}
|
||||
|
||||
|
||||
def run_transcription(job_id: str, input_path: str, mode: str, bend_policy: str = "auto") -> None:
|
||||
"""
|
||||
Background task: transcribe audio to MIDI.
|
||||
|
||||
@@ -23,6 +28,8 @@ def run_transcription(job_id: str, input_path: str, mode: str) -> None:
|
||||
'omnizart-vocal' → Omnizart vocal mode
|
||||
'omnizart-vocal-contour' → Omnizart vocal-contour mode
|
||||
"""
|
||||
resolved_bend_policy = _normalize_bend_policy(bend_policy)
|
||||
|
||||
if mode.startswith("omnizart-"):
|
||||
if not OMNIZART_ENABLED:
|
||||
jm.update_job(
|
||||
@@ -33,16 +40,16 @@ def run_transcription(job_id: str, input_path: str, mode: str) -> None:
|
||||
)
|
||||
return
|
||||
from .omnizart_service import run_omnizart_transcription
|
||||
run_omnizart_transcription(job_id, input_path, mode)
|
||||
run_omnizart_transcription(job_id, input_path, mode, resolved_bend_policy)
|
||||
else:
|
||||
_run_basic_pitch(job_id, input_path, mode)
|
||||
_run_basic_pitch(job_id, input_path, mode, resolved_bend_policy)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# basic-pitch (original engine — unchanged)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _run_basic_pitch(job_id: str, input_path: str, mode: str) -> None:
|
||||
def _run_basic_pitch(job_id: str, input_path: str, mode: str, bend_policy: str) -> None:
|
||||
# Reuse global GPU safety limits (VRAM reserve + TF32 settings) before
|
||||
# loading any transcription backend for basic-pitch.
|
||||
try:
|
||||
@@ -66,15 +73,30 @@ def _run_basic_pitch(job_id: str, input_path: str, mode: str) -> None:
|
||||
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
|
||||
return
|
||||
|
||||
params = TRANSCRIBE_PRESETS.get(mode, TRANSCRIBE_PRESETS["polyphonic"])
|
||||
params = dict(TRANSCRIBE_PRESETS.get(mode, TRANSCRIBE_PRESETS["polyphonic"]))
|
||||
# Always infer with bends enabled for adaptive modes; policy is applied
|
||||
# after prediction by either keeping or stripping bend events.
|
||||
if mode in {"polyphonic", "melody"}:
|
||||
params["multiple_pitch_bends"] = True
|
||||
jm.update_job(job_id, progress=f"Transcribing with basic-pitch ({mode})...")
|
||||
|
||||
_, midi_data, _ = predict(
|
||||
_, midi_data, note_events = predict(
|
||||
input_path,
|
||||
ICASSP_2022_MODEL_PATH,
|
||||
**params,
|
||||
)
|
||||
|
||||
confidence_summary = _summarize_confidence(note_events)
|
||||
stability = _estimate_note_stability(midi_data)
|
||||
allow_bends, decision = _resolve_basic_pitch_bend_policy(
|
||||
mode=mode,
|
||||
bend_policy=bend_policy,
|
||||
confidence_summary=confidence_summary,
|
||||
note_stability=stability,
|
||||
)
|
||||
if not allow_bends:
|
||||
_strip_pitch_bends(midi_data)
|
||||
|
||||
if jm.should_cancel(job_id):
|
||||
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
|
||||
return
|
||||
@@ -83,14 +105,121 @@ def _run_basic_pitch(job_id: str, input_path: str, mode: str) -> None:
|
||||
dest = output_dir / output_name
|
||||
midi_data.write(str(dest))
|
||||
|
||||
bend_event_count = _count_pitch_bends(midi_data)
|
||||
bend_stats = {
|
||||
"engine": "basic-pitch",
|
||||
"mode": mode,
|
||||
"bend_policy": bend_policy,
|
||||
"bend_policy_decision": decision,
|
||||
"bend_event_count": bend_event_count,
|
||||
"bend_confidence_summary": confidence_summary,
|
||||
"segment_stability": stability,
|
||||
"bends_enabled": bool(allow_bends),
|
||||
}
|
||||
|
||||
log.debug("[%s] basic-pitch complete — output=%s", job_id, dest.name)
|
||||
jm.update_job(
|
||||
job_id,
|
||||
status=jm.JobStatus.DONE,
|
||||
progress="Transcription complete.",
|
||||
files=[{"name": dest.name, "path": str(dest)}],
|
||||
result={"bend_stats": bend_stats},
|
||||
)
|
||||
|
||||
except Exception as exc:
|
||||
log.debug("[%s] basic-pitch failed — %s: %s", job_id, type(exc).__name__, exc)
|
||||
jm.update_job(job_id, status=jm.JobStatus.ERROR, error=str(exc), progress="Error.")
|
||||
|
||||
|
||||
def _normalize_bend_policy(value: str | None) -> str:
|
||||
cleaned = str(value or "auto").strip().lower()
|
||||
return cleaned if cleaned in _BEND_POLICY_VALUES else "auto"
|
||||
|
||||
|
||||
def _summarize_confidence(note_events: Any) -> dict[str, Any]:
|
||||
values: list[float] = []
|
||||
if isinstance(note_events, list):
|
||||
for item in note_events:
|
||||
if isinstance(item, dict):
|
||||
for key in ("confidence", "amplitude", "velocity", "score"):
|
||||
raw = item.get(key)
|
||||
if isinstance(raw, (int, float)):
|
||||
values.append(float(raw))
|
||||
break
|
||||
elif isinstance(item, (tuple, list)) and item:
|
||||
tail = item[-1]
|
||||
if isinstance(tail, (int, float)):
|
||||
values.append(float(tail))
|
||||
|
||||
if not values:
|
||||
return {
|
||||
"count": 0,
|
||||
"avg": None,
|
||||
"min": None,
|
||||
"max": None,
|
||||
}
|
||||
|
||||
return {
|
||||
"count": len(values),
|
||||
"avg": float(mean(values)),
|
||||
"min": float(min(values)),
|
||||
"max": float(max(values)),
|
||||
}
|
||||
|
||||
|
||||
def _estimate_note_stability(midi_data: Any) -> float:
|
||||
deltas: list[float] = []
|
||||
for instrument in getattr(midi_data, "instruments", []):
|
||||
notes = sorted(getattr(instrument, "notes", []) or [], key=lambda n: float(getattr(n, "start", 0.0)))
|
||||
for idx in range(1, len(notes)):
|
||||
prev = notes[idx - 1]
|
||||
cur = notes[idx]
|
||||
prev_pitch = float(getattr(prev, "pitch", 0.0))
|
||||
cur_pitch = float(getattr(cur, "pitch", 0.0))
|
||||
deltas.append(abs(cur_pitch - prev_pitch))
|
||||
|
||||
if not deltas:
|
||||
return 0.0
|
||||
|
||||
# 1.0 = highly stable, 0.0 = highly unstable.
|
||||
avg_delta = float(mean(deltas))
|
||||
return max(0.0, min(1.0, 1.0 - (avg_delta / 12.0)))
|
||||
|
||||
|
||||
def _resolve_basic_pitch_bend_policy(
|
||||
*,
|
||||
mode: str,
|
||||
bend_policy: str,
|
||||
confidence_summary: dict[str, Any],
|
||||
note_stability: float,
|
||||
) -> tuple[bool, str]:
|
||||
if bend_policy == "force_on":
|
||||
return True, "forced_on"
|
||||
if bend_policy == "force_off":
|
||||
return False, "forced_off"
|
||||
|
||||
if mode in {"guitar", "bass"}:
|
||||
return True, "auto_guitar_bass"
|
||||
if mode == "monophonic":
|
||||
return False, "auto_monophonic"
|
||||
|
||||
confidence_avg = confidence_summary.get("avg")
|
||||
enough_confidence = isinstance(confidence_avg, (int, float)) and float(confidence_avg) >= 0.45
|
||||
stable_segments = note_stability >= 0.55
|
||||
if enough_confidence and stable_segments:
|
||||
return True, "auto_adaptive_on"
|
||||
return False, "auto_adaptive_off"
|
||||
|
||||
|
||||
def _strip_pitch_bends(midi_data: Any) -> None:
|
||||
for instrument in getattr(midi_data, "instruments", []):
|
||||
if hasattr(instrument, "pitch_bends"):
|
||||
instrument.pitch_bends = []
|
||||
|
||||
|
||||
def _count_pitch_bends(midi_data: Any) -> int:
|
||||
count = 0
|
||||
for instrument in getattr(midi_data, "instruments", []):
|
||||
bends = getattr(instrument, "pitch_bends", []) or []
|
||||
count += len(bends)
|
||||
return count
|
||||
@@ -0,0 +1,279 @@
|
||||
# StemEngine Windows x64 Installable Executable Plan (Jellyfin-style Network Host)
|
||||
|
||||
Date: 2026-07-08
|
||||
Owner: StemEngine
|
||||
Status: proposed
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Ship StemEngine as an installable Windows x64 application that:
|
||||
|
||||
- runs as a local network-accessible service (LAN reachable), similar to Jellyfin's server model,
|
||||
- hosts the Web UI from the same local install,
|
||||
- installs/upgrades cleanly with an installer,
|
||||
- supports CPU and NVIDIA GPU execution modes,
|
||||
- is observable and recoverable by non-developer users.
|
||||
|
||||
Scope clarification for this local Windows distribution:
|
||||
|
||||
- Frontend target is the standard WebUI only.
|
||||
- SingUI is explicitly out of scope for this installer/runtime plan.
|
||||
- Packaging, QA, and release gates in this document apply to backend + standard WebUI only.
|
||||
|
||||
## 2. Jellyfin-style baseline to emulate
|
||||
|
||||
Jellyfin's model to mirror:
|
||||
|
||||
- server process hosted locally on Windows,
|
||||
- web client served by that process,
|
||||
- default localhost access plus configurable LAN bind,
|
||||
- predictable install location and upgrade path,
|
||||
- optional service semantics (auto-start/always-on behavior).
|
||||
|
||||
Target StemEngine shape:
|
||||
|
||||
- a single installed "StemEngine Server" runtime (backend + static Web UI assets),
|
||||
- browser access at a stable URL (default: http://localhost:3001),
|
||||
- optional LAN access (for other devices on the same network),
|
||||
- optional Windows service mode for headless/always-on operation.
|
||||
|
||||
Frontend note:
|
||||
|
||||
- "Web UI assets" here means the standard WebUI build artifacts.
|
||||
- SingUI assets are not included in the local Windows installer payload for this plan.
|
||||
|
||||
## 3. Packaging strategy
|
||||
|
||||
## 3.1 Runtime packaging approach
|
||||
|
||||
Primary approach (recommended):
|
||||
|
||||
- Freeze backend Python runtime into a packaged executable tree (not one-file), using PyInstaller or Nuitka.
|
||||
- Include prebuilt frontend dist in the install payload.
|
||||
- Run FastAPI with an embedded ASGI server process wrapper.
|
||||
- Bundle ffmpeg binaries required by processing paths.
|
||||
- Keep model assets externalized in a writable data directory to avoid huge installer updates.
|
||||
|
||||
Why this approach:
|
||||
|
||||
- predictable startup and dependency closure on Windows,
|
||||
- straightforward code signing and installer integration,
|
||||
- allows patching backend/frontend without requiring Docker.
|
||||
|
||||
## 3.2 Installer technology
|
||||
|
||||
Use WiX Toolset v4 or NSIS for MSI/EXE installer generation.
|
||||
|
||||
Installer must provide:
|
||||
|
||||
- per-machine install under Program Files,
|
||||
- Start Menu shortcuts,
|
||||
- optional desktop shortcut,
|
||||
- uninstall entry,
|
||||
- upgrade-safe migrations for config/data,
|
||||
- optional firewall rule creation for selected port,
|
||||
- optional service registration checkbox.
|
||||
|
||||
## 3.3 Service and process modes
|
||||
|
||||
Ship two launch modes:
|
||||
|
||||
- Desktop mode: user launches StemEngine app; tray icon and "Open Web UI" action.
|
||||
- Service mode: Windows service starts on boot and runs without user login.
|
||||
|
||||
Service wrapper options:
|
||||
|
||||
- winsw (Windows Service Wrapper) around backend executable, or
|
||||
- native service host in a tiny .NET helper.
|
||||
|
||||
## 4. Configuration and data layout (Windows)
|
||||
|
||||
Recommended directory layout:
|
||||
|
||||
- Binaries: C:\Program Files\StemEngine\
|
||||
- Mutable app data: C:\ProgramData\StemEngine\
|
||||
- Logs: C:\ProgramData\StemEngine\logs\
|
||||
- Settings/config: C:\ProgramData\StemEngine\config\settings.json
|
||||
- Model cache/downloads: C:\ProgramData\StemEngine\models\
|
||||
- Job output/projects: C:\ProgramData\StemEngine\projects\ and jobs\
|
||||
|
||||
Rules:
|
||||
|
||||
- keep Program Files read-only at runtime,
|
||||
- keep all mutable and large data in ProgramData,
|
||||
- preserve ProgramData across upgrades/uninstalls unless user explicitly chooses cleanup.
|
||||
|
||||
## 5. Networking plan (LAN-access similar to Jellyfin)
|
||||
|
||||
## 5.1 Bind behavior
|
||||
|
||||
Default:
|
||||
|
||||
- bind backend to 127.0.0.1 (safe local-only first run).
|
||||
|
||||
Optional LAN mode:
|
||||
|
||||
- bind to 0.0.0.0 with explicit UI toggle,
|
||||
- show warning + trust model text before enabling.
|
||||
|
||||
## 5.2 Port and discovery
|
||||
|
||||
Initial defaults:
|
||||
|
||||
- Web/API port: 3001 (or selected final port),
|
||||
- optional second internal port only if required by split services.
|
||||
|
||||
Future enhancement:
|
||||
|
||||
- optional mDNS/SSDP announcement for discovery on trusted LAN.
|
||||
|
||||
## 5.3 Windows Firewall integration
|
||||
|
||||
Installer option:
|
||||
|
||||
- "Allow StemEngine on private networks" (recommended),
|
||||
- creates inbound TCP rule for configured port/profile,
|
||||
- avoid public-network default opening.
|
||||
|
||||
## 6. GPU/CPU runtime design
|
||||
|
||||
Runtime policy:
|
||||
|
||||
- CPU mode always available.
|
||||
- GPU mode available when NVIDIA runtime is detected and dependencies load.
|
||||
- mismatch fallback: automatic downgrade to CPU with explicit status in UI/logs.
|
||||
|
||||
Packaging notes:
|
||||
|
||||
- keep CUDA-dependent Python wheels aligned and pinned,
|
||||
- provide a startup self-check screen/report listing detected providers,
|
||||
- persist user-selected separation mode in settings and respect it across restarts.
|
||||
|
||||
## 7. Security baseline for LAN mode
|
||||
|
||||
Minimum controls before broad LAN exposure:
|
||||
|
||||
- first-run admin credential setup (or strong local token) when LAN mode is enabled,
|
||||
- CSRF protections for mutating endpoints,
|
||||
- input/path hardening validation retained,
|
||||
- optional CORS narrowing in packaged defaults,
|
||||
- local audit logs for settings changes and remote requests.
|
||||
|
||||
## 8. Build and release pipeline
|
||||
|
||||
## 8.1 CI artifacts
|
||||
|
||||
Produce signed artifacts per release:
|
||||
|
||||
- StemEngine-Windows-x64-Installer.exe (or .msi),
|
||||
- portable zip (advanced users),
|
||||
- checksums + SBOM,
|
||||
- release notes with migration notes.
|
||||
|
||||
## 8.2 Signing
|
||||
|
||||
- sign executables and installer with code-sign certificate,
|
||||
- timestamp signatures,
|
||||
- verify SmartScreen reputation strategy (consistent publisher identity).
|
||||
|
||||
## 8.3 Upgrade mechanics
|
||||
|
||||
- in-place upgrade preserving ProgramData,
|
||||
- schema migration hooks for settings/jobs/projects metadata,
|
||||
- rollback-safe migration checkpoints for critical config changes.
|
||||
|
||||
## 9. Implementation phases
|
||||
|
||||
Phase 0: readiness audit (1-2 weeks)
|
||||
|
||||
- inventory Python runtime deps, GPU deps, ffmpeg requirements,
|
||||
- identify Linux-only assumptions and replace with cross-platform abstractions,
|
||||
- define Windows path/env defaults and migration rules.
|
||||
- confirm frontend packaging target is standard WebUI only (no SingUI build/dependency path).
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- Windows compatibility gap list and remediation tasks approved.
|
||||
|
||||
Phase 1: local Windows runnable build (1-2 weeks)
|
||||
|
||||
- backend launches on Windows via one command,
|
||||
- standard WebUI frontend dist served by backend,
|
||||
- basic processing flow works in CPU mode.
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- end-to-end separation/transcription on local host with persisted settings.
|
||||
|
||||
Phase 2: frozen runtime packaging (2-3 weeks)
|
||||
|
||||
- produce packaged backend runtime (PyInstaller/Nuitka),
|
||||
- embed standard WebUI static assets,
|
||||
- include ffmpeg/runtime probes.
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- portable unpacked build runs on clean Windows x64 VM.
|
||||
|
||||
Phase 3: installer + service mode (2-3 weeks)
|
||||
|
||||
- implement installer,
|
||||
- install/uninstall/upgrade paths,
|
||||
- optional service registration and firewall rule support.
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- clean install, upgrade, uninstall acceptance tests pass.
|
||||
|
||||
Phase 4: LAN + security hardening (2 weeks)
|
||||
|
||||
- LAN toggle UX and guardrails,
|
||||
- authentication hardening for network mode,
|
||||
- request logging and diagnostics UI.
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- LAN access from secondary device works with auth and clear operator controls.
|
||||
|
||||
Phase 5: release engineering + public beta (1-2 weeks)
|
||||
|
||||
- signed artifacts in CI,
|
||||
- beta channel telemetry + crash reporting,
|
||||
- docs and support playbooks.
|
||||
|
||||
Exit criteria:
|
||||
|
||||
- beta quality gate passed; release candidate approved.
|
||||
|
||||
## 10. Validation matrix
|
||||
|
||||
Must-pass test matrix:
|
||||
|
||||
- Windows 11 x64 clean VM install/uninstall/upgrade,
|
||||
- CPU-only machine flow,
|
||||
- NVIDIA GPU machine flow,
|
||||
- standard WebUI functional smoke (dashboard/settings/process flow),
|
||||
- LAN access from second device on private subnet,
|
||||
- service auto-start after reboot,
|
||||
- long-running job stability and log rotation,
|
||||
- data preservation across upgrades.
|
||||
|
||||
## 11. Documentation deliverables
|
||||
|
||||
Required docs before GA:
|
||||
|
||||
- Windows Installation Guide,
|
||||
- Standard WebUI on Windows (scope and entrypoint) Guide,
|
||||
- LAN Access and Security Guide,
|
||||
- GPU Setup and Troubleshooting,
|
||||
- Upgrade and Backup/Restore Guide,
|
||||
- Service Mode Operations Guide,
|
||||
- Known limitations and compatibility matrix (including SingUI out-of-scope for this local build).
|
||||
|
||||
## 12. Immediate next actions
|
||||
|
||||
1. Pick packaging stack: PyInstaller vs Nuitka and WiX vs NSIS.
|
||||
2. Stand up a Windows CI runner for reproducible x64 artifacts.
|
||||
3. Implement Phase 0 audit checklist and assign owners.
|
||||
4. Build a minimal Phase 1 Windows runnable prototype and measure startup/runtime behavior.
|
||||
5. Define LAN-mode auth requirement before opening firewall by default.
|
||||
@@ -0,0 +1,154 @@
|
||||
# StemEngine Image Size Analysis And Reduction Plan (2026-07-09)
|
||||
|
||||
## Scope
|
||||
|
||||
- Target image: `stemengine:latest`
|
||||
- Context: `/home/translator/docker/appdata/stemengine`
|
||||
- Goal: explain why image size is reported around 30 GB and define a practical shrink plan.
|
||||
|
||||
## Agent Guidance Used
|
||||
|
||||
- `docs/agents/agency-agents/adapted/software-architect.md`
|
||||
- `docs/agents/agency-agents/adapted/performance-benchmarker.md`
|
||||
- `docs/agents/agency-agents/adapted/technical-writer.md`
|
||||
- `docs/agents/agency-overrides/release-gate.md`
|
||||
|
||||
Applied interpretation:
|
||||
|
||||
- Prefer incremental, migration-safe changes.
|
||||
- Prioritize measurable size reductions with validation gates.
|
||||
- Keep this plan evidence-based and implementation-ready.
|
||||
|
||||
## Baseline Evidence
|
||||
|
||||
Collected from local Docker host:
|
||||
|
||||
- `docker image ls` / `docker system df -v` report `stemengine:latest` at about `30.3GB`.
|
||||
- `docker image inspect stemengine:latest --format '{{.Size}}'` returned `10890691183` bytes (~10.9 GB payload accounting).
|
||||
- `docker system df -v` also shows `Build cache usage: 18.04GB`.
|
||||
|
||||
Interpretation:
|
||||
|
||||
- You have two size vectors:
|
||||
- Image accounting shown by Docker UI/CLI (`~30.3GB`).
|
||||
- Content payload size from image metadata (`~10.9GB`).
|
||||
- Independent of accounting differences, the image stack is objectively heavyweight and the host also carries large build cache.
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
Primary contributors are visible in image history and Dockerfile design.
|
||||
|
||||
1. Heavy ML dependency layers in a single runtime image.
|
||||
- `RUN python3.11 -m uv pip install --system --no-cache -r /app/backend/requirements.txt` -> `8.61GB`
|
||||
- `RUN ... install torch==2.6.0 torchaudio==2.6.0 torchvision==0.21.0 --index-url .../cu124` -> `5.2GB`
|
||||
|
||||
2. Large GPU runtime base + additional system packages.
|
||||
- Base: `nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04`
|
||||
- Additional apt layer in Dockerfile contributes `1.96GB`.
|
||||
- Current image includes cuDNN runtime base and also installs `libcudnn8` for compatibility.
|
||||
|
||||
3. Dev tooling shipped in production backend image.
|
||||
- Node.js is installed in backend image for frontend watch workflow (`start.sh`).
|
||||
- Build toolchain (`build-essential`, `python3.11-dev`, `git`) is installed in runtime stage.
|
||||
|
||||
4. Monolithic "all capabilities in one image" architecture.
|
||||
- Main backend image carries separation + transcription + optional feature stacks together.
|
||||
- `requirements.txt` includes broad stacks (WhisperX, audio-separator[gpu], transformers, etc.).
|
||||
|
||||
5. Host-level Docker build cache adds major extra storage pressure.
|
||||
- Build cache currently around `18GB`, independent from running container writable layers.
|
||||
|
||||
## Plan To Reduce Size
|
||||
|
||||
## Phase 0: Measurement And Guardrails (No behavior change)
|
||||
|
||||
1. Add a size baseline script and capture per-build metrics:
|
||||
- `docker image inspect` bytes
|
||||
- `docker history` top layers
|
||||
- `docker system df -v` snapshot
|
||||
2. Define acceptance target bands:
|
||||
- `stemengine-runtime-core`: <= 8 GB
|
||||
- `stemengine-runtime-full`: <= 14 GB
|
||||
3. Keep functional parity tests for:
|
||||
- separation endpoints
|
||||
- transcription endpoints
|
||||
- startup health
|
||||
|
||||
## Phase 1: Low-Risk Structural Cuts
|
||||
|
||||
1. Split dev and production Dockerfiles.
|
||||
- Keep current live-reload behavior in `Dockerfile.dev`.
|
||||
- Create `Dockerfile.runtime` without Node.js/watch flow.
|
||||
|
||||
2. Remove dev-only packages from runtime stage.
|
||||
- Move `build-essential`, `python3.11-dev`, `git` to builder stage only.
|
||||
- Keep only runtime libs in final image.
|
||||
|
||||
3. Stop frontend watch responsibilities in backend runtime image.
|
||||
- Continue serving WebUI via existing separate web image (`stemengine-webui`) in production.
|
||||
|
||||
4. Expand `.dockerignore` to shrink build context and cache churn.
|
||||
- Add: `.git`, `tmp`, `jobs`, `uploads`, `imports`, `projects`, `stemengine-models`, `opt`, and local cache folders.
|
||||
|
||||
Expected impact: moderate immediate reduction and better cache efficiency.
|
||||
|
||||
## Phase 2: Dependency Profile Split
|
||||
|
||||
1. Split Python requirements by feature profile:
|
||||
- `requirements-core.txt` (API + essential separation path)
|
||||
- `requirements-separation-gpu.txt`
|
||||
- `requirements-transcription.txt`
|
||||
|
||||
2. Build separate image variants:
|
||||
- `stemengine-core`
|
||||
- `stemengine-full`
|
||||
- optional sidecar-based transcription image
|
||||
|
||||
3. Keep sidecars for optional heavy subsystems (already aligned by architecture):
|
||||
- Omnizart sidecar
|
||||
- Parakeet sidecar
|
||||
- consider WhisperX sidecar to decouple ASR weight from core separation runtime
|
||||
|
||||
Expected impact: largest controllable drop for users who do not need the full stack at all times.
|
||||
|
||||
## Phase 3: CUDA/cuDNN Rationalization
|
||||
|
||||
1. Validate if dual cuDNN footprint can be reduced by workload split:
|
||||
- keep separation runtime on one compatible CUDA/cuDNN stack
|
||||
- isolate components requiring alternate ABI in dedicated image/sidecar
|
||||
|
||||
2. Keep `onnxruntime-gpu==1.20.1` compatibility constraint intact unless a validated migration path exists.
|
||||
|
||||
Expected impact: moderate to high, but requires compatibility testing.
|
||||
|
||||
## Operational Cleanup Plan
|
||||
|
||||
1. Add scheduled cache housekeeping guidance:
|
||||
- `docker builder prune` for stale cache
|
||||
- policy for retaining only active build caches
|
||||
2. Document difference between:
|
||||
- image footprint
|
||||
- container writable layer size
|
||||
- volume/model-cache usage
|
||||
- build cache usage
|
||||
|
||||
## Proposed Execution Order
|
||||
|
||||
1. Implement Phase 1 (fastest, lowest risk).
|
||||
2. Measure size delta and run regression tests.
|
||||
3. Implement Phase 2 profile split.
|
||||
4. Implement/validate Phase 3 only after Phase 2 stabilizes.
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
- Build succeeds for each target image profile.
|
||||
- API health endpoint passes.
|
||||
- At least one GPU separation job succeeds.
|
||||
- At least one transcription job succeeds (when transcription profile enabled).
|
||||
- Reported image size and host disk usage deltas are recorded.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Clear production path that does not require dev/watch tooling in backend image.
|
||||
- Core runtime path substantially below current reported size.
|
||||
- Full runtime path materially reduced and documented with measured deltas.
|
||||
@@ -0,0 +1,488 @@
|
||||
# StemEngine Plan: PSARC to sloppak + Pitch Bend Integration (2026-07-09)
|
||||
|
||||
## Objective
|
||||
|
||||
Plan the implementation of:
|
||||
|
||||
1. PSARC to sloppak conversion in StemEngine with both:
|
||||
- discrete single-item conversion
|
||||
- batch conversion
|
||||
2. Integration of higher-efficiency queue and orchestration patterns observed in Slopsmith engineup.
|
||||
3. Automatic pitch bend detection for MIDI conversion in both basic-pitch and Omnizart flows.
|
||||
4. Pitch bend playback support in StemEngine's MIDI player.
|
||||
5. A lightweight in-app MIDI editor that persists user changes.
|
||||
|
||||
Source reference set analyzed:
|
||||
|
||||
- `E:/Docker/Slopsmith/engineup/slopsmith-plugin-sloppak-converter`
|
||||
- `E:/Docker/Slopsmith/engineup/slopsmith-plugin-midi`
|
||||
- `E:/Docker/Slopsmith/engineup/slopsmith-plugin-notedetect`
|
||||
- StemEngine backend/frontend MIDI and transcription codepaths.
|
||||
|
||||
## Agent Guidance Applied
|
||||
|
||||
- `docs/agents/agency-agents/adapted/software-architect.md`
|
||||
- `docs/agents/agency-agents/adapted/performance-benchmarker.md`
|
||||
- `docs/agents/agency-agents/adapted/technical-writer.md`
|
||||
- `docs/agents/agency-overrides/release-gate.md`
|
||||
|
||||
Interpretation used for this plan:
|
||||
|
||||
- Keep migration incremental and reversible.
|
||||
- Preserve existing APIs/flows unless extension is required.
|
||||
- Define measurable validation gates before closure.
|
||||
|
||||
## Current State (StemEngine)
|
||||
|
||||
1. No existing PSARC or sloppak conversion support in backend or frontend.
|
||||
2. MIDI transcription exists:
|
||||
- basic-pitch in-process (`transcriber_service.py`)
|
||||
- Omnizart sidecar/subprocess (`omnizart_service.py`)
|
||||
3. basic-pitch presets already include `multiple_pitch_bends` for guitar/bass modes in config, but no explicit auto bend policy orchestration across all modes.
|
||||
4. Frontend MIDI player currently schedules note events only (Tone.Part + triggerAttackRelease) and does not parse/play pitch bend events.
|
||||
5. No in-app MIDI editing workflow currently exists for note/bend edits and save-back.
|
||||
|
||||
## External Capability Map (Slopsmith engineup)
|
||||
|
||||
From `slopsmith-plugin-sloppak-converter`:
|
||||
|
||||
1. Queue-backed conversion jobs with explicit lifecycle states.
|
||||
2. Discrete enqueue + bulk enqueue API split.
|
||||
3. Operational controls: pause/resume, cancel queued, retry failed, clear finished.
|
||||
4. Live WebSocket queue/job updates with snapshot on connect.
|
||||
5. Queue persistence to disk and resume behavior after restart.
|
||||
6. Robust path validation and traversal guards.
|
||||
7. Thread-offloaded sync filesystem checks for large bulk operations.
|
||||
8. Result dedupe/skip reasons (`already_queued`, `already_running`, `already_converted`, etc.).
|
||||
9. Optional stem splitting and sidecar-friendly architecture.
|
||||
|
||||
From `slopsmith-plugin-notedetect` and plugin patterns:
|
||||
|
||||
1. Strong use of capability gating and fail-soft behavior.
|
||||
2. Practical support for multiple detection modes with deterministic fallback.
|
||||
3. Real-time event/state reporting suitable for user feedback.
|
||||
|
||||
## Architecture Rationale (3-6 bullets)
|
||||
|
||||
1. Reuse StemEngine's existing job manager and job storage model instead of introducing a separate queue subsystem.
|
||||
2. Introduce converter functionality as a new backend service module with a dedicated router, keeping project/transcription APIs stable.
|
||||
3. Extend MIDI metadata and frontend player types to include bend envelopes without breaking note-only playback.
|
||||
4. Add feature flags and capability probes for conversion dependencies (vgmstream/ffmpeg/optional split engine) and bend extraction.
|
||||
5. Keep heavy conversion and bend extraction work in background jobs to avoid blocking request handlers.
|
||||
6. Keep MIDI editing lightweight and deterministic (event editing only, no full DAW scope).
|
||||
|
||||
## Proposed Implementation Plan
|
||||
|
||||
## Phase 1: Converter Foundations (Backend)
|
||||
|
||||
1. Add new service module set:
|
||||
- `backend/services/psarc_sloppak_service.py`
|
||||
- `backend/services/psarc_sloppak_queue_service.py` (or integrated with job_manager wrappers)
|
||||
2. Add new router:
|
||||
- `backend/routers/sloppak_converter.py`
|
||||
3. Add API endpoints (v1):
|
||||
- `POST /api/sloppak/enqueue` (single/discrete)
|
||||
- `POST /api/sloppak/enqueue-bulk` (batch)
|
||||
- `GET /api/sloppak/jobs`
|
||||
- `POST /api/sloppak/jobs/cancel-queued`
|
||||
- `POST /api/sloppak/jobs/retry-failed`
|
||||
- `POST /api/sloppak/jobs/clear-finished`
|
||||
- `POST /api/sloppak/pause`
|
||||
- `POST /api/sloppak/resume`
|
||||
4. Input safety and dedupe:
|
||||
- enforce relative paths under configured root
|
||||
- reject path traversal and absolute paths
|
||||
- return explicit skip reasons for duplicates and missing files
|
||||
5. Add conversion result schema to job files metadata:
|
||||
- output path
|
||||
- generated stem list
|
||||
- output size
|
||||
- elapsed time
|
||||
|
||||
## Phase 2: Converter UX + Batch Operations (Frontend)
|
||||
|
||||
1. Add Conversions screen/page with queue table.
|
||||
2. Add discrete convert action in relevant file/project rows.
|
||||
3. Add batch selection + enqueue flow.
|
||||
4. Add queue control actions:
|
||||
- pause/resume
|
||||
- cancel queued
|
||||
- retry failed
|
||||
- clear finished
|
||||
5. Add polling first, then optional WebSocket live updates if needed for UX smoothness.
|
||||
|
||||
## Phase 3: Efficiency Integrations from engineup
|
||||
|
||||
Port efficiency patterns that are superior to current baseline:
|
||||
|
||||
1. Persistence of converter queue state across restart.
|
||||
2. Thread-offloaded preflight checks for large batch enqueue operations.
|
||||
3. Skip-reason taxonomy and idempotent enqueue semantics.
|
||||
4. Job snapshot + update stream for low-latency UI updates.
|
||||
5. Optional dependency capability flag in jobs snapshot, so UI can degrade gracefully.
|
||||
|
||||
Note:
|
||||
|
||||
- StemEngine already has strong background execution primitives in `job_manager.py`; this phase is primarily about adapter behavior and UX semantics, not replacing core executors.
|
||||
|
||||
## Phase 4: Automatic Pitch Bend Detection in MIDI Conversion
|
||||
|
||||
### 4A. basic-pitch path
|
||||
|
||||
1. Add explicit bend policy option layer per mode:
|
||||
- `auto` (default)
|
||||
- `force_on`
|
||||
- `force_off`
|
||||
2. Define auto policy:
|
||||
- guitar/bass modes: enabled
|
||||
- monophonic vocal strict: disabled by default unless confidence conditions are met
|
||||
- polyphonic/melody: adaptive based on confidence/segment stability
|
||||
3. Persist bend extraction diagnostics in job metadata:
|
||||
- bend_event_count
|
||||
- bend_confidence_summary
|
||||
|
||||
### 4B. Omnizart path
|
||||
|
||||
1. Add post-process bend derivation from Omnizart output where native bends are absent:
|
||||
- for `omnizart-vocal-contour`, derive pitch wheel events from F0 contour to nearest active note context
|
||||
- for other modes, keep opt-in derivation path with confidence gating
|
||||
2. Add mode-aware bend derivation guardrails:
|
||||
- avoid over-bending on dense polyphonic sections
|
||||
- cap bend range and smoothing windows
|
||||
3. Store derivation metadata in job result for transparency/debugging.
|
||||
|
||||
## Phase 5: MIDI Player Pitch Bend Playback
|
||||
|
||||
1. Extend frontend MIDI types in `ProjectDetail.tsx` to include track/channel bend events.
|
||||
2. Add bend scheduling pipeline:
|
||||
- parse pitch bend events from `@tonejs/midi` tracks
|
||||
- schedule bend curves in sync with notes
|
||||
3. Implement synth-side bend application strategy:
|
||||
- minimum viable: per-stem channel bend stream applied to active synth voice group
|
||||
- improved: per-track/per-channel synth partitioning for reduced bend cross-talk
|
||||
4. Add UI toggle:
|
||||
- Pitch Bend Playback On/Off (default on)
|
||||
- optional bend intensity scaling for audition
|
||||
|
||||
## Phase 6: Lightweight MIDI Editor + Save
|
||||
|
||||
1. Add MIDI editor panel in project detail view:
|
||||
- selectable track/channel
|
||||
- piano-roll lane with basic note blocks
|
||||
- bend lane for pitch wheel curve points
|
||||
2. Minimum edit operations (v1):
|
||||
- add/remove/move/resize note events
|
||||
- velocity adjust
|
||||
- add/move/remove bend points
|
||||
- quantize selection (coarse presets only)
|
||||
3. Persistence model:
|
||||
- keep edits as an internal normalized event list while editing
|
||||
- save action writes updated `.mid` artifact via backend endpoint
|
||||
- preserve original source MIDI and write an edited sibling output for rollback safety
|
||||
4. New API endpoints (v1):
|
||||
- `POST /api/midi/edit/sessions` (open edit session for source midi)
|
||||
- `PATCH /api/midi/edit/sessions/{session_id}` (apply edit ops)
|
||||
- `POST /api/midi/edit/sessions/{session_id}/save` (materialize edited midi)
|
||||
- `DELETE /api/midi/edit/sessions/{session_id}` (discard edit session)
|
||||
5. Save metadata:
|
||||
- editor version
|
||||
- source midi path/hash
|
||||
- edit timestamp
|
||||
- optional edit summary (counts of note/bend ops)
|
||||
|
||||
## Data Model and API Additions
|
||||
|
||||
1. Job metadata extensions:
|
||||
- `conversion_kind` (`psarc_to_sloppak`)
|
||||
- `skip_reason`
|
||||
- `bend_stats`
|
||||
2. MIDI editor metadata extensions:
|
||||
- `edited_midi_path`
|
||||
- `source_midi_path`
|
||||
- `midi_edit_summary`
|
||||
- `midi_editor_version`
|
||||
3. Optional app settings additions:
|
||||
- converter roots and dependency flags
|
||||
- bend policy defaults
|
||||
- editor snap/quantize defaults
|
||||
4. Response contracts should remain additive to avoid breaking current clients.
|
||||
|
||||
## Validation Plan
|
||||
|
||||
## Converter Validation
|
||||
|
||||
1. Single PSARC conversion success path.
|
||||
2. Batch conversion with mixed valid/invalid/already-converted entries and reason-code verification.
|
||||
3. Pause/resume behavior and cancellation semantics.
|
||||
4. Restart recovery of queued/in-progress converter jobs.
|
||||
5. Path traversal and absolute-path rejection tests.
|
||||
|
||||
## Pitch Bend Validation
|
||||
|
||||
1. basic-pitch guitar/bass outputs include pitch bend events when expected.
|
||||
2. Omnizart contour-derived bends generated with confidence thresholds.
|
||||
3. Frontend MIDI playback audibly follows bends and remains synchronized.
|
||||
4. No regression in note-only playback when bends absent.
|
||||
|
||||
## MIDI Editor Validation
|
||||
|
||||
1. User can perform note edits and save an updated midi artifact.
|
||||
2. User can perform bend edits and hear them during playback.
|
||||
3. Save operation preserves original midi and writes edited sibling output.
|
||||
4. Reloading project displays edited midi artifact and updated playback behavior.
|
||||
5. Session discard path leaves source midi unchanged.
|
||||
|
||||
## Required checks before closure
|
||||
|
||||
1. Backend compile checks for touched modules.
|
||||
2. Frontend typecheck for touched UI/player files.
|
||||
3. Evidence captured in job output samples and manual playback verification notes.
|
||||
4. Edited midi regression sample checked in test fixtures (or documented local artifact path).
|
||||
|
||||
## Non-Goals (for first delivery)
|
||||
|
||||
1. Full Slopsmith plugin runtime compatibility layer inside StemEngine.
|
||||
2. Perfect per-note MPE-grade bend rendering for dense polyphonic MIDI on first pass.
|
||||
3. Replacing existing StemEngine job manager architecture.
|
||||
4. Building a full DAW timeline with automation lanes beyond notes + pitch bend.
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
1. Dependency complexity for PSARC conversion stack.
|
||||
- Mitigation: explicit capability probing + fail-soft UI messaging.
|
||||
2. Bend overfitting or artifacts in polyphonic material.
|
||||
- Mitigation: conservative default auto policy + user toggle.
|
||||
3. Playback CPU overhead from bend scheduling.
|
||||
- Mitigation: throttle/simplify bend control points and offer UI disable toggle.
|
||||
4. Invalid MIDI serialization after complex edit sequences.
|
||||
- Mitigation: backend-side schema validation and round-trip parse checks before save success.
|
||||
|
||||
## Proposed Delivery Slices
|
||||
|
||||
1. Slice A: Converter backend + discrete endpoint + minimal frontend trigger.
|
||||
2. Slice B: Batch enqueue + queue controls + reason codes.
|
||||
3. Slice C: basic-pitch auto bend policy + metadata.
|
||||
4. Slice D: Omnizart bend derivation + safeguards.
|
||||
5. Slice E: MIDI player bend playback + UI controls.
|
||||
6. Slice F: lightweight MIDI editor UI + edit session endpoints.
|
||||
7. Slice G: save/discard workflow + artifact persistence + regression validation.
|
||||
|
||||
This sequencing keeps value incremental while containing regression risk.
|
||||
|
||||
## Implementation Update (2026-07-09)
|
||||
|
||||
Status update for requested execution order:
|
||||
|
||||
1. Slice F implemented (lightweight MIDI editor v1).
|
||||
2. Slice A implemented (converter backend + discrete enqueue).
|
||||
3. Slice B implemented (batch enqueue + queue controls + reasoned skip behavior).
|
||||
|
||||
### Implemented artifacts
|
||||
|
||||
Backend:
|
||||
|
||||
- `backend/routers/midi_editor.py`
|
||||
- `backend/services/midi_editor_service.py`
|
||||
- `backend/routers/sloppak_converter.py`
|
||||
- `backend/services/sloppak_converter_service.py`
|
||||
- `backend/main.py` (router registration)
|
||||
|
||||
Frontend:
|
||||
|
||||
- `frontend/src/components/MidiLightEditor.tsx`
|
||||
- `frontend/src/components/SloppakConverterPanel.tsx`
|
||||
- `frontend/src/pages/ProjectDetail.tsx` (integration points)
|
||||
- `frontend/src/api/client.ts` (new API client methods/types)
|
||||
- `frontend/src/types.ts` (new job_type values)
|
||||
|
||||
### Notes on Slice F scope delivered
|
||||
|
||||
- Session-based MIDI edit lifecycle is live:
|
||||
- open session from an existing transcribe job MIDI file
|
||||
- apply note/bend edit operations
|
||||
- save to a new sibling MIDI artifact (source remains unchanged)
|
||||
- discard session
|
||||
- Frontend editor supports lightweight note and bend edits, quantize, and save/download.
|
||||
|
||||
### Notes on Slices A/B scope delivered
|
||||
|
||||
- Converter queue API is live for:
|
||||
- single enqueue
|
||||
- bulk enqueue
|
||||
- pause/resume
|
||||
- cancel queued
|
||||
- retry failed
|
||||
- clear finished
|
||||
- Frontend queue panel is integrated in Project Detail with controls and download links.
|
||||
- Current conversion implementation writes a valid `.sloppak` zip package containing:
|
||||
- `manifest.json`
|
||||
- original `.psarc` payload under `source/`
|
||||
|
||||
This satisfies A/B operational flow and queue semantics; deeper Rocksmith-aware extraction/normalization can be layered as a follow-on enhancement.
|
||||
|
||||
### Validation evidence
|
||||
|
||||
- Backend compile check:
|
||||
|
||||
### Operationalization follow-up (2026-07-09)
|
||||
|
||||
To make deep extraction deployable without code edits:
|
||||
|
||||
1. Added runtime capability endpoint:
|
||||
- `GET /api/sloppak/capabilities`
|
||||
- reports unpacker availability and converter root readiness.
|
||||
2. Added environment wiring/documentation:
|
||||
- `stemengine.yml`: `SLOPPAK_SOURCE_ROOT`, `SLOPPAK_OUTPUT_ROOT`, `SLOPPAK_PSARC_UNPACK_CMD`
|
||||
- `.env.example`: includes concrete command-template example
|
||||
- `README.md`: setup + verification instructions
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- `python3 -m compileall backend/routers/sloppak_converter.py backend/services/sloppak_converter_service.py`
|
||||
- `python3 -m compileall backend/main.py backend/routers/sloppak_converter.py backend/routers/midi_editor.py backend/services/sloppak_converter_service.py backend/services/midi_editor_service.py`
|
||||
- Frontend typecheck:
|
||||
- `cd frontend && npm run -s typecheck`
|
||||
|
||||
## Follow-up Implementation Update (2026-07-09, later)
|
||||
|
||||
Requested enhancement implemented:
|
||||
|
||||
- Added deeper Rocksmith-aware extraction/normalization pass to converter packaging logic.
|
||||
|
||||
Implemented in:
|
||||
|
||||
- `backend/services/sloppak_converter_service.py`
|
||||
|
||||
Behavior added:
|
||||
|
||||
1. Optional PSARC unpack pipeline with graceful fallback:
|
||||
- preferred: `patcher.unpack_psarc` (when available)
|
||||
- fallback: env command template via `SLOPPAK_PSARC_UNPACK_CMD`
|
||||
2. Normalized metadata extraction from unpacked payload:
|
||||
- song fields: title/artist/album/year
|
||||
- arrangements: id/name/tuning/capo (deduplicated)
|
||||
- WEM inventory list and count
|
||||
3. Lyrics extraction from Rocksmith vocals XML (`<vocals><vocal .../>`), when present.
|
||||
4. Enriched `.sloppak` package contents:
|
||||
- `manifest.json` now includes normalized `metadata`, `rocksmith.unpack`, `rocksmith.arrangements`, and `rocksmith.audio` sections.
|
||||
- `metadata/rocksmith-summary.json` added to package.
|
||||
- `lyrics.json` added when lyrics were extractable.
|
||||
|
||||
Compatibility notes:
|
||||
|
||||
- Conversion remains successful even when unpack tools are unavailable.
|
||||
- In no-unpacker mode, package still includes source PSARC baseline content and reports unpack status in manifest.
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- `python3 -m compileall backend/services/sloppak_converter_service.py`
|
||||
|
||||
## Follow-up Implementation Update (2026-07-09, later) — Slices C, D, and E
|
||||
|
||||
Requested next slices implemented:
|
||||
|
||||
- Slice C: basic-pitch auto bend policy + bend diagnostics metadata.
|
||||
- Slice D: Omnizart bend derivation with guardrails + metadata transparency.
|
||||
- Slice E: frontend MIDI player pitch bend playback + UI controls.
|
||||
|
||||
### Slice C (basic-pitch) delivered
|
||||
|
||||
Implemented in:
|
||||
|
||||
- `backend/services/transcriber_service.py`
|
||||
- `backend/routers/transcribe.py`
|
||||
- `backend/routers/projects.py`
|
||||
- `backend/services/project_reprocess_service.py`
|
||||
|
||||
Behavior added:
|
||||
|
||||
1. New bend policy channel for transcription jobs:
|
||||
- `auto` (default)
|
||||
- `force_on`
|
||||
- `force_off`
|
||||
2. Auto-policy orchestration:
|
||||
- guitar/bass -> bends enabled
|
||||
- monophonic -> bends disabled
|
||||
- polyphonic/melody -> adaptive decision based on confidence summary + note stability
|
||||
3. Bend diagnostics persisted to job result metadata under `result.bend_stats`:
|
||||
- `bend_event_count`
|
||||
- `bend_confidence_summary`
|
||||
- `segment_stability`
|
||||
- policy decision fields
|
||||
|
||||
### Slice D (Omnizart) delivered
|
||||
|
||||
Implemented in:
|
||||
|
||||
- `backend/services/omnizart_service.py`
|
||||
|
||||
Behavior added:
|
||||
|
||||
1. Omnizart path now receives and honors bend policy (`auto|force_on|force_off`).
|
||||
2. Post-process derivation path when native bends are absent:
|
||||
- `auto`: derive for `vocal-contour`
|
||||
- `force_on`: derive for all Omnizart modes
|
||||
- `force_off`: never derive
|
||||
3. Guardrails to avoid over-bending:
|
||||
- dense polyphonic detection gate
|
||||
- capped semitone delta and MIDI bend value range
|
||||
- transition-only derivation (no uncontrolled continuous drift)
|
||||
4. Result metadata transparency in `result.bend_stats`:
|
||||
- native vs derived bend counts
|
||||
- final bend count
|
||||
- derivation applied/reason
|
||||
|
||||
### Slice E (frontend playback) delivered
|
||||
|
||||
Implemented in:
|
||||
|
||||
- `frontend/src/pages/ProjectDetail.tsx`
|
||||
|
||||
Behavior added:
|
||||
|
||||
1. MIDI parser/playback now reads `pitchBends` from `@tonejs/midi` track data.
|
||||
2. Pitch bend events are scheduled into playback transport per stem.
|
||||
3. New user controls in MIDI playback panel:
|
||||
- Bend playback toggle (On/Off, default On)
|
||||
- Bend intensity scaling slider (0-200%)
|
||||
4. Playback status now reports bend event counts alongside note counts.
|
||||
|
||||
Validation evidence:
|
||||
|
||||
- Backend compile:
|
||||
- `python3 -m compileall backend/services/transcriber_service.py backend/services/omnizart_service.py backend/routers/transcribe.py backend/routers/projects.py backend/services/project_reprocess_service.py`
|
||||
- Frontend typecheck:
|
||||
- `cd frontend && npm run -s typecheck`
|
||||
|
||||
## Remaining Work Checklist (as of 2026-07-09)
|
||||
|
||||
Completion status: implemented and closed.
|
||||
|
||||
- [x] Dedicated conversions page added (in addition to Project Detail panel):
|
||||
- `frontend/src/pages/Conversions.tsx`
|
||||
- route: `/conversions`
|
||||
- sidebar navigation entry added
|
||||
|
||||
- [x] Slice G closure evidence bundle captured:
|
||||
- Save/discard/source-immutability regression executed in runtime container (`python3.11`):
|
||||
- source: `/app/media/jobs/midi_regression_evidence/regression_source.mid`
|
||||
- edited output: `/app/media/jobs/midi_regression_evidence/regression_source.edited.mid`
|
||||
- `source_unchanged_after_save=true`
|
||||
- `source_unchanged_after_discard=true`
|
||||
|
||||
- [x] Phase 3 efficiency backlog delivered:
|
||||
- queue persistence/recovery on restart via `ensure_queue_hydrated()` + startup hook in `backend/main.py`
|
||||
- thread-offloaded batch preflight checks in `enqueue_bulk` (`ThreadPoolExecutor` path)
|
||||
- low-latency snapshot/update stream:
|
||||
- `GET /api/sloppak/snapshot`
|
||||
- `GET /api/sloppak/updates` (SSE)
|
||||
- frontend panel subscribes via `EventSource` and refetches on snapshot events
|
||||
|
||||
- [x] Validation-plan evidence captured in this plan doc:
|
||||
- converter hydration/snapshot runtime checks (container):
|
||||
- `ensure_queue_hydrated() -> {'resubmitted': 0, 'requeued_running': 0}`
|
||||
- snapshot contains paused/jobs/capabilities payload
|
||||
- update stream runtime check (container):
|
||||
- first chunks emitted: `retry: 1500`, `event: snapshot`
|
||||
- frontend validation:
|
||||
- `cd frontend && npm run -s typecheck`
|
||||
@@ -5,6 +5,7 @@ import { isFeatureEnabled, MOBILE_KARAOKE_BOX_V1 } from './utils/featureFlags'
|
||||
import MobileKaraokeShell from './components/singui/MobileKaraokeShell'
|
||||
import Separator from './pages/Separator'
|
||||
import ProjectDetail from './pages/ProjectDetail'
|
||||
import ConversionsPage from './pages/Conversions'
|
||||
import SettingsPage from './pages/Settings'
|
||||
import { ThemeProvider } from './context/ThemeContext'
|
||||
import { QueueProvider } from './context/QueueContext'
|
||||
@@ -24,6 +25,7 @@ import {
|
||||
PLAYLISTS_ROUTE,
|
||||
SING_ROUTE,
|
||||
SETTINGS_ROUTE,
|
||||
CONVERSIONS_ROUTE,
|
||||
VISUALIZER_LAB_ROUTE,
|
||||
KARAOKE_ROUTE,
|
||||
KARAOKE_TV_ROUTE,
|
||||
@@ -226,6 +228,7 @@ function WebUiShell() {
|
||||
}
|
||||
/>
|
||||
<Route path="/projects/:projectId" element={<ProjectDetailRoute />} />
|
||||
<Route path={CONVERSIONS_ROUTE} element={<ConversionsPage />} />
|
||||
<Route path={SETTINGS_ROUTE} element={<SettingsPage />} />
|
||||
<Route path="*" element={<Navigate to="/" replace />} />
|
||||
</Routes>
|
||||
|
||||
+149
-4
@@ -229,10 +229,11 @@ export const submitSeparate = (
|
||||
export const getTranscribeModes = () =>
|
||||
request<ModeOption[]>(`${BASE}/transcribe/modes`)
|
||||
|
||||
export const submitTranscribe = (file: File, mode: string) => {
|
||||
export const submitTranscribe = (file: File, mode: string, bendPolicy = 'auto') => {
|
||||
const fd = new FormData()
|
||||
fd.append('file', file)
|
||||
fd.append('mode', mode)
|
||||
fd.append('bend_policy', bendPolicy)
|
||||
return request<{ job_id: string }>(`${BASE}/transcribe`, { method: 'POST', body: fd })
|
||||
}
|
||||
|
||||
@@ -348,10 +349,11 @@ export const submitLrcAlignExisting = (
|
||||
|
||||
// ---------- Quick-action helpers (from existing project/job on server) ----------
|
||||
|
||||
export const submitTranscribeFromProject = (projectName: string, mode = 'music') => {
|
||||
export const submitTranscribeFromProject = (projectName: string, mode = 'music', bendPolicy = 'auto') => {
|
||||
const fd = new FormData()
|
||||
fd.append('project_name', projectName)
|
||||
fd.append('mode', mode)
|
||||
fd.append('bend_policy', bendPolicy)
|
||||
return request<{ job_id: string; stem: string }>(`${BASE}/transcribe/from-project`, { method: 'POST', body: fd })
|
||||
}
|
||||
|
||||
@@ -600,11 +602,11 @@ export const getProjectGloss = (id: string, targetLang: string, sourceLang?: str
|
||||
export const deleteProject = (id: string) =>
|
||||
request<{ detail: string }>(`${BASE}/projects/${id}`, { method: 'DELETE' })
|
||||
|
||||
export const transcribeProject = (id: string, stemName: string, mode: string) =>
|
||||
export const transcribeProject = (id: string, stemName: string, mode: string, bendPolicy = 'auto') =>
|
||||
request<{ job_id: string }>(`${BASE}/projects/${id}/transcribe`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ stem_name: stemName, mode }),
|
||||
body: JSON.stringify({ stem_name: stemName, mode, bend_policy: bendPolicy }),
|
||||
})
|
||||
|
||||
export const abcProject = (id: string, stemName: string, songTitle?: string) =>
|
||||
@@ -1462,3 +1464,146 @@ export const deletePixabayCacheEntry = (key: string) =>
|
||||
/** Clear the entire Pixabay video cache. */
|
||||
export const clearPixabayCache = () =>
|
||||
request<{ deleted: string }>(`${BASE}/pixabay/cache`, { method: 'DELETE' })
|
||||
|
||||
// ── Sloppak converter ───────────────────────────────────────────────────────
|
||||
|
||||
export interface SloppakJobListResponse {
|
||||
paused: boolean
|
||||
jobs: Job[]
|
||||
}
|
||||
|
||||
export interface SloppakEnqueueResult {
|
||||
ok?: boolean
|
||||
skipped?: boolean
|
||||
filename?: string
|
||||
reason?: string
|
||||
job_id?: string
|
||||
output_path?: string
|
||||
}
|
||||
|
||||
export interface SloppakBulkEnqueueResult {
|
||||
enqueued: SloppakEnqueueResult[]
|
||||
skipped: SloppakEnqueueResult[]
|
||||
summary: {
|
||||
requested: number
|
||||
enqueued: number
|
||||
skipped: number
|
||||
}
|
||||
}
|
||||
|
||||
export const listSloppakJobs = () =>
|
||||
request<SloppakJobListResponse>(`${BASE}/sloppak/jobs`)
|
||||
|
||||
export const getSloppakSnapshot = () =>
|
||||
request<SloppakJobListResponse & { capabilities: Record<string, unknown> }>(`${BASE}/sloppak/snapshot`)
|
||||
|
||||
export const hydrateSloppakQueue = () =>
|
||||
request<{ resubmitted: number; requeued_running: number }>(`${BASE}/sloppak/hydrate`, { method: 'POST' })
|
||||
|
||||
export const enqueueSloppak = (filename: string, reconvert = false) =>
|
||||
request<SloppakEnqueueResult>(`${BASE}/sloppak/enqueue`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ filename, reconvert }),
|
||||
})
|
||||
|
||||
export const enqueueSloppakBulk = (filenames: string[], reconvert = false) =>
|
||||
request<SloppakBulkEnqueueResult>(`${BASE}/sloppak/enqueue-bulk`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ filenames, reconvert }),
|
||||
})
|
||||
|
||||
export const pauseSloppakQueue = () =>
|
||||
request<{ paused: boolean }>(`${BASE}/sloppak/pause`, { method: 'POST' })
|
||||
|
||||
export const resumeSloppakQueue = () =>
|
||||
request<{ paused: boolean }>(`${BASE}/sloppak/resume`, { method: 'POST' })
|
||||
|
||||
export const cancelQueuedSloppak = () =>
|
||||
request<{ cancelled: number }>(`${BASE}/sloppak/jobs/cancel-queued`, { method: 'POST' })
|
||||
|
||||
export const retryFailedSloppak = () =>
|
||||
request<{ retried: number; skipped: number }>(`${BASE}/sloppak/jobs/retry-failed`, { method: 'POST' })
|
||||
|
||||
export const clearFinishedSloppak = () =>
|
||||
request<{ deleted: number }>(`${BASE}/sloppak/jobs/clear-finished`, { method: 'POST' })
|
||||
|
||||
// ── MIDI light editor ───────────────────────────────────────────────────────
|
||||
|
||||
export interface MidiEditorNote {
|
||||
id: string
|
||||
pitch: number
|
||||
start: number
|
||||
duration: number
|
||||
velocity: number
|
||||
}
|
||||
|
||||
export interface MidiEditorBend {
|
||||
id: string
|
||||
time: number
|
||||
value: number
|
||||
}
|
||||
|
||||
export interface MidiEditorTrack {
|
||||
index: number
|
||||
name: string
|
||||
program: number
|
||||
is_drum: boolean
|
||||
notes: MidiEditorNote[]
|
||||
bends: MidiEditorBend[]
|
||||
}
|
||||
|
||||
export interface MidiEditorSession {
|
||||
id: string
|
||||
created_at: string
|
||||
source_job_id: string
|
||||
source_filename: string
|
||||
source_path: string
|
||||
editor_version: string
|
||||
tracks: MidiEditorTrack[]
|
||||
last_saved_path?: string
|
||||
last_saved_job_id?: string
|
||||
}
|
||||
|
||||
export interface MidiEditOp {
|
||||
op: string
|
||||
track_index: number
|
||||
note_id?: string
|
||||
bend_id?: string
|
||||
pitch?: number
|
||||
start?: number
|
||||
duration?: number
|
||||
velocity?: number
|
||||
time?: number
|
||||
value?: number
|
||||
step?: number
|
||||
}
|
||||
|
||||
export const createMidiEditSession = (jobId: string, filename?: string) =>
|
||||
request<MidiEditorSession>(`${BASE}/midi/edit/sessions`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ job_id: jobId, filename }),
|
||||
})
|
||||
|
||||
export const getMidiEditSession = (sessionId: string) =>
|
||||
request<MidiEditorSession>(`${BASE}/midi/edit/sessions/${encodeURIComponent(sessionId)}`)
|
||||
|
||||
export const patchMidiEditSession = (sessionId: string, ops: MidiEditOp[]) =>
|
||||
request<MidiEditorSession>(`${BASE}/midi/edit/sessions/${encodeURIComponent(sessionId)}`, {
|
||||
method: 'PATCH',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ ops }),
|
||||
})
|
||||
|
||||
export const saveMidiEditSession = (sessionId: string) =>
|
||||
request<{ job_id: string; filename: string; path: string; summary: { track_count: number; note_count: number; bend_count: number } }>(
|
||||
`${BASE}/midi/edit/sessions/${encodeURIComponent(sessionId)}/save`,
|
||||
{ method: 'POST' }
|
||||
)
|
||||
|
||||
export const discardMidiEditSession = (sessionId: string) =>
|
||||
request<{ discarded: boolean }>(`${BASE}/midi/edit/sessions/${encodeURIComponent(sessionId)}`, {
|
||||
method: 'DELETE',
|
||||
})
|
||||
@@ -0,0 +1,282 @@
|
||||
import { useMemo, useState } from 'react'
|
||||
import { Download, Loader2, Save, Trash2 } from 'lucide-react'
|
||||
import {
|
||||
createMidiEditSession,
|
||||
discardMidiEditSession,
|
||||
patchMidiEditSession,
|
||||
saveMidiEditSession,
|
||||
type MidiEditOp,
|
||||
type MidiEditorSession,
|
||||
} from '../api/client'
|
||||
import { downloadUrl } from '../api/client'
|
||||
|
||||
interface StemRef {
|
||||
label: string
|
||||
jobId: string
|
||||
filename: string
|
||||
}
|
||||
|
||||
export default function MidiLightEditor({ stem }: { stem: StemRef }) {
|
||||
const [open, setOpen] = useState(false)
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [err, setErr] = useState<string | null>(null)
|
||||
const [info, setInfo] = useState<string | null>(null)
|
||||
const [session, setSession] = useState<MidiEditorSession | null>(null)
|
||||
const [trackIndex, setTrackIndex] = useState(0)
|
||||
const [quantize, setQuantize] = useState(0.25)
|
||||
const [savedJobId, setSavedJobId] = useState<string | null>(null)
|
||||
const [savedFilename, setSavedFilename] = useState<string | null>(null)
|
||||
|
||||
const activeTrack = useMemo(() => {
|
||||
if (!session) return null
|
||||
return session.tracks[trackIndex] ?? null
|
||||
}, [session, trackIndex])
|
||||
|
||||
async function openEditor() {
|
||||
setBusy(true)
|
||||
setErr(null)
|
||||
setInfo(null)
|
||||
try {
|
||||
const created = await createMidiEditSession(stem.jobId, stem.filename)
|
||||
setSession(created)
|
||||
setTrackIndex(0)
|
||||
setOpen(true)
|
||||
setSavedJobId(null)
|
||||
setSavedFilename(null)
|
||||
} catch (e) {
|
||||
setErr(String(e))
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function applyOps(ops: MidiEditOp[]) {
|
||||
if (!session) return
|
||||
setBusy(true)
|
||||
setErr(null)
|
||||
try {
|
||||
const updated = await patchMidiEditSession(session.id, ops)
|
||||
setSession(updated)
|
||||
} catch (e) {
|
||||
setErr(String(e))
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSave() {
|
||||
if (!session) return
|
||||
setBusy(true)
|
||||
setErr(null)
|
||||
setInfo(null)
|
||||
try {
|
||||
const res = await saveMidiEditSession(session.id)
|
||||
setSavedJobId(res.job_id)
|
||||
setSavedFilename(res.filename)
|
||||
setInfo(`Saved MIDI (${res.summary.note_count} notes, ${res.summary.bend_count} bends).`)
|
||||
} catch (e) {
|
||||
setErr(String(e))
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function closeEditor(discard: boolean) {
|
||||
if (discard && session) {
|
||||
try {
|
||||
await discardMidiEditSession(session.id)
|
||||
} catch (e) {
|
||||
setErr(String(e))
|
||||
}
|
||||
}
|
||||
setOpen(false)
|
||||
setSession(null)
|
||||
setTrackIndex(0)
|
||||
}
|
||||
|
||||
if (!open) {
|
||||
return (
|
||||
<button
|
||||
onClick={() => void openEditor()}
|
||||
disabled={busy}
|
||||
className="rounded-lg border ui-border px-2 py-1 text-[10px] ui-text-muted hover:border-brand-400 disabled:opacity-50"
|
||||
>
|
||||
{busy ? <Loader2 className="w-3 h-3 animate-spin" /> : 'Edit MIDI'}
|
||||
</button>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="rounded-lg border ui-border p-3 space-y-3 bg-slate-50/50 dark:bg-gray-900/40">
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<span className="text-xs font-medium ui-text">MIDI Editor · {stem.label}</span>
|
||||
<select
|
||||
value={trackIndex}
|
||||
onChange={(e) => setTrackIndex(Number(e.target.value))}
|
||||
className="rounded border ui-border ui-surface px-2 py-1 text-xs"
|
||||
>
|
||||
{session?.tracks.map((track) => (
|
||||
<option key={track.index} value={track.index}>
|
||||
{track.index + 1}: {track.name}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
|
||||
<select
|
||||
value={quantize}
|
||||
onChange={(e) => setQuantize(Number(e.target.value))}
|
||||
className="rounded border ui-border ui-surface px-2 py-1 text-xs"
|
||||
>
|
||||
<option value={0.125}>1/8</option>
|
||||
<option value={0.25}>1/4</option>
|
||||
<option value={0.5}>1/2</option>
|
||||
<option value={1}>1 bar</option>
|
||||
</select>
|
||||
<button
|
||||
onClick={() => void applyOps([{ op: 'quantize_notes', track_index: trackIndex, step: quantize }])}
|
||||
disabled={busy || !activeTrack}
|
||||
className="rounded border ui-border px-2 py-1 text-xs ui-text-muted disabled:opacity-50"
|
||||
>
|
||||
Quantize
|
||||
</button>
|
||||
|
||||
<button
|
||||
onClick={() => void handleSave()}
|
||||
disabled={busy || !session}
|
||||
className="ml-auto inline-flex items-center gap-1 rounded border border-emerald-400/50 bg-emerald-500/10 px-2 py-1 text-xs text-emerald-700 dark:text-emerald-400 disabled:opacity-50"
|
||||
>
|
||||
{busy ? <Loader2 className="w-3 h-3 animate-spin" /> : <Save className="w-3 h-3" />}
|
||||
Save MIDI
|
||||
</button>
|
||||
<button
|
||||
onClick={() => void closeEditor(true)}
|
||||
className="rounded border ui-border px-2 py-1 text-xs ui-text-muted"
|
||||
>
|
||||
Close
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{savedJobId && savedFilename && (
|
||||
<a
|
||||
href={downloadUrl(savedJobId, savedFilename)}
|
||||
download={savedFilename}
|
||||
className="inline-flex items-center gap-1 text-xs text-brand-600 dark:text-brand-400"
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
Download {savedFilename}
|
||||
</a>
|
||||
)}
|
||||
|
||||
{info && <p className="text-xs text-emerald-600 dark:text-emerald-400">{info}</p>}
|
||||
{err && <p className="text-xs text-red-500 break-all">{err}</p>}
|
||||
|
||||
{activeTrack && (
|
||||
<>
|
||||
<div className="space-y-1">
|
||||
<p className="text-[11px] font-medium ui-text-muted uppercase tracking-wider">Notes (first 16)</p>
|
||||
{activeTrack.notes.slice(0, 16).map((note) => (
|
||||
<div key={note.id} className="grid grid-cols-[56px_56px_72px_72px_64px_auto] gap-1 items-center">
|
||||
<input
|
||||
type="number"
|
||||
defaultValue={note.pitch}
|
||||
className="rounded border ui-border ui-surface px-1 py-0.5 text-xs"
|
||||
onBlur={(e) => {
|
||||
const pitch = Number(e.target.value)
|
||||
if (!Number.isFinite(pitch) || pitch === note.pitch) return
|
||||
void applyOps([{ op: 'update_note', track_index: trackIndex, note_id: note.id, pitch }])
|
||||
}}
|
||||
/>
|
||||
<input
|
||||
type="number"
|
||||
step="0.01"
|
||||
defaultValue={note.start}
|
||||
className="rounded border ui-border ui-surface px-1 py-0.5 text-xs"
|
||||
onBlur={(e) => {
|
||||
const start = Number(e.target.value)
|
||||
if (!Number.isFinite(start) || start === note.start) return
|
||||
void applyOps([{ op: 'update_note', track_index: trackIndex, note_id: note.id, start }])
|
||||
}}
|
||||
/>
|
||||
<input
|
||||
type="number"
|
||||
step="0.01"
|
||||
defaultValue={note.duration}
|
||||
className="rounded border ui-border ui-surface px-1 py-0.5 text-xs"
|
||||
onBlur={(e) => {
|
||||
const duration = Number(e.target.value)
|
||||
if (!Number.isFinite(duration) || duration === note.duration) return
|
||||
void applyOps([{ op: 'update_note', track_index: trackIndex, note_id: note.id, duration }])
|
||||
}}
|
||||
/>
|
||||
<input
|
||||
type="number"
|
||||
defaultValue={note.velocity}
|
||||
className="rounded border ui-border ui-surface px-1 py-0.5 text-xs"
|
||||
onBlur={(e) => {
|
||||
const velocity = Number(e.target.value)
|
||||
if (!Number.isFinite(velocity) || velocity === note.velocity) return
|
||||
void applyOps([{ op: 'update_note', track_index: trackIndex, note_id: note.id, velocity }])
|
||||
}}
|
||||
/>
|
||||
<button
|
||||
onClick={() => void applyOps([{ op: 'delete_note', track_index: trackIndex, note_id: note.id }])}
|
||||
className="inline-flex items-center justify-center rounded border ui-border px-1 py-0.5 text-xs ui-text-muted"
|
||||
>
|
||||
<Trash2 className="w-3 h-3" />
|
||||
</button>
|
||||
<span className="text-[10px] ui-text-muted">pitch/start/dur/vel</span>
|
||||
</div>
|
||||
))}
|
||||
<button
|
||||
onClick={() => void applyOps([{ op: 'add_note', track_index: trackIndex, pitch: 60, start: 0, duration: 0.5, velocity: 100 }])}
|
||||
className="rounded border ui-border px-2 py-1 text-xs ui-text-muted"
|
||||
>
|
||||
Add Note
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1 border-t ui-border pt-2">
|
||||
<p className="text-[11px] font-medium ui-text-muted uppercase tracking-wider">Pitch Bends (first 12)</p>
|
||||
{activeTrack.bends.slice(0, 12).map((bend) => (
|
||||
<div key={bend.id} className="grid grid-cols-[80px_80px_auto] gap-1 items-center">
|
||||
<input
|
||||
type="number"
|
||||
step="0.01"
|
||||
defaultValue={bend.time}
|
||||
className="rounded border ui-border ui-surface px-1 py-0.5 text-xs"
|
||||
onBlur={(e) => {
|
||||
const time = Number(e.target.value)
|
||||
if (!Number.isFinite(time) || time === bend.time) return
|
||||
void applyOps([{ op: 'update_bend', track_index: trackIndex, bend_id: bend.id, time }])
|
||||
}}
|
||||
/>
|
||||
<input
|
||||
type="number"
|
||||
defaultValue={bend.value}
|
||||
className="rounded border ui-border ui-surface px-1 py-0.5 text-xs"
|
||||
onBlur={(e) => {
|
||||
const value = Number(e.target.value)
|
||||
if (!Number.isFinite(value) || value === bend.value) return
|
||||
void applyOps([{ op: 'update_bend', track_index: trackIndex, bend_id: bend.id, value }])
|
||||
}}
|
||||
/>
|
||||
<button
|
||||
onClick={() => void applyOps([{ op: 'delete_bend', track_index: trackIndex, bend_id: bend.id }])}
|
||||
className="inline-flex items-center gap-1 rounded border ui-border px-2 py-1 text-xs ui-text-muted w-fit"
|
||||
>
|
||||
<Trash2 className="w-3 h-3" /> Remove
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
<button
|
||||
onClick={() => void applyOps([{ op: 'add_bend', track_index: trackIndex, time: 0, value: 0 }])}
|
||||
className="rounded border ui-border px-2 py-1 text-xs ui-text-muted"
|
||||
>
|
||||
Add Bend
|
||||
</button>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,11 +1,11 @@
|
||||
import React, { useState, useRef, useMemo, useEffect } from 'react'
|
||||
import { useQuery, useQueryClient } from '@tanstack/react-query'
|
||||
import { Link, useNavigate } from 'react-router-dom'
|
||||
import { Link, useLocation, useNavigate } from 'react-router-dom'
|
||||
import {
|
||||
ChevronLeft, ChevronRight, Loader2, CheckCircle, XCircle,
|
||||
Clock, AlertCircle, X, Music2, Mic2, FileCode, Workflow,
|
||||
FolderOpen, Type, ChevronDown, ChevronUp, Radio, Search,
|
||||
Download, CheckCircle2, Trash2, Zap,
|
||||
Download, CheckCircle2, Trash2, Zap, FileArchive,
|
||||
} from 'lucide-react'
|
||||
import {
|
||||
listJobs, cancelJob, getJob,
|
||||
@@ -15,7 +15,7 @@ import {
|
||||
} from '../api/client'
|
||||
import type { Job, Project, SubsonicGroupedSearchResults, SubsonicSong, ImportedFile } from '../types'
|
||||
import AutoPipelineDialog from './AutoPipelineDialog'
|
||||
import { albumRoute, artistRoute, PLAYLISTS_ROUTE, projectRoute, SING_SEARCH_ROUTE, trackRoute } from '../routes'
|
||||
import { albumRoute, artistRoute, CONVERSIONS_ROUTE, PLAYLISTS_ROUTE, projectRoute, SING_SEARCH_ROUTE, trackRoute } from '../routes'
|
||||
import { readSidebarProjectSearch, writeSidebarProjectSearch } from '../utils/uiState'
|
||||
import { publishProjectDeleted } from '../utils/crossTab'
|
||||
import { usePollingGate } from '../hooks/usePollingGate'
|
||||
@@ -872,6 +872,7 @@ interface Props {
|
||||
}
|
||||
|
||||
export default function Sidebar({ isOpen, onToggle, onDeleteProject, activeProjectId, onProjectNavigate }: Props) {
|
||||
const location = useLocation()
|
||||
const queryClient = useQueryClient()
|
||||
const [searchQuery, setSearchQuery] = useState(() => readSidebarProjectSearch())
|
||||
const canPollSidebar = usePollingGate(isOpen)
|
||||
@@ -993,6 +994,19 @@ export default function Sidebar({ isOpen, onToggle, onDeleteProject, activeProje
|
||||
)}
|
||||
</div>
|
||||
|
||||
<Link
|
||||
to={CONVERSIONS_ROUTE}
|
||||
onClick={onProjectNavigate}
|
||||
className={`w-full inline-flex items-center gap-2 rounded-lg border px-2.5 py-2 text-xs font-medium transition-colors ${
|
||||
location.pathname === CONVERSIONS_ROUTE
|
||||
? 'border-brand-400/50 bg-brand-50 dark:bg-brand-950/20 text-brand-700 dark:text-brand-300'
|
||||
: 'ui-border ui-surface hover:border-brand-400/50 ui-text-muted hover:text-brand-600 dark:hover:text-brand-400'
|
||||
}`}
|
||||
>
|
||||
<FileArchive className="w-3.5 h-3.5" />
|
||||
Conversions
|
||||
</Link>
|
||||
|
||||
{/* Search box for projects */}
|
||||
<div className="relative mb-2">
|
||||
<Search className="absolute left-2 top-1/2 -translate-y-1/2 w-3 h-3 ui-text-muted" />
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
import { useEffect, useMemo, useState } from 'react'
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import { Download, Loader2, Pause, Play, RefreshCcw, Trash2 } from 'lucide-react'
|
||||
import {
|
||||
cancelQueuedSloppak,
|
||||
clearFinishedSloppak,
|
||||
enqueueSloppak,
|
||||
enqueueSloppakBulk,
|
||||
getSloppakSnapshot,
|
||||
pauseSloppakQueue,
|
||||
resumeSloppakQueue,
|
||||
retryFailedSloppak,
|
||||
downloadUrl,
|
||||
} from '../api/client'
|
||||
import type { Job } from '../types'
|
||||
|
||||
const ACTIVE = new Set(['queued', 'running'])
|
||||
|
||||
function sourceFromJob(job: Job): string {
|
||||
const result = job.result
|
||||
if (!result) return ''
|
||||
const candidate = (result as Record<string, unknown>).source_filename
|
||||
return typeof candidate === 'string' ? candidate : ''
|
||||
}
|
||||
|
||||
export default function SloppakConverterPanel() {
|
||||
const [singlePath, setSinglePath] = useState('')
|
||||
const [batchText, setBatchText] = useState('')
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [msg, setMsg] = useState<string | null>(null)
|
||||
const [err, setErr] = useState<string | null>(null)
|
||||
|
||||
const { data, refetch } = useQuery({
|
||||
queryKey: ['sloppak-jobs'],
|
||||
queryFn: getSloppakSnapshot,
|
||||
refetchInterval: (query) => {
|
||||
const payload = query.state.data
|
||||
if (!payload?.jobs) return 4000
|
||||
return payload.jobs.some((j) => ACTIVE.has(j.status)) ? 2000 : 5000
|
||||
},
|
||||
})
|
||||
|
||||
useEffect(() => {
|
||||
if (typeof window === 'undefined' || typeof EventSource === 'undefined') {
|
||||
return
|
||||
}
|
||||
|
||||
const source = new EventSource('/api/sloppak/updates')
|
||||
const onSnapshot = () => { void refetch() }
|
||||
source.addEventListener('snapshot', onSnapshot)
|
||||
source.onmessage = () => { void refetch() }
|
||||
source.onerror = () => {
|
||||
// Passive fallback: query polling remains active when SSE drops.
|
||||
}
|
||||
return () => {
|
||||
source.removeEventListener('snapshot', onSnapshot)
|
||||
source.close()
|
||||
}
|
||||
}, [refetch])
|
||||
|
||||
const jobs = data?.jobs ?? []
|
||||
const paused = Boolean(data?.paused)
|
||||
const caps = (data?.capabilities ?? {}) as Record<string, unknown>
|
||||
const unpacker = (caps.unpacker ?? {}) as Record<string, unknown>
|
||||
const unpackReady = Boolean(unpacker.available)
|
||||
const unpackDetail = typeof unpacker.detail === 'string' ? unpacker.detail : ''
|
||||
|
||||
const stats = useMemo(() => {
|
||||
const counts = { queued: 0, running: 0, done: 0, error: 0, cancelled: 0 }
|
||||
jobs.forEach((job) => {
|
||||
if (job.status in counts) {
|
||||
counts[job.status as keyof typeof counts] += 1
|
||||
}
|
||||
})
|
||||
return counts
|
||||
}, [jobs])
|
||||
|
||||
async function runAction(fn: () => Promise<unknown>, success: string) {
|
||||
setBusy(true)
|
||||
setErr(null)
|
||||
setMsg(null)
|
||||
try {
|
||||
await fn()
|
||||
setMsg(success)
|
||||
await refetch()
|
||||
} catch (e) {
|
||||
setErr(String(e))
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function handleEnqueueSingle() {
|
||||
if (!singlePath.trim()) return
|
||||
await runAction(async () => {
|
||||
const res = await enqueueSloppak(singlePath.trim())
|
||||
if (!res.ok && res.reason) {
|
||||
throw new Error(`Skipped: ${res.reason}`)
|
||||
}
|
||||
setSinglePath('')
|
||||
}, 'Enqueued conversion.')
|
||||
}
|
||||
|
||||
async function handleEnqueueBatch() {
|
||||
const filenames = batchText
|
||||
.split('\n')
|
||||
.map((line) => line.trim())
|
||||
.filter(Boolean)
|
||||
if (!filenames.length) return
|
||||
|
||||
await runAction(async () => {
|
||||
const res = await enqueueSloppakBulk(filenames)
|
||||
setMsg(`Batch enqueue: ${res.summary.enqueued} enqueued, ${res.summary.skipped} skipped.`)
|
||||
setBatchText('')
|
||||
}, 'Batch enqueue completed.')
|
||||
}
|
||||
|
||||
return (
|
||||
<section>
|
||||
<div className="flex items-center justify-between mb-3">
|
||||
<h3 className="text-sm font-semibold ui-text">PSARC → sloppak Converter</h3>
|
||||
<button
|
||||
onClick={() => void refetch()}
|
||||
className="inline-flex items-center gap-1 rounded-lg border ui-border px-2.5 py-1 text-xs ui-text-muted hover:border-brand-400"
|
||||
>
|
||||
<RefreshCcw className="w-3 h-3" /> Refresh
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="rounded-xl border ui-border p-3 space-y-3">
|
||||
<div className="grid grid-cols-1 md:grid-cols-2 gap-3">
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs ui-text-muted uppercase tracking-wider">Single Convert</p>
|
||||
<div className="flex gap-2">
|
||||
<input
|
||||
value={singlePath}
|
||||
onChange={(e) => setSinglePath(e.target.value)}
|
||||
placeholder="relative/path/song.psarc"
|
||||
className="flex-1 rounded-lg border ui-border ui-surface-2 px-2.5 py-1.5 text-xs ui-text"
|
||||
/>
|
||||
<button
|
||||
onClick={() => void handleEnqueueSingle()}
|
||||
disabled={busy || !singlePath.trim()}
|
||||
className="rounded-lg border border-brand-400/40 bg-brand-500/10 px-3 py-1.5 text-xs text-brand-600 dark:text-brand-400 disabled:opacity-50"
|
||||
>
|
||||
{busy ? <Loader2 className="w-3 h-3 animate-spin" /> : 'Enqueue'}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<p className="text-xs ui-text-muted uppercase tracking-wider">Batch Convert</p>
|
||||
<textarea
|
||||
value={batchText}
|
||||
onChange={(e) => setBatchText(e.target.value)}
|
||||
rows={4}
|
||||
placeholder={['song-a.psarc', 'album/song-b.psarc'].join('\n')}
|
||||
className="w-full rounded-lg border ui-border ui-surface-2 px-2.5 py-1.5 text-xs ui-text"
|
||||
/>
|
||||
<button
|
||||
onClick={() => void handleEnqueueBatch()}
|
||||
disabled={busy || !batchText.trim()}
|
||||
className="rounded-lg border border-brand-400/40 bg-brand-500/10 px-3 py-1.5 text-xs text-brand-600 dark:text-brand-400 disabled:opacity-50"
|
||||
>
|
||||
Enqueue Batch
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex flex-wrap gap-2 border-t ui-border pt-3">
|
||||
<button
|
||||
onClick={() => void runAction(paused ? resumeSloppakQueue : pauseSloppakQueue, paused ? 'Queue resumed.' : 'Queue paused.')}
|
||||
disabled={busy}
|
||||
className="inline-flex items-center gap-1 rounded-lg border ui-border px-2.5 py-1 text-xs ui-text"
|
||||
>
|
||||
{paused ? <Play className="w-3 h-3" /> : <Pause className="w-3 h-3" />}
|
||||
{paused ? 'Resume' : 'Pause'}
|
||||
</button>
|
||||
<button
|
||||
onClick={() => void runAction(cancelQueuedSloppak, 'Queued jobs cancelled.')}
|
||||
disabled={busy}
|
||||
className="inline-flex items-center gap-1 rounded-lg border ui-border px-2.5 py-1 text-xs ui-text"
|
||||
>
|
||||
Cancel Queued
|
||||
</button>
|
||||
<button
|
||||
onClick={() => void runAction(retryFailedSloppak, 'Failed jobs retried.')}
|
||||
disabled={busy}
|
||||
className="inline-flex items-center gap-1 rounded-lg border ui-border px-2.5 py-1 text-xs ui-text"
|
||||
>
|
||||
Retry Failed
|
||||
</button>
|
||||
<button
|
||||
onClick={() => void runAction(clearFinishedSloppak, 'Finished jobs cleared.')}
|
||||
disabled={busy}
|
||||
className="inline-flex items-center gap-1 rounded-lg border ui-border px-2.5 py-1 text-xs text-red-600 dark:text-red-400"
|
||||
>
|
||||
<Trash2 className="w-3 h-3" /> Clear Finished
|
||||
</button>
|
||||
<span className="ml-auto text-xs ui-text-muted">
|
||||
q:{stats.queued} r:{stats.running} d:{stats.done} e:{stats.error}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{msg && <p className="text-xs text-emerald-600 dark:text-emerald-400">{msg}</p>}
|
||||
{err && <p className="text-xs text-red-500">{err}</p>}
|
||||
<p className={`text-xs ${unpackReady ? 'text-emerald-600 dark:text-emerald-400' : 'text-amber-600 dark:text-amber-400'}`}>
|
||||
Unpacker: {unpackReady ? 'ready' : 'not ready'}{unpackDetail ? ` (${unpackDetail})` : ''}
|
||||
</p>
|
||||
|
||||
<div className="space-y-1.5 border-t ui-border pt-3 max-h-56 overflow-y-auto">
|
||||
{jobs.length === 0 ? (
|
||||
<p className="text-xs ui-text-muted italic">No conversion jobs yet.</p>
|
||||
) : jobs.map((job) => {
|
||||
const source = sourceFromJob(job)
|
||||
return (
|
||||
<div key={job.id} className="rounded-lg border ui-border px-2.5 py-2 text-xs">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="font-medium ui-text">{source || job.id}</span>
|
||||
<span className="ui-text-muted">{job.status}</span>
|
||||
{job.files.length > 0 && (
|
||||
<a
|
||||
href={downloadUrl(job.id, job.files[0].name)}
|
||||
download={job.files[0].name}
|
||||
className="ml-auto inline-flex items-center gap-1 ui-text-muted hover:text-brand-500"
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
{job.files[0].name}
|
||||
</a>
|
||||
)}
|
||||
</div>
|
||||
{job.progress && <p className="ui-text-muted mt-1 truncate">{job.progress}</p>}
|
||||
{job.error && <p className="text-red-500 mt-1 break-all">{job.error}</p>}
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import SloppakConverterPanel from '../components/SloppakConverterPanel'
|
||||
|
||||
export default function ConversionsPage() {
|
||||
return (
|
||||
<div className="space-y-4">
|
||||
<div className="space-y-1">
|
||||
<h1 className="text-lg sm:text-xl font-semibold ui-text">PSARC to sloppak Conversions</h1>
|
||||
<p className="text-xs sm:text-sm ui-text-muted">
|
||||
Queue discrete or batch PSARC conversions and monitor status in one place.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<SloppakConverterPanel />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -7,10 +7,13 @@ import {
|
||||
ListMusic, Edit2, Check, X as XIcon, Wand2, RotateCcw, Copy, Users,
|
||||
Play, Pause, Square, Eye, EyeOff,
|
||||
FileArchive, Save,
|
||||
SlidersHorizontal,
|
||||
} from 'lucide-react'
|
||||
import { getProjectDetail, transcribeProject, abcProject, mixProject, lrcProject, lyricTranscribeProject, downloadUrl, reprocessStem, renameStem, deleteStem, getPerInstrumentModels, getTranscribeModes, getLyricModels, projectCoverArtUrl, getProjectKaraokeConfig, patchProjectKaraokeConfig, exportProjectSek } from '../api/client'
|
||||
import type { ProjectDetail as PD, Job, JobFile, ModeOption, LyricModelsPayload } from '../types'
|
||||
import DynamicMixPlayer from '../components/DynamicMixPlayer'
|
||||
import SloppakConverterPanel from '../components/SloppakConverterPanel'
|
||||
import MidiLightEditor from '../components/MidiLightEditor'
|
||||
import { publishProjectInvalidated } from '../utils/crossTab'
|
||||
import { usePollingGate } from '../hooks/usePollingGate'
|
||||
import { formatApiDate } from '../utils/datetime'
|
||||
@@ -133,11 +136,17 @@ interface MidiNoteLike {
|
||||
|
||||
interface MidiTrackLike {
|
||||
notes: MidiNoteLike[]
|
||||
pitchBends?: MidiPitchBendLike[]
|
||||
instrument?: {
|
||||
number?: number
|
||||
}
|
||||
}
|
||||
|
||||
interface MidiPitchBendLike {
|
||||
time: number
|
||||
value: number
|
||||
}
|
||||
|
||||
interface MidiLike {
|
||||
tracks: MidiTrackLike[]
|
||||
duration?: number
|
||||
@@ -168,7 +177,7 @@ interface ToneLike {
|
||||
Sampler: new (options: Record<string, unknown>) => ToneSynthLike
|
||||
Synth: unknown
|
||||
PolySynth: new (voice?: unknown, options?: unknown) => ToneSynthLike
|
||||
Part: new (callback: (time: number, note: MidiNoteLike) => void, events: Array<[number, MidiNoteLike]>) => TonePartLike
|
||||
Part: new (callback: (time: number, event: unknown) => void, events: Array<[number, unknown]>) => TonePartLike
|
||||
Volume: new (value: number) => ToneVolumeLike
|
||||
Transport: {
|
||||
stop(): void
|
||||
@@ -323,6 +332,12 @@ interface MidiGroupStem {
|
||||
jobId: string
|
||||
}
|
||||
|
||||
function bendValueToCents(raw: number, intensityScale: number): number {
|
||||
const normalized = Math.abs(raw) <= 1.0001 ? raw : raw / 8192
|
||||
const semitones = Math.max(-2, Math.min(2, normalized * 2))
|
||||
return semitones * 100 * intensityScale
|
||||
}
|
||||
|
||||
function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
const [open, setOpen] = useState(false)
|
||||
const [midis, setMidis] = useState<Record<number, MidiLike>>({})
|
||||
@@ -336,10 +351,13 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
const [mutedStems, setMutedStems] = useState<Record<number, boolean>>({})
|
||||
const [soloedStems, setSoloedStems] = useState<Record<number, boolean>>({})
|
||||
const [hiddenStems, setHiddenStems] = useState<Record<number, boolean>>({})
|
||||
const [bendPlaybackEnabled, setBendPlaybackEnabled] = useState(true)
|
||||
const [bendIntensity, setBendIntensity] = useState(100)
|
||||
|
||||
const ToneRef = useRef<ToneLike | null>(null)
|
||||
const synthRefs = useRef<Record<number, ToneSynthLike>>({})
|
||||
const partRefs = useRef<Record<number, TonePartLike>>({})
|
||||
const bendPartRefs = useRef<Record<number, TonePartLike>>({})
|
||||
const volRefs = useRef<Record<number, ToneVolumeLike>>({})
|
||||
const rafRef = useRef<number | null>(null)
|
||||
const playheadRef = useRef<SVGLineElement | null>(null)
|
||||
@@ -381,9 +399,11 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
ToneRef.current?.Transport.stop()
|
||||
ToneRef.current?.Transport.cancel()
|
||||
Object.values(partRefs.current).forEach((part) => { try { if (part?.dispose) part.dispose() } catch { /* ignore dispose errors */ } })
|
||||
Object.values(bendPartRefs.current).forEach((part) => { try { if (part?.dispose) part.dispose() } catch { /* ignore dispose errors */ } })
|
||||
Object.values(synthRefs.current).forEach((synth) => { try { if (synth?.dispose) synth.dispose() } catch { /* ignore dispose errors */ } })
|
||||
Object.values(volRefs.current).forEach((volume) => { try { if (volume?.dispose) volume.dispose() } catch { /* ignore dispose errors */ } })
|
||||
partRefs.current = {}
|
||||
bendPartRefs.current = {}
|
||||
synthRefs.current = {}
|
||||
volRefs.current = {}
|
||||
} catch { /* ignore transport cleanup errors */ }
|
||||
@@ -409,20 +429,66 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
rafRef.current = requestAnimationFrame(loop)
|
||||
}
|
||||
|
||||
function setSynthDetune(synth: ToneSynthLike, cents: number) {
|
||||
const value = Math.max(-2400, Math.min(2400, cents))
|
||||
try {
|
||||
const maybeSet = (synth as { set?: (options: Record<string, unknown>) => void }).set
|
||||
if (typeof maybeSet === 'function') {
|
||||
maybeSet.call(synth, { detune: value })
|
||||
}
|
||||
} catch {
|
||||
// Ignore synth-specific detune errors.
|
||||
}
|
||||
}
|
||||
|
||||
// One Part per stem — merges all tracks in that stem's MIDI into one synth
|
||||
function buildStemPart(Tone: ToneLike, si: number, synth: ToneSynthLike) {
|
||||
const midi = midis[si]
|
||||
if (!midi) return
|
||||
const allNotes: Array<[number, MidiNoteLike]> = []
|
||||
const allBends: Array<[number, MidiPitchBendLike]> = []
|
||||
midi.tracks.forEach((track) => {
|
||||
track.notes.forEach((note) => allNotes.push([note.time, note]))
|
||||
track.pitchBends?.forEach((bend) => allBends.push([bend.time, bend]))
|
||||
})
|
||||
if (!allNotes.length) return
|
||||
const part = new Tone.Part((time: number, note: MidiNoteLike) => {
|
||||
const part = new Tone.Part((time: number, event: unknown) => {
|
||||
const note = event as MidiNoteLike
|
||||
synth.triggerAttackRelease(note.name, note.duration, time, note.velocity ?? 0.8)
|
||||
}, allNotes)
|
||||
part.start(0)
|
||||
partRefs.current[si] = part
|
||||
|
||||
if (bendPlaybackEnabled && allBends.length) {
|
||||
const intensityScale = Math.max(0, Math.min(2, bendIntensity / 100))
|
||||
setSynthDetune(synth, 0)
|
||||
const bendPart = new Tone.Part((time: number, event: unknown) => {
|
||||
void time
|
||||
const bend = event as MidiPitchBendLike
|
||||
const cents = bendValueToCents(bend.value ?? 0, intensityScale)
|
||||
setSynthDetune(synth, cents)
|
||||
}, allBends)
|
||||
bendPart.start(0)
|
||||
bendPartRefs.current[si] = bendPart
|
||||
} else {
|
||||
setSynthDetune(synth, 0)
|
||||
}
|
||||
}
|
||||
|
||||
function clearParts() {
|
||||
Object.values(partRefs.current).forEach((part) => { try { if (part?.dispose) part.dispose() } catch { /* ignore dispose errors */ } })
|
||||
Object.values(bendPartRefs.current).forEach((part) => { try { if (part?.dispose) part.dispose() } catch { /* ignore dispose errors */ } })
|
||||
partRefs.current = {}
|
||||
bendPartRefs.current = {}
|
||||
}
|
||||
|
||||
function rebuildParts(Tone: ToneLike) {
|
||||
Tone.Transport.cancel()
|
||||
clearParts()
|
||||
stems.forEach((_, si) => {
|
||||
const synth = synthRefs.current[si]
|
||||
if (synth) buildStemPart(Tone, si, synth)
|
||||
})
|
||||
}
|
||||
|
||||
function applyMutes(overrides: {
|
||||
@@ -521,12 +587,7 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
Tone.Transport.stop()
|
||||
Tone.Transport.cancel()
|
||||
Tone.Transport.seconds = 0
|
||||
Object.values(partRefs.current).forEach((part) => { try { if (part?.dispose) part.dispose() } catch { /* ignore dispose errors */ } })
|
||||
partRefs.current = {}
|
||||
stems.forEach((_, si) => {
|
||||
const synth = synthRefs.current[si]
|
||||
if (synth) buildStemPart(Tone, si, synth)
|
||||
})
|
||||
rebuildParts(Tone)
|
||||
}
|
||||
resetPlayhead()
|
||||
setCurrentTime(0)
|
||||
@@ -542,12 +603,7 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
Tone.Transport.cancel()
|
||||
Tone.Transport.seconds = t
|
||||
setCurrentTime(t)
|
||||
Object.values(partRefs.current).forEach((part) => { try { if (part?.dispose) part.dispose() } catch { /* ignore dispose errors */ } })
|
||||
partRefs.current = {}
|
||||
stems.forEach((_, si) => {
|
||||
const synth = synthRefs.current[si]
|
||||
if (synth) buildStemPart(Tone, si, synth)
|
||||
})
|
||||
rebuildParts(Tone)
|
||||
if (playheadRef.current) {
|
||||
const pct = Math.min(1, t / (durationRef.current || 1))
|
||||
playheadRef.current.style.display = 'block'
|
||||
@@ -588,6 +644,11 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
if (part?.dispose) part.dispose()
|
||||
} catch { /* ignore dispose errors */ }
|
||||
delete partRefs.current[si]
|
||||
try {
|
||||
const bendPart = bendPartRefs.current[si]
|
||||
if (bendPart?.dispose) bendPart.dispose()
|
||||
} catch { /* ignore dispose errors */ }
|
||||
delete bendPartRefs.current[si]
|
||||
try {
|
||||
const synth = synthRefs.current[si]
|
||||
if (synth?.dispose) synth.dispose()
|
||||
@@ -602,21 +663,36 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
try { await newSynth.loaded } catch { /* ignore soundfont load errors */ }
|
||||
setSfState('ready')
|
||||
}
|
||||
// Rebuild all parts to stay in sync
|
||||
Tone.Transport.cancel()
|
||||
stems.forEach((_, idx) => {
|
||||
try {
|
||||
const part = partRefs.current[idx]
|
||||
if (part?.dispose) part.dispose()
|
||||
} catch { /* ignore dispose errors */ }
|
||||
const synth = synthRefs.current[idx]
|
||||
if (synth) buildStemPart(Tone, idx, synth)
|
||||
// Rebuild all parts to stay in sync.
|
||||
rebuildParts(Tone)
|
||||
}
|
||||
|
||||
function toggleBendPlayback() {
|
||||
setBendPlaybackEnabled((prev) => {
|
||||
const next = !prev
|
||||
const Tone = ToneRef.current
|
||||
if (Tone && sfState === 'ready') {
|
||||
rebuildParts(Tone)
|
||||
} else if (!next) {
|
||||
Object.values(synthRefs.current).forEach((synth) => setSynthDetune(synth, 0))
|
||||
}
|
||||
return next
|
||||
})
|
||||
}
|
||||
|
||||
function updateBendIntensity(value: number) {
|
||||
setBendIntensity(value)
|
||||
const Tone = ToneRef.current
|
||||
if (Tone && sfState === 'ready' && bendPlaybackEnabled) {
|
||||
rebuildParts(Tone)
|
||||
}
|
||||
}
|
||||
|
||||
const midiList = stems.map((_, si) => ({ midi: midis[si], stemIdx: si })).filter(x => x.midi)
|
||||
const totalNotes = midiList.reduce((s, { midi }) =>
|
||||
s + midi.tracks.reduce((s2: number, track) => s2 + track.notes.length, 0), 0)
|
||||
const totalBends = midiList.reduce((s, { midi }) =>
|
||||
s + midi.tracks.reduce((s2: number, track) => s2 + (track.pitchBends?.length ?? 0), 0), 0)
|
||||
const bpm = midiList[0]?.midi?.header?.tempos?.[0]?.bpm
|
||||
const durStr = formatTime(duration)
|
||||
const curStr = formatTime(currentTime)
|
||||
@@ -683,11 +759,33 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
<span className="ml-auto text-[10px] text-gray-600 select-none">
|
||||
{sfState === 'idle' && `${totalNotes} notes · click Play`}
|
||||
{sfState === 'loading' && 'Loading instruments…'}
|
||||
{sfState === 'ready' && `${totalNotes} notes`}
|
||||
{sfState === 'ready' && `${totalNotes} notes · ${totalBends} bends`}
|
||||
{sfState === 'error' && 'Failed to load'}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="flex items-center gap-2">
|
||||
<button
|
||||
onClick={toggleBendPlayback}
|
||||
className={`inline-flex items-center gap-1.5 rounded px-2 py-1 text-[10px] font-medium transition-colors ${bendPlaybackEnabled ? 'bg-cyan-600/20 text-cyan-300 border border-cyan-500/40' : 'bg-gray-800 text-gray-400 border border-gray-700'}`}
|
||||
title="Toggle pitch bend playback"
|
||||
>
|
||||
<SlidersHorizontal className="w-3 h-3" />
|
||||
Bend {bendPlaybackEnabled ? 'On' : 'Off'}
|
||||
</button>
|
||||
<input
|
||||
type="range"
|
||||
min={0}
|
||||
max={200}
|
||||
step={5}
|
||||
value={bendIntensity}
|
||||
onChange={(e) => updateBendIntensity(parseInt(e.target.value))}
|
||||
disabled={!bendPlaybackEnabled}
|
||||
className="flex-1 h-1.5 bg-gray-700 rounded-lg appearance-none cursor-pointer accent-cyan-500 disabled:opacity-40"
|
||||
/>
|
||||
<span className="text-[10px] tabular-nums text-gray-500 w-10 text-right">{bendIntensity}%</span>
|
||||
</div>
|
||||
|
||||
{/* Per-stem rows: M / S / label / instrument / download */}
|
||||
<div className="space-y-1.5 pt-1.5 border-t border-gray-800">
|
||||
{stems.map((stem, si) => {
|
||||
@@ -745,6 +843,7 @@ function MidiGroupPlayer({ stems }: { stems: MidiGroupStem[] }) {
|
||||
>
|
||||
<Download className="w-3 h-3" />
|
||||
</a>
|
||||
<MidiLightEditor stem={{ label: stem.label, jobId: stem.jobId, filename: stem.file.name }} />
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
@@ -3020,6 +3119,11 @@ export default function ProjectDetail({ id, onDelete }: Props) {
|
||||
/>
|
||||
</div>
|
||||
|
||||
{/* 3b — PSARC/sloppak Converter */}
|
||||
<div className="pt-6">
|
||||
<SloppakConverterPanel />
|
||||
</div>
|
||||
|
||||
{/* 4 — ABC */}
|
||||
<div className="pt-6">
|
||||
<AbcSection
|
||||
|
||||
@@ -3407,6 +3407,10 @@ function IntegrationsSettings() {
|
||||
const [clearingSeparatorAddress, setClearingSeparatorAddress] = useState(false)
|
||||
const [separatorStatus, setSeparatorStatus] = useState<'idle' | 'saved' | 'error'>('idle')
|
||||
const [separatorStatusText, setSeparatorStatusText] = useState('')
|
||||
const [separationDevice, setSeparationDevice] = useState<'gpu' | 'cpu'>('gpu')
|
||||
const [savingSeparationDevice, setSavingSeparationDevice] = useState(false)
|
||||
const [separationDeviceStatus, setSeparationDeviceStatus] = useState<'idle' | 'saved' | 'error'>('idle')
|
||||
const [separationDeviceStatusText, setSeparationDeviceStatusText] = useState('')
|
||||
|
||||
const [karaokeSourceMode, setKaraokeSourceMode] = useState<'projects' | 'sek-folder'>('sek-folder')
|
||||
const [karaokeSekFolderInput, setKaraokeSekFolderInput] = useState('')
|
||||
@@ -3468,6 +3472,7 @@ function IntegrationsSettings() {
|
||||
|
||||
api.getAppSettings().then((s) => {
|
||||
setHfTokenSet(s.hf_token_set)
|
||||
setSeparationDevice(s.cpu_separation ? 'cpu' : 'gpu')
|
||||
setKaraokeSourceMode((s.karaoke_source_mode ?? 'sek-folder') as 'projects' | 'sek-folder')
|
||||
setKaraokeSekFolderInput(s.karaoke_sek_folder ?? '')
|
||||
setMusicLibraryPathInput(s.music_library_path ?? '')
|
||||
@@ -3712,6 +3717,30 @@ function IntegrationsSettings() {
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSaveSeparationDevice() {
|
||||
setSavingSeparationDevice(true)
|
||||
setSeparationDeviceStatus('idle')
|
||||
setSeparationDeviceStatusText('')
|
||||
try {
|
||||
const cpuSeparation = separationDevice === 'cpu'
|
||||
await api.patchAppSettings({
|
||||
cpu_separation: cpuSeparation,
|
||||
hw_profile: 'custom',
|
||||
})
|
||||
setSeparationDeviceStatus('saved')
|
||||
setSeparationDeviceStatusText(
|
||||
cpuSeparation
|
||||
? 'Saved. Separation will run on CPU only.'
|
||||
: 'Saved. Separation will use GPU when CUDA is available.'
|
||||
)
|
||||
} catch (error: any) {
|
||||
setSeparationDeviceStatus('error')
|
||||
setSeparationDeviceStatusText(error?.message || 'Failed to save separation device setting.')
|
||||
} finally {
|
||||
setSavingSeparationDevice(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSaveToken() {
|
||||
if (!hfTokenInput.trim()) return
|
||||
setSaving(true)
|
||||
@@ -3932,6 +3961,68 @@ function IntegrationsSettings() {
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Separation device */}
|
||||
<div className="space-y-3 pt-2 border-t border-slate-200 dark:border-gray-700">
|
||||
<div className="flex items-center gap-2">
|
||||
<Wrench className="w-4 h-4 text-slate-400 dark:text-gray-500" />
|
||||
<label className="text-sm font-medium text-slate-700 dark:text-gray-300">
|
||||
Separation device
|
||||
</label>
|
||||
</div>
|
||||
<p className="text-xs text-slate-500 dark:text-gray-400">
|
||||
Choose whether stem separation should run on GPU (faster) or CPU (more compatible, much slower).
|
||||
</p>
|
||||
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setSeparationDevice('gpu')}
|
||||
className={`px-3 py-1.5 rounded-md border text-sm font-medium transition-all ${
|
||||
separationDevice === 'gpu'
|
||||
? 'bg-brand-50 dark:bg-brand-950/30 border-brand-500 text-brand-700 dark:text-brand-300 shadow-sm'
|
||||
: 'bg-white dark:bg-gray-800 border-slate-200 dark:border-gray-700 text-slate-600 dark:text-gray-400 hover:border-brand-400 hover:bg-slate-50 dark:hover:bg-gray-750'
|
||||
}`}
|
||||
>
|
||||
GPU (auto when available)
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setSeparationDevice('cpu')}
|
||||
className={`px-3 py-1.5 rounded-md border text-sm font-medium transition-all ${
|
||||
separationDevice === 'cpu'
|
||||
? 'bg-brand-50 dark:bg-brand-950/30 border-brand-500 text-brand-700 dark:text-brand-300 shadow-sm'
|
||||
: 'bg-white dark:bg-gray-800 border-slate-200 dark:border-gray-700 text-slate-600 dark:text-gray-400 hover:border-brand-400 hover:bg-slate-50 dark:hover:bg-gray-750'
|
||||
}`}
|
||||
>
|
||||
CPU only
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="flex gap-2">
|
||||
<button
|
||||
onClick={handleSaveSeparationDevice}
|
||||
disabled={savingSeparationDevice}
|
||||
className="flex items-center gap-1.5 px-4 py-2 text-sm font-medium rounded-lg bg-brand-500 hover:bg-brand-600 text-white disabled:opacity-40 transition-colors"
|
||||
>
|
||||
<Save className="w-4 h-4" />
|
||||
{savingSeparationDevice ? 'Saving…' : 'Save'}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{separationDeviceStatus === 'saved' && separationDeviceStatusText && (
|
||||
<p className="flex items-center gap-1.5 text-xs text-emerald-600 dark:text-emerald-400">
|
||||
<CheckCircle className="w-3.5 h-3.5" />
|
||||
{separationDeviceStatusText}
|
||||
</p>
|
||||
)}
|
||||
{separationDeviceStatus === 'error' && separationDeviceStatusText && (
|
||||
<p className="flex items-center gap-1.5 text-xs text-red-500 dark:text-red-400">
|
||||
<AlertCircle className="w-3.5 h-3.5" />
|
||||
{separationDeviceStatusText}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* HuggingFace Token */}
|
||||
<div className="space-y-3 pt-2 border-t border-slate-200 dark:border-gray-700">
|
||||
<div className="flex items-center gap-2">
|
||||
|
||||
@@ -5,6 +5,7 @@ export const ARTISTS_ROUTE = '/artists'
|
||||
export const PLAYLISTS_ROUTE = '/playlists'
|
||||
export const SING_ROUTE = '/sing'
|
||||
export const SETTINGS_ROUTE = '/settings'
|
||||
export const CONVERSIONS_ROUTE = '/conversions'
|
||||
export const VISUALIZER_LAB_ROUTE = '/visualizer-lab'
|
||||
export const KARAOKE_ROUTE = '/karaoke'
|
||||
export const KARAOKE_TV_ROUTE = '/karaoke/tv'
|
||||
|
||||
+11
-1
@@ -10,7 +10,17 @@ export interface JobFile {
|
||||
|
||||
export interface Job {
|
||||
id: string
|
||||
job_type: 'separate' | 'transcribe' | 'abc-convert' | 'pipeline' | 'lrc_align' | 'mix' | 'lyric-transcribe' | 'ytdl-import'
|
||||
job_type:
|
||||
| 'separate'
|
||||
| 'transcribe'
|
||||
| 'abc-convert'
|
||||
| 'pipeline'
|
||||
| 'lrc_align'
|
||||
| 'mix'
|
||||
| 'lyric-transcribe'
|
||||
| 'ytdl-import'
|
||||
| 'sloppak_convert'
|
||||
| 'midi-edit-save'
|
||||
status: JobStatus
|
||||
progress: string
|
||||
error: string
|
||||
|
||||
@@ -115,6 +115,12 @@ services:
|
||||
- PARAKEET_TRANSCRIBE_PATH=${PARAKEET_TRANSCRIBE_PATH:-/transcribe}
|
||||
- PARAKEET_TIMEOUT_SEC=${PARAKEET_TIMEOUT_SEC:-180}
|
||||
- DEFAULT_PARAKEET_MODEL=${DEFAULT_PARAKEET_MODEL:-parakeet-1.1b-rnnt-multilingual-asr}
|
||||
# PSARC -> sloppak converter settings
|
||||
- SLOPPAK_SOURCE_ROOT=${SLOPPAK_SOURCE_ROOT:-/app/media/uploads}
|
||||
- SLOPPAK_OUTPUT_ROOT=${SLOPPAK_OUTPUT_ROOT:-/app/media/sloppak}
|
||||
# Optional unpack command template for deep Rocksmith extraction when
|
||||
# python patcher module is unavailable. Must include {input} and {output}.
|
||||
- SLOPPAK_PSARC_UNPACK_CMD=${SLOPPAK_PSARC_UNPACK_CMD:-}
|
||||
# Parakeet multilingual transcription (self-hosted only)
|
||||
- PARAKEET_ENABLED=${PARAKEET_ENABLED:-false}
|
||||
- PARAKEET_SIDECAR_URL=${PARAKEET_SIDECAR_URL:-http://parakeet-sidecar:8777}
|
||||
|
||||
Reference in new issue
Block a user