Myriad improvements.
This commit is contained in:
1 parent
ed72ded98f
commit
9a02f0f321
19 files changed
+2208
-24
No files matched your search
@@ -0,0 +1,47 @@
|
||||
## Summary
|
||||
|
||||
<!-- What changed and why? -->
|
||||
|
||||
## Change type
|
||||
|
||||
- [ ] Feature
|
||||
- [ ] Bug fix
|
||||
- [ ] Refactor
|
||||
- [ ] Docs only
|
||||
- [ ] Chore
|
||||
|
||||
## Validation
|
||||
|
||||
- [ ] Backend tests or targeted checks run
|
||||
- [ ] Frontend checks (typecheck/lint/build) run when relevant
|
||||
- [ ] Manual verification completed
|
||||
|
||||
## Documentation impact (required)
|
||||
|
||||
- [ ] I reviewed documentation impact for this PR.
|
||||
- [ ] No documentation update required (internal-only change with no behavior/API/workflow impact).
|
||||
- [ ] Documentation updated in this PR.
|
||||
|
||||
If documentation was updated, list files:
|
||||
|
||||
- [ ] docs/USER_GUIDE.md
|
||||
- [ ] docs/DEVELOPER_GUIDE.md
|
||||
- [ ] docs/API_REFERENCE.md
|
||||
- [ ] docs/TROUBLESHOOTING_MATRIX.md
|
||||
- [ ] docs/CONFIGURATION_REFERENCE.md
|
||||
- [ ] docs/DOC_GOVERNANCE.md
|
||||
- [ ] docs/templates/FEATURE_DOC_TEMPLATE.md
|
||||
- [ ] docs/templates/MIGRATION_NOTE_TEMPLATE.md
|
||||
- [ ] other: ______________________
|
||||
|
||||
## Release notes / migration
|
||||
|
||||
- [ ] No release note needed
|
||||
- [ ] Release note entry added
|
||||
- [ ] Migration note added (if required)
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Backward compatibility considered
|
||||
- [ ] Error cases handled
|
||||
- [ ] Security/permissions impact considered
|
||||
@@ -3510,3 +3510,483 @@ Validation:
|
||||
- frontend diagnostics: clean
|
||||
- frontend typecheck: `EXIT:0`
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation (documentation program — Phases 1, 2, and 3)
|
||||
|
||||
Requested:
|
||||
- Create documentation for users and developers.
|
||||
- Implement Phase 2 (API reference, troubleshooting matrix, configuration guidance).
|
||||
- Implement Phase 3 (governance and maintenance workflow).
|
||||
- Record documentation changes in this running fixes log.
|
||||
|
||||
Scope:
|
||||
- `docs/README.md`
|
||||
- `docs/USER_GUIDE.md`
|
||||
- `docs/DEVELOPER_GUIDE.md`
|
||||
- `docs/DOCUMENTATION_PLAN.md`
|
||||
- `docs/API_REFERENCE.md`
|
||||
- `docs/TROUBLESHOOTING_MATRIX.md`
|
||||
- `docs/CONFIGURATION_REFERENCE.md`
|
||||
- `docs/DOC_GOVERNANCE.md`
|
||||
- `docs/RELEASE_CHECKLIST.md`
|
||||
- `docs/templates/FEATURE_DOC_TEMPLATE.md`
|
||||
- `docs/templates/MIGRATION_NOTE_TEMPLATE.md`
|
||||
- `.github/PULL_REQUEST_TEMPLATE.md`
|
||||
- `README.md`
|
||||
|
||||
#### Implemented
|
||||
|
||||
1) Documentation baseline (Phase 1)
|
||||
- Added dedicated docs entrypoint and split audience docs:
|
||||
- user guide
|
||||
- developer guide
|
||||
- delivery plan
|
||||
|
||||
2) Documentation depth (Phase 2)
|
||||
- Added API reference by endpoint groups with practical request examples.
|
||||
- Added troubleshooting matrix for GPU/runtime/model-download/sidecar/alignment paths.
|
||||
- Added configuration reference with operational guidance beyond raw env-var listing.
|
||||
|
||||
3) Documentation governance (Phase 3)
|
||||
- Added PR template with mandatory documentation-impact checklist.
|
||||
- Added governance policy with:
|
||||
- per-PR doc review requirements
|
||||
- per-release/monthly freshness cadence
|
||||
- owner rotation/steward responsibilities
|
||||
- Added release checklist that includes explicit docs sign-off.
|
||||
- Added reusable templates for:
|
||||
- feature documentation
|
||||
- migration notes
|
||||
|
||||
4) Documentation wiring
|
||||
- Updated docs index and root README links so all new docs are discoverable.
|
||||
- Updated developer guide to point to governance/process artifacts.
|
||||
- Updated documentation plan status to mark Phase 2 and Phase 3 complete.
|
||||
|
||||
Outcome:
|
||||
- Documentation initiative now has full lifecycle coverage:
|
||||
- **Phase 1** baseline audience docs
|
||||
- **Phase 2** operational/technical depth docs
|
||||
- **Phase 3** governance and update process to prevent drift
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 note — CI enforcement of docs-impact PR fields (on hold)
|
||||
|
||||
Discussed: enforcing the Documentation Impact section of `.github/PULL_REQUEST_TEMPLATE.md` via a
|
||||
GitHub Actions workflow that fails PRs where the docs-impact fields are left blank/skipped.
|
||||
|
||||
Decision: **hold for now.**
|
||||
|
||||
When ready, the implementation would be a small GitHub Actions job that parses the PR body and
|
||||
requires either a "files updated" checkbox or an explicit "No documentation changes needed"
|
||||
declaration. Without it the gate blocks merging. Currently the template is on the honor system.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 plan — replace lyric line segmentation with `karaoke-lyrics-processor`
|
||||
|
||||
Requested:
|
||||
- Plan migration from current lyric line segmentation heuristics to
|
||||
`nomadkaraoke/karaoke-lyrics-processor`.
|
||||
- Document the plan.
|
||||
|
||||
Implemented:
|
||||
- Added implementation plan document:
|
||||
- `20260318_Lyric_Line_Segmentation_Replacement_Plan.md`
|
||||
- Plan covers:
|
||||
- current baseline segmentation functions in `lyric_transcriber_service.py`
|
||||
- adapter-first integration strategy
|
||||
- feature-flagged rollout (`legacy` / `karaoke_processor` / `auto`)
|
||||
- validation matrix (unit/integration/regression corpus)
|
||||
- risks and mitigations (timestamp remap drift, dependency pinning, fallback policy)
|
||||
- expected implementation file touch list + docs updates
|
||||
|
||||
Status:
|
||||
- Planning/documentation complete.
|
||||
- Code migration intentionally not started in this step.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — lyric segmentation replacement (`karaoke-lyrics-processor`)
|
||||
|
||||
Requested:
|
||||
- Implement the planned lyric line segmentation replacement.
|
||||
- Document the implementation.
|
||||
|
||||
Scope:
|
||||
- `backend/config.py`
|
||||
- `backend/requirements.txt`
|
||||
- `backend/services/lyric_transcriber_service.py`
|
||||
- `README.md`
|
||||
- `docs/CONFIGURATION_REFERENCE.md`
|
||||
- `docs/TROUBLESHOOTING_MATRIX.md`
|
||||
- `docs/DEVELOPER_GUIDE.md`
|
||||
- `20260318_Lyric_Line_Segmentation_Replacement_Plan.md`
|
||||
|
||||
Implemented:
|
||||
- Added backend dependency: `karaoke-lyrics-processor>=1.0.0`.
|
||||
- Added env-configured segmentation controls:
|
||||
- `STEMENGINE_LYRIC_SEGMENTER` (`auto` / `karaoke_processor` / `legacy`)
|
||||
- `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (default `36`)
|
||||
- Replaced plain-text lyric segmentation path with karaoke processor adapter + safe fallback to legacy heuristic splitter.
|
||||
- Replaced WhisperX long-segment split path with karaoke processor-driven splitting and conservative word-token remap back onto timestamped words.
|
||||
- Added ambiguity/runtime fallback behavior to legacy splitting to avoid job failures when remap or dependency edge cases occur.
|
||||
- Updated operational docs (README + config + troubleshooting + developer guide) with new controls and fallback behavior.
|
||||
- Updated the 2026-03-18 plan doc to mark implementation complete.
|
||||
|
||||
Outcome:
|
||||
- Lyric line segmentation now defaults to karaoke-oriented splitting while preserving backward compatibility and operational safety via explicit fallback modes.
|
||||
|
||||
Validation:
|
||||
- Focused segmentation smoke pass executed (2026-03-18) comparing `auto` vs `legacy` modes using representative long-line samples and timestamped word segments.
|
||||
- Command context: local Python runtime with writable temp media paths (`/tmp/stemengine_smoke`) and direct helper invocation from `lyric_transcriber_service.py`.
|
||||
- Results:
|
||||
- `auto`:
|
||||
- active mode: `auto`
|
||||
- plain-text line count: `4`
|
||||
- WhisperX segment split count: `3`
|
||||
- word timestamp order monotonic after remap: `true`
|
||||
- `legacy`:
|
||||
- active mode: `legacy`
|
||||
- plain-text line count: `3`
|
||||
- WhisperX segment split count: `2`
|
||||
- word timestamp order monotonic after split: `true`
|
||||
- Interpretation: new segmentation path is active and produces tighter karaoke-style line wrapping while preserving timestamp monotonicity in split output.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 plan — karaoke player platform (StemEngine ecosystem)
|
||||
|
||||
Requested:
|
||||
- Process and plan a karaoke player with:
|
||||
1) source browsing/import (Subsonic, local folder, YouTube, SoulSeek)
|
||||
2) StemEngine preset processing + auto-play
|
||||
3) existing Two.js renderer playback with stem mute controls, `projectM` visualizer option, and video playback coupling
|
||||
4) in-app lyrics/style editor using same Two.js renderer
|
||||
5) companion mobile PWA for queue/room operations
|
||||
- Document the plan and hold implementation.
|
||||
|
||||
Implemented:
|
||||
- Added architecture and delivery plan document:
|
||||
- `20260318_Karaoke_Player_Platform_Plan.md`
|
||||
- Plan includes:
|
||||
- scope and explicit out-of-scope boundaries
|
||||
- target multi-component architecture (desktop player, orchestration API, StemEngine integration, mobile PWA)
|
||||
- capability breakdown for each requested feature
|
||||
- proposed domain model and API additions
|
||||
- security/roles policy outline for rooms and queue control
|
||||
- phased rollout (A: foundation, B: editor/visuals, C: rooms/PWA, D: source expansion/hardening)
|
||||
- risks and mitigations
|
||||
|
||||
Status:
|
||||
- Planning/documentation complete.
|
||||
- **On hold** (no implementation started in this step).
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 update — security considerations added to karaoke player platform plan
|
||||
|
||||
Requested:
|
||||
- Add security measures required for the planned karaoke player platform.
|
||||
- Update the plan and document the considerations.
|
||||
|
||||
Implemented:
|
||||
- Expanded `20260318_Karaoke_Player_Platform_Plan.md` security coverage to include:
|
||||
- identity/session hardening (token rotation, revocation, MFA for elevated roles)
|
||||
- RBAC + server-side room policy enforcement and moderation auditability
|
||||
- API/transport controls (TLS, strict CORS, CSRF, endpoint rate-limits, service auth)
|
||||
- ingestion/source hardening for Subsonic/YouTube/SoulSeek/local adapters
|
||||
- path traversal and filesystem isolation requirements
|
||||
- secrets management and encryption-at-rest requirements
|
||||
- XSS/CSP/schema-validation controls for lyrics/style editor and playback UI
|
||||
- abuse prevention quotas and destructive-action safeguards
|
||||
- audit logging, observability, and incident playbook requirements
|
||||
- privacy/retention/legal baseline considerations
|
||||
- Added security-specific pre-implementation gates section:
|
||||
- threat model approval
|
||||
- authN/authZ design review
|
||||
- secret management decision
|
||||
- ingestion sandbox design
|
||||
- security testing plan
|
||||
- audit schema/retention approval
|
||||
- incident runbook drafting
|
||||
|
||||
Status:
|
||||
- Planning/documentation update complete.
|
||||
- **On hold** (implementation not started).
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — PlayaK Phase A scaffold kickoff
|
||||
|
||||
Requested:
|
||||
- Begin PlayaK implementation work and document progress.
|
||||
|
||||
Implemented:
|
||||
- Bootstrapped PlayaK runtime/service scaffold under `/home/translator/docker/appdata/playak`:
|
||||
- `docker-compose.yml`
|
||||
- `.env.example`
|
||||
- `.gitignore`
|
||||
- `backend/Dockerfile`
|
||||
- `backend/requirements.txt`
|
||||
- Added initial FastAPI control-plane app:
|
||||
- `backend/app/main.py`
|
||||
- `backend/app/models.py`
|
||||
- `backend/app/store.py`
|
||||
- `backend/app/security.py`
|
||||
- Initial API endpoints scaffolded:
|
||||
- `GET /api/playak/health`
|
||||
- `GET /api/playak/roles/matrix`
|
||||
- room/queue CRUD skeleton endpoints for Phase A orchestration prototyping
|
||||
- Added/updated documentation:
|
||||
- `/home/translator/docker/appdata/playak/README.md` (quick start + scaffold inventory)
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Karaoke_Player_Platform_Plan.md` status moved from hold to active scaffold phase
|
||||
|
||||
Status:
|
||||
- Phase A scaffold started.
|
||||
- Persistence/auth/provider adapters pending next pass.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — PlayaK workspace bootstrap + plan relocation
|
||||
|
||||
Requested:
|
||||
- Implement and document a new container/workspace named `PlayaK` under:
|
||||
- `~/docker/appdata/playak`
|
||||
- Move the karaoke player platform plan to PlayaK.
|
||||
- Create a new markdown progress document in the root folder.
|
||||
|
||||
Implemented:
|
||||
- Created PlayaK workspace path:
|
||||
- `/home/translator/docker/appdata/playak`
|
||||
- Added PlayaK root documentation:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
- Relocated planning document into PlayaK:
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Karaoke_Player_Platform_Plan.md`
|
||||
- Removed old plan location from StemEngine root:
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_Karaoke_Player_Platform_Plan.md`
|
||||
- Added root-level progress tracker:
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
|
||||
Status:
|
||||
- PlayaK workspace bootstrap complete.
|
||||
- Planning artifacts moved and documented.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 update — explicit global role matrix added (admin/user/guest)
|
||||
|
||||
Requested:
|
||||
- Plan and document a complete global role matrix including:
|
||||
- `platform_admin` (system-wide)
|
||||
- `authenticated_user` (regular signed-in user)
|
||||
- `guest` (anonymous/limited)
|
||||
|
||||
Implemented:
|
||||
- Updated `20260318_Karaoke_Player_Platform_Plan.md` to add:
|
||||
- explicit global-role table with capability-by-capability permissions
|
||||
- mapping between global roles and room roles (`owner`, `moderator`, `member`, `viewer`)
|
||||
- policy notes for guest restrictions and admin scope
|
||||
- deterministic permission-resolution order for queue/room actions
|
||||
- server-side audit requirements for authorization decisions (`user_id`, role context, action, decision)
|
||||
|
||||
Status:
|
||||
- Planning/documentation update complete.
|
||||
- **On hold** (implementation not started).
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — PlayaK Phase A persistence (SQLite)
|
||||
|
||||
Requested:
|
||||
- Continue PlayaK implementation and document progress.
|
||||
|
||||
Implemented:
|
||||
- Replaced in-memory room/queue state with SQLite-backed persistence in PlayaK:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/db.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/store.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py` (startup `init_db` via lifespan)
|
||||
- Added runtime DB configuration and persistent host mount:
|
||||
- `/home/translator/docker/appdata/playak/.env.example` (`PLAYAK_DB_PATH`)
|
||||
- `/home/translator/docker/appdata/playak/docker-compose.yml` (`PLAYAK_DB_PATH` + `./data:/data`)
|
||||
- Updated implementation docs:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
|
||||
Status:
|
||||
- Phase A scaffold now includes persisted room/queue state.
|
||||
- Next: auth/session + server-side role enforcement, then source adapter interfaces.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — PlayaK Phase A auth skeleton + role enforcement
|
||||
|
||||
Requested:
|
||||
- Continue PlayaK implementation and document progress.
|
||||
|
||||
Implemented:
|
||||
- Added auth/authorization scaffold to PlayaK API:
|
||||
- header-based actor context: `X-Global-Role`, `X-User-Id`
|
||||
- role/permission helpers in `/home/translator/docker/appdata/playak/backend/app/security.py`
|
||||
- Added room membership persistence:
|
||||
- SQLite `room_members` table in `/home/translator/docker/appdata/playak/backend/app/db.py`
|
||||
- membership queries/mutations in `/home/translator/docker/appdata/playak/backend/app/store.py`
|
||||
- Added role-aware room endpoints:
|
||||
- `POST /api/playak/rooms/{room_id}/join`
|
||||
- `GET /api/playak/rooms/{room_id}/state`
|
||||
- Enforced queue mutation rules in `/home/translator/docker/appdata/playak/backend/app/main.py`:
|
||||
- enqueue requires permission (`platform_admin` or room member role)
|
||||
- non-admin enqueue identity must match `requested_by`
|
||||
- queue removal allowed for admin, room owner/moderator, or member removing own item
|
||||
|
||||
Documentation updated:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
|
||||
Status:
|
||||
- Phase A now includes persistence + server-side role checks on room/queue control paths.
|
||||
- Next: source-ingestion adapter interfaces and orchestration job submission wiring.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — PlayaK Phase A source adapters + orchestration submission wiring
|
||||
|
||||
Requested:
|
||||
- Continue PlayaK implementation and document progress.
|
||||
|
||||
Implemented:
|
||||
- Added source-adapter interface module in PlayaK:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/sources.py`
|
||||
- adapters + validation rules for `subsonic`, `local`, `youtube`, `soulseek`
|
||||
- Added media library persistence model:
|
||||
- SQLite `media_items` table + index in `/home/translator/docker/appdata/playak/backend/app/db.py`
|
||||
- media CRUD helpers in `/home/translator/docker/appdata/playak/backend/app/store.py`
|
||||
- Added orchestration submission bridge:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py`
|
||||
- `stub` mode (default) issues deterministic local submit IDs
|
||||
- optional `http` mode posts payloads to configured endpoint
|
||||
- Extended queue model/state:
|
||||
- added `processing_preset`, `orchestration_state`, `stemengine_job_id`
|
||||
- DB migration guard added for existing queue tables (`ALTER TABLE` when columns missing)
|
||||
- Added new API endpoints in `/home/translator/docker/appdata/playak/backend/app/main.py`:
|
||||
- `GET /api/playak/library`
|
||||
- `POST /api/playak/import`
|
||||
- `POST /api/playak/rooms/{room_id}/queue/from-media`
|
||||
- `POST /api/playak/rooms/{room_id}/queue/{item_id}/submit`
|
||||
- Updated runtime configuration wiring:
|
||||
- `/home/translator/docker/appdata/playak/.env.example`
|
||||
- `/home/translator/docker/appdata/playak/docker-compose.yml`
|
||||
- new vars: `PLAYAK_ORCHESTRATION_MODE`, `PLAYAK_ORCHESTRATION_HTTP_URL`, `PLAYAK_ORCHESTRATION_HTTP_TIMEOUT_SEC`
|
||||
|
||||
Documentation updated:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
|
||||
Status:
|
||||
- Phase A now includes source import scaffolding and explicit queue submission lifecycle wiring.
|
||||
- Next: replace adapter validation stubs with provider-backed integrations and bind `http` orchestration to concrete StemEngine submit/poll routes.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — PlayaK local scan ingestion + queue status synchronization
|
||||
|
||||
Requested:
|
||||
- Continue implementation and document progress.
|
||||
|
||||
Implemented:
|
||||
- Added local provider-backed scan ingestion path:
|
||||
- endpoint: `POST /api/playak/import/local-scan`
|
||||
- scanning implemented in `/home/translator/docker/appdata/playak/backend/app/sources.py`
|
||||
- allowlist enforcement via `PLAYAK_LOCAL_LIBRARY_ROOTS`
|
||||
- recursive import for common audio extensions
|
||||
- Added media deduplication in store layer by (`source`, `source_ref`) to avoid duplicate local scan inserts.
|
||||
- Added orchestration status synchronization path:
|
||||
- endpoint: `POST /api/playak/rooms/{room_id}/queue/{item_id}/sync-status`
|
||||
- logic in `/home/translator/docker/appdata/playak/backend/app/orchestration.py`
|
||||
- `stub` mode: marks submitted jobs as `ready`
|
||||
- `http` mode: polls using `PLAYAK_ORCHESTRATION_HTTP_STATUS_URL_TEMPLATE`
|
||||
- Added queue status update helper in `/home/translator/docker/appdata/playak/backend/app/store.py`.
|
||||
- Updated runtime configuration wiring:
|
||||
- `/home/translator/docker/appdata/playak/.env.example`
|
||||
- `/home/translator/docker/appdata/playak/docker-compose.yml`
|
||||
- new vars: `PLAYAK_ORCHESTRATION_HTTP_STATUS_URL_TEMPLATE`, `PLAYAK_LOCAL_LIBRARY_ROOTS`
|
||||
|
||||
Documentation updated:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
|
||||
Status:
|
||||
- Phase A now includes a real local scan ingestion workflow and queue status synchronization loop.
|
||||
- Next: integrate concrete Subsonic adapter and bind HTTP orchestration to real StemEngine endpoints.
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 implementation — PlayaK real StemEngine bridge + Subsonic provider integration
|
||||
|
||||
Requested:
|
||||
- Go ahead with the next implementation step.
|
||||
|
||||
Implemented:
|
||||
- Reworked PlayaK `http` orchestration mode to support real StemEngine routing when `PLAYAK_STEMENGINE_BASE_URL` is configured:
|
||||
- local source → `POST /api/separate/from-path`
|
||||
- Subsonic source → `POST /api/subsonic/import` then `POST /api/separate/from-path`
|
||||
- job sync → `GET /api/jobs/{job_id}`
|
||||
- Added env-driven preset mapping for StemEngine model/output/power configuration.
|
||||
- Added concrete Subsonic provider bridge helpers in:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/stemengine_bridge.py`
|
||||
- Added new provider endpoints in PlayaK:
|
||||
- `GET /api/playak/providers/subsonic/search`
|
||||
- `POST /api/playak/providers/subsonic/import`
|
||||
- Updated PlayaK runtime environment config for StemEngine route/path overrides and preset defaults.
|
||||
|
||||
Documentation updated:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
|
||||
Status:
|
||||
- PlayaK now supports real StemEngine-backed submit/poll flows for local and Subsonic sources in `http` mode.
|
||||
- Next: add targeted smoke tests and extend concrete bridge support to YouTube/SoulSeek flows.
|
||||
|
||||
Validation:
|
||||
- Host-side live smoke against `http://localhost:8000`: passed
|
||||
- Focused API regression suite (`backend/tests/test_bridge_api.py`): `2 tests, OK`
|
||||
|
||||
---
|
||||
|
||||
### 2026-03-18 validation — PlayaK localhost real-bridge smoke
|
||||
|
||||
Requested:
|
||||
- Use `http://localhost:8000` as the base URL for testing and document the results.
|
||||
|
||||
Implemented:
|
||||
- Added host-side smoke assets:
|
||||
- `/home/translator/docker/appdata/playak/backend/scripts/smoke_real_bridge.py`
|
||||
- `/home/translator/docker/appdata/playak/.env.host-test.example`
|
||||
- Ran live smoke against local StemEngine at `http://localhost:8000`.
|
||||
- Observed successful end-to-end control-plane behavior:
|
||||
- PlayaK registered a local media item using a StemEngine runtime-visible import path
|
||||
- queue submit returned real StemEngine job id: `bd95ecc8-adec-4479-af33-b07b948a5c8e`
|
||||
- queue status sync returned `status=processing`, `orchestration_state=submitted`
|
||||
|
||||
Important finding:
|
||||
- Real `POST /api/separate/from-path` submit requires a path visible to the running StemEngine process (e.g. `/app/media/imports/...`), not a raw host path.
|
||||
- The smoke runner now auto-discovers a valid runtime-visible source path from StemEngine `/api/subsonic/imports` to avoid false negatives.
|
||||
|
||||
Documentation updated:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
|
||||
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
|
||||
|
||||
Status:
|
||||
- Localhost real-bridge smoke passed.
|
||||
- Next: convert this smoke path into automated regression coverage and extend real bridge support to YouTube/SoulSeek.
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# 2026-03-18 — Plan: Replace lyric line segmentation with `karaoke-lyrics-processor`
|
||||
|
||||
## Implementation status (2026-03-18)
|
||||
|
||||
✅ Implemented.
|
||||
|
||||
Delivered:
|
||||
- backend dependency added: `karaoke-lyrics-processor`
|
||||
- `lyric_transcriber_service.py` now supports `STEMENGINE_LYRIC_SEGMENTER` modes:
|
||||
- `auto` (default)
|
||||
- `karaoke_processor`
|
||||
- `legacy`
|
||||
- max line-length control via `STEMENGINE_LYRIC_MAX_LINE_LENGTH`
|
||||
- plain-text lyrics segmentation path switched to karaoke processor with safe fallback
|
||||
- WhisperX long-segment split path switched to karaoke processor with conservative word-timestamp remap and legacy fallback on ambiguity
|
||||
- docs updated in root README + configuration reference + troubleshooting matrix + developer guide
|
||||
|
||||
Smoke validation snapshot:
|
||||
- `auto`: line_count=4, split_segment_count=3, timestamps_monotonic=true
|
||||
- `legacy`: line_count=3, split_segment_count=2, timestamps_monotonic=true
|
||||
|
||||
## Objective
|
||||
Replace the in-house line segmentation heuristics used in lyric transcription with the `KaraokeLyricsProcessor` implementation from:
|
||||
- https://github.com/nomadkaraoke/karaoke-lyrics-processor/tree/main/karaoke_lyrics_processor
|
||||
|
||||
Target outcome:
|
||||
- better line breaks for karaoke readability
|
||||
- fewer unnatural splits in long plain-text lyrics
|
||||
- one consistent segmentation strategy for forced alignment input
|
||||
|
||||
## Current behavior (baseline)
|
||||
Current segmentation is implemented in `backend/services/lyric_transcriber_service.py` via:
|
||||
- `_chunk_words_sentence_aware()`
|
||||
- `_segment_plain_lyrics_lines()`
|
||||
- `_split_long_segments()`
|
||||
- `_lyrics_to_whisperx_segments()` (plain-text path)
|
||||
|
||||
Current logic relies on punctuation/uppercase/max-word heuristics.
|
||||
|
||||
## Proposed approach
|
||||
|
||||
### 1) Introduce a segmentation adapter (no big-bang rewrite)
|
||||
Create an internal wrapper function (same module or a small helper module) that calls `KaraokeLyricsProcessor` with `input_lyrics_text` and returns normalized lines.
|
||||
|
||||
Proposed internal API:
|
||||
- `segment_lyrics_lines(text: str, max_line_length: int = 36) -> list[str]`
|
||||
|
||||
Normalization rules:
|
||||
- trim whitespace lines
|
||||
- preserve order
|
||||
- ignore blank results
|
||||
- return fallback line list if external processor raises
|
||||
|
||||
### 2) Dependency strategy
|
||||
Add Python dependency in backend requirements:
|
||||
- `karaoke-lyrics-processor` (pinned major/minor after validation)
|
||||
|
||||
Keep existing heuristic functions temporarily as fallback until rollout is complete.
|
||||
|
||||
### 3) Integration points
|
||||
Replace segmentation calls in two places first:
|
||||
1. Plain-text forced-alignment path in `_lyrics_to_whisperx_segments()`.
|
||||
2. Long-segment post-processing in `_split_long_segments()` for WhisperX output.
|
||||
|
||||
Implementation note:
|
||||
- For WhisperX word-timestamp chunks, segment using text and remap grouped words conservatively (do not degrade timing fidelity).
|
||||
- If remap confidence is low (token mismatch), keep original segment unchanged.
|
||||
|
||||
### 4) Backward compatibility controls
|
||||
Add feature flag:
|
||||
- `STEMENGINE_LYRIC_SEGMENTER` with values:
|
||||
- `legacy` (existing behavior)
|
||||
- `karaoke_processor` (new behavior)
|
||||
- `auto` (default after bake-in period; new with safe fallback)
|
||||
|
||||
Optional tuning variable:
|
||||
- `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (default `36`)
|
||||
|
||||
### 5) Rollout phases
|
||||
|
||||
#### Phase A — Shadow mode (no behavior change)
|
||||
- Wire adapter and dependency.
|
||||
- Execute both legacy and new segmenters for sampled jobs.
|
||||
- Store debug comparison in logs (line count, average chars/line, large mismatch markers).
|
||||
|
||||
#### Phase B — Opt-in
|
||||
- Enable `karaoke_processor` via env flag in staging.
|
||||
- Validate output quality on multilingual and punctuation-poor lyrics.
|
||||
|
||||
#### Phase C — Default-on with fallback
|
||||
- Set `auto` default.
|
||||
- Auto-fallback to legacy on adapter exceptions or mapping mismatch.
|
||||
|
||||
#### Phase D — Cleanup
|
||||
- Remove dead legacy paths after stability window.
|
||||
- Keep small compatibility shim if needed for old jobs.
|
||||
|
||||
## Validation plan
|
||||
|
||||
### Unit tests
|
||||
- segmentation adapter returns stable non-empty lines for normal input
|
||||
- preserves parentheses/quoted text handling expected by karaoke UI
|
||||
- enforces max line length behavior
|
||||
- safe fallback when processor throws
|
||||
|
||||
### Integration tests
|
||||
- forced alignment with plain text lyrics still produces valid LRC/ELRC
|
||||
- existing LRC input path remains unchanged
|
||||
- WhisperX post-segmentation preserves timestamp monotonicity and non-overlap
|
||||
|
||||
### Regression corpus
|
||||
Test against representative lyrics sets:
|
||||
- lowercase/no punctuation
|
||||
- rap/ad-libs with dense tokens
|
||||
- multilingual text with non-ASCII characters
|
||||
- parenthetical structure
|
||||
|
||||
Acceptance criteria:
|
||||
- no increase in failed lyric jobs
|
||||
- improved line readability in manual QA samples
|
||||
- no timestamp regressions in LRC/ELRC exports
|
||||
|
||||
## Risk register
|
||||
- **Word-timestamp remap drift** when splitting WhisperX segments by text alone.
|
||||
- Mitigation: conservative remap; fallback to original segment when uncertain.
|
||||
- **Dependency behavior changes** in upstream processor.
|
||||
- Mitigation: pin version + smoke tests in CI.
|
||||
- **Performance overhead** on long lyrics.
|
||||
- Mitigation: benchmark and keep adapter lightweight.
|
||||
|
||||
## Files expected to change (implementation phase)
|
||||
- `backend/requirements.txt`
|
||||
- `backend/services/lyric_transcriber_service.py`
|
||||
- `backend/tests/*` (new unit/integration tests for adapter + mapping)
|
||||
- `README.md` (new env vars)
|
||||
- `docs/CONFIGURATION_REFERENCE.md` (segmenter flags)
|
||||
- `docs/TROUBLESHOOTING_MATRIX.md` (new failure mode/fallback notes)
|
||||
|
||||
## Documentation updates included now
|
||||
This plan is documented in:
|
||||
- `20260318_Lyric_Line_Segmentation_Replacement_Plan.md`
|
||||
- `20260313_Fixes (Copilot).md` (decision log entry)
|
||||
@@ -0,0 +1,123 @@
|
||||
# 2026-03-18 — PlayaK implementation progress
|
||||
|
||||
## Request
|
||||
|
||||
Create a new container/workspace named **PlayaK** at:
|
||||
- `/home/translator/docker/appdata/playak`
|
||||
|
||||
Move the karaoke player platform plan to that location and document progress.
|
||||
|
||||
## Completed
|
||||
|
||||
1. Created PlayaK workspace directory:
|
||||
- `/home/translator/docker/appdata/playak`
|
||||
2. Added PlayaK root documentation:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
3. Moved planning artifact into PlayaK (recreated in new location):
|
||||
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Karaoke_Player_Platform_Plan.md`
|
||||
|
||||
## Additional completed (implementation kickoff)
|
||||
|
||||
4. Added PlayaK runtime scaffold:
|
||||
- `/home/translator/docker/appdata/playak/docker-compose.yml`
|
||||
- `/home/translator/docker/appdata/playak/.env.example`
|
||||
- `/home/translator/docker/appdata/playak/.gitignore`
|
||||
5. Added backend API scaffold:
|
||||
- `/home/translator/docker/appdata/playak/backend/Dockerfile`
|
||||
- `/home/translator/docker/appdata/playak/backend/requirements.txt`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/models.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/store.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/security.py`
|
||||
6. Updated PlayaK root docs with run instructions and scaffold status:
|
||||
- `/home/translator/docker/appdata/playak/README.md`
|
||||
7. Added SQLite persistence for PlayaK room/queue state:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/db.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/store.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py` (startup DB init)
|
||||
- `/home/translator/docker/appdata/playak/docker-compose.yml` (`./data:/data` + `PLAYAK_DB_PATH`)
|
||||
8. Added auth/authorization scaffold with room-role enforcement:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/security.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/store.py` (`room_members` helpers)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py` (permission checks + new endpoints)
|
||||
- new endpoints:
|
||||
- `POST /api/playak/rooms/{room_id}/join`
|
||||
- `GET /api/playak/rooms/{room_id}/state`
|
||||
9. Added source-ingestion adapter interfaces + orchestration submission wiring:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/sources.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py`
|
||||
- `/home/translator/docker/appdata/playak/backend/app/db.py` (`media_items` + queue orchestration columns)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/store.py` (media persistence + queue submission updates)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new import/library/submit endpoints)
|
||||
- new endpoints:
|
||||
- `GET /api/playak/library`
|
||||
- `POST /api/playak/import`
|
||||
- `POST /api/playak/rooms/{room_id}/queue/from-media`
|
||||
- `POST /api/playak/rooms/{room_id}/queue/{item_id}/submit`
|
||||
10. Added local provider scan + orchestration status sync wiring:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/sources.py` (scan allowlist + recursive local discovery)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py` (status polling support)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/store.py` (media dedupe + queue status update)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new endpoints)
|
||||
- new endpoints:
|
||||
- `POST /api/playak/import/local-scan`
|
||||
- `POST /api/playak/rooms/{room_id}/queue/{item_id}/sync-status`
|
||||
11. Added real StemEngine HTTP bridge + Subsonic provider bridge:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py` (local + Subsonic real submit/poll)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/stemengine_bridge.py` (Subsonic search/import proxy)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new provider endpoints)
|
||||
- new endpoints:
|
||||
- `GET /api/playak/providers/subsonic/search`
|
||||
- `POST /api/playak/providers/subsonic/import`
|
||||
12. Added host-side smoke validation for the real bridge:
|
||||
- `/home/translator/docker/appdata/playak/backend/scripts/smoke_real_bridge.py`
|
||||
- `/home/translator/docker/appdata/playak/.env.host-test.example`
|
||||
- validated against `http://localhost:8000`
|
||||
- observed real submit job id + successful `processing` sync state
|
||||
13. Added focused API regression coverage for the bridge path:
|
||||
- `/home/translator/docker/appdata/playak/backend/tests/test_bridge_api.py`
|
||||
- covers import → queue → submit → sync and Subsonic provider registration/deduplication
|
||||
- validated: `2 tests, OK`
|
||||
14. Added Phase B/C backend control-plane scaffolding:
|
||||
- `/home/translator/docker/appdata/playak/backend/app/db.py` (room policies, playback sessions, lyric/style revisions, room events, queue ordering)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/store.py` (policy/playback/editor/events CRUD + queue reorder)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new policy/moderation/playback/editor/event endpoints)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/models.py` (new Phase B/C request/response models)
|
||||
- `/home/translator/docker/appdata/playak/backend/app/security.py` (policy-aware moderation/playback checks)
|
||||
- new endpoints:
|
||||
- `GET|PATCH /api/playak/rooms/{room_id}/policy`
|
||||
- `POST /api/playak/rooms/{room_id}/leave`
|
||||
- `PATCH /api/playak/rooms/{room_id}/queue/reorder`
|
||||
- `POST /api/playak/rooms/{room_id}/playback/sessions`
|
||||
- `GET /api/playak/rooms/{room_id}/playback/session`
|
||||
- `POST /api/playak/playback/sessions/{session_id}/command`
|
||||
- `POST|GET /api/playak/media/{media_id}/lyrics-style`
|
||||
- `GET /api/playak/rooms/{room_id}/events`
|
||||
- expanded regression suite validation: `10 tests, OK`
|
||||
15. Added Phase B/C frontend surfaces (Two.js player/editor + companion PWA scaffold):
|
||||
- `/home/translator/docker/appdata/playak/frontend/index.html`
|
||||
- `/home/translator/docker/appdata/playak/frontend/controller.html`
|
||||
- `/home/translator/docker/appdata/playak/frontend/player.js`
|
||||
- `/home/translator/docker/appdata/playak/frontend/controller.js`
|
||||
- `/home/translator/docker/appdata/playak/frontend/api.js`
|
||||
- `/home/translator/docker/appdata/playak/frontend/app.css`
|
||||
- `/home/translator/docker/appdata/playak/frontend/manifest.webmanifest`
|
||||
- `/home/translator/docker/appdata/playak/frontend/sw.js`
|
||||
- `/home/translator/docker/appdata/playak/frontend/assets/icon.svg`
|
||||
- compose runtime for static hosting (`playak-frontend` service on `PLAYAK_FRONTEND_PORT`, default `18101`)
|
||||
- API CORS wiring added (`PLAYAK_CORS_ORIGINS`)
|
||||
- Current data layer persists to SQLite (room/queue state survives container restarts when `./data` is retained).
|
||||
- Media library state also persists to SQLite (`media_items`).
|
||||
- Auth is currently header-based (`X-Global-Role`, `X-User-Id`) and intended as a scaffold before full session/token integration.
|
||||
- Local source now has provider-backed scan/import path (allowlist-enforced).
|
||||
- Subsonic now has concrete search/import bridge paths through StemEngine.
|
||||
- YouTube/SoulSeek adapters still need concrete provider runtime integration.
|
||||
- SoulSeek adapter still needs concrete provider runtime integration (held as requested).
|
||||
- Orchestration bridge now supports real StemEngine local/Subsonic submit + job polling when `PLAYAK_STEMENGINE_BASE_URL` is configured.
|
||||
- Live smoke validation passed against local StemEngine using a runtime-visible imports path.
|
||||
- Phase B/C backend APIs are now available for frontend and PWA integration.
|
||||
- Phase B/C frontend surfaces are now scaffolded and connected to those APIs.
|
||||
|
||||
## Next suggested step
|
||||
|
||||
- Add live YouTube ytdl end-to-end smoke against StemEngine and harden frontend/PWA scaffold (state management, realtime transport, session auth).
|
||||
@@ -4,6 +4,19 @@
|
||||
|
||||
---
|
||||
|
||||
## Documentation
|
||||
|
||||
- User docs: [docs/USER_GUIDE.md](docs/USER_GUIDE.md)
|
||||
- Developer docs: [docs/DEVELOPER_GUIDE.md](docs/DEVELOPER_GUIDE.md)
|
||||
- Documentation plan: [docs/DOCUMENTATION_PLAN.md](docs/DOCUMENTATION_PLAN.md)
|
||||
- API reference: [docs/API_REFERENCE.md](docs/API_REFERENCE.md)
|
||||
- Troubleshooting matrix: [docs/TROUBLESHOOTING_MATRIX.md](docs/TROUBLESHOOTING_MATRIX.md)
|
||||
- Configuration reference: [docs/CONFIGURATION_REFERENCE.md](docs/CONFIGURATION_REFERENCE.md)
|
||||
- Documentation governance: [docs/DOC_GOVERNANCE.md](docs/DOC_GOVERNANCE.md)
|
||||
- Release checklist: [docs/RELEASE_CHECKLIST.md](docs/RELEASE_CHECKLIST.md)
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **Stem Separation** — 40+ models across 10 model families
|
||||
@@ -82,6 +95,8 @@ All configuration is via environment variables. The `docker-compose.yml` accepts
|
||||
| `MAX_UPLOAD_MB` | `500` | Maximum upload file size in MB |
|
||||
| `STEMENGINE_GPU_VRAM_RESERVE_MB` | `2048` | Keeps this many MB free on the GPU (WSL2-safe headroom) |
|
||||
| `OMNIZART_GPU_VRAM_RESERVE_MB` | `STEMENGINE_GPU_VRAM_RESERVE_MB` | Optional sidecar override for Omnizart TensorFlow GPU cap |
|
||||
| `STEMENGINE_LYRIC_SEGMENTER` | `auto` | Lyric line splitter strategy: `auto`, `karaoke_processor`, `legacy` |
|
||||
| `STEMENGINE_LYRIC_MAX_LINE_LENGTH` | `36` | Max line length target used by karaoke lyric segmentation |
|
||||
| `TZ` | `UTC` | Timezone |
|
||||
| `NAVSERVER` | — | Subsonic server URL (can also be set in the UI) |
|
||||
| `NAVUSER` | — | Subsonic username |
|
||||
|
||||
@@ -362,6 +362,17 @@ DEFAULT_PARAKEET_MODEL = os.environ.get(
|
||||
"parakeet-1.1b-rnnt-multilingual-asr",
|
||||
)
|
||||
|
||||
# Lyric line segmentation strategy
|
||||
# - legacy: current punctuation/uppercase/max-word heuristics
|
||||
# - karaoke_processor: always use karaoke-lyrics-processor splitter
|
||||
# - auto: karaoke-lyrics-processor with safe fallback to legacy heuristics
|
||||
LYRIC_SEGMENTER: str = os.environ.get("STEMENGINE_LYRIC_SEGMENTER", "auto").strip().lower() or "auto"
|
||||
|
||||
try:
|
||||
LYRIC_MAX_LINE_LENGTH: int = max(8, int(os.environ.get("STEMENGINE_LYRIC_MAX_LINE_LENGTH", "36")))
|
||||
except ValueError:
|
||||
LYRIC_MAX_LINE_LENGTH = 36
|
||||
|
||||
# Create directories on import
|
||||
UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
|
||||
JOBS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -75,6 +75,9 @@ audio-separator[gpu]>=0.28.0
|
||||
# Lyric transcription via WhisperX (faster-whisper backend + phoneme-level word alignment)
|
||||
whisperx>=3.8.1
|
||||
|
||||
# Karaoke-oriented lyric line segmentation
|
||||
karaoke-lyrics-processor>=1.0.0
|
||||
|
||||
# Phoneme-level syllable alignment (requires espeak-ng system package)
|
||||
# Used by the "Phoneme alignment" option in AI Transcribe to produce
|
||||
# syllable-level ELRC timestamps via facebook/wav2vec2-xlsr-53-espeak-cv-ft.
|
||||
|
||||
@@ -66,6 +66,8 @@ from ..config import (
|
||||
PARAKEET_TRANSCRIBE_PATH,
|
||||
PARAKEET_TIMEOUT_SEC,
|
||||
DEFAULT_PARAKEET_MODEL,
|
||||
LYRIC_SEGMENTER,
|
||||
LYRIC_MAX_LINE_LENGTH,
|
||||
)
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -783,6 +785,16 @@ def _ortho_split(word_text: str, n: int) -> List[str]:
|
||||
|
||||
|
||||
_PUNCT_BREAK = frozenset('.!?,;')
|
||||
_SEGMENTER_MODES = {"legacy", "karaoke_processor", "auto"}
|
||||
|
||||
if LYRIC_SEGMENTER not in _SEGMENTER_MODES:
|
||||
log.warning(
|
||||
"Invalid STEMENGINE_LYRIC_SEGMENTER=%r (expected one of %s); defaulting to 'auto'.",
|
||||
LYRIC_SEGMENTER,
|
||||
sorted(_SEGMENTER_MODES),
|
||||
)
|
||||
|
||||
_ACTIVE_LYRIC_SEGMENTER = LYRIC_SEGMENTER if LYRIC_SEGMENTER in _SEGMENTER_MODES else "auto"
|
||||
|
||||
|
||||
def _starts_with_sentence_capital(token: str) -> bool:
|
||||
@@ -843,8 +855,8 @@ def _chunk_words_sentence_aware(words: list, max_words: int = 10, min_sentence_w
|
||||
return chunks
|
||||
|
||||
|
||||
def _segment_plain_lyrics_lines(lyrics_text: str, max_words: int = 10) -> list[str]:
|
||||
"""Split plain lyrics into sentence-like lines with lowercase fallback."""
|
||||
def _segment_plain_lyrics_lines_legacy(lyrics_text: str, max_words: int = 10) -> list[str]:
|
||||
"""Legacy splitter: sentence-like chunking with lowercase fallback."""
|
||||
src_lines = [ln.strip() for ln in lyrics_text.splitlines() if ln.strip()]
|
||||
out: list[str] = []
|
||||
|
||||
@@ -867,22 +879,49 @@ def _segment_plain_lyrics_lines(lyrics_text: str, max_words: int = 10) -> list[s
|
||||
return out
|
||||
|
||||
|
||||
def _split_long_segments(segments: list, max_words: int = 10) -> list:
|
||||
"""
|
||||
Split over-long WhisperX segments into shorter lines suitable for LRC display.
|
||||
def _norm_word_token(token: str) -> str:
|
||||
"""Normalize a token for conservative line→word remapping."""
|
||||
return "".join(ch for ch in (token or "").lower() if ch.isalnum() or ch == "'")
|
||||
|
||||
Strategy (applied per segment):
|
||||
1. Walk the word list looking for *punctuation boundaries* — words whose
|
||||
text ends with a character in _PUNCT_BREAK. When such a boundary is
|
||||
found AND the current chunk already has at least 3 words, emit a new
|
||||
sub-segment there.
|
||||
2. If no punctuation boundary is found within `max_words` words, force-
|
||||
split at `max_words` regardless.
|
||||
|
||||
Each sub-segment inherits the speaker label and gets timestamps from the
|
||||
first / last word in the slice. Segments with fewer than 2 words are left
|
||||
unchanged.
|
||||
"""
|
||||
def _line_tokens_for_mapping(text: str) -> list[str]:
|
||||
"""Tokenize line text for word-level mapping (keeps apostrophes)."""
|
||||
raw_tokens = _re.findall(r"[\w']+", text or "", flags=_re.UNICODE)
|
||||
return [t for t in (_norm_word_token(tok) for tok in raw_tokens) if t]
|
||||
|
||||
|
||||
def _segment_with_karaoke_processor(lyrics_text: str, max_line_length: int) -> list[str]:
|
||||
"""Split lyrics with `karaoke-lyrics-processor` and normalize output lines."""
|
||||
from karaoke_lyrics_processor import KaraokeLyricsProcessor
|
||||
|
||||
processor = KaraokeLyricsProcessor(
|
||||
input_lyrics_text=lyrics_text,
|
||||
max_line_length=max_line_length,
|
||||
log_level=logging.WARNING,
|
||||
)
|
||||
processed = processor.process()
|
||||
return [ln.strip() for ln in processed.splitlines() if ln.strip()]
|
||||
|
||||
|
||||
def _segment_plain_lyrics_lines(lyrics_text: str, max_words: int = 10) -> list[str]:
|
||||
"""Split plain lyrics into line-sized chunks suitable for karaoke display."""
|
||||
if _ACTIVE_LYRIC_SEGMENTER == "legacy":
|
||||
return _segment_plain_lyrics_lines_legacy(lyrics_text, max_words=max_words)
|
||||
|
||||
try:
|
||||
return _segment_with_karaoke_processor(lyrics_text, max_line_length=LYRIC_MAX_LINE_LENGTH)
|
||||
except Exception as exc:
|
||||
# `auto` and `karaoke_processor` both fail-safe to legacy so lyric jobs
|
||||
# do not fail on dependency/runtime edge cases.
|
||||
log.warning(
|
||||
"karaoke lyric segmentation unavailable (%s); falling back to legacy segmentation.",
|
||||
exc,
|
||||
)
|
||||
return _segment_plain_lyrics_lines_legacy(lyrics_text, max_words=max_words)
|
||||
|
||||
|
||||
def _split_long_segments_legacy(segments: list, max_words: int = 10) -> list:
|
||||
"""Legacy splitter for WhisperX segments using punctuation/max-word heuristics."""
|
||||
if max_words <= 0:
|
||||
return segments
|
||||
|
||||
@@ -893,13 +932,7 @@ def _split_long_segments(segments: list, max_words: int = 10) -> list:
|
||||
result.append(seg)
|
||||
continue
|
||||
|
||||
# Sentence-aware chunking:
|
||||
# 1) punctuation boundaries
|
||||
# 2) uppercase sentence starts
|
||||
# 3) max-words fallback for lowercase/unpunctuated lyrics
|
||||
chunks = _chunk_words_sentence_aware(words, max_words=max_words, min_sentence_words=3)
|
||||
|
||||
# If no effective split happened, keep original segment.
|
||||
if len(chunks) <= 1:
|
||||
result.append(seg)
|
||||
continue
|
||||
@@ -918,6 +951,105 @@ def _split_long_segments(segments: list, max_words: int = 10) -> list:
|
||||
return result
|
||||
|
||||
|
||||
def _remap_segment_words_to_lines(seg: dict, split_lines: list[str]) -> Optional[list[dict]]:
|
||||
"""Map line-split text back to contiguous WhisperX word slices conservatively."""
|
||||
words = seg.get("words") or []
|
||||
if len(words) <= 1 or len(split_lines) <= 1:
|
||||
return None
|
||||
|
||||
remapped: list[dict] = []
|
||||
cursor = 0
|
||||
|
||||
for line in split_lines:
|
||||
line_tokens = _line_tokens_for_mapping(line)
|
||||
if not line_tokens:
|
||||
continue
|
||||
|
||||
start_idx: Optional[int] = None
|
||||
for token in line_tokens:
|
||||
matched = False
|
||||
while cursor < len(words):
|
||||
word_token = _norm_word_token(words[cursor].get("word", ""))
|
||||
if not word_token:
|
||||
cursor += 1
|
||||
continue
|
||||
if word_token == token:
|
||||
if start_idx is None:
|
||||
start_idx = cursor
|
||||
cursor += 1
|
||||
matched = True
|
||||
break
|
||||
return None
|
||||
if not matched:
|
||||
return None
|
||||
|
||||
if start_idx is None:
|
||||
return None
|
||||
|
||||
end_idx = max(start_idx, cursor - 1)
|
||||
chunk_words = words[start_idx:end_idx + 1]
|
||||
if not chunk_words:
|
||||
return None
|
||||
|
||||
first, last = chunk_words[0], chunk_words[-1]
|
||||
remapped.append({
|
||||
"start": first.get("start") if first.get("start") is not None else seg["start"],
|
||||
"end": last.get("end") if last.get("end") is not None else seg["end"],
|
||||
"text": line.strip(),
|
||||
"words": chunk_words,
|
||||
"speaker": seg.get("speaker"),
|
||||
})
|
||||
|
||||
# If we did not consume all meaningful residual tokens, mapping was unsafe.
|
||||
if any(_norm_word_token(w.get("word", "")) for w in words[cursor:]):
|
||||
return None
|
||||
|
||||
return remapped if len(remapped) > 1 else None
|
||||
|
||||
|
||||
def _split_long_segments(segments: list, max_words: int = 10) -> list:
|
||||
"""
|
||||
Split over-long WhisperX segments into karaoke-friendly lines.
|
||||
|
||||
`legacy` mode uses punctuation/uppercase/max-word heuristics.
|
||||
`karaoke_processor`/`auto` mode uses karaoke-lyrics-processor first, then
|
||||
conservatively remaps generated lines back to contiguous word slices.
|
||||
"""
|
||||
if _ACTIVE_LYRIC_SEGMENTER == "legacy":
|
||||
return _split_long_segments_legacy(segments, max_words=max_words)
|
||||
|
||||
result: list[dict] = []
|
||||
for seg in segments:
|
||||
words = seg.get("words") or []
|
||||
if len(words) <= 1:
|
||||
result.append(seg)
|
||||
continue
|
||||
|
||||
try:
|
||||
split_lines = _segment_with_karaoke_processor(seg.get("text", ""), max_line_length=LYRIC_MAX_LINE_LENGTH)
|
||||
except Exception as exc:
|
||||
log.warning(
|
||||
"karaoke segment split unavailable for WhisperX segment (%s); falling back to legacy split.",
|
||||
exc,
|
||||
)
|
||||
result.extend(_split_long_segments_legacy([seg], max_words=max_words))
|
||||
continue
|
||||
|
||||
if len(split_lines) <= 1:
|
||||
result.append(seg)
|
||||
continue
|
||||
|
||||
remapped = _remap_segment_words_to_lines(seg, split_lines)
|
||||
if remapped:
|
||||
result.extend(remapped)
|
||||
else:
|
||||
# Safe fallback if token remap is ambiguous (punctuation-heavy,
|
||||
# language-specific tokenization, etc.).
|
||||
result.extend(_split_long_segments_legacy([seg], max_words=max_words))
|
||||
|
||||
return result
|
||||
|
||||
|
||||
_VOCAL_KEYWORDS = frozenset(('vocal', 'vocals', 'voice', 'lead', 'sing'))
|
||||
_AUDIO_EXTS = frozenset(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac'))
|
||||
|
||||
@@ -2004,8 +2136,9 @@ def run_lyric_transcription(
|
||||
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
|
||||
return
|
||||
|
||||
# Split over-long segments into shorter lyric lines before converting.
|
||||
# Punctuation boundaries are preferred; falls back to a hard word-count cut.
|
||||
# Split over-long segments into karaoke-friendly lyric lines before converting.
|
||||
# Uses karaoke-lyrics-processor with conservative word-timestamp remap,
|
||||
# then falls back to legacy heuristics when remap is ambiguous.
|
||||
raw_segments = _split_long_segments(wx_result["segments"], max_words=10)
|
||||
|
||||
# Convert WhisperX output → LyricTranscriptionResult
|
||||
|
||||
@@ -0,0 +1,397 @@
|
||||
# StemEngine API Reference (Phase 2)
|
||||
|
||||
This is a practical API map for contributors. It focuses on endpoint groups, request shape, and common response patterns.
|
||||
|
||||
## Base conventions
|
||||
|
||||
- Base URL (default): `http://localhost:8000`
|
||||
- Most long-running operations return a `job_id` and run asynchronously.
|
||||
- Poll status via `GET /api/jobs/{job_id}`.
|
||||
- Download job outputs via `GET /api/jobs/{job_id}/download/{filename}` when status is `done`.
|
||||
|
||||
## Common async pattern
|
||||
|
||||
1. Start operation (for example separation/transcription/alignment).
|
||||
2. Receive `{ "job_id": "..." }`.
|
||||
3. Poll job until `status` is `done`, `error`, or `cancelled`.
|
||||
4. Read `files` and `result` from job payload.
|
||||
|
||||
---
|
||||
|
||||
## System and operations
|
||||
|
||||
### GET /api/system/health
|
||||
Lightweight liveness probe.
|
||||
|
||||
### GET /api/system/readiness
|
||||
Readiness checks for DB and writable directories.
|
||||
|
||||
### GET /api/system/executors
|
||||
Queue and worker state from the job manager.
|
||||
|
||||
### GET /api/system/gpu-info
|
||||
GPU availability + VRAM figures + transcription CUDA health flag.
|
||||
|
||||
### POST /api/system/gpu-reset
|
||||
Admin-protected reset path for GPU recovery and cache flush.
|
||||
|
||||
---
|
||||
|
||||
## Jobs
|
||||
|
||||
### GET /api/jobs
|
||||
List jobs.
|
||||
|
||||
### GET /api/jobs/{job_id}
|
||||
Get one job payload.
|
||||
|
||||
### DELETE /api/jobs/{job_id}
|
||||
Request cancellation.
|
||||
|
||||
### GET /api/jobs/{job_id}/download/{filename}
|
||||
Download one output file from a completed job.
|
||||
|
||||
---
|
||||
|
||||
## Separation
|
||||
|
||||
### GET /api/separate/models
|
||||
Returns available separation models.
|
||||
|
||||
### GET /api/separate/formats
|
||||
Returns output formats.
|
||||
|
||||
### GET /api/separate/presets
|
||||
Returns preset types and static options.
|
||||
|
||||
### GET /api/separate/per-instrument-models
|
||||
Returns models usable for per-stem reprocessing.
|
||||
|
||||
### POST /api/separate
|
||||
Multipart upload separation request.
|
||||
|
||||
Form fields:
|
||||
- `file` (required)
|
||||
- `model`
|
||||
- `output_format`
|
||||
- `preset_type`
|
||||
- `static_option`
|
||||
- `dynamic_levels` (JSON string)
|
||||
- `project_name`
|
||||
- `instrument_models` (JSON string)
|
||||
- `sep_power_preset`
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"job_id": "f52d...",
|
||||
"preset_type": "stems_only",
|
||||
"project_name": "Artist - Title"
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/separate/existing
|
||||
Re-run separation on an existing project source.
|
||||
|
||||
### POST /api/separate/from-path
|
||||
Run separation from a server-side file path (restricted to imports directory).
|
||||
|
||||
---
|
||||
|
||||
## MIDI transcription
|
||||
|
||||
### GET /api/transcribe/modes
|
||||
Returns available modes and availability flags.
|
||||
|
||||
### POST /api/transcribe
|
||||
Multipart upload request:
|
||||
- `file`
|
||||
- `mode`
|
||||
|
||||
### POST /api/transcribe/from-project
|
||||
Form request:
|
||||
- `project_name`
|
||||
- `mode`
|
||||
|
||||
Selects the best non-drum stem by preference order.
|
||||
|
||||
---
|
||||
|
||||
## Lyric transcription (ASR)
|
||||
|
||||
### GET /api/lyric-transcribe/models
|
||||
Returns engines/models/defaults and runtime availability.
|
||||
|
||||
### GET /api/lyric-transcribe/languages
|
||||
Returns supported language choices.
|
||||
|
||||
### GET /api/lyric-transcribe/formats
|
||||
Returns output formats (`lrc`, `elrc`, `json`, `txt`, `srt`, `vtt`).
|
||||
|
||||
### POST /api/lyric-transcribe
|
||||
Multipart upload request.
|
||||
|
||||
Form fields:
|
||||
- `file`
|
||||
- `engine` (`whisperx` | `parakeet`)
|
||||
- `model`
|
||||
- `language`
|
||||
- `task` (`transcribe` | `translate`, with engine constraints)
|
||||
- `output_formats` (comma-separated)
|
||||
- `initial_prompt`
|
||||
- `temperature`
|
||||
|
||||
### POST /api/lyric-transcribe/from-project
|
||||
Form fields include:
|
||||
- `project_name`
|
||||
- `stem_name` (optional; vocals auto-detection if absent)
|
||||
- transcription options above
|
||||
|
||||
### POST /api/lyric-transcribe/with-diarization
|
||||
Multipart upload + diarization options.
|
||||
|
||||
### POST /api/lyric-transcribe/project/{project_id}
|
||||
JSON body version for project-stem lyric transcription.
|
||||
|
||||
Example JSON body:
|
||||
|
||||
```json
|
||||
{
|
||||
"stem_name": "vocals",
|
||||
"engine": "whisperx",
|
||||
"model": "large-v2",
|
||||
"task": "transcribe",
|
||||
"output_formats": ["elrc", "json", "txt"],
|
||||
"temperature": 0.0,
|
||||
"condition_on_previous_text": false,
|
||||
"word_timestamps": true
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## LRC / ELRC alignment
|
||||
|
||||
### GET /api/lrc/sources
|
||||
Returns source modes: `auto`, `tag`, `file`, `text`.
|
||||
|
||||
### POST /api/lrc/extract
|
||||
Extracts lyrics from uploaded audio metadata/source.
|
||||
|
||||
### POST /api/lrc/align
|
||||
Multipart request:
|
||||
- `audio`
|
||||
- `lyrics_source`
|
||||
- `lyrics_text` (required when source=`text`)
|
||||
- `output_filename`
|
||||
- `style_options` (JSON string)
|
||||
|
||||
### POST /api/lrc/align-with-file
|
||||
Multipart request with `audio` + `lyrics_file`.
|
||||
|
||||
### POST /api/lrc/align-existing
|
||||
Align using existing project vocals stem.
|
||||
|
||||
---
|
||||
|
||||
## Projects
|
||||
|
||||
### GET /api/projects
|
||||
List projects.
|
||||
|
||||
### GET /api/projects/{project_id}
|
||||
Get project detail with linked jobs.
|
||||
|
||||
### GET /api/projects/{project_id}/cover-art
|
||||
Returns embedded art bytes if available.
|
||||
|
||||
### DELETE /api/projects/{project_id}
|
||||
Delete project and related assets.
|
||||
|
||||
### POST /api/projects/{project_id}/transcribe
|
||||
JSON body:
|
||||
|
||||
```json
|
||||
{
|
||||
"stem_name": "other",
|
||||
"mode": "polyphonic"
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/projects/{project_id}/abc
|
||||
JSON body:
|
||||
|
||||
```json
|
||||
{
|
||||
"stem_name": "other",
|
||||
"song_title": "My Song"
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/projects/{project_id}/mix
|
||||
JSON body:
|
||||
|
||||
```json
|
||||
{
|
||||
"preset_type": "dynamic",
|
||||
"dynamic_levels": {"vocals": 0.8, "drums": 1.1}
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/projects/{project_id}/lrc
|
||||
JSON body supports lyrics source, style, timing precision, and power preset.
|
||||
|
||||
### POST /api/projects/{project_id}/lyric-transcribe
|
||||
JSON body supports engine, model, alignment options, diarization, and existing lyrics alignment paths.
|
||||
|
||||
### POST /api/projects/{project_id}/reprocess
|
||||
Reprocess a single stem with a selected model.
|
||||
|
||||
### POST /api/projects/{project_id}/rename-stem
|
||||
Rename tracked stem file.
|
||||
|
||||
### POST /api/projects/{project_id}/rescan-stems
|
||||
Rescan stem directory and register untracked files.
|
||||
|
||||
### Maintenance (admin-protected)
|
||||
- `GET /api/projects/cleanup/verify`
|
||||
- `POST /api/projects/cleanup/write-metadata`
|
||||
- `POST /api/projects/cleanup/rename-stems`
|
||||
- `POST /api/projects/cleanup/purge-orphaned`
|
||||
- `POST /api/projects/cleanup/clean-lrc-files`
|
||||
|
||||
---
|
||||
|
||||
## Pipeline (end-to-end)
|
||||
|
||||
### POST /api/pipeline
|
||||
Multipart request combining separation + transcription + ABC path.
|
||||
|
||||
Form fields:
|
||||
- `file`
|
||||
- `sep_model`
|
||||
- `sep_format`
|
||||
- `transcribe_mode`
|
||||
- `song_title`
|
||||
- `instrument_models` (JSON string)
|
||||
- `lyric_engine`
|
||||
- `lyric_model`
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{ "job_id": "..." }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ABC conversion
|
||||
|
||||
### POST /api/abc-convert
|
||||
Upload one or more MIDI files and convert to ABC.
|
||||
|
||||
### POST /api/abc-convert/from-job
|
||||
Reuse MIDI files produced by an existing completed job.
|
||||
|
||||
---
|
||||
|
||||
## Model download management
|
||||
|
||||
### GET /api/models/list
|
||||
Returns model catalog and download status.
|
||||
|
||||
### POST /api/models/download
|
||||
Admin-protected start/resume download.
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{ "model_id": "bs_roformer_4stem" }
|
||||
```
|
||||
|
||||
### POST /api/models/pause
|
||||
### POST /api/models/resume
|
||||
### POST /api/models/cancel
|
||||
Admin-protected download control.
|
||||
|
||||
### GET /api/models/progress/{model_id}
|
||||
Get one model progress.
|
||||
|
||||
### GET /api/models/progress
|
||||
Get all active progress entries.
|
||||
|
||||
### DELETE /api/models/{model_id}
|
||||
Admin-protected model deletion.
|
||||
|
||||
---
|
||||
|
||||
## Settings
|
||||
|
||||
### GET /api/settings
|
||||
Returns persisted settings with sensitive values redacted (`*_set` flags).
|
||||
|
||||
### PATCH /api/settings
|
||||
Partial update. Common keys include:
|
||||
- `hf_token`
|
||||
- `whisper_model`
|
||||
- `sep_power`
|
||||
- `whisper_word_ts`
|
||||
- `whisper_condition_prev`
|
||||
- `whisper_phoneme_align`
|
||||
- `auto_steps`
|
||||
- `auto_model`
|
||||
- `auto_format`
|
||||
- `auto_presets`
|
||||
|
||||
Example patch:
|
||||
|
||||
```json
|
||||
{
|
||||
"whisper_model": "large-v2",
|
||||
"sep_power": "standard",
|
||||
"whisper_word_ts": true
|
||||
}
|
||||
```
|
||||
|
||||
### DELETE /api/settings/hf-token
|
||||
Delete stored token; env-var fallback can still apply.
|
||||
|
||||
---
|
||||
|
||||
## Subsonic integration
|
||||
|
||||
### GET /api/subsonic/config
|
||||
Returns configured state (no password echoed).
|
||||
|
||||
### POST /api/subsonic/config
|
||||
Validates and stores connection config.
|
||||
|
||||
### DELETE /api/subsonic/config
|
||||
Removes stored config.
|
||||
|
||||
### GET /api/subsonic/search
|
||||
Query params support text + title/artist/album filtering.
|
||||
|
||||
### POST /api/subsonic/import
|
||||
Imports one Subsonic song into local imports.
|
||||
|
||||
### POST /api/subsonic/import-ytdl
|
||||
Enqueues yt-dlp import job.
|
||||
|
||||
### GET /api/subsonic/imports
|
||||
Lists local imports.
|
||||
|
||||
### DELETE /api/subsonic/imports/{filename}
|
||||
Deletes one imported file.
|
||||
|
||||
---
|
||||
|
||||
## Errors and status codes (common)
|
||||
|
||||
- `400` validation/unsupported option
|
||||
- `403` security/path constraints
|
||||
- `404` missing job/project/file/model
|
||||
- `503` optional runtime unavailable (for example Omnizart/Parakeet/WhisperX)
|
||||
|
||||
For production-grade clients, treat all non-2xx as recoverable with user-readable error surfacing and retry logic where appropriate.
|
||||
@@ -0,0 +1,193 @@
|
||||
# StemEngine Configuration Reference (Phase 2)
|
||||
|
||||
This guide expands configuration from raw env var listings to practical usage guidance.
|
||||
|
||||
## 1) Configuration layers
|
||||
|
||||
StemEngine reads config from container environment variables. Typical deployment flow:
|
||||
|
||||
1. Compose file defines environment keys.
|
||||
2. `.env` provides values for compose substitutions.
|
||||
3. Backend reads env vars on startup.
|
||||
|
||||
When a setting is also available in UI (for example some integration settings), backend env still defines baseline behavior and fallbacks.
|
||||
|
||||
---
|
||||
|
||||
## 2) Core required configuration
|
||||
|
||||
| Variable | Why it matters | Recommended practice |
|
||||
|---|---|---|
|
||||
| `SECRET_KEY` | Encrypts sensitive stored values | Set a strong random value and keep it stable across restarts |
|
||||
| `SEFOLDER` (compose substitution) | Host mount root for backend/media/model paths | Use persistent storage, not ephemeral temp dirs |
|
||||
| `PROJECTS` (compose substitution) | Host project output location | Put on a volume with enough space and backups |
|
||||
|
||||
---
|
||||
|
||||
## 3) Storage and path layout
|
||||
|
||||
Backend path keys (resolved in runtime):
|
||||
|
||||
- `MEDIA_DIR`
|
||||
- `UPLOAD_DIR`
|
||||
- `JOBS_DIR`
|
||||
- `PROJECTS_DIR`
|
||||
- `DB_PATH`
|
||||
- `LRC_SYNC_DIR`
|
||||
|
||||
Model cache/directory keys include:
|
||||
|
||||
- `BS_ROFORMER_MODEL_DIR`
|
||||
- `MEL_BAND_MODEL_DIR`
|
||||
- `MDXNET_MODEL_DIR`
|
||||
- `SCNET_MODEL_DIR`
|
||||
- `MEDLEYVOX_MODEL_DIR`
|
||||
- `APOLLO_MODEL_DIR`
|
||||
- `ASPIRATION_MODEL_DIR`
|
||||
- `BANDIT_MODEL_DIR`
|
||||
|
||||
Guidance:
|
||||
|
||||
- Keep all writable paths on persistent mounts.
|
||||
- Avoid sharing the same path between unrelated services unless intended.
|
||||
- Verify write permissions for the runtime user.
|
||||
|
||||
---
|
||||
|
||||
## 4) GPU and performance tuning
|
||||
|
||||
| Variable | Effect | Practical usage |
|
||||
|---|---|---|
|
||||
| `CPU_SEPARATION` | Forces CPU path when true | Use only when no GPU is available |
|
||||
| `STEMENGINE_GPU_VRAM_RESERVE_MB` | Keeps VRAM headroom free | Increase on WSL2/shared desktop GPU systems |
|
||||
| `OMNIZART_GPU_VRAM_RESERVE_MB` | Sidecar TensorFlow VRAM reserve | Keep aligned with backend reserve policy |
|
||||
| `PYTORCH_CUDA_ALLOC_CONF` | Allocator behavior | `expandable_segments:True` helps fragmentation scenarios |
|
||||
|
||||
Recommendations:
|
||||
|
||||
- For desktop + WSL2 use, reserve enough VRAM to avoid compositor starvation.
|
||||
- If you see intermittent OOM, reduce model size and concurrency before raising complexity.
|
||||
|
||||
---
|
||||
|
||||
## 5) CORS and network access
|
||||
|
||||
| Variable | Use |
|
||||
|---|---|
|
||||
| `CORS_ORIGINS` | Exact allowed origins (comma-separated) |
|
||||
| `CORS_ORIGIN_REGEX` | Pattern for dynamic/local/private hosts |
|
||||
|
||||
Guidance:
|
||||
|
||||
- Keep CORS narrow in production.
|
||||
- Prefer explicit origins where possible.
|
||||
- Use regex only for known internal hostname ranges.
|
||||
|
||||
---
|
||||
|
||||
## 6) Optional integration settings
|
||||
|
||||
### Subsonic
|
||||
|
||||
- `SUBSONIC_URL`
|
||||
- `SUBSONIC_USERNAME`
|
||||
- `SUBSONIC_PASSWORD`
|
||||
|
||||
Notes:
|
||||
|
||||
- Can also be configured through Settings API/UI.
|
||||
- Validate credentials immediately after changes.
|
||||
|
||||
### Hugging Face token
|
||||
|
||||
- `HF_TOKEN`
|
||||
|
||||
Notes:
|
||||
|
||||
- Needed for some gated model workflows (for example diarization dependencies).
|
||||
- If both env and stored settings exist, runtime fallback behavior applies as implemented.
|
||||
|
||||
---
|
||||
|
||||
## 7) Sidecar configuration
|
||||
|
||||
### Omnizart
|
||||
|
||||
- `OMNIZART_ENABLED`
|
||||
- `OMNIZART_SIDECAR_URL`
|
||||
|
||||
Use when enabling Omnizart transcription modes. If disabled, Omnizart modes remain unavailable.
|
||||
|
||||
### Parakeet
|
||||
|
||||
- `PARAKEET_ENABLED`
|
||||
- `PARAKEET_SIDECAR_URL`
|
||||
- `PARAKEET_HEALTH_PATH`
|
||||
- `PARAKEET_TRANSCRIBE_PATH`
|
||||
- `PARAKEET_TIMEOUT_SEC`
|
||||
- `DEFAULT_PARAKEET_MODEL`
|
||||
|
||||
Use when enabling Parakeet lyric transcription. Ensure sidecar health endpoint is reachable from backend container network.
|
||||
|
||||
### Lyric line segmentation
|
||||
|
||||
- `STEMENGINE_LYRIC_SEGMENTER` (`auto` / `karaoke_processor` / `legacy`)
|
||||
- `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (default `36`)
|
||||
|
||||
Behavior:
|
||||
|
||||
- `auto`: use `karaoke-lyrics-processor` first, fallback to legacy heuristics on runtime/dependency mismatch.
|
||||
- `karaoke_processor`: prefer karaoke processor for line breaking; still fail-safe to legacy behavior if the dependency is unavailable.
|
||||
- `legacy`: keep existing punctuation/uppercase/max-word heuristic splitter only.
|
||||
|
||||
Recommended rollout:
|
||||
|
||||
1. Start with `auto`.
|
||||
2. Validate line quality against your lyrics corpus.
|
||||
3. Use `legacy` only as a temporary compatibility mode.
|
||||
|
||||
---
|
||||
|
||||
## 8) Operational profiles
|
||||
|
||||
### Profile A: single-GPU workstation
|
||||
|
||||
- Keep `STEMENGINE_GPU_VRAM_RESERVE_MB` non-zero (for example 2048 or higher).
|
||||
- Avoid launching multiple heavy jobs at once.
|
||||
- Prefer balanced defaults (`standard` presets) first.
|
||||
|
||||
### Profile B: dedicated server
|
||||
|
||||
- Use tighter CORS.
|
||||
- Enable only required sidecars.
|
||||
- Keep model/cache paths on high-throughput persistent storage.
|
||||
|
||||
### Profile C: CPU fallback
|
||||
|
||||
- Set `CPU_SEPARATION=true`.
|
||||
- Expect much slower separation/transcription.
|
||||
- Consider limiting upload sizes and queue depth.
|
||||
|
||||
---
|
||||
|
||||
## 9) Validation checklist after config changes
|
||||
|
||||
1. Restart services.
|
||||
2. Verify `/api/system/health` and `/api/system/readiness`.
|
||||
3. Verify model list and availability endpoints:
|
||||
- `/api/separate/models`
|
||||
- `/api/transcribe/modes`
|
||||
- `/api/lyric-transcribe/models`
|
||||
4. Run one end-to-end smoke job.
|
||||
|
||||
---
|
||||
|
||||
## 10) Common misconfigurations
|
||||
|
||||
- Missing/unstable `SECRET_KEY` causing settings/cipher issues.
|
||||
- Wrong sidecar URL or disabled profile while feature is enabled in UI.
|
||||
- Non-persistent model directories causing repeated downloads.
|
||||
- Overly broad or missing CORS causing UI/API connectivity failures.
|
||||
- Insufficient VRAM reserve on shared desktop systems.
|
||||
|
||||
Keep configuration changes small and verify incrementally.
|
||||
@@ -0,0 +1,100 @@
|
||||
# StemEngine Developer Guide
|
||||
|
||||
## 1) Architecture overview
|
||||
|
||||
StemEngine is a full-stack system:
|
||||
|
||||
- **Frontend**: React + TypeScript + Vite
|
||||
- **Backend**: FastAPI + Python services
|
||||
- **Sidecars**: optional model-specific services (for example Parakeet, Omnizart)
|
||||
- **Persistence**: project directories, job output directories, DB-backed metadata
|
||||
|
||||
High-level flow:
|
||||
|
||||
1. UI submits jobs to API.
|
||||
2. Backend routes requests to service layers.
|
||||
3. Job manager tracks async progress and status.
|
||||
4. Outputs are written to configured project/output paths.
|
||||
|
||||
## 2) Repository map
|
||||
|
||||
- `backend/` API, job orchestration, services, tests
|
||||
- `frontend/` React UI
|
||||
- `docker/` compose and deployment assets
|
||||
- `lyrics-sync/` alignment utilities
|
||||
- `parakeet-sidecar/` Parakeet integration runtime
|
||||
|
||||
## 3) Backend API surface (router prefixes)
|
||||
|
||||
Key API groups:
|
||||
|
||||
- `/api/separate`
|
||||
- `/api/transcribe`
|
||||
- `/api/lyric-transcribe`
|
||||
- `/api/lrc`
|
||||
- `/api/projects`
|
||||
- `/api/pipeline`
|
||||
- `/api/jobs`
|
||||
- `/api/models`
|
||||
- `/api/system`
|
||||
- `/api/settings`
|
||||
- `/api/subsonic`
|
||||
- `/api/abc-convert`
|
||||
|
||||
## 4) Local development workflows
|
||||
|
||||
### Backend
|
||||
|
||||
- Build image: `make build`
|
||||
- Run tests: `make test`
|
||||
- Smoke tests: `make test-smoke`
|
||||
- Migrations: `make alembic-upgrade`
|
||||
|
||||
### Frontend
|
||||
|
||||
From `frontend/`:
|
||||
|
||||
- Install: `npm install`
|
||||
- Dev server: `npm run dev`
|
||||
- Type/lint/build check: `npm run check`
|
||||
|
||||
## 5) Coding and change strategy
|
||||
|
||||
When adding a feature:
|
||||
|
||||
1. Define API contract changes first.
|
||||
2. Add/update service layer behavior.
|
||||
3. Extend router validations.
|
||||
4. Update frontend types + UI behavior.
|
||||
5. Add or update tests.
|
||||
6. Update user/developer docs in `docs/`.
|
||||
|
||||
## 6) Operational notes
|
||||
|
||||
- GPU safety limits are centrally important for stability.
|
||||
- Whisper/WhisperX/sidecar paths can have different feature support.
|
||||
- Lyric line segmentation now uses `karaoke-lyrics-processor` by default in `auto` mode, with a legacy fallback path controlled by `STEMENGINE_LYRIC_SEGMENTER`.
|
||||
- Keep long-running tasks cancellable and progress-reporting through the job manager.
|
||||
|
||||
## 7) Documentation rules for contributors
|
||||
|
||||
- Every user-facing feature change must update [USER_GUIDE.md](./USER_GUIDE.md).
|
||||
- Every architecture/API/process change must update this guide.
|
||||
- Large implementation work should include a dated plan or ADR-style note.
|
||||
- Every PR must complete documentation-impact checks via [../.github/PULL_REQUEST_TEMPLATE.md](../.github/PULL_REQUEST_TEMPLATE.md).
|
||||
|
||||
## 8) Documentation governance (Phase 3)
|
||||
|
||||
- Governance process: [DOC_GOVERNANCE.md](./DOC_GOVERNANCE.md)
|
||||
- Release verification: [RELEASE_CHECKLIST.md](./RELEASE_CHECKLIST.md)
|
||||
- Feature template: [templates/FEATURE_DOC_TEMPLATE.md](./templates/FEATURE_DOC_TEMPLATE.md)
|
||||
- Migration template: [templates/MIGRATION_NOTE_TEMPLATE.md](./templates/MIGRATION_NOTE_TEMPLATE.md)
|
||||
|
||||
## 9) Suggested next technical docs
|
||||
|
||||
- API endpoint reference with request/response examples (**implemented**: [API_REFERENCE.md](./API_REFERENCE.md))
|
||||
- Troubleshooting matrix for GPU/runtime/sidecar failures (**implemented**: [TROUBLESHOOTING_MATRIX.md](./TROUBLESHOOTING_MATRIX.md))
|
||||
- Configuration usage guide (**implemented**: [CONFIGURATION_REFERENCE.md](./CONFIGURATION_REFERENCE.md))
|
||||
- Docs governance + templates (**implemented**: [DOC_GOVERNANCE.md](./DOC_GOVERNANCE.md))
|
||||
- Data model and migration policy
|
||||
- Job lifecycle/state machine documentation
|
||||
@@ -0,0 +1,115 @@
|
||||
# StemEngine Documentation Plan
|
||||
|
||||
## Purpose
|
||||
|
||||
Create and maintain complete, audience-specific documentation for:
|
||||
|
||||
1. **Users** (how to operate StemEngine successfully)
|
||||
2. **Developers** (how to build, maintain, and extend StemEngine)
|
||||
|
||||
This plan defines scope, phases, owners, and acceptance criteria.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
- Reduce onboarding time for both users and contributors.
|
||||
- Make major workflows discoverable and repeatable.
|
||||
- Keep documentation synchronized with actual behavior.
|
||||
- Introduce a lightweight maintenance process so docs do not drift.
|
||||
|
||||
---
|
||||
|
||||
## Audience and scope
|
||||
|
||||
### User documentation scope
|
||||
|
||||
- Installation and first run
|
||||
- Common task workflows (separation, transcription, lyric sync)
|
||||
- File outputs and where to find them
|
||||
- Basic troubleshooting and known limits
|
||||
|
||||
### Developer documentation scope
|
||||
|
||||
- Architecture overview and component boundaries
|
||||
- API/route map and service responsibilities
|
||||
- Development setup and test workflow
|
||||
- Contribution and documentation update rules
|
||||
|
||||
---
|
||||
|
||||
## Delivery phases
|
||||
|
||||
## Phase 1 — Baseline docs (completed)
|
||||
|
||||
Deliverables:
|
||||
|
||||
- `docs/README.md` (docs index)
|
||||
- `docs/USER_GUIDE.md` (user baseline)
|
||||
- `docs/DEVELOPER_GUIDE.md` (developer baseline)
|
||||
- `docs/DOCUMENTATION_PLAN.md` (this plan)
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Both audiences have a dedicated entry point.
|
||||
- Core workflows and architecture are documented at a high level.
|
||||
- Documentation ownership expectations are defined.
|
||||
|
||||
## Phase 2 — Expand depth (completed)
|
||||
|
||||
Deliverables:
|
||||
|
||||
- API reference doc with endpoint groups and payload examples.
|
||||
- Troubleshooting matrix by subsystem (GPU, model download, sidecars, alignment).
|
||||
- Configuration reference expanded from env-var tables to usage guidance.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- New contributors can run tests and add a feature without tribal knowledge.
|
||||
- Users can self-serve common failures from docs.
|
||||
|
||||
## Phase 3 — Quality and governance (completed)
|
||||
|
||||
Deliverables:
|
||||
|
||||
- PR checklist item: docs impact reviewed.
|
||||
- Doc freshness review cadence (for example monthly or per release).
|
||||
- Standard template for feature docs and migration notes.
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- Documentation updates become part of normal development flow.
|
||||
- Drift between implementation and docs is reduced.
|
||||
|
||||
---
|
||||
|
||||
## Work breakdown and owners
|
||||
|
||||
- **Product/maintainers**: approve user workflow priority and terminology.
|
||||
- **Backend contributors**: maintain API/service/ops sections.
|
||||
- **Frontend contributors**: maintain UI flow and screenshots (if added).
|
||||
- **Release owner**: verify doc updates before release tagging.
|
||||
|
||||
---
|
||||
|
||||
## Tracking checklist
|
||||
|
||||
- [x] Create docs index
|
||||
- [x] Create user guide baseline
|
||||
- [x] Create developer guide baseline
|
||||
- [x] Document multi-phase plan
|
||||
- [x] Add API reference with examples
|
||||
- [x] Add troubleshooting matrix
|
||||
- [x] Add configuration reference with usage guidance
|
||||
- [x] Add PR/release doc update checklist
|
||||
- [x] Define review cadence and owner rotation
|
||||
|
||||
---
|
||||
|
||||
## Definition of done
|
||||
|
||||
Documentation initiative is considered operational when:
|
||||
|
||||
- Baseline docs exist for both audiences (done).
|
||||
- Expansion docs are published and linked from docs index.
|
||||
- Team workflow requires doc updates for meaningful behavior changes.
|
||||
@@ -0,0 +1,76 @@
|
||||
# Documentation Governance (Phase 3)
|
||||
|
||||
This document defines how documentation stays fresh as StemEngine evolves.
|
||||
|
||||
## 1) Policy
|
||||
|
||||
- Documentation updates are required for meaningful behavior changes.
|
||||
- Documentation review is part of pull request completion.
|
||||
- Release readiness includes documentation verification.
|
||||
|
||||
## 2) Cadence
|
||||
|
||||
Two cadences are enforced:
|
||||
|
||||
1. **Per PR**
|
||||
- Author must declare documentation impact in PR template.
|
||||
- Reviewer must confirm docs are updated or explicitly not required.
|
||||
|
||||
2. **Per release / monthly review**
|
||||
- Run a docs freshness pass at each release.
|
||||
- If releases are infrequent, run at least once per month.
|
||||
|
||||
## 3) Owner rotation
|
||||
|
||||
Use a rotating documentation steward role.
|
||||
|
||||
- **Rotation period**: one release cycle (or one month)
|
||||
- **Owner**: assigned in planning/release issue
|
||||
- **Backup owner**: one additional maintainer
|
||||
|
||||
Steward responsibilities:
|
||||
|
||||
- Validate docs are linked and discoverable.
|
||||
- Verify API docs reflect current endpoints.
|
||||
- Verify troubleshooting and config references are still accurate.
|
||||
- Track stale or missing docs as follow-up work.
|
||||
|
||||
## 4) PR requirements
|
||||
|
||||
Every PR must answer documentation impact via template:
|
||||
|
||||
- docs updated, or
|
||||
- docs explicitly not needed with rationale.
|
||||
|
||||
Common triggers requiring docs update:
|
||||
|
||||
- New/changed endpoint or request shape
|
||||
- New/changed environment variable
|
||||
- Feature behavior changes visible to users
|
||||
- New sidecar/runtime dependency
|
||||
- New operational or troubleshooting behavior
|
||||
|
||||
## 5) Release checklist integration
|
||||
|
||||
Before tagging a release:
|
||||
|
||||
1. Confirm documentation-impact PR checks are present in merged changes.
|
||||
2. Update user/developer guides for behavior changes.
|
||||
3. Update API, troubleshooting, and configuration references as needed.
|
||||
4. Add migration note when compatibility or operations changed.
|
||||
|
||||
## 6) Escalation for stale docs
|
||||
|
||||
If docs drift is found:
|
||||
|
||||
- Open a documentation debt issue in the same milestone.
|
||||
- Label as `docs-debt` and include impacted files.
|
||||
- Assign to current documentation steward.
|
||||
|
||||
## 7) Definition of healthy docs process
|
||||
|
||||
The process is healthy when:
|
||||
|
||||
- PRs consistently declare documentation impact.
|
||||
- Release checks catch stale docs before release.
|
||||
- New contributors can ship changes without tribal knowledge.
|
||||
@@ -0,0 +1,29 @@
|
||||
# StemEngine Documentation
|
||||
|
||||
This directory contains documentation for two audiences:
|
||||
|
||||
- **Users** who run StemEngine to separate stems, transcribe audio, and sync lyrics.
|
||||
- **Developers** who build, debug, and extend StemEngine.
|
||||
|
||||
## Start here
|
||||
|
||||
- [User Guide](./USER_GUIDE.md)
|
||||
- [Developer Guide](./DEVELOPER_GUIDE.md)
|
||||
- [Documentation Plan](./DOCUMENTATION_PLAN.md)
|
||||
|
||||
## Phase 2 references
|
||||
|
||||
- [API Reference](./API_REFERENCE.md)
|
||||
- [Troubleshooting Matrix](./TROUBLESHOOTING_MATRIX.md)
|
||||
- [Configuration Reference](./CONFIGURATION_REFERENCE.md)
|
||||
|
||||
## Phase 3 governance
|
||||
|
||||
- [Documentation Governance](./DOC_GOVERNANCE.md)
|
||||
- [Release Checklist](./RELEASE_CHECKLIST.md)
|
||||
- [Feature Doc Template](./templates/FEATURE_DOC_TEMPLATE.md)
|
||||
- [Migration Note Template](./templates/MIGRATION_NOTE_TEMPLATE.md)
|
||||
|
||||
## Documentation status
|
||||
|
||||
These docs are a structured baseline. They should be treated as living docs and updated with each feature change.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Release Checklist (Documentation + Quality)
|
||||
|
||||
Use this checklist before creating a release tag.
|
||||
|
||||
## Core checks
|
||||
|
||||
- [ ] Backend checks/tests passed
|
||||
- [ ] Frontend checks passed
|
||||
- [ ] Critical workflows smoke-tested
|
||||
|
||||
## Documentation checks
|
||||
|
||||
- [ ] User-facing changes reflected in docs/USER_GUIDE.md
|
||||
- [ ] Developer-facing changes reflected in docs/DEVELOPER_GUIDE.md
|
||||
- [ ] API changes reflected in docs/API_REFERENCE.md
|
||||
- [ ] Ops/runtime changes reflected in docs/TROUBLESHOOTING_MATRIX.md
|
||||
- [ ] Env/config changes reflected in docs/CONFIGURATION_REFERENCE.md
|
||||
- [ ] Governance notes current in docs/DOC_GOVERNANCE.md
|
||||
|
||||
## Migration / compatibility
|
||||
|
||||
- [ ] No migration required
|
||||
- [ ] Migration note added from docs/templates/MIGRATION_NOTE_TEMPLATE.md
|
||||
- [ ] Backward compatibility statement included in release notes
|
||||
|
||||
## Sign-off
|
||||
|
||||
- [ ] Documentation steward reviewed
|
||||
- [ ] Release owner approved
|
||||
@@ -0,0 +1,87 @@
|
||||
# StemEngine Troubleshooting Matrix (Phase 2)
|
||||
|
||||
Use this matrix to triage by subsystem. Each row gives symptoms, likely causes, checks, and corrective actions.
|
||||
|
||||
## Quick triage order
|
||||
|
||||
1. Check container health (`/api/system/health`, `/api/system/readiness`).
|
||||
2. Check job payload (`/api/jobs/{job_id}`) for exact failure stage.
|
||||
3. Check subsystem-specific row below.
|
||||
|
||||
---
|
||||
|
||||
## Matrix
|
||||
|
||||
| Subsystem | Symptom | Likely cause | What to check | Corrective action |
|
||||
|---|---|---|---|---|
|
||||
| GPU runtime | CUDA errors, OOM, jobs fail after earlier GPU-heavy tasks | VRAM pressure or fragmented allocator state | `/api/system/gpu-info`, recent job history | Use smaller model/preset, increase VRAM reserve, run `/api/system/gpu-reset` (admin), retry |
|
||||
| GPU scheduling (WSL2) | Desktop/UI stutter while jobs run | GPU headroom too low | Reserve env vars and current workload | Increase `STEMENGINE_GPU_VRAM_RESERVE_MB`, avoid concurrent heavy jobs |
|
||||
| Model downloads | Model stuck in `error` or not downloaded | Network issue, invalid/placeholder repo, permissions | `/api/models/progress`, `/api/models/list` | Retry download, verify network and model metadata, ensure model dirs writable |
|
||||
| Separation | `Unknown model` / poor output | Invalid model id or unsuitable model family | `/api/separate/models` and request payload | Use valid `model`, choose family matched to source material |
|
||||
| Transcribe (MIDI) | `Omnizart not available` | Sidecar disabled/unreachable | `OMNIZART_ENABLED`, sidecar URL/health, mode selected | Enable sidecar or switch to basic-pitch mode |
|
||||
| Lyric transcription (WhisperX) | Whisper/WhisperX unavailable | Package not installed in runtime image | `/api/lyric-transcribe/models` availability flags | Install/enable runtime dependencies, rebuild image |
|
||||
| Lyric line segmentation | Odd line breaks or unexpected fallback style | `karaoke-lyrics-processor` unavailable or remap fallback triggered | backend logs, `STEMENGINE_LYRIC_SEGMENTER`, `STEMENGINE_LYRIC_MAX_LINE_LENGTH` | Use `auto` mode, verify dependency in runtime image, tune max line length, compare against `legacy` |
|
||||
| Lyric transcription (Parakeet) | `Parakeet runtime unavailable` | Sidecar disabled/unreachable | `PARAKEET_ENABLED`, sidecar URL, `/health` | Enable profile, correct URL/path, verify network reachability |
|
||||
| LRC alignment | No aligned lines / low quality timing | Wrong source lyrics, weak vocals stem, language mismatch | Source mode, stem quality, selected language | Use vocals stem, provide clean lyrics text/file, re-run with corrected source |
|
||||
| Project operations | `Project not found` / missing stems | Project ID mismatch, files moved manually | `/api/projects`, project detail, disk paths | Rescan stems endpoint, restore/move files back, avoid out-of-band edits |
|
||||
| Jobs/downloads | Download endpoint returns `Job not complete` | Job still running/failed | `/api/jobs/{job_id}` status | Poll until done, then download exact `files[].name` |
|
||||
| Settings/auth | Admin endpoints denied | Missing/incorrect admin token | server env + request headers | Provide valid admin token for protected routes |
|
||||
| Subsonic import | Search/import fails | Bad credentials/server URL, upstream issue | `/api/subsonic/config`, upstream server reachability | Re-save config with valid credentials, check external server health |
|
||||
|
||||
---
|
||||
|
||||
## Focused playbooks
|
||||
|
||||
### A) GPU OOM recovery playbook
|
||||
|
||||
1. Stop launching new heavy jobs.
|
||||
2. Check GPU state via `/api/system/gpu-info`.
|
||||
3. Call `/api/system/gpu-reset` (admin).
|
||||
4. Re-run with lower-cost options:
|
||||
- smaller separation model
|
||||
- lower `sep_power_preset`
|
||||
- reduced concurrent workload
|
||||
5. If persistent, restart container.
|
||||
|
||||
### B) Parakeet sidecar recovery playbook
|
||||
|
||||
1. Verify `PARAKEET_ENABLED=true` in backend.
|
||||
2. Verify `PARAKEET_SIDECAR_URL` points to the reachable service name.
|
||||
3. Check sidecar `GET /health` returns 200 from backend network context.
|
||||
4. Re-test `/api/lyric-transcribe/models` availability fields.
|
||||
|
||||
### C) Model download recovery playbook
|
||||
|
||||
1. Check `/api/models/progress/{model_id}`.
|
||||
2. If paused/cancelled unexpectedly, resume or restart download.
|
||||
3. Validate writable model directory mounts.
|
||||
4. If corrupted partial files are suspected, cancel and restart.
|
||||
|
||||
### D) Alignment quality recovery playbook
|
||||
|
||||
1. Use a clear vocals-focused stem.
|
||||
2. Prefer accurate, line-broken lyrics input when forcing alignment.
|
||||
3. Match language hint to track language.
|
||||
4. Re-run with engine/options tuned for material.
|
||||
|
||||
### E) Lyric segmentation quality playbook
|
||||
|
||||
1. Confirm runtime has `karaoke-lyrics-processor` installed.
|
||||
2. Verify `STEMENGINE_LYRIC_SEGMENTER` is set as intended (`auto` recommended).
|
||||
3. Tune `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (typical karaoke range: 32–42).
|
||||
4. Re-test with the same track and compare `auto` vs `legacy` output for regressions.
|
||||
|
||||
---
|
||||
|
||||
## Signals to capture in bug reports
|
||||
|
||||
Include:
|
||||
|
||||
- Endpoint called and payload summary
|
||||
- `job_id`
|
||||
- Full job payload (`status`, `progress`, `error`)
|
||||
- `/api/system/readiness` result
|
||||
- `/api/system/gpu-info` result (for GPU issues)
|
||||
- Relevant sidecar enablement/env summary
|
||||
|
||||
This keeps issue triage reproducible and fast.
|
||||
@@ -0,0 +1,85 @@
|
||||
# StemEngine User Guide
|
||||
|
||||
## 1) What StemEngine does
|
||||
|
||||
StemEngine helps you:
|
||||
|
||||
- Separate songs into stems (vocals, drums, bass, etc.)
|
||||
- Transcribe audio to MIDI
|
||||
- Generate and align lyrics (LRC/ELRC)
|
||||
- Create karaoke-ready project outputs
|
||||
|
||||
## 2) Requirements
|
||||
|
||||
- Docker + Docker Compose
|
||||
- NVIDIA GPU + NVIDIA Container Toolkit for best performance
|
||||
- Browser access to the StemEngine UI
|
||||
|
||||
## 3) Quick start
|
||||
|
||||
1. Copy `.env.example` to `.env`.
|
||||
2. Set required variables (`SEFOLDER`, `ENGINEFOLDER`, `PROJECTS`, `SECRET_KEY`).
|
||||
3. Start StemEngine:
|
||||
|
||||
- `docker compose up -d --build`
|
||||
|
||||
4. Open `http://localhost:8000`.
|
||||
|
||||
## 4) Core workflows
|
||||
|
||||
### A. Separate a track into stems
|
||||
|
||||
1. Go to the separation workflow in the UI.
|
||||
2. Upload audio.
|
||||
3. Choose a model family and model.
|
||||
4. Start job and monitor progress.
|
||||
5. Download outputs or open the generated project.
|
||||
|
||||
### B. Transcribe to MIDI
|
||||
|
||||
1. Select a project/audio source.
|
||||
2. Run MIDI transcription (polyphonic or melody mode).
|
||||
3. Export MIDI and optionally convert to ABC notation.
|
||||
|
||||
### C. Create synced lyrics
|
||||
|
||||
1. Open Lyric Transcription.
|
||||
2. Choose engine:
|
||||
- `WhisperX` for richer options (alignment/diarization paths)
|
||||
- `Parakeet` for sidecar ASR flow
|
||||
3. Choose output format (`lrc`, `elrc`, `txt`, `json`, `srt`, `vtt`).
|
||||
4. Run job and save outputs.
|
||||
|
||||
### D. Align existing lyrics
|
||||
|
||||
1. Open LRC/Lyrics alignment workflow.
|
||||
2. Paste/upload existing lyrics.
|
||||
3. Run alignment to generate timed output.
|
||||
|
||||
## 5) Output locations
|
||||
|
||||
Generated artifacts are stored in the configured project/output directories. Typical output includes:
|
||||
|
||||
- Source audio copy
|
||||
- Separated stems (`Stems/`)
|
||||
- Optional mixes (`Instrumental/`, `Static/`, `Dynamic/`)
|
||||
- Lyrics artifacts (`.lrc`, `.elrc`, `.txt`, `.json`, `.srt`, `.vtt`)
|
||||
|
||||
## 6) Troubleshooting
|
||||
|
||||
- **GPU memory issues**: reduce model size, lower workload, or increase reserve tuning variables.
|
||||
- **Slow runs**: verify GPU container runtime is enabled.
|
||||
- **Parakeet unavailable**: confirm sidecar URL and health endpoint configuration.
|
||||
- **No/poor lyric alignment**: prefer vocals stem input and verify language settings.
|
||||
|
||||
## 7) Best practices
|
||||
|
||||
- Use vocals-focused stems for lyric transcription quality.
|
||||
- Start with balanced models, then move to large models only when needed.
|
||||
- Keep project names and source metadata clean for easier downstream workflows.
|
||||
|
||||
## 8) Known limits
|
||||
|
||||
- Some advanced options are engine-specific.
|
||||
- Sidecar-dependent features require correctly configured auxiliary services.
|
||||
- CPU-only operation is supported but can be significantly slower.
|
||||
+70
@@ -0,0 +1,70 @@
|
||||
# Feature Documentation Template
|
||||
|
||||
Use this template for any feature that changes user workflows, API behavior, or operations.
|
||||
|
||||
## 1) Feature summary
|
||||
|
||||
- **Feature name**:
|
||||
- **Status**: draft | implemented | released
|
||||
- **Date**:
|
||||
- **Owner**:
|
||||
|
||||
## 2) Problem and goal
|
||||
|
||||
- What problem does this feature solve?
|
||||
- What is the intended user/developer outcome?
|
||||
|
||||
## 3) Scope
|
||||
|
||||
### In scope
|
||||
-
|
||||
|
||||
### Out of scope
|
||||
-
|
||||
|
||||
## 4) User workflow changes
|
||||
|
||||
- Before:
|
||||
- After:
|
||||
- UI/API touchpoints:
|
||||
|
||||
## 5) Technical changes
|
||||
|
||||
- Backend:
|
||||
- Frontend:
|
||||
- Sidecars/runtime:
|
||||
- Data/storage:
|
||||
|
||||
## 6) Configuration impact
|
||||
|
||||
- New variables:
|
||||
- Changed variables:
|
||||
- Defaults and migration behavior:
|
||||
|
||||
## 7) API impact
|
||||
|
||||
- New endpoints:
|
||||
- Changed endpoints:
|
||||
- Deprecated endpoints:
|
||||
- Request/response examples:
|
||||
|
||||
## 8) Failure modes and troubleshooting
|
||||
|
||||
- Likely failures:
|
||||
- Detection signals:
|
||||
- Recovery steps:
|
||||
|
||||
## 9) Validation
|
||||
|
||||
- Automated tests:
|
||||
- Manual verification:
|
||||
- Known limits:
|
||||
|
||||
## 10) Documentation updates
|
||||
|
||||
- [ ] docs/USER_GUIDE.md
|
||||
- [ ] docs/DEVELOPER_GUIDE.md
|
||||
- [ ] docs/API_REFERENCE.md
|
||||
- [ ] docs/TROUBLESHOOTING_MATRIX.md
|
||||
- [ ] docs/CONFIGURATION_REFERENCE.md
|
||||
- [ ] Release notes updated
|
||||
+49
@@ -0,0 +1,49 @@
|
||||
# Migration Note Template
|
||||
|
||||
Use this when a change affects compatibility, operations, configuration, or stored data.
|
||||
|
||||
## 1) Migration title
|
||||
|
||||
- **Title**:
|
||||
- **Date**:
|
||||
- **Owner**:
|
||||
- **Applies to versions**:
|
||||
|
||||
## 2) What changed
|
||||
|
||||
- Summary of behavior/API/config/data changes.
|
||||
|
||||
## 3) Who is impacted
|
||||
|
||||
- Users:
|
||||
- Operators:
|
||||
- Developers/integrators:
|
||||
|
||||
## 4) Required actions
|
||||
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
|
||||
## 5) Backward compatibility
|
||||
|
||||
- Compatible without action: yes/no
|
||||
- Deprecated behavior and timeline:
|
||||
|
||||
## 6) Rollback plan
|
||||
|
||||
- Preconditions:
|
||||
- Steps:
|
||||
- Data recovery notes:
|
||||
|
||||
## 7) Validation after migration
|
||||
|
||||
- Health/readiness checks:
|
||||
- Functional checks:
|
||||
- Observability checks:
|
||||
|
||||
## 8) References
|
||||
|
||||
- PR:
|
||||
- Related docs:
|
||||
- Incident/issues (if any):
|
||||
Reference in new issue
Block a user