Myriad improvements.

This commit is contained in:
translator committed 2026-03-18 17:55:21 +09:00
1 parent ed72ded98f
commit 9a02f0f321
19 files changed
+2208 -24

No files matched your search

+47
View File
@@ -0,0 +1,47 @@
## Summary
<!-- What changed and why? -->
## Change type
- [ ] Feature
- [ ] Bug fix
- [ ] Refactor
- [ ] Docs only
- [ ] Chore
## Validation
- [ ] Backend tests or targeted checks run
- [ ] Frontend checks (typecheck/lint/build) run when relevant
- [ ] Manual verification completed
## Documentation impact (required)
- [ ] I reviewed documentation impact for this PR.
- [ ] No documentation update required (internal-only change with no behavior/API/workflow impact).
- [ ] Documentation updated in this PR.
If documentation was updated, list files:
- [ ] docs/USER_GUIDE.md
- [ ] docs/DEVELOPER_GUIDE.md
- [ ] docs/API_REFERENCE.md
- [ ] docs/TROUBLESHOOTING_MATRIX.md
- [ ] docs/CONFIGURATION_REFERENCE.md
- [ ] docs/DOC_GOVERNANCE.md
- [ ] docs/templates/FEATURE_DOC_TEMPLATE.md
- [ ] docs/templates/MIGRATION_NOTE_TEMPLATE.md
- [ ] other: ______________________
## Release notes / migration
- [ ] No release note needed
- [ ] Release note entry added
- [ ] Migration note added (if required)
## Checklist
- [ ] Backward compatibility considered
- [ ] Error cases handled
- [ ] Security/permissions impact considered
+480
View File
@@ -3510,3 +3510,483 @@ Validation:
- frontend diagnostics: clean
- frontend typecheck: `EXIT:0`
---
### 2026-03-18 implementation (documentation program — Phases 1, 2, and 3)
Requested:
- Create documentation for users and developers.
- Implement Phase 2 (API reference, troubleshooting matrix, configuration guidance).
- Implement Phase 3 (governance and maintenance workflow).
- Record documentation changes in this running fixes log.
Scope:
- `docs/README.md`
- `docs/USER_GUIDE.md`
- `docs/DEVELOPER_GUIDE.md`
- `docs/DOCUMENTATION_PLAN.md`
- `docs/API_REFERENCE.md`
- `docs/TROUBLESHOOTING_MATRIX.md`
- `docs/CONFIGURATION_REFERENCE.md`
- `docs/DOC_GOVERNANCE.md`
- `docs/RELEASE_CHECKLIST.md`
- `docs/templates/FEATURE_DOC_TEMPLATE.md`
- `docs/templates/MIGRATION_NOTE_TEMPLATE.md`
- `.github/PULL_REQUEST_TEMPLATE.md`
- `README.md`
#### Implemented
1) Documentation baseline (Phase 1)
- Added dedicated docs entrypoint and split audience docs:
- user guide
- developer guide
- delivery plan
2) Documentation depth (Phase 2)
- Added API reference by endpoint groups with practical request examples.
- Added troubleshooting matrix for GPU/runtime/model-download/sidecar/alignment paths.
- Added configuration reference with operational guidance beyond raw env-var listing.
3) Documentation governance (Phase 3)
- Added PR template with mandatory documentation-impact checklist.
- Added governance policy with:
- per-PR doc review requirements
- per-release/monthly freshness cadence
- owner rotation/steward responsibilities
- Added release checklist that includes explicit docs sign-off.
- Added reusable templates for:
- feature documentation
- migration notes
4) Documentation wiring
- Updated docs index and root README links so all new docs are discoverable.
- Updated developer guide to point to governance/process artifacts.
- Updated documentation plan status to mark Phase 2 and Phase 3 complete.
Outcome:
- Documentation initiative now has full lifecycle coverage:
- **Phase 1** baseline audience docs
- **Phase 2** operational/technical depth docs
- **Phase 3** governance and update process to prevent drift
---
### 2026-03-18 note — CI enforcement of docs-impact PR fields (on hold)
Discussed: enforcing the Documentation Impact section of `.github/PULL_REQUEST_TEMPLATE.md` via a
GitHub Actions workflow that fails PRs where the docs-impact fields are left blank/skipped.
Decision: **hold for now.**
When ready, the implementation would be a small GitHub Actions job that parses the PR body and
requires either a "files updated" checkbox or an explicit "No documentation changes needed"
declaration. Without it the gate blocks merging. Currently the template is on the honor system.
---
### 2026-03-18 plan — replace lyric line segmentation with `karaoke-lyrics-processor`
Requested:
- Plan migration from current lyric line segmentation heuristics to
`nomadkaraoke/karaoke-lyrics-processor`.
- Document the plan.
Implemented:
- Added implementation plan document:
- `20260318_Lyric_Line_Segmentation_Replacement_Plan.md`
- Plan covers:
- current baseline segmentation functions in `lyric_transcriber_service.py`
- adapter-first integration strategy
- feature-flagged rollout (`legacy` / `karaoke_processor` / `auto`)
- validation matrix (unit/integration/regression corpus)
- risks and mitigations (timestamp remap drift, dependency pinning, fallback policy)
- expected implementation file touch list + docs updates
Status:
- Planning/documentation complete.
- Code migration intentionally not started in this step.
---
### 2026-03-18 implementation — lyric segmentation replacement (`karaoke-lyrics-processor`)
Requested:
- Implement the planned lyric line segmentation replacement.
- Document the implementation.
Scope:
- `backend/config.py`
- `backend/requirements.txt`
- `backend/services/lyric_transcriber_service.py`
- `README.md`
- `docs/CONFIGURATION_REFERENCE.md`
- `docs/TROUBLESHOOTING_MATRIX.md`
- `docs/DEVELOPER_GUIDE.md`
- `20260318_Lyric_Line_Segmentation_Replacement_Plan.md`
Implemented:
- Added backend dependency: `karaoke-lyrics-processor>=1.0.0`.
- Added env-configured segmentation controls:
- `STEMENGINE_LYRIC_SEGMENTER` (`auto` / `karaoke_processor` / `legacy`)
- `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (default `36`)
- Replaced plain-text lyric segmentation path with karaoke processor adapter + safe fallback to legacy heuristic splitter.
- Replaced WhisperX long-segment split path with karaoke processor-driven splitting and conservative word-token remap back onto timestamped words.
- Added ambiguity/runtime fallback behavior to legacy splitting to avoid job failures when remap or dependency edge cases occur.
- Updated operational docs (README + config + troubleshooting + developer guide) with new controls and fallback behavior.
- Updated the 2026-03-18 plan doc to mark implementation complete.
Outcome:
- Lyric line segmentation now defaults to karaoke-oriented splitting while preserving backward compatibility and operational safety via explicit fallback modes.
Validation:
- Focused segmentation smoke pass executed (2026-03-18) comparing `auto` vs `legacy` modes using representative long-line samples and timestamped word segments.
- Command context: local Python runtime with writable temp media paths (`/tmp/stemengine_smoke`) and direct helper invocation from `lyric_transcriber_service.py`.
- Results:
- `auto`:
- active mode: `auto`
- plain-text line count: `4`
- WhisperX segment split count: `3`
- word timestamp order monotonic after remap: `true`
- `legacy`:
- active mode: `legacy`
- plain-text line count: `3`
- WhisperX segment split count: `2`
- word timestamp order monotonic after split: `true`
- Interpretation: new segmentation path is active and produces tighter karaoke-style line wrapping while preserving timestamp monotonicity in split output.
---
### 2026-03-18 plan — karaoke player platform (StemEngine ecosystem)
Requested:
- Process and plan a karaoke player with:
1) source browsing/import (Subsonic, local folder, YouTube, SoulSeek)
2) StemEngine preset processing + auto-play
3) existing Two.js renderer playback with stem mute controls, `projectM` visualizer option, and video playback coupling
4) in-app lyrics/style editor using same Two.js renderer
5) companion mobile PWA for queue/room operations
- Document the plan and hold implementation.
Implemented:
- Added architecture and delivery plan document:
- `20260318_Karaoke_Player_Platform_Plan.md`
- Plan includes:
- scope and explicit out-of-scope boundaries
- target multi-component architecture (desktop player, orchestration API, StemEngine integration, mobile PWA)
- capability breakdown for each requested feature
- proposed domain model and API additions
- security/roles policy outline for rooms and queue control
- phased rollout (A: foundation, B: editor/visuals, C: rooms/PWA, D: source expansion/hardening)
- risks and mitigations
Status:
- Planning/documentation complete.
- **On hold** (no implementation started in this step).
---
### 2026-03-18 update — security considerations added to karaoke player platform plan
Requested:
- Add security measures required for the planned karaoke player platform.
- Update the plan and document the considerations.
Implemented:
- Expanded `20260318_Karaoke_Player_Platform_Plan.md` security coverage to include:
- identity/session hardening (token rotation, revocation, MFA for elevated roles)
- RBAC + server-side room policy enforcement and moderation auditability
- API/transport controls (TLS, strict CORS, CSRF, endpoint rate-limits, service auth)
- ingestion/source hardening for Subsonic/YouTube/SoulSeek/local adapters
- path traversal and filesystem isolation requirements
- secrets management and encryption-at-rest requirements
- XSS/CSP/schema-validation controls for lyrics/style editor and playback UI
- abuse prevention quotas and destructive-action safeguards
- audit logging, observability, and incident playbook requirements
- privacy/retention/legal baseline considerations
- Added security-specific pre-implementation gates section:
- threat model approval
- authN/authZ design review
- secret management decision
- ingestion sandbox design
- security testing plan
- audit schema/retention approval
- incident runbook drafting
Status:
- Planning/documentation update complete.
- **On hold** (implementation not started).
---
### 2026-03-18 implementation — PlayaK Phase A scaffold kickoff
Requested:
- Begin PlayaK implementation work and document progress.
Implemented:
- Bootstrapped PlayaK runtime/service scaffold under `/home/translator/docker/appdata/playak`:
- `docker-compose.yml`
- `.env.example`
- `.gitignore`
- `backend/Dockerfile`
- `backend/requirements.txt`
- Added initial FastAPI control-plane app:
- `backend/app/main.py`
- `backend/app/models.py`
- `backend/app/store.py`
- `backend/app/security.py`
- Initial API endpoints scaffolded:
- `GET /api/playak/health`
- `GET /api/playak/roles/matrix`
- room/queue CRUD skeleton endpoints for Phase A orchestration prototyping
- Added/updated documentation:
- `/home/translator/docker/appdata/playak/README.md` (quick start + scaffold inventory)
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Karaoke_Player_Platform_Plan.md` status moved from hold to active scaffold phase
Status:
- Phase A scaffold started.
- Persistence/auth/provider adapters pending next pass.
---
### 2026-03-18 implementation — PlayaK workspace bootstrap + plan relocation
Requested:
- Implement and document a new container/workspace named `PlayaK` under:
- `~/docker/appdata/playak`
- Move the karaoke player platform plan to PlayaK.
- Create a new markdown progress document in the root folder.
Implemented:
- Created PlayaK workspace path:
- `/home/translator/docker/appdata/playak`
- Added PlayaK root documentation:
- `/home/translator/docker/appdata/playak/README.md`
- Relocated planning document into PlayaK:
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Karaoke_Player_Platform_Plan.md`
- Removed old plan location from StemEngine root:
- `/home/translator/docker/appdata/stemengine/20260318_Karaoke_Player_Platform_Plan.md`
- Added root-level progress tracker:
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
Status:
- PlayaK workspace bootstrap complete.
- Planning artifacts moved and documented.
---
### 2026-03-18 update — explicit global role matrix added (admin/user/guest)
Requested:
- Plan and document a complete global role matrix including:
- `platform_admin` (system-wide)
- `authenticated_user` (regular signed-in user)
- `guest` (anonymous/limited)
Implemented:
- Updated `20260318_Karaoke_Player_Platform_Plan.md` to add:
- explicit global-role table with capability-by-capability permissions
- mapping between global roles and room roles (`owner`, `moderator`, `member`, `viewer`)
- policy notes for guest restrictions and admin scope
- deterministic permission-resolution order for queue/room actions
- server-side audit requirements for authorization decisions (`user_id`, role context, action, decision)
Status:
- Planning/documentation update complete.
- **On hold** (implementation not started).
---
### 2026-03-18 implementation — PlayaK Phase A persistence (SQLite)
Requested:
- Continue PlayaK implementation and document progress.
Implemented:
- Replaced in-memory room/queue state with SQLite-backed persistence in PlayaK:
- `/home/translator/docker/appdata/playak/backend/app/db.py`
- `/home/translator/docker/appdata/playak/backend/app/store.py`
- `/home/translator/docker/appdata/playak/backend/app/main.py` (startup `init_db` via lifespan)
- Added runtime DB configuration and persistent host mount:
- `/home/translator/docker/appdata/playak/.env.example` (`PLAYAK_DB_PATH`)
- `/home/translator/docker/appdata/playak/docker-compose.yml` (`PLAYAK_DB_PATH` + `./data:/data`)
- Updated implementation docs:
- `/home/translator/docker/appdata/playak/README.md`
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
Status:
- Phase A scaffold now includes persisted room/queue state.
- Next: auth/session + server-side role enforcement, then source adapter interfaces.
---
### 2026-03-18 implementation — PlayaK Phase A auth skeleton + role enforcement
Requested:
- Continue PlayaK implementation and document progress.
Implemented:
- Added auth/authorization scaffold to PlayaK API:
- header-based actor context: `X-Global-Role`, `X-User-Id`
- role/permission helpers in `/home/translator/docker/appdata/playak/backend/app/security.py`
- Added room membership persistence:
- SQLite `room_members` table in `/home/translator/docker/appdata/playak/backend/app/db.py`
- membership queries/mutations in `/home/translator/docker/appdata/playak/backend/app/store.py`
- Added role-aware room endpoints:
- `POST /api/playak/rooms/{room_id}/join`
- `GET /api/playak/rooms/{room_id}/state`
- Enforced queue mutation rules in `/home/translator/docker/appdata/playak/backend/app/main.py`:
- enqueue requires permission (`platform_admin` or room member role)
- non-admin enqueue identity must match `requested_by`
- queue removal allowed for admin, room owner/moderator, or member removing own item
Documentation updated:
- `/home/translator/docker/appdata/playak/README.md`
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
Status:
- Phase A now includes persistence + server-side role checks on room/queue control paths.
- Next: source-ingestion adapter interfaces and orchestration job submission wiring.
---
### 2026-03-18 implementation — PlayaK Phase A source adapters + orchestration submission wiring
Requested:
- Continue PlayaK implementation and document progress.
Implemented:
- Added source-adapter interface module in PlayaK:
- `/home/translator/docker/appdata/playak/backend/app/sources.py`
- adapters + validation rules for `subsonic`, `local`, `youtube`, `soulseek`
- Added media library persistence model:
- SQLite `media_items` table + index in `/home/translator/docker/appdata/playak/backend/app/db.py`
- media CRUD helpers in `/home/translator/docker/appdata/playak/backend/app/store.py`
- Added orchestration submission bridge:
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py`
- `stub` mode (default) issues deterministic local submit IDs
- optional `http` mode posts payloads to configured endpoint
- Extended queue model/state:
- added `processing_preset`, `orchestration_state`, `stemengine_job_id`
- DB migration guard added for existing queue tables (`ALTER TABLE` when columns missing)
- Added new API endpoints in `/home/translator/docker/appdata/playak/backend/app/main.py`:
- `GET /api/playak/library`
- `POST /api/playak/import`
- `POST /api/playak/rooms/{room_id}/queue/from-media`
- `POST /api/playak/rooms/{room_id}/queue/{item_id}/submit`
- Updated runtime configuration wiring:
- `/home/translator/docker/appdata/playak/.env.example`
- `/home/translator/docker/appdata/playak/docker-compose.yml`
- new vars: `PLAYAK_ORCHESTRATION_MODE`, `PLAYAK_ORCHESTRATION_HTTP_URL`, `PLAYAK_ORCHESTRATION_HTTP_TIMEOUT_SEC`
Documentation updated:
- `/home/translator/docker/appdata/playak/README.md`
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
Status:
- Phase A now includes source import scaffolding and explicit queue submission lifecycle wiring.
- Next: replace adapter validation stubs with provider-backed integrations and bind `http` orchestration to concrete StemEngine submit/poll routes.
---
### 2026-03-18 implementation — PlayaK local scan ingestion + queue status synchronization
Requested:
- Continue implementation and document progress.
Implemented:
- Added local provider-backed scan ingestion path:
- endpoint: `POST /api/playak/import/local-scan`
- scanning implemented in `/home/translator/docker/appdata/playak/backend/app/sources.py`
- allowlist enforcement via `PLAYAK_LOCAL_LIBRARY_ROOTS`
- recursive import for common audio extensions
- Added media deduplication in store layer by (`source`, `source_ref`) to avoid duplicate local scan inserts.
- Added orchestration status synchronization path:
- endpoint: `POST /api/playak/rooms/{room_id}/queue/{item_id}/sync-status`
- logic in `/home/translator/docker/appdata/playak/backend/app/orchestration.py`
- `stub` mode: marks submitted jobs as `ready`
- `http` mode: polls using `PLAYAK_ORCHESTRATION_HTTP_STATUS_URL_TEMPLATE`
- Added queue status update helper in `/home/translator/docker/appdata/playak/backend/app/store.py`.
- Updated runtime configuration wiring:
- `/home/translator/docker/appdata/playak/.env.example`
- `/home/translator/docker/appdata/playak/docker-compose.yml`
- new vars: `PLAYAK_ORCHESTRATION_HTTP_STATUS_URL_TEMPLATE`, `PLAYAK_LOCAL_LIBRARY_ROOTS`
Documentation updated:
- `/home/translator/docker/appdata/playak/README.md`
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
Status:
- Phase A now includes a real local scan ingestion workflow and queue status synchronization loop.
- Next: integrate concrete Subsonic adapter and bind HTTP orchestration to real StemEngine endpoints.
---
### 2026-03-18 implementation — PlayaK real StemEngine bridge + Subsonic provider integration
Requested:
- Go ahead with the next implementation step.
Implemented:
- Reworked PlayaK `http` orchestration mode to support real StemEngine routing when `PLAYAK_STEMENGINE_BASE_URL` is configured:
- local source → `POST /api/separate/from-path`
- Subsonic source → `POST /api/subsonic/import` then `POST /api/separate/from-path`
- job sync → `GET /api/jobs/{job_id}`
- Added env-driven preset mapping for StemEngine model/output/power configuration.
- Added concrete Subsonic provider bridge helpers in:
- `/home/translator/docker/appdata/playak/backend/app/stemengine_bridge.py`
- Added new provider endpoints in PlayaK:
- `GET /api/playak/providers/subsonic/search`
- `POST /api/playak/providers/subsonic/import`
- Updated PlayaK runtime environment config for StemEngine route/path overrides and preset defaults.
Documentation updated:
- `/home/translator/docker/appdata/playak/README.md`
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
Status:
- PlayaK now supports real StemEngine-backed submit/poll flows for local and Subsonic sources in `http` mode.
- Next: add targeted smoke tests and extend concrete bridge support to YouTube/SoulSeek flows.
Validation:
- Host-side live smoke against `http://localhost:8000`: passed
- Focused API regression suite (`backend/tests/test_bridge_api.py`): `2 tests, OK`
---
### 2026-03-18 validation — PlayaK localhost real-bridge smoke
Requested:
- Use `http://localhost:8000` as the base URL for testing and document the results.
Implemented:
- Added host-side smoke assets:
- `/home/translator/docker/appdata/playak/backend/scripts/smoke_real_bridge.py`
- `/home/translator/docker/appdata/playak/.env.host-test.example`
- Ran live smoke against local StemEngine at `http://localhost:8000`.
- Observed successful end-to-end control-plane behavior:
- PlayaK registered a local media item using a StemEngine runtime-visible import path
- queue submit returned real StemEngine job id: `bd95ecc8-adec-4479-af33-b07b948a5c8e`
- queue status sync returned `status=processing`, `orchestration_state=submitted`
Important finding:
- Real `POST /api/separate/from-path` submit requires a path visible to the running StemEngine process (e.g. `/app/media/imports/...`), not a raw host path.
- The smoke runner now auto-discovers a valid runtime-visible source path from StemEngine `/api/subsonic/imports` to avoid false negatives.
Documentation updated:
- `/home/translator/docker/appdata/playak/README.md`
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Work_Log.md`
- `/home/translator/docker/appdata/stemengine/20260318_PlayaK_Progress.md`
Status:
- Localhost real-bridge smoke passed.
- Next: convert this smoke path into automated regression coverage and extend real bridge support to YouTube/SoulSeek.
@@ -0,0 +1,142 @@
# 2026-03-18 — Plan: Replace lyric line segmentation with `karaoke-lyrics-processor`
## Implementation status (2026-03-18)
✅ Implemented.
Delivered:
- backend dependency added: `karaoke-lyrics-processor`
- `lyric_transcriber_service.py` now supports `STEMENGINE_LYRIC_SEGMENTER` modes:
- `auto` (default)
- `karaoke_processor`
- `legacy`
- max line-length control via `STEMENGINE_LYRIC_MAX_LINE_LENGTH`
- plain-text lyrics segmentation path switched to karaoke processor with safe fallback
- WhisperX long-segment split path switched to karaoke processor with conservative word-timestamp remap and legacy fallback on ambiguity
- docs updated in root README + configuration reference + troubleshooting matrix + developer guide
Smoke validation snapshot:
- `auto`: line_count=4, split_segment_count=3, timestamps_monotonic=true
- `legacy`: line_count=3, split_segment_count=2, timestamps_monotonic=true
## Objective
Replace the in-house line segmentation heuristics used in lyric transcription with the `KaraokeLyricsProcessor` implementation from:
- https://github.com/nomadkaraoke/karaoke-lyrics-processor/tree/main/karaoke_lyrics_processor
Target outcome:
- better line breaks for karaoke readability
- fewer unnatural splits in long plain-text lyrics
- one consistent segmentation strategy for forced alignment input
## Current behavior (baseline)
Current segmentation is implemented in `backend/services/lyric_transcriber_service.py` via:
- `_chunk_words_sentence_aware()`
- `_segment_plain_lyrics_lines()`
- `_split_long_segments()`
- `_lyrics_to_whisperx_segments()` (plain-text path)
Current logic relies on punctuation/uppercase/max-word heuristics.
## Proposed approach
### 1) Introduce a segmentation adapter (no big-bang rewrite)
Create an internal wrapper function (same module or a small helper module) that calls `KaraokeLyricsProcessor` with `input_lyrics_text` and returns normalized lines.
Proposed internal API:
- `segment_lyrics_lines(text: str, max_line_length: int = 36) -> list[str]`
Normalization rules:
- trim whitespace lines
- preserve order
- ignore blank results
- return fallback line list if external processor raises
### 2) Dependency strategy
Add Python dependency in backend requirements:
- `karaoke-lyrics-processor` (pinned major/minor after validation)
Keep existing heuristic functions temporarily as fallback until rollout is complete.
### 3) Integration points
Replace segmentation calls in two places first:
1. Plain-text forced-alignment path in `_lyrics_to_whisperx_segments()`.
2. Long-segment post-processing in `_split_long_segments()` for WhisperX output.
Implementation note:
- For WhisperX word-timestamp chunks, segment using text and remap grouped words conservatively (do not degrade timing fidelity).
- If remap confidence is low (token mismatch), keep original segment unchanged.
### 4) Backward compatibility controls
Add feature flag:
- `STEMENGINE_LYRIC_SEGMENTER` with values:
- `legacy` (existing behavior)
- `karaoke_processor` (new behavior)
- `auto` (default after bake-in period; new with safe fallback)
Optional tuning variable:
- `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (default `36`)
### 5) Rollout phases
#### Phase A — Shadow mode (no behavior change)
- Wire adapter and dependency.
- Execute both legacy and new segmenters for sampled jobs.
- Store debug comparison in logs (line count, average chars/line, large mismatch markers).
#### Phase B — Opt-in
- Enable `karaoke_processor` via env flag in staging.
- Validate output quality on multilingual and punctuation-poor lyrics.
#### Phase C — Default-on with fallback
- Set `auto` default.
- Auto-fallback to legacy on adapter exceptions or mapping mismatch.
#### Phase D — Cleanup
- Remove dead legacy paths after stability window.
- Keep small compatibility shim if needed for old jobs.
## Validation plan
### Unit tests
- segmentation adapter returns stable non-empty lines for normal input
- preserves parentheses/quoted text handling expected by karaoke UI
- enforces max line length behavior
- safe fallback when processor throws
### Integration tests
- forced alignment with plain text lyrics still produces valid LRC/ELRC
- existing LRC input path remains unchanged
- WhisperX post-segmentation preserves timestamp monotonicity and non-overlap
### Regression corpus
Test against representative lyrics sets:
- lowercase/no punctuation
- rap/ad-libs with dense tokens
- multilingual text with non-ASCII characters
- parenthetical structure
Acceptance criteria:
- no increase in failed lyric jobs
- improved line readability in manual QA samples
- no timestamp regressions in LRC/ELRC exports
## Risk register
- **Word-timestamp remap drift** when splitting WhisperX segments by text alone.
- Mitigation: conservative remap; fallback to original segment when uncertain.
- **Dependency behavior changes** in upstream processor.
- Mitigation: pin version + smoke tests in CI.
- **Performance overhead** on long lyrics.
- Mitigation: benchmark and keep adapter lightweight.
## Files expected to change (implementation phase)
- `backend/requirements.txt`
- `backend/services/lyric_transcriber_service.py`
- `backend/tests/*` (new unit/integration tests for adapter + mapping)
- `README.md` (new env vars)
- `docs/CONFIGURATION_REFERENCE.md` (segmenter flags)
- `docs/TROUBLESHOOTING_MATRIX.md` (new failure mode/fallback notes)
## Documentation updates included now
This plan is documented in:
- `20260318_Lyric_Line_Segmentation_Replacement_Plan.md`
- `20260313_Fixes (Copilot).md` (decision log entry)
+123
View File
@@ -0,0 +1,123 @@
# 2026-03-18 — PlayaK implementation progress
## Request
Create a new container/workspace named **PlayaK** at:
- `/home/translator/docker/appdata/playak`
Move the karaoke player platform plan to that location and document progress.
## Completed
1. Created PlayaK workspace directory:
- `/home/translator/docker/appdata/playak`
2. Added PlayaK root documentation:
- `/home/translator/docker/appdata/playak/README.md`
3. Moved planning artifact into PlayaK (recreated in new location):
- `/home/translator/docker/appdata/playak/20260318_PlayaK_Karaoke_Player_Platform_Plan.md`
## Additional completed (implementation kickoff)
4. Added PlayaK runtime scaffold:
- `/home/translator/docker/appdata/playak/docker-compose.yml`
- `/home/translator/docker/appdata/playak/.env.example`
- `/home/translator/docker/appdata/playak/.gitignore`
5. Added backend API scaffold:
- `/home/translator/docker/appdata/playak/backend/Dockerfile`
- `/home/translator/docker/appdata/playak/backend/requirements.txt`
- `/home/translator/docker/appdata/playak/backend/app/main.py`
- `/home/translator/docker/appdata/playak/backend/app/models.py`
- `/home/translator/docker/appdata/playak/backend/app/store.py`
- `/home/translator/docker/appdata/playak/backend/app/security.py`
6. Updated PlayaK root docs with run instructions and scaffold status:
- `/home/translator/docker/appdata/playak/README.md`
7. Added SQLite persistence for PlayaK room/queue state:
- `/home/translator/docker/appdata/playak/backend/app/db.py`
- `/home/translator/docker/appdata/playak/backend/app/store.py`
- `/home/translator/docker/appdata/playak/backend/app/main.py` (startup DB init)
- `/home/translator/docker/appdata/playak/docker-compose.yml` (`./data:/data` + `PLAYAK_DB_PATH`)
8. Added auth/authorization scaffold with room-role enforcement:
- `/home/translator/docker/appdata/playak/backend/app/security.py`
- `/home/translator/docker/appdata/playak/backend/app/store.py` (`room_members` helpers)
- `/home/translator/docker/appdata/playak/backend/app/main.py` (permission checks + new endpoints)
- new endpoints:
- `POST /api/playak/rooms/{room_id}/join`
- `GET /api/playak/rooms/{room_id}/state`
9. Added source-ingestion adapter interfaces + orchestration submission wiring:
- `/home/translator/docker/appdata/playak/backend/app/sources.py`
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py`
- `/home/translator/docker/appdata/playak/backend/app/db.py` (`media_items` + queue orchestration columns)
- `/home/translator/docker/appdata/playak/backend/app/store.py` (media persistence + queue submission updates)
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new import/library/submit endpoints)
- new endpoints:
- `GET /api/playak/library`
- `POST /api/playak/import`
- `POST /api/playak/rooms/{room_id}/queue/from-media`
- `POST /api/playak/rooms/{room_id}/queue/{item_id}/submit`
10. Added local provider scan + orchestration status sync wiring:
- `/home/translator/docker/appdata/playak/backend/app/sources.py` (scan allowlist + recursive local discovery)
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py` (status polling support)
- `/home/translator/docker/appdata/playak/backend/app/store.py` (media dedupe + queue status update)
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new endpoints)
- new endpoints:
- `POST /api/playak/import/local-scan`
- `POST /api/playak/rooms/{room_id}/queue/{item_id}/sync-status`
11. Added real StemEngine HTTP bridge + Subsonic provider bridge:
- `/home/translator/docker/appdata/playak/backend/app/orchestration.py` (local + Subsonic real submit/poll)
- `/home/translator/docker/appdata/playak/backend/app/stemengine_bridge.py` (Subsonic search/import proxy)
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new provider endpoints)
- new endpoints:
- `GET /api/playak/providers/subsonic/search`
- `POST /api/playak/providers/subsonic/import`
12. Added host-side smoke validation for the real bridge:
- `/home/translator/docker/appdata/playak/backend/scripts/smoke_real_bridge.py`
- `/home/translator/docker/appdata/playak/.env.host-test.example`
- validated against `http://localhost:8000`
- observed real submit job id + successful `processing` sync state
13. Added focused API regression coverage for the bridge path:
- `/home/translator/docker/appdata/playak/backend/tests/test_bridge_api.py`
- covers import → queue → submit → sync and Subsonic provider registration/deduplication
- validated: `2 tests, OK`
14. Added Phase B/C backend control-plane scaffolding:
- `/home/translator/docker/appdata/playak/backend/app/db.py` (room policies, playback sessions, lyric/style revisions, room events, queue ordering)
- `/home/translator/docker/appdata/playak/backend/app/store.py` (policy/playback/editor/events CRUD + queue reorder)
- `/home/translator/docker/appdata/playak/backend/app/main.py` (new policy/moderation/playback/editor/event endpoints)
- `/home/translator/docker/appdata/playak/backend/app/models.py` (new Phase B/C request/response models)
- `/home/translator/docker/appdata/playak/backend/app/security.py` (policy-aware moderation/playback checks)
- new endpoints:
- `GET|PATCH /api/playak/rooms/{room_id}/policy`
- `POST /api/playak/rooms/{room_id}/leave`
- `PATCH /api/playak/rooms/{room_id}/queue/reorder`
- `POST /api/playak/rooms/{room_id}/playback/sessions`
- `GET /api/playak/rooms/{room_id}/playback/session`
- `POST /api/playak/playback/sessions/{session_id}/command`
- `POST|GET /api/playak/media/{media_id}/lyrics-style`
- `GET /api/playak/rooms/{room_id}/events`
- expanded regression suite validation: `10 tests, OK`
15. Added Phase B/C frontend surfaces (Two.js player/editor + companion PWA scaffold):
- `/home/translator/docker/appdata/playak/frontend/index.html`
- `/home/translator/docker/appdata/playak/frontend/controller.html`
- `/home/translator/docker/appdata/playak/frontend/player.js`
- `/home/translator/docker/appdata/playak/frontend/controller.js`
- `/home/translator/docker/appdata/playak/frontend/api.js`
- `/home/translator/docker/appdata/playak/frontend/app.css`
- `/home/translator/docker/appdata/playak/frontend/manifest.webmanifest`
- `/home/translator/docker/appdata/playak/frontend/sw.js`
- `/home/translator/docker/appdata/playak/frontend/assets/icon.svg`
- compose runtime for static hosting (`playak-frontend` service on `PLAYAK_FRONTEND_PORT`, default `18101`)
- API CORS wiring added (`PLAYAK_CORS_ORIGINS`)
- Current data layer persists to SQLite (room/queue state survives container restarts when `./data` is retained).
- Media library state also persists to SQLite (`media_items`).
- Auth is currently header-based (`X-Global-Role`, `X-User-Id`) and intended as a scaffold before full session/token integration.
- Local source now has provider-backed scan/import path (allowlist-enforced).
- Subsonic now has concrete search/import bridge paths through StemEngine.
- YouTube/SoulSeek adapters still need concrete provider runtime integration.
- SoulSeek adapter still needs concrete provider runtime integration (held as requested).
- Orchestration bridge now supports real StemEngine local/Subsonic submit + job polling when `PLAYAK_STEMENGINE_BASE_URL` is configured.
- Live smoke validation passed against local StemEngine using a runtime-visible imports path.
- Phase B/C backend APIs are now available for frontend and PWA integration.
- Phase B/C frontend surfaces are now scaffolded and connected to those APIs.
## Next suggested step
- Add live YouTube ytdl end-to-end smoke against StemEngine and harden frontend/PWA scaffold (state management, realtime transport, session auth).
+15
View File
@@ -4,6 +4,19 @@
---
## Documentation
- User docs: [docs/USER_GUIDE.md](docs/USER_GUIDE.md)
- Developer docs: [docs/DEVELOPER_GUIDE.md](docs/DEVELOPER_GUIDE.md)
- Documentation plan: [docs/DOCUMENTATION_PLAN.md](docs/DOCUMENTATION_PLAN.md)
- API reference: [docs/API_REFERENCE.md](docs/API_REFERENCE.md)
- Troubleshooting matrix: [docs/TROUBLESHOOTING_MATRIX.md](docs/TROUBLESHOOTING_MATRIX.md)
- Configuration reference: [docs/CONFIGURATION_REFERENCE.md](docs/CONFIGURATION_REFERENCE.md)
- Documentation governance: [docs/DOC_GOVERNANCE.md](docs/DOC_GOVERNANCE.md)
- Release checklist: [docs/RELEASE_CHECKLIST.md](docs/RELEASE_CHECKLIST.md)
---
## Features
- **Stem Separation** — 40+ models across 10 model families
@@ -82,6 +95,8 @@ All configuration is via environment variables. The `docker-compose.yml` accepts
| `MAX_UPLOAD_MB` | `500` | Maximum upload file size in MB |
| `STEMENGINE_GPU_VRAM_RESERVE_MB` | `2048` | Keeps this many MB free on the GPU (WSL2-safe headroom) |
| `OMNIZART_GPU_VRAM_RESERVE_MB` | `STEMENGINE_GPU_VRAM_RESERVE_MB` | Optional sidecar override for Omnizart TensorFlow GPU cap |
| `STEMENGINE_LYRIC_SEGMENTER` | `auto` | Lyric line splitter strategy: `auto`, `karaoke_processor`, `legacy` |
| `STEMENGINE_LYRIC_MAX_LINE_LENGTH` | `36` | Max line length target used by karaoke lyric segmentation |
| `TZ` | `UTC` | Timezone |
| `NAVSERVER` | — | Subsonic server URL (can also be set in the UI) |
| `NAVUSER` | — | Subsonic username |
+11
View File
@@ -362,6 +362,17 @@ DEFAULT_PARAKEET_MODEL = os.environ.get(
"parakeet-1.1b-rnnt-multilingual-asr",
)
# Lyric line segmentation strategy
# - legacy: current punctuation/uppercase/max-word heuristics
# - karaoke_processor: always use karaoke-lyrics-processor splitter
# - auto: karaoke-lyrics-processor with safe fallback to legacy heuristics
LYRIC_SEGMENTER: str = os.environ.get("STEMENGINE_LYRIC_SEGMENTER", "auto").strip().lower() or "auto"
try:
LYRIC_MAX_LINE_LENGTH: int = max(8, int(os.environ.get("STEMENGINE_LYRIC_MAX_LINE_LENGTH", "36")))
except ValueError:
LYRIC_MAX_LINE_LENGTH = 36
# Create directories on import
UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
JOBS_DIR.mkdir(parents=True, exist_ok=True)
+3
View File
@@ -75,6 +75,9 @@ audio-separator[gpu]>=0.28.0
# Lyric transcription via WhisperX (faster-whisper backend + phoneme-level word alignment)
whisperx>=3.8.1
# Karaoke-oriented lyric line segmentation
karaoke-lyrics-processor>=1.0.0
# Phoneme-level syllable alignment (requires espeak-ng system package)
# Used by the "Phoneme alignment" option in AI Transcribe to produce
# syllable-level ELRC timestamps via facebook/wav2vec2-xlsr-53-espeak-cv-ft.
+157 -24
View File
@@ -66,6 +66,8 @@ from ..config import (
PARAKEET_TRANSCRIBE_PATH,
PARAKEET_TIMEOUT_SEC,
DEFAULT_PARAKEET_MODEL,
LYRIC_SEGMENTER,
LYRIC_MAX_LINE_LENGTH,
)
log = logging.getLogger(__name__)
@@ -783,6 +785,16 @@ def _ortho_split(word_text: str, n: int) -> List[str]:
_PUNCT_BREAK = frozenset('.!?,;')
_SEGMENTER_MODES = {"legacy", "karaoke_processor", "auto"}
if LYRIC_SEGMENTER not in _SEGMENTER_MODES:
log.warning(
"Invalid STEMENGINE_LYRIC_SEGMENTER=%r (expected one of %s); defaulting to 'auto'.",
LYRIC_SEGMENTER,
sorted(_SEGMENTER_MODES),
)
_ACTIVE_LYRIC_SEGMENTER = LYRIC_SEGMENTER if LYRIC_SEGMENTER in _SEGMENTER_MODES else "auto"
def _starts_with_sentence_capital(token: str) -> bool:
@@ -843,8 +855,8 @@ def _chunk_words_sentence_aware(words: list, max_words: int = 10, min_sentence_w
return chunks
def _segment_plain_lyrics_lines(lyrics_text: str, max_words: int = 10) -> list[str]:
"""Split plain lyrics into sentence-like lines with lowercase fallback."""
def _segment_plain_lyrics_lines_legacy(lyrics_text: str, max_words: int = 10) -> list[str]:
"""Legacy splitter: sentence-like chunking with lowercase fallback."""
src_lines = [ln.strip() for ln in lyrics_text.splitlines() if ln.strip()]
out: list[str] = []
@@ -867,22 +879,49 @@ def _segment_plain_lyrics_lines(lyrics_text: str, max_words: int = 10) -> list[s
return out
def _split_long_segments(segments: list, max_words: int = 10) -> list:
"""
Split over-long WhisperX segments into shorter lines suitable for LRC display.
def _norm_word_token(token: str) -> str:
"""Normalize a token for conservative line→word remapping."""
return "".join(ch for ch in (token or "").lower() if ch.isalnum() or ch == "'")
Strategy (applied per segment):
1. Walk the word list looking for *punctuation boundaries* — words whose
text ends with a character in _PUNCT_BREAK. When such a boundary is
found AND the current chunk already has at least 3 words, emit a new
sub-segment there.
2. If no punctuation boundary is found within `max_words` words, force-
split at `max_words` regardless.
Each sub-segment inherits the speaker label and gets timestamps from the
first / last word in the slice. Segments with fewer than 2 words are left
unchanged.
"""
def _line_tokens_for_mapping(text: str) -> list[str]:
"""Tokenize line text for word-level mapping (keeps apostrophes)."""
raw_tokens = _re.findall(r"[\w']+", text or "", flags=_re.UNICODE)
return [t for t in (_norm_word_token(tok) for tok in raw_tokens) if t]
def _segment_with_karaoke_processor(lyrics_text: str, max_line_length: int) -> list[str]:
"""Split lyrics with `karaoke-lyrics-processor` and normalize output lines."""
from karaoke_lyrics_processor import KaraokeLyricsProcessor
processor = KaraokeLyricsProcessor(
input_lyrics_text=lyrics_text,
max_line_length=max_line_length,
log_level=logging.WARNING,
)
processed = processor.process()
return [ln.strip() for ln in processed.splitlines() if ln.strip()]
def _segment_plain_lyrics_lines(lyrics_text: str, max_words: int = 10) -> list[str]:
"""Split plain lyrics into line-sized chunks suitable for karaoke display."""
if _ACTIVE_LYRIC_SEGMENTER == "legacy":
return _segment_plain_lyrics_lines_legacy(lyrics_text, max_words=max_words)
try:
return _segment_with_karaoke_processor(lyrics_text, max_line_length=LYRIC_MAX_LINE_LENGTH)
except Exception as exc:
# `auto` and `karaoke_processor` both fail-safe to legacy so lyric jobs
# do not fail on dependency/runtime edge cases.
log.warning(
"karaoke lyric segmentation unavailable (%s); falling back to legacy segmentation.",
exc,
)
return _segment_plain_lyrics_lines_legacy(lyrics_text, max_words=max_words)
def _split_long_segments_legacy(segments: list, max_words: int = 10) -> list:
"""Legacy splitter for WhisperX segments using punctuation/max-word heuristics."""
if max_words <= 0:
return segments
@@ -893,13 +932,7 @@ def _split_long_segments(segments: list, max_words: int = 10) -> list:
result.append(seg)
continue
# Sentence-aware chunking:
# 1) punctuation boundaries
# 2) uppercase sentence starts
# 3) max-words fallback for lowercase/unpunctuated lyrics
chunks = _chunk_words_sentence_aware(words, max_words=max_words, min_sentence_words=3)
# If no effective split happened, keep original segment.
if len(chunks) <= 1:
result.append(seg)
continue
@@ -918,6 +951,105 @@ def _split_long_segments(segments: list, max_words: int = 10) -> list:
return result
def _remap_segment_words_to_lines(seg: dict, split_lines: list[str]) -> Optional[list[dict]]:
"""Map line-split text back to contiguous WhisperX word slices conservatively."""
words = seg.get("words") or []
if len(words) <= 1 or len(split_lines) <= 1:
return None
remapped: list[dict] = []
cursor = 0
for line in split_lines:
line_tokens = _line_tokens_for_mapping(line)
if not line_tokens:
continue
start_idx: Optional[int] = None
for token in line_tokens:
matched = False
while cursor < len(words):
word_token = _norm_word_token(words[cursor].get("word", ""))
if not word_token:
cursor += 1
continue
if word_token == token:
if start_idx is None:
start_idx = cursor
cursor += 1
matched = True
break
return None
if not matched:
return None
if start_idx is None:
return None
end_idx = max(start_idx, cursor - 1)
chunk_words = words[start_idx:end_idx + 1]
if not chunk_words:
return None
first, last = chunk_words[0], chunk_words[-1]
remapped.append({
"start": first.get("start") if first.get("start") is not None else seg["start"],
"end": last.get("end") if last.get("end") is not None else seg["end"],
"text": line.strip(),
"words": chunk_words,
"speaker": seg.get("speaker"),
})
# If we did not consume all meaningful residual tokens, mapping was unsafe.
if any(_norm_word_token(w.get("word", "")) for w in words[cursor:]):
return None
return remapped if len(remapped) > 1 else None
def _split_long_segments(segments: list, max_words: int = 10) -> list:
"""
Split over-long WhisperX segments into karaoke-friendly lines.
`legacy` mode uses punctuation/uppercase/max-word heuristics.
`karaoke_processor`/`auto` mode uses karaoke-lyrics-processor first, then
conservatively remaps generated lines back to contiguous word slices.
"""
if _ACTIVE_LYRIC_SEGMENTER == "legacy":
return _split_long_segments_legacy(segments, max_words=max_words)
result: list[dict] = []
for seg in segments:
words = seg.get("words") or []
if len(words) <= 1:
result.append(seg)
continue
try:
split_lines = _segment_with_karaoke_processor(seg.get("text", ""), max_line_length=LYRIC_MAX_LINE_LENGTH)
except Exception as exc:
log.warning(
"karaoke segment split unavailable for WhisperX segment (%s); falling back to legacy split.",
exc,
)
result.extend(_split_long_segments_legacy([seg], max_words=max_words))
continue
if len(split_lines) <= 1:
result.append(seg)
continue
remapped = _remap_segment_words_to_lines(seg, split_lines)
if remapped:
result.extend(remapped)
else:
# Safe fallback if token remap is ambiguous (punctuation-heavy,
# language-specific tokenization, etc.).
result.extend(_split_long_segments_legacy([seg], max_words=max_words))
return result
_VOCAL_KEYWORDS = frozenset(('vocal', 'vocals', 'voice', 'lead', 'sing'))
_AUDIO_EXTS = frozenset(('.mp3', '.wav', '.flac', '.ogg', '.m4a', '.aac'))
@@ -2004,8 +2136,9 @@ def run_lyric_transcription(
jm.update_job(job_id, status=jm.JobStatus.CANCELLED, progress="Cancelled.")
return
# Split over-long segments into shorter lyric lines before converting.
# Punctuation boundaries are preferred; falls back to a hard word-count cut.
# Split over-long segments into karaoke-friendly lyric lines before converting.
# Uses karaoke-lyrics-processor with conservative word-timestamp remap,
# then falls back to legacy heuristics when remap is ambiguous.
raw_segments = _split_long_segments(wx_result["segments"], max_words=10)
# Convert WhisperX output → LyricTranscriptionResult
+397
View File
@@ -0,0 +1,397 @@
# StemEngine API Reference (Phase 2)
This is a practical API map for contributors. It focuses on endpoint groups, request shape, and common response patterns.
## Base conventions
- Base URL (default): `http://localhost:8000`
- Most long-running operations return a `job_id` and run asynchronously.
- Poll status via `GET /api/jobs/{job_id}`.
- Download job outputs via `GET /api/jobs/{job_id}/download/{filename}` when status is `done`.
## Common async pattern
1. Start operation (for example separation/transcription/alignment).
2. Receive `{ "job_id": "..." }`.
3. Poll job until `status` is `done`, `error`, or `cancelled`.
4. Read `files` and `result` from job payload.
---
## System and operations
### GET /api/system/health
Lightweight liveness probe.
### GET /api/system/readiness
Readiness checks for DB and writable directories.
### GET /api/system/executors
Queue and worker state from the job manager.
### GET /api/system/gpu-info
GPU availability + VRAM figures + transcription CUDA health flag.
### POST /api/system/gpu-reset
Admin-protected reset path for GPU recovery and cache flush.
---
## Jobs
### GET /api/jobs
List jobs.
### GET /api/jobs/{job_id}
Get one job payload.
### DELETE /api/jobs/{job_id}
Request cancellation.
### GET /api/jobs/{job_id}/download/{filename}
Download one output file from a completed job.
---
## Separation
### GET /api/separate/models
Returns available separation models.
### GET /api/separate/formats
Returns output formats.
### GET /api/separate/presets
Returns preset types and static options.
### GET /api/separate/per-instrument-models
Returns models usable for per-stem reprocessing.
### POST /api/separate
Multipart upload separation request.
Form fields:
- `file` (required)
- `model`
- `output_format`
- `preset_type`
- `static_option`
- `dynamic_levels` (JSON string)
- `project_name`
- `instrument_models` (JSON string)
- `sep_power_preset`
Example response:
```json
{
"job_id": "f52d...",
"preset_type": "stems_only",
"project_name": "Artist - Title"
}
```
### POST /api/separate/existing
Re-run separation on an existing project source.
### POST /api/separate/from-path
Run separation from a server-side file path (restricted to imports directory).
---
## MIDI transcription
### GET /api/transcribe/modes
Returns available modes and availability flags.
### POST /api/transcribe
Multipart upload request:
- `file`
- `mode`
### POST /api/transcribe/from-project
Form request:
- `project_name`
- `mode`
Selects the best non-drum stem by preference order.
---
## Lyric transcription (ASR)
### GET /api/lyric-transcribe/models
Returns engines/models/defaults and runtime availability.
### GET /api/lyric-transcribe/languages
Returns supported language choices.
### GET /api/lyric-transcribe/formats
Returns output formats (`lrc`, `elrc`, `json`, `txt`, `srt`, `vtt`).
### POST /api/lyric-transcribe
Multipart upload request.
Form fields:
- `file`
- `engine` (`whisperx` | `parakeet`)
- `model`
- `language`
- `task` (`transcribe` | `translate`, with engine constraints)
- `output_formats` (comma-separated)
- `initial_prompt`
- `temperature`
### POST /api/lyric-transcribe/from-project
Form fields include:
- `project_name`
- `stem_name` (optional; vocals auto-detection if absent)
- transcription options above
### POST /api/lyric-transcribe/with-diarization
Multipart upload + diarization options.
### POST /api/lyric-transcribe/project/{project_id}
JSON body version for project-stem lyric transcription.
Example JSON body:
```json
{
"stem_name": "vocals",
"engine": "whisperx",
"model": "large-v2",
"task": "transcribe",
"output_formats": ["elrc", "json", "txt"],
"temperature": 0.0,
"condition_on_previous_text": false,
"word_timestamps": true
}
```
---
## LRC / ELRC alignment
### GET /api/lrc/sources
Returns source modes: `auto`, `tag`, `file`, `text`.
### POST /api/lrc/extract
Extracts lyrics from uploaded audio metadata/source.
### POST /api/lrc/align
Multipart request:
- `audio`
- `lyrics_source`
- `lyrics_text` (required when source=`text`)
- `output_filename`
- `style_options` (JSON string)
### POST /api/lrc/align-with-file
Multipart request with `audio` + `lyrics_file`.
### POST /api/lrc/align-existing
Align using existing project vocals stem.
---
## Projects
### GET /api/projects
List projects.
### GET /api/projects/{project_id}
Get project detail with linked jobs.
### GET /api/projects/{project_id}/cover-art
Returns embedded art bytes if available.
### DELETE /api/projects/{project_id}
Delete project and related assets.
### POST /api/projects/{project_id}/transcribe
JSON body:
```json
{
"stem_name": "other",
"mode": "polyphonic"
}
```
### POST /api/projects/{project_id}/abc
JSON body:
```json
{
"stem_name": "other",
"song_title": "My Song"
}
```
### POST /api/projects/{project_id}/mix
JSON body:
```json
{
"preset_type": "dynamic",
"dynamic_levels": {"vocals": 0.8, "drums": 1.1}
}
```
### POST /api/projects/{project_id}/lrc
JSON body supports lyrics source, style, timing precision, and power preset.
### POST /api/projects/{project_id}/lyric-transcribe
JSON body supports engine, model, alignment options, diarization, and existing lyrics alignment paths.
### POST /api/projects/{project_id}/reprocess
Reprocess a single stem with a selected model.
### POST /api/projects/{project_id}/rename-stem
Rename tracked stem file.
### POST /api/projects/{project_id}/rescan-stems
Rescan stem directory and register untracked files.
### Maintenance (admin-protected)
- `GET /api/projects/cleanup/verify`
- `POST /api/projects/cleanup/write-metadata`
- `POST /api/projects/cleanup/rename-stems`
- `POST /api/projects/cleanup/purge-orphaned`
- `POST /api/projects/cleanup/clean-lrc-files`
---
## Pipeline (end-to-end)
### POST /api/pipeline
Multipart request combining separation + transcription + ABC path.
Form fields:
- `file`
- `sep_model`
- `sep_format`
- `transcribe_mode`
- `song_title`
- `instrument_models` (JSON string)
- `lyric_engine`
- `lyric_model`
Response:
```json
{ "job_id": "..." }
```
---
## ABC conversion
### POST /api/abc-convert
Upload one or more MIDI files and convert to ABC.
### POST /api/abc-convert/from-job
Reuse MIDI files produced by an existing completed job.
---
## Model download management
### GET /api/models/list
Returns model catalog and download status.
### POST /api/models/download
Admin-protected start/resume download.
Request:
```json
{ "model_id": "bs_roformer_4stem" }
```
### POST /api/models/pause
### POST /api/models/resume
### POST /api/models/cancel
Admin-protected download control.
### GET /api/models/progress/{model_id}
Get one model progress.
### GET /api/models/progress
Get all active progress entries.
### DELETE /api/models/{model_id}
Admin-protected model deletion.
---
## Settings
### GET /api/settings
Returns persisted settings with sensitive values redacted (`*_set` flags).
### PATCH /api/settings
Partial update. Common keys include:
- `hf_token`
- `whisper_model`
- `sep_power`
- `whisper_word_ts`
- `whisper_condition_prev`
- `whisper_phoneme_align`
- `auto_steps`
- `auto_model`
- `auto_format`
- `auto_presets`
Example patch:
```json
{
"whisper_model": "large-v2",
"sep_power": "standard",
"whisper_word_ts": true
}
```
### DELETE /api/settings/hf-token
Delete stored token; env-var fallback can still apply.
---
## Subsonic integration
### GET /api/subsonic/config
Returns configured state (no password echoed).
### POST /api/subsonic/config
Validates and stores connection config.
### DELETE /api/subsonic/config
Removes stored config.
### GET /api/subsonic/search
Query params support text + title/artist/album filtering.
### POST /api/subsonic/import
Imports one Subsonic song into local imports.
### POST /api/subsonic/import-ytdl
Enqueues yt-dlp import job.
### GET /api/subsonic/imports
Lists local imports.
### DELETE /api/subsonic/imports/{filename}
Deletes one imported file.
---
## Errors and status codes (common)
- `400` validation/unsupported option
- `403` security/path constraints
- `404` missing job/project/file/model
- `503` optional runtime unavailable (for example Omnizart/Parakeet/WhisperX)
For production-grade clients, treat all non-2xx as recoverable with user-readable error surfacing and retry logic where appropriate.
+193
View File
@@ -0,0 +1,193 @@
# StemEngine Configuration Reference (Phase 2)
This guide expands configuration from raw env var listings to practical usage guidance.
## 1) Configuration layers
StemEngine reads config from container environment variables. Typical deployment flow:
1. Compose file defines environment keys.
2. `.env` provides values for compose substitutions.
3. Backend reads env vars on startup.
When a setting is also available in UI (for example some integration settings), backend env still defines baseline behavior and fallbacks.
---
## 2) Core required configuration
| Variable | Why it matters | Recommended practice |
|---|---|---|
| `SECRET_KEY` | Encrypts sensitive stored values | Set a strong random value and keep it stable across restarts |
| `SEFOLDER` (compose substitution) | Host mount root for backend/media/model paths | Use persistent storage, not ephemeral temp dirs |
| `PROJECTS` (compose substitution) | Host project output location | Put on a volume with enough space and backups |
---
## 3) Storage and path layout
Backend path keys (resolved in runtime):
- `MEDIA_DIR`
- `UPLOAD_DIR`
- `JOBS_DIR`
- `PROJECTS_DIR`
- `DB_PATH`
- `LRC_SYNC_DIR`
Model cache/directory keys include:
- `BS_ROFORMER_MODEL_DIR`
- `MEL_BAND_MODEL_DIR`
- `MDXNET_MODEL_DIR`
- `SCNET_MODEL_DIR`
- `MEDLEYVOX_MODEL_DIR`
- `APOLLO_MODEL_DIR`
- `ASPIRATION_MODEL_DIR`
- `BANDIT_MODEL_DIR`
Guidance:
- Keep all writable paths on persistent mounts.
- Avoid sharing the same path between unrelated services unless intended.
- Verify write permissions for the runtime user.
---
## 4) GPU and performance tuning
| Variable | Effect | Practical usage |
|---|---|---|
| `CPU_SEPARATION` | Forces CPU path when true | Use only when no GPU is available |
| `STEMENGINE_GPU_VRAM_RESERVE_MB` | Keeps VRAM headroom free | Increase on WSL2/shared desktop GPU systems |
| `OMNIZART_GPU_VRAM_RESERVE_MB` | Sidecar TensorFlow VRAM reserve | Keep aligned with backend reserve policy |
| `PYTORCH_CUDA_ALLOC_CONF` | Allocator behavior | `expandable_segments:True` helps fragmentation scenarios |
Recommendations:
- For desktop + WSL2 use, reserve enough VRAM to avoid compositor starvation.
- If you see intermittent OOM, reduce model size and concurrency before raising complexity.
---
## 5) CORS and network access
| Variable | Use |
|---|---|
| `CORS_ORIGINS` | Exact allowed origins (comma-separated) |
| `CORS_ORIGIN_REGEX` | Pattern for dynamic/local/private hosts |
Guidance:
- Keep CORS narrow in production.
- Prefer explicit origins where possible.
- Use regex only for known internal hostname ranges.
---
## 6) Optional integration settings
### Subsonic
- `SUBSONIC_URL`
- `SUBSONIC_USERNAME`
- `SUBSONIC_PASSWORD`
Notes:
- Can also be configured through Settings API/UI.
- Validate credentials immediately after changes.
### Hugging Face token
- `HF_TOKEN`
Notes:
- Needed for some gated model workflows (for example diarization dependencies).
- If both env and stored settings exist, runtime fallback behavior applies as implemented.
---
## 7) Sidecar configuration
### Omnizart
- `OMNIZART_ENABLED`
- `OMNIZART_SIDECAR_URL`
Use when enabling Omnizart transcription modes. If disabled, Omnizart modes remain unavailable.
### Parakeet
- `PARAKEET_ENABLED`
- `PARAKEET_SIDECAR_URL`
- `PARAKEET_HEALTH_PATH`
- `PARAKEET_TRANSCRIBE_PATH`
- `PARAKEET_TIMEOUT_SEC`
- `DEFAULT_PARAKEET_MODEL`
Use when enabling Parakeet lyric transcription. Ensure sidecar health endpoint is reachable from backend container network.
### Lyric line segmentation
- `STEMENGINE_LYRIC_SEGMENTER` (`auto` / `karaoke_processor` / `legacy`)
- `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (default `36`)
Behavior:
- `auto`: use `karaoke-lyrics-processor` first, fallback to legacy heuristics on runtime/dependency mismatch.
- `karaoke_processor`: prefer karaoke processor for line breaking; still fail-safe to legacy behavior if the dependency is unavailable.
- `legacy`: keep existing punctuation/uppercase/max-word heuristic splitter only.
Recommended rollout:
1. Start with `auto`.
2. Validate line quality against your lyrics corpus.
3. Use `legacy` only as a temporary compatibility mode.
---
## 8) Operational profiles
### Profile A: single-GPU workstation
- Keep `STEMENGINE_GPU_VRAM_RESERVE_MB` non-zero (for example 2048 or higher).
- Avoid launching multiple heavy jobs at once.
- Prefer balanced defaults (`standard` presets) first.
### Profile B: dedicated server
- Use tighter CORS.
- Enable only required sidecars.
- Keep model/cache paths on high-throughput persistent storage.
### Profile C: CPU fallback
- Set `CPU_SEPARATION=true`.
- Expect much slower separation/transcription.
- Consider limiting upload sizes and queue depth.
---
## 9) Validation checklist after config changes
1. Restart services.
2. Verify `/api/system/health` and `/api/system/readiness`.
3. Verify model list and availability endpoints:
- `/api/separate/models`
- `/api/transcribe/modes`
- `/api/lyric-transcribe/models`
4. Run one end-to-end smoke job.
---
## 10) Common misconfigurations
- Missing/unstable `SECRET_KEY` causing settings/cipher issues.
- Wrong sidecar URL or disabled profile while feature is enabled in UI.
- Non-persistent model directories causing repeated downloads.
- Overly broad or missing CORS causing UI/API connectivity failures.
- Insufficient VRAM reserve on shared desktop systems.
Keep configuration changes small and verify incrementally.
+100
View File
@@ -0,0 +1,100 @@
# StemEngine Developer Guide
## 1) Architecture overview
StemEngine is a full-stack system:
- **Frontend**: React + TypeScript + Vite
- **Backend**: FastAPI + Python services
- **Sidecars**: optional model-specific services (for example Parakeet, Omnizart)
- **Persistence**: project directories, job output directories, DB-backed metadata
High-level flow:
1. UI submits jobs to API.
2. Backend routes requests to service layers.
3. Job manager tracks async progress and status.
4. Outputs are written to configured project/output paths.
## 2) Repository map
- `backend/` API, job orchestration, services, tests
- `frontend/` React UI
- `docker/` compose and deployment assets
- `lyrics-sync/` alignment utilities
- `parakeet-sidecar/` Parakeet integration runtime
## 3) Backend API surface (router prefixes)
Key API groups:
- `/api/separate`
- `/api/transcribe`
- `/api/lyric-transcribe`
- `/api/lrc`
- `/api/projects`
- `/api/pipeline`
- `/api/jobs`
- `/api/models`
- `/api/system`
- `/api/settings`
- `/api/subsonic`
- `/api/abc-convert`
## 4) Local development workflows
### Backend
- Build image: `make build`
- Run tests: `make test`
- Smoke tests: `make test-smoke`
- Migrations: `make alembic-upgrade`
### Frontend
From `frontend/`:
- Install: `npm install`
- Dev server: `npm run dev`
- Type/lint/build check: `npm run check`
## 5) Coding and change strategy
When adding a feature:
1. Define API contract changes first.
2. Add/update service layer behavior.
3. Extend router validations.
4. Update frontend types + UI behavior.
5. Add or update tests.
6. Update user/developer docs in `docs/`.
## 6) Operational notes
- GPU safety limits are centrally important for stability.
- Whisper/WhisperX/sidecar paths can have different feature support.
- Lyric line segmentation now uses `karaoke-lyrics-processor` by default in `auto` mode, with a legacy fallback path controlled by `STEMENGINE_LYRIC_SEGMENTER`.
- Keep long-running tasks cancellable and progress-reporting through the job manager.
## 7) Documentation rules for contributors
- Every user-facing feature change must update [USER_GUIDE.md](./USER_GUIDE.md).
- Every architecture/API/process change must update this guide.
- Large implementation work should include a dated plan or ADR-style note.
- Every PR must complete documentation-impact checks via [../.github/PULL_REQUEST_TEMPLATE.md](../.github/PULL_REQUEST_TEMPLATE.md).
## 8) Documentation governance (Phase 3)
- Governance process: [DOC_GOVERNANCE.md](./DOC_GOVERNANCE.md)
- Release verification: [RELEASE_CHECKLIST.md](./RELEASE_CHECKLIST.md)
- Feature template: [templates/FEATURE_DOC_TEMPLATE.md](./templates/FEATURE_DOC_TEMPLATE.md)
- Migration template: [templates/MIGRATION_NOTE_TEMPLATE.md](./templates/MIGRATION_NOTE_TEMPLATE.md)
## 9) Suggested next technical docs
- API endpoint reference with request/response examples (**implemented**: [API_REFERENCE.md](./API_REFERENCE.md))
- Troubleshooting matrix for GPU/runtime/sidecar failures (**implemented**: [TROUBLESHOOTING_MATRIX.md](./TROUBLESHOOTING_MATRIX.md))
- Configuration usage guide (**implemented**: [CONFIGURATION_REFERENCE.md](./CONFIGURATION_REFERENCE.md))
- Docs governance + templates (**implemented**: [DOC_GOVERNANCE.md](./DOC_GOVERNANCE.md))
- Data model and migration policy
- Job lifecycle/state machine documentation
+115
View File
@@ -0,0 +1,115 @@
# StemEngine Documentation Plan
## Purpose
Create and maintain complete, audience-specific documentation for:
1. **Users** (how to operate StemEngine successfully)
2. **Developers** (how to build, maintain, and extend StemEngine)
This plan defines scope, phases, owners, and acceptance criteria.
---
## Goals
- Reduce onboarding time for both users and contributors.
- Make major workflows discoverable and repeatable.
- Keep documentation synchronized with actual behavior.
- Introduce a lightweight maintenance process so docs do not drift.
---
## Audience and scope
### User documentation scope
- Installation and first run
- Common task workflows (separation, transcription, lyric sync)
- File outputs and where to find them
- Basic troubleshooting and known limits
### Developer documentation scope
- Architecture overview and component boundaries
- API/route map and service responsibilities
- Development setup and test workflow
- Contribution and documentation update rules
---
## Delivery phases
## Phase 1 — Baseline docs (completed)
Deliverables:
- `docs/README.md` (docs index)
- `docs/USER_GUIDE.md` (user baseline)
- `docs/DEVELOPER_GUIDE.md` (developer baseline)
- `docs/DOCUMENTATION_PLAN.md` (this plan)
Acceptance criteria:
- Both audiences have a dedicated entry point.
- Core workflows and architecture are documented at a high level.
- Documentation ownership expectations are defined.
## Phase 2 — Expand depth (completed)
Deliverables:
- API reference doc with endpoint groups and payload examples.
- Troubleshooting matrix by subsystem (GPU, model download, sidecars, alignment).
- Configuration reference expanded from env-var tables to usage guidance.
Acceptance criteria:
- New contributors can run tests and add a feature without tribal knowledge.
- Users can self-serve common failures from docs.
## Phase 3 — Quality and governance (completed)
Deliverables:
- PR checklist item: docs impact reviewed.
- Doc freshness review cadence (for example monthly or per release).
- Standard template for feature docs and migration notes.
Acceptance criteria:
- Documentation updates become part of normal development flow.
- Drift between implementation and docs is reduced.
---
## Work breakdown and owners
- **Product/maintainers**: approve user workflow priority and terminology.
- **Backend contributors**: maintain API/service/ops sections.
- **Frontend contributors**: maintain UI flow and screenshots (if added).
- **Release owner**: verify doc updates before release tagging.
---
## Tracking checklist
- [x] Create docs index
- [x] Create user guide baseline
- [x] Create developer guide baseline
- [x] Document multi-phase plan
- [x] Add API reference with examples
- [x] Add troubleshooting matrix
- [x] Add configuration reference with usage guidance
- [x] Add PR/release doc update checklist
- [x] Define review cadence and owner rotation
---
## Definition of done
Documentation initiative is considered operational when:
- Baseline docs exist for both audiences (done).
- Expansion docs are published and linked from docs index.
- Team workflow requires doc updates for meaningful behavior changes.
+76
View File
@@ -0,0 +1,76 @@
# Documentation Governance (Phase 3)
This document defines how documentation stays fresh as StemEngine evolves.
## 1) Policy
- Documentation updates are required for meaningful behavior changes.
- Documentation review is part of pull request completion.
- Release readiness includes documentation verification.
## 2) Cadence
Two cadences are enforced:
1. **Per PR**
- Author must declare documentation impact in PR template.
- Reviewer must confirm docs are updated or explicitly not required.
2. **Per release / monthly review**
- Run a docs freshness pass at each release.
- If releases are infrequent, run at least once per month.
## 3) Owner rotation
Use a rotating documentation steward role.
- **Rotation period**: one release cycle (or one month)
- **Owner**: assigned in planning/release issue
- **Backup owner**: one additional maintainer
Steward responsibilities:
- Validate docs are linked and discoverable.
- Verify API docs reflect current endpoints.
- Verify troubleshooting and config references are still accurate.
- Track stale or missing docs as follow-up work.
## 4) PR requirements
Every PR must answer documentation impact via template:
- docs updated, or
- docs explicitly not needed with rationale.
Common triggers requiring docs update:
- New/changed endpoint or request shape
- New/changed environment variable
- Feature behavior changes visible to users
- New sidecar/runtime dependency
- New operational or troubleshooting behavior
## 5) Release checklist integration
Before tagging a release:
1. Confirm documentation-impact PR checks are present in merged changes.
2. Update user/developer guides for behavior changes.
3. Update API, troubleshooting, and configuration references as needed.
4. Add migration note when compatibility or operations changed.
## 6) Escalation for stale docs
If docs drift is found:
- Open a documentation debt issue in the same milestone.
- Label as `docs-debt` and include impacted files.
- Assign to current documentation steward.
## 7) Definition of healthy docs process
The process is healthy when:
- PRs consistently declare documentation impact.
- Release checks catch stale docs before release.
- New contributors can ship changes without tribal knowledge.
+29
View File
@@ -0,0 +1,29 @@
# StemEngine Documentation
This directory contains documentation for two audiences:
- **Users** who run StemEngine to separate stems, transcribe audio, and sync lyrics.
- **Developers** who build, debug, and extend StemEngine.
## Start here
- [User Guide](./USER_GUIDE.md)
- [Developer Guide](./DEVELOPER_GUIDE.md)
- [Documentation Plan](./DOCUMENTATION_PLAN.md)
## Phase 2 references
- [API Reference](./API_REFERENCE.md)
- [Troubleshooting Matrix](./TROUBLESHOOTING_MATRIX.md)
- [Configuration Reference](./CONFIGURATION_REFERENCE.md)
## Phase 3 governance
- [Documentation Governance](./DOC_GOVERNANCE.md)
- [Release Checklist](./RELEASE_CHECKLIST.md)
- [Feature Doc Template](./templates/FEATURE_DOC_TEMPLATE.md)
- [Migration Note Template](./templates/MIGRATION_NOTE_TEMPLATE.md)
## Documentation status
These docs are a structured baseline. They should be treated as living docs and updated with each feature change.
+29
View File
@@ -0,0 +1,29 @@
# Release Checklist (Documentation + Quality)
Use this checklist before creating a release tag.
## Core checks
- [ ] Backend checks/tests passed
- [ ] Frontend checks passed
- [ ] Critical workflows smoke-tested
## Documentation checks
- [ ] User-facing changes reflected in docs/USER_GUIDE.md
- [ ] Developer-facing changes reflected in docs/DEVELOPER_GUIDE.md
- [ ] API changes reflected in docs/API_REFERENCE.md
- [ ] Ops/runtime changes reflected in docs/TROUBLESHOOTING_MATRIX.md
- [ ] Env/config changes reflected in docs/CONFIGURATION_REFERENCE.md
- [ ] Governance notes current in docs/DOC_GOVERNANCE.md
## Migration / compatibility
- [ ] No migration required
- [ ] Migration note added from docs/templates/MIGRATION_NOTE_TEMPLATE.md
- [ ] Backward compatibility statement included in release notes
## Sign-off
- [ ] Documentation steward reviewed
- [ ] Release owner approved
+87
View File
@@ -0,0 +1,87 @@
# StemEngine Troubleshooting Matrix (Phase 2)
Use this matrix to triage by subsystem. Each row gives symptoms, likely causes, checks, and corrective actions.
## Quick triage order
1. Check container health (`/api/system/health`, `/api/system/readiness`).
2. Check job payload (`/api/jobs/{job_id}`) for exact failure stage.
3. Check subsystem-specific row below.
---
## Matrix
| Subsystem | Symptom | Likely cause | What to check | Corrective action |
|---|---|---|---|---|
| GPU runtime | CUDA errors, OOM, jobs fail after earlier GPU-heavy tasks | VRAM pressure or fragmented allocator state | `/api/system/gpu-info`, recent job history | Use smaller model/preset, increase VRAM reserve, run `/api/system/gpu-reset` (admin), retry |
| GPU scheduling (WSL2) | Desktop/UI stutter while jobs run | GPU headroom too low | Reserve env vars and current workload | Increase `STEMENGINE_GPU_VRAM_RESERVE_MB`, avoid concurrent heavy jobs |
| Model downloads | Model stuck in `error` or not downloaded | Network issue, invalid/placeholder repo, permissions | `/api/models/progress`, `/api/models/list` | Retry download, verify network and model metadata, ensure model dirs writable |
| Separation | `Unknown model` / poor output | Invalid model id or unsuitable model family | `/api/separate/models` and request payload | Use valid `model`, choose family matched to source material |
| Transcribe (MIDI) | `Omnizart not available` | Sidecar disabled/unreachable | `OMNIZART_ENABLED`, sidecar URL/health, mode selected | Enable sidecar or switch to basic-pitch mode |
| Lyric transcription (WhisperX) | Whisper/WhisperX unavailable | Package not installed in runtime image | `/api/lyric-transcribe/models` availability flags | Install/enable runtime dependencies, rebuild image |
| Lyric line segmentation | Odd line breaks or unexpected fallback style | `karaoke-lyrics-processor` unavailable or remap fallback triggered | backend logs, `STEMENGINE_LYRIC_SEGMENTER`, `STEMENGINE_LYRIC_MAX_LINE_LENGTH` | Use `auto` mode, verify dependency in runtime image, tune max line length, compare against `legacy` |
| Lyric transcription (Parakeet) | `Parakeet runtime unavailable` | Sidecar disabled/unreachable | `PARAKEET_ENABLED`, sidecar URL, `/health` | Enable profile, correct URL/path, verify network reachability |
| LRC alignment | No aligned lines / low quality timing | Wrong source lyrics, weak vocals stem, language mismatch | Source mode, stem quality, selected language | Use vocals stem, provide clean lyrics text/file, re-run with corrected source |
| Project operations | `Project not found` / missing stems | Project ID mismatch, files moved manually | `/api/projects`, project detail, disk paths | Rescan stems endpoint, restore/move files back, avoid out-of-band edits |
| Jobs/downloads | Download endpoint returns `Job not complete` | Job still running/failed | `/api/jobs/{job_id}` status | Poll until done, then download exact `files[].name` |
| Settings/auth | Admin endpoints denied | Missing/incorrect admin token | server env + request headers | Provide valid admin token for protected routes |
| Subsonic import | Search/import fails | Bad credentials/server URL, upstream issue | `/api/subsonic/config`, upstream server reachability | Re-save config with valid credentials, check external server health |
---
## Focused playbooks
### A) GPU OOM recovery playbook
1. Stop launching new heavy jobs.
2. Check GPU state via `/api/system/gpu-info`.
3. Call `/api/system/gpu-reset` (admin).
4. Re-run with lower-cost options:
- smaller separation model
- lower `sep_power_preset`
- reduced concurrent workload
5. If persistent, restart container.
### B) Parakeet sidecar recovery playbook
1. Verify `PARAKEET_ENABLED=true` in backend.
2. Verify `PARAKEET_SIDECAR_URL` points to the reachable service name.
3. Check sidecar `GET /health` returns 200 from backend network context.
4. Re-test `/api/lyric-transcribe/models` availability fields.
### C) Model download recovery playbook
1. Check `/api/models/progress/{model_id}`.
2. If paused/cancelled unexpectedly, resume or restart download.
3. Validate writable model directory mounts.
4. If corrupted partial files are suspected, cancel and restart.
### D) Alignment quality recovery playbook
1. Use a clear vocals-focused stem.
2. Prefer accurate, line-broken lyrics input when forcing alignment.
3. Match language hint to track language.
4. Re-run with engine/options tuned for material.
### E) Lyric segmentation quality playbook
1. Confirm runtime has `karaoke-lyrics-processor` installed.
2. Verify `STEMENGINE_LYRIC_SEGMENTER` is set as intended (`auto` recommended).
3. Tune `STEMENGINE_LYRIC_MAX_LINE_LENGTH` (typical karaoke range: 32–42).
4. Re-test with the same track and compare `auto` vs `legacy` output for regressions.
---
## Signals to capture in bug reports
Include:
- Endpoint called and payload summary
- `job_id`
- Full job payload (`status`, `progress`, `error`)
- `/api/system/readiness` result
- `/api/system/gpu-info` result (for GPU issues)
- Relevant sidecar enablement/env summary
This keeps issue triage reproducible and fast.
+85
View File
@@ -0,0 +1,85 @@
# StemEngine User Guide
## 1) What StemEngine does
StemEngine helps you:
- Separate songs into stems (vocals, drums, bass, etc.)
- Transcribe audio to MIDI
- Generate and align lyrics (LRC/ELRC)
- Create karaoke-ready project outputs
## 2) Requirements
- Docker + Docker Compose
- NVIDIA GPU + NVIDIA Container Toolkit for best performance
- Browser access to the StemEngine UI
## 3) Quick start
1. Copy `.env.example` to `.env`.
2. Set required variables (`SEFOLDER`, `ENGINEFOLDER`, `PROJECTS`, `SECRET_KEY`).
3. Start StemEngine:
- `docker compose up -d --build`
4. Open `http://localhost:8000`.
## 4) Core workflows
### A. Separate a track into stems
1. Go to the separation workflow in the UI.
2. Upload audio.
3. Choose a model family and model.
4. Start job and monitor progress.
5. Download outputs or open the generated project.
### B. Transcribe to MIDI
1. Select a project/audio source.
2. Run MIDI transcription (polyphonic or melody mode).
3. Export MIDI and optionally convert to ABC notation.
### C. Create synced lyrics
1. Open Lyric Transcription.
2. Choose engine:
- `WhisperX` for richer options (alignment/diarization paths)
- `Parakeet` for sidecar ASR flow
3. Choose output format (`lrc`, `elrc`, `txt`, `json`, `srt`, `vtt`).
4. Run job and save outputs.
### D. Align existing lyrics
1. Open LRC/Lyrics alignment workflow.
2. Paste/upload existing lyrics.
3. Run alignment to generate timed output.
## 5) Output locations
Generated artifacts are stored in the configured project/output directories. Typical output includes:
- Source audio copy
- Separated stems (`Stems/`)
- Optional mixes (`Instrumental/`, `Static/`, `Dynamic/`)
- Lyrics artifacts (`.lrc`, `.elrc`, `.txt`, `.json`, `.srt`, `.vtt`)
## 6) Troubleshooting
- **GPU memory issues**: reduce model size, lower workload, or increase reserve tuning variables.
- **Slow runs**: verify GPU container runtime is enabled.
- **Parakeet unavailable**: confirm sidecar URL and health endpoint configuration.
- **No/poor lyric alignment**: prefer vocals stem input and verify language settings.
## 7) Best practices
- Use vocals-focused stems for lyric transcription quality.
- Start with balanced models, then move to large models only when needed.
- Keep project names and source metadata clean for easier downstream workflows.
## 8) Known limits
- Some advanced options are engine-specific.
- Sidecar-dependent features require correctly configured auxiliary services.
- CPU-only operation is supported but can be significantly slower.
+70
View File
@@ -0,0 +1,70 @@
# Feature Documentation Template
Use this template for any feature that changes user workflows, API behavior, or operations.
## 1) Feature summary
- **Feature name**:
- **Status**: draft | implemented | released
- **Date**:
- **Owner**:
## 2) Problem and goal
- What problem does this feature solve?
- What is the intended user/developer outcome?
## 3) Scope
### In scope
-
### Out of scope
-
## 4) User workflow changes
- Before:
- After:
- UI/API touchpoints:
## 5) Technical changes
- Backend:
- Frontend:
- Sidecars/runtime:
- Data/storage:
## 6) Configuration impact
- New variables:
- Changed variables:
- Defaults and migration behavior:
## 7) API impact
- New endpoints:
- Changed endpoints:
- Deprecated endpoints:
- Request/response examples:
## 8) Failure modes and troubleshooting
- Likely failures:
- Detection signals:
- Recovery steps:
## 9) Validation
- Automated tests:
- Manual verification:
- Known limits:
## 10) Documentation updates
- [ ] docs/USER_GUIDE.md
- [ ] docs/DEVELOPER_GUIDE.md
- [ ] docs/API_REFERENCE.md
- [ ] docs/TROUBLESHOOTING_MATRIX.md
- [ ] docs/CONFIGURATION_REFERENCE.md
- [ ] Release notes updated
+49
View File
@@ -0,0 +1,49 @@
# Migration Note Template
Use this when a change affects compatibility, operations, configuration, or stored data.
## 1) Migration title
- **Title**:
- **Date**:
- **Owner**:
- **Applies to versions**:
## 2) What changed
- Summary of behavior/API/config/data changes.
## 3) Who is impacted
- Users:
- Operators:
- Developers/integrators:
## 4) Required actions
1.
2.
3.
## 5) Backward compatibility
- Compatible without action: yes/no
- Deprecated behavior and timeline:
## 6) Rollback plan
- Preconditions:
- Steps:
- Data recovery notes:
## 7) Validation after migration
- Health/readiness checks:
- Functional checks:
- Observability checks:
## 8) References
- PR:
- Related docs:
- Incident/issues (if any):