11 KiB
Implementation Checklist: UI + Per-Instrument Models + Parakeet
Date: 2026-03-16 Scope: Main Separator, Project pages, Auto-pipeline, backend APIs/services
Decision lock (2026-03-16):
- Parakeet integration must be self-hosted (no hosted trial/API dependency).
- Parakeet uses alternate handling vs WhisperX for unsupported features.
- Parakeet scope includes multilingual transcription support.
Milestone A — Spec + Contract
Status: ✅ Completed (2026-03-16)
A1. Finalize data contracts
- Define
instrument_modelsJSON schema and allowed stem keys - Define lyric engine contract (
engine,model, optional constraints) - Define default/fallback behavior for missing stems and unknown models
- Document backward-compatibility behavior when new fields are absent
Acceptance criteria:
- Team-approved contract doc exists
- Example requests/responses provided for separate, project, and pipeline flows
Milestone B — Backend Foundation
Status: 🟨 In progress (runtime Parakeet E2E verification pending)
B1. Config/constants updates
Target:
- backend/config.py
Tasks:
- Add/organize constants for supported lyric engines/models
- Keep existing model lists stable for current clients
Acceptance criteria:
- Existing model endpoints still return current values
- New engine/model metadata is available to routers/services
B2. Separation API extension
Target:
- backend/routers/separate.py
Tasks:
- Accept optional
instrument_modelsin/api/separateand/api/separate/from-path - Validate schema and return clear 400 errors for invalid payloads
- Preserve existing behavior if field not present
Acceptance criteria:
- Legacy clients succeed without changes
- Invalid mapping is rejected with actionable error details
B3. Separation service orchestration
Target:
- backend/services/separator_service.py
Tasks:
- Implement base separation + optional per-stem refinement pass
- Use existing reprocess logic where possible to avoid duplication
- Persist selected per-instrument mapping in job/project metadata
Acceptance criteria:
- Job result includes applied mapping summary
- Refined stems are discoverable in project outputs
B4. Pipeline API extension
Target:
- backend/routers/pipeline.py
- backend/services/pipeline_service.py (and/or pipeline stages)
Tasks:
- Accept pass-through for
instrument_models, lyricengine, lyricmodel - Keep defaults if new values are omitted
Acceptance criteria:
- Existing pipeline requests still function unchanged
- New fields affect pipeline execution as expected
Notes:
instrument_modelspass-through and result metadata are wired for pipeline.- Lyric
engine/modelpass-through is accepted and surfaced in pipeline job result metadata. - Full lyric-engine stage execution behavior is pending with Parakeet integration (B5).
- Separation now performs optional per-stem refinement and promotes refined outputs as canonical stem entries.
B5. Parakeet engine integration
Target:
- backend/services/lyric_transcriber_service.py
- backend/routers/lyric_transcribe.py
Tasks:
- Add engine dispatch layer (
whisperxandparakeet) - Add self-hosted Parakeet runtime adapter (container/service URL + health check)
- Add Parakeet runner path with multilingual language handling + validation
- Expose engine-aware model listing from lyric model endpoint
- Add alternate handling policy for WhisperX-only options (
diarize,phoneme_align,lsync_align,two_pass)
Acceptance criteria:
- WhisperX path remains fully functional
- Parakeet path runs and returns supported output formats
- Parakeet path supports explicit language selection + auto-detect where available
- Unsupported Parakeet usage returns clear validation errors
- Parakeet requests fail fast with actionable error when self-hosted runtime is unavailable
Current caveat:
- Initial Parakeet sidecar scaffold returns segment-level timing and may emit empty word-level timing (
words=[]).
B6. Project endpoints support
Target:
- backend/routers/projects.py
- backend/services/project_reprocess_service.py
Tasks:
- Allow project transcription endpoints to pass
engine+model - Ensure project job linkage remains intact
Acceptance criteria:
- Project page can trigger Parakeet and WhisperX jobs
- Job records and project references remain correct
Milestone C — Frontend Main Window (Separator)
Status: ✅ Completed (2026-03-16)
C1. Types and API client updates
Target:
- frontend/src/types.ts
- frontend/src/api/client.ts
Tasks:
- Add typing for per-instrument mapping and lyric engine metadata
- Send
instrument_modelson separation submission
Acceptance criteria:
- Type-safe compile without
anyregressions in touched paths - Network payload contains expected new fields
C2. Separator UI panel
Target:
- frontend/src/pages/Separator.tsx
Tasks:
- Add “Per-instrument refinement” section
- Add enable toggle and per-stem selectors
- Add preset/profile quick picks
- Show summary before submit
Acceptance criteria:
- User can configure per-stem models in main window
- Settings persist for current session (or localStorage if chosen)
Milestone D — Frontend Project Page
Status: ✅ Completed (2026-03-16)
D1. Reprocess UX enhancement
Target:
- frontend/src/pages/ProjectDetail.tsx
Tasks:
- Add batch profile apply for stem reprocess
- Keep single-stem reprocess UX intact
Acceptance criteria:
- User can apply multi-stem model profile from project page
- Existing single-stem flow still works
D2. Lyric engine selector on project actions
Target:
- frontend/src/pages/ProjectDetail.tsx
- frontend/src/api/client.ts
Tasks:
- Add engine selector + engine-filtered model selector
- Pass selected engine/model to project lyric transcription requests
Acceptance criteria:
- User can choose WhisperX or Parakeet from project page
- Validation errors display clearly in UI
Milestone E — Auto-Pipeline
Status: ✅ Completed (2026-03-16)
E1. Auto step schema extension
Target:
- frontend/src/utils/autoPipeline.ts
Tasks:
- Add fields for per-instrument mapping and lyric engine/model
- Add migration for old saved steps in localStorage
Acceptance criteria:
- Existing users do not lose current auto-pipeline settings
- Old configs load with sane defaults
E2. Settings editor updates
Target:
- frontend/src/pages/Settings.tsx
Tasks:
- Extend Auto Pipeline step editor to configure new fields
- Update labels/help text to reflect engines and per-instrument separation
Acceptance criteria:
- New fields are editable and persisted
- Preset save/load includes new fields
E3. Auto runner updates
Target:
- frontend/src/components/AutoPipelineDialog.tsx
Tasks:
- Pass per-instrument mapping into separation stage
- Pass lyric engine/model into lyric stage
- Ensure progress text reflects selected engine/model
Acceptance criteria:
- End-to-end auto run uses configured advanced options
- Retry/cancel behavior remains unchanged
Milestone F — Testing + Docs + Rollout
Status: 🟨 In progress
F1. Backend tests
Target:
- backend/tests/*
Tasks:
- Add router/service tests for new payloads and validation
- Add Parakeet engine tests (happy path + invalid constraints)
- Add compatibility tests for old payloads
Acceptance criteria:
- New tests pass in CI
- No regressions in existing tests
Notes:
- Added targeted backend tests for Parakeet engine routing/validation in
backend/tests/test_lyric_engine_parakeet.py. - Updated project lyric transcription test linkage expectation to engine-specific key
in
backend/tests/test_project_lyric_transcribe.py. - Added separation validation coverage for
instrument_modelspayload handling and legacy request compatibility inbackend/tests/test_separate_validation.py. - Added pipeline validation coverage for
instrument_modelsand lyric engine/model payloads inbackend/tests/test_pipeline_validation.py. - Runtime test command executed in backend container (Python 3.11):
unittestsuite for lyric/parakeet/project/separate/pipeline checks passed (16 tests, OK).
F2. Frontend verification
Tasks:
- Add/refresh component and integration tests where available
- Manual QA matrix: main window, project page, auto-pipeline
Acceptance criteria:
- QA checklist completed for all three surfaces
Notes:
- Added QA matrix document:
20260316_QA_Matrix_UI_Models_Parakeet.md - Frontend validation command executed:
npm run check(typecheck + lint + build:local) passed for current changes.
F3. Documentation + release notes
Tasks:
- Update user docs and settings guidance
- Add migration notes for saved auto-pipeline configs
- Add troubleshooting section for Parakeet constraints
Acceptance criteria:
- Docs merged with feature rollout
Suggested PR Breakdown (10-14 PRs)
- PR1: contract/spec + config constants
- PR2: separate router/service payload support
- PR3: pipeline payload support
- PR4: lyric engine abstraction
- PR5: Parakeet runner + validations
- PR6: project endpoint engine support
- PR7: frontend API/types update
- PR8: Separator UI per-instrument panel
- PR9: ProjectDetail reprocess profile + lyric engine UI
- PR10: autoPipeline schema + migration
- PR11: Settings auto editor updates
- PR12: AutoPipelineDialog runner updates
- PR13: backend tests
- PR14: docs + release notes
Definition of Done
- Per-instrument model selection works in main window, project page, and auto-pipeline
- Parakeet English transcription works in all three surfaces
- Backward compatibility preserved for existing requests and saved settings
- Tests and docs updated
Milestone G — MT3 Transcription Integration (New)
Status: ⬜ Not started (planned 2026-03-16)
Plan reference:
20260316_MT3_Integration_Plan.md
G1. MT3 sidecar/runtime
Tasks:
- Add MT3 sidecar profile and health/transcribe endpoints
- Define checkpoint bootstrap/cache strategy
- Validate sidecar startup + health in compose profile
G2. Backend engine adapter
Tasks:
- Add MT3 config/env constants and defaults
- Extend engine dispatch and model catalog payload
- Add runtime unavailable fail-fast behavior and validation messages
G3. Project and pipeline wiring
Tasks:
- Allow project transcription endpoints to accept
engine=mt3 - Add pipeline pass-through handling for MT3 engine/model
- Ensure project job linkage uses engine-specific keys
G4. Frontend controls
Tasks:
- Add MT3 engine option + model selection in project page
- Add MT3 support in auto-pipeline settings/runner
- Gate non-MT3-compatible controls with clear helper text
G5. Testing + rollout
Tasks:
- Add backend tests for MT3 routing/validation/linkage
- Run frontend check + QA matrix for MT3 paths
- Update README troubleshooting and release notes for MT3
Acceptance criteria:
- MT3 is selectable and functional from required UI surfaces
- Runtime failures are clear/actionable
- Existing engines remain unaffected