Files
stemengine_old/20260316_UI_Models_Parakeet_Checklist.md
T
2026-03-16 16:35:08 +09:00

11 KiB

Implementation Checklist: UI + Per-Instrument Models + Parakeet

Date: 2026-03-16 Scope: Main Separator, Project pages, Auto-pipeline, backend APIs/services

Decision lock (2026-03-16):

  • Parakeet integration must be self-hosted (no hosted trial/API dependency).
  • Parakeet uses alternate handling vs WhisperX for unsupported features.
  • Parakeet scope includes multilingual transcription support.

Milestone A — Spec + Contract

Status: ✅ Completed (2026-03-16)

A1. Finalize data contracts

  • Define instrument_models JSON schema and allowed stem keys
  • Define lyric engine contract (engine, model, optional constraints)
  • Define default/fallback behavior for missing stems and unknown models
  • Document backward-compatibility behavior when new fields are absent

Acceptance criteria:

  • Team-approved contract doc exists
  • Example requests/responses provided for separate, project, and pipeline flows

Milestone B — Backend Foundation

Status: 🟨 In progress (runtime Parakeet E2E verification pending)

B1. Config/constants updates

Target:

  • backend/config.py

Tasks:

  • Add/organize constants for supported lyric engines/models
  • Keep existing model lists stable for current clients

Acceptance criteria:

  • Existing model endpoints still return current values
  • New engine/model metadata is available to routers/services

B2. Separation API extension

Target:

  • backend/routers/separate.py

Tasks:

  • Accept optional instrument_models in /api/separate and /api/separate/from-path
  • Validate schema and return clear 400 errors for invalid payloads
  • Preserve existing behavior if field not present

Acceptance criteria:

  • Legacy clients succeed without changes
  • Invalid mapping is rejected with actionable error details

B3. Separation service orchestration

Target:

  • backend/services/separator_service.py

Tasks:

  • Implement base separation + optional per-stem refinement pass
  • Use existing reprocess logic where possible to avoid duplication
  • Persist selected per-instrument mapping in job/project metadata

Acceptance criteria:

  • Job result includes applied mapping summary
  • Refined stems are discoverable in project outputs

B4. Pipeline API extension

Target:

  • backend/routers/pipeline.py
  • backend/services/pipeline_service.py (and/or pipeline stages)

Tasks:

  • Accept pass-through for instrument_models, lyric engine, lyric model
  • Keep defaults if new values are omitted

Acceptance criteria:

  • Existing pipeline requests still function unchanged
  • New fields affect pipeline execution as expected

Notes:

  • instrument_models pass-through and result metadata are wired for pipeline.
  • Lyric engine/model pass-through is accepted and surfaced in pipeline job result metadata.
  • Full lyric-engine stage execution behavior is pending with Parakeet integration (B5).
  • Separation now performs optional per-stem refinement and promotes refined outputs as canonical stem entries.

B5. Parakeet engine integration

Target:

  • backend/services/lyric_transcriber_service.py
  • backend/routers/lyric_transcribe.py

Tasks:

  • Add engine dispatch layer (whisperx and parakeet)
  • Add self-hosted Parakeet runtime adapter (container/service URL + health check)
  • Add Parakeet runner path with multilingual language handling + validation
  • Expose engine-aware model listing from lyric model endpoint
  • Add alternate handling policy for WhisperX-only options (diarize, phoneme_align, lsync_align, two_pass)

Acceptance criteria:

  • WhisperX path remains fully functional
  • Parakeet path runs and returns supported output formats
  • Parakeet path supports explicit language selection + auto-detect where available
  • Unsupported Parakeet usage returns clear validation errors
  • Parakeet requests fail fast with actionable error when self-hosted runtime is unavailable

Current caveat:

  • Initial Parakeet sidecar scaffold returns segment-level timing and may emit empty word-level timing (words=[]).

B6. Project endpoints support

Target:

  • backend/routers/projects.py
  • backend/services/project_reprocess_service.py

Tasks:

  • Allow project transcription endpoints to pass engine + model
  • Ensure project job linkage remains intact

Acceptance criteria:

  • Project page can trigger Parakeet and WhisperX jobs
  • Job records and project references remain correct

Milestone C — Frontend Main Window (Separator)

Status: ✅ Completed (2026-03-16)

C1. Types and API client updates

Target:

  • frontend/src/types.ts
  • frontend/src/api/client.ts

Tasks:

  • Add typing for per-instrument mapping and lyric engine metadata
  • Send instrument_models on separation submission

Acceptance criteria:

  • Type-safe compile without any regressions in touched paths
  • Network payload contains expected new fields

C2. Separator UI panel

Target:

  • frontend/src/pages/Separator.tsx

Tasks:

  • Add “Per-instrument refinement” section
  • Add enable toggle and per-stem selectors
  • Add preset/profile quick picks
  • Show summary before submit

Acceptance criteria:

  • User can configure per-stem models in main window
  • Settings persist for current session (or localStorage if chosen)

Milestone D — Frontend Project Page

Status: ✅ Completed (2026-03-16)

D1. Reprocess UX enhancement

Target:

  • frontend/src/pages/ProjectDetail.tsx

Tasks:

  • Add batch profile apply for stem reprocess
  • Keep single-stem reprocess UX intact

Acceptance criteria:

  • User can apply multi-stem model profile from project page
  • Existing single-stem flow still works

D2. Lyric engine selector on project actions

Target:

  • frontend/src/pages/ProjectDetail.tsx
  • frontend/src/api/client.ts

Tasks:

  • Add engine selector + engine-filtered model selector
  • Pass selected engine/model to project lyric transcription requests

Acceptance criteria:

  • User can choose WhisperX or Parakeet from project page
  • Validation errors display clearly in UI

Milestone E — Auto-Pipeline

Status: ✅ Completed (2026-03-16)

E1. Auto step schema extension

Target:

  • frontend/src/utils/autoPipeline.ts

Tasks:

  • Add fields for per-instrument mapping and lyric engine/model
  • Add migration for old saved steps in localStorage

Acceptance criteria:

  • Existing users do not lose current auto-pipeline settings
  • Old configs load with sane defaults

E2. Settings editor updates

Target:

  • frontend/src/pages/Settings.tsx

Tasks:

  • Extend Auto Pipeline step editor to configure new fields
  • Update labels/help text to reflect engines and per-instrument separation

Acceptance criteria:

  • New fields are editable and persisted
  • Preset save/load includes new fields

E3. Auto runner updates

Target:

  • frontend/src/components/AutoPipelineDialog.tsx

Tasks:

  • Pass per-instrument mapping into separation stage
  • Pass lyric engine/model into lyric stage
  • Ensure progress text reflects selected engine/model

Acceptance criteria:

  • End-to-end auto run uses configured advanced options
  • Retry/cancel behavior remains unchanged

Milestone F — Testing + Docs + Rollout

Status: 🟨 In progress

F1. Backend tests

Target:

  • backend/tests/*

Tasks:

  • Add router/service tests for new payloads and validation
  • Add Parakeet engine tests (happy path + invalid constraints)
  • Add compatibility tests for old payloads

Acceptance criteria:

  • New tests pass in CI
  • No regressions in existing tests

Notes:

  • Added targeted backend tests for Parakeet engine routing/validation in backend/tests/test_lyric_engine_parakeet.py.
  • Updated project lyric transcription test linkage expectation to engine-specific key in backend/tests/test_project_lyric_transcribe.py.
  • Added separation validation coverage for instrument_models payload handling and legacy request compatibility in backend/tests/test_separate_validation.py.
  • Added pipeline validation coverage for instrument_models and lyric engine/model payloads in backend/tests/test_pipeline_validation.py.
  • Runtime test command executed in backend container (Python 3.11): unittest suite for lyric/parakeet/project/separate/pipeline checks passed (16 tests, OK).

F2. Frontend verification

Tasks:

  • Add/refresh component and integration tests where available
  • Manual QA matrix: main window, project page, auto-pipeline

Acceptance criteria:

  • QA checklist completed for all three surfaces

Notes:

  • Added QA matrix document: 20260316_QA_Matrix_UI_Models_Parakeet.md
  • Frontend validation command executed: npm run check (typecheck + lint + build:local) passed for current changes.

F3. Documentation + release notes

Tasks:

  • Update user docs and settings guidance
  • Add migration notes for saved auto-pipeline configs
  • Add troubleshooting section for Parakeet constraints

Acceptance criteria:

  • Docs merged with feature rollout

Suggested PR Breakdown (10-14 PRs)

  • PR1: contract/spec + config constants
  • PR2: separate router/service payload support
  • PR3: pipeline payload support
  • PR4: lyric engine abstraction
  • PR5: Parakeet runner + validations
  • PR6: project endpoint engine support
  • PR7: frontend API/types update
  • PR8: Separator UI per-instrument panel
  • PR9: ProjectDetail reprocess profile + lyric engine UI
  • PR10: autoPipeline schema + migration
  • PR11: Settings auto editor updates
  • PR12: AutoPipelineDialog runner updates
  • PR13: backend tests
  • PR14: docs + release notes

Definition of Done

  • Per-instrument model selection works in main window, project page, and auto-pipeline
  • Parakeet English transcription works in all three surfaces
  • Backward compatibility preserved for existing requests and saved settings
  • Tests and docs updated

Milestone G — MT3 Transcription Integration (New)

Status: ⬜ Not started (planned 2026-03-16)

Plan reference:

  • 20260316_MT3_Integration_Plan.md

G1. MT3 sidecar/runtime

Tasks:

  • Add MT3 sidecar profile and health/transcribe endpoints
  • Define checkpoint bootstrap/cache strategy
  • Validate sidecar startup + health in compose profile

G2. Backend engine adapter

Tasks:

  • Add MT3 config/env constants and defaults
  • Extend engine dispatch and model catalog payload
  • Add runtime unavailable fail-fast behavior and validation messages

G3. Project and pipeline wiring

Tasks:

  • Allow project transcription endpoints to accept engine=mt3
  • Add pipeline pass-through handling for MT3 engine/model
  • Ensure project job linkage uses engine-specific keys

G4. Frontend controls

Tasks:

  • Add MT3 engine option + model selection in project page
  • Add MT3 support in auto-pipeline settings/runner
  • Gate non-MT3-compatible controls with clear helper text

G5. Testing + rollout

Tasks:

  • Add backend tests for MT3 routing/validation/linkage
  • Run frontend check + QA matrix for MT3 paths
  • Update README troubleshooting and release notes for MT3

Acceptance criteria:

  • MT3 is selectable and functional from required UI surfaces
  • Runtime failures are clear/actionable
  • Existing engines remain unaffected