8.4 KiB
8.4 KiB
Visualizer Integration — Implementation Checklist
Date: 2026-04-08
Source plan: 2026-04-08_Visualizer_Integration_Plan.md
0) Non-negotiable color policy
- Use
manifest.metadata.dynamic_paletteas the visualizer color source of truth. (2026-04-08:useVisualizerPaletteprefers manifest palette and is now wired intoLyricsLivePreview+ karaoke pages.) - Do not treat visualizer colors as free/random runtime state when manifest palette is present. (2026-04-08: Muon/Fluid renderers consume
paletteRef; Fluid palette sampling replaces random color source.) - Apply the same palette policy to both
muonandfluidmodes. - Allow audio to modulate motion/intensity only; keep color drift constrained to palette family. (2026-04-08: Muon hue drift limited to seed-hue offset from treble; geometry/velocity driven by audio.)
Acceptance:
- Two clients opening the same
.sekrender matching base visualizer color family. (2026-04-08: SingUI Playwright harnesskaraoke-sek-parity.spec.tsvalidates two clients on/karaokewith the same.sekproduce matching manifest palette seed/highlight.)
1) Backend manifest + API surface
- Ensure
.sekexport writesmetadata.dynamic_paletteintomanifest.json. - Ensure
metadata.dynamic_paletteincludes:sourceseed_colorbackground_colortext_colorhighlight_colorword_highlight_colorspeaker_palette
- Ensure SEK manifest/list endpoints return
manifest.metadata.dynamic_palette. (2026-04-08: list payload now includesmeta_dynamic_palette; manifest endpoint already returns fullmanifest.)
Acceptance:
GETmanifest payload includesmetadata.dynamic_palettefor newly exported packages. (Code path validated: export writes palette; manifest endpoint returns full manifest object.)
2) Frontend types and palette resolver
- Add/verify
dynamic_palettetyping in SEK manifest response model. (2026-04-08: added sharedSekDynamicPalettetype; applied to bothSekManifestResponse.manifest.metadata.dynamic_paletteandSekLibraryItem.meta_dynamic_palette.) - Implement
useVisualizerPalettehook:- Input:
sekManifest?.manifest?.metadata?.dynamic_palette,coverArtUrl,thumbnailUrl - Output: stable
paletteRef
- Input:
- Fallback chain:
- Manifest palette
- Runtime extraction from cover/thumbnail
- Static neutral defaults
Acceptance:
- Missing manifest palette does not break rendering. (hook falls back to runtime extraction/default palette)
- Runtime fallback only used when manifest palette is absent.
3) Audio feature layer
- Implement/verify
useAudioFeatureswith analyser + Meyda. (2026-04-08: hook added with Meyda + FFT fallback path.) - Keep feature transport in
featuresRef(no 60fps React state updates). - Expose at least:
bass,treble,rms,beatDetected,drumIntensity. (also exposesenergy,loudness,spectralCentroid,freqData)
Acceptance:
- No double-audio output. (analyser tap is routed through silent gain node)
- Cleanup releases analyser/Meyda resources. (disconnect + analyzer stop implemented in hook cleanup)
4) Muon visualizer integration
- Add
paletteRefinput toMuonVisualizerCanvas. - Map
drumIntensityto geometry controls (radiusMultiplier,aperture,spacing). - Seed color from manifest palette (
seed_color/highlight_color). - Restrict hue drift to small offset around seed (treble-driven).
- Keep fallback to defaults if palette unavailable.
Acceptance:
- Muon keeps package-consistent color identity while reacting to audio.
5) Fluid visualizer integration
- Add
paletteRefinput toFluidVisualizerCanvas. - Replace random color generation with palette sampling (
speaker_palette,highlight_color,word_highlight_color). - Keep beat/rms driving splat force and density behavior.
Acceptance:
- Fluid colors are deterministic per package, not random per session.
6) Backdrop switcher + LyricsLivePreview wiring
- Add
paletteReftoVisualizerBackdropprops and forward to both modes. - Add
useVisualizerPalettecall inLyricsLivePreview. (2026-04-08: hook now invoked inLyricsLivePreviewwith optionalsekDynamicPalette+ cover/thumbnail fallbacks.) - Ensure backdrop renders under lyrics layer (
z-indexorder preserved). - Extend style union with
muon | fluidand validate guards. (2026-04-08:LyricsVisualStyle+ saved-style allowlist updated inLyricsLivePreview.) - Add style-picker options for
MuonandFluid. (2026-04-08: added renderer selector buttons in style panel: Karaoke/Video/Muon/Fluid.)
Acceptance:
- Switching styles does not reset playback or break lyric rendering. (2026-04-08: Playwright regression
lyrics-live-preview.spec.tsverifies playback continues and lyrics remain visible while switching Karaoke → Muon → Fluid.)
7) Visualizer settings persistence
- Keep
manifest.visualizerfor mode + geometry/performance settings. (2026-04-08: SEK export now writes normalizedmanifest.visualizerfrom karaoke config with mode/quality/fps and muon/fluid geometry/intensity controls.) - Keep color data in
manifest.metadata.dynamic_palette. (No visual color fields were added tomanifest.visualizer; colors remain inmetadata.dynamic_paletteonly.) - Avoid duplicate/conflicting color settings in
manifest.visualizer.
Acceptance:
- Package stores one authoritative color definition path.
8) Performance and safety checks
- Lazy-load heavy visualizer bundles where possible. (2026-04-08:
VisualizerBackdropnow lazy-loads Muon/Fluid canvases withReact.lazy+Suspense.) - Confirm GPU/CPU use remains acceptable in fullscreen and windowed modes. (2026-04-08: Playwright frame-cadence smoke test verifies healthy RAF cadence in both windowed and fullscreen visualizer preview.)
- Verify graceful behavior when WebGL is unavailable. (Current visualizers are Canvas2D-based and bail out cleanly if context init fails.)
Acceptance:
- No regressions in non-visualizer lyric modes. (2026-04-08: preview regression starts in standard live-preview flow, then switches renderers while transport/lyrics continue without reset.)
9) QA matrix
.sekwith fulldynamic_palette→ Muon colors match manifest. (SingUI Playwright harness:/karaokeassertsdata-palette-source=manifest, seed#123456in Muon mode.).sekwith fulldynamic_palette→ Fluid colors match manifest. (SingUI Playwright harness:/karaokeassertsdata-palette-source=manifest, seed#123456in Fluid mode.)- Legacy
.sek(no palette) → runtime fallback path works. (SingUI Playwright harness: legacy-with-cover resolvesruntime-artwork; no-cover resolvesdefault.) - No-cover/no-thumbnail project → static default fallback works. (2026-04-08: Playwright regression verifies
data-palette-source=defaultand default seed color when cover art is unavailable.) - TV/performance mode renders same color policy as standard mode. (SingUI Playwright harness:
/karaoke/tvparity assertion matches/karaokepalette source/seed/highlight for the same.sek.)
10) Definition of done
- Backend emits and serves
manifest.metadata.dynamic_palette. - Frontend consumes palette via
paletteRefin both visualizers. - Color behavior is deterministic per package.
- Audio reactivity remains smooth and independent from color source policy. (Playback continuity test under renderer switching passes; SEK-route parity tests confirm identical color policy behavior independent from audio-driven mode dynamics.)
- Checklist and implementation notes are committed.
Validation evidence captured:
- Frontend typecheck passes. (
npm run typecheck) - Backend regression passes. (
python3 -m unittest backend.tests.test_karaoke_sek_manifest_export -v) - Focused Playwright regressions pass. (
npx playwright test e2e/lyrics-live-preview.spec.ts --grep "switching renderers keeps playback running|artwork palette stays consistent across clients and renderers|visualizer palette falls back to static defaults without artwork") - SingUI
.sekkaraoke/TV parity harness passes. (npx playwright test -c playwright.sing.config.ts e2e/karaoke-sek-parity.spec.ts)