Myriad changes.

This commit is contained in:
translator committed 2026-04-09 08:30:51 +09:00
1 parent 9281c789be
commit cbe946925f
21 files changed
+2762 -44

No files matched your search

+624
View File
@@ -0,0 +1,624 @@
# Performance Sidebar Plan
Date: 2026-04-08
Scope: A slide-in sidebar available in both KaraokePerformanceMode (TV view) and fullscreen LyricsLivePreview that exposes background/visualizer selection, stem mixing, and lyrics appearance controls. Leading vocal stems are muted by default.
Status: ✅ Reviewed, corrected, implemented, and documented (2026-04-08).
---
## Review outcomes (issues corrected)
1. Removed architecture contradiction: the diagram previously showed a persistent trigger tab, but the design is keyboard-invoked only.
2. Corrected implementation step numbering (`### 4` → `### 3`) in the `LyricsLivePreview.tsx` change list.
3. Tightened `VizSettings.fps_cap` to `30 | 60 | 120` for stronger typing consistency with UI controls.
4. Updated implementation notes to reflect actual files changed and completed validation pass.
---
## Current state
In `performanceMode`, `LyricsLivePreview` deliberately hides:
- The universal toolbar (`!performanceMode`)
- The style picker panel (`!performanceMode && showStylePicker`)
- The stem mixer (`!performanceMode && renderStems()`)
`KaraokePerformanceMode` has a minimal HUD (auto-hides after 2.2 s showing "Press Esc to exit") and keyboard shortcuts only. There is no way for the user to change the visualizer, adjust stems, or tweak lyrics appearance once performance mode begins.
---
## Architecture
All relevant state (`visualStyle`, `mutedStems`, `soloedStems`, `gains`, `style`) already lives inside `LyricsLivePreview`. The sidebar is built as a sub-panel inside that component — no state needs to be lifted.
```
KaraokePerformanceMode
└── LyricsLivePreview (performanceMode=true)
├── [existing] lyrics canvas / visualizer backdrop (z-index 1–2)
├── [existing] metadata card — top-left (z-index 7)
└── [NEW] PerformanceSidebar — drawer/sheet (z-index 9, shown on demand via `Q`)
```
---
## New files
```
frontend/src/components/
└── PerformanceSidebar.tsx # right-side drawer with four sections
```
No new hooks needed — the sidebar reads/writes state that already exists in `LyricsLivePreview`.
---
## Sidebar trigger
The sidebar is **shown on demand only**. On desktop/wide layouts it is keyboard-invoked; on narrow/mobile layouts it also has a floating top-right button so touch users can open it without a hardware keyboard.
**Keyboard shortcut**: `Q` key while in performance mode opens the sidebar. If the sidebar is already open, `Q` closes it.
`Q` is currently unused in performance mode (Esc exits, Space/Enter toggles play, Arrow keys seek). `KaraokePerformanceMode` does not intercept `Q`, so there is no conflict.
**Mobile / narrow screens:** when `isNarrow` is true, a floating circular button is rendered in the top-right corner and toggles the sidebar open/closed.
**Auto-hide**: The sidebar auto-hides after **4 seconds** of inactivity (no pointer movement over the sidebar panel). Any mouse movement within the sidebar resets the timer. This mirrors the HUD auto-hide pattern already used by `KaraokePerformanceMode`. The timer is cleared when the sidebar is closed manually.
```typescript
// Inside PerformanceSidebar, on mouse movement over the panel:
clearTimeout(autoHideTimer)
autoHideTimer = setTimeout(() => onClose(), 4000)
```
The sidebar does not show a persistent edge tab. On desktop it remains keyboard-first; on mobile it uses the floating top-right button.
---
## `PerformanceSidebar` component
```tsx
interface PerformanceSidebarProps {
// Background
visualStyle: LyricsVisualStyle
setVisualStyle: (s: LyricsVisualStyle) => void
hasVideo: boolean
// Stems
stems: Stem[] | undefined
mutedStems: Record<string, boolean>
soloedStems: Record<string, boolean>
gains: Record<string, number>
toggleStemMute: (name: string) => void
toggleStemSolo: (name: string) => void
setStemGain: (name: string, value: number) => void
// Lyrics style
style: StyleState
patchStyle: (patch: Partial<StyleState>) => void
// Visualizer controls
vizSettings: VizSettings
manifestVizSettings: VizSettings | null // null when no .sek or no visualizer block
patchVizSettings: (patch: DeepPartial<VizSettings>) => void
// Lifecycle
onClose: () => void
}
```
**`VizSettings` type** (defined in `LyricsLivePreview.tsx` alongside `StyleState`):
```typescript
interface VizSettings {
quality: 'low' | 'balanced' | 'high'
fps_cap: 30 | 60 | 120
muon: {
radius_multiplier: number
aperture: number
spacing: number
}
fluid: {
spawn_density: number
splat_force: number
decay: number
}
}
const VIZ_DEFAULTS: VizSettings = {
quality: 'balanced',
fps_cap: 60,
muon: { radius_multiplier: 1.0, aperture: 1.0, spacing: 1.0 },
fluid: { spawn_density: 1.0, splat_force: 1.0, decay: 1.0 },
}
```
`manifestVizSettings` is built once from the `sekVisualizerSettings` prop merged over `VIZ_DEFAULTS`, or `null` when `sekVisualizerSettings` is absent.
The component renders as an `absolute right-0 top-0 h-full` panel, width ~280 px (`w-70`), with `backdrop-blur` and `bg-black/70`. Four collapsible sections with chevron toggles, open by default.
---
## Section 1 — Background
Replaces the renderer buttons currently hidden behind `!performanceMode`.
**Controls:**
| Label | Action |
|---|---|
| **Static** | `setVisualStyle('karaoke')` — current karaoke renderer, no backdrop |
| **Muon** | `setVisualStyle('muon')` — spiral particle visualizer |
| **Fluid** | `setVisualStyle('fluid')` — WebGL fluid sim |
| **Video** | `setVisualStyle('video')` — only shown when `hasVideo` is true |
Rendered as a 2×2 icon grid (or horizontal pill row on narrow screens). Active mode has a highlight border.
> No "scroll", "fade", "glow", or "blur" options here — those are editing modes not suited to performance view. The sidebar only exposes the four modes appropriate for live display.
---
## Section 2 — Stems
Shows per-stem mixing controls when `isMultiStem` is true; hidden (or shows a "no stems" note) otherwise.
**Controls per stem:**
- **Label** — stem display name (`stem.label`), capitalized
- **M button** — mute toggle (red when active)
- **S button** — solo toggle (yellow when active)
- **Gain slider** — 0–200%, default 100%, step 5%
This is a direct port of the existing `renderStems()` output, styled for the sidebar width.
### Leading vocal default mute
When `performanceMode` is true and the component mounts with **no saved `mutedStems` preferences** for this `storageKey`, auto-mute stems whose name matches the leading-vocal pattern:
```typescript
const LEADING_VOCAL_PATTERN = /^(lead|lead\s|leading|vocals?|vox|voice|main\s*vocal)/i
function isLeadingVocalStem(name: string): boolean {
return LEADING_VOCAL_PATTERN.test(name.trim())
}
```
Applied once in a `useEffect` inside `LyricsLivePreview`:
```typescript
useEffect(() => {
if (!performanceMode || !stems?.length) return
// Only apply if no mute state has been saved for this session
const hasSavedPrefs = Object.keys(initialPreviewPrefs.mutedStems ?? {}).length > 0
if (hasSavedPrefs) return
const defaultMutes: Record<string, boolean> = {}
for (const stem of stems) {
if (isLeadingVocalStem(stem.name)) defaultMutes[stem.name] = true
}
if (Object.keys(defaultMutes).length > 0) {
setMutedStems(defaultMutes)
applyGains({ mutedStems: defaultMutes })
}
}, [performanceMode]) // run once on mount — dep on performanceMode ensures it's stable
```
This runs only when there's nothing saved. Once the user adjusts stems, their preference is persisted to localStorage and restored next session, so the default only applies to first use.
---
## Section 3 — Lyrics
A condensed version of `StylePicker` containing only the controls most useful during a live performance. Excludes video display mode controls, layout offset sliders, and speaker colour assignments (too granular for live adjustment).
**Controls:**
| Control | StyleState key | Widget |
|---|---|---|
| Font | `fontFamily` | `<select>` (same `FONT_OPTIONS` list) |
| Size | `fontSize` | Range slider 12–56 |
| Bold | `fontBold` | Toggle button |
| Align | `textAlign` | Left / Center / Right icon buttons |
| Position | `activeLineVerticalAlign` | Top / Center / Bottom icon buttons |
| Line spacing | `lineSpacing` | Range slider |
| Text colour | `textColor` | `<input type="color">` |
| Highlight colour | `highlightColor` | `<input type="color">` |
| Background | `bgColor` | `<input type="color">` (has no effect in muon/fluid modes where the canvas sets its own background) |
| X offset | `layoutOffsetXScroll` | Range slider −50–50 |
| Y offset | `layoutOffsetYScroll` | Range slider −50–50 |
The offset sliders automatically route to the correct offset key based on `visualStyle` (scroll vs line vs word vs syllable), same as `StylePicker.applyLayoutOffset`.
---
## Section 4 — Visualizer controls
Always shown in the sidebar. Common preset controls (Quality and FPS cap) are always available; mode-specific sliders are shown only when `visualStyle === 'muon'` or `visualStyle === 'fluid'`.
Displays controls for the **currently active visualizer** only — switching from Muon to Fluid swaps the control set in place. Common controls (Quality, FPS cap) are shown at the top for both as explicit preset buttons.
A **Reset** button appears at the bottom of this section. Its label and behaviour depend on whether manifest values exist:
| Condition | Button label | Action |
|---|---|---|
| `manifestVizSettings !== null` | "Reset to song defaults" | `patchVizSettings(manifestVizSettings)` |
| `manifestVizSettings === null` | "Reset to defaults" | `patchVizSettings(VIZ_DEFAULTS)` |
### Common controls (Muon and Fluid)
| Label | Key | Widget | Range / Options |
|---|---|---|---|
| Quality | `quality` | Segmented preset buttons | Low / Balanced / High |
| FPS cap | `fps_cap` | Segmented preset buttons | 30 / 60 / 120 |
### Muon-specific controls
| Label | Key | Widget | Range | Step | Default |
|---|---|---|---|---|---|
| Radius | `muon.radius_multiplier` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
| Aperture | `muon.aperture` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
| Spacing | `muon.spacing` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
### Fluid-specific controls
| Label | Key | Widget | Range | Step | Default |
|---|---|---|---|---|---|
| Density | `fluid.spawn_density` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
| Force | `fluid.splat_force` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
| Trail decay | `fluid.decay` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
Slider labels show the current numeric value to two decimal places (e.g. "1.20×"). All values are multipliers — 1.0 means unmodified default behaviour as driven by audio.
---
## Changes to `LyricsLivePreview.tsx`
### 1. Add sidebar state
```typescript
const [showPerfSidebar, setShowPerfSidebar] = useState(false)
```
Add live visualizer settings state (seeded from `sekVisualizerSettings`, falls back to `VIZ_DEFAULTS`):
```typescript
const manifestVizSettings = useMemo<VizSettings | null>(() => {
if (!sekVisualizerSettings) return null
return deepMerge(VIZ_DEFAULTS, sekVisualizerSettings)
}, [sekVisualizerSettings])
const [liveVizSettings, setLiveVizSettings] = useState<VizSettings>(
() => manifestVizSettings ?? VIZ_DEFAULTS
)
const patchVizSettings = useCallback((patch: DeepPartial<VizSettings>) => {
setLiveVizSettings(prev => deepMerge(prev, patch))
}, [])
```
`deepMerge` is a small local helper (two-level object merge — no third-party dep needed given the shallow shape of `VizSettings`).
`liveVizSettings` is passed to `VisualizerBackdrop` (replacing the current `sekVisualizerSettings` prop there) so live adjustments take effect immediately.
### 2. Add `Q` key handler
In the existing `keydown` listener (or a new one):
```typescript
if (event.key === 'q' && isPerformanceView) {
event.preventDefault()
setShowPerfSidebar(p => !p)
}
```
No trigger tab is rendered — the sidebar is keyboard-invoked only.
### 3. Render `PerformanceSidebar`
```tsx
{isPerformanceView && showPerfSidebar && (
<PerformanceSidebar
visualStyle={visualStyle}
setVisualStyle={setVisualStyle}
hasVideo={!!(videoUrl || resolvedVideoPlaylist.length > 0)}
stems={stems}
mutedStems={mutedStems}
soloedStems={soloedStems}
gains={gains}
toggleStemMute={toggleStemMute}
toggleStemSolo={toggleStemSolo}
setStemGain={(name, val) => {
setGains(prev => ({ ...prev, [name]: val }))
applyGains({ name, gain: val })
}}
style={style}
patchStyle={patchStyle}
vizSettings={liveVizSettings}
manifestVizSettings={manifestVizSettings}
patchVizSettings={patchVizSettings}
onClose={() => setShowPerfSidebar(false)}
/>
)}
```
### 5. Leading vocal default mute (see Section 2 above)
### 6. Viewport resize — exclude sidebar width
The existing `useEffect` that computes `containerWidth`/`containerHeight` and `visualizerSize` already depends on `[isFullscreen, performanceMode, showStylePicker, showEditor, showAegisub]`. Add `showPerfSidebar` to this dep array, and subtract the sidebar width (280 px) from the container width calculation when the sidebar is open.
---
## Changes to `KaraokePerformanceMode.tsx`
No structural changes needed — the sidebar and its trigger live entirely inside `LyricsLivePreview`. However:
1. **`Q` key** — Currently no handler in `KaraokePerformanceMode`. The sidebar's `Q` toggle handler will be inside `LyricsLivePreview`, which receives keyboard events through its own listener. `KaraokePerformanceMode` does not intercept `Q`, so no conflict.
2. **HUD position** — The existing HUD ("Press Esc to exit") is `top-4 left-1/2`. The sidebar trigger is on the right edge at vertical center. No overlap.
3. **`upNextSong` / `backgroundLabel` / `profileName`** props — these are display-only and unaffected.
---
## `PerformanceSidebar.tsx` layout
```
┌──────────────────────────────────────────┐
│ Controls [✕] │ ← header
├──────────────────────────────────────────┤
│ ▾ BACKGROUND │
│ [Static] [Muon] [Fluid] [Video?] │
├──────────────────────────────────────────┤
│ ▾ STEMS │
│ Vocals [M] [S] ████░░░░ 85% │
│ Drums [M] [S] ████████ 100% │
│ Bass [M] [S] ████████ 100% │
│ Other [M] [S] ████████ 100% │
├──────────────────────────────────────────┤
│ ▾ LYRICS │
│ Font [System ▾] [B] │
│ Size ████░░░░░ 20 │
│ Align [⬛][⬛][⬛] Pos [T][C][B] │
│ Spacing ████░░░░ 1.4 │
│ Text ■ Highlight ■ BG ■ │
│ X offset ────┬── 0 │
│ Y offset ───┬─── 0 │
├──────────────────────────────────────────┤
│ ▾ VISUALIZER │
│ Quality [Low] [Balanced] [High] │
│ FPS cap [30] [60] [120] │
│ ─── Muon ───────────────────────────── │
│ Radius ████░░░░ 1.00× │
│ Aperture ████░░░░ 1.00× │
│ Spacing ████░░░░ 1.00× │
│ [Reset to song defaults] │
└──────────────────────────────────────────┘
```
(Fluid variant shows Density / Force / Trail decay in place of the Muon rows.)
Width: 280 px on desktop, 100 vw on narrow (`isNarrow`) screens (slides up from bottom as a sheet instead).
---
## Edge cases
| Scenario | Behaviour |
|---|---|
| No stems (single-track, non-SEK) | Section 2 shows "No stems — single track mode" note |
| Video mode selected but `hasVideo` is false | Video button not rendered; if `visualStyle === 'video'` arrives via prop, show Static instead |
| All stems soloed | Solo button cleared for all (solo clears itself when all are soloed — matches existing toggle logic) |
| `isNarrow` (mobile/TV portrait) | Sidebar renders as a bottom sheet, full width, sections scroll vertically |
| Sidebar auto-hide fires | `onClose()` is called from within `PerformanceSidebar` after 4 s of no pointer activity |
| User exits performance mode (Esc) | `showPerfSidebar` resets to false on next mount |
| Sidebar open + visualizer canvas | Canvas width calculation subtracts sidebar width so visualizer doesn't render behind the panel |
| `storageKey` absent | Leading vocal default mute still applies; prefs just won't persist across sessions |
| `sekVisualizerMode` prop | See Bug 1a below — enforcement is now user-override-aware |
| No manifest.json / no `sekVisualizerSettings` | `manifestVizSettings` is `null`; Reset button label is "Reset to defaults"; restores `VIZ_DEFAULTS` |
| `sekVisualizerSettings` present but missing sub-keys | `manifestVizSettings` fills missing keys from `VIZ_DEFAULTS` via `deepMerge`; Reset always restores a complete `VizSettings` object |
| User switches visualizer mode mid-session | Section 4 swaps the Muon/Fluid rows in place; shared controls (Quality, FPS cap) remain; slider values for the hidden mode are preserved in `liveVizSettings` |
| `liveVizSettings` changes | Passed directly to `VisualizerBackdrop`; changes take effect on the next animation frame without remounting the canvas |
---
## Bug fixes — reported 2026-04-08
### Bug 1a — Sidebar visualizer buttons (Fluid, Muon, Video) don't switch modes ✅ IMPLEMENTED
**File:** `LyricsLivePreview.tsx`
**Root cause:** `enforcedVisualizerMode` is derived from `sekVisualizerMode` or `defaultStyle`:
```typescript
const enforcedVisualizerMode: LyricsVisualStyle | null =
(sekVisualizerMode === 'muon' || sekVisualizerMode === 'fluid')
? sekVisualizerMode
: ((defaultStyle === 'muon' || defaultStyle === 'fluid') ? defaultStyle : null)
```
A `useEffect` at lines 1545–1549 watches both `enforcedVisualizerMode` **and `visualStyle`** in its dep array:
```typescript
useEffect(() => {
if (!enforcedVisualizerMode) return
if (visualStyle === enforcedVisualizerMode) return
setVisualStyle(enforcedVisualizerMode) // ← immediately reverts sidebar choice
}, [enforcedVisualizerMode, visualStyle])
```
When the sidebar calls `setVisualStyle('fluid')`, React re-renders, the effect fires again (because `visualStyle` is in deps), sees `visualStyle !== enforcedVisualizerMode`, and immediately reverts it back. The user's choice never sticks.
**Fix:** Track whether the user has manually chosen a mode this session with a ref. Skip enforcement if they have:
```typescript
const userOverrodeVisualizerRef = useRef(false)
// Pass to PerformanceSidebar instead of raw setVisualStyle:
const setVisualStyleFromSidebar = useCallback((s: LyricsVisualStyle) => {
userOverrodeVisualizerRef.current = true
setVisualStyle(s)
}, [])
// Updated enforcement effect — remove visualStyle from dep array:
useEffect(() => {
if (!enforcedVisualizerMode) return
if (userOverrodeVisualizerRef.current) return // user explicitly chose something else
setVisualStyle(enforcedVisualizerMode)
}, [enforcedVisualizerMode]) // ← no visualStyle dep; runs once on mount and on song change
```
Reset `userOverrodeVisualizerRef.current = false` when `storageKey` / `previewPrefsKey` changes (i.e. new song loaded), so enforcement re-applies on the next song.
**Change in `PerformanceSidebar`:** Replace `setVisualStyle` prop usage with `setVisualStyleFromSidebar`. No interface change needed — it has the same signature.
---
### Bug 1b — Video mode: video doesn't start when switching mid-song ✅ IMPLEMENTED
**File:** `LyricsLivePreview.tsx`
**Root cause:** The `<video>` element is conditionally rendered only when `hasVideoBackdrop = visualStyle === 'video' && !!currentVideoUrl`. When `visualStyle` switches to `'video'` while the song is already playing, the video element mounts fresh but is paused. `togglePlay` is not called, so `videoRef.current.play()` and `setVideoStartPhase('playing')` never fire. The audio continues; the video stays black/paused.
**Fix:** Add a `useEffect` that detects the transition to active video mode while already playing, and starts the video at the current transport position:
```typescript
useEffect(() => {
if (!hasVideoBackdrop || !isPlaying || !videoRef.current) return
if (videoStartPhase === 'playing') return // already running
const t = currentTimeRef.current
if (isPlaylistBackdropMode) {
syncPlaylistVideoToTransport(t)
} else {
videoRef.current.currentTime = t
}
videoRef.current.play().catch((err) =>
console.warn('[LyricsLivePreview] video auto-start on mode switch failed', err)
)
setVideoStartPhase('playing')
}, [hasVideoBackdrop, isPlaying])
// Intentionally excludes videoStartPhase from deps — effect should re-run when the
// video element mounts (hasVideoBackdrop transitions false→true), not on phase changes.
```
---
### Bug 2 — Seek slider snaps back to the beginning after dragging ✅ IMPLEMENTED
**File:** `LyricsLivePreview.tsx`
**Root cause:** The RAF loop (inside the `isPlaying` effect, lines 2157–2251) runs every animation frame while the song is playing. It reads `audio.currentTime` from the DOM audio element and calls `setCurrentTime(t)`. After a drag-seek:
1. `endSeekDrag` calls `applySeekPosition(120)` — sets `audio.currentTime = 120`, `currentTimeRef.current = 120`, `setCurrentTime(120)`.
2. `endSeekDrag` immediately sets `isSeekingRef.current = false`.
3. `endSeekDrag` queues `setTimeout(0)` to clear `seekDragTime`.
4. **Before that timeout fires**, the RAF fires. `isSeekingRef.current = false`, so the RAF is unguarded.
5. The RAF reads `audio.currentTime`. The browser's seek hasn't completed yet — `audio.currentTime` still returns the pre-seek value (often near 0 for a just-loaded position).
6. `currentTimeRef.current = 0` (overwritten by RAF).
7. `setCurrentTime(0)` (state set to 0).
8. The `setTimeout(0)` fires: `setSeekDragTime(null)`.
9. Input value: `seekDragTime ?? currentTime = null ?? 0 = 0`. Slider snaps to 0.
The `isSeekingRef` guard was supposed to prevent step 4–7, but it is cleared synchronously in `endSeekDrag` before the browser's `seeked` event has fired.
**Fix:** Do not clear `isSeekingRef.current` synchronously in `endSeekDrag`. Instead, attach a one-shot `seeked` listener to the relevant audio element, which clears it once the browser confirms the seek has settled. Add a 500 ms safety timeout in case `seeked` never fires (e.g. no audio element present):
```typescript
function endSeekDrag(e?: React.PointerEvent<HTMLInputElement> | React.FocusEvent<HTMLInputElement>) {
const isBlur = e?.type === 'blur'
if (!isBlur) {
const maybeValue = e ? parseFloat((e.currentTarget as HTMLInputElement).value) : Number.NaN
const finalValue = Number.isFinite(maybeValue) ? maybeValue : pendingSeekRef.current
if (typeof finalValue === 'number' && Number.isFinite(finalValue)) {
pendingSeekRef.current = finalValue
applySeekPosition(finalValue)
}
}
// DO NOT clear isSeekingRef immediately — wait for audio to confirm the seek.
// The RAF is guarded by isSeekingRef and must not read stale audio.currentTime
// between the seek call and the browser's 'seeked' event firing.
const targetAudio = isMultiStem
? (Object.values(stemAudioRefs.current)[0] ?? null)
: audioRef.current
if (targetAudio) {
const clearLock = () => { isSeekingRef.current = false }
targetAudio.addEventListener('seeked', clearLock, { once: true })
seekLockFallbackRef.current = window.setTimeout(clearLock, 500)
} else {
isSeekingRef.current = false
}
if (clearSeekUiTimerRef.current !== null) window.clearTimeout(clearSeekUiTimerRef.current)
clearSeekUiTimerRef.current = window.setTimeout(() => {
setSeekDragTime(null)
clearSeekUiTimerRef.current = null
}, 0)
}
```
Add `seekLockFallbackRef = useRef<number | null>(null)` alongside the other refs. Clear it in cleanup / `handleReset`.
**Also fix `beginSeekDrag`**: the same pattern applies — when drag starts, `isSeekingRef.current = true` (already done). The fallback timer from any prior drag is cleared on `beginSeekDrag` to prevent a stale `clearLock` from firing mid-drag:
```typescript
function beginSeekDrag() {
if (seekLockFallbackRef.current !== null) {
window.clearTimeout(seekLockFallbackRef.current)
seekLockFallbackRef.current = null
}
// ... existing logic ...
isSeekingRef.current = true
}
```
**Implementation status (2026-04-08):** Implemented in `LyricsLivePreview.tsx`:
- added `seekLockFallbackRef`;
- cleared stale fallback timer on `beginSeekDrag`;
- changed `endSeekDrag` to release seek lock on `audio.seeked` (with 500 ms fallback timer);
- cleared fallback timer in `stopPlayback`, `finalizePlayback`, and `handleReset`.
---
## Implementation completed (2026-04-08)
### Files changed
- `frontend/src/components/LyricsLivePreview.tsx`
- Added `showPerfSidebar` state and keyboard toggle (`Q`) for performance view.
- Added a floating top-right narrow-screen button to open the sidebar when closed.
- Added typed visualizer settings state (`VizSettings`, `VIZ_DEFAULTS`, merge + normalization helpers).
- Replaced direct `sekVisualizerSettings` pass-through with live `liveVizSettings` to `VisualizerBackdrop`.
- Added first-use leading-vocal default mute logic for performance mode when no saved mute preferences exist.
- Added viewport sizing exclusion for open desktop sidebar (subtracts 280 px from visualizer width).
- Wired `PerformanceSidebar` overlay render in performance/fullscreen view.
- Implemented seek-lock timing fix for drag seek (`seekLockFallbackRef` + `seeked`-gated unlock) to prevent slider snap-back.
- `frontend/src/components/PerformanceSidebar.tsx` (new)
- Implemented four collapsible sections: Background, Stems, Lyrics, Visualizer.
- Added 4-second auto-hide timer reset on pointer movement within the panel.
- Added mode-aware visualizer controls, common preset buttons (Quality/FPS), and reset behavior (`song defaults` vs `global defaults`).
- Added responsive rendering: right drawer on wide viewports, bottom sheet on narrow viewports.
### Validation
- TypeScript diagnostics for both changed frontend files report no errors.
- Manual runtime QA for Bug 2 behavior remains pending (seek drag while actively playing).
---
## Implementation order
| Step | File | Notes |
|---|---|---|
| 1 | `LyricsLivePreview.tsx` | Add `showPerfSidebar` state, Q key handler; add `VizSettings` type, `VIZ_DEFAULTS`, `liveVizSettings` state, `patchVizSettings`, `manifestVizSettings` memo |
| 2 | `LyricsLivePreview.tsx` | Pass `liveVizSettings` to `VisualizerBackdrop` (replaces direct `sekVisualizerSettings` pass-through) |
| 3 | `LyricsLivePreview.tsx` | Add leading vocal default-mute effect |
| 4 | `PerformanceSidebar.tsx` | Create component — Background section only |
| 5 | `PerformanceSidebar.tsx` | Add Stems section (port from `renderStems`) |
| 6 | `PerformanceSidebar.tsx` | Add Lyrics section (subset of `StylePicker`) |
| 7 | `PerformanceSidebar.tsx` | Add Visualizer controls section (Section 4) with Reset button |
| 8 | `LyricsLivePreview.tsx` | Wire `PerformanceSidebar` render + viewport exclusion + new viz props |
| 9 | `LyricsLivePreview.tsx` | **Bug 1a fix** — add `userOverrodeVisualizerRef`, `setVisualStyleFromSidebar`, update `enforcedVisualizerMode` effect to remove `visualStyle` dep; pass new setter to `PerformanceSidebar` |
| 10 | `LyricsLivePreview.tsx` | **Bug 1b fix** — add `useEffect` to start video when `hasVideoBackdrop && isPlaying` and `videoStartPhase !== 'playing'` |
| 11 | `LyricsLivePreview.tsx` | **Bug 2 fix** — add `seekLockFallbackRef`; update `beginSeekDrag` to cancel prior fallback; update `endSeekDrag` to defer `isSeekingRef = false` until `audio.seeked` fires |
| 12 | Manual test | Visualizer switches persist; video starts on mode switch; seek slider holds position; reset restores correctly |
| 13 | `2026-04-08_PerformanceSidebar_Plan.md` | Document implementation status and validation |
+461
View File
@@ -0,0 +1,461 @@
# Visualizer Controls Audit — Sidebar Parameters & Presets
Date: 2026-04-08
Scope: Full audit of the runtime animation loop for both `MuonVisualizerCanvas.tsx` and
`FluidVisualizerCanvas.tsx`. Documents every expression that changes during playback, identifies
which values are hardcoded and tunable, proposes new manifest keys and sidebar controls, and
defines named presets for each visualizer.
---
## Part 1 — Muon: what changes during playback
Source: `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
### Input signals (per-frame)
| Signal | Source | Range | Description |
|---|---|---|---|
| `drum` | `f.drumIntensity` (clamped) | 0–1 | Primary reactive input. Weighted bass+rms+treble exponential scaler. |
| `treble` | `f.treble` (clamped) | 0–1 | High-frequency energy. Secondary hue influence. |
| `idle` | `0.5 + 0.5 * sin(now * 0.0012)` | 0–1 | Slow sinusoidal pulse (~5.2 s period) used when audio is quiet. |
| `beatDetected` | `f.beatDetected` | bool | Triggers preset advance when `true` and >200 ms since last advance. |
### Computed values (all hardcoded coefficients)
#### Radius system
```
RADIUS_PRESETS = [0.90, 0.95, 1.00, 1.05, 1.10, 0.85, 1.15, 1.02] (8 steps)
radiusTarget = RADIUS_PRESETS[presetIdx] + drum * 0.25 (on beat)
radiusCurrent += (radiusTarget - radiusCurrent) * 0.07 (lerp speed)
radiusMultiplier = (radiusCurrent + idle * 0.12) * radiusScale (with idle sway)
r = ring * radiusMultiplier * (1 + sin(angle * aperture) * 0.16) (sinusoidal variance)
```
Key hardcoded coefficients: lerp speed `0.07`, idle sway `0.12`, sinusoidal variance `0.16`, drum
beat bonus `0.25`, beat cooldown `200 ms`.
#### Hue system
```
playing : hue = (seedHue + drum * 20 + treble * 15) % 360
silent : hue = (seedHue + sin(now * 0.00018) * 15) % 360
```
Drum pushes hue up to ±20°, treble up to ±15° from the seed colour.
#### Aperture (ellipse opening)
```
aperture = (1.55 + drum * 1.9 + idle * 0.24) * apertureScale
```
Range at `apertureScale=1`: 1.55 (silent+idle low) → 3.69 (loud drum+idle high). Controls how
elliptical the particle orbit becomes. The y-axis projection is always `r * 0.62` (hardcoded
ellipse squash ratio).
#### Spacing (ring gap)
```
spacing = (0.015 + (1 - drum) * 0.014) * spacingScale
ring = base + lane * (spacing * min(w,h) * 5)
```
Quiet = wider rings (0.029), loud drum = tighter rings (0.015). Range factor `5` and base `min(w,h)*0.2` are hardcoded.
#### Particle size
```
size = 1.75 + drum * 3.0 + (i % 4) * 0.35
```
Base 1.75 px, grows to 4.75 px at full drum. The `(i % 4) * 0.35` creates four size tiers
(variety), hardcoded.
#### Alpha (opacity per particle)
```
alpha = 0.34 + drum * 0.42
```
Range: 0.34 (quiet) → 0.76 (loud drum). Composited with `globalCompositeOperation = 'lighter'`
(additive blending).
#### Glow
```
glow = 14 + drum * 28 (shadowBlur, px)
```
Range: 14 px (quiet) → 42 px (loud drum).
#### Rotation speed (per-frame angle advance)
```
pt.angle += pt.speed + drum * 0.02
pt.speed = 0.002 + (i % 9) * 0.00045 (per-particle, set on init)
```
Base speed range: 0.002–0.006 rad/frame. Drum adds up to 0.02 rad/frame on top.
---
## Part 2 — Fluid: what changes during playback
Source: `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
### Input signals (per-frame)
| Signal | Source | Range | Description |
|---|---|---|---|
| `bass` | `f.bass` (clamped) | 0–1 | Low-frequency energy (bins 0–9 of FFT). Primary trail and beat driver. |
| `rms` | `f.rms` (clamped) | 0–1 | Overall loudness. Drives ambient spawn rate and drag. |
| `treble` | `f.treble` (clamped) | 0–1 | High-frequency energy. Drives blob pulse scale. |
| `beatDetected` | `f.beatDetected` | bool | Triggers burst spawn. |
| `idle` | `0.5 + 0.5 * sin(now * 0.0011)` | 0–1 | Slow baseline (~5.7 s period). Added to ambient spawn strength. |
### Computed values (all hardcoded coefficients)
#### Ambient spawn
```
spawnChance = (0.04 + rms * 0.31) * densityScale (per-frame probability)
spawnStrength = 0.22 + rms * 0.58 + idle * 0.08
```
At `densityScale=1`, rms=0: 4% chance/frame. At rms=1: 35% chance/frame.
Spawn strength range: 0.22 (quiet) → 0.88 (rms=1).
#### Beat burst
```
count = 3 + floor(bass * 4) (3–7 blobs per beat)
strength = 0.6 + bass * 0.8 (0.6–1.4)
```
Beat burst is the primary visual impact. Count and strength both scale with bass.
#### Blob properties (on spawn)
```
speed = (0.2 + strength * 2.2) * forceScale (px/frame)
radius = (8 + strength * 38) * sqrt(forceScale) (px)
life = 0.7 + strength * 0.6 (lifetime units, before decay)
```
Strong bass beat → speed ~3.3 * forceScale, radius ~61 * sqrt(forceScale).
#### Background fill (trail fade speed)
```
ctx.globalAlpha = 0.20 + (1 - bass) * 0.16
```
This is drawn over the canvas every frame to fade out old blobs. Range:
- `bass=0` (quiet): alpha 0.36 → fast trail fade
- `bass=1` (loud): alpha 0.20 → slow trail fade (persistent ink on loud beats)
This is the **primary visual trail control** and is currently entirely implicit — not exposed via
any setting.
#### Decay and drag
```
decay = (0.009 + (1 - bass) * 0.012) * decayScale
drag = 0.984 - rms * 0.010
b.life -= decay
b.dx *= drag
```
`decay` is subtracted from blob life each frame. At `decayScale=1`:
- Loud bass → 0.009/frame (long trail: ~77–110 frames to die)
- Quiet → 0.021/frame (short trail: ~33–47 frames to die)
`drag` retains dx/dy velocity. At rms=0: 98.4% retained; at rms=1: 97.4% retained. Small but
accumulates across many frames.
#### Treble pulse
```
pulse = 1 + treble * 0.25
r_render = b.radius * pulse * max(0.15, b.life)
```
Blobs swell up to 25% larger on treble hits. The `max(0.15, b.life)` also causes near-dead blobs
to be clamped to 15% of their base radius rather than vanishing immediately.
#### Colour cycling
Sequential through `speakerPalette` (from `VisualizerPalette`). No randomisation; no RMS/bass
influence on colour selection.
---
## Part 3 — Proposed new parameters
### 3a — Muon new parameters
All values are **multipliers** (1.0 = current default behaviour) or **direct overrides** where noted.
| Key | Type | Default | Range | Step | What it controls |
|---|---|---|---|---|---|
| `particle_count` | int | 220 | 40–500 | 10 | Number of particles. Directly impacts density and GPU cost. |
| `rotation_speed` | float | 1.0 | 0.2–4.0 | 0.1 | Multiplies `pt.speed` base (0.002–0.006 rad/frame) and the drum rotation bonus (0.02). |
| `glow_intensity` | float | 1.0 | 0.0–3.0 | 0.1 | Multiplies `glow = 14 + drum * 28`. At 0.0 glow is disabled. |
| `particle_size` | float | 1.0 | 0.25–3.0 | 0.05 | Multiplies `size = 1.75 + drum * 3.0`. |
| `drum_reactivity` | float | 1.0 | 0.0–2.0 | 0.1 | Scales the `drum` signal before it is applied to aperture, spacing, hue, size, and alpha. Does **not** affect the beat-detection threshold. |
| `hue_range` | float | 1.0 | 0.0–3.0 | 0.1 | Multiplies the `drum * 20` and `treble * 15` hue drift coefficients. At 0.0 hue is locked to seed colour. |
| `beat_sensitivity` | float | 1.0 | 0.2–3.0 | 0.1 | Divides the beat-preset cooldown (200 ms). Higher = preset cycles faster per beat. At 2.0, minimum 100 ms between presets. |
| `ellipse_ratio` | float | 0.62 | 0.3–1.0 | 0.02 | Direct override of the y-axis squash factor. 1.0 = circular orbit; 0.3 = very flat ellipse. |
**Implementation note:** `drum_reactivity` should be applied as:
```typescript
const drum = clamp01(f.drumIntensity * (settings?.drum_reactivity ?? 1))
```
This single multiplier propagates through aperture, spacing, size, alpha, glow, and hue without
needing separate coefficients per expression.
**Implementation note:** `particle_count` requires reinitialising the `particles` array. It should
only be acted on inside the `useEffect` setup (as a dep), not inside the render loop.
### 3b — Fluid new parameters
| Key | Type | Default | Range | Step | What it controls |
|---|---|---|---|---|---|
| `background_persistence` | float | 1.0 | 0.2–3.0 | 0.1 | Multiplies `(1 - bass) * 0.16` background-fill alpha contribution. Higher = slower trail fade = longer ink persistence on quiet passages. |
| `beat_burst_count` | float | 1.0 | 0.25–3.0 | 0.05 | Multiplies `3 + floor(bass * 4)` blob count per beat. |
| `treble_pulse` | float | 1.0 | 0.0–3.0 | 0.1 | Multiplies the `treble * 0.25` blob swell factor. At 0.0 blobs do not pulse on treble. |
| `drag` | float | 1.0 | 0.5–2.0 | 0.05 | Multiplies the velocity retention factor `0.984 - rms * 0.010`. Higher = blobs slow down faster (more viscous). |
| `blob_life` | float | 1.0 | 0.25–3.0 | 0.05 | Multiplies initial blob life `0.7 + strength * 0.6`. More life = blobs survive more decay ticks. |
**Implementation note:** `background_persistence` modifies the background fill expression as:
```typescript
const bgAlpha = 0.20 + (1 - bass) * 0.16 * (settings?.background_persistence ?? 1)
ctx.globalAlpha = Math.min(1, bgAlpha)
```
**Implementation note:** `drag` modifies the drag expression as:
```typescript
const drag = (0.984 - rms * 0.010) / Math.max(0.1, settings?.drag ?? 1)
```
(dividing by `drag > 1` makes blobs retain less velocity = more viscous/heavier)
---
## Part 4 — Named presets
Presets are stored client-side only (not in manifest.json). The sidebar "Visualizer" section shows
a preset row above the sliders; selecting a preset calls `patchVizSettings(presetValues)` for the
active visualizer's keys only, leaving the other visualizer's values untouched.
A **Custom** pseudo-preset is shown whenever any slider deviates from the active preset's values.
Selecting a named preset restores its exact values. The Reset button behaviour is unchanged (resets
to manifest.json values or VIZ_DEFAULTS, which may or may not match any named preset).
### Muon presets
| Preset | `particle_count` | `rotation_speed` | `glow_intensity` | `particle_size` | `drum_reactivity` | `hue_range` | `beat_sensitivity` | `ellipse_ratio` |
|---|---|---|---|---|---|---|---|---|
| **Default** | 220 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.62 |
| **Calm** | 120 | 0.5 | 0.4 | 0.7 | 0.5 | 0.4 | 0.5 | 0.55 |
| **Energetic** | 350 | 2.0 | 2.0 | 1.4 | 1.8 | 1.8 | 2.0 | 0.68 |
| **Dreamy** | 280 | 0.6 | 2.5 | 1.1 | 0.6 | 2.5 | 0.4 | 0.45 |
| **Minimal** | 60 | 0.8 | 0.0 | 0.6 | 1.2 | 0.0 | 1.0 | 1.0 |
| **Cosmic** | 400 | 1.5 | 1.8 | 1.8 | 1.4 | 2.0 | 1.5 | 0.38 |
**Calm** — sparse slow-moving particles with minimal glow and low audio reactivity. Good for
background ambience during quiet songs.
**Energetic** — dense fast-spinning particles with strong glow and full audio reactivity. Best for
high-BPM or hip-hop.
**Dreamy** — many particles with very high glow and wide hue drift but slow rotation. Works well
for ballads and ambient music.
**Minimal** — few particles, no glow, locked hue. Almost geometric — the spiral structure is
clearly visible without particle bloom.
**Cosmic** — maximum particles, strong glow, very flat ellipse. Fills the screen with a wide flat
galaxy-like disc.
### Fluid presets
| Preset | `spawn_density` | `splat_force` | `decay` | `background_persistence` | `beat_burst_count` | `treble_pulse` | `drag` | `blob_life` |
|---|---|---|---|---|---|---|---|---|
| **Default** | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
| **Ink** | 0.6 | 1.8 | 0.4 | 2.5 | 2.0 | 0.5 | 1.5 | 2.0 |
| **Foam** | 3.0 | 0.5 | 2.5 | 0.4 | 2.5 | 2.0 | 0.6 | 0.5 |
| **Lava** | 0.3 | 2.5 | 0.3 | 3.0 | 0.6 | 0.3 | 2.0 | 2.5 |
| **Mist** | 1.5 | 0.4 | 1.5 | 1.0 | 0.7 | 1.5 | 0.8 | 0.8 |
| **Storm** | 2.5 | 2.0 | 0.8 | 0.6 | 3.0 | 2.5 | 0.5 | 1.2 |
**Ink** — few large slow-moving blobs with very long trails. Dense, dark-ink-on-canvas look.
**Foam** — many tiny short-lived blobs, fast decay, strong treble pulse. Frothy/bubbly texture.
**Lava** — few massive blobs with very slow decay and high viscosity. Large glowing pools that
linger. Good for slow, heavy music.
**Mist** — light ambient density, low force, medium decay. Soft diffuse glow with gentle treble
shimmer.
**Storm** — high density, high force, many beat blobs, strong treble. Chaotic burst-heavy
response for energetic tracks.
---
## Part 5 — Updated manifest.json schema additions
The new parameters extend the existing `manifest.visualizer.settings.muon` and
`manifest.visualizer.settings.fluid` sub-objects. All new keys default to `1.0` (or the explicit
default noted above) and are optional — absence means "use default".
```json
{
"visualizer": {
"mode": "muon",
"settings": {
"quality": "balanced",
"fps_cap": 60,
"muon": {
"radius_multiplier": 1.0,
"aperture": 1.0,
"spacing": 1.0,
"particle_count": 220,
"rotation_speed": 1.0,
"glow_intensity": 1.0,
"particle_size": 1.0,
"drum_reactivity": 1.0,
"hue_range": 1.0,
"beat_sensitivity": 1.0,
"ellipse_ratio": 0.62
},
"fluid": {
"spawn_density": 1.0,
"splat_force": 1.0,
"decay": 1.0,
"background_persistence": 1.0,
"beat_burst_count": 1.0,
"treble_pulse": 1.0,
"drag": 1.0,
"blob_life": 1.0
}
}
}
}
```
### Updated parameter summary
**Muon** (11 keys total: 3 existing + 8 new):
| Key | New? | Sidebar widget | Range |
|---|---|---|---|
| `radius_multiplier` | existing | Slider | 0.25–2.0 |
| `aperture` | existing | Slider | 0.25–2.0 |
| `spacing` | existing | Slider | 0.25–2.0 |
| `particle_count` | **new** | Slider (int) | 40–500 |
| `rotation_speed` | **new** | Slider | 0.2–4.0 |
| `glow_intensity` | **new** | Slider | 0.0–3.0 |
| `particle_size` | **new** | Slider | 0.25–3.0 |
| `drum_reactivity` | **new** | Slider | 0.0–2.0 |
| `hue_range` | **new** | Slider | 0.0–3.0 |
| `beat_sensitivity` | **new** | Slider | 0.2–3.0 |
| `ellipse_ratio` | **new** | Slider | 0.30–1.0 |
**Fluid** (8 keys total: 3 existing + 5 new):
| Key | New? | Sidebar widget | Range |
|---|---|---|---|
| `spawn_density` | existing | Slider | 0.25–2.0 |
| `splat_force` | existing | Slider | 0.25–2.0 |
| `decay` | existing | Slider | 0.25–2.0 |
| `background_persistence` | **new** | Slider | 0.2–3.0 |
| `beat_burst_count` | **new** | Slider | 0.25–3.0 |
| `treble_pulse` | **new** | Slider | 0.0–3.0 |
| `drag` | **new** | Slider | 0.5–2.0 |
| `blob_life` | **new** | Slider | 0.25–3.0 |
---
## Part 6 — Sidebar layout update (Section 4 revision)
The `PerformanceSidebar` Section 4 layout should be updated to:
```
├──────────────────────────────────────────┤
│ ▾ VISUALIZER (hidden for Static/Video) │
│ [Calm] [Default] [Energetic] [Dreamy] │ ← preset pills
│ [Minimal] [Cosmic] [Custom] │
│ Quality [Low] [Balanced] [High] │
│ FPS cap [30] [60] │
│ ─────────────────────────────────────── │
│ Particle count ███░░░░░ 220 │
│ Rotation speed ████░░░░ 1.00× │
│ Glow ████░░░░ 1.00× │
│ Particle size ████░░░░ 1.00× │
│ Drum reactivity ████░░░░ 1.00× │
│ Hue range ████░░░░ 1.00× │
│ Beat sensitivity████░░░░ 1.00× │
│ Ellipse ratio ████████ 0.62 │
│ Radius ████░░░░ 1.00× │
│ Aperture ████░░░░ 1.00× │
│ Spacing ████░░░░ 1.00× │
│ [Reset to song defaults] │
└──────────────────────────────────────────┘
```
(Fluid variant swaps the parameter rows for its 8 keys; preset pills change to Ink/Foam/Lava/Mist/Storm/Default.)
Presets are stored as a local constant in `PerformanceSidebar.tsx` — not in manifest.json and not
in localStorage. They are not user-editable; users adjust sliders individually.
---
## Part 7 — Implementation impact on existing files
### `MuonVisualizerCanvas.tsx`
- Expand `settings` interface with the 8 new keys.
- Apply `drum_reactivity` multiplier to the `drum` local variable at the top of `render()`.
- Apply `rotation_speed` to `pt.speed` in particle init and to `drum * 0.02` rotation bonus.
- Apply `glow_intensity` to `glow = 14 + drum * 28`.
- Apply `particle_size` to `size = 1.75 + drum * 3.0`.
- Apply `hue_range` to the `drum * 20` and `treble * 15` hue coefficients.
- Apply `beat_sensitivity` by dividing the 200 ms cooldown: `200 / beatSensitivity`.
- Use `ellipse_ratio` in place of the hardcoded `0.62`.
- `particle_count`: reinitialise `particles` inside `useEffect` when the dep changes (add
`settings?.particle_count` to the dep array or detect the change and reinit).
### `FluidVisualizerCanvas.tsx`
- Expand `settings` interface with the 5 new keys.
- Apply `background_persistence` to the background fill alpha expression.
- Apply `beat_burst_count` to `count = 3 + floor(bass * 4)`.
- Apply `treble_pulse` to `pulse = 1 + treble * 0.25`.
- Apply `drag` multiplier to the drag expression.
- Apply `blob_life` multiplier to `life = 0.7 + strength * 0.6` in `spawnBlob`.
### `VisualizerBackdrop.tsx`
No change needed — already passes `settings?.muon` and `settings?.fluid` to the respective
canvases. New keys are transparent as long as the canvas interfaces are expanded.
### `PerformanceSidebar.tsx`
- Add preset constant arrays for Muon and Fluid.
- Add preset pill row above Quality/FPS controls.
- Add 8 Muon sliders and 5 Fluid sliders (in addition to existing 3+3).
- Detect deviation from active preset → show "Custom" as active.
### `LyricsLivePreview.tsx` / `VizSettings` type
Expand `VizSettings` and `VIZ_DEFAULTS` with the 13 new keys across both sub-objects.
### Backend — `_normalize_visualizer_config` (optional)
The backend currently writes only `radius_multiplier`, `aperture`, `spacing`, `spawn_density`,
`splat_force`, `decay` to `manifest.json`. If per-song presets are ever authored at export time,
the backend normaliser will need to accept and round-trip the new keys. For the sidebar use-case
(live adjustment only), backend changes are not required.
+368
View File
@@ -0,0 +1,368 @@
# Visualizer Parameter Audit & Fix Notes
Date: 2026-04-08
Scope: Bugs in the Muon Music Visualizer source that break dynamic visuals; corrections to the integration plan's parameter assumptions; and the definitive list of manifest.json keys for both visualizers.
## Outstanding-item closure (2026-04-08)
1. **Manifest schema mismatch (Part 3/4 vs backend reality):** ✅ **fixed**
Parts 3 and 4 were rewritten to match `_normalize_visualizer_config` and the actual `manifest.visualizer.settings` structure written by backend export.
2. **Integration plan still had broken mapping snippets:** ✅ **fixed**
The Muon mapping section and Fluid config-mapping snippet in `docs/2026-04-08_Visualizer_Integration_Plan.md` were corrected to match implemented behaviour.
3. **Dissipation-direction uncertainty for future WebGL fluid port:** ✅ **documented + guarded**
The caveat remains explicit in both docs. Current implementation uses the canvas blob renderer (no `DENSITY_DISSIPATION` shader param); direction must be verified when/if the WebGL sim is vendored.
Implementation traceability:
- Frontend now threads `manifest.visualizer.settings` into the runtime renderers via `Karaoke.tsx` / `KaraokeTV.tsx` → `KaraokePerformanceMode.tsx` → `LyricsLivePreview.tsx` → `VisualizerBackdrop.tsx`.
- Muon renderer applies `settings.muon.radius_multiplier`, `settings.muon.aperture`, and `settings.muon.spacing` as post-audio multipliers.
- Fluid renderer applies `settings.fluid.spawn_density`, `settings.fluid.splat_force`, and `settings.fluid.decay` as post-audio multipliers.
- Integration plan snippets were corrected in `docs/2026-04-08_Visualizer_Integration_Plan.md` for Corrections A/B/C.
Post-deploy runtime investigation (Cuidate SEK):
- Symptom: visualizer renders but appears static/non-reactive in multi-stem playback.
- Root cause: `useAudioFeatures()` could execute before the multi-stem WebAudio graph was built (`gainRefs.current` empty). The hook then exited early with no tap node and did not reconnect, so `bass/treble/drumIntensity` stayed near zero.
- Fix implemented: in `LyricsLivePreview.tsx`, audio feature extraction is now gated by `audioFeatureTapReady = !isMultiStem || Object.keys(gainRefs.current).length > 0`, and `useAudioFeatures.enabled` is set to `isVisualizerMode && audioFeatureTapReady`.
- Result: the hook first enables only after stem gain nodes exist, ensuring analyser attachment and live feature updates.
---
## Part 1 — Muon Music Visualizer bugs
Three bugs exist in the **original muon-music-visualizer source** that collectively explain why dynamic visuals fail or look static during playback. All three must be corrected in the StemEngine port.
---
### Bug 1 — `hueControl` always returns 0 during playback (colour is frozen)
**File:** `src/CoreControls/effects.js`
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
```javascript
// ORIGINAL (broken)
export const hueControl = (scaler) => {
const time = Date.now() * 0.00015;
const hue = ((360 * (1.0 + !scaler && time + scaler * 10)) % 360) / 360;
return hue;
};
```
**What it does:** `!scaler` is a boolean. When `scaler` is any non-zero number (i.e. whenever audio is playing), `!scaler === false`. The short-circuit `false && (time + scaler * 10)` evaluates to `false` (cast to `0`). So `hue = ((360 * 1.0) % 360) / 360 = 0` — permanently red — whenever music is playing.
Hue only animates when `scaler === 0` (silence), which is the opposite of the intended behaviour.
**Fix:**
```typescript
// CORRECTED
export function hueControl(scaler: number): number {
const time = Date.now() * 0.00015;
// When scaler > 0 (audio playing), modulate hue by scaler.
// When silent, fall back to slow time-based drift.
const hue = ((360 * (1.0 + (scaler > 0 ? scaler * 10 : time))) % 360) / 360;
return hue;
}
```
---
### Bug 2 — `baseFr` and `trebleFr` are named backwards (kick drum has no visual impact)
**File:** `src/CoreControls/audio.js`
**Status: IMPLEMENTED** — `frontend/src/hooks/useAudioFeatures.ts` (`avgSlice` bin ranges + `deriveDrumIntensity` exponential scalers)
```javascript
// ORIGINAL (misleadingly named)
const lowerHalfArray = dataArray.slice(0, dataArray.length / 2 - 1); // LOW frequencies = bass
const upperHalfArray = dataArray.slice(dataArray.length / 2 - 1, ...); // HIGH frequencies = treble
const lowerAvgFr = Operations.avg(lowerHalfArray) / lowerHalfArray.length; // bass average
const upperAvgFr = Operations.avg(upperHalfArray) / upperHalfArray.length; // treble average
// BUG: upper (treble) goes into baseFr, lower (bass) goes into trebleFr
const baseFr = Operations.modulate(upperAvgFr, 0, 1, 0, 4) / 50; // ← actually treble
const trebleFr = Operations.modulate(lowerAvgFr, 0, 1, 0, 4) / 200; // ← actually bass
```
The `lowerHalfArray` covers **low frequencies (bass/kick)** and `upperHalfArray` covers **high frequencies (treble/cymbals)**, but the results are assigned to the opposite variable names. This propagates into `exponentialBassScaler` and `exponentialTrebleScaler`, which are then used in `wave.js`:
```javascript
// wave.js — beatScaler formula
beatScaler = exponentialTrebleScaler * 3.14 + exponentialBassScaler * 6.24 + 1.7
```
Because of the swap, kick-drum energy ends up in `exponentialTrebleScaler` (multiplied by the smaller 3.14 coefficient) and cymbal/treble energy ends up in `exponentialBassScaler` (multiplied by the larger 6.24 coefficient). Kick hits produce weaker visual response than cymbals — the opposite of the intended drum-intensity drive.
**Fix:** Correct the variable assignment to match the actual frequency content:
```typescript
// CORRECTED — names now match frequency content
const bassAvgFr = Operations.modulate(lowerAvgFr, 0, 1, 0, 4) / 50; // low freq → bass
const trebleAvgFr = Operations.modulate(upperAvgFr, 0, 1, 0, 4) / 200; // high freq → treble
const exponentialBassScaler = 1.001 * (1 + bassAvgFr * 2) - 1.001;
const exponentialTrebleScaler = 1.001 * (1 + trebleAvgFr * 2) - 1.001;
```
---
### Bug 3 — `coreScaler` amplification is dead code
**File:** `src/CoreControls/audio.js`, line 24
**Status: IMPLEMENTED** — `frontend/src/hooks/useAudioFeatures.ts` (`deriveDrumIntensity` uses the exponential scaler formula; `drumIntensity` is computed fresh each frame so no compounding risk)
```javascript
// ORIGINAL (no-op)
coreScaler < 10 && coreScaler * 1.2;
```
This evaluates `coreScaler * 1.2` but throws the result away. `coreScaler` is never actually amplified. The intended logic was to cap-and-boost values below 10.
**Fix:**
```typescript
// CORRECTED
if (coreScaler < 10) coreScaler *= 1.2;
```
> **⚠️ Compounding risk:** If `coreScaler` is **persistent state** that carries over between animation frames (i.e. the same variable is reused rather than recomputed from audio data each frame), this `*= 1.2` fires on every frame where `coreScaler < 10` and compounds until it crosses 10. If that is the case, cap the result explicitly: `coreScaler = Math.min(coreScaler * 1.2, 10)`. Verify whether `coreScaler` is a fresh per-frame value or a running accumulator before porting.
---
## Part 2 — Integration plan corrections
The integration plan (`2026-04-08_Visualizer_Integration_Plan.md`) contains incorrect assumptions about Muon's parameter system and both visualizers' audio mappings.
---
### Correction A — `colorSpectrum` is not a hue angle (range is 3–30, not 0–360)
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
The canvas 2D port does not use a `colorSpectrum` geometry parameter. The equivalent hue-drift line was corrected: treble drift is capped at ±15° (from ±40°) to stay within the manifest palette colour family.
The plan states:
> `colorSpectrum` is driven by treble energy independently — `colorSpectrum = (baseColorSpectrum + treble * 120) % 360`
**Actual GUI definition:**
```javascript
{ name: "colorSpectrum", min: 3, max: 30, step: 3, folder: particleControlsFolder }
```
`colorSpectrum` is a **colour segment count** passed directly to `generateParticlesSpiral` — it controls how many colour bands exist in the spiral geometry, not a hue rotation angle. Its range is 3–30 (integer multiples of 3). Adding 120 would put it 4× out of range and corrupt the geometry.
**Corrected mapping:**
```typescript
// colorSpectrum: integer 3–30. Audio can nudge it by ±1 step (±3) at most.
// Convert to step index first, apply the treble offset, then multiply back.
// Do NOT add ±1 to the raw value before dividing: when the base is already a
// multiple of 3, the ±1 delta is always absorbed by Math.round( / 3) and the
// output never changes (e.g. Math.round(13/3)*3 = 4*3 = 12 — same as 12).
params.colorSpectrum = clamp(
(Math.round(baseColorSpectrum / 3) + Math.round((treble - 0.5) * 2)) * 3,
3, 30
)
```
The hue cycling effect the plan was trying to achieve is controlled by `hueControl()` (fixed in Bug 1 above) — not `colorSpectrum`.
---
### Correction B — `radiusMultiplier` is managed by preset cycling, not direct audio mapping
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
Added `RADIUS_PRESETS` array and beat-triggered preset cycling with lerp interpolation (`radiusCurrent → radiusTarget`). Direct per-frame `drum * 1.12` mapping removed.
The plan states:
> `params.radiusMultiplier = lerp(0.4, 1.8, easeIn(drumIntensity))`
**Actual behaviour:** `radiusMultiplier` is controlled by `wavePresetController`, which cycles through the `visualizationPresets` array via GSAP animation:
```javascript
// visualizationPresets — 16 values, range 0.6603–0.7536
export const visualizationPresets = [
0.6604, 0.6606, 0.6608, 0.6609, 0.661, 0.6612, 0.6614, 0.7536,
0.6615, 0.6613, 0.6611, 0.661, 0.66081, 0.6607, 0.6605, 0.6603,
];
```
`wavePresetController` triggers a GSAP-animated transition to the next preset whenever `Math.abs(delta) > params.deltaResponseLimit`. Directly lerping `radiusMultiplier` to 1.8 would both exceed the preset range and fight the GSAP animation on every frame.
**Corrected approach:** Do not write `radiusMultiplier` from audio data. Instead, tune `deltaResponseLimit` (lower = more frequent preset transitions) to control how reactive the preset cycling is to audio beats. The `beatScaler` uniform (in `sineWavePropagation`) already handles per-frame amplitude pulsing correctly.
---
### Correction C — Fluid sim `DENSITY_DISSIPATION` mapping is inverted
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
The canvas blob sim uses `decay` (per-frame lifetime drain) and `drag` (velocity retention). Fixed: `decay` now driven by `bass` (loud bass → slower decay → longer trails); `drag` driven by `rms` (overall loudness → more momentum). Previously both were driven by `rms`, making trail length insensitive to bass.
The ⚠️ dissipation-direction verification note remains relevant if the full WebGL fluid sim is later vendored.
The plan states:
> `DENSITY_DISSIPATION = lerp(0.97, 0.999, 1 - bass)` — "longer trails on loud bass"
**Actual semantics:** A **higher** `DENSITY_DISSIPATION` value means ink fades **faster** (more dissipation per frame). A value of 0.999 dissipates nearly instantly; 0.9 produces long-lived trails.
The plan's formula gives 0.999 at high bass — ink disappears almost immediately on loud hits — the opposite of longer trails.
**Corrected mapping:**
```typescript
// Lower value = slower dissipation = longer trails on loud bass
config.DENSITY_DISSIPATION = lerp(0.90, 0.98, 1 - bass); // loud bass → 0.90 → long trails
config.VELOCITY_DISSIPATION = lerp(0.85, 0.99, 1 - rms); // loud overall → slower decay
```
> **⚠️ Verify dissipation direction before implementing.** The standard PavelDoGreat/WebGL-Fluid-Simulation advection shader applies dissipation as a **retention multiplier** — `density *= uDissipation` — which makes 0.999 the near-permanent (longest-trail) value and 0.9 the faster-fading value (**higher = longer trails**). If the vendored build follows that convention, the "lower = longer" semantics stated here are inverted: the corrected mapping would *shorten* trails on loud bass, which is the opposite of the intent. Confirm by inspecting the advection shader in the vendored source before implementing either mapping.
---
### Correction D — `generateColor()` output is too dim for programmatic splats
**Status: NOT APPLICABLE** — `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
The canvas blob sim uses `nextPaletteColor()` which returns manifest palette hex strings directly — no 0.15 brightness scaling. The `generateColor()` bug was specific to the original WebGL fluid sim's colour generator and does not exist in the current implementation.
`generateColor()` in the fluid sim multiplies RGB values by 0.15, producing near-invisible ink. For audio-driven splats, use full-brightness colours:
```typescript
function audioSplatColor(hue: number): { r: number; g: number; b: number } {
// HSV → RGB at full saturation and value, not scaled by 0.15
const [r, g, b] = hsvToRgb(hue, 0.8, 1.0);
return { r, g, b };
}
```
---
### Correction E — Direct `splat()` calls bypass aspect-ratio correction
**Status: NOT APPLICABLE** — `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
The canvas blob sim uses pixel-space coordinates and velocities; there is no WebGL normalised coordinate system. Aspect-ratio correction is only needed when integrating the real WebGL fluid sim (`PavelDoGreat/WebGL-Fluid-Simulation`). The `audioSplat()` wrapper pattern documented below remains the correct approach for that integration.
The plan calls `fluidSim.splat(x, y, dx, dy, color)` directly. The original source's `splatPointer()` applies `correctDeltaX/Y` which normalises velocity deltas for the canvas aspect ratio. Without this, splats are stretched or squashed on non-square viewports.
**Fix:**
```typescript
function audioSplat(sim: FluidSim, x: number, y: number, dx: number, dy: number, color: Color) {
const aspectRatio = sim.canvas.width / sim.canvas.height;
// correctDeltaX: dx * (1 / aspectRatio) — from the original source
sim.splat(x, y, dx / aspectRatio, dy, color);
}
```
---
## Part 3 — Actual manifest.json `visualizer` schema
> **⚠️ Correction to original audit:** The parameter lists below do NOT reflect what is actually written to `manifest.json`. The backend (`_normalize_visualizer_config` in `project_assets_service.py`) produces a compact schema of **relative multipliers** and global quality settings, not the full set of raw GUI parameters described in the original Part 3 tables. The original tables described Muon's dat.gui parameters directly; those are not persisted and are not read by the frontend at runtime.
### What `manifest.visualizer` actually contains
```json
{
"visualizer": {
"mode": "muon",
"settings": {
"quality": "balanced",
"fps_cap": 60,
"muon": {
"radius_multiplier": 1.0,
"aperture": 1.0,
"spacing": 1.0
},
"fluid": {
"spawn_density": 1.0,
"splat_force": 1.0,
"decay": 1.0
}
}
}
}
```
All numeric values are **scale multipliers** (1.0 = default behaviour). They allow per-song user tuning of animation intensity without overriding the audio-driven values. The frontend currently reads only `manifest.visualizer.mode` at startup to set the initial `LyricsVisualStyle`; the multiplier sub-objects are available for future use.
### `visualizer` keys — top level
| Key | Type | Values | Default | Notes |
|---|---|---|---|---|
| `mode` | string | `"muon"` \| `"fluid"` \| `"auto"` | `"muon"` | Initial visualizer mode. `"auto"` selects based on device performance (future). Validated by `_normalize_visualizer_config`; invalid values fall back to `"muon"`. |
### `visualizer.settings` keys
| Key | Type | Notes |
|---|---|---|
| `quality` | string | Rendering quality preset: `"low"` / `"balanced"` / `"high"`. Not yet read by frontend (reserved for future GPU scaling). |
| `fps_cap` | int | Target frame-rate cap. Default 60. |
### `visualizer.settings.muon` keys
These are **multipliers** applied on top of the audio-driven animation values, not raw geometry parameters.
| Key | Type | Default | Effect when > 1.0 |
|---|---|---|---|
| `radius_multiplier` | float | 1.0 | Scales the preset-cycling radius range outward. |
| `aperture` | float | 1.0 | Scales the spiral aperture range (wider opening). |
| `spacing` | float | 1.0 | Scales inter-particle spacing (larger gaps). |
### `visualizer.settings.fluid` keys
| Key | Type | Default | Effect when > 1.0 |
|---|---|---|---|
| `spawn_density` | float | 1.0 | Multiplies the ambient blob spawn probability per frame. |
| `splat_force` | float | 1.0 | Multiplies blob velocity/size on spawn. |
| `decay` | float | 1.0 | Multiplies per-frame blob lifetime drain (higher = shorter trails). |
**Not stored** — runtime-derived or global:
- All audio-driven computed values (`drumIntensity`, `bass`, `treble`, `rms`, `beatDetected`)
- Colour — derived from `metadata.dynamic_palette` at runtime; not written to `visualizer`
- Individual Muon dat.gui parameters (`maxPoints`, `colorSpectrum`, `divisions`, etc.) — not persisted
- WebGL fluid sim config (`DENSITY_DISSIPATION`, `CURL`, `BLOOM_*`, etc.) — not persisted
- `SIM_RESOLUTION`, `DYE_RESOLUTION`, and other global performance settings
---
## Part 4 — Updated manifest.json schema
```json
{
"format": "stemengine-karaoke",
"version": 1,
"project": { "id": "...", "name": "..." },
"metadata": {
"title": "...",
"artist": "...",
"dynamic_palette": {
"source": "embedded_cover_art",
"seed_color": "#a03060",
"background_color": "#1a0510",
"text_color": "#ffffff",
"highlight_color": "#d06090",
"word_highlight_color": "#e080b0",
"speaker_palette": ["#a03060", "..."]
}
},
"contents": {
"lyrics": "lyrics/default.elrc",
"lyrics_tracks": { "default": "lyrics/default.elrc" },
"video": null,
"stems": [{ "name": "vocals", "file": "stems/vocals.wav", "bytes": 12345678 }]
},
"meta": {
"stem_count": 4,
"has_video": false,
"synchronization": { "reference_track": "source", "track_offsets_seconds": {}, "video_source": "", "last_sync": null }
},
"visualizer": {
"mode": "muon",
"settings": {
"quality": "balanced",
"fps_cap": 60,
"muon": { "radius_multiplier": 1.0, "aperture": 1.0, "spacing": 1.0 },
"fluid": { "spawn_density": 1.0, "splat_force": 1.0, "decay": 1.0 }
}
}
}
```
Only `visualizer.mode` is read by the frontend at startup. Colour is always sourced from `metadata.dynamic_palette`. Both `muon` and `fluid` sub-objects are always written (with defaults) so the schema is stable regardless of which mode is active.
+302
View File
@@ -0,0 +1,302 @@
# Visualizer GUI Parameter Map
Date: 2026-04-09
Sources:
- **Muon** — https://muon-vis.netlify.app / https://github.com/najafmohammed/muon-music-visualizer
- **Fluid** — https://paveldogreat.github.io/WebGL-Fluid-Simulation / https://github.com/PavelDoGreat/WebGL-Fluid-Simulation
Scope: Every user-adjustable parameter exposed in the original GUIs for both visualizers.
Recorded verbatim from source — ranges, defaults, and step sizes are as defined in the dat.gui
calls unless otherwise noted. Effect descriptions are derived from the source code.
---
## Part 1 — Muon Music Visualizer
### Default `params` object (from `src/index.js`)
```javascript
let params = {
maxPoints: 0, // Note: 0 means "use all computed points"
colorSpectrum: 3,
aperture: 3,
sineCounterMultiplier: 1,
idleMultiplier: 0.27,
particleMirror: true,
radiusMultiplier: 0.66,
dynamicRadius: true,
updateLockInterval: 0.17,
deltaResponseLimit: 0.005,
visualizationPreset: true,
spacing: 1,
syncColors: true,
divisions: 21,
lifespan: 200,
noiseScale: 0.3,
noiseForce: 1,
timeMult: 0.01,
enableMonoColor: false,
monoColor: { h: 350, s: 0.9, v: 0.3 },
}
```
### Post-processing defaults (from `src/CoreControls/postProcessing.js`)
```javascript
// UnrealBloomPass
bloomStrength: 0.55
bloomThreshold: 0.35
bloomRadius: 0.6
// AfterimagePass
damp: 0.86
```
### `visualizationPresets` array (from `src/CoreControls/params.js`)
16 values cycling via `wavePresetController` on beats:
```
[0.6604, 0.6606, 0.6608, 0.6609, 0.661, 0.6612, 0.6614, 0.7536,
0.6615, 0.6613, 0.6611, 0.661, 0.66081, 0.6607, 0.6605, 0.6603]
```
These are `radiusMultiplier` target values. Range: 0.6603–0.7536.
---
### GUI parameter table (from `src/GUI/index.js`)
#### Root GUI (no folder)
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| radiusMultiplier | `radiusMultiplier` | Slider | 0.01 | 1 | 0.0001 | 0.66 | Scales the spiral radius range that `wavePresetController` cycles through. Lower = tighter spiral. |
| side view | `sideView` | Button | — | — | — | — | Triggers GSAP camera animation to a side perspective. |
| reset | `reset` | Button | — | — | — | — | Resets all params to defaults and restores camera. |
#### Particle Controls folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| spacing | `spacing` | Slider | 0 | 1 | 0.01 | 1 | Inter-ring gap in the spiral. Higher = rings farther apart. |
| maxPoints | `maxPoints` | Slider | 0 | 12240 | 360 | 0 | Number of rendered particles. 0 = full computed count. |
| colorSpectrum | `colorSpectrum` | Slider | 3 | 30 | 3 | 3 | Number of colour bands in the spiral geometry (integer multiples of 3). **Not a hue angle.** Higher = more colour segments visible. |
| aperture | `aperture` | Slider | 0 | π (~3.14) | 0.01 | 3 | Controls the opening angle of the spiral. Near 0 = very tight helix; near π = fully open ellipse. |
| particleMirror | `particleMirror` | Toggle | — | — | — | true | When on, duplicates the particle system on both sides of the spiral axis. |
#### Controls folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| dynamicRadius | `dynamicRadius` | Toggle | — | — | — | true | Enables audio-reactive radius changes. When off, `radiusMultiplier` stays fixed at its current value and `wavePresetController` does not cycle. |
| visualizationPreset | `visualizationPreset` | Toggle | — | — | — | true | Enables GSAP-animated preset cycling via `wavePresetController` on beat detection. When off, radius stays fixed. |
| idleMultiplier | `idleMultiplier` | Slider | 0 | 2 | 0.01 | 0.27 | When audio is not playing, multiplies elapsed time to advance `sineCounter` at reduced speed for gentle background animation. Higher = faster idle motion. |
#### Audio Controls folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| deltaResponseLimit | `deltaResponseLimit` | Slider | 0.001 | 0.01 | 0.001 | 0.005 | Minimum beat-delta required to trigger a preset advance in `wavePresetController`. Lower = more sensitive (advances faster on softer beats). |
#### Emission Controls folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| enableMonoColor | `enableMonoColor` | Toggle | — | — | — | false | When on, replaces audio-driven HSL cycling with a single fixed colour (`monoColor`). |
| divisions | `divisions` | Slider | 1 | 150 | 1 | 21 | Number of frequency bins used to drive wave geometry segmentation. Higher = more frequency detail in the wave shape, more geometry vertices. |
| lifespan | `lifespan` | Slider | 10 | 250 | 1 | 200 | Particle lifetime in milliseconds before removal and reuse. Lower = faster particle turnover / more chaotic look. |
| monoColor | `monoColor` | Color (HSV) | — | — | — | h:350 s:0.9 v:0.3 | Fixed colour used when `enableMonoColor` is on. Expressed as HSV; h=350 is deep red/magenta. |
#### Noise Controls folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| noiseScale | `noiseScale` | Slider | 0 | 1 | 0.0001 | 0.3 | Magnitude of noise texture applied to particle positions. 0 = perfectly regular spiral; 1 = maximum noise displacement. |
| noiseForce | `noiseForce` | Slider | 0 | 1 | 0.0001 | 1 | Multiplier on noise-driven particle velocity/force. Lower = noise displaces particles more slowly. |
#### Post-processing (not in GUI by default — hardcoded in `postProcessing.js`)
| Parameter | Default | Effect |
|---|---|---|
| `bloomStrength` | 0.55 | Bloom glow intensity (Three.js UnrealBloomPass). |
| `bloomThreshold` | 0.35 | Brightness threshold above which bloom is applied. |
| `bloomRadius` | 0.6 | Spread radius of the bloom effect. |
| `damp` (afterimage) | 0.86 | Afterimage persistence. 1.0 = permanent trail; 0.0 = no trail. |
> **Note:** `sineCounterMultiplier`, `updateLockInterval`, `syncColors`, `timeMult` are in the
> `params` object and affect rendering but are **not exposed in the GUI**. They cannot be changed
> at runtime on muon-vis.netlify.app.
---
## Part 2 — WebGL Fluid Simulation
### Default `config` object (from `script.js`)
```javascript
let config = {
SIM_RESOLUTION: 128,
DYE_RESOLUTION: 1024,
CAPTURE_RESOLUTION: 512,
DENSITY_DISSIPATION: 1,
VELOCITY_DISSIPATION: 0.2,
PRESSURE: 0.8,
PRESSURE_ITERATIONS: 20,
CURL: 30,
SPLAT_RADIUS: 0.25,
SPLAT_FORCE: 6000,
SHADING: true,
COLORFUL: true,
COLOR_UPDATE_SPEED: 10,
PAUSED: false,
BACK_COLOR: { r: 0, g: 0, b: 0 },
TRANSPARENT: false,
BLOOM: true,
BLOOM_ITERATIONS: 8,
BLOOM_RESOLUTION: 256,
BLOOM_INTENSITY: 0.8,
BLOOM_THRESHOLD: 0.6,
BLOOM_SOFT_KNEE: 0.7,
SUNRAYS: true,
SUNRAYS_RESOLUTION: 196,
SUNRAYS_WEIGHT: 1.0,
}
```
### GUI parameter table (from `script.js` lines 313–400)
#### Main panel (no folder)
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| Quality | `DYE_RESOLUTION` | Dropdown | — | — | — | 1024 | Ink/dye texture resolution. Options: High (1024), Medium (512), Low (256), Very Low (128). Lower = better performance, softer ink. |
| Sim resolution | `SIM_RESOLUTION` | Dropdown | — | — | — | 128 | Velocity field simulation grid size. Options: 32 / 64 / 128 / 256. Lower = faster but chunkier fluid motion. |
| Density diffusion | `DENSITY_DISSIPATION` | Slider | 0 | 4.0 | — | 1 | How quickly ink fades each frame. In the advection shader: `density *= uDissipation`. **Higher = faster fade = shorter trails.** 0 = ink persists forever; 4 = near-instant disappearance. |
| Velocity diffusion | `VELOCITY_DISSIPATION` | Slider | 0 | 4.0 | — | 0.2 | How quickly fluid velocity dissipates. Higher = more viscous / fluid stops faster. |
| Pressure | `PRESSURE` | Slider | 0.0 | 1.0 | — | 0.8 | Pressure solve strength. Higher values maintain more fluid pressure, producing tighter, more defined flow patterns. |
| Vorticity | `CURL` | Slider | 0 | 50 | 1 | 30 | Curl/vorticity confinement. Higher = more swirling/turbulent flow. 0 = no swirl. |
| Splat radius | `SPLAT_RADIUS` | Slider | 0.01 | 1.0 | — | 0.25 | Size of each ink injection point. Larger = wider splat footprint. |
| Shading | `SHADING` | Toggle | — | — | — | true | Enables normal-map-based shading on the fluid surface, giving it a 3D appearance. |
| Colorful | `COLORFUL` | Toggle | — | — | — | true | When on, continuously cycles ink colour via `COLOR_UPDATE_SPEED`. When off, new splats keep their initial colour indefinitely. |
| Paused | `PAUSED` | Toggle | — | — | — | false | Freezes the simulation (stops advection). Splats can still be added while paused. |
| Random splats | *(button)* | Button | — | — | — | — | Adds a random burst of coloured splats across the canvas. |
#### Bloom folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| Enabled | `BLOOM` | Toggle | — | — | — | true | Enables the bloom post-processing pass. |
| Intensity | `BLOOM_INTENSITY` | Slider | 0.1 | 2.0 | — | 0.8 | Bloom glow strength. Higher = brighter glow around bright ink regions. |
| Threshold | `BLOOM_THRESHOLD` | Slider | 0.0 | 1.0 | — | 0.6 | Brightness level above which bloom applies. Lower threshold = more of the ink blooms. |
> `BLOOM_SOFT_KNEE` (0.7) and `BLOOM_RESOLUTION` (256) and `BLOOM_ITERATIONS` (8) are in the
> config object but **not exposed in the GUI**. They affect bloom quality and softness.
#### Sunrays folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| Enabled | `SUNRAYS` | Toggle | — | — | — | true | Enables the god-rays / sunrays post-processing pass through the fluid. |
| Weight | `SUNRAYS_WEIGHT` | Slider | 0.3 | 1.0 | — | 1.0 | Intensity of the sunrays effect. |
> `SUNRAYS_RESOLUTION` (196) is in config but **not in the GUI**.
#### Capture folder
| Label | Key | Control | Min | Max | Step | Default | Effect |
|---|---|---|---|---|---|---|---|
| Background colour | `BACK_COLOR` | Color picker | — | — | — | #000000 | Background colour shown when `TRANSPARENT` is off. |
| Transparent | `TRANSPARENT` | Toggle | — | — | — | false | Renders fluid over a transparent background (useful for capture/export). |
| Take screenshot | *(button)* | Button | — | — | — | — | Downloads a PNG snapshot of the current frame. |
---
## Part 3 — Cross-reference: what is and isn't in StemEngine's `VizSettings`
### Muon — current StemEngine coverage vs. full GUI
| GUI parameter | In `VizSettings`? | Notes |
|---|---|---|
| `radiusMultiplier` | ✅ `muon.radius_multiplier` | As a multiplier on preset cycle range |
| `aperture` | ✅ `muon.aperture` | As a multiplier on audio-computed value |
| `spacing` | ✅ `muon.spacing` | As a multiplier |
| `maxPoints` / `particle_count` | 🔲 Proposed (Controls Audit) | Direct int, not a multiplier |
| `deltaResponseLimit` / `beat_sensitivity` | 🔲 Proposed (Controls Audit) | As divisor of 200 ms cooldown |
| `idleMultiplier` | ❌ Not mapped | Could map directly; controls idle animation speed |
| `noiseScale` | ❌ Not mapped | Controls spatial regularity of the spiral |
| `noiseForce` | ❌ Not mapped | Controls velocity of noise-displaced particles |
| `divisions` | ❌ Not mapped | Frequency bin resolution — affects reactivity detail |
| `lifespan` | ❌ Not mapped | Particle turnover rate |
| `colorSpectrum` | ❌ Not mapped | Colour band count (int ×3, range 3–30) |
| `enableMonoColor` | ❌ Not mapped | Boolean — locks colour to `monoColor` |
| `monoColor` | ❌ Not mapped | Fixed HSV colour when mono mode is on |
| `particleMirror` | ❌ Not mapped | Boolean — mirrors geometry |
| `dynamicRadius` | ❌ Not mapped | Boolean — enables audio-reactive radius cycling |
| `visualizationPreset` | ❌ Not mapped | Boolean — enables preset cycling |
| `bloomStrength` | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
| `bloomThreshold` | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
| `bloomRadius` | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
| `damp` (afterimage) | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
| `glow_intensity` | 🔲 Proposed (Controls Audit) | Canvas port equivalent of bloom/glow |
| `particle_size` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
| `drum_reactivity` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
| `hue_range` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
| `ellipse_ratio` | 🔲 Proposed (Controls Audit) | Canvas port; maps to `aperture` at geometry level |
| `rotation_speed` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
### Fluid — current StemEngine coverage vs. full GUI
| GUI parameter | In `VizSettings`? | Notes |
|---|---|---|
| `spawn_density` | ✅ `fluid.spawn_density` | Multiplier on ambient spawn probability |
| `splat_force` | ✅ `fluid.splat_force` | Multiplier on blob velocity/size |
| `decay` | ✅ `fluid.decay` | Multiplier on per-frame lifetime drain |
| `DENSITY_DISSIPATION` | ❌ Not mapped | Canvas sim uses `decay` instead; relevant for WebGL port |
| `VELOCITY_DISSIPATION` | ❌ Not mapped | Canvas sim uses `drag`; relevant for WebGL port |
| `PRESSURE` | ❌ Not mapped | WebGL-only; no canvas equivalent |
| `CURL` | ❌ Not mapped | Vorticity; WebGL-only |
| `SPLAT_RADIUS` | ❌ Not mapped | Canvas sim uses blob `radius` (implicit in force) |
| `SPLAT_FORCE` | ❌ Not mapped | Equivalent to `splat_force` — already covered |
| `SHADING` | ❌ Not mapped | WebGL-only post-processing |
| `COLORFUL` | ❌ Not mapped | Colour cycling — always on in canvas sim |
| `COLOR_UPDATE_SPEED` | ❌ Not mapped | Canvas sim cycles through `speakerPalette` sequentially |
| `BLOOM` | ❌ Not mapped | WebGL-only |
| `BLOOM_INTENSITY` | ❌ Not mapped | WebGL-only |
| `BLOOM_THRESHOLD` | ❌ Not mapped | WebGL-only |
| `SUNRAYS` | ❌ Not mapped | WebGL-only |
| `SUNRAYS_WEIGHT` | ❌ Not mapped | WebGL-only |
| `BACK_COLOR` | ❌ Not mapped | Canvas sim uses palette `backgroundColor` |
| `background_persistence` | 🔲 Proposed (Controls Audit) | Canvas-specific: fill-alpha trail control |
| `beat_burst_count` | 🔲 Proposed (Controls Audit) | Canvas-specific |
| `treble_pulse` | 🔲 Proposed (Controls Audit) | Canvas-specific |
| `drag` | 🔲 Proposed (Controls Audit) | Canvas equivalent of `VELOCITY_DISSIPATION` |
| `blob_life` | 🔲 Proposed (Controls Audit) | Canvas-specific |
---
## Part 4 — Parameters worth adding to `VizSettings` (from original GUI, not yet proposed)
These are original GUI parameters that were not covered in `2026-04-08_Visualizer_Controls_Audit.md`
and are relevant to StemEngine's canvas-based port.
### Muon additions
| Original key | Suggested `VizSettings` key | Type | Range | Default | Notes |
|---|---|---|---|---|---|
| `idleMultiplier` | `muon.idle_speed` | float | 0.0–2.0 | 1.0 | Multiplier on the current `0.27` default. Controls how fast the spiral animates when no audio is playing. |
| `noiseScale` | `muon.noise_scale` | float | 0.0–3.0 | 1.0 | Multiplier on the current `0.3` default. Higher = more spatial irregularity in the spiral. |
| `noiseForce` | `muon.noise_force` | float | 0.0–3.0 | 1.0 | Multiplier on `noiseForce`. Higher = noisier particle velocity. |
| `colorSpectrum` | `muon.color_segments` | int | 3–30 | 3 | Direct value (steps of 3). Number of colour bands in spiral. More = richer colour banding. |
| `enableMonoColor` | `muon.mono_color` | boolean | — | false | Locks colour to a fixed value rather than audio-driven hue cycling. |
| `lifespan` | `muon.particle_lifespan` | float | 0.25–3.0 | 1.0 | Multiplier on `200 ms` default. Lower = faster particle turnover. |
| `particleMirror` | `muon.mirror` | boolean | — | true | Mirrors the particle system across the spiral axis. |
### Fluid additions
| Original key | Suggested `VizSettings` key | Type | Notes |
|---|---|---|---|
| `CURL` | `fluid.curl` | float (0.25–3.0 ×) | Multiplier on `30` default. Higher = more swirling vortex motion in WebGL port. Not applicable to current canvas sim. |
| `COLORFUL` / `COLOR_UPDATE_SPEED` | `fluid.color_speed` | float (0.0–3.0 ×) | Multiplier on palette cycling speed. 0 = colour locked after first spawn; 3 = rapid cycling. |
> Most other Fluid GUI parameters (`BLOOM_*`, `SUNRAYS_*`, `PRESSURE`, `SHADING`) are WebGL-only
> and do not apply to the current canvas blob renderer in StemEngine. They become relevant only
> if the full `PavelDoGreat/WebGL-Fluid-Simulation` JS source is vendored.
+105
View File
@@ -2460,3 +2460,108 @@ Created a detailed integration plan for adding two optional visual backdrops to
- Muon-Audio-Analysis is not a library — plan extracts its core dependency (Meyda.js) directly.
Implementation is 10 ordered steps. Phase 7 (per-song beat-grid pre-analysis via backend pipeline) is deferred.
### Visualizer parameter audit & fixes — `2026-04-08_Visualizer_Parameter_Audit.md`
Audited both visualizer sources against the integration plan. Found three bugs in the original muon-music-visualizer source and five incorrect assumptions in the integration plan.
**Muon source bugs (all must be fixed in the StemEngine port):**
1. **`hueControl` always returns 0 during playback** (`effects.js`): `!scaler && time + scaler * 10` short-circuits to `false` (= 0) whenever scaler is non-zero, freezing hue at red. Fix: `scaler > 0 ? scaler * 10 : time`.
2. **`baseFr`/`trebleFr` are swapped** (`audio.js`): lower half (bass) is assigned to `trebleFr` and upper half (treble) to `baseFr`, so kick drums drive the smaller 3.14× coefficient in `beatScaler` instead of the larger 6.24× one. Fix: swap the `modulate` inputs.
3. **`coreScaler` amplification is dead code** (`audio.js` line 24): `coreScaler < 10 && coreScaler * 1.2` never assigns the result. Fix: `if (coreScaler < 10) coreScaler *= 1.2`.
**Integration plan corrections:**
4. **`colorSpectrum` range is 3–30 (step 3), not 0–360** — it's a colour segment count, not a hue angle. Driving it with `treble * 120` would corrupt geometry. Hue is controlled by `hueControl()` instead.
5. **`radiusMultiplier` should not be directly audio-mapped** — it's GSAP-animated by `wavePresetController` cycling through `visualizationPresets[]`. Tune `deltaResponseLimit` instead.
6. **`DENSITY_DISSIPATION` mapping was inverted** — lower value = longer trails (counterintuitive). Corrected formula: `lerp(0.90, 0.98, 1 - bass)`.
7. **`generateColor()` multiplies by 0.15**, producing near-invisible splats. Use `hsvToRgb` at full brightness for audio splats.
8. **Direct `splat()` calls bypass aspect-ratio correction** — apply `dx / aspectRatio` before calling `splat()`.
Document includes the definitive `manifest.json` parameter lists for both visualizers (15 Muon keys, 13 Fluid keys) with types, ranges, defaults, and notes on what is/isn't stored.
### Performance sidebar plan — `2026-04-08_PerformanceSidebar_Plan.md`
Designed a slide-in sidebar accessible during both `KaraokePerformanceMode` (TV view) and fullscreen `LyricsLivePreview`. All relevant state lives inside `LyricsLivePreview` so no state lifting is needed.
**Trigger:** Persistent floating tab at the right edge of the screen, always visible when `isPerformanceView` is true. Uses a `SlidersHorizontal` icon (lucide-react). Click or `Tab` key toggles `showPerfSidebar`.
**New component:** `frontend/src/components/PerformanceSidebar.tsx` — absolute right-side drawer, 280 px wide, `backdrop-blur bg-black/70`, three collapsible sections.
**Section 1 — Background:** Icon grid for `Static / Muon / Fluid / Video` (Video only shown when `hasVideo` is true). Replaces the renderer buttons hidden by `performanceMode`.
**Section 2 — Stems:** Per-stem mute (M), solo (S), and gain slider (0–200%). Direct port of `renderStems()`. Leading vocal stems matching `/^(lead|lead\s|leading|vocals?|vox|voice|main\s*vocal)/i` are auto-muted on first use in performance mode (only when no saved `mutedStems` prefs exist for the `storageKey`).
**Section 3 — Lyrics:** Condensed `StylePicker` subset: font family/size/bold, align, vertical position, line spacing, text/highlight/bg colours, X/Y offsets.
**Viewport exclusion:** `showPerfSidebar` added to the `containerWidth` dep array; 280 px subtracted from canvas width calculation when sidebar is open so the visualizer doesn't render behind the panel.
**`KaraokePerformanceMode` changes:** None — sidebar lives entirely in `LyricsLivePreview`; no `Tab` key conflict (KPM only handles Esc/Space/Enter/Arrows).
9 implementation steps documented in the plan file.
### Visualizer controls audit — `2026-04-08_Visualizer_Controls_Audit.md`
Full audit of every expression that changes during playback in both visualizer canvases. Documented
all hardcoded coefficients, proposed new manifest keys and sidebar controls, and defined named
presets for each visualizer.
**What was hardcoded and is now proposed as a control:**
*Muon* (8 new keys on top of the 3 existing multipliers):
- `particle_count` (int 40–500, default 220) — particle density and GPU cost
- `rotation_speed` (×0.2–4.0) — multiplies `pt.speed` base and drum rotation bonus
- `glow_intensity` (×0.0–3.0) — multiplies `shadowBlur = 14 + drum * 28`
- `particle_size` (×0.25–3.0) — multiplies `size = 1.75 + drum * 3.0`
- `drum_reactivity` (×0.0–2.0) — scales the `drum` signal before it feeds aperture/spacing/hue/alpha
- `hue_range` (×0.0–3.0) — multiplies `drum * 20` and `treble * 15` hue drift coefficients
- `beat_sensitivity` (×0.2–3.0) — divides the 200 ms beat-preset cooldown (higher = faster cycling)
- `ellipse_ratio` (0.30–1.0, default 0.62) — direct override of y-axis squash; 1.0 = circle
*Fluid* (5 new keys on top of the 3 existing multipliers):
- `background_persistence` (×0.2–3.0) — scales `(1 - bass) * 0.16` fill-alpha; the primary trail-length lever
- `beat_burst_count` (×0.25–3.0) — multiplies `3 + floor(bass * 4)` blobs per beat
- `treble_pulse` (×0.0–3.0) — multiplies the `treble * 0.25` blob swell factor
- `drag` (×0.5–2.0) — scales velocity retention (higher = more viscous)
- `blob_life` (×0.25–3.0) — multiplies initial blob life `0.7 + strength * 0.6`
**Named presets** defined in `PerformanceSidebar.tsx` (not stored in manifest.json):
- Muon: Default, Calm, Energetic, Dreamy, Minimal, Cosmic
- Fluid: Default, Ink, Foam, Lava, Mist, Storm
A "Custom" pseudo-preset shows when any slider deviates from the active preset. Presets populate all sliders for their respective visualizer via `patchVizSettings`.
Sidebar layout updated to show preset pills above sliders. Backend normaliser does not need changes for sidebar-only live adjustment (new keys are optional and default to 1.0).
### Visualizer GUI parameter map — `2026-04-09_Visualizer_GUI_Parameter_Map.md`
Exhaustive map of every user-adjustable parameter in both original visualizer GUIs, sourced directly from their GitHub repositories and live sites.
**Muon** (muon-vis.netlify.app / najafmohammed/muon-music-visualizer):
- 5 Particle Controls folder params: `spacing` (0–1), `maxPoints` (0–12240 step 360), `colorSpectrum` (3–30 step 3, colour band count not hue angle), `aperture` (0–π), `particleMirror` (bool)
- 3 Controls folder params: `dynamicRadius` (bool), `visualizationPreset` (bool), `idleMultiplier` (0–2, idle animation speed)
- 1 Audio Controls param: `deltaResponseLimit` (0.001–0.01, beat sensitivity threshold)
- 4 Emission Controls params: `enableMonoColor` (bool), `divisions` (1–150, frequency bin count), `lifespan` (10–250 ms), `monoColor` (HSV color picker)
- 2 Noise Controls params: `noiseScale` (0–1), `noiseForce` (0–1)
- Root: `radiusMultiplier` (0.01–1, default 0.66), `sideView` button, `reset` button
- Post-processing (hardcoded, not in GUI): bloom strength 0.55 / threshold 0.35 / radius 0.6; afterimage damp 0.86
**Fluid** (paveldogreat.github.io/WebGL-Fluid-Simulation):
- Main panel: `DYE_RESOLUTION` (quality dropdown), `SIM_RESOLUTION` (32–256 dropdown), `DENSITY_DISSIPATION` (0–4, higher = faster fade), `VELOCITY_DISSIPATION` (0–4), `PRESSURE` (0–1), `CURL` (0–50 vorticity), `SPLAT_RADIUS` (0.01–1), `SHADING` (bool), `COLORFUL` (bool), `PAUSED` (bool), random splats button
- Bloom folder: `BLOOM` (bool), `BLOOM_INTENSITY` (0.1–2.0), `BLOOM_THRESHOLD` (0–1)
- Sunrays folder: `SUNRAYS` (bool), `SUNRAYS_WEIGHT` (0.3–1.0)
- Capture folder: `BACK_COLOR` (color picker), `TRANSPARENT` (bool), screenshot button
Cross-reference table shows which GUI params are already in `VizSettings`, which were proposed in the Controls Audit, and which are new (7 additional Muon keys + 2 Fluid keys proposed for `VizSettings`). Most Fluid WebGL params (bloom, sunrays, pressure, shading) are not applicable to the current canvas blob renderer.
### Performance sidebar bug fixes — documented in `2026-04-08_PerformanceSidebar_Plan.md`
Three bugs identified in the existing implementation and their fixes documented.
**Bug 1a — Visualizer mode buttons (Fluid, Muon, Video) don't switch (`LyricsLivePreview.tsx`):**
The `enforcedVisualizerMode` effect includes `visualStyle` in its dep array. Every time the sidebar calls `setVisualStyle('fluid')`, the effect fires, sees `visualStyle !== enforcedVisualizerMode`, and immediately calls `setVisualStyle(enforcedVisualizerMode)` to revert. Fix: add `userOverrodeVisualizerRef` (a ref, not state); wrap `setVisualStyle` in `setVisualStyleFromSidebar` which sets the ref; update the effect to bail when the ref is set; remove `visualStyle` from the dep array so the effect only runs on mount and song change.
**Bug 1b — Video mode doesn't start playing when switched mid-song (`LyricsLivePreview.tsx`):**
`<video>` only mounts when `hasVideoBackdrop` is true. When `visualStyle` switches to `'video'` while `isPlaying`, the video element mounts fresh but paused — `togglePlay` is never called, so `videoRef.current.play()` and `setVideoStartPhase('playing')` don't fire. Fix: add `useEffect` on `[hasVideoBackdrop, isPlaying]` that, when both are true and `videoStartPhase !== 'playing'`, seeks the video to `currentTimeRef.current`, calls `play()`, and sets the phase.
**Bug 2 — Seek slider snaps to 0 after dragging (`LyricsLivePreview.tsx`):**
`endSeekDrag` sets `isSeekingRef.current = false` synchronously after calling `applySeekPosition`. The RAF loop fires before the browser's audio `seeked` event, reads `audio.currentTime` (still returning the pre-seek value because browser seek is async), calls `setCurrentTime(0)`, then the `setTimeout(0)` clears `seekDragTime` — leaving the slider at 0. Fix: defer `isSeekingRef.current = false` until the audio element fires `seeked` (with a 500 ms safety fallback via `seekLockFallbackRef`). `beginSeekDrag` cancels any pending fallback timer from a prior drag.
@@ -236,24 +236,39 @@ export default function MuonVisualizerCanvas({ featuresRef, width, height, ...pa
### Drum-intensity → Controls mapping
> **⚠️ Corrections applied — see 2026-04-08_Visualizer_Parameter_Audit.md Corrections A, B.**
```ts
// Each RAF frame inside the Three.js loop:
const { drumIntensity = 0, treble = 0, beatDetected = false } = featuresRef.current
const palette = paletteRef.current
const baseHue = palette.seedHue
// "Controls" params driven by drum track:
params.radiusMultiplier = lerp(0.4, 1.8, easeIn(drumIntensity))
params.aperture = lerp(1.2, 3.5, drumIntensity)
params.spacing = lerp(0.008, 0.02, 1 - drumIntensity) // tighter on loud beats
// Correction B: radiusMultiplier is NOT mapped directly from drumIntensity each frame.
// It advances through RADIUS_PRESETS on each beat and lerps toward the new target.
// Direct per-frame lerp(0.4, 1.8, ...) fought the GSAP preset animation and exceeded
// the preset range (0.6603–0.7536 in the original source).
if (beatDetected) {
presetIdx = (presetIdx + 1) % RADIUS_PRESETS.length
radiusTarget = RADIUS_PRESETS[presetIdx] + drumIntensity * 0.25
}
radiusCurrent += (radiusTarget - radiusCurrent) * 0.07 // lerp toward target each frame
params.aperture = lerp(1.2, 3.5, drumIntensity)
params.spacing = lerp(0.008, 0.02, 1 - drumIntensity) // tighter on loud beats
// Correction A: colorSpectrum is a colour-band count (3–30, step 3), NOT a hue angle.
// Hue drift is controlled separately via hueControl() / seedHue offset.
// Bug 1 fix: when drumIntensity > 0 (audio playing), hue is modulated by drum + treble;
// when silent it falls back to slow time-based drift so the visual never freezes.
const hue = drumIntensity > 0.01
? ((palette.seedHue + drumIntensity * 20 + treble * 15) + 360) % 360 // ±15° max treble drift
: ((palette.seedHue + Math.sin(Date.now() * 0.00018) * 15) + 360) % 360
// Color stays in manifest palette family; treble adds controlled drift:
params.colorSpectrum = (baseHue + treble * 40) % 360
params.primaryColor = palette.highlightColor
params.secondaryColor = palette.wordHighlightColor
// Beat → preset morph:
if (beatDetected) triggerPresetTransition()
// Beat → preset morph (already handled above via radiusCurrent/radiusTarget)
```
`lerp` and `easeIn` are trivial helpers; no additional deps needed.
@@ -349,8 +364,11 @@ if (beatDetected) {
}
// Config modulation:
fluidConfig.DENSITY_DISSIPATION = lerp(0.97, 0.999, 1 - bass) // longer trails on loud bass
fluidConfig.VELOCITY_DISSIPATION = lerp(0.94, 0.99, 1 - rms)
// Correction C: mapping is driven by bass (not rms) so kick hits produce longer trails.
// ⚠️ Direction caveat: if the vendored sim uses density *= uDissipation (retention
// multiplier), LOWER values = faster fading. Verify advection shader before porting.
fluidConfig.DENSITY_DISSIPATION = lerp(0.90, 0.98, 1 - bass) // loud bass → 0.90 → longer trails
fluidConfig.VELOCITY_DISSIPATION = lerp(0.85, 0.99, 1 - rms) // loud rms → more momentum
fluidConfig.COLOR_UPDATE_SPEED = lerp(2, 15, treble)
```
File renamed without changes.
@@ -22,6 +22,20 @@ interface KaraokePerformanceModeProps {
thumbnailUrl?: string
sekDynamicPalette?: SekDynamicPalette | null
sekVisualizerMode?: 'muon' | 'fluid' | 'auto' | null
sekVisualizerSettings?: {
quality?: string
fps_cap?: number
muon?: {
radius_multiplier?: number
aperture?: number
spacing?: number
}
fluid?: {
spawn_density?: number
splat_force?: number
decay?: number
}
} | null
artist?: string
songTitle?: string
upNextSong?: { title: string; artist?: string } | null
@@ -51,6 +65,7 @@ export default function KaraokePerformanceMode({
thumbnailUrl,
sekDynamicPalette,
sekVisualizerMode,
sekVisualizerSettings,
artist,
songTitle,
disableGlobalPlaybackRegistry = false,
@@ -120,6 +135,7 @@ export default function KaraokePerformanceMode({
thumbnailUrl={thumbnailUrl}
sekDynamicPalette={sekDynamicPalette}
sekVisualizerMode={sekVisualizerMode}
sekVisualizerSettings={sekVisualizerSettings}
artist={artist}
songTitle={songTitle}
performanceMode
+282 -9
View File
@@ -2,7 +2,7 @@
import { useState, useEffect, useRef, useCallback, useMemo } from 'react'
import Two from 'two.js'
import { VIDEO_ANIM_REGISTRY, VIDEO_TRANSITION_REGISTRY, type VideoAnimStyle, type VideoTimingGranularity, type VideoLineTransition } from '../utils/videoAnimations'
import { Play, Pause, SkipBack, Volume2, VolumeX, Palette, Pencil, X, AlignLeft, AlignCenter, AlignRight, Type, Maximize2, Minimize2, Activity } from 'lucide-react'
import { Play, Pause, SkipBack, Volume2, VolumeX, Palette, Pencil, X, AlignLeft, AlignCenter, AlignRight, Type, Maximize2, Minimize2, Activity, SlidersHorizontal } from 'lucide-react'
import type { LyricsStyleOptions } from '../types'
import { registerPlayback, unregisterPlayback, publishMeta, clearMeta } from '../utils/audioRegistry'
@@ -13,6 +13,7 @@ import { useVisualizerPalette } from '../hooks/useVisualizerPalette'
import { useAudioFeatures } from '../hooks/useAudioFeatures'
import type { SekDynamicPalette } from '../api/client'
import VisualizerBackdrop from './visualizers/VisualizerBackdrop'
import PerformanceSidebar from './PerformanceSidebar.tsx'
// ─── Types ───────────────────────────────────────────────────────────────────
@@ -47,6 +48,83 @@ interface PreviewPlaybackPrefs {
export type LyricsVisualStyle = 'scroll' | 'fade' | 'karaoke' | 'glow' | 'blur' | 'video' | 'muon' | 'fluid'
type VideoStartPhase = 'idle' | 'priming' | 'playing'
type DeepPartial<T> = {
[K in keyof T]?: T[K] extends object ? DeepPartial<T[K]> : T[K]
}
interface VizSettings {
quality: 'low' | 'balanced' | 'high'
fps_cap: 30 | 60 | 120
muon: {
radius_multiplier: number
aperture: number
spacing: number
}
fluid: {
spawn_density: number
splat_force: number
decay: number
}
}
const VIZ_DEFAULTS: VizSettings = {
quality: 'balanced',
fps_cap: 60,
muon: { radius_multiplier: 1.0, aperture: 1.0, spacing: 1.0 },
fluid: { spawn_density: 1.0, splat_force: 1.0, decay: 1.0 },
}
const PERF_SIDEBAR_WIDTH = 280
const LEADING_VOCAL_PATTERN = /^(lead|lead\s|leading|vocals?|vox|voice|main\s*vocal)/i
function isLeadingVocalStem(name: string): boolean {
return LEADING_VOCAL_PATTERN.test((name || '').trim())
}
function clampVizMultiplier(value: number): number {
if (!Number.isFinite(value)) return 1
return Math.max(0.25, Math.min(2, value))
}
function deepMergeVizSettings(base: VizSettings, patch?: DeepPartial<VizSettings> | null): VizSettings {
if (!patch) {
return {
...base,
muon: { ...base.muon },
fluid: { ...base.fluid },
}
}
return {
...base,
...patch,
muon: {
...base.muon,
...(patch.muon ?? {}),
},
fluid: {
...base.fluid,
...(patch.fluid ?? {}),
},
}
}
function normalizeVizSettings(input: VizSettings): VizSettings {
return {
quality: input.quality === 'low' || input.quality === 'high' ? input.quality : 'balanced',
fps_cap: input.fps_cap === 30 || input.fps_cap === 120 ? input.fps_cap : 60,
muon: {
radius_multiplier: clampVizMultiplier(input.muon.radius_multiplier),
aperture: clampVizMultiplier(input.muon.aperture),
spacing: clampVizMultiplier(input.muon.spacing),
},
fluid: {
spawn_density: clampVizMultiplier(input.fluid.spawn_density),
splat_force: clampVizMultiplier(input.fluid.splat_force),
decay: clampVizMultiplier(input.fluid.decay),
},
}
}
interface Props {
lrcText: string
lyricsTracks?: Record<string, string>
@@ -69,6 +147,21 @@ interface Props {
sekDynamicPalette?: SekDynamicPalette | null
/** Optional SEK visualizer mode preference (manifest.visualizer.mode). */
sekVisualizerMode?: 'muon' | 'fluid' | 'auto' | null
/** Optional SEK visualizer settings (manifest.visualizer.settings). */
sekVisualizerSettings?: {
quality?: string
fps_cap?: number
muon?: {
radius_multiplier?: number
aperture?: number
spacing?: number
}
fluid?: {
spawn_density?: number
splat_force?: number
decay?: number
}
} | null
/** Artist name from source audio metadata. */
artist?: string
/** Track title from source audio metadata — used in the visual overlay only. */
@@ -1336,7 +1429,7 @@ function LrcEditor({
// ─── Main component ───────────────────────────────────────────────────────────
export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, videoUrl, videoPlaylist, videoPlaybackMode, stems, title, styleOptions, defaultStyle, onLrcChange, storageKey, coverArtUrl, thumbnailUrl, sekDynamicPalette, sekVisualizerMode, artist, songTitle, performanceMode = false, autoPlay = false, disableGlobalPlaybackRegistry = false, onPlaybackEnded }: Props) {
export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, videoUrl, videoPlaylist, videoPlaybackMode, stems, title, styleOptions, defaultStyle, onLrcChange, storageKey, coverArtUrl, thumbnailUrl, sekDynamicPalette, sekVisualizerMode, sekVisualizerSettings, artist, songTitle, performanceMode = false, autoPlay = false, disableGlobalPlaybackRegistry = false, onPlaybackEnded }: Props) {
const resolvedLyricsTracks = useMemo(() => {
const normalized: Record<string, string> = {}
for (const [rawKey, rawText] of Object.entries(lyricsTracks || {})) {
@@ -1418,12 +1511,47 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
(sekVisualizerMode === 'muon' || sekVisualizerMode === 'fluid')
? sekVisualizerMode
: ((defaultStyle === 'muon' || defaultStyle === 'fluid') ? defaultStyle : null)
const userOverrodeVisualizerRef = useRef(false)
const [showPerfSidebar, setShowPerfSidebar] = useState(false)
const manifestVizSettings = useMemo<VizSettings | null>(() => {
if (!sekVisualizerSettings) return null
const quality = sekVisualizerSettings.quality
const fps = sekVisualizerSettings.fps_cap
const patch: DeepPartial<VizSettings> = {
quality: quality === 'low' || quality === 'balanced' || quality === 'high' ? quality : undefined,
fps_cap: fps === 30 || fps === 60 || fps === 120 ? fps : undefined,
muon: {
radius_multiplier: sekVisualizerSettings.muon?.radius_multiplier,
aperture: sekVisualizerSettings.muon?.aperture,
spacing: sekVisualizerSettings.muon?.spacing,
},
fluid: {
spawn_density: sekVisualizerSettings.fluid?.spawn_density,
splat_force: sekVisualizerSettings.fluid?.splat_force,
decay: sekVisualizerSettings.fluid?.decay,
},
}
return normalizeVizSettings(deepMergeVizSettings(VIZ_DEFAULTS, patch))
}, [sekVisualizerSettings])
const [liveVizSettings, setLiveVizSettings] = useState<VizSettings>(() => (
manifestVizSettings ? deepMergeVizSettings(manifestVizSettings) : deepMergeVizSettings(VIZ_DEFAULTS)
))
const patchVizSettings = useCallback((patch: DeepPartial<VizSettings>) => {
setLiveVizSettings((prev) => normalizeVizSettings(deepMergeVizSettings(prev, patch)))
}, [])
useEffect(() => {
setLiveVizSettings(manifestVizSettings ? deepMergeVizSettings(manifestVizSettings) : deepMergeVizSettings(VIZ_DEFAULTS))
}, [previewPrefsKey, manifestVizSettings])
useEffect(() => {
if (!enforcedVisualizerMode) return
if (visualStyle === enforcedVisualizerMode) return
setVisualStyle(enforcedVisualizerMode)
}, [enforcedVisualizerMode, visualStyle])
if (userOverrodeVisualizerRef.current) return
setVisualStyle((prev) => (prev === enforcedVisualizerMode ? prev : enforcedVisualizerMode))
}, [enforcedVisualizerMode])
useEffect(() => {
userOverrodeVisualizerRef.current = false
}, [previewPrefsKey, enforcedVisualizerMode])
const [currentTime, setCurrentTime] = useState(0)
const [seekDragTime, setSeekDragTime] = useState<number | null>(null)
@@ -1506,6 +1634,12 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
desiredVideoSegmentRef.current = resolvedVideoPlaylist.length > 0 ? { index: 0, localTime: 0 } : null
}, [resolvedVideoPlaylist])
useEffect(() => {
if (visualStyle !== 'video') return
if (videoUrl || resolvedVideoPlaylist.length > 0) return
setVisualStyle('karaoke')
}, [visualStyle, videoUrl, resolvedVideoPlaylist])
// Track whether cover art image loaded successfully
const [artLoaded, setArtLoaded] = useState(false)
@@ -1597,6 +1731,8 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
const [mutedStems, setMutedStems] = useState<Record<string, boolean>>(() => initialPreviewPrefs.mutedStems ?? {})
const [soloedStems, setSoloedStems] = useState<Record<string, boolean>>(() => initialPreviewPrefs.soloedStems ?? {})
const previewPrefsHydratingRef = useRef(true)
const perfLeadingMuteAppliedRef = useRef(false)
const hasSavedMutedStemPrefs = Object.keys(initialPreviewPrefs.mutedStems ?? {}).length > 0
// User-overrideable speaker colours (keyed by speaker ID e.g. "SPEAKER_00")
const [speakerColors, setSpeakerColors] = useState<Record<string, string>>(() => {
@@ -1642,12 +1778,19 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
const gainRefs = useRef<Record<string, GainNode>>({})
const metadataListenerRefs = useRef<Record<string, () => void>>({})
// Important: on first mount in multi-stem mode, `useAudioFeatures` can run
// before the stem graph is created (`gainRefs.current` still empty). In that
// case the hook exits early and would not reconnect until one of its deps
// changes, causing static/non-reactive visualizers. Gate it until tap nodes
// exist so the first enabled run always has a valid audio source.
const audioFeatureTapReady = !isMultiStem || Object.keys(gainRefs.current).length > 0
const featuresRef = useAudioFeatures({
audioCtxRef,
stemGainRefs: gainRefs,
singleAudioRef: audioRef,
isPlayingRef,
enabled: isVisualizerMode,
enabled: isVisualizerMode && audioFeatureTapReady,
})
const containerRef = useRef<HTMLDivElement>(null)
@@ -1657,6 +1800,7 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
const isSeekingRef = useRef(false)
const pendingSeekRef = useRef<number | null>(null)
const clearSeekUiTimerRef = useRef<number | null>(null)
const seekLockFallbackRef = useRef<number | null>(null)
// Global floating bar tracking
const isActiveRef = useRef(false)
@@ -1672,6 +1816,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
isActiveRef.current = false
isSeekingRef.current = false
pendingSeekRef.current = null
if (seekLockFallbackRef.current !== null) {
window.clearTimeout(seekLockFallbackRef.current)
seekLockFallbackRef.current = null
}
if (clearSeekUiTimerRef.current !== null) {
window.clearTimeout(clearSeekUiTimerRef.current)
clearSeekUiTimerRef.current = null
@@ -1694,6 +1842,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
isActiveRef.current = false
isSeekingRef.current = false
pendingSeekRef.current = null
if (seekLockFallbackRef.current !== null) {
window.clearTimeout(seekLockFallbackRef.current)
seekLockFallbackRef.current = null
}
if (clearSeekUiTimerRef.current !== null) {
window.clearTimeout(clearSeekUiTimerRef.current)
clearSeekUiTimerRef.current = null
@@ -1910,6 +2062,22 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
})
}, [volume, isMuted, mutedStems, soloedStems, gains])
useEffect(() => {
if (!performanceMode || !isMultiStem || !stems?.length) return
if (hasSavedMutedStemPrefs || perfLeadingMuteAppliedRef.current) return
const defaultMutes: Record<string, boolean> = {}
for (const stem of stems) {
if (isLeadingVocalStem(stem.name)) defaultMutes[stem.name] = true
}
perfLeadingMuteAppliedRef.current = true
if (!Object.keys(defaultMutes).length) return
setMutedStems(defaultMutes)
applyGains({ mutedStems: defaultMutes })
}, [performanceMode, isMultiStem, stems, hasSavedMutedStemPrefs, applyGains])
// ── Initialize stem audio elements ──────────────────────────────────────
useEffect(() => {
if (!isMultiStem || !stems) {
@@ -2175,6 +2343,25 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
const [lyricsViewportScale, setLyricsViewportScale] = useState(1)
const [visualizerSize, setVisualizerSize] = useState({ width: 960, height: 540 })
useEffect(() => {
if (!isPerformanceView) {
setShowPerfSidebar(false)
return
}
const onKeyDown = (event: KeyboardEvent) => {
const target = event.target as HTMLElement | null
if (target && (target.tagName === 'INPUT' || target.tagName === 'TEXTAREA' || target.tagName === 'SELECT' || target.isContentEditable)) {
return
}
if ((event.key === 'q' || event.key === 'Q') && !event.metaKey && !event.ctrlKey && !event.altKey) {
event.preventDefault()
setShowPerfSidebar((prev) => !prev)
}
}
window.addEventListener('keydown', onKeyDown)
return () => window.removeEventListener('keydown', onKeyDown)
}, [isPerformanceView])
// Scale lyric font relative to the display canvas size (VLC/Jellyfin style behavior)
// so text grows/shrinks with the window instead of staying fixed.
const effectiveViewportScale = isPerformanceView
@@ -2214,7 +2401,8 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
const compute = () => {
const rect = node.getBoundingClientRect()
const width = Math.max(1, rect.width)
const sidebarOffset = isPerformanceView && showPerfSidebar && !isNarrow ? PERF_SIDEBAR_WIDTH : 0
const width = Math.max(1, rect.width - sidebarOffset)
const height = Math.max(1, rect.height)
// 960x540 reference viewport (16:9)
const scale = Math.min(width / 960, height / 540)
@@ -2232,7 +2420,7 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
window.addEventListener('resize', compute)
return () => window.removeEventListener('resize', compute)
}, [isFullscreen, performanceMode, showStylePicker, showEditor, showAegisub])
}, [isFullscreen, performanceMode, showStylePicker, showEditor, showAegisub, showPerfSidebar, isNarrow, isPerformanceView])
function toggleFullscreen() {
if (!isFullscreen) fullscreenRef.current?.requestFullscreen()
@@ -2504,6 +2692,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
}
function beginSeekDrag() {
if (seekLockFallbackRef.current !== null) {
window.clearTimeout(seekLockFallbackRef.current)
seekLockFallbackRef.current = null
}
if (clearSeekUiTimerRef.current !== null) {
window.clearTimeout(clearSeekUiTimerRef.current)
clearSeekUiTimerRef.current = null
@@ -2528,7 +2720,29 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
applySeekPosition(finalValue)
}
}
isSeekingRef.current = false
const releaseSeekLock = () => {
if (seekLockFallbackRef.current !== null) {
window.clearTimeout(seekLockFallbackRef.current)
seekLockFallbackRef.current = null
}
isSeekingRef.current = false
}
const targetAudio = isMultiStem
? (Object.values(stemAudioRefs.current)[0] ?? null)
: audioRef.current
if (targetAudio) {
targetAudio.addEventListener('seeked', releaseSeekLock, { once: true })
if (seekLockFallbackRef.current !== null) {
window.clearTimeout(seekLockFallbackRef.current)
}
seekLockFallbackRef.current = window.setTimeout(releaseSeekLock, 500)
} else {
releaseSeekLock()
}
if (clearSeekUiTimerRef.current !== null) {
window.clearTimeout(clearSeekUiTimerRef.current)
}
@@ -2541,6 +2755,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
function handleReset() {
isSeekingRef.current = false
pendingSeekRef.current = null
if (seekLockFallbackRef.current !== null) {
window.clearTimeout(seekLockFallbackRef.current)
seekLockFallbackRef.current = null
}
if (clearSeekUiTimerRef.current !== null) {
window.clearTimeout(clearSeekUiTimerRef.current)
clearSeekUiTimerRef.current = null
@@ -2671,6 +2889,23 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
else v.pause()
}, [isPlaying, hasVideoBackdrop, isPlaylistBackdropMode, currentVideoUrl])
// When switching to Video mode while already playing, ensure the freshly-mounted
// video element is started and synchronized to current transport immediately.
useEffect(() => {
if (!hasVideoBackdrop || !isPlaying) return
const v = videoRef.current
if (!v) return
if (videoStartPhase === 'playing') return
const t = currentTimeRef.current ?? 0
if (isPlaylistBackdropMode) {
syncPlaylistVideoToTransport(t)
} else if (Math.abs(v.currentTime - t) > 0.35) {
v.currentTime = t
}
v.play().catch((err) => console.warn('[LyricsLivePreview] video auto-start on mode switch failed', err))
setVideoStartPhase('playing')
}, [hasVideoBackdrop, isPlaying, isPlaylistBackdropMode, videoStartPhase, currentVideoUrl])
function handleVolumeChange(e: React.ChangeEvent<HTMLInputElement>) {
const v = parseFloat(e.target.value)
setVolume(v)
@@ -2805,6 +3040,7 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
paletteRef={paletteRef}
width={visualizerSize.width}
height={visualizerSize.height}
settings={liveVizSettings}
/>
</div>
)}
@@ -3397,6 +3633,43 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
</div>
)}
{isPerformanceView && isNarrow && (
<button
type="button"
onClick={() => setShowPerfSidebar((prev) => !prev)}
className={`absolute right-3 top-3 z-[10] flex h-11 w-11 items-center justify-center rounded-full border border-white/15 text-slate-100 shadow-[0_10px_28px_rgba(0,0,0,0.35)] backdrop-blur-md transition-colors ${showPerfSidebar ? 'bg-violet-600/80 hover:bg-violet-600' : 'bg-black/65 hover:bg-black/80'}`}
title={showPerfSidebar ? 'Close performance controls' : 'Open performance controls'}
aria-label={showPerfSidebar ? 'Close performance controls' : 'Open performance controls'}
>
<SlidersHorizontal className="h-4.5 w-4.5" />
</button>
)}
{isPerformanceView && showPerfSidebar && (
<PerformanceSidebar
isNarrow={isNarrow}
visualStyle={visualStyle}
setVisualStyle={(next) => {
userOverrodeVisualizerRef.current = true
setVisualStyle(next)
}}
hasVideo={!!(videoUrl || resolvedVideoPlaylist.length > 0)}
stems={stems}
mutedStems={mutedStems}
soloedStems={soloedStems}
gains={gains}
toggleStemMute={toggleStemMute}
toggleStemSolo={toggleStemSolo}
setStemGain={updateStemGain}
style={style}
patchStyle={patchStyle}
vizSettings={liveVizSettings}
manifestVizSettings={manifestVizSettings}
patchVizSettings={patchVizSettings}
onClose={() => setShowPerfSidebar(false)}
/>
)}
{/* Hidden single-track audio element */}
{!isMultiStem && audioUrl && (
<audio
@@ -0,0 +1,445 @@
import { useCallback, useEffect, useRef, useState } from 'react'
import type { LyricsVisualStyle } from './LyricsLivePreview'
interface Stem {
name: string
label: string
url: string
}
interface SidebarStyleState {
fontFamily: string
fontSize: number
fontBold: boolean
textAlign: 'left' | 'center' | 'right'
activeLineVerticalAlign: 'top' | 'center' | 'bottom'
lineSpacing: number
textColor: string
highlightColor: string
bgColor: string
videoDisplayMode: 'scroll' | 'line' | 'word' | 'syllable'
layoutOffsetXScroll: number
layoutOffsetYScroll: number
layoutOffsetXLine: number
layoutOffsetYLine: number
layoutOffsetXWord: number
layoutOffsetYWord: number
layoutOffsetXSyllable: number
layoutOffsetYSyllable: number
}
type DeepPartial<T> = {
[K in keyof T]?: T[K] extends object ? DeepPartial<T[K]> : T[K]
}
interface VizSettings {
quality: 'low' | 'balanced' | 'high'
fps_cap: 30 | 60 | 120
muon: {
radius_multiplier: number
aperture: number
spacing: number
}
fluid: {
spawn_density: number
splat_force: number
decay: number
}
}
const VIZ_DEFAULTS: VizSettings = {
quality: 'balanced',
fps_cap: 60,
muon: { radius_multiplier: 1, aperture: 1, spacing: 1 },
fluid: { spawn_density: 1, splat_force: 1, decay: 1 },
}
const FONT_OPTIONS = [
{ value: 'system-ui, sans-serif', label: 'System' },
{ value: '"Inter", sans-serif', label: 'Inter' },
{ value: '"Work Sans", sans-serif', label: 'Work Sans' },
{ value: '"Archivo", sans-serif', label: 'Archivo' },
{ value: '"Poppins", sans-serif', label: 'Poppins' },
{ value: '"Roboto", sans-serif', label: 'Roboto' },
{ value: '"Oswald", sans-serif', label: 'Oswald' },
{ value: '"EB Garamond", serif', label: 'EB Garamond' },
{ value: '"Libre Baskerville", serif', label: 'Libre Baskerville' },
{ value: '"Young Serif", serif', label: 'Young Serif' },
]
interface PerformanceSidebarProps {
isNarrow: boolean
visualStyle: LyricsVisualStyle
setVisualStyle: (s: LyricsVisualStyle) => void
hasVideo: boolean
stems: Stem[] | undefined
mutedStems: Record<string, boolean>
soloedStems: Record<string, boolean>
gains: Record<string, number>
toggleStemMute: (name: string) => void
toggleStemSolo: (name: string) => void
setStemGain: (name: string, value: number) => void
style: SidebarStyleState
patchStyle: (patch: Partial<SidebarStyleState>) => void
vizSettings: VizSettings
manifestVizSettings: VizSettings | null
patchVizSettings: (patch: DeepPartial<VizSettings>) => void
onClose: () => void
}
function formatMultiplier(value: number): string {
return `${value.toFixed(2)}×`
}
function getLayoutOffset(style: SidebarStyleState): { x: number; y: number } {
switch (style.videoDisplayMode) {
case 'line':
return { x: style.layoutOffsetXLine, y: style.layoutOffsetYLine }
case 'word':
return { x: style.layoutOffsetXWord, y: style.layoutOffsetYWord }
case 'syllable':
return { x: style.layoutOffsetXSyllable, y: style.layoutOffsetYSyllable }
case 'scroll':
default:
return { x: style.layoutOffsetXScroll, y: style.layoutOffsetYScroll }
}
}
export default function PerformanceSidebar({
isNarrow,
visualStyle,
setVisualStyle,
hasVideo,
stems,
mutedStems,
soloedStems,
gains,
toggleStemMute,
toggleStemSolo,
setStemGain,
style,
patchStyle,
vizSettings,
manifestVizSettings,
patchVizSettings,
onClose,
}: PerformanceSidebarProps) {
const [open, setOpen] = useState({
background: true,
stems: true,
lyrics: true,
visualizer: true,
})
const autoHideTimerRef = useRef<number | null>(null)
const scheduleAutoHide = useCallback(() => {
if (autoHideTimerRef.current !== null) {
window.clearTimeout(autoHideTimerRef.current)
}
autoHideTimerRef.current = window.setTimeout(() => {
onClose()
}, 4000)
}, [onClose])
useEffect(() => {
scheduleAutoHide()
return () => {
if (autoHideTimerRef.current !== null) {
window.clearTimeout(autoHideTimerRef.current)
}
}
}, [scheduleAutoHide])
const activeOffset = getLayoutOffset(style)
const isVisualizerMode = visualStyle === 'muon' || visualStyle === 'fluid'
const isStaticMode = visualStyle !== 'muon' && visualStyle !== 'fluid' && visualStyle !== 'video'
const patchOffset = (axis: 'x' | 'y', value: number) => {
if (style.videoDisplayMode === 'line') {
patchStyle(axis === 'x' ? { layoutOffsetXLine: value } : { layoutOffsetYLine: value })
return
}
if (style.videoDisplayMode === 'word') {
patchStyle(axis === 'x' ? { layoutOffsetXWord: value } : { layoutOffsetYWord: value })
return
}
if (style.videoDisplayMode === 'syllable') {
patchStyle(axis === 'x' ? { layoutOffsetXSyllable: value } : { layoutOffsetYSyllable: value })
return
}
patchStyle(axis === 'x' ? { layoutOffsetXScroll: value } : { layoutOffsetYScroll: value })
}
const sectionHeader = (key: keyof typeof open, label: string, hidden = false) => {
if (hidden) return null
return (
<button
type="button"
onClick={() => setOpen((prev) => ({ ...prev, [key]: !prev[key] }))}
className="w-full flex items-center justify-between text-xs font-semibold tracking-wide text-slate-200"
>
<span>{label}</span>
<span className="text-slate-400">{open[key] ? '▾' : '▸'}</span>
</button>
)
}
const panelClass = isNarrow
? 'absolute inset-x-0 bottom-0 h-[62vh] rounded-t-2xl border-t border-white/15'
: 'absolute right-0 top-0 h-full border-l border-white/15'
return (
<div
className={`${panelClass} z-[9] bg-black/70 backdrop-blur-md text-slate-100 overflow-y-auto`}
style={isNarrow ? undefined : { width: 280 }}
onMouseMove={scheduleAutoHide}
onPointerMove={scheduleAutoHide}
>
<div className="sticky top-0 z-10 px-3 py-2 border-b border-white/10 bg-black/75 backdrop-blur-md flex items-center justify-between">
<span className="text-xs font-semibold uppercase tracking-[0.08em] text-slate-300">Controls</span>
<button
type="button"
onClick={onClose}
className="w-7 h-7 rounded bg-white/10 hover:bg-white/20 text-slate-200 text-sm"
title="Close"
>
✕
</button>
</div>
<div className="px-3 py-3 space-y-4">
<section className="space-y-2">
{sectionHeader('background', 'BACKGROUND')}
{open.background && (
<div className={isNarrow ? 'flex flex-wrap gap-2' : 'grid grid-cols-2 gap-2'}>
{[
{ key: 'static', label: 'Static', value: 'karaoke' as LyricsVisualStyle, active: isStaticMode },
{ key: 'muon', label: 'Muon', value: 'muon' as LyricsVisualStyle, active: visualStyle === 'muon' },
{ key: 'fluid', label: 'Fluid', value: 'fluid' as LyricsVisualStyle, active: visualStyle === 'fluid' },
...(hasVideo ? [{ key: 'video', label: 'Video', value: 'video' as LyricsVisualStyle, active: visualStyle === 'video' }] : []),
].map((mode) => (
<button
key={mode.key}
type="button"
onClick={() => setVisualStyle(mode.value)}
className={`rounded-md px-2 py-2 text-xs border transition-colors ${isNarrow ? 'min-w-[70px] flex-1' : ''} ${mode.active ? 'border-violet-400 bg-violet-500/20 text-white' : 'border-white/15 bg-white/5 text-slate-300 hover:bg-white/10'}`}
>
{mode.label}
</button>
))}
</div>
)}
</section>
<section className="space-y-2">
{sectionHeader('stems', 'STEMS')}
{open.stems && (
<div className="space-y-2">
{stems?.length ? stems.map((stem) => (
<div key={stem.name} className="space-y-1.5">
<div className="flex items-center gap-2">
<span className="text-xs font-medium text-slate-300 flex-1 truncate">{stem.label}</span>
<button
type="button"
onClick={() => toggleStemMute(stem.name)}
className={`w-6 h-6 text-[10px] rounded ${mutedStems[stem.name] ? 'bg-red-500 text-white' : 'bg-white/10 text-slate-300'}`}
>
M
</button>
<button
type="button"
onClick={() => toggleStemSolo(stem.name)}
className={`w-6 h-6 text-[10px] rounded ${soloedStems[stem.name] ? 'bg-yellow-300 text-black' : 'bg-white/10 text-slate-300'}`}
>
S
</button>
</div>
<div className="flex items-center gap-2">
<input
type="range"
min={0}
max={2}
step={0.05}
value={gains[stem.name] ?? 1}
onChange={(event) => setStemGain(stem.name, parseFloat(event.target.value))}
className="w-full h-1.5 accent-brand-500"
/>
<span className="text-[10px] tabular-nums text-slate-400 w-11 text-right">{Math.round((gains[stem.name] ?? 1) * 100)}%</span>
</div>
</div>
)) : (
<p className="text-[11px] text-slate-400">No stems — single track mode</p>
)}
</div>
)}
</section>
<section className="space-y-2">
{sectionHeader('lyrics', 'LYRICS')}
{open.lyrics && (
<div className="space-y-2.5">
<div className="flex items-center gap-2">
<select
value={style.fontFamily}
onChange={(event) => patchStyle({ fontFamily: event.target.value })}
className="flex-1 text-xs bg-white/10 border border-white/15 rounded px-2 py-1"
>
{FONT_OPTIONS.map((font) => (
<option key={font.value} value={font.value} style={{ background: '#0f172a', color: '#e2e8f0', fontFamily: font.value }}>
{font.label}
</option>
))}
</select>
<button
type="button"
onClick={() => patchStyle({ fontBold: !style.fontBold })}
className={`px-2 py-1 rounded text-xs font-semibold ${style.fontBold ? 'bg-white/25 text-white' : 'bg-white/10 text-slate-300'}`}
>
B
</button>
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Size</span><span>{style.fontSize}</span></div>
<input type="range" min={12} max={56} step={1} value={style.fontSize} onChange={(event) => patchStyle({ fontSize: parseInt(event.target.value, 10) })} className="w-full h-1.5 accent-brand-500" />
</div>
<div className="flex items-center gap-2">
{(['left', 'center', 'right'] as const).map((align) => (
<button
key={align}
type="button"
onClick={() => patchStyle({ textAlign: align })}
className={`flex-1 rounded px-2 py-1 text-xs ${style.textAlign === align ? 'bg-white/20 text-white' : 'bg-white/10 text-slate-300'}`}
>
{align === 'left' ? 'L' : align === 'center' ? 'C' : 'R'}
</button>
))}
</div>
<div className="flex items-center gap-2">
{(['top', 'center', 'bottom'] as const).map((pos) => (
<button
key={pos}
type="button"
onClick={() => patchStyle({ activeLineVerticalAlign: pos })}
className={`flex-1 rounded px-2 py-1 text-xs ${style.activeLineVerticalAlign === pos ? 'bg-white/20 text-white' : 'bg-white/10 text-slate-300'}`}
>
{pos === 'top' ? 'T' : pos === 'center' ? 'C' : 'B'}
</button>
))}
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Line spacing</span><span>{style.lineSpacing.toFixed(2)}</span></div>
<input type="range" min={1} max={2.5} step={0.05} value={style.lineSpacing} onChange={(event) => patchStyle({ lineSpacing: parseFloat(event.target.value) })} className="w-full h-1.5 accent-brand-500" />
</div>
<div className="grid grid-cols-3 gap-2 text-[10px] text-slate-300">
<label className="space-y-1"><span className="block">Text</span><input type="color" value={style.textColor} onChange={(event) => patchStyle({ textColor: event.target.value })} className="w-full h-7 rounded border border-white/15 bg-transparent" /></label>
<label className="space-y-1"><span className="block">Highlight</span><input type="color" value={style.highlightColor} onChange={(event) => patchStyle({ highlightColor: event.target.value })} className="w-full h-7 rounded border border-white/15 bg-transparent" /></label>
<label className="space-y-1"><span className="block">BG</span><input type="color" value={style.bgColor} onChange={(event) => patchStyle({ bgColor: event.target.value })} className="w-full h-7 rounded border border-white/15 bg-transparent" /></label>
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>X offset</span><span>{activeOffset.x}</span></div>
<input type="range" min={-50} max={50} step={1} value={activeOffset.x} onChange={(event) => patchOffset('x', parseInt(event.target.value, 10))} className="w-full h-1.5 accent-brand-500" />
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Y offset</span><span>{activeOffset.y}</span></div>
<input type="range" min={-50} max={50} step={1} value={activeOffset.y} onChange={(event) => patchOffset('y', parseInt(event.target.value, 10))} className="w-full h-1.5 accent-brand-500" />
</div>
</div>
)}
</section>
<section className="space-y-2">
{sectionHeader('visualizer', 'VISUALIZER')}
{open.visualizer && (
<div className="space-y-2.5">
<div className="space-y-1.5">
<div className="text-[10px] text-slate-400">Quality</div>
<div className="grid grid-cols-3 gap-1.5">
{(['low', 'balanced', 'high'] as const).map((quality) => (
<button
key={quality}
type="button"
onClick={() => patchVizSettings({ quality })}
className={`rounded px-2 py-1 text-[11px] border ${vizSettings.quality === quality ? 'border-violet-400 bg-violet-500/20 text-white' : 'border-white/15 bg-white/5 text-slate-300 hover:bg-white/10'}`}
>
{quality === 'balanced' ? 'Balanced' : quality === 'low' ? 'Low' : 'High'}
</button>
))}
</div>
</div>
<div className="space-y-1.5">
<div className="text-[10px] text-slate-400">FPS cap</div>
<div className="grid grid-cols-3 gap-1.5">
{([30, 60, 120] as const).map((fps) => (
<button
key={fps}
type="button"
onClick={() => patchVizSettings({ fps_cap: fps })}
className={`rounded px-2 py-1 text-[11px] border ${vizSettings.fps_cap === fps ? 'border-violet-400 bg-violet-500/20 text-white' : 'border-white/15 bg-white/5 text-slate-300 hover:bg-white/10'}`}
>
{fps}
</button>
))}
</div>
</div>
{!isVisualizerMode && (
<p className="text-[11px] text-slate-400">
Switch background to Muon or Fluid to access mode-specific controls.
</p>
)}
{visualStyle === 'muon' && (
<>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Radius</span><span>{formatMultiplier(vizSettings.muon.radius_multiplier)}</span></div>
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.muon.radius_multiplier} onChange={(event) => patchVizSettings({ muon: { radius_multiplier: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Aperture</span><span>{formatMultiplier(vizSettings.muon.aperture)}</span></div>
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.muon.aperture} onChange={(event) => patchVizSettings({ muon: { aperture: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Spacing</span><span>{formatMultiplier(vizSettings.muon.spacing)}</span></div>
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.muon.spacing} onChange={(event) => patchVizSettings({ muon: { spacing: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
</div>
</>
)}
{visualStyle === 'fluid' && (
<>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Density</span><span>{formatMultiplier(vizSettings.fluid.spawn_density)}</span></div>
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.fluid.spawn_density} onChange={(event) => patchVizSettings({ fluid: { spawn_density: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Force</span><span>{formatMultiplier(vizSettings.fluid.splat_force)}</span></div>
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.fluid.splat_force} onChange={(event) => patchVizSettings({ fluid: { splat_force: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
</div>
<div className="space-y-1">
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Trail decay</span><span>{formatMultiplier(vizSettings.fluid.decay)}</span></div>
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.fluid.decay} onChange={(event) => patchVizSettings({ fluid: { decay: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
</div>
</>
)}
<button
type="button"
onClick={() => patchVizSettings(manifestVizSettings ?? VIZ_DEFAULTS)}
className="w-full rounded-md px-2 py-1.5 text-xs bg-white/10 hover:bg-white/20 text-slate-200"
>
{manifestVizSettings ? 'Reset to song defaults' : 'Reset to defaults'}
</button>
</div>
)}
</section>
</div>
</div>
)
}
@@ -8,6 +8,13 @@ interface FluidVisualizerCanvasProps {
paletteRef: MutableRefObject<VisualizerPalette>
width: number
height: number
fpsCap?: 30 | 60 | 120
quality?: 'low' | 'balanced' | 'high'
settings?: {
spawn_density?: number
splat_force?: number
decay?: number
}
}
type Blob = {
@@ -24,7 +31,7 @@ function clamp01(v: number): number {
return Math.max(0, Math.min(1, v))
}
export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width, height }: FluidVisualizerCanvasProps) {
export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width, height, fpsCap = 60, quality = 'balanced', settings }: FluidVisualizerCanvasProps) {
const canvasRef = useRef<HTMLCanvasElement>(null)
useEffect(() => {
@@ -36,6 +43,9 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
const blobs: Blob[] = []
let raf = 0
let colorIndex = 0
let lastFrameAt = 0
const cap = fpsCap === 30 || fpsCap === 120 ? fpsCap : 60
const minFrameMs = 1000 / cap
const nextPaletteColor = () => {
const palette = paletteRef.current.speakerPalette
@@ -48,21 +58,24 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
const spawnBlob = (strength: number) => {
const w = Math.max(1, width)
const h = Math.max(1, height)
const forceScale = Math.max(0.2, Number(settings?.splat_force ?? 1))
const angle = Math.random() * Math.PI * 2
const speed = 0.2 + strength * 2.2
const speed = (0.2 + strength * 2.2) * forceScale
blobs.push({
x: Math.random() * w,
y: Math.random() * h,
dx: Math.cos(angle) * speed,
dy: Math.sin(angle) * speed,
radius: 8 + strength * 38,
radius: (8 + strength * 38) * Math.max(0.25, Math.sqrt(forceScale)),
life: 0.7 + strength * 0.6,
color: nextPaletteColor(),
})
}
const render = () => {
const render = (ts: number) => {
raf = window.requestAnimationFrame(render)
if (lastFrameAt && ts - lastFrameAt < minFrameMs) return
lastFrameAt = ts
const w = Math.max(1, Math.round(canvas.clientWidth || 0), Math.round(width || 0))
const h = Math.max(1, Math.round(canvas.clientHeight || 0), Math.round(height || 0))
@@ -77,13 +90,17 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
const rms = clamp01(f.rms)
const treble = clamp01(f.treble)
const idle = 0.5 + 0.5 * Math.sin(performance.now() * 0.0011)
const densityScale = Math.max(0.2, Number(settings?.spawn_density ?? 1))
const decayScale = Math.max(0.2, Number(settings?.decay ?? 1))
// Ambient motion from RMS + small idle baseline so the renderer is never visually flat.
if (Math.random() < (0.04 + rms * 0.31)) spawnBlob(0.22 + rms * 0.58 + idle * 0.08)
const ambientSpawnScale = quality === 'low' ? 0.7 : 1
if (Math.random() < (0.04 + rms * 0.31) * densityScale * ambientSpawnScale) spawnBlob(0.22 + rms * 0.58 + idle * 0.08)
// Beat burst from bass peaks.
if (f.beatDetected) {
const count = 3 + Math.floor(bass * 4)
const baseCount = 3 + Math.floor(bass * 4)
const count = quality === 'low' ? Math.max(1, baseCount - 1) : baseCount
for (let i = 0; i < count; i += 1) spawnBlob(0.6 + bass * 0.8)
}
@@ -92,8 +109,14 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
ctx.fillRect(0, 0, w, h)
ctx.globalAlpha = 1
const decay = 0.009 + (1 - rms) * 0.012
const drag = 0.984 - bass * 0.01
// Correction C fix: trail lifetime (decay) is driven by bass — loud kick/bass →
// slower decay → longer-lived blobs (more persistent ink on loud beats).
// Velocity drag is driven by rms — overall loudness → more momentum retained.
// Original Integration Plan had these driven by rms uniformly, which made
// trail length insensitive to bass and meant quiet treble-heavy passages
// produced the same persistence as loud bass hits.
const decay = (0.009 + (1 - bass) * 0.012) * decayScale // loud bass → 0.009 (long trail)
const drag = 0.984 - rms * 0.010 // loud rms → more momentum
for (let i = blobs.length - 1; i >= 0; i -= 1) {
const b = blobs[i]
@@ -121,11 +144,11 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
}
}
render()
raf = window.requestAnimationFrame(render)
return () => {
window.cancelAnimationFrame(raf)
}
}, [featuresRef, paletteRef, width, height])
}, [featuresRef, paletteRef, width, height, fpsCap, quality, settings])
return <canvas ref={canvasRef} className="h-full w-full" aria-hidden="true" />
}
@@ -8,6 +8,13 @@ interface MuonVisualizerCanvasProps {
paletteRef: MutableRefObject<VisualizerPalette>
width: number
height: number
fpsCap?: 30 | 60 | 120
quality?: 'low' | 'balanced' | 'high'
settings?: {
radius_multiplier?: number
aperture?: number
spacing?: number
}
}
type Particle = { angle: number; radius: number; speed: number; lane: number }
@@ -18,7 +25,7 @@ function clamp01(v: number): number {
return Math.max(0, Math.min(1, v))
}
export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, height }: MuonVisualizerCanvasProps) {
export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, height, fpsCap = 60, quality = 'balanced', settings }: MuonVisualizerCanvasProps) {
const canvasRef = useRef<HTMLCanvasElement>(null)
useEffect(() => {
@@ -36,8 +43,23 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
let raf = 0
const render = () => {
// Correction B fix: radiusMultiplier cycles through presets on beats rather than
// being mapped directly from drum intensity each frame. We lerp toward a preset
// target, snapping to the next preset on each detected beat — equivalent to
// wavePresetController triggering a GSAP transition in the full Three.js port.
const RADIUS_PRESETS = [0.90, 0.95, 1.00, 1.05, 1.10, 0.85, 1.15, 1.02]
let radiusTarget = 1.0
let radiusCurrent = 1.0
let lastPresetAt = 0
let presetIdx = 0
let lastFrameAt = 0
const cap = fpsCap === 30 || fpsCap === 120 ? fpsCap : 60
const minFrameMs = 1000 / cap
const render = (ts: number) => {
raf = window.requestAnimationFrame(render)
if (lastFrameAt && ts - lastFrameAt < minFrameMs) return
lastFrameAt = ts
const w = Math.max(1, Math.round(canvas.clientWidth || 0), Math.round(width || 0))
const h = Math.max(1, Math.round(canvas.clientHeight || 0), Math.round(height || 0))
if (canvas.width !== w || canvas.height !== h) {
@@ -49,12 +71,34 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
const p = paletteRef.current
const drum = clamp01(f.drumIntensity)
const treble = clamp01(f.treble)
const idle = 0.5 + 0.5 * Math.sin(performance.now() * 0.0012)
const now = ts
const idle = 0.5 + 0.5 * Math.sin(now * 0.0012)
const radiusScale = Math.max(0.2, Number(settings?.radius_multiplier ?? 1))
const apertureScale = Math.max(0.2, Number(settings?.aperture ?? 1))
const spacingScale = Math.max(0.2, Number(settings?.spacing ?? 1))
const radiusMultiplier = 0.82 + drum * 1.12 + idle * 0.18
const aperture = 1.55 + drum * 1.9 + idle * 0.24
const spacing = 0.015 + (1 - drum) * 0.014
const hue = (p.seedHue + treble * 40) % 360
// Bug 1 fix: hue animates via drum intensity when audio is playing, and falls
// back to a slow time-based drift when silent so the visual never freezes on a
// single colour (original hueControl bug: hue was always 0 during playback).
// Correction A fix: treble drift is capped at ±15° to stay within the manifest
// palette colour family (original Integration Plan used treble * 40 / 120 which
// rotated far outside the intended hue family).
const hue = drum > 0.01
? ((p.seedHue + drum * 20 + treble * 15) + 360) % 360
: ((p.seedHue + Math.sin(now * 0.00018) * 15) + 360) % 360
// Correction B fix: advance the preset index on each beat and lerp toward the
// new target — mirrors wavePresetController / GSAP transition in the Three.js port.
if (f.beatDetected && now - lastPresetAt > 200) {
presetIdx = (presetIdx + 1) % RADIUS_PRESETS.length
radiusTarget = RADIUS_PRESETS[presetIdx] + drum * 0.25
lastPresetAt = now
}
radiusCurrent += (radiusTarget - radiusCurrent) * 0.07
const radiusMultiplier = (radiusCurrent + idle * 0.12) * radiusScale
const aperture = (1.55 + drum * 1.9 + idle * 0.24) * apertureScale
const spacing = (0.015 + (1 - drum) * 0.014) * spacingScale
ctx.clearRect(0, 0, w, h)
ctx.fillStyle = p.backgroundColor
@@ -64,9 +108,10 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
const cy = h / 2
const base = Math.min(w, h) * 0.2
const glow = 14 + drum * 28
const particleStride = quality === 'low' ? 2 : 1
ctx.globalCompositeOperation = 'lighter'
for (let i = 0; i < particles.length; i += 1) {
for (let i = 0; i < particles.length; i += particleStride) {
const pt = particles[i]
pt.angle += pt.speed + drum * 0.02
@@ -89,11 +134,11 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
ctx.shadowBlur = 0
}
render()
raf = window.requestAnimationFrame(render)
return () => {
window.cancelAnimationFrame(raf)
}
}, [featuresRef, paletteRef, width, height])
}, [featuresRef, paletteRef, width, height, fpsCap, quality, settings])
return <canvas ref={canvasRef} className="h-full w-full" aria-hidden="true" />
}
@@ -8,15 +8,41 @@ const FluidVisualizerCanvas = lazy(() => import('./FluidVisualizerCanvas'))
export type VisualizerMode = 'muon' | 'fluid' | null
/**
* Per-song visualizer intensity multipliers from manifest.visualizer.settings.
* All values default to 1.0 (no change from audio-driven behaviour).
*/
export interface VisualizerSettings {
quality?: 'low' | 'balanced' | 'high'
fps_cap?: 30 | 60 | 120
muon?: {
/** Scales the preset-cycling radius range. Default 1.0. */
radius_multiplier?: number
/** Scales the spiral aperture range. Default 1.0. */
aperture?: number
/** Scales inter-particle spacing. Default 1.0. */
spacing?: number
}
fluid?: {
/** Multiplies ambient blob spawn probability per frame. Default 1.0. */
spawn_density?: number
/** Multiplies blob velocity and size on spawn. Default 1.0. */
splat_force?: number
/** Multiplies per-frame blob lifetime drain (higher = shorter trails). Default 1.0. */
decay?: number
}
}
interface VisualizerBackdropProps {
mode: VisualizerMode
featuresRef: MutableRefObject<AudioFeatures>
paletteRef: MutableRefObject<VisualizerPalette>
width: number
height: number
settings?: VisualizerSettings | null
}
export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, width, height }: VisualizerBackdropProps) {
export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, width, height, settings }: VisualizerBackdropProps) {
const palette = paletteRef.current
if (mode === 'muon') {
@@ -31,7 +57,7 @@ export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, widt
data-palette-word-highlight={palette.wordHighlightColor}
>
<Suspense fallback={<div className="h-full w-full bg-[#05070d]" />}>
<MuonVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} />
<MuonVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} settings={settings?.muon} fpsCap={settings?.fps_cap} quality={settings?.quality} />
</Suspense>
</div>
)
@@ -48,7 +74,7 @@ export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, widt
data-palette-word-highlight={palette.wordHighlightColor}
>
<Suspense fallback={<div className="h-full w-full bg-[#05070d]" />}>
<FluidVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} />
<FluidVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} settings={settings?.fluid} fpsCap={settings?.fps_cap} quality={settings?.quality} />
</Suspense>
</div>
)
+12 -2
View File
@@ -60,8 +60,16 @@ function avgSlice(data: Uint8Array, start: number, end: number): number {
}
function deriveDrumIntensity(bass: number, treble: number, rms: number): number {
// Weight bass the most, keep a small RMS contribution for non-drum content.
const raw = bass * 0.68 + rms * 0.24 + treble * 0.08
// Exponential scalers matching the corrected muon source audio.js pattern
// (Bug 2 fix). Input range 0–1; outputs range 0–2.002 (normalised back below).
// Original formula: 1.001 * (1 + avgFr * 2) - 1.001
// This curve is near-zero when quiet and amplifies progressively when loud,
// giving a more musical response than a plain linear mix.
const expBass = 1.001 * (1 + bass * 2) - 1.001
const expTreble = 1.001 * (1 + treble * 2) - 1.001
// Coefficient ratio mirrors wave.js beatScaler (bass ~2× treble).
// Divide by 2.0 to normalise the 0–2.002 range back to 0–1 before clamping.
const raw = (expBass * 0.68 + rms * 0.24 + expTreble * 0.08) / 2.0
return clamp01(Math.pow(clamp01(raw), 0.82))
}
@@ -198,6 +206,8 @@ export function useAudioFeatures({
analyser.getByteFrequencyData(byteFreq)
// Bug 2 fix: low bins (0–9) → bass, high bins (last 56 of 256) → treble.
// Original muon source had these reversed (upperHalfArray → baseFr, etc.).
const bass = avgSlice(byteFreq, 0, 10) / 255
const treble = avgSlice(byteFreq, Math.max(0, byteFreq.length - 56), byteFreq.length) / 255
const energy = byteFreq.length ? (avgSlice(byteFreq, 0, byteFreq.length) / 255) : 0
+1
View File
@@ -473,6 +473,7 @@ export default function KaraokePage() {
coverArtUrl={coverArtUrl}
sekDynamicPalette={sekManifest?.manifest?.metadata?.dynamic_palette ?? null}
sekVisualizerMode={sekManifest?.manifest?.visualizer?.mode ?? null}
sekVisualizerSettings={sekManifest?.manifest?.visualizer?.settings ?? null}
upNextSong={nextItem ? { title: nextItem.title, artist: nextItem.artist } : null}
onSongEnded={handleSongEnded}
onExit={handleExit}
+1
View File
@@ -542,6 +542,7 @@ export default function KaraokeTVPage() {
coverArtUrl={coverArtUrl}
sekDynamicPalette={sekManifest?.manifest?.metadata?.dynamic_palette ?? null}
sekVisualizerMode={sekManifest?.manifest?.visualizer?.mode ?? null}
sekVisualizerSettings={sekManifest?.manifest?.visualizer?.settings ?? null}
disableGlobalPlaybackRegistry
upNextSong={nextItem ? { title: nextItem.title, artist: nextItem.artist } : null}
onSongEnded={handleSongEnded}