Myriad changes.
This commit is contained in:
1 parent
9281c789be
commit
cbe946925f
21 files changed
+2762
-44
No files matched your search
@@ -0,0 +1,624 @@
|
||||
# Performance Sidebar Plan
|
||||
|
||||
Date: 2026-04-08
|
||||
Scope: A slide-in sidebar available in both KaraokePerformanceMode (TV view) and fullscreen LyricsLivePreview that exposes background/visualizer selection, stem mixing, and lyrics appearance controls. Leading vocal stems are muted by default.
|
||||
|
||||
Status: ✅ Reviewed, corrected, implemented, and documented (2026-04-08).
|
||||
|
||||
---
|
||||
|
||||
## Review outcomes (issues corrected)
|
||||
|
||||
1. Removed architecture contradiction: the diagram previously showed a persistent trigger tab, but the design is keyboard-invoked only.
|
||||
2. Corrected implementation step numbering (`### 4` → `### 3`) in the `LyricsLivePreview.tsx` change list.
|
||||
3. Tightened `VizSettings.fps_cap` to `30 | 60 | 120` for stronger typing consistency with UI controls.
|
||||
4. Updated implementation notes to reflect actual files changed and completed validation pass.
|
||||
|
||||
---
|
||||
|
||||
## Current state
|
||||
|
||||
In `performanceMode`, `LyricsLivePreview` deliberately hides:
|
||||
- The universal toolbar (`!performanceMode`)
|
||||
- The style picker panel (`!performanceMode && showStylePicker`)
|
||||
- The stem mixer (`!performanceMode && renderStems()`)
|
||||
|
||||
`KaraokePerformanceMode` has a minimal HUD (auto-hides after 2.2 s showing "Press Esc to exit") and keyboard shortcuts only. There is no way for the user to change the visualizer, adjust stems, or tweak lyrics appearance once performance mode begins.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
All relevant state (`visualStyle`, `mutedStems`, `soloedStems`, `gains`, `style`) already lives inside `LyricsLivePreview`. The sidebar is built as a sub-panel inside that component — no state needs to be lifted.
|
||||
|
||||
```
|
||||
KaraokePerformanceMode
|
||||
└── LyricsLivePreview (performanceMode=true)
|
||||
├── [existing] lyrics canvas / visualizer backdrop (z-index 1–2)
|
||||
├── [existing] metadata card — top-left (z-index 7)
|
||||
└── [NEW] PerformanceSidebar — drawer/sheet (z-index 9, shown on demand via `Q`)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## New files
|
||||
|
||||
```
|
||||
frontend/src/components/
|
||||
└── PerformanceSidebar.tsx # right-side drawer with four sections
|
||||
```
|
||||
|
||||
No new hooks needed — the sidebar reads/writes state that already exists in `LyricsLivePreview`.
|
||||
|
||||
---
|
||||
|
||||
## Sidebar trigger
|
||||
|
||||
The sidebar is **shown on demand only**. On desktop/wide layouts it is keyboard-invoked; on narrow/mobile layouts it also has a floating top-right button so touch users can open it without a hardware keyboard.
|
||||
|
||||
**Keyboard shortcut**: `Q` key while in performance mode opens the sidebar. If the sidebar is already open, `Q` closes it.
|
||||
|
||||
`Q` is currently unused in performance mode (Esc exits, Space/Enter toggles play, Arrow keys seek). `KaraokePerformanceMode` does not intercept `Q`, so there is no conflict.
|
||||
|
||||
**Mobile / narrow screens:** when `isNarrow` is true, a floating circular button is rendered in the top-right corner and toggles the sidebar open/closed.
|
||||
|
||||
**Auto-hide**: The sidebar auto-hides after **4 seconds** of inactivity (no pointer movement over the sidebar panel). Any mouse movement within the sidebar resets the timer. This mirrors the HUD auto-hide pattern already used by `KaraokePerformanceMode`. The timer is cleared when the sidebar is closed manually.
|
||||
|
||||
```typescript
|
||||
// Inside PerformanceSidebar, on mouse movement over the panel:
|
||||
clearTimeout(autoHideTimer)
|
||||
autoHideTimer = setTimeout(() => onClose(), 4000)
|
||||
```
|
||||
|
||||
The sidebar does not show a persistent edge tab. On desktop it remains keyboard-first; on mobile it uses the floating top-right button.
|
||||
|
||||
---
|
||||
|
||||
## `PerformanceSidebar` component
|
||||
|
||||
```tsx
|
||||
interface PerformanceSidebarProps {
|
||||
// Background
|
||||
visualStyle: LyricsVisualStyle
|
||||
setVisualStyle: (s: LyricsVisualStyle) => void
|
||||
hasVideo: boolean
|
||||
|
||||
// Stems
|
||||
stems: Stem[] | undefined
|
||||
mutedStems: Record<string, boolean>
|
||||
soloedStems: Record<string, boolean>
|
||||
gains: Record<string, number>
|
||||
toggleStemMute: (name: string) => void
|
||||
toggleStemSolo: (name: string) => void
|
||||
setStemGain: (name: string, value: number) => void
|
||||
|
||||
// Lyrics style
|
||||
style: StyleState
|
||||
patchStyle: (patch: Partial<StyleState>) => void
|
||||
|
||||
// Visualizer controls
|
||||
vizSettings: VizSettings
|
||||
manifestVizSettings: VizSettings | null // null when no .sek or no visualizer block
|
||||
patchVizSettings: (patch: DeepPartial<VizSettings>) => void
|
||||
|
||||
// Lifecycle
|
||||
onClose: () => void
|
||||
}
|
||||
```
|
||||
|
||||
**`VizSettings` type** (defined in `LyricsLivePreview.tsx` alongside `StyleState`):
|
||||
|
||||
```typescript
|
||||
interface VizSettings {
|
||||
quality: 'low' | 'balanced' | 'high'
|
||||
fps_cap: 30 | 60 | 120
|
||||
muon: {
|
||||
radius_multiplier: number
|
||||
aperture: number
|
||||
spacing: number
|
||||
}
|
||||
fluid: {
|
||||
spawn_density: number
|
||||
splat_force: number
|
||||
decay: number
|
||||
}
|
||||
}
|
||||
|
||||
const VIZ_DEFAULTS: VizSettings = {
|
||||
quality: 'balanced',
|
||||
fps_cap: 60,
|
||||
muon: { radius_multiplier: 1.0, aperture: 1.0, spacing: 1.0 },
|
||||
fluid: { spawn_density: 1.0, splat_force: 1.0, decay: 1.0 },
|
||||
}
|
||||
```
|
||||
|
||||
`manifestVizSettings` is built once from the `sekVisualizerSettings` prop merged over `VIZ_DEFAULTS`, or `null` when `sekVisualizerSettings` is absent.
|
||||
|
||||
The component renders as an `absolute right-0 top-0 h-full` panel, width ~280 px (`w-70`), with `backdrop-blur` and `bg-black/70`. Four collapsible sections with chevron toggles, open by default.
|
||||
|
||||
---
|
||||
|
||||
## Section 1 — Background
|
||||
|
||||
Replaces the renderer buttons currently hidden behind `!performanceMode`.
|
||||
|
||||
**Controls:**
|
||||
|
||||
| Label | Action |
|
||||
|---|---|
|
||||
| **Static** | `setVisualStyle('karaoke')` — current karaoke renderer, no backdrop |
|
||||
| **Muon** | `setVisualStyle('muon')` — spiral particle visualizer |
|
||||
| **Fluid** | `setVisualStyle('fluid')` — WebGL fluid sim |
|
||||
| **Video** | `setVisualStyle('video')` — only shown when `hasVideo` is true |
|
||||
|
||||
Rendered as a 2×2 icon grid (or horizontal pill row on narrow screens). Active mode has a highlight border.
|
||||
|
||||
> No "scroll", "fade", "glow", or "blur" options here — those are editing modes not suited to performance view. The sidebar only exposes the four modes appropriate for live display.
|
||||
|
||||
---
|
||||
|
||||
## Section 2 — Stems
|
||||
|
||||
Shows per-stem mixing controls when `isMultiStem` is true; hidden (or shows a "no stems" note) otherwise.
|
||||
|
||||
**Controls per stem:**
|
||||
|
||||
- **Label** — stem display name (`stem.label`), capitalized
|
||||
- **M button** — mute toggle (red when active)
|
||||
- **S button** — solo toggle (yellow when active)
|
||||
- **Gain slider** — 0–200%, default 100%, step 5%
|
||||
|
||||
This is a direct port of the existing `renderStems()` output, styled for the sidebar width.
|
||||
|
||||
### Leading vocal default mute
|
||||
|
||||
When `performanceMode` is true and the component mounts with **no saved `mutedStems` preferences** for this `storageKey`, auto-mute stems whose name matches the leading-vocal pattern:
|
||||
|
||||
```typescript
|
||||
const LEADING_VOCAL_PATTERN = /^(lead|lead\s|leading|vocals?|vox|voice|main\s*vocal)/i
|
||||
|
||||
function isLeadingVocalStem(name: string): boolean {
|
||||
return LEADING_VOCAL_PATTERN.test(name.trim())
|
||||
}
|
||||
```
|
||||
|
||||
Applied once in a `useEffect` inside `LyricsLivePreview`:
|
||||
|
||||
```typescript
|
||||
useEffect(() => {
|
||||
if (!performanceMode || !stems?.length) return
|
||||
// Only apply if no mute state has been saved for this session
|
||||
const hasSavedPrefs = Object.keys(initialPreviewPrefs.mutedStems ?? {}).length > 0
|
||||
if (hasSavedPrefs) return
|
||||
|
||||
const defaultMutes: Record<string, boolean> = {}
|
||||
for (const stem of stems) {
|
||||
if (isLeadingVocalStem(stem.name)) defaultMutes[stem.name] = true
|
||||
}
|
||||
if (Object.keys(defaultMutes).length > 0) {
|
||||
setMutedStems(defaultMutes)
|
||||
applyGains({ mutedStems: defaultMutes })
|
||||
}
|
||||
}, [performanceMode]) // run once on mount — dep on performanceMode ensures it's stable
|
||||
```
|
||||
|
||||
This runs only when there's nothing saved. Once the user adjusts stems, their preference is persisted to localStorage and restored next session, so the default only applies to first use.
|
||||
|
||||
---
|
||||
|
||||
## Section 3 — Lyrics
|
||||
|
||||
A condensed version of `StylePicker` containing only the controls most useful during a live performance. Excludes video display mode controls, layout offset sliders, and speaker colour assignments (too granular for live adjustment).
|
||||
|
||||
**Controls:**
|
||||
|
||||
| Control | StyleState key | Widget |
|
||||
|---|---|---|
|
||||
| Font | `fontFamily` | `<select>` (same `FONT_OPTIONS` list) |
|
||||
| Size | `fontSize` | Range slider 12–56 |
|
||||
| Bold | `fontBold` | Toggle button |
|
||||
| Align | `textAlign` | Left / Center / Right icon buttons |
|
||||
| Position | `activeLineVerticalAlign` | Top / Center / Bottom icon buttons |
|
||||
| Line spacing | `lineSpacing` | Range slider |
|
||||
| Text colour | `textColor` | `<input type="color">` |
|
||||
| Highlight colour | `highlightColor` | `<input type="color">` |
|
||||
| Background | `bgColor` | `<input type="color">` (has no effect in muon/fluid modes where the canvas sets its own background) |
|
||||
| X offset | `layoutOffsetXScroll` | Range slider −50–50 |
|
||||
| Y offset | `layoutOffsetYScroll` | Range slider −50–50 |
|
||||
|
||||
The offset sliders automatically route to the correct offset key based on `visualStyle` (scroll vs line vs word vs syllable), same as `StylePicker.applyLayoutOffset`.
|
||||
|
||||
---
|
||||
|
||||
## Section 4 — Visualizer controls
|
||||
|
||||
Always shown in the sidebar. Common preset controls (Quality and FPS cap) are always available; mode-specific sliders are shown only when `visualStyle === 'muon'` or `visualStyle === 'fluid'`.
|
||||
|
||||
Displays controls for the **currently active visualizer** only — switching from Muon to Fluid swaps the control set in place. Common controls (Quality, FPS cap) are shown at the top for both as explicit preset buttons.
|
||||
|
||||
A **Reset** button appears at the bottom of this section. Its label and behaviour depend on whether manifest values exist:
|
||||
|
||||
| Condition | Button label | Action |
|
||||
|---|---|---|
|
||||
| `manifestVizSettings !== null` | "Reset to song defaults" | `patchVizSettings(manifestVizSettings)` |
|
||||
| `manifestVizSettings === null` | "Reset to defaults" | `patchVizSettings(VIZ_DEFAULTS)` |
|
||||
|
||||
### Common controls (Muon and Fluid)
|
||||
|
||||
| Label | Key | Widget | Range / Options |
|
||||
|---|---|---|---|
|
||||
| Quality | `quality` | Segmented preset buttons | Low / Balanced / High |
|
||||
| FPS cap | `fps_cap` | Segmented preset buttons | 30 / 60 / 120 |
|
||||
|
||||
### Muon-specific controls
|
||||
|
||||
| Label | Key | Widget | Range | Step | Default |
|
||||
|---|---|---|---|---|---|
|
||||
| Radius | `muon.radius_multiplier` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
|
||||
| Aperture | `muon.aperture` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
|
||||
| Spacing | `muon.spacing` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
|
||||
|
||||
### Fluid-specific controls
|
||||
|
||||
| Label | Key | Widget | Range | Step | Default |
|
||||
|---|---|---|---|---|---|
|
||||
| Density | `fluid.spawn_density` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
|
||||
| Force | `fluid.splat_force` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
|
||||
| Trail decay | `fluid.decay` | Range slider | 0.25–2.0 | 0.05 | 1.0 |
|
||||
|
||||
Slider labels show the current numeric value to two decimal places (e.g. "1.20×"). All values are multipliers — 1.0 means unmodified default behaviour as driven by audio.
|
||||
|
||||
---
|
||||
|
||||
## Changes to `LyricsLivePreview.tsx`
|
||||
|
||||
### 1. Add sidebar state
|
||||
|
||||
```typescript
|
||||
const [showPerfSidebar, setShowPerfSidebar] = useState(false)
|
||||
```
|
||||
|
||||
Add live visualizer settings state (seeded from `sekVisualizerSettings`, falls back to `VIZ_DEFAULTS`):
|
||||
|
||||
```typescript
|
||||
const manifestVizSettings = useMemo<VizSettings | null>(() => {
|
||||
if (!sekVisualizerSettings) return null
|
||||
return deepMerge(VIZ_DEFAULTS, sekVisualizerSettings)
|
||||
}, [sekVisualizerSettings])
|
||||
|
||||
const [liveVizSettings, setLiveVizSettings] = useState<VizSettings>(
|
||||
() => manifestVizSettings ?? VIZ_DEFAULTS
|
||||
)
|
||||
|
||||
const patchVizSettings = useCallback((patch: DeepPartial<VizSettings>) => {
|
||||
setLiveVizSettings(prev => deepMerge(prev, patch))
|
||||
}, [])
|
||||
```
|
||||
|
||||
`deepMerge` is a small local helper (two-level object merge — no third-party dep needed given the shallow shape of `VizSettings`).
|
||||
|
||||
`liveVizSettings` is passed to `VisualizerBackdrop` (replacing the current `sekVisualizerSettings` prop there) so live adjustments take effect immediately.
|
||||
|
||||
### 2. Add `Q` key handler
|
||||
|
||||
In the existing `keydown` listener (or a new one):
|
||||
|
||||
```typescript
|
||||
if (event.key === 'q' && isPerformanceView) {
|
||||
event.preventDefault()
|
||||
setShowPerfSidebar(p => !p)
|
||||
}
|
||||
```
|
||||
|
||||
No trigger tab is rendered — the sidebar is keyboard-invoked only.
|
||||
|
||||
### 3. Render `PerformanceSidebar`
|
||||
|
||||
```tsx
|
||||
{isPerformanceView && showPerfSidebar && (
|
||||
<PerformanceSidebar
|
||||
visualStyle={visualStyle}
|
||||
setVisualStyle={setVisualStyle}
|
||||
hasVideo={!!(videoUrl || resolvedVideoPlaylist.length > 0)}
|
||||
stems={stems}
|
||||
mutedStems={mutedStems}
|
||||
soloedStems={soloedStems}
|
||||
gains={gains}
|
||||
toggleStemMute={toggleStemMute}
|
||||
toggleStemSolo={toggleStemSolo}
|
||||
setStemGain={(name, val) => {
|
||||
setGains(prev => ({ ...prev, [name]: val }))
|
||||
applyGains({ name, gain: val })
|
||||
}}
|
||||
style={style}
|
||||
patchStyle={patchStyle}
|
||||
vizSettings={liveVizSettings}
|
||||
manifestVizSettings={manifestVizSettings}
|
||||
patchVizSettings={patchVizSettings}
|
||||
onClose={() => setShowPerfSidebar(false)}
|
||||
/>
|
||||
)}
|
||||
```
|
||||
|
||||
### 5. Leading vocal default mute (see Section 2 above)
|
||||
|
||||
### 6. Viewport resize — exclude sidebar width
|
||||
|
||||
The existing `useEffect` that computes `containerWidth`/`containerHeight` and `visualizerSize` already depends on `[isFullscreen, performanceMode, showStylePicker, showEditor, showAegisub]`. Add `showPerfSidebar` to this dep array, and subtract the sidebar width (280 px) from the container width calculation when the sidebar is open.
|
||||
|
||||
---
|
||||
|
||||
## Changes to `KaraokePerformanceMode.tsx`
|
||||
|
||||
No structural changes needed — the sidebar and its trigger live entirely inside `LyricsLivePreview`. However:
|
||||
|
||||
1. **`Q` key** — Currently no handler in `KaraokePerformanceMode`. The sidebar's `Q` toggle handler will be inside `LyricsLivePreview`, which receives keyboard events through its own listener. `KaraokePerformanceMode` does not intercept `Q`, so no conflict.
|
||||
|
||||
2. **HUD position** — The existing HUD ("Press Esc to exit") is `top-4 left-1/2`. The sidebar trigger is on the right edge at vertical center. No overlap.
|
||||
|
||||
3. **`upNextSong` / `backgroundLabel` / `profileName`** props — these are display-only and unaffected.
|
||||
|
||||
---
|
||||
|
||||
## `PerformanceSidebar.tsx` layout
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────┐
|
||||
│ Controls [✕] │ ← header
|
||||
├──────────────────────────────────────────┤
|
||||
│ ▾ BACKGROUND │
|
||||
│ [Static] [Muon] [Fluid] [Video?] │
|
||||
├──────────────────────────────────────────┤
|
||||
│ ▾ STEMS │
|
||||
│ Vocals [M] [S] ████░░░░ 85% │
|
||||
│ Drums [M] [S] ████████ 100% │
|
||||
│ Bass [M] [S] ████████ 100% │
|
||||
│ Other [M] [S] ████████ 100% │
|
||||
├──────────────────────────────────────────┤
|
||||
│ ▾ LYRICS │
|
||||
│ Font [System ▾] [B] │
|
||||
│ Size ████░░░░░ 20 │
|
||||
│ Align [⬛][⬛][⬛] Pos [T][C][B] │
|
||||
│ Spacing ████░░░░ 1.4 │
|
||||
│ Text ■ Highlight ■ BG ■ │
|
||||
│ X offset ────┬── 0 │
|
||||
│ Y offset ───┬─── 0 │
|
||||
├──────────────────────────────────────────┤
|
||||
│ ▾ VISUALIZER │
|
||||
│ Quality [Low] [Balanced] [High] │
|
||||
│ FPS cap [30] [60] [120] │
|
||||
│ ─── Muon ───────────────────────────── │
|
||||
│ Radius ████░░░░ 1.00× │
|
||||
│ Aperture ████░░░░ 1.00× │
|
||||
│ Spacing ████░░░░ 1.00× │
|
||||
│ [Reset to song defaults] │
|
||||
└──────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
(Fluid variant shows Density / Force / Trail decay in place of the Muon rows.)
|
||||
|
||||
Width: 280 px on desktop, 100 vw on narrow (`isNarrow`) screens (slides up from bottom as a sheet instead).
|
||||
|
||||
---
|
||||
|
||||
## Edge cases
|
||||
|
||||
| Scenario | Behaviour |
|
||||
|---|---|
|
||||
| No stems (single-track, non-SEK) | Section 2 shows "No stems — single track mode" note |
|
||||
| Video mode selected but `hasVideo` is false | Video button not rendered; if `visualStyle === 'video'` arrives via prop, show Static instead |
|
||||
| All stems soloed | Solo button cleared for all (solo clears itself when all are soloed — matches existing toggle logic) |
|
||||
| `isNarrow` (mobile/TV portrait) | Sidebar renders as a bottom sheet, full width, sections scroll vertically |
|
||||
| Sidebar auto-hide fires | `onClose()` is called from within `PerformanceSidebar` after 4 s of no pointer activity |
|
||||
| User exits performance mode (Esc) | `showPerfSidebar` resets to false on next mount |
|
||||
| Sidebar open + visualizer canvas | Canvas width calculation subtracts sidebar width so visualizer doesn't render behind the panel |
|
||||
| `storageKey` absent | Leading vocal default mute still applies; prefs just won't persist across sessions |
|
||||
| `sekVisualizerMode` prop | See Bug 1a below — enforcement is now user-override-aware |
|
||||
| No manifest.json / no `sekVisualizerSettings` | `manifestVizSettings` is `null`; Reset button label is "Reset to defaults"; restores `VIZ_DEFAULTS` |
|
||||
| `sekVisualizerSettings` present but missing sub-keys | `manifestVizSettings` fills missing keys from `VIZ_DEFAULTS` via `deepMerge`; Reset always restores a complete `VizSettings` object |
|
||||
| User switches visualizer mode mid-session | Section 4 swaps the Muon/Fluid rows in place; shared controls (Quality, FPS cap) remain; slider values for the hidden mode are preserved in `liveVizSettings` |
|
||||
| `liveVizSettings` changes | Passed directly to `VisualizerBackdrop`; changes take effect on the next animation frame without remounting the canvas |
|
||||
|
||||
---
|
||||
|
||||
## Bug fixes — reported 2026-04-08
|
||||
|
||||
### Bug 1a — Sidebar visualizer buttons (Fluid, Muon, Video) don't switch modes ✅ IMPLEMENTED
|
||||
|
||||
**File:** `LyricsLivePreview.tsx`
|
||||
|
||||
**Root cause:** `enforcedVisualizerMode` is derived from `sekVisualizerMode` or `defaultStyle`:
|
||||
|
||||
```typescript
|
||||
const enforcedVisualizerMode: LyricsVisualStyle | null =
|
||||
(sekVisualizerMode === 'muon' || sekVisualizerMode === 'fluid')
|
||||
? sekVisualizerMode
|
||||
: ((defaultStyle === 'muon' || defaultStyle === 'fluid') ? defaultStyle : null)
|
||||
```
|
||||
|
||||
A `useEffect` at lines 1545–1549 watches both `enforcedVisualizerMode` **and `visualStyle`** in its dep array:
|
||||
|
||||
```typescript
|
||||
useEffect(() => {
|
||||
if (!enforcedVisualizerMode) return
|
||||
if (visualStyle === enforcedVisualizerMode) return
|
||||
setVisualStyle(enforcedVisualizerMode) // ← immediately reverts sidebar choice
|
||||
}, [enforcedVisualizerMode, visualStyle])
|
||||
```
|
||||
|
||||
When the sidebar calls `setVisualStyle('fluid')`, React re-renders, the effect fires again (because `visualStyle` is in deps), sees `visualStyle !== enforcedVisualizerMode`, and immediately reverts it back. The user's choice never sticks.
|
||||
|
||||
**Fix:** Track whether the user has manually chosen a mode this session with a ref. Skip enforcement if they have:
|
||||
|
||||
```typescript
|
||||
const userOverrodeVisualizerRef = useRef(false)
|
||||
|
||||
// Pass to PerformanceSidebar instead of raw setVisualStyle:
|
||||
const setVisualStyleFromSidebar = useCallback((s: LyricsVisualStyle) => {
|
||||
userOverrodeVisualizerRef.current = true
|
||||
setVisualStyle(s)
|
||||
}, [])
|
||||
|
||||
// Updated enforcement effect — remove visualStyle from dep array:
|
||||
useEffect(() => {
|
||||
if (!enforcedVisualizerMode) return
|
||||
if (userOverrodeVisualizerRef.current) return // user explicitly chose something else
|
||||
setVisualStyle(enforcedVisualizerMode)
|
||||
}, [enforcedVisualizerMode]) // ← no visualStyle dep; runs once on mount and on song change
|
||||
```
|
||||
|
||||
Reset `userOverrodeVisualizerRef.current = false` when `storageKey` / `previewPrefsKey` changes (i.e. new song loaded), so enforcement re-applies on the next song.
|
||||
|
||||
**Change in `PerformanceSidebar`:** Replace `setVisualStyle` prop usage with `setVisualStyleFromSidebar`. No interface change needed — it has the same signature.
|
||||
|
||||
---
|
||||
|
||||
### Bug 1b — Video mode: video doesn't start when switching mid-song ✅ IMPLEMENTED
|
||||
|
||||
**File:** `LyricsLivePreview.tsx`
|
||||
|
||||
**Root cause:** The `<video>` element is conditionally rendered only when `hasVideoBackdrop = visualStyle === 'video' && !!currentVideoUrl`. When `visualStyle` switches to `'video'` while the song is already playing, the video element mounts fresh but is paused. `togglePlay` is not called, so `videoRef.current.play()` and `setVideoStartPhase('playing')` never fire. The audio continues; the video stays black/paused.
|
||||
|
||||
**Fix:** Add a `useEffect` that detects the transition to active video mode while already playing, and starts the video at the current transport position:
|
||||
|
||||
```typescript
|
||||
useEffect(() => {
|
||||
if (!hasVideoBackdrop || !isPlaying || !videoRef.current) return
|
||||
if (videoStartPhase === 'playing') return // already running
|
||||
const t = currentTimeRef.current
|
||||
if (isPlaylistBackdropMode) {
|
||||
syncPlaylistVideoToTransport(t)
|
||||
} else {
|
||||
videoRef.current.currentTime = t
|
||||
}
|
||||
videoRef.current.play().catch((err) =>
|
||||
console.warn('[LyricsLivePreview] video auto-start on mode switch failed', err)
|
||||
)
|
||||
setVideoStartPhase('playing')
|
||||
}, [hasVideoBackdrop, isPlaying])
|
||||
// Intentionally excludes videoStartPhase from deps — effect should re-run when the
|
||||
// video element mounts (hasVideoBackdrop transitions false→true), not on phase changes.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Bug 2 — Seek slider snaps back to the beginning after dragging ✅ IMPLEMENTED
|
||||
|
||||
**File:** `LyricsLivePreview.tsx`
|
||||
|
||||
**Root cause:** The RAF loop (inside the `isPlaying` effect, lines 2157–2251) runs every animation frame while the song is playing. It reads `audio.currentTime` from the DOM audio element and calls `setCurrentTime(t)`. After a drag-seek:
|
||||
|
||||
1. `endSeekDrag` calls `applySeekPosition(120)` — sets `audio.currentTime = 120`, `currentTimeRef.current = 120`, `setCurrentTime(120)`.
|
||||
2. `endSeekDrag` immediately sets `isSeekingRef.current = false`.
|
||||
3. `endSeekDrag` queues `setTimeout(0)` to clear `seekDragTime`.
|
||||
4. **Before that timeout fires**, the RAF fires. `isSeekingRef.current = false`, so the RAF is unguarded.
|
||||
5. The RAF reads `audio.currentTime`. The browser's seek hasn't completed yet — `audio.currentTime` still returns the pre-seek value (often near 0 for a just-loaded position).
|
||||
6. `currentTimeRef.current = 0` (overwritten by RAF).
|
||||
7. `setCurrentTime(0)` (state set to 0).
|
||||
8. The `setTimeout(0)` fires: `setSeekDragTime(null)`.
|
||||
9. Input value: `seekDragTime ?? currentTime = null ?? 0 = 0`. Slider snaps to 0.
|
||||
|
||||
The `isSeekingRef` guard was supposed to prevent step 4–7, but it is cleared synchronously in `endSeekDrag` before the browser's `seeked` event has fired.
|
||||
|
||||
**Fix:** Do not clear `isSeekingRef.current` synchronously in `endSeekDrag`. Instead, attach a one-shot `seeked` listener to the relevant audio element, which clears it once the browser confirms the seek has settled. Add a 500 ms safety timeout in case `seeked` never fires (e.g. no audio element present):
|
||||
|
||||
```typescript
|
||||
function endSeekDrag(e?: React.PointerEvent<HTMLInputElement> | React.FocusEvent<HTMLInputElement>) {
|
||||
const isBlur = e?.type === 'blur'
|
||||
if (!isBlur) {
|
||||
const maybeValue = e ? parseFloat((e.currentTarget as HTMLInputElement).value) : Number.NaN
|
||||
const finalValue = Number.isFinite(maybeValue) ? maybeValue : pendingSeekRef.current
|
||||
if (typeof finalValue === 'number' && Number.isFinite(finalValue)) {
|
||||
pendingSeekRef.current = finalValue
|
||||
applySeekPosition(finalValue)
|
||||
}
|
||||
}
|
||||
// DO NOT clear isSeekingRef immediately — wait for audio to confirm the seek.
|
||||
// The RAF is guarded by isSeekingRef and must not read stale audio.currentTime
|
||||
// between the seek call and the browser's 'seeked' event firing.
|
||||
const targetAudio = isMultiStem
|
||||
? (Object.values(stemAudioRefs.current)[0] ?? null)
|
||||
: audioRef.current
|
||||
if (targetAudio) {
|
||||
const clearLock = () => { isSeekingRef.current = false }
|
||||
targetAudio.addEventListener('seeked', clearLock, { once: true })
|
||||
seekLockFallbackRef.current = window.setTimeout(clearLock, 500)
|
||||
} else {
|
||||
isSeekingRef.current = false
|
||||
}
|
||||
|
||||
if (clearSeekUiTimerRef.current !== null) window.clearTimeout(clearSeekUiTimerRef.current)
|
||||
clearSeekUiTimerRef.current = window.setTimeout(() => {
|
||||
setSeekDragTime(null)
|
||||
clearSeekUiTimerRef.current = null
|
||||
}, 0)
|
||||
}
|
||||
```
|
||||
|
||||
Add `seekLockFallbackRef = useRef<number | null>(null)` alongside the other refs. Clear it in cleanup / `handleReset`.
|
||||
|
||||
**Also fix `beginSeekDrag`**: the same pattern applies — when drag starts, `isSeekingRef.current = true` (already done). The fallback timer from any prior drag is cleared on `beginSeekDrag` to prevent a stale `clearLock` from firing mid-drag:
|
||||
|
||||
```typescript
|
||||
function beginSeekDrag() {
|
||||
if (seekLockFallbackRef.current !== null) {
|
||||
window.clearTimeout(seekLockFallbackRef.current)
|
||||
seekLockFallbackRef.current = null
|
||||
}
|
||||
// ... existing logic ...
|
||||
isSeekingRef.current = true
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation status (2026-04-08):** Implemented in `LyricsLivePreview.tsx`:
|
||||
- added `seekLockFallbackRef`;
|
||||
- cleared stale fallback timer on `beginSeekDrag`;
|
||||
- changed `endSeekDrag` to release seek lock on `audio.seeked` (with 500 ms fallback timer);
|
||||
- cleared fallback timer in `stopPlayback`, `finalizePlayback`, and `handleReset`.
|
||||
|
||||
---
|
||||
|
||||
## Implementation completed (2026-04-08)
|
||||
|
||||
### Files changed
|
||||
|
||||
- `frontend/src/components/LyricsLivePreview.tsx`
|
||||
- Added `showPerfSidebar` state and keyboard toggle (`Q`) for performance view.
|
||||
- Added a floating top-right narrow-screen button to open the sidebar when closed.
|
||||
- Added typed visualizer settings state (`VizSettings`, `VIZ_DEFAULTS`, merge + normalization helpers).
|
||||
- Replaced direct `sekVisualizerSettings` pass-through with live `liveVizSettings` to `VisualizerBackdrop`.
|
||||
- Added first-use leading-vocal default mute logic for performance mode when no saved mute preferences exist.
|
||||
- Added viewport sizing exclusion for open desktop sidebar (subtracts 280 px from visualizer width).
|
||||
- Wired `PerformanceSidebar` overlay render in performance/fullscreen view.
|
||||
- Implemented seek-lock timing fix for drag seek (`seekLockFallbackRef` + `seeked`-gated unlock) to prevent slider snap-back.
|
||||
|
||||
- `frontend/src/components/PerformanceSidebar.tsx` (new)
|
||||
- Implemented four collapsible sections: Background, Stems, Lyrics, Visualizer.
|
||||
- Added 4-second auto-hide timer reset on pointer movement within the panel.
|
||||
- Added mode-aware visualizer controls, common preset buttons (Quality/FPS), and reset behavior (`song defaults` vs `global defaults`).
|
||||
- Added responsive rendering: right drawer on wide viewports, bottom sheet on narrow viewports.
|
||||
|
||||
### Validation
|
||||
|
||||
- TypeScript diagnostics for both changed frontend files report no errors.
|
||||
- Manual runtime QA for Bug 2 behavior remains pending (seek drag while actively playing).
|
||||
|
||||
---
|
||||
|
||||
## Implementation order
|
||||
|
||||
| Step | File | Notes |
|
||||
|---|---|---|
|
||||
| 1 | `LyricsLivePreview.tsx` | Add `showPerfSidebar` state, Q key handler; add `VizSettings` type, `VIZ_DEFAULTS`, `liveVizSettings` state, `patchVizSettings`, `manifestVizSettings` memo |
|
||||
| 2 | `LyricsLivePreview.tsx` | Pass `liveVizSettings` to `VisualizerBackdrop` (replaces direct `sekVisualizerSettings` pass-through) |
|
||||
| 3 | `LyricsLivePreview.tsx` | Add leading vocal default-mute effect |
|
||||
| 4 | `PerformanceSidebar.tsx` | Create component — Background section only |
|
||||
| 5 | `PerformanceSidebar.tsx` | Add Stems section (port from `renderStems`) |
|
||||
| 6 | `PerformanceSidebar.tsx` | Add Lyrics section (subset of `StylePicker`) |
|
||||
| 7 | `PerformanceSidebar.tsx` | Add Visualizer controls section (Section 4) with Reset button |
|
||||
| 8 | `LyricsLivePreview.tsx` | Wire `PerformanceSidebar` render + viewport exclusion + new viz props |
|
||||
| 9 | `LyricsLivePreview.tsx` | **Bug 1a fix** — add `userOverrodeVisualizerRef`, `setVisualStyleFromSidebar`, update `enforcedVisualizerMode` effect to remove `visualStyle` dep; pass new setter to `PerformanceSidebar` |
|
||||
| 10 | `LyricsLivePreview.tsx` | **Bug 1b fix** — add `useEffect` to start video when `hasVideoBackdrop && isPlaying` and `videoStartPhase !== 'playing'` |
|
||||
| 11 | `LyricsLivePreview.tsx` | **Bug 2 fix** — add `seekLockFallbackRef`; update `beginSeekDrag` to cancel prior fallback; update `endSeekDrag` to defer `isSeekingRef = false` until `audio.seeked` fires |
|
||||
| 12 | Manual test | Visualizer switches persist; video starts on mode switch; seek slider holds position; reset restores correctly |
|
||||
| 13 | `2026-04-08_PerformanceSidebar_Plan.md` | Document implementation status and validation |
|
||||
@@ -0,0 +1,461 @@
|
||||
# Visualizer Controls Audit — Sidebar Parameters & Presets
|
||||
|
||||
Date: 2026-04-08
|
||||
Scope: Full audit of the runtime animation loop for both `MuonVisualizerCanvas.tsx` and
|
||||
`FluidVisualizerCanvas.tsx`. Documents every expression that changes during playback, identifies
|
||||
which values are hardcoded and tunable, proposes new manifest keys and sidebar controls, and
|
||||
defines named presets for each visualizer.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Muon: what changes during playback
|
||||
|
||||
Source: `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
|
||||
|
||||
### Input signals (per-frame)
|
||||
|
||||
| Signal | Source | Range | Description |
|
||||
|---|---|---|---|
|
||||
| `drum` | `f.drumIntensity` (clamped) | 0–1 | Primary reactive input. Weighted bass+rms+treble exponential scaler. |
|
||||
| `treble` | `f.treble` (clamped) | 0–1 | High-frequency energy. Secondary hue influence. |
|
||||
| `idle` | `0.5 + 0.5 * sin(now * 0.0012)` | 0–1 | Slow sinusoidal pulse (~5.2 s period) used when audio is quiet. |
|
||||
| `beatDetected` | `f.beatDetected` | bool | Triggers preset advance when `true` and >200 ms since last advance. |
|
||||
|
||||
### Computed values (all hardcoded coefficients)
|
||||
|
||||
#### Radius system
|
||||
|
||||
```
|
||||
RADIUS_PRESETS = [0.90, 0.95, 1.00, 1.05, 1.10, 0.85, 1.15, 1.02] (8 steps)
|
||||
radiusTarget = RADIUS_PRESETS[presetIdx] + drum * 0.25 (on beat)
|
||||
radiusCurrent += (radiusTarget - radiusCurrent) * 0.07 (lerp speed)
|
||||
radiusMultiplier = (radiusCurrent + idle * 0.12) * radiusScale (with idle sway)
|
||||
r = ring * radiusMultiplier * (1 + sin(angle * aperture) * 0.16) (sinusoidal variance)
|
||||
```
|
||||
|
||||
Key hardcoded coefficients: lerp speed `0.07`, idle sway `0.12`, sinusoidal variance `0.16`, drum
|
||||
beat bonus `0.25`, beat cooldown `200 ms`.
|
||||
|
||||
#### Hue system
|
||||
|
||||
```
|
||||
playing : hue = (seedHue + drum * 20 + treble * 15) % 360
|
||||
silent : hue = (seedHue + sin(now * 0.00018) * 15) % 360
|
||||
```
|
||||
|
||||
Drum pushes hue up to ±20°, treble up to ±15° from the seed colour.
|
||||
|
||||
#### Aperture (ellipse opening)
|
||||
|
||||
```
|
||||
aperture = (1.55 + drum * 1.9 + idle * 0.24) * apertureScale
|
||||
```
|
||||
|
||||
Range at `apertureScale=1`: 1.55 (silent+idle low) → 3.69 (loud drum+idle high). Controls how
|
||||
elliptical the particle orbit becomes. The y-axis projection is always `r * 0.62` (hardcoded
|
||||
ellipse squash ratio).
|
||||
|
||||
#### Spacing (ring gap)
|
||||
|
||||
```
|
||||
spacing = (0.015 + (1 - drum) * 0.014) * spacingScale
|
||||
ring = base + lane * (spacing * min(w,h) * 5)
|
||||
```
|
||||
|
||||
Quiet = wider rings (0.029), loud drum = tighter rings (0.015). Range factor `5` and base `min(w,h)*0.2` are hardcoded.
|
||||
|
||||
#### Particle size
|
||||
|
||||
```
|
||||
size = 1.75 + drum * 3.0 + (i % 4) * 0.35
|
||||
```
|
||||
|
||||
Base 1.75 px, grows to 4.75 px at full drum. The `(i % 4) * 0.35` creates four size tiers
|
||||
(variety), hardcoded.
|
||||
|
||||
#### Alpha (opacity per particle)
|
||||
|
||||
```
|
||||
alpha = 0.34 + drum * 0.42
|
||||
```
|
||||
|
||||
Range: 0.34 (quiet) → 0.76 (loud drum). Composited with `globalCompositeOperation = 'lighter'`
|
||||
(additive blending).
|
||||
|
||||
#### Glow
|
||||
|
||||
```
|
||||
glow = 14 + drum * 28 (shadowBlur, px)
|
||||
```
|
||||
|
||||
Range: 14 px (quiet) → 42 px (loud drum).
|
||||
|
||||
#### Rotation speed (per-frame angle advance)
|
||||
|
||||
```
|
||||
pt.angle += pt.speed + drum * 0.02
|
||||
pt.speed = 0.002 + (i % 9) * 0.00045 (per-particle, set on init)
|
||||
```
|
||||
|
||||
Base speed range: 0.002–0.006 rad/frame. Drum adds up to 0.02 rad/frame on top.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Fluid: what changes during playback
|
||||
|
||||
Source: `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
|
||||
|
||||
### Input signals (per-frame)
|
||||
|
||||
| Signal | Source | Range | Description |
|
||||
|---|---|---|---|
|
||||
| `bass` | `f.bass` (clamped) | 0–1 | Low-frequency energy (bins 0–9 of FFT). Primary trail and beat driver. |
|
||||
| `rms` | `f.rms` (clamped) | 0–1 | Overall loudness. Drives ambient spawn rate and drag. |
|
||||
| `treble` | `f.treble` (clamped) | 0–1 | High-frequency energy. Drives blob pulse scale. |
|
||||
| `beatDetected` | `f.beatDetected` | bool | Triggers burst spawn. |
|
||||
| `idle` | `0.5 + 0.5 * sin(now * 0.0011)` | 0–1 | Slow baseline (~5.7 s period). Added to ambient spawn strength. |
|
||||
|
||||
### Computed values (all hardcoded coefficients)
|
||||
|
||||
#### Ambient spawn
|
||||
|
||||
```
|
||||
spawnChance = (0.04 + rms * 0.31) * densityScale (per-frame probability)
|
||||
spawnStrength = 0.22 + rms * 0.58 + idle * 0.08
|
||||
```
|
||||
|
||||
At `densityScale=1`, rms=0: 4% chance/frame. At rms=1: 35% chance/frame.
|
||||
Spawn strength range: 0.22 (quiet) → 0.88 (rms=1).
|
||||
|
||||
#### Beat burst
|
||||
|
||||
```
|
||||
count = 3 + floor(bass * 4) (3–7 blobs per beat)
|
||||
strength = 0.6 + bass * 0.8 (0.6–1.4)
|
||||
```
|
||||
|
||||
Beat burst is the primary visual impact. Count and strength both scale with bass.
|
||||
|
||||
#### Blob properties (on spawn)
|
||||
|
||||
```
|
||||
speed = (0.2 + strength * 2.2) * forceScale (px/frame)
|
||||
radius = (8 + strength * 38) * sqrt(forceScale) (px)
|
||||
life = 0.7 + strength * 0.6 (lifetime units, before decay)
|
||||
```
|
||||
|
||||
Strong bass beat → speed ~3.3 * forceScale, radius ~61 * sqrt(forceScale).
|
||||
|
||||
#### Background fill (trail fade speed)
|
||||
|
||||
```
|
||||
ctx.globalAlpha = 0.20 + (1 - bass) * 0.16
|
||||
```
|
||||
|
||||
This is drawn over the canvas every frame to fade out old blobs. Range:
|
||||
- `bass=0` (quiet): alpha 0.36 → fast trail fade
|
||||
- `bass=1` (loud): alpha 0.20 → slow trail fade (persistent ink on loud beats)
|
||||
|
||||
This is the **primary visual trail control** and is currently entirely implicit — not exposed via
|
||||
any setting.
|
||||
|
||||
#### Decay and drag
|
||||
|
||||
```
|
||||
decay = (0.009 + (1 - bass) * 0.012) * decayScale
|
||||
drag = 0.984 - rms * 0.010
|
||||
b.life -= decay
|
||||
b.dx *= drag
|
||||
```
|
||||
|
||||
`decay` is subtracted from blob life each frame. At `decayScale=1`:
|
||||
- Loud bass → 0.009/frame (long trail: ~77–110 frames to die)
|
||||
- Quiet → 0.021/frame (short trail: ~33–47 frames to die)
|
||||
|
||||
`drag` retains dx/dy velocity. At rms=0: 98.4% retained; at rms=1: 97.4% retained. Small but
|
||||
accumulates across many frames.
|
||||
|
||||
#### Treble pulse
|
||||
|
||||
```
|
||||
pulse = 1 + treble * 0.25
|
||||
r_render = b.radius * pulse * max(0.15, b.life)
|
||||
```
|
||||
|
||||
Blobs swell up to 25% larger on treble hits. The `max(0.15, b.life)` also causes near-dead blobs
|
||||
to be clamped to 15% of their base radius rather than vanishing immediately.
|
||||
|
||||
#### Colour cycling
|
||||
|
||||
Sequential through `speakerPalette` (from `VisualizerPalette`). No randomisation; no RMS/bass
|
||||
influence on colour selection.
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Proposed new parameters
|
||||
|
||||
### 3a — Muon new parameters
|
||||
|
||||
All values are **multipliers** (1.0 = current default behaviour) or **direct overrides** where noted.
|
||||
|
||||
| Key | Type | Default | Range | Step | What it controls |
|
||||
|---|---|---|---|---|---|
|
||||
| `particle_count` | int | 220 | 40–500 | 10 | Number of particles. Directly impacts density and GPU cost. |
|
||||
| `rotation_speed` | float | 1.0 | 0.2–4.0 | 0.1 | Multiplies `pt.speed` base (0.002–0.006 rad/frame) and the drum rotation bonus (0.02). |
|
||||
| `glow_intensity` | float | 1.0 | 0.0–3.0 | 0.1 | Multiplies `glow = 14 + drum * 28`. At 0.0 glow is disabled. |
|
||||
| `particle_size` | float | 1.0 | 0.25–3.0 | 0.05 | Multiplies `size = 1.75 + drum * 3.0`. |
|
||||
| `drum_reactivity` | float | 1.0 | 0.0–2.0 | 0.1 | Scales the `drum` signal before it is applied to aperture, spacing, hue, size, and alpha. Does **not** affect the beat-detection threshold. |
|
||||
| `hue_range` | float | 1.0 | 0.0–3.0 | 0.1 | Multiplies the `drum * 20` and `treble * 15` hue drift coefficients. At 0.0 hue is locked to seed colour. |
|
||||
| `beat_sensitivity` | float | 1.0 | 0.2–3.0 | 0.1 | Divides the beat-preset cooldown (200 ms). Higher = preset cycles faster per beat. At 2.0, minimum 100 ms between presets. |
|
||||
| `ellipse_ratio` | float | 0.62 | 0.3–1.0 | 0.02 | Direct override of the y-axis squash factor. 1.0 = circular orbit; 0.3 = very flat ellipse. |
|
||||
|
||||
**Implementation note:** `drum_reactivity` should be applied as:
|
||||
```typescript
|
||||
const drum = clamp01(f.drumIntensity * (settings?.drum_reactivity ?? 1))
|
||||
```
|
||||
This single multiplier propagates through aperture, spacing, size, alpha, glow, and hue without
|
||||
needing separate coefficients per expression.
|
||||
|
||||
**Implementation note:** `particle_count` requires reinitialising the `particles` array. It should
|
||||
only be acted on inside the `useEffect` setup (as a dep), not inside the render loop.
|
||||
|
||||
### 3b — Fluid new parameters
|
||||
|
||||
| Key | Type | Default | Range | Step | What it controls |
|
||||
|---|---|---|---|---|---|
|
||||
| `background_persistence` | float | 1.0 | 0.2–3.0 | 0.1 | Multiplies `(1 - bass) * 0.16` background-fill alpha contribution. Higher = slower trail fade = longer ink persistence on quiet passages. |
|
||||
| `beat_burst_count` | float | 1.0 | 0.25–3.0 | 0.05 | Multiplies `3 + floor(bass * 4)` blob count per beat. |
|
||||
| `treble_pulse` | float | 1.0 | 0.0–3.0 | 0.1 | Multiplies the `treble * 0.25` blob swell factor. At 0.0 blobs do not pulse on treble. |
|
||||
| `drag` | float | 1.0 | 0.5–2.0 | 0.05 | Multiplies the velocity retention factor `0.984 - rms * 0.010`. Higher = blobs slow down faster (more viscous). |
|
||||
| `blob_life` | float | 1.0 | 0.25–3.0 | 0.05 | Multiplies initial blob life `0.7 + strength * 0.6`. More life = blobs survive more decay ticks. |
|
||||
|
||||
**Implementation note:** `background_persistence` modifies the background fill expression as:
|
||||
```typescript
|
||||
const bgAlpha = 0.20 + (1 - bass) * 0.16 * (settings?.background_persistence ?? 1)
|
||||
ctx.globalAlpha = Math.min(1, bgAlpha)
|
||||
```
|
||||
|
||||
**Implementation note:** `drag` modifies the drag expression as:
|
||||
```typescript
|
||||
const drag = (0.984 - rms * 0.010) / Math.max(0.1, settings?.drag ?? 1)
|
||||
```
|
||||
(dividing by `drag > 1` makes blobs retain less velocity = more viscous/heavier)
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Named presets
|
||||
|
||||
Presets are stored client-side only (not in manifest.json). The sidebar "Visualizer" section shows
|
||||
a preset row above the sliders; selecting a preset calls `patchVizSettings(presetValues)` for the
|
||||
active visualizer's keys only, leaving the other visualizer's values untouched.
|
||||
|
||||
A **Custom** pseudo-preset is shown whenever any slider deviates from the active preset's values.
|
||||
Selecting a named preset restores its exact values. The Reset button behaviour is unchanged (resets
|
||||
to manifest.json values or VIZ_DEFAULTS, which may or may not match any named preset).
|
||||
|
||||
### Muon presets
|
||||
|
||||
| Preset | `particle_count` | `rotation_speed` | `glow_intensity` | `particle_size` | `drum_reactivity` | `hue_range` | `beat_sensitivity` | `ellipse_ratio` |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| **Default** | 220 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 0.62 |
|
||||
| **Calm** | 120 | 0.5 | 0.4 | 0.7 | 0.5 | 0.4 | 0.5 | 0.55 |
|
||||
| **Energetic** | 350 | 2.0 | 2.0 | 1.4 | 1.8 | 1.8 | 2.0 | 0.68 |
|
||||
| **Dreamy** | 280 | 0.6 | 2.5 | 1.1 | 0.6 | 2.5 | 0.4 | 0.45 |
|
||||
| **Minimal** | 60 | 0.8 | 0.0 | 0.6 | 1.2 | 0.0 | 1.0 | 1.0 |
|
||||
| **Cosmic** | 400 | 1.5 | 1.8 | 1.8 | 1.4 | 2.0 | 1.5 | 0.38 |
|
||||
|
||||
**Calm** — sparse slow-moving particles with minimal glow and low audio reactivity. Good for
|
||||
background ambience during quiet songs.
|
||||
|
||||
**Energetic** — dense fast-spinning particles with strong glow and full audio reactivity. Best for
|
||||
high-BPM or hip-hop.
|
||||
|
||||
**Dreamy** — many particles with very high glow and wide hue drift but slow rotation. Works well
|
||||
for ballads and ambient music.
|
||||
|
||||
**Minimal** — few particles, no glow, locked hue. Almost geometric — the spiral structure is
|
||||
clearly visible without particle bloom.
|
||||
|
||||
**Cosmic** — maximum particles, strong glow, very flat ellipse. Fills the screen with a wide flat
|
||||
galaxy-like disc.
|
||||
|
||||
### Fluid presets
|
||||
|
||||
| Preset | `spawn_density` | `splat_force` | `decay` | `background_persistence` | `beat_burst_count` | `treble_pulse` | `drag` | `blob_life` |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| **Default** | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 |
|
||||
| **Ink** | 0.6 | 1.8 | 0.4 | 2.5 | 2.0 | 0.5 | 1.5 | 2.0 |
|
||||
| **Foam** | 3.0 | 0.5 | 2.5 | 0.4 | 2.5 | 2.0 | 0.6 | 0.5 |
|
||||
| **Lava** | 0.3 | 2.5 | 0.3 | 3.0 | 0.6 | 0.3 | 2.0 | 2.5 |
|
||||
| **Mist** | 1.5 | 0.4 | 1.5 | 1.0 | 0.7 | 1.5 | 0.8 | 0.8 |
|
||||
| **Storm** | 2.5 | 2.0 | 0.8 | 0.6 | 3.0 | 2.5 | 0.5 | 1.2 |
|
||||
|
||||
**Ink** — few large slow-moving blobs with very long trails. Dense, dark-ink-on-canvas look.
|
||||
|
||||
**Foam** — many tiny short-lived blobs, fast decay, strong treble pulse. Frothy/bubbly texture.
|
||||
|
||||
**Lava** — few massive blobs with very slow decay and high viscosity. Large glowing pools that
|
||||
linger. Good for slow, heavy music.
|
||||
|
||||
**Mist** — light ambient density, low force, medium decay. Soft diffuse glow with gentle treble
|
||||
shimmer.
|
||||
|
||||
**Storm** — high density, high force, many beat blobs, strong treble. Chaotic burst-heavy
|
||||
response for energetic tracks.
|
||||
|
||||
---
|
||||
|
||||
## Part 5 — Updated manifest.json schema additions
|
||||
|
||||
The new parameters extend the existing `manifest.visualizer.settings.muon` and
|
||||
`manifest.visualizer.settings.fluid` sub-objects. All new keys default to `1.0` (or the explicit
|
||||
default noted above) and are optional — absence means "use default".
|
||||
|
||||
```json
|
||||
{
|
||||
"visualizer": {
|
||||
"mode": "muon",
|
||||
"settings": {
|
||||
"quality": "balanced",
|
||||
"fps_cap": 60,
|
||||
"muon": {
|
||||
"radius_multiplier": 1.0,
|
||||
"aperture": 1.0,
|
||||
"spacing": 1.0,
|
||||
"particle_count": 220,
|
||||
"rotation_speed": 1.0,
|
||||
"glow_intensity": 1.0,
|
||||
"particle_size": 1.0,
|
||||
"drum_reactivity": 1.0,
|
||||
"hue_range": 1.0,
|
||||
"beat_sensitivity": 1.0,
|
||||
"ellipse_ratio": 0.62
|
||||
},
|
||||
"fluid": {
|
||||
"spawn_density": 1.0,
|
||||
"splat_force": 1.0,
|
||||
"decay": 1.0,
|
||||
"background_persistence": 1.0,
|
||||
"beat_burst_count": 1.0,
|
||||
"treble_pulse": 1.0,
|
||||
"drag": 1.0,
|
||||
"blob_life": 1.0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Updated parameter summary
|
||||
|
||||
**Muon** (11 keys total: 3 existing + 8 new):
|
||||
|
||||
| Key | New? | Sidebar widget | Range |
|
||||
|---|---|---|---|
|
||||
| `radius_multiplier` | existing | Slider | 0.25–2.0 |
|
||||
| `aperture` | existing | Slider | 0.25–2.0 |
|
||||
| `spacing` | existing | Slider | 0.25–2.0 |
|
||||
| `particle_count` | **new** | Slider (int) | 40–500 |
|
||||
| `rotation_speed` | **new** | Slider | 0.2–4.0 |
|
||||
| `glow_intensity` | **new** | Slider | 0.0–3.0 |
|
||||
| `particle_size` | **new** | Slider | 0.25–3.0 |
|
||||
| `drum_reactivity` | **new** | Slider | 0.0–2.0 |
|
||||
| `hue_range` | **new** | Slider | 0.0–3.0 |
|
||||
| `beat_sensitivity` | **new** | Slider | 0.2–3.0 |
|
||||
| `ellipse_ratio` | **new** | Slider | 0.30–1.0 |
|
||||
|
||||
**Fluid** (8 keys total: 3 existing + 5 new):
|
||||
|
||||
| Key | New? | Sidebar widget | Range |
|
||||
|---|---|---|---|
|
||||
| `spawn_density` | existing | Slider | 0.25–2.0 |
|
||||
| `splat_force` | existing | Slider | 0.25–2.0 |
|
||||
| `decay` | existing | Slider | 0.25–2.0 |
|
||||
| `background_persistence` | **new** | Slider | 0.2–3.0 |
|
||||
| `beat_burst_count` | **new** | Slider | 0.25–3.0 |
|
||||
| `treble_pulse` | **new** | Slider | 0.0–3.0 |
|
||||
| `drag` | **new** | Slider | 0.5–2.0 |
|
||||
| `blob_life` | **new** | Slider | 0.25–3.0 |
|
||||
|
||||
---
|
||||
|
||||
## Part 6 — Sidebar layout update (Section 4 revision)
|
||||
|
||||
The `PerformanceSidebar` Section 4 layout should be updated to:
|
||||
|
||||
```
|
||||
├──────────────────────────────────────────┤
|
||||
│ ▾ VISUALIZER (hidden for Static/Video) │
|
||||
│ [Calm] [Default] [Energetic] [Dreamy] │ ← preset pills
|
||||
│ [Minimal] [Cosmic] [Custom] │
|
||||
│ Quality [Low] [Balanced] [High] │
|
||||
│ FPS cap [30] [60] │
|
||||
│ ─────────────────────────────────────── │
|
||||
│ Particle count ███░░░░░ 220 │
|
||||
│ Rotation speed ████░░░░ 1.00× │
|
||||
│ Glow ████░░░░ 1.00× │
|
||||
│ Particle size ████░░░░ 1.00× │
|
||||
│ Drum reactivity ████░░░░ 1.00× │
|
||||
│ Hue range ████░░░░ 1.00× │
|
||||
│ Beat sensitivity████░░░░ 1.00× │
|
||||
│ Ellipse ratio ████████ 0.62 │
|
||||
│ Radius ████░░░░ 1.00× │
|
||||
│ Aperture ████░░░░ 1.00× │
|
||||
│ Spacing ████░░░░ 1.00× │
|
||||
│ [Reset to song defaults] │
|
||||
└──────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
(Fluid variant swaps the parameter rows for its 8 keys; preset pills change to Ink/Foam/Lava/Mist/Storm/Default.)
|
||||
|
||||
Presets are stored as a local constant in `PerformanceSidebar.tsx` — not in manifest.json and not
|
||||
in localStorage. They are not user-editable; users adjust sliders individually.
|
||||
|
||||
---
|
||||
|
||||
## Part 7 — Implementation impact on existing files
|
||||
|
||||
### `MuonVisualizerCanvas.tsx`
|
||||
|
||||
- Expand `settings` interface with the 8 new keys.
|
||||
- Apply `drum_reactivity` multiplier to the `drum` local variable at the top of `render()`.
|
||||
- Apply `rotation_speed` to `pt.speed` in particle init and to `drum * 0.02` rotation bonus.
|
||||
- Apply `glow_intensity` to `glow = 14 + drum * 28`.
|
||||
- Apply `particle_size` to `size = 1.75 + drum * 3.0`.
|
||||
- Apply `hue_range` to the `drum * 20` and `treble * 15` hue coefficients.
|
||||
- Apply `beat_sensitivity` by dividing the 200 ms cooldown: `200 / beatSensitivity`.
|
||||
- Use `ellipse_ratio` in place of the hardcoded `0.62`.
|
||||
- `particle_count`: reinitialise `particles` inside `useEffect` when the dep changes (add
|
||||
`settings?.particle_count` to the dep array or detect the change and reinit).
|
||||
|
||||
### `FluidVisualizerCanvas.tsx`
|
||||
|
||||
- Expand `settings` interface with the 5 new keys.
|
||||
- Apply `background_persistence` to the background fill alpha expression.
|
||||
- Apply `beat_burst_count` to `count = 3 + floor(bass * 4)`.
|
||||
- Apply `treble_pulse` to `pulse = 1 + treble * 0.25`.
|
||||
- Apply `drag` multiplier to the drag expression.
|
||||
- Apply `blob_life` multiplier to `life = 0.7 + strength * 0.6` in `spawnBlob`.
|
||||
|
||||
### `VisualizerBackdrop.tsx`
|
||||
|
||||
No change needed — already passes `settings?.muon` and `settings?.fluid` to the respective
|
||||
canvases. New keys are transparent as long as the canvas interfaces are expanded.
|
||||
|
||||
### `PerformanceSidebar.tsx`
|
||||
|
||||
- Add preset constant arrays for Muon and Fluid.
|
||||
- Add preset pill row above Quality/FPS controls.
|
||||
- Add 8 Muon sliders and 5 Fluid sliders (in addition to existing 3+3).
|
||||
- Detect deviation from active preset → show "Custom" as active.
|
||||
|
||||
### `LyricsLivePreview.tsx` / `VizSettings` type
|
||||
|
||||
Expand `VizSettings` and `VIZ_DEFAULTS` with the 13 new keys across both sub-objects.
|
||||
|
||||
### Backend — `_normalize_visualizer_config` (optional)
|
||||
|
||||
The backend currently writes only `radius_multiplier`, `aperture`, `spacing`, `spawn_density`,
|
||||
`splat_force`, `decay` to `manifest.json`. If per-song presets are ever authored at export time,
|
||||
the backend normaliser will need to accept and round-trip the new keys. For the sidebar use-case
|
||||
(live adjustment only), backend changes are not required.
|
||||
@@ -0,0 +1,368 @@
|
||||
# Visualizer Parameter Audit & Fix Notes
|
||||
|
||||
Date: 2026-04-08
|
||||
Scope: Bugs in the Muon Music Visualizer source that break dynamic visuals; corrections to the integration plan's parameter assumptions; and the definitive list of manifest.json keys for both visualizers.
|
||||
|
||||
## Outstanding-item closure (2026-04-08)
|
||||
|
||||
1. **Manifest schema mismatch (Part 3/4 vs backend reality):** ✅ **fixed**
|
||||
Parts 3 and 4 were rewritten to match `_normalize_visualizer_config` and the actual `manifest.visualizer.settings` structure written by backend export.
|
||||
2. **Integration plan still had broken mapping snippets:** ✅ **fixed**
|
||||
The Muon mapping section and Fluid config-mapping snippet in `docs/2026-04-08_Visualizer_Integration_Plan.md` were corrected to match implemented behaviour.
|
||||
3. **Dissipation-direction uncertainty for future WebGL fluid port:** ✅ **documented + guarded**
|
||||
The caveat remains explicit in both docs. Current implementation uses the canvas blob renderer (no `DENSITY_DISSIPATION` shader param); direction must be verified when/if the WebGL sim is vendored.
|
||||
|
||||
Implementation traceability:
|
||||
- Frontend now threads `manifest.visualizer.settings` into the runtime renderers via `Karaoke.tsx` / `KaraokeTV.tsx` → `KaraokePerformanceMode.tsx` → `LyricsLivePreview.tsx` → `VisualizerBackdrop.tsx`.
|
||||
- Muon renderer applies `settings.muon.radius_multiplier`, `settings.muon.aperture`, and `settings.muon.spacing` as post-audio multipliers.
|
||||
- Fluid renderer applies `settings.fluid.spawn_density`, `settings.fluid.splat_force`, and `settings.fluid.decay` as post-audio multipliers.
|
||||
- Integration plan snippets were corrected in `docs/2026-04-08_Visualizer_Integration_Plan.md` for Corrections A/B/C.
|
||||
|
||||
Post-deploy runtime investigation (Cuidate SEK):
|
||||
- Symptom: visualizer renders but appears static/non-reactive in multi-stem playback.
|
||||
- Root cause: `useAudioFeatures()` could execute before the multi-stem WebAudio graph was built (`gainRefs.current` empty). The hook then exited early with no tap node and did not reconnect, so `bass/treble/drumIntensity` stayed near zero.
|
||||
- Fix implemented: in `LyricsLivePreview.tsx`, audio feature extraction is now gated by `audioFeatureTapReady = !isMultiStem || Object.keys(gainRefs.current).length > 0`, and `useAudioFeatures.enabled` is set to `isVisualizerMode && audioFeatureTapReady`.
|
||||
- Result: the hook first enables only after stem gain nodes exist, ensuring analyser attachment and live feature updates.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Muon Music Visualizer bugs
|
||||
|
||||
Three bugs exist in the **original muon-music-visualizer source** that collectively explain why dynamic visuals fail or look static during playback. All three must be corrected in the StemEngine port.
|
||||
|
||||
---
|
||||
|
||||
### Bug 1 — `hueControl` always returns 0 during playback (colour is frozen)
|
||||
|
||||
**File:** `src/CoreControls/effects.js`
|
||||
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
|
||||
|
||||
```javascript
|
||||
// ORIGINAL (broken)
|
||||
export const hueControl = (scaler) => {
|
||||
const time = Date.now() * 0.00015;
|
||||
const hue = ((360 * (1.0 + !scaler && time + scaler * 10)) % 360) / 360;
|
||||
return hue;
|
||||
};
|
||||
```
|
||||
|
||||
**What it does:** `!scaler` is a boolean. When `scaler` is any non-zero number (i.e. whenever audio is playing), `!scaler === false`. The short-circuit `false && (time + scaler * 10)` evaluates to `false` (cast to `0`). So `hue = ((360 * 1.0) % 360) / 360 = 0` — permanently red — whenever music is playing.
|
||||
|
||||
Hue only animates when `scaler === 0` (silence), which is the opposite of the intended behaviour.
|
||||
|
||||
**Fix:**
|
||||
|
||||
```typescript
|
||||
// CORRECTED
|
||||
export function hueControl(scaler: number): number {
|
||||
const time = Date.now() * 0.00015;
|
||||
// When scaler > 0 (audio playing), modulate hue by scaler.
|
||||
// When silent, fall back to slow time-based drift.
|
||||
const hue = ((360 * (1.0 + (scaler > 0 ? scaler * 10 : time))) % 360) / 360;
|
||||
return hue;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Bug 2 — `baseFr` and `trebleFr` are named backwards (kick drum has no visual impact)
|
||||
|
||||
**File:** `src/CoreControls/audio.js`
|
||||
**Status: IMPLEMENTED** — `frontend/src/hooks/useAudioFeatures.ts` (`avgSlice` bin ranges + `deriveDrumIntensity` exponential scalers)
|
||||
|
||||
```javascript
|
||||
// ORIGINAL (misleadingly named)
|
||||
const lowerHalfArray = dataArray.slice(0, dataArray.length / 2 - 1); // LOW frequencies = bass
|
||||
const upperHalfArray = dataArray.slice(dataArray.length / 2 - 1, ...); // HIGH frequencies = treble
|
||||
|
||||
const lowerAvgFr = Operations.avg(lowerHalfArray) / lowerHalfArray.length; // bass average
|
||||
const upperAvgFr = Operations.avg(upperHalfArray) / upperHalfArray.length; // treble average
|
||||
|
||||
// BUG: upper (treble) goes into baseFr, lower (bass) goes into trebleFr
|
||||
const baseFr = Operations.modulate(upperAvgFr, 0, 1, 0, 4) / 50; // ← actually treble
|
||||
const trebleFr = Operations.modulate(lowerAvgFr, 0, 1, 0, 4) / 200; // ← actually bass
|
||||
```
|
||||
|
||||
The `lowerHalfArray` covers **low frequencies (bass/kick)** and `upperHalfArray` covers **high frequencies (treble/cymbals)**, but the results are assigned to the opposite variable names. This propagates into `exponentialBassScaler` and `exponentialTrebleScaler`, which are then used in `wave.js`:
|
||||
|
||||
```javascript
|
||||
// wave.js — beatScaler formula
|
||||
beatScaler = exponentialTrebleScaler * 3.14 + exponentialBassScaler * 6.24 + 1.7
|
||||
```
|
||||
|
||||
Because of the swap, kick-drum energy ends up in `exponentialTrebleScaler` (multiplied by the smaller 3.14 coefficient) and cymbal/treble energy ends up in `exponentialBassScaler` (multiplied by the larger 6.24 coefficient). Kick hits produce weaker visual response than cymbals — the opposite of the intended drum-intensity drive.
|
||||
|
||||
**Fix:** Correct the variable assignment to match the actual frequency content:
|
||||
|
||||
```typescript
|
||||
// CORRECTED — names now match frequency content
|
||||
const bassAvgFr = Operations.modulate(lowerAvgFr, 0, 1, 0, 4) / 50; // low freq → bass
|
||||
const trebleAvgFr = Operations.modulate(upperAvgFr, 0, 1, 0, 4) / 200; // high freq → treble
|
||||
|
||||
const exponentialBassScaler = 1.001 * (1 + bassAvgFr * 2) - 1.001;
|
||||
const exponentialTrebleScaler = 1.001 * (1 + trebleAvgFr * 2) - 1.001;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Bug 3 — `coreScaler` amplification is dead code
|
||||
|
||||
**File:** `src/CoreControls/audio.js`, line 24
|
||||
**Status: IMPLEMENTED** — `frontend/src/hooks/useAudioFeatures.ts` (`deriveDrumIntensity` uses the exponential scaler formula; `drumIntensity` is computed fresh each frame so no compounding risk)
|
||||
|
||||
```javascript
|
||||
// ORIGINAL (no-op)
|
||||
coreScaler < 10 && coreScaler * 1.2;
|
||||
```
|
||||
|
||||
This evaluates `coreScaler * 1.2` but throws the result away. `coreScaler` is never actually amplified. The intended logic was to cap-and-boost values below 10.
|
||||
|
||||
**Fix:**
|
||||
|
||||
```typescript
|
||||
// CORRECTED
|
||||
if (coreScaler < 10) coreScaler *= 1.2;
|
||||
```
|
||||
|
||||
> **⚠️ Compounding risk:** If `coreScaler` is **persistent state** that carries over between animation frames (i.e. the same variable is reused rather than recomputed from audio data each frame), this `*= 1.2` fires on every frame where `coreScaler < 10` and compounds until it crosses 10. If that is the case, cap the result explicitly: `coreScaler = Math.min(coreScaler * 1.2, 10)`. Verify whether `coreScaler` is a fresh per-frame value or a running accumulator before porting.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Integration plan corrections
|
||||
|
||||
The integration plan (`2026-04-08_Visualizer_Integration_Plan.md`) contains incorrect assumptions about Muon's parameter system and both visualizers' audio mappings.
|
||||
|
||||
---
|
||||
|
||||
### Correction A — `colorSpectrum` is not a hue angle (range is 3–30, not 0–360)
|
||||
|
||||
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
|
||||
The canvas 2D port does not use a `colorSpectrum` geometry parameter. The equivalent hue-drift line was corrected: treble drift is capped at ±15° (from ±40°) to stay within the manifest palette colour family.
|
||||
|
||||
The plan states:
|
||||
> `colorSpectrum` is driven by treble energy independently — `colorSpectrum = (baseColorSpectrum + treble * 120) % 360`
|
||||
|
||||
**Actual GUI definition:**
|
||||
```javascript
|
||||
{ name: "colorSpectrum", min: 3, max: 30, step: 3, folder: particleControlsFolder }
|
||||
```
|
||||
|
||||
`colorSpectrum` is a **colour segment count** passed directly to `generateParticlesSpiral` — it controls how many colour bands exist in the spiral geometry, not a hue rotation angle. Its range is 3–30 (integer multiples of 3). Adding 120 would put it 4× out of range and corrupt the geometry.
|
||||
|
||||
**Corrected mapping:**
|
||||
|
||||
```typescript
|
||||
// colorSpectrum: integer 3–30. Audio can nudge it by ±1 step (±3) at most.
|
||||
// Convert to step index first, apply the treble offset, then multiply back.
|
||||
// Do NOT add ±1 to the raw value before dividing: when the base is already a
|
||||
// multiple of 3, the ±1 delta is always absorbed by Math.round( / 3) and the
|
||||
// output never changes (e.g. Math.round(13/3)*3 = 4*3 = 12 — same as 12).
|
||||
params.colorSpectrum = clamp(
|
||||
(Math.round(baseColorSpectrum / 3) + Math.round((treble - 0.5) * 2)) * 3,
|
||||
3, 30
|
||||
)
|
||||
```
|
||||
|
||||
The hue cycling effect the plan was trying to achieve is controlled by `hueControl()` (fixed in Bug 1 above) — not `colorSpectrum`.
|
||||
|
||||
---
|
||||
|
||||
### Correction B — `radiusMultiplier` is managed by preset cycling, not direct audio mapping
|
||||
|
||||
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/MuonVisualizerCanvas.tsx`
|
||||
Added `RADIUS_PRESETS` array and beat-triggered preset cycling with lerp interpolation (`radiusCurrent → radiusTarget`). Direct per-frame `drum * 1.12` mapping removed.
|
||||
|
||||
The plan states:
|
||||
> `params.radiusMultiplier = lerp(0.4, 1.8, easeIn(drumIntensity))`
|
||||
|
||||
**Actual behaviour:** `radiusMultiplier` is controlled by `wavePresetController`, which cycles through the `visualizationPresets` array via GSAP animation:
|
||||
|
||||
```javascript
|
||||
// visualizationPresets — 16 values, range 0.6603–0.7536
|
||||
export const visualizationPresets = [
|
||||
0.6604, 0.6606, 0.6608, 0.6609, 0.661, 0.6612, 0.6614, 0.7536,
|
||||
0.6615, 0.6613, 0.6611, 0.661, 0.66081, 0.6607, 0.6605, 0.6603,
|
||||
];
|
||||
```
|
||||
|
||||
`wavePresetController` triggers a GSAP-animated transition to the next preset whenever `Math.abs(delta) > params.deltaResponseLimit`. Directly lerping `radiusMultiplier` to 1.8 would both exceed the preset range and fight the GSAP animation on every frame.
|
||||
|
||||
**Corrected approach:** Do not write `radiusMultiplier` from audio data. Instead, tune `deltaResponseLimit` (lower = more frequent preset transitions) to control how reactive the preset cycling is to audio beats. The `beatScaler` uniform (in `sineWavePropagation`) already handles per-frame amplitude pulsing correctly.
|
||||
|
||||
---
|
||||
|
||||
### Correction C — Fluid sim `DENSITY_DISSIPATION` mapping is inverted
|
||||
|
||||
**Status: IMPLEMENTED** — `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
|
||||
The canvas blob sim uses `decay` (per-frame lifetime drain) and `drag` (velocity retention). Fixed: `decay` now driven by `bass` (loud bass → slower decay → longer trails); `drag` driven by `rms` (overall loudness → more momentum). Previously both were driven by `rms`, making trail length insensitive to bass.
|
||||
The ⚠️ dissipation-direction verification note remains relevant if the full WebGL fluid sim is later vendored.
|
||||
|
||||
The plan states:
|
||||
> `DENSITY_DISSIPATION = lerp(0.97, 0.999, 1 - bass)` — "longer trails on loud bass"
|
||||
|
||||
**Actual semantics:** A **higher** `DENSITY_DISSIPATION` value means ink fades **faster** (more dissipation per frame). A value of 0.999 dissipates nearly instantly; 0.9 produces long-lived trails.
|
||||
|
||||
The plan's formula gives 0.999 at high bass — ink disappears almost immediately on loud hits — the opposite of longer trails.
|
||||
|
||||
**Corrected mapping:**
|
||||
|
||||
```typescript
|
||||
// Lower value = slower dissipation = longer trails on loud bass
|
||||
config.DENSITY_DISSIPATION = lerp(0.90, 0.98, 1 - bass); // loud bass → 0.90 → long trails
|
||||
config.VELOCITY_DISSIPATION = lerp(0.85, 0.99, 1 - rms); // loud overall → slower decay
|
||||
```
|
||||
|
||||
> **⚠️ Verify dissipation direction before implementing.** The standard PavelDoGreat/WebGL-Fluid-Simulation advection shader applies dissipation as a **retention multiplier** — `density *= uDissipation` — which makes 0.999 the near-permanent (longest-trail) value and 0.9 the faster-fading value (**higher = longer trails**). If the vendored build follows that convention, the "lower = longer" semantics stated here are inverted: the corrected mapping would *shorten* trails on loud bass, which is the opposite of the intent. Confirm by inspecting the advection shader in the vendored source before implementing either mapping.
|
||||
|
||||
---
|
||||
|
||||
### Correction D — `generateColor()` output is too dim for programmatic splats
|
||||
|
||||
**Status: NOT APPLICABLE** — `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
|
||||
The canvas blob sim uses `nextPaletteColor()` which returns manifest palette hex strings directly — no 0.15 brightness scaling. The `generateColor()` bug was specific to the original WebGL fluid sim's colour generator and does not exist in the current implementation.
|
||||
|
||||
`generateColor()` in the fluid sim multiplies RGB values by 0.15, producing near-invisible ink. For audio-driven splats, use full-brightness colours:
|
||||
|
||||
```typescript
|
||||
function audioSplatColor(hue: number): { r: number; g: number; b: number } {
|
||||
// HSV → RGB at full saturation and value, not scaled by 0.15
|
||||
const [r, g, b] = hsvToRgb(hue, 0.8, 1.0);
|
||||
return { r, g, b };
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Correction E — Direct `splat()` calls bypass aspect-ratio correction
|
||||
|
||||
**Status: NOT APPLICABLE** — `frontend/src/components/visualizers/FluidVisualizerCanvas.tsx`
|
||||
The canvas blob sim uses pixel-space coordinates and velocities; there is no WebGL normalised coordinate system. Aspect-ratio correction is only needed when integrating the real WebGL fluid sim (`PavelDoGreat/WebGL-Fluid-Simulation`). The `audioSplat()` wrapper pattern documented below remains the correct approach for that integration.
|
||||
|
||||
The plan calls `fluidSim.splat(x, y, dx, dy, color)` directly. The original source's `splatPointer()` applies `correctDeltaX/Y` which normalises velocity deltas for the canvas aspect ratio. Without this, splats are stretched or squashed on non-square viewports.
|
||||
|
||||
**Fix:**
|
||||
|
||||
```typescript
|
||||
function audioSplat(sim: FluidSim, x: number, y: number, dx: number, dy: number, color: Color) {
|
||||
const aspectRatio = sim.canvas.width / sim.canvas.height;
|
||||
// correctDeltaX: dx * (1 / aspectRatio) — from the original source
|
||||
sim.splat(x, y, dx / aspectRatio, dy, color);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Actual manifest.json `visualizer` schema
|
||||
|
||||
> **⚠️ Correction to original audit:** The parameter lists below do NOT reflect what is actually written to `manifest.json`. The backend (`_normalize_visualizer_config` in `project_assets_service.py`) produces a compact schema of **relative multipliers** and global quality settings, not the full set of raw GUI parameters described in the original Part 3 tables. The original tables described Muon's dat.gui parameters directly; those are not persisted and are not read by the frontend at runtime.
|
||||
|
||||
### What `manifest.visualizer` actually contains
|
||||
|
||||
```json
|
||||
{
|
||||
"visualizer": {
|
||||
"mode": "muon",
|
||||
"settings": {
|
||||
"quality": "balanced",
|
||||
"fps_cap": 60,
|
||||
"muon": {
|
||||
"radius_multiplier": 1.0,
|
||||
"aperture": 1.0,
|
||||
"spacing": 1.0
|
||||
},
|
||||
"fluid": {
|
||||
"spawn_density": 1.0,
|
||||
"splat_force": 1.0,
|
||||
"decay": 1.0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
All numeric values are **scale multipliers** (1.0 = default behaviour). They allow per-song user tuning of animation intensity without overriding the audio-driven values. The frontend currently reads only `manifest.visualizer.mode` at startup to set the initial `LyricsVisualStyle`; the multiplier sub-objects are available for future use.
|
||||
|
||||
### `visualizer` keys — top level
|
||||
|
||||
| Key | Type | Values | Default | Notes |
|
||||
|---|---|---|---|---|
|
||||
| `mode` | string | `"muon"` \| `"fluid"` \| `"auto"` | `"muon"` | Initial visualizer mode. `"auto"` selects based on device performance (future). Validated by `_normalize_visualizer_config`; invalid values fall back to `"muon"`. |
|
||||
|
||||
### `visualizer.settings` keys
|
||||
|
||||
| Key | Type | Notes |
|
||||
|---|---|---|
|
||||
| `quality` | string | Rendering quality preset: `"low"` / `"balanced"` / `"high"`. Not yet read by frontend (reserved for future GPU scaling). |
|
||||
| `fps_cap` | int | Target frame-rate cap. Default 60. |
|
||||
|
||||
### `visualizer.settings.muon` keys
|
||||
|
||||
These are **multipliers** applied on top of the audio-driven animation values, not raw geometry parameters.
|
||||
|
||||
| Key | Type | Default | Effect when > 1.0 |
|
||||
|---|---|---|---|
|
||||
| `radius_multiplier` | float | 1.0 | Scales the preset-cycling radius range outward. |
|
||||
| `aperture` | float | 1.0 | Scales the spiral aperture range (wider opening). |
|
||||
| `spacing` | float | 1.0 | Scales inter-particle spacing (larger gaps). |
|
||||
|
||||
### `visualizer.settings.fluid` keys
|
||||
|
||||
| Key | Type | Default | Effect when > 1.0 |
|
||||
|---|---|---|---|
|
||||
| `spawn_density` | float | 1.0 | Multiplies the ambient blob spawn probability per frame. |
|
||||
| `splat_force` | float | 1.0 | Multiplies blob velocity/size on spawn. |
|
||||
| `decay` | float | 1.0 | Multiplies per-frame blob lifetime drain (higher = shorter trails). |
|
||||
|
||||
**Not stored** — runtime-derived or global:
|
||||
- All audio-driven computed values (`drumIntensity`, `bass`, `treble`, `rms`, `beatDetected`)
|
||||
- Colour — derived from `metadata.dynamic_palette` at runtime; not written to `visualizer`
|
||||
- Individual Muon dat.gui parameters (`maxPoints`, `colorSpectrum`, `divisions`, etc.) — not persisted
|
||||
- WebGL fluid sim config (`DENSITY_DISSIPATION`, `CURL`, `BLOOM_*`, etc.) — not persisted
|
||||
- `SIM_RESOLUTION`, `DYE_RESOLUTION`, and other global performance settings
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Updated manifest.json schema
|
||||
|
||||
```json
|
||||
{
|
||||
"format": "stemengine-karaoke",
|
||||
"version": 1,
|
||||
"project": { "id": "...", "name": "..." },
|
||||
"metadata": {
|
||||
"title": "...",
|
||||
"artist": "...",
|
||||
"dynamic_palette": {
|
||||
"source": "embedded_cover_art",
|
||||
"seed_color": "#a03060",
|
||||
"background_color": "#1a0510",
|
||||
"text_color": "#ffffff",
|
||||
"highlight_color": "#d06090",
|
||||
"word_highlight_color": "#e080b0",
|
||||
"speaker_palette": ["#a03060", "..."]
|
||||
}
|
||||
},
|
||||
"contents": {
|
||||
"lyrics": "lyrics/default.elrc",
|
||||
"lyrics_tracks": { "default": "lyrics/default.elrc" },
|
||||
"video": null,
|
||||
"stems": [{ "name": "vocals", "file": "stems/vocals.wav", "bytes": 12345678 }]
|
||||
},
|
||||
"meta": {
|
||||
"stem_count": 4,
|
||||
"has_video": false,
|
||||
"synchronization": { "reference_track": "source", "track_offsets_seconds": {}, "video_source": "", "last_sync": null }
|
||||
},
|
||||
"visualizer": {
|
||||
"mode": "muon",
|
||||
"settings": {
|
||||
"quality": "balanced",
|
||||
"fps_cap": 60,
|
||||
"muon": { "radius_multiplier": 1.0, "aperture": 1.0, "spacing": 1.0 },
|
||||
"fluid": { "spawn_density": 1.0, "splat_force": 1.0, "decay": 1.0 }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Only `visualizer.mode` is read by the frontend at startup. Colour is always sourced from `metadata.dynamic_palette`. Both `muon` and `fluid` sub-objects are always written (with defaults) so the schema is stable regardless of which mode is active.
|
||||
@@ -0,0 +1,302 @@
|
||||
# Visualizer GUI Parameter Map
|
||||
|
||||
Date: 2026-04-09
|
||||
Sources:
|
||||
- **Muon** — https://muon-vis.netlify.app / https://github.com/najafmohammed/muon-music-visualizer
|
||||
- **Fluid** — https://paveldogreat.github.io/WebGL-Fluid-Simulation / https://github.com/PavelDoGreat/WebGL-Fluid-Simulation
|
||||
|
||||
Scope: Every user-adjustable parameter exposed in the original GUIs for both visualizers.
|
||||
Recorded verbatim from source — ranges, defaults, and step sizes are as defined in the dat.gui
|
||||
calls unless otherwise noted. Effect descriptions are derived from the source code.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Muon Music Visualizer
|
||||
|
||||
### Default `params` object (from `src/index.js`)
|
||||
|
||||
```javascript
|
||||
let params = {
|
||||
maxPoints: 0, // Note: 0 means "use all computed points"
|
||||
colorSpectrum: 3,
|
||||
aperture: 3,
|
||||
sineCounterMultiplier: 1,
|
||||
idleMultiplier: 0.27,
|
||||
particleMirror: true,
|
||||
radiusMultiplier: 0.66,
|
||||
dynamicRadius: true,
|
||||
updateLockInterval: 0.17,
|
||||
deltaResponseLimit: 0.005,
|
||||
visualizationPreset: true,
|
||||
spacing: 1,
|
||||
syncColors: true,
|
||||
divisions: 21,
|
||||
lifespan: 200,
|
||||
noiseScale: 0.3,
|
||||
noiseForce: 1,
|
||||
timeMult: 0.01,
|
||||
enableMonoColor: false,
|
||||
monoColor: { h: 350, s: 0.9, v: 0.3 },
|
||||
}
|
||||
```
|
||||
|
||||
### Post-processing defaults (from `src/CoreControls/postProcessing.js`)
|
||||
|
||||
```javascript
|
||||
// UnrealBloomPass
|
||||
bloomStrength: 0.55
|
||||
bloomThreshold: 0.35
|
||||
bloomRadius: 0.6
|
||||
|
||||
// AfterimagePass
|
||||
damp: 0.86
|
||||
```
|
||||
|
||||
### `visualizationPresets` array (from `src/CoreControls/params.js`)
|
||||
|
||||
16 values cycling via `wavePresetController` on beats:
|
||||
```
|
||||
[0.6604, 0.6606, 0.6608, 0.6609, 0.661, 0.6612, 0.6614, 0.7536,
|
||||
0.6615, 0.6613, 0.6611, 0.661, 0.66081, 0.6607, 0.6605, 0.6603]
|
||||
```
|
||||
These are `radiusMultiplier` target values. Range: 0.6603–0.7536.
|
||||
|
||||
---
|
||||
|
||||
### GUI parameter table (from `src/GUI/index.js`)
|
||||
|
||||
#### Root GUI (no folder)
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| radiusMultiplier | `radiusMultiplier` | Slider | 0.01 | 1 | 0.0001 | 0.66 | Scales the spiral radius range that `wavePresetController` cycles through. Lower = tighter spiral. |
|
||||
| side view | `sideView` | Button | — | — | — | — | Triggers GSAP camera animation to a side perspective. |
|
||||
| reset | `reset` | Button | — | — | — | — | Resets all params to defaults and restores camera. |
|
||||
|
||||
#### Particle Controls folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| spacing | `spacing` | Slider | 0 | 1 | 0.01 | 1 | Inter-ring gap in the spiral. Higher = rings farther apart. |
|
||||
| maxPoints | `maxPoints` | Slider | 0 | 12240 | 360 | 0 | Number of rendered particles. 0 = full computed count. |
|
||||
| colorSpectrum | `colorSpectrum` | Slider | 3 | 30 | 3 | 3 | Number of colour bands in the spiral geometry (integer multiples of 3). **Not a hue angle.** Higher = more colour segments visible. |
|
||||
| aperture | `aperture` | Slider | 0 | π (~3.14) | 0.01 | 3 | Controls the opening angle of the spiral. Near 0 = very tight helix; near π = fully open ellipse. |
|
||||
| particleMirror | `particleMirror` | Toggle | — | — | — | true | When on, duplicates the particle system on both sides of the spiral axis. |
|
||||
|
||||
#### Controls folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| dynamicRadius | `dynamicRadius` | Toggle | — | — | — | true | Enables audio-reactive radius changes. When off, `radiusMultiplier` stays fixed at its current value and `wavePresetController` does not cycle. |
|
||||
| visualizationPreset | `visualizationPreset` | Toggle | — | — | — | true | Enables GSAP-animated preset cycling via `wavePresetController` on beat detection. When off, radius stays fixed. |
|
||||
| idleMultiplier | `idleMultiplier` | Slider | 0 | 2 | 0.01 | 0.27 | When audio is not playing, multiplies elapsed time to advance `sineCounter` at reduced speed for gentle background animation. Higher = faster idle motion. |
|
||||
|
||||
#### Audio Controls folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| deltaResponseLimit | `deltaResponseLimit` | Slider | 0.001 | 0.01 | 0.001 | 0.005 | Minimum beat-delta required to trigger a preset advance in `wavePresetController`. Lower = more sensitive (advances faster on softer beats). |
|
||||
|
||||
#### Emission Controls folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| enableMonoColor | `enableMonoColor` | Toggle | — | — | — | false | When on, replaces audio-driven HSL cycling with a single fixed colour (`monoColor`). |
|
||||
| divisions | `divisions` | Slider | 1 | 150 | 1 | 21 | Number of frequency bins used to drive wave geometry segmentation. Higher = more frequency detail in the wave shape, more geometry vertices. |
|
||||
| lifespan | `lifespan` | Slider | 10 | 250 | 1 | 200 | Particle lifetime in milliseconds before removal and reuse. Lower = faster particle turnover / more chaotic look. |
|
||||
| monoColor | `monoColor` | Color (HSV) | — | — | — | h:350 s:0.9 v:0.3 | Fixed colour used when `enableMonoColor` is on. Expressed as HSV; h=350 is deep red/magenta. |
|
||||
|
||||
#### Noise Controls folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| noiseScale | `noiseScale` | Slider | 0 | 1 | 0.0001 | 0.3 | Magnitude of noise texture applied to particle positions. 0 = perfectly regular spiral; 1 = maximum noise displacement. |
|
||||
| noiseForce | `noiseForce` | Slider | 0 | 1 | 0.0001 | 1 | Multiplier on noise-driven particle velocity/force. Lower = noise displaces particles more slowly. |
|
||||
|
||||
#### Post-processing (not in GUI by default — hardcoded in `postProcessing.js`)
|
||||
|
||||
| Parameter | Default | Effect |
|
||||
|---|---|---|
|
||||
| `bloomStrength` | 0.55 | Bloom glow intensity (Three.js UnrealBloomPass). |
|
||||
| `bloomThreshold` | 0.35 | Brightness threshold above which bloom is applied. |
|
||||
| `bloomRadius` | 0.6 | Spread radius of the bloom effect. |
|
||||
| `damp` (afterimage) | 0.86 | Afterimage persistence. 1.0 = permanent trail; 0.0 = no trail. |
|
||||
|
||||
> **Note:** `sineCounterMultiplier`, `updateLockInterval`, `syncColors`, `timeMult` are in the
|
||||
> `params` object and affect rendering but are **not exposed in the GUI**. They cannot be changed
|
||||
> at runtime on muon-vis.netlify.app.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — WebGL Fluid Simulation
|
||||
|
||||
### Default `config` object (from `script.js`)
|
||||
|
||||
```javascript
|
||||
let config = {
|
||||
SIM_RESOLUTION: 128,
|
||||
DYE_RESOLUTION: 1024,
|
||||
CAPTURE_RESOLUTION: 512,
|
||||
DENSITY_DISSIPATION: 1,
|
||||
VELOCITY_DISSIPATION: 0.2,
|
||||
PRESSURE: 0.8,
|
||||
PRESSURE_ITERATIONS: 20,
|
||||
CURL: 30,
|
||||
SPLAT_RADIUS: 0.25,
|
||||
SPLAT_FORCE: 6000,
|
||||
SHADING: true,
|
||||
COLORFUL: true,
|
||||
COLOR_UPDATE_SPEED: 10,
|
||||
PAUSED: false,
|
||||
BACK_COLOR: { r: 0, g: 0, b: 0 },
|
||||
TRANSPARENT: false,
|
||||
BLOOM: true,
|
||||
BLOOM_ITERATIONS: 8,
|
||||
BLOOM_RESOLUTION: 256,
|
||||
BLOOM_INTENSITY: 0.8,
|
||||
BLOOM_THRESHOLD: 0.6,
|
||||
BLOOM_SOFT_KNEE: 0.7,
|
||||
SUNRAYS: true,
|
||||
SUNRAYS_RESOLUTION: 196,
|
||||
SUNRAYS_WEIGHT: 1.0,
|
||||
}
|
||||
```
|
||||
|
||||
### GUI parameter table (from `script.js` lines 313–400)
|
||||
|
||||
#### Main panel (no folder)
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| Quality | `DYE_RESOLUTION` | Dropdown | — | — | — | 1024 | Ink/dye texture resolution. Options: High (1024), Medium (512), Low (256), Very Low (128). Lower = better performance, softer ink. |
|
||||
| Sim resolution | `SIM_RESOLUTION` | Dropdown | — | — | — | 128 | Velocity field simulation grid size. Options: 32 / 64 / 128 / 256. Lower = faster but chunkier fluid motion. |
|
||||
| Density diffusion | `DENSITY_DISSIPATION` | Slider | 0 | 4.0 | — | 1 | How quickly ink fades each frame. In the advection shader: `density *= uDissipation`. **Higher = faster fade = shorter trails.** 0 = ink persists forever; 4 = near-instant disappearance. |
|
||||
| Velocity diffusion | `VELOCITY_DISSIPATION` | Slider | 0 | 4.0 | — | 0.2 | How quickly fluid velocity dissipates. Higher = more viscous / fluid stops faster. |
|
||||
| Pressure | `PRESSURE` | Slider | 0.0 | 1.0 | — | 0.8 | Pressure solve strength. Higher values maintain more fluid pressure, producing tighter, more defined flow patterns. |
|
||||
| Vorticity | `CURL` | Slider | 0 | 50 | 1 | 30 | Curl/vorticity confinement. Higher = more swirling/turbulent flow. 0 = no swirl. |
|
||||
| Splat radius | `SPLAT_RADIUS` | Slider | 0.01 | 1.0 | — | 0.25 | Size of each ink injection point. Larger = wider splat footprint. |
|
||||
| Shading | `SHADING` | Toggle | — | — | — | true | Enables normal-map-based shading on the fluid surface, giving it a 3D appearance. |
|
||||
| Colorful | `COLORFUL` | Toggle | — | — | — | true | When on, continuously cycles ink colour via `COLOR_UPDATE_SPEED`. When off, new splats keep their initial colour indefinitely. |
|
||||
| Paused | `PAUSED` | Toggle | — | — | — | false | Freezes the simulation (stops advection). Splats can still be added while paused. |
|
||||
| Random splats | *(button)* | Button | — | — | — | — | Adds a random burst of coloured splats across the canvas. |
|
||||
|
||||
#### Bloom folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| Enabled | `BLOOM` | Toggle | — | — | — | true | Enables the bloom post-processing pass. |
|
||||
| Intensity | `BLOOM_INTENSITY` | Slider | 0.1 | 2.0 | — | 0.8 | Bloom glow strength. Higher = brighter glow around bright ink regions. |
|
||||
| Threshold | `BLOOM_THRESHOLD` | Slider | 0.0 | 1.0 | — | 0.6 | Brightness level above which bloom applies. Lower threshold = more of the ink blooms. |
|
||||
|
||||
> `BLOOM_SOFT_KNEE` (0.7) and `BLOOM_RESOLUTION` (256) and `BLOOM_ITERATIONS` (8) are in the
|
||||
> config object but **not exposed in the GUI**. They affect bloom quality and softness.
|
||||
|
||||
#### Sunrays folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| Enabled | `SUNRAYS` | Toggle | — | — | — | true | Enables the god-rays / sunrays post-processing pass through the fluid. |
|
||||
| Weight | `SUNRAYS_WEIGHT` | Slider | 0.3 | 1.0 | — | 1.0 | Intensity of the sunrays effect. |
|
||||
|
||||
> `SUNRAYS_RESOLUTION` (196) is in config but **not in the GUI**.
|
||||
|
||||
#### Capture folder
|
||||
|
||||
| Label | Key | Control | Min | Max | Step | Default | Effect |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| Background colour | `BACK_COLOR` | Color picker | — | — | — | #000000 | Background colour shown when `TRANSPARENT` is off. |
|
||||
| Transparent | `TRANSPARENT` | Toggle | — | — | — | false | Renders fluid over a transparent background (useful for capture/export). |
|
||||
| Take screenshot | *(button)* | Button | — | — | — | — | Downloads a PNG snapshot of the current frame. |
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Cross-reference: what is and isn't in StemEngine's `VizSettings`
|
||||
|
||||
### Muon — current StemEngine coverage vs. full GUI
|
||||
|
||||
| GUI parameter | In `VizSettings`? | Notes |
|
||||
|---|---|---|
|
||||
| `radiusMultiplier` | ✅ `muon.radius_multiplier` | As a multiplier on preset cycle range |
|
||||
| `aperture` | ✅ `muon.aperture` | As a multiplier on audio-computed value |
|
||||
| `spacing` | ✅ `muon.spacing` | As a multiplier |
|
||||
| `maxPoints` / `particle_count` | 🔲 Proposed (Controls Audit) | Direct int, not a multiplier |
|
||||
| `deltaResponseLimit` / `beat_sensitivity` | 🔲 Proposed (Controls Audit) | As divisor of 200 ms cooldown |
|
||||
| `idleMultiplier` | ❌ Not mapped | Could map directly; controls idle animation speed |
|
||||
| `noiseScale` | ❌ Not mapped | Controls spatial regularity of the spiral |
|
||||
| `noiseForce` | ❌ Not mapped | Controls velocity of noise-displaced particles |
|
||||
| `divisions` | ❌ Not mapped | Frequency bin resolution — affects reactivity detail |
|
||||
| `lifespan` | ❌ Not mapped | Particle turnover rate |
|
||||
| `colorSpectrum` | ❌ Not mapped | Colour band count (int ×3, range 3–30) |
|
||||
| `enableMonoColor` | ❌ Not mapped | Boolean — locks colour to `monoColor` |
|
||||
| `monoColor` | ❌ Not mapped | Fixed HSV colour when mono mode is on |
|
||||
| `particleMirror` | ❌ Not mapped | Boolean — mirrors geometry |
|
||||
| `dynamicRadius` | ❌ Not mapped | Boolean — enables audio-reactive radius cycling |
|
||||
| `visualizationPreset` | ❌ Not mapped | Boolean — enables preset cycling |
|
||||
| `bloomStrength` | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
|
||||
| `bloomThreshold` | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
|
||||
| `bloomRadius` | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
|
||||
| `damp` (afterimage) | ❌ Not mapped | Post-processing — not in GUI, hardcoded |
|
||||
| `glow_intensity` | 🔲 Proposed (Controls Audit) | Canvas port equivalent of bloom/glow |
|
||||
| `particle_size` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
|
||||
| `drum_reactivity` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
|
||||
| `hue_range` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
|
||||
| `ellipse_ratio` | 🔲 Proposed (Controls Audit) | Canvas port; maps to `aperture` at geometry level |
|
||||
| `rotation_speed` | 🔲 Proposed (Controls Audit) | Canvas port parameter |
|
||||
|
||||
### Fluid — current StemEngine coverage vs. full GUI
|
||||
|
||||
| GUI parameter | In `VizSettings`? | Notes |
|
||||
|---|---|---|
|
||||
| `spawn_density` | ✅ `fluid.spawn_density` | Multiplier on ambient spawn probability |
|
||||
| `splat_force` | ✅ `fluid.splat_force` | Multiplier on blob velocity/size |
|
||||
| `decay` | ✅ `fluid.decay` | Multiplier on per-frame lifetime drain |
|
||||
| `DENSITY_DISSIPATION` | ❌ Not mapped | Canvas sim uses `decay` instead; relevant for WebGL port |
|
||||
| `VELOCITY_DISSIPATION` | ❌ Not mapped | Canvas sim uses `drag`; relevant for WebGL port |
|
||||
| `PRESSURE` | ❌ Not mapped | WebGL-only; no canvas equivalent |
|
||||
| `CURL` | ❌ Not mapped | Vorticity; WebGL-only |
|
||||
| `SPLAT_RADIUS` | ❌ Not mapped | Canvas sim uses blob `radius` (implicit in force) |
|
||||
| `SPLAT_FORCE` | ❌ Not mapped | Equivalent to `splat_force` — already covered |
|
||||
| `SHADING` | ❌ Not mapped | WebGL-only post-processing |
|
||||
| `COLORFUL` | ❌ Not mapped | Colour cycling — always on in canvas sim |
|
||||
| `COLOR_UPDATE_SPEED` | ❌ Not mapped | Canvas sim cycles through `speakerPalette` sequentially |
|
||||
| `BLOOM` | ❌ Not mapped | WebGL-only |
|
||||
| `BLOOM_INTENSITY` | ❌ Not mapped | WebGL-only |
|
||||
| `BLOOM_THRESHOLD` | ❌ Not mapped | WebGL-only |
|
||||
| `SUNRAYS` | ❌ Not mapped | WebGL-only |
|
||||
| `SUNRAYS_WEIGHT` | ❌ Not mapped | WebGL-only |
|
||||
| `BACK_COLOR` | ❌ Not mapped | Canvas sim uses palette `backgroundColor` |
|
||||
| `background_persistence` | 🔲 Proposed (Controls Audit) | Canvas-specific: fill-alpha trail control |
|
||||
| `beat_burst_count` | 🔲 Proposed (Controls Audit) | Canvas-specific |
|
||||
| `treble_pulse` | 🔲 Proposed (Controls Audit) | Canvas-specific |
|
||||
| `drag` | 🔲 Proposed (Controls Audit) | Canvas equivalent of `VELOCITY_DISSIPATION` |
|
||||
| `blob_life` | 🔲 Proposed (Controls Audit) | Canvas-specific |
|
||||
|
||||
---
|
||||
|
||||
## Part 4 — Parameters worth adding to `VizSettings` (from original GUI, not yet proposed)
|
||||
|
||||
These are original GUI parameters that were not covered in `2026-04-08_Visualizer_Controls_Audit.md`
|
||||
and are relevant to StemEngine's canvas-based port.
|
||||
|
||||
### Muon additions
|
||||
|
||||
| Original key | Suggested `VizSettings` key | Type | Range | Default | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| `idleMultiplier` | `muon.idle_speed` | float | 0.0–2.0 | 1.0 | Multiplier on the current `0.27` default. Controls how fast the spiral animates when no audio is playing. |
|
||||
| `noiseScale` | `muon.noise_scale` | float | 0.0–3.0 | 1.0 | Multiplier on the current `0.3` default. Higher = more spatial irregularity in the spiral. |
|
||||
| `noiseForce` | `muon.noise_force` | float | 0.0–3.0 | 1.0 | Multiplier on `noiseForce`. Higher = noisier particle velocity. |
|
||||
| `colorSpectrum` | `muon.color_segments` | int | 3–30 | 3 | Direct value (steps of 3). Number of colour bands in spiral. More = richer colour banding. |
|
||||
| `enableMonoColor` | `muon.mono_color` | boolean | — | false | Locks colour to a fixed value rather than audio-driven hue cycling. |
|
||||
| `lifespan` | `muon.particle_lifespan` | float | 0.25–3.0 | 1.0 | Multiplier on `200 ms` default. Lower = faster particle turnover. |
|
||||
| `particleMirror` | `muon.mirror` | boolean | — | true | Mirrors the particle system across the spiral axis. |
|
||||
|
||||
### Fluid additions
|
||||
|
||||
| Original key | Suggested `VizSettings` key | Type | Notes |
|
||||
|---|---|---|---|
|
||||
| `CURL` | `fluid.curl` | float (0.25–3.0 ×) | Multiplier on `30` default. Higher = more swirling vortex motion in WebGL port. Not applicable to current canvas sim. |
|
||||
| `COLORFUL` / `COLOR_UPDATE_SPEED` | `fluid.color_speed` | float (0.0–3.0 ×) | Multiplier on palette cycling speed. 0 = colour locked after first spawn; 3 = rapid cycling. |
|
||||
|
||||
> Most other Fluid GUI parameters (`BLOOM_*`, `SUNRAYS_*`, `PRESSURE`, `SHADING`) are WebGL-only
|
||||
> and do not apply to the current canvas blob renderer in StemEngine. They become relevant only
|
||||
> if the full `PavelDoGreat/WebGL-Fluid-Simulation` JS source is vendored.
|
||||
@@ -2460,3 +2460,108 @@ Created a detailed integration plan for adding two optional visual backdrops to
|
||||
- Muon-Audio-Analysis is not a library — plan extracts its core dependency (Meyda.js) directly.
|
||||
|
||||
Implementation is 10 ordered steps. Phase 7 (per-song beat-grid pre-analysis via backend pipeline) is deferred.
|
||||
|
||||
### Visualizer parameter audit & fixes — `2026-04-08_Visualizer_Parameter_Audit.md`
|
||||
|
||||
Audited both visualizer sources against the integration plan. Found three bugs in the original muon-music-visualizer source and five incorrect assumptions in the integration plan.
|
||||
|
||||
**Muon source bugs (all must be fixed in the StemEngine port):**
|
||||
1. **`hueControl` always returns 0 during playback** (`effects.js`): `!scaler && time + scaler * 10` short-circuits to `false` (= 0) whenever scaler is non-zero, freezing hue at red. Fix: `scaler > 0 ? scaler * 10 : time`.
|
||||
2. **`baseFr`/`trebleFr` are swapped** (`audio.js`): lower half (bass) is assigned to `trebleFr` and upper half (treble) to `baseFr`, so kick drums drive the smaller 3.14× coefficient in `beatScaler` instead of the larger 6.24× one. Fix: swap the `modulate` inputs.
|
||||
3. **`coreScaler` amplification is dead code** (`audio.js` line 24): `coreScaler < 10 && coreScaler * 1.2` never assigns the result. Fix: `if (coreScaler < 10) coreScaler *= 1.2`.
|
||||
|
||||
**Integration plan corrections:**
|
||||
4. **`colorSpectrum` range is 3–30 (step 3), not 0–360** — it's a colour segment count, not a hue angle. Driving it with `treble * 120` would corrupt geometry. Hue is controlled by `hueControl()` instead.
|
||||
5. **`radiusMultiplier` should not be directly audio-mapped** — it's GSAP-animated by `wavePresetController` cycling through `visualizationPresets[]`. Tune `deltaResponseLimit` instead.
|
||||
6. **`DENSITY_DISSIPATION` mapping was inverted** — lower value = longer trails (counterintuitive). Corrected formula: `lerp(0.90, 0.98, 1 - bass)`.
|
||||
7. **`generateColor()` multiplies by 0.15**, producing near-invisible splats. Use `hsvToRgb` at full brightness for audio splats.
|
||||
8. **Direct `splat()` calls bypass aspect-ratio correction** — apply `dx / aspectRatio` before calling `splat()`.
|
||||
|
||||
Document includes the definitive `manifest.json` parameter lists for both visualizers (15 Muon keys, 13 Fluid keys) with types, ranges, defaults, and notes on what is/isn't stored.
|
||||
|
||||
### Performance sidebar plan — `2026-04-08_PerformanceSidebar_Plan.md`
|
||||
|
||||
Designed a slide-in sidebar accessible during both `KaraokePerformanceMode` (TV view) and fullscreen `LyricsLivePreview`. All relevant state lives inside `LyricsLivePreview` so no state lifting is needed.
|
||||
|
||||
**Trigger:** Persistent floating tab at the right edge of the screen, always visible when `isPerformanceView` is true. Uses a `SlidersHorizontal` icon (lucide-react). Click or `Tab` key toggles `showPerfSidebar`.
|
||||
|
||||
**New component:** `frontend/src/components/PerformanceSidebar.tsx` — absolute right-side drawer, 280 px wide, `backdrop-blur bg-black/70`, three collapsible sections.
|
||||
|
||||
**Section 1 — Background:** Icon grid for `Static / Muon / Fluid / Video` (Video only shown when `hasVideo` is true). Replaces the renderer buttons hidden by `performanceMode`.
|
||||
|
||||
**Section 2 — Stems:** Per-stem mute (M), solo (S), and gain slider (0–200%). Direct port of `renderStems()`. Leading vocal stems matching `/^(lead|lead\s|leading|vocals?|vox|voice|main\s*vocal)/i` are auto-muted on first use in performance mode (only when no saved `mutedStems` prefs exist for the `storageKey`).
|
||||
|
||||
**Section 3 — Lyrics:** Condensed `StylePicker` subset: font family/size/bold, align, vertical position, line spacing, text/highlight/bg colours, X/Y offsets.
|
||||
|
||||
**Viewport exclusion:** `showPerfSidebar` added to the `containerWidth` dep array; 280 px subtracted from canvas width calculation when sidebar is open so the visualizer doesn't render behind the panel.
|
||||
|
||||
**`KaraokePerformanceMode` changes:** None — sidebar lives entirely in `LyricsLivePreview`; no `Tab` key conflict (KPM only handles Esc/Space/Enter/Arrows).
|
||||
|
||||
9 implementation steps documented in the plan file.
|
||||
|
||||
### Visualizer controls audit — `2026-04-08_Visualizer_Controls_Audit.md`
|
||||
|
||||
Full audit of every expression that changes during playback in both visualizer canvases. Documented
|
||||
all hardcoded coefficients, proposed new manifest keys and sidebar controls, and defined named
|
||||
presets for each visualizer.
|
||||
|
||||
**What was hardcoded and is now proposed as a control:**
|
||||
|
||||
*Muon* (8 new keys on top of the 3 existing multipliers):
|
||||
- `particle_count` (int 40–500, default 220) — particle density and GPU cost
|
||||
- `rotation_speed` (×0.2–4.0) — multiplies `pt.speed` base and drum rotation bonus
|
||||
- `glow_intensity` (×0.0–3.0) — multiplies `shadowBlur = 14 + drum * 28`
|
||||
- `particle_size` (×0.25–3.0) — multiplies `size = 1.75 + drum * 3.0`
|
||||
- `drum_reactivity` (×0.0–2.0) — scales the `drum` signal before it feeds aperture/spacing/hue/alpha
|
||||
- `hue_range` (×0.0–3.0) — multiplies `drum * 20` and `treble * 15` hue drift coefficients
|
||||
- `beat_sensitivity` (×0.2–3.0) — divides the 200 ms beat-preset cooldown (higher = faster cycling)
|
||||
- `ellipse_ratio` (0.30–1.0, default 0.62) — direct override of y-axis squash; 1.0 = circle
|
||||
|
||||
*Fluid* (5 new keys on top of the 3 existing multipliers):
|
||||
- `background_persistence` (×0.2–3.0) — scales `(1 - bass) * 0.16` fill-alpha; the primary trail-length lever
|
||||
- `beat_burst_count` (×0.25–3.0) — multiplies `3 + floor(bass * 4)` blobs per beat
|
||||
- `treble_pulse` (×0.0–3.0) — multiplies the `treble * 0.25` blob swell factor
|
||||
- `drag` (×0.5–2.0) — scales velocity retention (higher = more viscous)
|
||||
- `blob_life` (×0.25–3.0) — multiplies initial blob life `0.7 + strength * 0.6`
|
||||
|
||||
**Named presets** defined in `PerformanceSidebar.tsx` (not stored in manifest.json):
|
||||
- Muon: Default, Calm, Energetic, Dreamy, Minimal, Cosmic
|
||||
- Fluid: Default, Ink, Foam, Lava, Mist, Storm
|
||||
|
||||
A "Custom" pseudo-preset shows when any slider deviates from the active preset. Presets populate all sliders for their respective visualizer via `patchVizSettings`.
|
||||
|
||||
Sidebar layout updated to show preset pills above sliders. Backend normaliser does not need changes for sidebar-only live adjustment (new keys are optional and default to 1.0).
|
||||
|
||||
### Visualizer GUI parameter map — `2026-04-09_Visualizer_GUI_Parameter_Map.md`
|
||||
|
||||
Exhaustive map of every user-adjustable parameter in both original visualizer GUIs, sourced directly from their GitHub repositories and live sites.
|
||||
|
||||
**Muon** (muon-vis.netlify.app / najafmohammed/muon-music-visualizer):
|
||||
- 5 Particle Controls folder params: `spacing` (0–1), `maxPoints` (0–12240 step 360), `colorSpectrum` (3–30 step 3, colour band count not hue angle), `aperture` (0–π), `particleMirror` (bool)
|
||||
- 3 Controls folder params: `dynamicRadius` (bool), `visualizationPreset` (bool), `idleMultiplier` (0–2, idle animation speed)
|
||||
- 1 Audio Controls param: `deltaResponseLimit` (0.001–0.01, beat sensitivity threshold)
|
||||
- 4 Emission Controls params: `enableMonoColor` (bool), `divisions` (1–150, frequency bin count), `lifespan` (10–250 ms), `monoColor` (HSV color picker)
|
||||
- 2 Noise Controls params: `noiseScale` (0–1), `noiseForce` (0–1)
|
||||
- Root: `radiusMultiplier` (0.01–1, default 0.66), `sideView` button, `reset` button
|
||||
- Post-processing (hardcoded, not in GUI): bloom strength 0.55 / threshold 0.35 / radius 0.6; afterimage damp 0.86
|
||||
|
||||
**Fluid** (paveldogreat.github.io/WebGL-Fluid-Simulation):
|
||||
- Main panel: `DYE_RESOLUTION` (quality dropdown), `SIM_RESOLUTION` (32–256 dropdown), `DENSITY_DISSIPATION` (0–4, higher = faster fade), `VELOCITY_DISSIPATION` (0–4), `PRESSURE` (0–1), `CURL` (0–50 vorticity), `SPLAT_RADIUS` (0.01–1), `SHADING` (bool), `COLORFUL` (bool), `PAUSED` (bool), random splats button
|
||||
- Bloom folder: `BLOOM` (bool), `BLOOM_INTENSITY` (0.1–2.0), `BLOOM_THRESHOLD` (0–1)
|
||||
- Sunrays folder: `SUNRAYS` (bool), `SUNRAYS_WEIGHT` (0.3–1.0)
|
||||
- Capture folder: `BACK_COLOR` (color picker), `TRANSPARENT` (bool), screenshot button
|
||||
|
||||
Cross-reference table shows which GUI params are already in `VizSettings`, which were proposed in the Controls Audit, and which are new (7 additional Muon keys + 2 Fluid keys proposed for `VizSettings`). Most Fluid WebGL params (bloom, sunrays, pressure, shading) are not applicable to the current canvas blob renderer.
|
||||
|
||||
### Performance sidebar bug fixes — documented in `2026-04-08_PerformanceSidebar_Plan.md`
|
||||
|
||||
Three bugs identified in the existing implementation and their fixes documented.
|
||||
|
||||
**Bug 1a — Visualizer mode buttons (Fluid, Muon, Video) don't switch (`LyricsLivePreview.tsx`):**
|
||||
The `enforcedVisualizerMode` effect includes `visualStyle` in its dep array. Every time the sidebar calls `setVisualStyle('fluid')`, the effect fires, sees `visualStyle !== enforcedVisualizerMode`, and immediately calls `setVisualStyle(enforcedVisualizerMode)` to revert. Fix: add `userOverrodeVisualizerRef` (a ref, not state); wrap `setVisualStyle` in `setVisualStyleFromSidebar` which sets the ref; update the effect to bail when the ref is set; remove `visualStyle` from the dep array so the effect only runs on mount and song change.
|
||||
|
||||
**Bug 1b — Video mode doesn't start playing when switched mid-song (`LyricsLivePreview.tsx`):**
|
||||
`<video>` only mounts when `hasVideoBackdrop` is true. When `visualStyle` switches to `'video'` while `isPlaying`, the video element mounts fresh but paused — `togglePlay` is never called, so `videoRef.current.play()` and `setVideoStartPhase('playing')` don't fire. Fix: add `useEffect` on `[hasVideoBackdrop, isPlaying]` that, when both are true and `videoStartPhase !== 'playing'`, seeks the video to `currentTimeRef.current`, calls `play()`, and sets the phase.
|
||||
|
||||
**Bug 2 — Seek slider snaps to 0 after dragging (`LyricsLivePreview.tsx`):**
|
||||
`endSeekDrag` sets `isSeekingRef.current = false` synchronously after calling `applySeekPosition`. The RAF loop fires before the browser's audio `seeked` event, reads `audio.currentTime` (still returning the pre-seek value because browser seek is async), calls `setCurrentTime(0)`, then the `setTimeout(0)` clears `seekDragTime` — leaving the slider at 0. Fix: defer `isSeekingRef.current = false` until the audio element fires `seeked` (with a 500 ms safety fallback via `seekLockFallbackRef`). `beginSeekDrag` cancels any pending fallback timer from a prior drag.
|
||||
File renamed without changes.
File renamed without changes.
+28
-10
@@ -236,24 +236,39 @@ export default function MuonVisualizerCanvas({ featuresRef, width, height, ...pa
|
||||
|
||||
### Drum-intensity → Controls mapping
|
||||
|
||||
> **⚠️ Corrections applied — see 2026-04-08_Visualizer_Parameter_Audit.md Corrections A, B.**
|
||||
|
||||
```ts
|
||||
// Each RAF frame inside the Three.js loop:
|
||||
const { drumIntensity = 0, treble = 0, beatDetected = false } = featuresRef.current
|
||||
const palette = paletteRef.current
|
||||
const baseHue = palette.seedHue
|
||||
|
||||
// "Controls" params driven by drum track:
|
||||
params.radiusMultiplier = lerp(0.4, 1.8, easeIn(drumIntensity))
|
||||
params.aperture = lerp(1.2, 3.5, drumIntensity)
|
||||
params.spacing = lerp(0.008, 0.02, 1 - drumIntensity) // tighter on loud beats
|
||||
// Correction B: radiusMultiplier is NOT mapped directly from drumIntensity each frame.
|
||||
// It advances through RADIUS_PRESETS on each beat and lerps toward the new target.
|
||||
// Direct per-frame lerp(0.4, 1.8, ...) fought the GSAP preset animation and exceeded
|
||||
// the preset range (0.6603–0.7536 in the original source).
|
||||
if (beatDetected) {
|
||||
presetIdx = (presetIdx + 1) % RADIUS_PRESETS.length
|
||||
radiusTarget = RADIUS_PRESETS[presetIdx] + drumIntensity * 0.25
|
||||
}
|
||||
radiusCurrent += (radiusTarget - radiusCurrent) * 0.07 // lerp toward target each frame
|
||||
|
||||
params.aperture = lerp(1.2, 3.5, drumIntensity)
|
||||
params.spacing = lerp(0.008, 0.02, 1 - drumIntensity) // tighter on loud beats
|
||||
|
||||
// Correction A: colorSpectrum is a colour-band count (3–30, step 3), NOT a hue angle.
|
||||
// Hue drift is controlled separately via hueControl() / seedHue offset.
|
||||
// Bug 1 fix: when drumIntensity > 0 (audio playing), hue is modulated by drum + treble;
|
||||
// when silent it falls back to slow time-based drift so the visual never freezes.
|
||||
const hue = drumIntensity > 0.01
|
||||
? ((palette.seedHue + drumIntensity * 20 + treble * 15) + 360) % 360 // ±15° max treble drift
|
||||
: ((palette.seedHue + Math.sin(Date.now() * 0.00018) * 15) + 360) % 360
|
||||
|
||||
// Color stays in manifest palette family; treble adds controlled drift:
|
||||
params.colorSpectrum = (baseHue + treble * 40) % 360
|
||||
params.primaryColor = palette.highlightColor
|
||||
params.secondaryColor = palette.wordHighlightColor
|
||||
|
||||
// Beat → preset morph:
|
||||
if (beatDetected) triggerPresetTransition()
|
||||
// Beat → preset morph (already handled above via radiusCurrent/radiusTarget)
|
||||
```
|
||||
|
||||
`lerp` and `easeIn` are trivial helpers; no additional deps needed.
|
||||
@@ -349,8 +364,11 @@ if (beatDetected) {
|
||||
}
|
||||
|
||||
// Config modulation:
|
||||
fluidConfig.DENSITY_DISSIPATION = lerp(0.97, 0.999, 1 - bass) // longer trails on loud bass
|
||||
fluidConfig.VELOCITY_DISSIPATION = lerp(0.94, 0.99, 1 - rms)
|
||||
// Correction C: mapping is driven by bass (not rms) so kick hits produce longer trails.
|
||||
// ⚠️ Direction caveat: if the vendored sim uses density *= uDissipation (retention
|
||||
// multiplier), LOWER values = faster fading. Verify advection shader before porting.
|
||||
fluidConfig.DENSITY_DISSIPATION = lerp(0.90, 0.98, 1 - bass) // loud bass → 0.90 → longer trails
|
||||
fluidConfig.VELOCITY_DISSIPATION = lerp(0.85, 0.99, 1 - rms) // loud rms → more momentum
|
||||
fluidConfig.COLOR_UPDATE_SPEED = lerp(2, 15, treble)
|
||||
```
|
||||
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -22,6 +22,20 @@ interface KaraokePerformanceModeProps {
|
||||
thumbnailUrl?: string
|
||||
sekDynamicPalette?: SekDynamicPalette | null
|
||||
sekVisualizerMode?: 'muon' | 'fluid' | 'auto' | null
|
||||
sekVisualizerSettings?: {
|
||||
quality?: string
|
||||
fps_cap?: number
|
||||
muon?: {
|
||||
radius_multiplier?: number
|
||||
aperture?: number
|
||||
spacing?: number
|
||||
}
|
||||
fluid?: {
|
||||
spawn_density?: number
|
||||
splat_force?: number
|
||||
decay?: number
|
||||
}
|
||||
} | null
|
||||
artist?: string
|
||||
songTitle?: string
|
||||
upNextSong?: { title: string; artist?: string } | null
|
||||
@@ -51,6 +65,7 @@ export default function KaraokePerformanceMode({
|
||||
thumbnailUrl,
|
||||
sekDynamicPalette,
|
||||
sekVisualizerMode,
|
||||
sekVisualizerSettings,
|
||||
artist,
|
||||
songTitle,
|
||||
disableGlobalPlaybackRegistry = false,
|
||||
@@ -120,6 +135,7 @@ export default function KaraokePerformanceMode({
|
||||
thumbnailUrl={thumbnailUrl}
|
||||
sekDynamicPalette={sekDynamicPalette}
|
||||
sekVisualizerMode={sekVisualizerMode}
|
||||
sekVisualizerSettings={sekVisualizerSettings}
|
||||
artist={artist}
|
||||
songTitle={songTitle}
|
||||
performanceMode
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
import { useState, useEffect, useRef, useCallback, useMemo } from 'react'
|
||||
import Two from 'two.js'
|
||||
import { VIDEO_ANIM_REGISTRY, VIDEO_TRANSITION_REGISTRY, type VideoAnimStyle, type VideoTimingGranularity, type VideoLineTransition } from '../utils/videoAnimations'
|
||||
import { Play, Pause, SkipBack, Volume2, VolumeX, Palette, Pencil, X, AlignLeft, AlignCenter, AlignRight, Type, Maximize2, Minimize2, Activity } from 'lucide-react'
|
||||
import { Play, Pause, SkipBack, Volume2, VolumeX, Palette, Pencil, X, AlignLeft, AlignCenter, AlignRight, Type, Maximize2, Minimize2, Activity, SlidersHorizontal } from 'lucide-react'
|
||||
import type { LyricsStyleOptions } from '../types'
|
||||
import { registerPlayback, unregisterPlayback, publishMeta, clearMeta } from '../utils/audioRegistry'
|
||||
|
||||
@@ -13,6 +13,7 @@ import { useVisualizerPalette } from '../hooks/useVisualizerPalette'
|
||||
import { useAudioFeatures } from '../hooks/useAudioFeatures'
|
||||
import type { SekDynamicPalette } from '../api/client'
|
||||
import VisualizerBackdrop from './visualizers/VisualizerBackdrop'
|
||||
import PerformanceSidebar from './PerformanceSidebar.tsx'
|
||||
|
||||
// ─── Types ───────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -47,6 +48,83 @@ interface PreviewPlaybackPrefs {
|
||||
export type LyricsVisualStyle = 'scroll' | 'fade' | 'karaoke' | 'glow' | 'blur' | 'video' | 'muon' | 'fluid'
|
||||
type VideoStartPhase = 'idle' | 'priming' | 'playing'
|
||||
|
||||
type DeepPartial<T> = {
|
||||
[K in keyof T]?: T[K] extends object ? DeepPartial<T[K]> : T[K]
|
||||
}
|
||||
|
||||
interface VizSettings {
|
||||
quality: 'low' | 'balanced' | 'high'
|
||||
fps_cap: 30 | 60 | 120
|
||||
muon: {
|
||||
radius_multiplier: number
|
||||
aperture: number
|
||||
spacing: number
|
||||
}
|
||||
fluid: {
|
||||
spawn_density: number
|
||||
splat_force: number
|
||||
decay: number
|
||||
}
|
||||
}
|
||||
|
||||
const VIZ_DEFAULTS: VizSettings = {
|
||||
quality: 'balanced',
|
||||
fps_cap: 60,
|
||||
muon: { radius_multiplier: 1.0, aperture: 1.0, spacing: 1.0 },
|
||||
fluid: { spawn_density: 1.0, splat_force: 1.0, decay: 1.0 },
|
||||
}
|
||||
|
||||
const PERF_SIDEBAR_WIDTH = 280
|
||||
const LEADING_VOCAL_PATTERN = /^(lead|lead\s|leading|vocals?|vox|voice|main\s*vocal)/i
|
||||
|
||||
function isLeadingVocalStem(name: string): boolean {
|
||||
return LEADING_VOCAL_PATTERN.test((name || '').trim())
|
||||
}
|
||||
|
||||
function clampVizMultiplier(value: number): number {
|
||||
if (!Number.isFinite(value)) return 1
|
||||
return Math.max(0.25, Math.min(2, value))
|
||||
}
|
||||
|
||||
function deepMergeVizSettings(base: VizSettings, patch?: DeepPartial<VizSettings> | null): VizSettings {
|
||||
if (!patch) {
|
||||
return {
|
||||
...base,
|
||||
muon: { ...base.muon },
|
||||
fluid: { ...base.fluid },
|
||||
}
|
||||
}
|
||||
return {
|
||||
...base,
|
||||
...patch,
|
||||
muon: {
|
||||
...base.muon,
|
||||
...(patch.muon ?? {}),
|
||||
},
|
||||
fluid: {
|
||||
...base.fluid,
|
||||
...(patch.fluid ?? {}),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeVizSettings(input: VizSettings): VizSettings {
|
||||
return {
|
||||
quality: input.quality === 'low' || input.quality === 'high' ? input.quality : 'balanced',
|
||||
fps_cap: input.fps_cap === 30 || input.fps_cap === 120 ? input.fps_cap : 60,
|
||||
muon: {
|
||||
radius_multiplier: clampVizMultiplier(input.muon.radius_multiplier),
|
||||
aperture: clampVizMultiplier(input.muon.aperture),
|
||||
spacing: clampVizMultiplier(input.muon.spacing),
|
||||
},
|
||||
fluid: {
|
||||
spawn_density: clampVizMultiplier(input.fluid.spawn_density),
|
||||
splat_force: clampVizMultiplier(input.fluid.splat_force),
|
||||
decay: clampVizMultiplier(input.fluid.decay),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
interface Props {
|
||||
lrcText: string
|
||||
lyricsTracks?: Record<string, string>
|
||||
@@ -69,6 +147,21 @@ interface Props {
|
||||
sekDynamicPalette?: SekDynamicPalette | null
|
||||
/** Optional SEK visualizer mode preference (manifest.visualizer.mode). */
|
||||
sekVisualizerMode?: 'muon' | 'fluid' | 'auto' | null
|
||||
/** Optional SEK visualizer settings (manifest.visualizer.settings). */
|
||||
sekVisualizerSettings?: {
|
||||
quality?: string
|
||||
fps_cap?: number
|
||||
muon?: {
|
||||
radius_multiplier?: number
|
||||
aperture?: number
|
||||
spacing?: number
|
||||
}
|
||||
fluid?: {
|
||||
spawn_density?: number
|
||||
splat_force?: number
|
||||
decay?: number
|
||||
}
|
||||
} | null
|
||||
/** Artist name from source audio metadata. */
|
||||
artist?: string
|
||||
/** Track title from source audio metadata — used in the visual overlay only. */
|
||||
@@ -1336,7 +1429,7 @@ function LrcEditor({
|
||||
|
||||
// ─── Main component ───────────────────────────────────────────────────────────
|
||||
|
||||
export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, videoUrl, videoPlaylist, videoPlaybackMode, stems, title, styleOptions, defaultStyle, onLrcChange, storageKey, coverArtUrl, thumbnailUrl, sekDynamicPalette, sekVisualizerMode, artist, songTitle, performanceMode = false, autoPlay = false, disableGlobalPlaybackRegistry = false, onPlaybackEnded }: Props) {
|
||||
export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, videoUrl, videoPlaylist, videoPlaybackMode, stems, title, styleOptions, defaultStyle, onLrcChange, storageKey, coverArtUrl, thumbnailUrl, sekDynamicPalette, sekVisualizerMode, sekVisualizerSettings, artist, songTitle, performanceMode = false, autoPlay = false, disableGlobalPlaybackRegistry = false, onPlaybackEnded }: Props) {
|
||||
const resolvedLyricsTracks = useMemo(() => {
|
||||
const normalized: Record<string, string> = {}
|
||||
for (const [rawKey, rawText] of Object.entries(lyricsTracks || {})) {
|
||||
@@ -1418,12 +1511,47 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
(sekVisualizerMode === 'muon' || sekVisualizerMode === 'fluid')
|
||||
? sekVisualizerMode
|
||||
: ((defaultStyle === 'muon' || defaultStyle === 'fluid') ? defaultStyle : null)
|
||||
const userOverrodeVisualizerRef = useRef(false)
|
||||
const [showPerfSidebar, setShowPerfSidebar] = useState(false)
|
||||
const manifestVizSettings = useMemo<VizSettings | null>(() => {
|
||||
if (!sekVisualizerSettings) return null
|
||||
const quality = sekVisualizerSettings.quality
|
||||
const fps = sekVisualizerSettings.fps_cap
|
||||
const patch: DeepPartial<VizSettings> = {
|
||||
quality: quality === 'low' || quality === 'balanced' || quality === 'high' ? quality : undefined,
|
||||
fps_cap: fps === 30 || fps === 60 || fps === 120 ? fps : undefined,
|
||||
muon: {
|
||||
radius_multiplier: sekVisualizerSettings.muon?.radius_multiplier,
|
||||
aperture: sekVisualizerSettings.muon?.aperture,
|
||||
spacing: sekVisualizerSettings.muon?.spacing,
|
||||
},
|
||||
fluid: {
|
||||
spawn_density: sekVisualizerSettings.fluid?.spawn_density,
|
||||
splat_force: sekVisualizerSettings.fluid?.splat_force,
|
||||
decay: sekVisualizerSettings.fluid?.decay,
|
||||
},
|
||||
}
|
||||
return normalizeVizSettings(deepMergeVizSettings(VIZ_DEFAULTS, patch))
|
||||
}, [sekVisualizerSettings])
|
||||
const [liveVizSettings, setLiveVizSettings] = useState<VizSettings>(() => (
|
||||
manifestVizSettings ? deepMergeVizSettings(manifestVizSettings) : deepMergeVizSettings(VIZ_DEFAULTS)
|
||||
))
|
||||
const patchVizSettings = useCallback((patch: DeepPartial<VizSettings>) => {
|
||||
setLiveVizSettings((prev) => normalizeVizSettings(deepMergeVizSettings(prev, patch)))
|
||||
}, [])
|
||||
useEffect(() => {
|
||||
setLiveVizSettings(manifestVizSettings ? deepMergeVizSettings(manifestVizSettings) : deepMergeVizSettings(VIZ_DEFAULTS))
|
||||
}, [previewPrefsKey, manifestVizSettings])
|
||||
|
||||
useEffect(() => {
|
||||
if (!enforcedVisualizerMode) return
|
||||
if (visualStyle === enforcedVisualizerMode) return
|
||||
setVisualStyle(enforcedVisualizerMode)
|
||||
}, [enforcedVisualizerMode, visualStyle])
|
||||
if (userOverrodeVisualizerRef.current) return
|
||||
setVisualStyle((prev) => (prev === enforcedVisualizerMode ? prev : enforcedVisualizerMode))
|
||||
}, [enforcedVisualizerMode])
|
||||
|
||||
useEffect(() => {
|
||||
userOverrodeVisualizerRef.current = false
|
||||
}, [previewPrefsKey, enforcedVisualizerMode])
|
||||
|
||||
const [currentTime, setCurrentTime] = useState(0)
|
||||
const [seekDragTime, setSeekDragTime] = useState<number | null>(null)
|
||||
@@ -1506,6 +1634,12 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
desiredVideoSegmentRef.current = resolvedVideoPlaylist.length > 0 ? { index: 0, localTime: 0 } : null
|
||||
}, [resolvedVideoPlaylist])
|
||||
|
||||
useEffect(() => {
|
||||
if (visualStyle !== 'video') return
|
||||
if (videoUrl || resolvedVideoPlaylist.length > 0) return
|
||||
setVisualStyle('karaoke')
|
||||
}, [visualStyle, videoUrl, resolvedVideoPlaylist])
|
||||
|
||||
// Track whether cover art image loaded successfully
|
||||
const [artLoaded, setArtLoaded] = useState(false)
|
||||
|
||||
@@ -1597,6 +1731,8 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
const [mutedStems, setMutedStems] = useState<Record<string, boolean>>(() => initialPreviewPrefs.mutedStems ?? {})
|
||||
const [soloedStems, setSoloedStems] = useState<Record<string, boolean>>(() => initialPreviewPrefs.soloedStems ?? {})
|
||||
const previewPrefsHydratingRef = useRef(true)
|
||||
const perfLeadingMuteAppliedRef = useRef(false)
|
||||
const hasSavedMutedStemPrefs = Object.keys(initialPreviewPrefs.mutedStems ?? {}).length > 0
|
||||
|
||||
// User-overrideable speaker colours (keyed by speaker ID e.g. "SPEAKER_00")
|
||||
const [speakerColors, setSpeakerColors] = useState<Record<string, string>>(() => {
|
||||
@@ -1642,12 +1778,19 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
const gainRefs = useRef<Record<string, GainNode>>({})
|
||||
const metadataListenerRefs = useRef<Record<string, () => void>>({})
|
||||
|
||||
// Important: on first mount in multi-stem mode, `useAudioFeatures` can run
|
||||
// before the stem graph is created (`gainRefs.current` still empty). In that
|
||||
// case the hook exits early and would not reconnect until one of its deps
|
||||
// changes, causing static/non-reactive visualizers. Gate it until tap nodes
|
||||
// exist so the first enabled run always has a valid audio source.
|
||||
const audioFeatureTapReady = !isMultiStem || Object.keys(gainRefs.current).length > 0
|
||||
|
||||
const featuresRef = useAudioFeatures({
|
||||
audioCtxRef,
|
||||
stemGainRefs: gainRefs,
|
||||
singleAudioRef: audioRef,
|
||||
isPlayingRef,
|
||||
enabled: isVisualizerMode,
|
||||
enabled: isVisualizerMode && audioFeatureTapReady,
|
||||
})
|
||||
|
||||
const containerRef = useRef<HTMLDivElement>(null)
|
||||
@@ -1657,6 +1800,7 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
const isSeekingRef = useRef(false)
|
||||
const pendingSeekRef = useRef<number | null>(null)
|
||||
const clearSeekUiTimerRef = useRef<number | null>(null)
|
||||
const seekLockFallbackRef = useRef<number | null>(null)
|
||||
|
||||
// Global floating bar tracking
|
||||
const isActiveRef = useRef(false)
|
||||
@@ -1672,6 +1816,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
isActiveRef.current = false
|
||||
isSeekingRef.current = false
|
||||
pendingSeekRef.current = null
|
||||
if (seekLockFallbackRef.current !== null) {
|
||||
window.clearTimeout(seekLockFallbackRef.current)
|
||||
seekLockFallbackRef.current = null
|
||||
}
|
||||
if (clearSeekUiTimerRef.current !== null) {
|
||||
window.clearTimeout(clearSeekUiTimerRef.current)
|
||||
clearSeekUiTimerRef.current = null
|
||||
@@ -1694,6 +1842,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
isActiveRef.current = false
|
||||
isSeekingRef.current = false
|
||||
pendingSeekRef.current = null
|
||||
if (seekLockFallbackRef.current !== null) {
|
||||
window.clearTimeout(seekLockFallbackRef.current)
|
||||
seekLockFallbackRef.current = null
|
||||
}
|
||||
if (clearSeekUiTimerRef.current !== null) {
|
||||
window.clearTimeout(clearSeekUiTimerRef.current)
|
||||
clearSeekUiTimerRef.current = null
|
||||
@@ -1910,6 +2062,22 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
})
|
||||
}, [volume, isMuted, mutedStems, soloedStems, gains])
|
||||
|
||||
useEffect(() => {
|
||||
if (!performanceMode || !isMultiStem || !stems?.length) return
|
||||
if (hasSavedMutedStemPrefs || perfLeadingMuteAppliedRef.current) return
|
||||
|
||||
const defaultMutes: Record<string, boolean> = {}
|
||||
for (const stem of stems) {
|
||||
if (isLeadingVocalStem(stem.name)) defaultMutes[stem.name] = true
|
||||
}
|
||||
|
||||
perfLeadingMuteAppliedRef.current = true
|
||||
if (!Object.keys(defaultMutes).length) return
|
||||
|
||||
setMutedStems(defaultMutes)
|
||||
applyGains({ mutedStems: defaultMutes })
|
||||
}, [performanceMode, isMultiStem, stems, hasSavedMutedStemPrefs, applyGains])
|
||||
|
||||
// ── Initialize stem audio elements ──────────────────────────────────────
|
||||
useEffect(() => {
|
||||
if (!isMultiStem || !stems) {
|
||||
@@ -2175,6 +2343,25 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
const [lyricsViewportScale, setLyricsViewportScale] = useState(1)
|
||||
const [visualizerSize, setVisualizerSize] = useState({ width: 960, height: 540 })
|
||||
|
||||
useEffect(() => {
|
||||
if (!isPerformanceView) {
|
||||
setShowPerfSidebar(false)
|
||||
return
|
||||
}
|
||||
const onKeyDown = (event: KeyboardEvent) => {
|
||||
const target = event.target as HTMLElement | null
|
||||
if (target && (target.tagName === 'INPUT' || target.tagName === 'TEXTAREA' || target.tagName === 'SELECT' || target.isContentEditable)) {
|
||||
return
|
||||
}
|
||||
if ((event.key === 'q' || event.key === 'Q') && !event.metaKey && !event.ctrlKey && !event.altKey) {
|
||||
event.preventDefault()
|
||||
setShowPerfSidebar((prev) => !prev)
|
||||
}
|
||||
}
|
||||
window.addEventListener('keydown', onKeyDown)
|
||||
return () => window.removeEventListener('keydown', onKeyDown)
|
||||
}, [isPerformanceView])
|
||||
|
||||
// Scale lyric font relative to the display canvas size (VLC/Jellyfin style behavior)
|
||||
// so text grows/shrinks with the window instead of staying fixed.
|
||||
const effectiveViewportScale = isPerformanceView
|
||||
@@ -2214,7 +2401,8 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
|
||||
const compute = () => {
|
||||
const rect = node.getBoundingClientRect()
|
||||
const width = Math.max(1, rect.width)
|
||||
const sidebarOffset = isPerformanceView && showPerfSidebar && !isNarrow ? PERF_SIDEBAR_WIDTH : 0
|
||||
const width = Math.max(1, rect.width - sidebarOffset)
|
||||
const height = Math.max(1, rect.height)
|
||||
// 960x540 reference viewport (16:9)
|
||||
const scale = Math.min(width / 960, height / 540)
|
||||
@@ -2232,7 +2420,7 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
|
||||
window.addEventListener('resize', compute)
|
||||
return () => window.removeEventListener('resize', compute)
|
||||
}, [isFullscreen, performanceMode, showStylePicker, showEditor, showAegisub])
|
||||
}, [isFullscreen, performanceMode, showStylePicker, showEditor, showAegisub, showPerfSidebar, isNarrow, isPerformanceView])
|
||||
|
||||
function toggleFullscreen() {
|
||||
if (!isFullscreen) fullscreenRef.current?.requestFullscreen()
|
||||
@@ -2504,6 +2692,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
}
|
||||
|
||||
function beginSeekDrag() {
|
||||
if (seekLockFallbackRef.current !== null) {
|
||||
window.clearTimeout(seekLockFallbackRef.current)
|
||||
seekLockFallbackRef.current = null
|
||||
}
|
||||
if (clearSeekUiTimerRef.current !== null) {
|
||||
window.clearTimeout(clearSeekUiTimerRef.current)
|
||||
clearSeekUiTimerRef.current = null
|
||||
@@ -2528,7 +2720,29 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
applySeekPosition(finalValue)
|
||||
}
|
||||
}
|
||||
isSeekingRef.current = false
|
||||
|
||||
const releaseSeekLock = () => {
|
||||
if (seekLockFallbackRef.current !== null) {
|
||||
window.clearTimeout(seekLockFallbackRef.current)
|
||||
seekLockFallbackRef.current = null
|
||||
}
|
||||
isSeekingRef.current = false
|
||||
}
|
||||
|
||||
const targetAudio = isMultiStem
|
||||
? (Object.values(stemAudioRefs.current)[0] ?? null)
|
||||
: audioRef.current
|
||||
|
||||
if (targetAudio) {
|
||||
targetAudio.addEventListener('seeked', releaseSeekLock, { once: true })
|
||||
if (seekLockFallbackRef.current !== null) {
|
||||
window.clearTimeout(seekLockFallbackRef.current)
|
||||
}
|
||||
seekLockFallbackRef.current = window.setTimeout(releaseSeekLock, 500)
|
||||
} else {
|
||||
releaseSeekLock()
|
||||
}
|
||||
|
||||
if (clearSeekUiTimerRef.current !== null) {
|
||||
window.clearTimeout(clearSeekUiTimerRef.current)
|
||||
}
|
||||
@@ -2541,6 +2755,10 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
function handleReset() {
|
||||
isSeekingRef.current = false
|
||||
pendingSeekRef.current = null
|
||||
if (seekLockFallbackRef.current !== null) {
|
||||
window.clearTimeout(seekLockFallbackRef.current)
|
||||
seekLockFallbackRef.current = null
|
||||
}
|
||||
if (clearSeekUiTimerRef.current !== null) {
|
||||
window.clearTimeout(clearSeekUiTimerRef.current)
|
||||
clearSeekUiTimerRef.current = null
|
||||
@@ -2671,6 +2889,23 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
else v.pause()
|
||||
}, [isPlaying, hasVideoBackdrop, isPlaylistBackdropMode, currentVideoUrl])
|
||||
|
||||
// When switching to Video mode while already playing, ensure the freshly-mounted
|
||||
// video element is started and synchronized to current transport immediately.
|
||||
useEffect(() => {
|
||||
if (!hasVideoBackdrop || !isPlaying) return
|
||||
const v = videoRef.current
|
||||
if (!v) return
|
||||
if (videoStartPhase === 'playing') return
|
||||
const t = currentTimeRef.current ?? 0
|
||||
if (isPlaylistBackdropMode) {
|
||||
syncPlaylistVideoToTransport(t)
|
||||
} else if (Math.abs(v.currentTime - t) > 0.35) {
|
||||
v.currentTime = t
|
||||
}
|
||||
v.play().catch((err) => console.warn('[LyricsLivePreview] video auto-start on mode switch failed', err))
|
||||
setVideoStartPhase('playing')
|
||||
}, [hasVideoBackdrop, isPlaying, isPlaylistBackdropMode, videoStartPhase, currentVideoUrl])
|
||||
|
||||
function handleVolumeChange(e: React.ChangeEvent<HTMLInputElement>) {
|
||||
const v = parseFloat(e.target.value)
|
||||
setVolume(v)
|
||||
@@ -2805,6 +3040,7 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
paletteRef={paletteRef}
|
||||
width={visualizerSize.width}
|
||||
height={visualizerSize.height}
|
||||
settings={liveVizSettings}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
@@ -3397,6 +3633,43 @@ export default function LyricsLivePreview({ lrcText, lyricsTracks, audioUrl, vid
|
||||
</div>
|
||||
)}
|
||||
|
||||
{isPerformanceView && isNarrow && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowPerfSidebar((prev) => !prev)}
|
||||
className={`absolute right-3 top-3 z-[10] flex h-11 w-11 items-center justify-center rounded-full border border-white/15 text-slate-100 shadow-[0_10px_28px_rgba(0,0,0,0.35)] backdrop-blur-md transition-colors ${showPerfSidebar ? 'bg-violet-600/80 hover:bg-violet-600' : 'bg-black/65 hover:bg-black/80'}`}
|
||||
title={showPerfSidebar ? 'Close performance controls' : 'Open performance controls'}
|
||||
aria-label={showPerfSidebar ? 'Close performance controls' : 'Open performance controls'}
|
||||
>
|
||||
<SlidersHorizontal className="h-4.5 w-4.5" />
|
||||
</button>
|
||||
)}
|
||||
|
||||
{isPerformanceView && showPerfSidebar && (
|
||||
<PerformanceSidebar
|
||||
isNarrow={isNarrow}
|
||||
visualStyle={visualStyle}
|
||||
setVisualStyle={(next) => {
|
||||
userOverrodeVisualizerRef.current = true
|
||||
setVisualStyle(next)
|
||||
}}
|
||||
hasVideo={!!(videoUrl || resolvedVideoPlaylist.length > 0)}
|
||||
stems={stems}
|
||||
mutedStems={mutedStems}
|
||||
soloedStems={soloedStems}
|
||||
gains={gains}
|
||||
toggleStemMute={toggleStemMute}
|
||||
toggleStemSolo={toggleStemSolo}
|
||||
setStemGain={updateStemGain}
|
||||
style={style}
|
||||
patchStyle={patchStyle}
|
||||
vizSettings={liveVizSettings}
|
||||
manifestVizSettings={manifestVizSettings}
|
||||
patchVizSettings={patchVizSettings}
|
||||
onClose={() => setShowPerfSidebar(false)}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Hidden single-track audio element */}
|
||||
{!isMultiStem && audioUrl && (
|
||||
<audio
|
||||
|
||||
@@ -0,0 +1,445 @@
|
||||
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
import type { LyricsVisualStyle } from './LyricsLivePreview'
|
||||
|
||||
interface Stem {
|
||||
name: string
|
||||
label: string
|
||||
url: string
|
||||
}
|
||||
|
||||
interface SidebarStyleState {
|
||||
fontFamily: string
|
||||
fontSize: number
|
||||
fontBold: boolean
|
||||
textAlign: 'left' | 'center' | 'right'
|
||||
activeLineVerticalAlign: 'top' | 'center' | 'bottom'
|
||||
lineSpacing: number
|
||||
textColor: string
|
||||
highlightColor: string
|
||||
bgColor: string
|
||||
videoDisplayMode: 'scroll' | 'line' | 'word' | 'syllable'
|
||||
layoutOffsetXScroll: number
|
||||
layoutOffsetYScroll: number
|
||||
layoutOffsetXLine: number
|
||||
layoutOffsetYLine: number
|
||||
layoutOffsetXWord: number
|
||||
layoutOffsetYWord: number
|
||||
layoutOffsetXSyllable: number
|
||||
layoutOffsetYSyllable: number
|
||||
}
|
||||
|
||||
type DeepPartial<T> = {
|
||||
[K in keyof T]?: T[K] extends object ? DeepPartial<T[K]> : T[K]
|
||||
}
|
||||
|
||||
interface VizSettings {
|
||||
quality: 'low' | 'balanced' | 'high'
|
||||
fps_cap: 30 | 60 | 120
|
||||
muon: {
|
||||
radius_multiplier: number
|
||||
aperture: number
|
||||
spacing: number
|
||||
}
|
||||
fluid: {
|
||||
spawn_density: number
|
||||
splat_force: number
|
||||
decay: number
|
||||
}
|
||||
}
|
||||
|
||||
const VIZ_DEFAULTS: VizSettings = {
|
||||
quality: 'balanced',
|
||||
fps_cap: 60,
|
||||
muon: { radius_multiplier: 1, aperture: 1, spacing: 1 },
|
||||
fluid: { spawn_density: 1, splat_force: 1, decay: 1 },
|
||||
}
|
||||
|
||||
const FONT_OPTIONS = [
|
||||
{ value: 'system-ui, sans-serif', label: 'System' },
|
||||
{ value: '"Inter", sans-serif', label: 'Inter' },
|
||||
{ value: '"Work Sans", sans-serif', label: 'Work Sans' },
|
||||
{ value: '"Archivo", sans-serif', label: 'Archivo' },
|
||||
{ value: '"Poppins", sans-serif', label: 'Poppins' },
|
||||
{ value: '"Roboto", sans-serif', label: 'Roboto' },
|
||||
{ value: '"Oswald", sans-serif', label: 'Oswald' },
|
||||
{ value: '"EB Garamond", serif', label: 'EB Garamond' },
|
||||
{ value: '"Libre Baskerville", serif', label: 'Libre Baskerville' },
|
||||
{ value: '"Young Serif", serif', label: 'Young Serif' },
|
||||
]
|
||||
|
||||
interface PerformanceSidebarProps {
|
||||
isNarrow: boolean
|
||||
visualStyle: LyricsVisualStyle
|
||||
setVisualStyle: (s: LyricsVisualStyle) => void
|
||||
hasVideo: boolean
|
||||
stems: Stem[] | undefined
|
||||
mutedStems: Record<string, boolean>
|
||||
soloedStems: Record<string, boolean>
|
||||
gains: Record<string, number>
|
||||
toggleStemMute: (name: string) => void
|
||||
toggleStemSolo: (name: string) => void
|
||||
setStemGain: (name: string, value: number) => void
|
||||
style: SidebarStyleState
|
||||
patchStyle: (patch: Partial<SidebarStyleState>) => void
|
||||
vizSettings: VizSettings
|
||||
manifestVizSettings: VizSettings | null
|
||||
patchVizSettings: (patch: DeepPartial<VizSettings>) => void
|
||||
onClose: () => void
|
||||
}
|
||||
|
||||
function formatMultiplier(value: number): string {
|
||||
return `${value.toFixed(2)}×`
|
||||
}
|
||||
|
||||
function getLayoutOffset(style: SidebarStyleState): { x: number; y: number } {
|
||||
switch (style.videoDisplayMode) {
|
||||
case 'line':
|
||||
return { x: style.layoutOffsetXLine, y: style.layoutOffsetYLine }
|
||||
case 'word':
|
||||
return { x: style.layoutOffsetXWord, y: style.layoutOffsetYWord }
|
||||
case 'syllable':
|
||||
return { x: style.layoutOffsetXSyllable, y: style.layoutOffsetYSyllable }
|
||||
case 'scroll':
|
||||
default:
|
||||
return { x: style.layoutOffsetXScroll, y: style.layoutOffsetYScroll }
|
||||
}
|
||||
}
|
||||
|
||||
export default function PerformanceSidebar({
|
||||
isNarrow,
|
||||
visualStyle,
|
||||
setVisualStyle,
|
||||
hasVideo,
|
||||
stems,
|
||||
mutedStems,
|
||||
soloedStems,
|
||||
gains,
|
||||
toggleStemMute,
|
||||
toggleStemSolo,
|
||||
setStemGain,
|
||||
style,
|
||||
patchStyle,
|
||||
vizSettings,
|
||||
manifestVizSettings,
|
||||
patchVizSettings,
|
||||
onClose,
|
||||
}: PerformanceSidebarProps) {
|
||||
const [open, setOpen] = useState({
|
||||
background: true,
|
||||
stems: true,
|
||||
lyrics: true,
|
||||
visualizer: true,
|
||||
})
|
||||
const autoHideTimerRef = useRef<number | null>(null)
|
||||
|
||||
const scheduleAutoHide = useCallback(() => {
|
||||
if (autoHideTimerRef.current !== null) {
|
||||
window.clearTimeout(autoHideTimerRef.current)
|
||||
}
|
||||
autoHideTimerRef.current = window.setTimeout(() => {
|
||||
onClose()
|
||||
}, 4000)
|
||||
}, [onClose])
|
||||
|
||||
useEffect(() => {
|
||||
scheduleAutoHide()
|
||||
return () => {
|
||||
if (autoHideTimerRef.current !== null) {
|
||||
window.clearTimeout(autoHideTimerRef.current)
|
||||
}
|
||||
}
|
||||
}, [scheduleAutoHide])
|
||||
|
||||
const activeOffset = getLayoutOffset(style)
|
||||
const isVisualizerMode = visualStyle === 'muon' || visualStyle === 'fluid'
|
||||
const isStaticMode = visualStyle !== 'muon' && visualStyle !== 'fluid' && visualStyle !== 'video'
|
||||
|
||||
const patchOffset = (axis: 'x' | 'y', value: number) => {
|
||||
if (style.videoDisplayMode === 'line') {
|
||||
patchStyle(axis === 'x' ? { layoutOffsetXLine: value } : { layoutOffsetYLine: value })
|
||||
return
|
||||
}
|
||||
if (style.videoDisplayMode === 'word') {
|
||||
patchStyle(axis === 'x' ? { layoutOffsetXWord: value } : { layoutOffsetYWord: value })
|
||||
return
|
||||
}
|
||||
if (style.videoDisplayMode === 'syllable') {
|
||||
patchStyle(axis === 'x' ? { layoutOffsetXSyllable: value } : { layoutOffsetYSyllable: value })
|
||||
return
|
||||
}
|
||||
patchStyle(axis === 'x' ? { layoutOffsetXScroll: value } : { layoutOffsetYScroll: value })
|
||||
}
|
||||
|
||||
const sectionHeader = (key: keyof typeof open, label: string, hidden = false) => {
|
||||
if (hidden) return null
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setOpen((prev) => ({ ...prev, [key]: !prev[key] }))}
|
||||
className="w-full flex items-center justify-between text-xs font-semibold tracking-wide text-slate-200"
|
||||
>
|
||||
<span>{label}</span>
|
||||
<span className="text-slate-400">{open[key] ? '▾' : '▸'}</span>
|
||||
</button>
|
||||
)
|
||||
}
|
||||
|
||||
const panelClass = isNarrow
|
||||
? 'absolute inset-x-0 bottom-0 h-[62vh] rounded-t-2xl border-t border-white/15'
|
||||
: 'absolute right-0 top-0 h-full border-l border-white/15'
|
||||
|
||||
return (
|
||||
<div
|
||||
className={`${panelClass} z-[9] bg-black/70 backdrop-blur-md text-slate-100 overflow-y-auto`}
|
||||
style={isNarrow ? undefined : { width: 280 }}
|
||||
onMouseMove={scheduleAutoHide}
|
||||
onPointerMove={scheduleAutoHide}
|
||||
>
|
||||
<div className="sticky top-0 z-10 px-3 py-2 border-b border-white/10 bg-black/75 backdrop-blur-md flex items-center justify-between">
|
||||
<span className="text-xs font-semibold uppercase tracking-[0.08em] text-slate-300">Controls</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClose}
|
||||
className="w-7 h-7 rounded bg-white/10 hover:bg-white/20 text-slate-200 text-sm"
|
||||
title="Close"
|
||||
>
|
||||
✕
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="px-3 py-3 space-y-4">
|
||||
<section className="space-y-2">
|
||||
{sectionHeader('background', 'BACKGROUND')}
|
||||
{open.background && (
|
||||
<div className={isNarrow ? 'flex flex-wrap gap-2' : 'grid grid-cols-2 gap-2'}>
|
||||
{[
|
||||
{ key: 'static', label: 'Static', value: 'karaoke' as LyricsVisualStyle, active: isStaticMode },
|
||||
{ key: 'muon', label: 'Muon', value: 'muon' as LyricsVisualStyle, active: visualStyle === 'muon' },
|
||||
{ key: 'fluid', label: 'Fluid', value: 'fluid' as LyricsVisualStyle, active: visualStyle === 'fluid' },
|
||||
...(hasVideo ? [{ key: 'video', label: 'Video', value: 'video' as LyricsVisualStyle, active: visualStyle === 'video' }] : []),
|
||||
].map((mode) => (
|
||||
<button
|
||||
key={mode.key}
|
||||
type="button"
|
||||
onClick={() => setVisualStyle(mode.value)}
|
||||
className={`rounded-md px-2 py-2 text-xs border transition-colors ${isNarrow ? 'min-w-[70px] flex-1' : ''} ${mode.active ? 'border-violet-400 bg-violet-500/20 text-white' : 'border-white/15 bg-white/5 text-slate-300 hover:bg-white/10'}`}
|
||||
>
|
||||
{mode.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
|
||||
<section className="space-y-2">
|
||||
{sectionHeader('stems', 'STEMS')}
|
||||
{open.stems && (
|
||||
<div className="space-y-2">
|
||||
{stems?.length ? stems.map((stem) => (
|
||||
<div key={stem.name} className="space-y-1.5">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-xs font-medium text-slate-300 flex-1 truncate">{stem.label}</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => toggleStemMute(stem.name)}
|
||||
className={`w-6 h-6 text-[10px] rounded ${mutedStems[stem.name] ? 'bg-red-500 text-white' : 'bg-white/10 text-slate-300'}`}
|
||||
>
|
||||
M
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => toggleStemSolo(stem.name)}
|
||||
className={`w-6 h-6 text-[10px] rounded ${soloedStems[stem.name] ? 'bg-yellow-300 text-black' : 'bg-white/10 text-slate-300'}`}
|
||||
>
|
||||
S
|
||||
</button>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<input
|
||||
type="range"
|
||||
min={0}
|
||||
max={2}
|
||||
step={0.05}
|
||||
value={gains[stem.name] ?? 1}
|
||||
onChange={(event) => setStemGain(stem.name, parseFloat(event.target.value))}
|
||||
className="w-full h-1.5 accent-brand-500"
|
||||
/>
|
||||
<span className="text-[10px] tabular-nums text-slate-400 w-11 text-right">{Math.round((gains[stem.name] ?? 1) * 100)}%</span>
|
||||
</div>
|
||||
</div>
|
||||
)) : (
|
||||
<p className="text-[11px] text-slate-400">No stems — single track mode</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
|
||||
<section className="space-y-2">
|
||||
{sectionHeader('lyrics', 'LYRICS')}
|
||||
{open.lyrics && (
|
||||
<div className="space-y-2.5">
|
||||
<div className="flex items-center gap-2">
|
||||
<select
|
||||
value={style.fontFamily}
|
||||
onChange={(event) => patchStyle({ fontFamily: event.target.value })}
|
||||
className="flex-1 text-xs bg-white/10 border border-white/15 rounded px-2 py-1"
|
||||
>
|
||||
{FONT_OPTIONS.map((font) => (
|
||||
<option key={font.value} value={font.value} style={{ background: '#0f172a', color: '#e2e8f0', fontFamily: font.value }}>
|
||||
{font.label}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => patchStyle({ fontBold: !style.fontBold })}
|
||||
className={`px-2 py-1 rounded text-xs font-semibold ${style.fontBold ? 'bg-white/25 text-white' : 'bg-white/10 text-slate-300'}`}
|
||||
>
|
||||
B
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Size</span><span>{style.fontSize}</span></div>
|
||||
<input type="range" min={12} max={56} step={1} value={style.fontSize} onChange={(event) => patchStyle({ fontSize: parseInt(event.target.value, 10) })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
|
||||
<div className="flex items-center gap-2">
|
||||
{(['left', 'center', 'right'] as const).map((align) => (
|
||||
<button
|
||||
key={align}
|
||||
type="button"
|
||||
onClick={() => patchStyle({ textAlign: align })}
|
||||
className={`flex-1 rounded px-2 py-1 text-xs ${style.textAlign === align ? 'bg-white/20 text-white' : 'bg-white/10 text-slate-300'}`}
|
||||
>
|
||||
{align === 'left' ? 'L' : align === 'center' ? 'C' : 'R'}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div className="flex items-center gap-2">
|
||||
{(['top', 'center', 'bottom'] as const).map((pos) => (
|
||||
<button
|
||||
key={pos}
|
||||
type="button"
|
||||
onClick={() => patchStyle({ activeLineVerticalAlign: pos })}
|
||||
className={`flex-1 rounded px-2 py-1 text-xs ${style.activeLineVerticalAlign === pos ? 'bg-white/20 text-white' : 'bg-white/10 text-slate-300'}`}
|
||||
>
|
||||
{pos === 'top' ? 'T' : pos === 'center' ? 'C' : 'B'}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Line spacing</span><span>{style.lineSpacing.toFixed(2)}</span></div>
|
||||
<input type="range" min={1} max={2.5} step={0.05} value={style.lineSpacing} onChange={(event) => patchStyle({ lineSpacing: parseFloat(event.target.value) })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-3 gap-2 text-[10px] text-slate-300">
|
||||
<label className="space-y-1"><span className="block">Text</span><input type="color" value={style.textColor} onChange={(event) => patchStyle({ textColor: event.target.value })} className="w-full h-7 rounded border border-white/15 bg-transparent" /></label>
|
||||
<label className="space-y-1"><span className="block">Highlight</span><input type="color" value={style.highlightColor} onChange={(event) => patchStyle({ highlightColor: event.target.value })} className="w-full h-7 rounded border border-white/15 bg-transparent" /></label>
|
||||
<label className="space-y-1"><span className="block">BG</span><input type="color" value={style.bgColor} onChange={(event) => patchStyle({ bgColor: event.target.value })} className="w-full h-7 rounded border border-white/15 bg-transparent" /></label>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>X offset</span><span>{activeOffset.x}</span></div>
|
||||
<input type="range" min={-50} max={50} step={1} value={activeOffset.x} onChange={(event) => patchOffset('x', parseInt(event.target.value, 10))} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Y offset</span><span>{activeOffset.y}</span></div>
|
||||
<input type="range" min={-50} max={50} step={1} value={activeOffset.y} onChange={(event) => patchOffset('y', parseInt(event.target.value, 10))} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
|
||||
<section className="space-y-2">
|
||||
{sectionHeader('visualizer', 'VISUALIZER')}
|
||||
{open.visualizer && (
|
||||
<div className="space-y-2.5">
|
||||
<div className="space-y-1.5">
|
||||
<div className="text-[10px] text-slate-400">Quality</div>
|
||||
<div className="grid grid-cols-3 gap-1.5">
|
||||
{(['low', 'balanced', 'high'] as const).map((quality) => (
|
||||
<button
|
||||
key={quality}
|
||||
type="button"
|
||||
onClick={() => patchVizSettings({ quality })}
|
||||
className={`rounded px-2 py-1 text-[11px] border ${vizSettings.quality === quality ? 'border-violet-400 bg-violet-500/20 text-white' : 'border-white/15 bg-white/5 text-slate-300 hover:bg-white/10'}`}
|
||||
>
|
||||
{quality === 'balanced' ? 'Balanced' : quality === 'low' ? 'Low' : 'High'}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1.5">
|
||||
<div className="text-[10px] text-slate-400">FPS cap</div>
|
||||
<div className="grid grid-cols-3 gap-1.5">
|
||||
{([30, 60, 120] as const).map((fps) => (
|
||||
<button
|
||||
key={fps}
|
||||
type="button"
|
||||
onClick={() => patchVizSettings({ fps_cap: fps })}
|
||||
className={`rounded px-2 py-1 text-[11px] border ${vizSettings.fps_cap === fps ? 'border-violet-400 bg-violet-500/20 text-white' : 'border-white/15 bg-white/5 text-slate-300 hover:bg-white/10'}`}
|
||||
>
|
||||
{fps}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{!isVisualizerMode && (
|
||||
<p className="text-[11px] text-slate-400">
|
||||
Switch background to Muon or Fluid to access mode-specific controls.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{visualStyle === 'muon' && (
|
||||
<>
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Radius</span><span>{formatMultiplier(vizSettings.muon.radius_multiplier)}</span></div>
|
||||
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.muon.radius_multiplier} onChange={(event) => patchVizSettings({ muon: { radius_multiplier: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Aperture</span><span>{formatMultiplier(vizSettings.muon.aperture)}</span></div>
|
||||
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.muon.aperture} onChange={(event) => patchVizSettings({ muon: { aperture: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Spacing</span><span>{formatMultiplier(vizSettings.muon.spacing)}</span></div>
|
||||
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.muon.spacing} onChange={(event) => patchVizSettings({ muon: { spacing: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
{visualStyle === 'fluid' && (
|
||||
<>
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Density</span><span>{formatMultiplier(vizSettings.fluid.spawn_density)}</span></div>
|
||||
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.fluid.spawn_density} onChange={(event) => patchVizSettings({ fluid: { spawn_density: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Force</span><span>{formatMultiplier(vizSettings.fluid.splat_force)}</span></div>
|
||||
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.fluid.splat_force} onChange={(event) => patchVizSettings({ fluid: { splat_force: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
<div className="space-y-1">
|
||||
<div className="flex items-center justify-between text-[10px] text-slate-400"><span>Trail decay</span><span>{formatMultiplier(vizSettings.fluid.decay)}</span></div>
|
||||
<input type="range" min={0.25} max={2} step={0.05} value={vizSettings.fluid.decay} onChange={(event) => patchVizSettings({ fluid: { decay: parseFloat(event.target.value) } })} className="w-full h-1.5 accent-brand-500" />
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => patchVizSettings(manifestVizSettings ?? VIZ_DEFAULTS)}
|
||||
className="w-full rounded-md px-2 py-1.5 text-xs bg-white/10 hover:bg-white/20 text-slate-200"
|
||||
>
|
||||
{manifestVizSettings ? 'Reset to song defaults' : 'Reset to defaults'}
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -8,6 +8,13 @@ interface FluidVisualizerCanvasProps {
|
||||
paletteRef: MutableRefObject<VisualizerPalette>
|
||||
width: number
|
||||
height: number
|
||||
fpsCap?: 30 | 60 | 120
|
||||
quality?: 'low' | 'balanced' | 'high'
|
||||
settings?: {
|
||||
spawn_density?: number
|
||||
splat_force?: number
|
||||
decay?: number
|
||||
}
|
||||
}
|
||||
|
||||
type Blob = {
|
||||
@@ -24,7 +31,7 @@ function clamp01(v: number): number {
|
||||
return Math.max(0, Math.min(1, v))
|
||||
}
|
||||
|
||||
export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width, height }: FluidVisualizerCanvasProps) {
|
||||
export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width, height, fpsCap = 60, quality = 'balanced', settings }: FluidVisualizerCanvasProps) {
|
||||
const canvasRef = useRef<HTMLCanvasElement>(null)
|
||||
|
||||
useEffect(() => {
|
||||
@@ -36,6 +43,9 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
|
||||
const blobs: Blob[] = []
|
||||
let raf = 0
|
||||
let colorIndex = 0
|
||||
let lastFrameAt = 0
|
||||
const cap = fpsCap === 30 || fpsCap === 120 ? fpsCap : 60
|
||||
const minFrameMs = 1000 / cap
|
||||
|
||||
const nextPaletteColor = () => {
|
||||
const palette = paletteRef.current.speakerPalette
|
||||
@@ -48,21 +58,24 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
|
||||
const spawnBlob = (strength: number) => {
|
||||
const w = Math.max(1, width)
|
||||
const h = Math.max(1, height)
|
||||
const forceScale = Math.max(0.2, Number(settings?.splat_force ?? 1))
|
||||
const angle = Math.random() * Math.PI * 2
|
||||
const speed = 0.2 + strength * 2.2
|
||||
const speed = (0.2 + strength * 2.2) * forceScale
|
||||
blobs.push({
|
||||
x: Math.random() * w,
|
||||
y: Math.random() * h,
|
||||
dx: Math.cos(angle) * speed,
|
||||
dy: Math.sin(angle) * speed,
|
||||
radius: 8 + strength * 38,
|
||||
radius: (8 + strength * 38) * Math.max(0.25, Math.sqrt(forceScale)),
|
||||
life: 0.7 + strength * 0.6,
|
||||
color: nextPaletteColor(),
|
||||
})
|
||||
}
|
||||
|
||||
const render = () => {
|
||||
const render = (ts: number) => {
|
||||
raf = window.requestAnimationFrame(render)
|
||||
if (lastFrameAt && ts - lastFrameAt < minFrameMs) return
|
||||
lastFrameAt = ts
|
||||
|
||||
const w = Math.max(1, Math.round(canvas.clientWidth || 0), Math.round(width || 0))
|
||||
const h = Math.max(1, Math.round(canvas.clientHeight || 0), Math.round(height || 0))
|
||||
@@ -77,13 +90,17 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
|
||||
const rms = clamp01(f.rms)
|
||||
const treble = clamp01(f.treble)
|
||||
const idle = 0.5 + 0.5 * Math.sin(performance.now() * 0.0011)
|
||||
const densityScale = Math.max(0.2, Number(settings?.spawn_density ?? 1))
|
||||
const decayScale = Math.max(0.2, Number(settings?.decay ?? 1))
|
||||
|
||||
// Ambient motion from RMS + small idle baseline so the renderer is never visually flat.
|
||||
if (Math.random() < (0.04 + rms * 0.31)) spawnBlob(0.22 + rms * 0.58 + idle * 0.08)
|
||||
const ambientSpawnScale = quality === 'low' ? 0.7 : 1
|
||||
if (Math.random() < (0.04 + rms * 0.31) * densityScale * ambientSpawnScale) spawnBlob(0.22 + rms * 0.58 + idle * 0.08)
|
||||
|
||||
// Beat burst from bass peaks.
|
||||
if (f.beatDetected) {
|
||||
const count = 3 + Math.floor(bass * 4)
|
||||
const baseCount = 3 + Math.floor(bass * 4)
|
||||
const count = quality === 'low' ? Math.max(1, baseCount - 1) : baseCount
|
||||
for (let i = 0; i < count; i += 1) spawnBlob(0.6 + bass * 0.8)
|
||||
}
|
||||
|
||||
@@ -92,8 +109,14 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
|
||||
ctx.fillRect(0, 0, w, h)
|
||||
ctx.globalAlpha = 1
|
||||
|
||||
const decay = 0.009 + (1 - rms) * 0.012
|
||||
const drag = 0.984 - bass * 0.01
|
||||
// Correction C fix: trail lifetime (decay) is driven by bass — loud kick/bass →
|
||||
// slower decay → longer-lived blobs (more persistent ink on loud beats).
|
||||
// Velocity drag is driven by rms — overall loudness → more momentum retained.
|
||||
// Original Integration Plan had these driven by rms uniformly, which made
|
||||
// trail length insensitive to bass and meant quiet treble-heavy passages
|
||||
// produced the same persistence as loud bass hits.
|
||||
const decay = (0.009 + (1 - bass) * 0.012) * decayScale // loud bass → 0.009 (long trail)
|
||||
const drag = 0.984 - rms * 0.010 // loud rms → more momentum
|
||||
|
||||
for (let i = blobs.length - 1; i >= 0; i -= 1) {
|
||||
const b = blobs[i]
|
||||
@@ -121,11 +144,11 @@ export default function FluidVisualizerCanvas({ featuresRef, paletteRef, width,
|
||||
}
|
||||
}
|
||||
|
||||
render()
|
||||
raf = window.requestAnimationFrame(render)
|
||||
return () => {
|
||||
window.cancelAnimationFrame(raf)
|
||||
}
|
||||
}, [featuresRef, paletteRef, width, height])
|
||||
}, [featuresRef, paletteRef, width, height, fpsCap, quality, settings])
|
||||
|
||||
return <canvas ref={canvasRef} className="h-full w-full" aria-hidden="true" />
|
||||
}
|
||||
@@ -8,6 +8,13 @@ interface MuonVisualizerCanvasProps {
|
||||
paletteRef: MutableRefObject<VisualizerPalette>
|
||||
width: number
|
||||
height: number
|
||||
fpsCap?: 30 | 60 | 120
|
||||
quality?: 'low' | 'balanced' | 'high'
|
||||
settings?: {
|
||||
radius_multiplier?: number
|
||||
aperture?: number
|
||||
spacing?: number
|
||||
}
|
||||
}
|
||||
|
||||
type Particle = { angle: number; radius: number; speed: number; lane: number }
|
||||
@@ -18,7 +25,7 @@ function clamp01(v: number): number {
|
||||
return Math.max(0, Math.min(1, v))
|
||||
}
|
||||
|
||||
export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, height }: MuonVisualizerCanvasProps) {
|
||||
export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, height, fpsCap = 60, quality = 'balanced', settings }: MuonVisualizerCanvasProps) {
|
||||
const canvasRef = useRef<HTMLCanvasElement>(null)
|
||||
|
||||
useEffect(() => {
|
||||
@@ -36,8 +43,23 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
|
||||
|
||||
let raf = 0
|
||||
|
||||
const render = () => {
|
||||
// Correction B fix: radiusMultiplier cycles through presets on beats rather than
|
||||
// being mapped directly from drum intensity each frame. We lerp toward a preset
|
||||
// target, snapping to the next preset on each detected beat — equivalent to
|
||||
// wavePresetController triggering a GSAP transition in the full Three.js port.
|
||||
const RADIUS_PRESETS = [0.90, 0.95, 1.00, 1.05, 1.10, 0.85, 1.15, 1.02]
|
||||
let radiusTarget = 1.0
|
||||
let radiusCurrent = 1.0
|
||||
let lastPresetAt = 0
|
||||
let presetIdx = 0
|
||||
let lastFrameAt = 0
|
||||
const cap = fpsCap === 30 || fpsCap === 120 ? fpsCap : 60
|
||||
const minFrameMs = 1000 / cap
|
||||
|
||||
const render = (ts: number) => {
|
||||
raf = window.requestAnimationFrame(render)
|
||||
if (lastFrameAt && ts - lastFrameAt < minFrameMs) return
|
||||
lastFrameAt = ts
|
||||
const w = Math.max(1, Math.round(canvas.clientWidth || 0), Math.round(width || 0))
|
||||
const h = Math.max(1, Math.round(canvas.clientHeight || 0), Math.round(height || 0))
|
||||
if (canvas.width !== w || canvas.height !== h) {
|
||||
@@ -49,12 +71,34 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
|
||||
const p = paletteRef.current
|
||||
const drum = clamp01(f.drumIntensity)
|
||||
const treble = clamp01(f.treble)
|
||||
const idle = 0.5 + 0.5 * Math.sin(performance.now() * 0.0012)
|
||||
const now = ts
|
||||
const idle = 0.5 + 0.5 * Math.sin(now * 0.0012)
|
||||
const radiusScale = Math.max(0.2, Number(settings?.radius_multiplier ?? 1))
|
||||
const apertureScale = Math.max(0.2, Number(settings?.aperture ?? 1))
|
||||
const spacingScale = Math.max(0.2, Number(settings?.spacing ?? 1))
|
||||
|
||||
const radiusMultiplier = 0.82 + drum * 1.12 + idle * 0.18
|
||||
const aperture = 1.55 + drum * 1.9 + idle * 0.24
|
||||
const spacing = 0.015 + (1 - drum) * 0.014
|
||||
const hue = (p.seedHue + treble * 40) % 360
|
||||
// Bug 1 fix: hue animates via drum intensity when audio is playing, and falls
|
||||
// back to a slow time-based drift when silent so the visual never freezes on a
|
||||
// single colour (original hueControl bug: hue was always 0 during playback).
|
||||
// Correction A fix: treble drift is capped at ±15° to stay within the manifest
|
||||
// palette colour family (original Integration Plan used treble * 40 / 120 which
|
||||
// rotated far outside the intended hue family).
|
||||
const hue = drum > 0.01
|
||||
? ((p.seedHue + drum * 20 + treble * 15) + 360) % 360
|
||||
: ((p.seedHue + Math.sin(now * 0.00018) * 15) + 360) % 360
|
||||
|
||||
// Correction B fix: advance the preset index on each beat and lerp toward the
|
||||
// new target — mirrors wavePresetController / GSAP transition in the Three.js port.
|
||||
if (f.beatDetected && now - lastPresetAt > 200) {
|
||||
presetIdx = (presetIdx + 1) % RADIUS_PRESETS.length
|
||||
radiusTarget = RADIUS_PRESETS[presetIdx] + drum * 0.25
|
||||
lastPresetAt = now
|
||||
}
|
||||
radiusCurrent += (radiusTarget - radiusCurrent) * 0.07
|
||||
const radiusMultiplier = (radiusCurrent + idle * 0.12) * radiusScale
|
||||
|
||||
const aperture = (1.55 + drum * 1.9 + idle * 0.24) * apertureScale
|
||||
const spacing = (0.015 + (1 - drum) * 0.014) * spacingScale
|
||||
|
||||
ctx.clearRect(0, 0, w, h)
|
||||
ctx.fillStyle = p.backgroundColor
|
||||
@@ -64,9 +108,10 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
|
||||
const cy = h / 2
|
||||
const base = Math.min(w, h) * 0.2
|
||||
const glow = 14 + drum * 28
|
||||
const particleStride = quality === 'low' ? 2 : 1
|
||||
|
||||
ctx.globalCompositeOperation = 'lighter'
|
||||
for (let i = 0; i < particles.length; i += 1) {
|
||||
for (let i = 0; i < particles.length; i += particleStride) {
|
||||
const pt = particles[i]
|
||||
pt.angle += pt.speed + drum * 0.02
|
||||
|
||||
@@ -89,11 +134,11 @@ export default function MuonVisualizerCanvas({ featuresRef, paletteRef, width, h
|
||||
ctx.shadowBlur = 0
|
||||
}
|
||||
|
||||
render()
|
||||
raf = window.requestAnimationFrame(render)
|
||||
return () => {
|
||||
window.cancelAnimationFrame(raf)
|
||||
}
|
||||
}, [featuresRef, paletteRef, width, height])
|
||||
}, [featuresRef, paletteRef, width, height, fpsCap, quality, settings])
|
||||
|
||||
return <canvas ref={canvasRef} className="h-full w-full" aria-hidden="true" />
|
||||
}
|
||||
@@ -8,15 +8,41 @@ const FluidVisualizerCanvas = lazy(() => import('./FluidVisualizerCanvas'))
|
||||
|
||||
export type VisualizerMode = 'muon' | 'fluid' | null
|
||||
|
||||
/**
|
||||
* Per-song visualizer intensity multipliers from manifest.visualizer.settings.
|
||||
* All values default to 1.0 (no change from audio-driven behaviour).
|
||||
*/
|
||||
export interface VisualizerSettings {
|
||||
quality?: 'low' | 'balanced' | 'high'
|
||||
fps_cap?: 30 | 60 | 120
|
||||
muon?: {
|
||||
/** Scales the preset-cycling radius range. Default 1.0. */
|
||||
radius_multiplier?: number
|
||||
/** Scales the spiral aperture range. Default 1.0. */
|
||||
aperture?: number
|
||||
/** Scales inter-particle spacing. Default 1.0. */
|
||||
spacing?: number
|
||||
}
|
||||
fluid?: {
|
||||
/** Multiplies ambient blob spawn probability per frame. Default 1.0. */
|
||||
spawn_density?: number
|
||||
/** Multiplies blob velocity and size on spawn. Default 1.0. */
|
||||
splat_force?: number
|
||||
/** Multiplies per-frame blob lifetime drain (higher = shorter trails). Default 1.0. */
|
||||
decay?: number
|
||||
}
|
||||
}
|
||||
|
||||
interface VisualizerBackdropProps {
|
||||
mode: VisualizerMode
|
||||
featuresRef: MutableRefObject<AudioFeatures>
|
||||
paletteRef: MutableRefObject<VisualizerPalette>
|
||||
width: number
|
||||
height: number
|
||||
settings?: VisualizerSettings | null
|
||||
}
|
||||
|
||||
export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, width, height }: VisualizerBackdropProps) {
|
||||
export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, width, height, settings }: VisualizerBackdropProps) {
|
||||
const palette = paletteRef.current
|
||||
|
||||
if (mode === 'muon') {
|
||||
@@ -31,7 +57,7 @@ export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, widt
|
||||
data-palette-word-highlight={palette.wordHighlightColor}
|
||||
>
|
||||
<Suspense fallback={<div className="h-full w-full bg-[#05070d]" />}>
|
||||
<MuonVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} />
|
||||
<MuonVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} settings={settings?.muon} fpsCap={settings?.fps_cap} quality={settings?.quality} />
|
||||
</Suspense>
|
||||
</div>
|
||||
)
|
||||
@@ -48,7 +74,7 @@ export default function VisualizerBackdrop({ mode, featuresRef, paletteRef, widt
|
||||
data-palette-word-highlight={palette.wordHighlightColor}
|
||||
>
|
||||
<Suspense fallback={<div className="h-full w-full bg-[#05070d]" />}>
|
||||
<FluidVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} />
|
||||
<FluidVisualizerCanvas featuresRef={featuresRef} paletteRef={paletteRef} width={width} height={height} settings={settings?.fluid} fpsCap={settings?.fps_cap} quality={settings?.quality} />
|
||||
</Suspense>
|
||||
</div>
|
||||
)
|
||||
|
||||
@@ -60,8 +60,16 @@ function avgSlice(data: Uint8Array, start: number, end: number): number {
|
||||
}
|
||||
|
||||
function deriveDrumIntensity(bass: number, treble: number, rms: number): number {
|
||||
// Weight bass the most, keep a small RMS contribution for non-drum content.
|
||||
const raw = bass * 0.68 + rms * 0.24 + treble * 0.08
|
||||
// Exponential scalers matching the corrected muon source audio.js pattern
|
||||
// (Bug 2 fix). Input range 0–1; outputs range 0–2.002 (normalised back below).
|
||||
// Original formula: 1.001 * (1 + avgFr * 2) - 1.001
|
||||
// This curve is near-zero when quiet and amplifies progressively when loud,
|
||||
// giving a more musical response than a plain linear mix.
|
||||
const expBass = 1.001 * (1 + bass * 2) - 1.001
|
||||
const expTreble = 1.001 * (1 + treble * 2) - 1.001
|
||||
// Coefficient ratio mirrors wave.js beatScaler (bass ~2× treble).
|
||||
// Divide by 2.0 to normalise the 0–2.002 range back to 0–1 before clamping.
|
||||
const raw = (expBass * 0.68 + rms * 0.24 + expTreble * 0.08) / 2.0
|
||||
return clamp01(Math.pow(clamp01(raw), 0.82))
|
||||
}
|
||||
|
||||
@@ -198,6 +206,8 @@ export function useAudioFeatures({
|
||||
|
||||
analyser.getByteFrequencyData(byteFreq)
|
||||
|
||||
// Bug 2 fix: low bins (0–9) → bass, high bins (last 56 of 256) → treble.
|
||||
// Original muon source had these reversed (upperHalfArray → baseFr, etc.).
|
||||
const bass = avgSlice(byteFreq, 0, 10) / 255
|
||||
const treble = avgSlice(byteFreq, Math.max(0, byteFreq.length - 56), byteFreq.length) / 255
|
||||
const energy = byteFreq.length ? (avgSlice(byteFreq, 0, byteFreq.length) / 255) : 0
|
||||
|
||||
@@ -473,6 +473,7 @@ export default function KaraokePage() {
|
||||
coverArtUrl={coverArtUrl}
|
||||
sekDynamicPalette={sekManifest?.manifest?.metadata?.dynamic_palette ?? null}
|
||||
sekVisualizerMode={sekManifest?.manifest?.visualizer?.mode ?? null}
|
||||
sekVisualizerSettings={sekManifest?.manifest?.visualizer?.settings ?? null}
|
||||
upNextSong={nextItem ? { title: nextItem.title, artist: nextItem.artist } : null}
|
||||
onSongEnded={handleSongEnded}
|
||||
onExit={handleExit}
|
||||
|
||||
@@ -542,6 +542,7 @@ export default function KaraokeTVPage() {
|
||||
coverArtUrl={coverArtUrl}
|
||||
sekDynamicPalette={sekManifest?.manifest?.metadata?.dynamic_palette ?? null}
|
||||
sekVisualizerMode={sekManifest?.manifest?.visualizer?.mode ?? null}
|
||||
sekVisualizerSettings={sekManifest?.manifest?.visualizer?.settings ?? null}
|
||||
disableGlobalPlaybackRegistry
|
||||
upNextSong={nextItem ? { title: nextItem.title, artist: nextItem.artist } : null}
|
||||
onSongEnded={handleSongEnded}
|
||||
|
||||
Reference in new issue
Block a user