diff --git a/docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md b/docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md new file mode 100644 index 0000000..19f96f7 --- /dev/null +++ b/docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md @@ -0,0 +1,78 @@ +# Muon Visualizer Current Per-Mode Functionality + +Date: 2026-04-27 +Environment: StemEngine track preview (http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb) +Mode set under test: `default`, `fluid`, `tendrils`, `spirit` + +## Scope +This document captures the CURRENT observed behavior for Muon style modes in Track Detail preview, with focus on repeat-play reactivity. + +Reactivity definition used here: +- cycle 1 reactive: visual output changes after first Play preview +- cycle 2 reactive: visual output still changes after Pause -> Play replay + +## Summary +- `default`: functional with caveats +- `fluid`: reproducible bug on replay (cycle 2 non-reactive) +- `tendrils`: functional in current sweep, but keep under watch (user-reported instability exists) +- `spirit`: functional in current sweep, but keep under watch (user-reported instability exists) + +## Per-Mode Status + +### default +Status: Functional with caveats +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: reactive (3/3 runs) +Caveats: +- Intermittent stream-layer noise still appears in session logs/events (`ERR_ABORTED`, occasional `502`), but did not break reactivity in this sweep. + +### fluid +Status: Buggy (reproducible) +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: NOT reactive (3/3 runs) +Evidence: +- Motion diff for cycle 2 repeatedly dropped to `0` +- Console repeatedly logs: `[StemEngine][Fluid] FBO incomplete: 36054` +Impact: +- Replay reliability is broken for fluid mode under current build/runtime combination. + +### tendrils +Status: Functional in current session (watchlist) +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: reactive (3/3 runs) +Caveat: +- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep. + +### spirit +Status: Functional in current session (watchlist) +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: reactive (3/3 runs) +Caveat: +- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep. + +## Raw Sweep Snapshot (Automated) +Run 1: +- default: c1 reactive, c2 reactive +- fluid: c1 reactive, c2 non-reactive +- tendrils: c1 reactive, c2 reactive +- spirit: c1 reactive, c2 reactive + +Run 2: +- default: c1 reactive, c2 reactive +- fluid: c1 reactive, c2 non-reactive +- tendrils: c1 reactive, c2 reactive +- spirit: c1 reactive, c2 reactive + +Run 3: +- default: c1 reactive, c2 reactive +- fluid: c1 reactive, c2 non-reactive +- tendrils: c1 reactive, c2 reactive +- spirit: c1 reactive, c2 reactive + +## Notes +- This report is intentionally point-in-time. Re-run after renderer/transport changes to keep this file current. +- If you want, add a companion matrix file for Visualizer Lab page and project-page embed to compare behavior across surfaces. diff --git a/docs/muon-mode-status/2026-04-28-export-bundle-manifest.md b/docs/muon-mode-status/2026-04-28-export-bundle-manifest.md new file mode 100644 index 0000000..febd1a0 --- /dev/null +++ b/docs/muon-mode-status/2026-04-28-export-bundle-manifest.md @@ -0,0 +1,95 @@ +# Muon Review Export Bundle Manifest + +Date: 2026-04-28 +Purpose: frozen manifest for external sharing of the Muon mode review package. + +## Bundle Name +Suggested bundle label: +- `muon-mode-review-package-2026-04-28` + +## Bundle Scope +This export bundle is intended for third-party reviewers who need: +- current operational status by mode, +- surface-by-surface testability notes, +- reference-comparison analysis, +- a reviewer scoring rubric, +- a single index file for navigation. + +## Files To Include + +### Core review docs +1. `docs/muon-mode-status/README.md` + - Entry point and reading order. + +2. `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` + - Point-in-time Track Detail mode functionality and replay observations. + +3. `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` + - Surface matrix across Track Detail, Visualizer Lab, and project/embed views. + +4. `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` + - Reference comparison for `fluid`, `tendrils`, and `spirit`. + +5. `docs/muon-mode-status/2026-04-28-reviewer-checklist.md` + - Third-party scoring and signoff rubric. + +### Optional supporting source files +Include these if the reviewer needs implementation-level evidence: +6. `muon-iframe/fluid-render-graph.js` +7. `muon-iframe/tendrils-render-graph.js` +8. `muon-iframe/spirit-render-graph.js` +9. `muon-iframe/stemengine-router.js` +10. `muon-iframe/stemengine-bootstrap.js` + +## Minimal Bundle +If the recipient should receive docs only, export: +- `docs/muon-mode-status/README.md` +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` +- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` +- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md` + +## Full Evidence Bundle +If the recipient should receive both review docs and implementation evidence, export: +- all files from the Minimal Bundle +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` + +## Suggested Folder Layout For Sharing +```text +muon-mode-review-package-2026-04-28/ + README.md + docs/ + 2026-04-27-current-per-mode-functionality.md + 2026-04-28-surface-functionality-matrix.md + 2026-04-28-reference-comparison-third-party-review.md + 2026-04-28-reviewer-checklist.md + source-evidence/ + fluid-render-graph.js + tendrils-render-graph.js + spirit-render-graph.js + stemengine-router.js + stemengine-bootstrap.js +``` + +## Export Notes +- This manifest freezes the intended review set as of 2026-04-28. +- The bundle is evidence-oriented, not a claim of source-equivalent parity with external references. +- Reference-comparison conclusions include inference and confidence levels; reviewers should read those caveats before scoring fidelity. + +## Reviewer Guidance +Recommended reading sequence inside the exported bundle: +1. `README.md` +2. `2026-04-28-reference-comparison-third-party-review.md` +3. `2026-04-27-current-per-mode-functionality.md` +4. `2026-04-28-surface-functionality-matrix.md` +5. `2026-04-28-reviewer-checklist.md` + +## Current Package Status +As of this manifest: +- `fluid` remains the main concern because of replay/FBO reliability issues. +- `tendrils` and `spirit` are documented as in-family approximations, not proven literal ports. +- Visualizer Lab and project/embed surfaces remain limited or non-testable for this exact replay matrix without additional setup. diff --git a/docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md b/docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md new file mode 100644 index 0000000..c6aa9e2 --- /dev/null +++ b/docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md @@ -0,0 +1,141 @@ +# Muon Mode Comparison vs Reference Demos (Third-Party Review) + +Date: 2026-04-28 +Prepared for: external/third-party technical review +Scope: infer how local `fluid`, `tendrils`, and `spirit` modes compare to their likely inspiration references. + +References under review: +- https://paveldogreat.github.io/WebGL-Fluid-Simulation/ +- https://keeffeoghan.github.io/work/tendrils/ (redirects to `epok.tech` content) +- https://edankwan.com/experiments/the-spirit/ + +Local implementation inputs reviewed: +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` + +## Method Notes +This review intentionally separates: +- verified local facts (from readable source files), and +- inferred reference characteristics (from live pages + publicly visible descriptions + agent-assisted analysis). + +A `ui-designer` subagent was used to produce an independent reference summary, then cross-checked against local code. + +## Executive Summary +1. `fluid` in this fork is structurally the closest to its reference target (PavelDoGreat fluid simulation), because it implements a recognizable multi-pass fluid pipeline (splat, divergence, pressure Jacobi, gradient subtract, advection, bloom). +2. `tendrils` and `spirit` in this fork are stylistic approximations, not faithful reproductions of the original interactive systems. They are compact shader-feedback variants optimized for integration into Muon’s transport/audio stack. +3. The local `fluid` path currently has a practical stability/replay issue in your environment (`FBO incomplete: 36054` warnings and cycle-2 non-reactivity), which materially affects parity with the expected “real thing” feel. + +## Comparison Matrix + +| Mode | Local Implementation (verified) | Reference “Real Thing” (inferred) | Similarity Assessment | +|---|---|---|---| +| fluid | Full-screen multipass FBO fluid pipeline with pressure solve, advection, bloom, and audio splat injection | Interactive Navier-Stokes style GPU fluid sim with user splats and polished post FX | High structural similarity; medium experiential similarity due to current stability issues and integration constraints | +| tendrils | Feedback texture + procedural flow/noise line structures in one update shader, audio-driven decay/flow | Emergent particle-field ecosystem with richer control surface, presets, and interaction channels (keyboard/camera/mic) | Medium visual similarity; low-to-medium system similarity | +| spirit | Layered atmospheric feedback shader with fbm/noise, halo/strand/pulse composition, audio-driven fade | Highly polished ethereal particle/trail experience with strong cinematic motion language | Medium stylistic similarity; low implementation-parity confidence | + +## Per-Mode Detailed Assessment + +### A) `fluid` +Local verified facts: +- Uses explicit fluid-solver style passes: + - curl, divergence, pressure Jacobi, gradient subtract, velocity advection, dye advection. +- Uses ping-pong FBOs for velocity/dye/pressure and optional bloom chain by quality tier. +- Audio coupling includes bass/sub/treble/rms/beat/beatPhase/drumIntensity mapped to splat radius/force/color. +- Quality tier controls pressure iterations and bloom enablement. + +Evidence (local code): +- `muon-iframe/fluid-render-graph.js` (pipeline and FBO management) +- `muon-iframe/stemengine-router.js` (style lifecycle + audio feature push) + +Reference inferred characteristics: +- Canonical GPU fluid simulation style with strong interactive splat behavior and stable continuous flow. + +Assessment: +- Architecture match is strong. +- Main gap is operational reliability in your current stack: observed `FBO incomplete: 36054` warnings and replay-cycle failures reduce real-world parity. + +Confidence: +- Local implementation confidence: High +- Reference internals confidence: Medium +- Similarity conclusion confidence: Medium-High + +### B) `tendrils` +Local verified facts: +- Two-program design (`update` + `blit`) with framebuffer feedback loop. +- Motion built from angle/radius flow fields + hash noise + trail decay. +- Audio mapping primarily modulates flow rate, turbulence, color injection, and decay. +- No explicit particle-buffer architecture exposed; behavior emerges from texture feedback math. + +Evidence (local code): +- `muon-iframe/tendrils-render-graph.js` + +Reference inferred characteristics: +- Public text describes interacting particle system with advection-like behavior, optional optical flow/camera/mic interactions, rich controls and presets. + +Assessment: +- The local mode captures the “organic tendril motion” aesthetic. +- It appears materially simplified versus the broader interaction/control ecosystem and potentially deeper particle mechanics of the reference. + +Confidence: +- Local implementation confidence: High +- Reference internals confidence: Medium-Low +- Similarity conclusion confidence: Medium + +### C) `spirit` +Local verified facts: +- Two-pass feedback renderer with atmospheric layering: + - fbm-like noise term, + - radial halo decay, + - strand/pulse functions, + - additive glow and nonlinear output shaping. +- Audio mapping influences drift, fade persistence, pulse intensity, and color blend. + +Evidence (local code): +- `muon-iframe/spirit-render-graph.js` + +Reference inferred characteristics: +- Highly polished ethereal motion piece with strong artistic direction and likely richer scene dynamics. + +Assessment: +- The local mode appears designed as a stylistic homage (ethereal layered glow) rather than a strict algorithmic clone. +- Without source-level access to reference internals, parity can only be judged at visual-language level. + +Confidence: +- Local implementation confidence: High +- Reference internals confidence: Low-Medium +- Similarity conclusion confidence: Medium-Low + +## Integration Context (Important for Reviewers) +The local renderers are not standalone art demos; they are embedded into Muon transport and controlled by host commands. + +Observed integration traits: +- Router-driven style switching and lifecycle disposal/instantiation. +- Host-driven audio feature push throttled to iframe RAF cadence. +- Host-driven pause/play/seek/rate semantics through injected audio engine. +- Baseline WebGL extension checks in bootstrap layer. + +Implication: +- Even when visual math resembles a reference demo, behavior can diverge due to transport timing, iframe lifecycle, and browser/driver extension availability. + +## Known Gaps and Risks +1. Reference internals are partly opaque (minified/demo-only), so comparison is inference-based, not source-equivalence. +2. `fluid` currently shows environment-specific fragility (`FBO incomplete: 36054` warnings observed in session), impacting practical parity. +3. `tendrils`/`spirit` appear intentionally compact; they likely trade breadth/depth of original interactivity for predictable embedding in Muon. + +## Third-Party Review Conclusion +- If the review criterion is “same algorithm/system as originals”: not demonstrated for `tendrils` or `spirit`; only partial for `fluid` (closest). +- If criterion is “same visual family integrated into Muon with audio transport”: yes, with `fluid` currently hindered by reliability caveats. +- Recommended interpretation: these are reference-inspired implementations with varying structural closeness, not literal ports. + +## Suggested Evidence Bundle for Auditors +Include these files in review packets: +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` diff --git a/docs/muon-mode-status/2026-04-28-reviewer-checklist.md b/docs/muon-mode-status/2026-04-28-reviewer-checklist.md new file mode 100644 index 0000000..4eaa326 --- /dev/null +++ b/docs/muon-mode-status/2026-04-28-reviewer-checklist.md @@ -0,0 +1,152 @@ +# Muon Reviewer Checklist + +Date: 2026-04-28 +Purpose: quick third-party parity and functionality review for Muon `fluid`, `tendrils`, and `spirit` modes. + +Related documents: +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` +- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` + +## How To Use +For each mode, mark: +- PASS: clearly meets the criterion +- PARTIAL: present but degraded, unstable, or notably simplified +- FAIL: absent, broken, or materially unlike expected behavior +- N/A: not exposed on the reviewed surface + +Recommended review surfaces: +- Track Detail preview +- Visualizer Lab +- Any project/embed surface exposing Muon controls + +## Reviewer Metadata +- Reviewer: +- Date: +- Browser / version: +- GPU / driver: +- Surface tested: +- Audio source used: +- Quality tier tested: + +## Section 1: Basic Functionality + +### Mode availability +- `fluid` is selectable +- `tendrils` is selectable +- `spirit` is selectable +- Mode switch does not visibly break iframe/canvas + +### Playback / transport +- First play starts visual motion +- Pause stops or meaningfully freezes progression +- Replay after pause resumes visual motion +- Seek while playing preserves visual responsiveness +- Mode remains reactive for at least two play cycles + +### Stability +- No blank canvas on mode switch +- No persistent frozen frame after replay +- No repeated fatal errors in console +- No WebGL context loss during normal use + +## Section 2: Visual Parity vs Reference + +### `fluid` +Expected reference family: +- Fluid-like motion with continuous advection +- Visible diffusion / swirling / dye transport feel +- Stronger impulse on beats or major peaks +- Cohesive full-frame flow rather than isolated sprite effects + +Review marks: +- Flow behavior resembles fluid simulation +- Beat/bass energy visibly changes motion intensity +- Color field behaves like transported dye, not only flashing overlays +- Motion remains continuous across replay cycles +- Presentation feels structurally close to the PavelDoGreat reference + +Overall `fluid` score: PASS / PARTIAL / FAIL +Notes: + +### `tendrils` +Expected reference family: +- Organic line or filament structures +- Emergent motion with feedback / trace persistence +- Sense of interconnected or fibrous forms +- More than a simple radial pulse or stock waveform + +Review marks: +- Visual output forms tendril/filament-like structures +- Trails persist and evolve over time +- Motion feels emergent rather than strictly canned +- Audio affects density, color, or movement in a noticeable way +- Presentation feels recognizably related to the Tendrils reference family + +Overall `tendrils` score: PASS / PARTIAL / FAIL +Notes: + +### `spirit` +Expected reference family: +- Ethereal / atmospheric glow language +- Layered soft motion rather than sharp geometric tracing +- Strong sense of aura, halo, strand, smoke, or apparition-like motion +- Audio-linked pulse or drift without collapsing into a basic particle burst + +Review marks: +- Visual output feels atmospheric / spectral / ethereal +- Glow and layering read clearly on screen +- Audio changes drift, pulse, or luminosity in a visible way +- Motion remains smooth across play / pause / replay +- Presentation feels recognizably related to the Spirit reference family + +Overall `spirit` score: PASS / PARTIAL / FAIL +Notes: + +## Section 3: Surface Matrix Check +Mark each surface: + +### Track Detail +- `fluid`: PASS / PARTIAL / FAIL / N/A +- `tendrils`: PASS / PARTIAL / FAIL / N/A +- `spirit`: PASS / PARTIAL / FAIL / N/A +Notes: + +### Visualizer Lab +- `fluid`: PASS / PARTIAL / FAIL / N/A +- `tendrils`: PASS / PARTIAL / FAIL / N/A +- `spirit`: PASS / PARTIAL / FAIL / N/A +Notes: + +### Project / embed surface +- `fluid`: PASS / PARTIAL / FAIL / N/A +- `tendrils`: PASS / PARTIAL / FAIL / N/A +- `spirit`: PASS / PARTIAL / FAIL / N/A +Notes: + +## Section 4: Red Flags +Check any that apply: +- Replay fails only on some modes +- Mode works on first play but freezes on second play +- Console shows framebuffer / FBO warnings +- Visuals appear materially simpler than reference expectations +- Surface exposes controls but is not actually testable +- Behavior differs substantially between surfaces + +## Section 5: Final Verdict +Use this rubric: +- PASS: mode is stable and recognizably in the same reference family +- PARTIAL: mode works but has obvious reliability, fidelity, or surface gaps +- FAIL: mode is broken or not meaningfully comparable to its reference family + +Final verdict by mode: +- `fluid`: +- `tendrils`: +- `spirit`: + +Overall release recommendation: +- APPROVE for review +- APPROVE WITH CAVEATS +- DO NOT APPROVE + +Summary comments: diff --git a/docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md b/docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md new file mode 100644 index 0000000..42b5be3 --- /dev/null +++ b/docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md @@ -0,0 +1,65 @@ +# Muon Surface Functionality Matrix + +Date: 2026-04-28 + +Surfaces covered: +- Track Detail preview (`/tracks/:trackId`) +- Visualizer Lab (`/visualizer-lab`) +- Project page (`/projects/:projectId` on webui host) + +## Legend +- PASS: verified reactive in current session +- FAIL: reproducible non-reactive behavior in current session +- BLOCKED: surface/control exists but cannot execute mode replay test without additional input/state +- N/A: control path for this Muon mode test is not exposed on this surface + +## Matrix (Current) + +| Surface | default | fluid | tendrils | spirit | Notes | +|---|---|---|---|---|---| +| Track Detail preview | PASS | FAIL | PASS | PASS | `fluid` fails on cycle 2 replay (non-reactive), others reactive in same run | +| Visualizer Lab | BLOCKED | BLOCKED | BLOCKED | BLOCKED | Style combobox is present, but song is not loaded (`Song: None selected`), play/reset disabled | +| Project page embed | N/A | N/A | N/A | N/A | No Muon replay controls or mode selector exposed in this project surface snapshot | + +## Evidence Snapshot + +### 1) Track Detail (`http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb`) +Single sweep results (cycle1/cycle2): +- default: reactive / reactive +- fluid: reactive / non-reactive +- tendrils: reactive / reactive +- spirit: reactive / reactive + +Representative diffs: +- default: c1=20141, c2=23362 +- fluid: c1=1552, c2=0 +- tendrils: c1=43459, c2=45607 +- spirit: c1=9952, c2=9313 + +Known caveat in session: +- intermittent stream/network noise still appears in browser events (`ERR_ABORTED`, occasional `502`), but does not explain the mode-specific fluid replay failure by itself. + +### 2) Visualizer Lab (`http://work:3001/visualizer-lab`) +Observed: +- Muon iframe present +- Mode style options are available via `Style` combobox (`default`, `fluid`, `tendrils`, `spirit`) +- `Song: None selected` +- `Play preview` is disabled + +Implication: +- per-mode replay reactivity cannot be validated until an audio source is loaded. + +### 3) Project page (`http://work:5174/projects/e98f2b8a-0cad-4b4e-b33e-1ca2c8c87a0a`) +Observed in current snapshot: +- No Muon iframe preview controls for replay testing +- No mode buttons (`default`, `fluid`, `tendrils`, `spirit`) exposed +- No preview play/pause controls exposed for this Muon test path + +Implication: +- Mode-by-mode replay matrix is N/A on this surface as currently rendered. + +## Delta vs Prior Note +See prior Track Detail-focused note for earlier run context: +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` + +This file extends that status into a cross-surface matrix and explicitly marks where tests are blocked or not applicable. diff --git a/docs/muon-mode-status/README.md b/docs/muon-mode-status/README.md new file mode 100644 index 0000000..60a9c21 --- /dev/null +++ b/docs/muon-mode-status/README.md @@ -0,0 +1,87 @@ +# Muon Mode Status Review Index + +Date: 2026-04-28 +Purpose: single entry point for current Muon mode functionality, surface coverage, reference comparison, and third-party review. + +## Document Set + +### 1. Current Per-Mode Functionality +File: +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` + +Use this first for: +- Track Detail preview mode-by-mode status +- Current replay behavior by mode +- Quick snapshot of what is working vs buggy + +Current headline: +- `default`: functional with caveats +- `fluid`: reproducible replay bug +- `tendrils`: functional in tested session, watchlist +- `spirit`: functional in tested session, watchlist + +### 2. Cross-Surface Matrix +File: +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` + +Use this for: +- Comparing Track Detail, Visualizer Lab, and project/embed surfaces +- Seeing which surfaces are testable, blocked, or N/A +- Understanding where mode controls are or are not exposed + +Current headline: +- Track Detail is testable and currently shows `fluid` replay failure +- Visualizer Lab is blocked without a loaded song +- Project page does not currently expose this Muon replay test path + +### 3. Reference Comparison for Third-Party Review +File: +- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` + +Use this for: +- Evaluating how local `fluid`, `tendrils`, and `spirit` compare to reference demos +- Distinguishing verified local facts from inferred reference traits +- Reviewing confidence levels and known uncertainty + +Current headline: +- `fluid` is the closest structural match to its reference +- `tendrils` and `spirit` are better described as reference-inspired approximations than literal ports + +### 4. Reviewer Checklist +File: +- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md` + +Use this for: +- PASS / PARTIAL / FAIL / N/A scoring +- Consistent third-party review rubric +- Capturing final approval / caveat decisions + +## Recommended Reading Order +1. `2026-04-27-current-per-mode-functionality.md` +2. `2026-04-28-surface-functionality-matrix.md` +3. `2026-04-28-reference-comparison-third-party-review.md` +4. `2026-04-28-reviewer-checklist.md` + +## Reviewer Short Summary +If a reviewer reads only one page first, start with: +- `2026-04-28-reference-comparison-third-party-review.md` + +If a reviewer wants only operational status first, start with: +- `2026-04-27-current-per-mode-functionality.md` + +If a reviewer is validating reproducibility across UI surfaces, start with: +- `2026-04-28-surface-functionality-matrix.md` + +## Current Overall Assessment +- `fluid`: closest to reference in structure, but currently weakened by replay/FBO reliability issues +- `tendrils`: visually in-family, but likely a simplified embedded variant rather than full reference parity +- `spirit`: visually in-family, but should be treated as stylistic approximation unless stronger source-level evidence emerges + +## Evidence Scope Notice +This folder contains a mix of: +- direct source-code review, +- live browser observation, +- runtime probing, +- inference from public reference pages. + +It should be treated as a technical review package, not a claim of source-equivalent implementation. diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/README.md b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/README.md new file mode 100644 index 0000000..545e16f --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/README.md @@ -0,0 +1,100 @@ +# Muon Mode Status Review Index + +Date: 2026-04-28 +Purpose: single entry point for current Muon mode functionality, surface coverage, reference comparison, and third-party review. + +## Document Set + +### 1. Current Per-Mode Functionality +File: +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` + +Use this first for: +- Track Detail preview mode-by-mode status +- Current replay behavior by mode +- Quick snapshot of what is working vs buggy + +Current headline: +- `default`: functional with caveats +- `fluid`: reproducible replay bug +- `tendrils`: functional in tested session, watchlist +- `spirit`: functional in tested session, watchlist + +### 2. Cross-Surface Matrix +File: +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` + +Use this for: +- Comparing Track Detail, Visualizer Lab, and project/embed surfaces +- Seeing which surfaces are testable, blocked, or N/A +- Understanding where mode controls are or are not exposed + +Current headline: +- Track Detail is testable and currently shows `fluid` replay failure +- Visualizer Lab is blocked without a loaded song +- Project page does not currently expose this Muon replay test path + +### 3. Reference Comparison for Third-Party Review +File: +- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` + +Use this for: +- Evaluating how local `fluid`, `tendrils`, and `spirit` compare to reference demos +- Distinguishing verified local facts from inferred reference traits +- Reviewing confidence levels and known uncertainty + +Current headline: +- `fluid` is the closest structural match to its reference +- `tendrils` and `spirit` are better described as reference-inspired approximations than literal ports + +### 4. Reviewer Checklist +File: +- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md` + +Use this for: +- PASS / PARTIAL / FAIL / N/A scoring +- Consistent third-party review rubric +- Capturing final approval / caveat decisions + +### 5. Local Iframe Architecture Plan +File: +- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md` + +Use this for: +- The implementation path for moving non-default styles onto a local Muon-like iframe flow +- Phase-by-phase execution guidance +- Host/iframe touch points, risks, and validation gates + +## Recommended Reading Order +1. `2026-04-27-current-per-mode-functionality.md` +2. `2026-04-28-surface-functionality-matrix.md` +3. `2026-04-28-reference-comparison-third-party-review.md` +4. `2026-04-28-reviewer-checklist.md` +5. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md` + +## Reviewer Short Summary +If a reviewer reads only one page first, start with: +- `2026-04-28-reference-comparison-third-party-review.md` + +If a reviewer wants only operational status first, start with: +- `2026-04-27-current-per-mode-functionality.md` + +If a reviewer is validating reproducibility across UI surfaces, start with: +- `2026-04-28-surface-functionality-matrix.md` + +If a reviewer is planning implementation work from these findings, continue with: +- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md` + +## Current Overall Assessment +- `fluid`: closest to reference in structure, but currently weakened by replay/FBO reliability issues +- `tendrils`: visually in-family, but likely a simplified embedded variant rather than full reference parity +- `spirit`: visually in-family, but should be treated as stylistic approximation unless stronger source-level evidence emerges + +## Evidence Scope Notice +This folder contains a mix of: +- direct source-code review, +- live browser observation, +- runtime probing, +- inference from public reference pages. + +It should be treated as a technical review package, not a claim of source-equivalent implementation. diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-27-current-per-mode-functionality.md b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-27-current-per-mode-functionality.md new file mode 100644 index 0000000..19f96f7 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-27-current-per-mode-functionality.md @@ -0,0 +1,78 @@ +# Muon Visualizer Current Per-Mode Functionality + +Date: 2026-04-27 +Environment: StemEngine track preview (http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb) +Mode set under test: `default`, `fluid`, `tendrils`, `spirit` + +## Scope +This document captures the CURRENT observed behavior for Muon style modes in Track Detail preview, with focus on repeat-play reactivity. + +Reactivity definition used here: +- cycle 1 reactive: visual output changes after first Play preview +- cycle 2 reactive: visual output still changes after Pause -> Play replay + +## Summary +- `default`: functional with caveats +- `fluid`: reproducible bug on replay (cycle 2 non-reactive) +- `tendrils`: functional in current sweep, but keep under watch (user-reported instability exists) +- `spirit`: functional in current sweep, but keep under watch (user-reported instability exists) + +## Per-Mode Status + +### default +Status: Functional with caveats +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: reactive (3/3 runs) +Caveats: +- Intermittent stream-layer noise still appears in session logs/events (`ERR_ABORTED`, occasional `502`), but did not break reactivity in this sweep. + +### fluid +Status: Buggy (reproducible) +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: NOT reactive (3/3 runs) +Evidence: +- Motion diff for cycle 2 repeatedly dropped to `0` +- Console repeatedly logs: `[StemEngine][Fluid] FBO incomplete: 36054` +Impact: +- Replay reliability is broken for fluid mode under current build/runtime combination. + +### tendrils +Status: Functional in current session (watchlist) +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: reactive (3/3 runs) +Caveat: +- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep. + +### spirit +Status: Functional in current session (watchlist) +Observed behavior: +- Cycle 1: reactive (3/3 runs) +- Cycle 2: reactive (3/3 runs) +Caveat: +- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep. + +## Raw Sweep Snapshot (Automated) +Run 1: +- default: c1 reactive, c2 reactive +- fluid: c1 reactive, c2 non-reactive +- tendrils: c1 reactive, c2 reactive +- spirit: c1 reactive, c2 reactive + +Run 2: +- default: c1 reactive, c2 reactive +- fluid: c1 reactive, c2 non-reactive +- tendrils: c1 reactive, c2 reactive +- spirit: c1 reactive, c2 reactive + +Run 3: +- default: c1 reactive, c2 reactive +- fluid: c1 reactive, c2 non-reactive +- tendrils: c1 reactive, c2 reactive +- spirit: c1 reactive, c2 reactive + +## Notes +- This report is intentionally point-in-time. Re-run after renderer/transport changes to keep this file current. +- If you want, add a companion matrix file for Visualizer Lab page and project-page embed to compare behavior across surfaces. diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-export-bundle-manifest.md b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-export-bundle-manifest.md new file mode 100644 index 0000000..a9e6653 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-export-bundle-manifest.md @@ -0,0 +1,104 @@ +# Muon Review Export Bundle Manifest + +Date: 2026-04-28 +Purpose: frozen manifest for external sharing of the Muon mode review package. + +## Bundle Name +Suggested bundle label: +- `muon-mode-review-package-2026-04-28` + +## Bundle Scope +This export bundle is intended for third-party reviewers who need: +- current operational status by mode, +- surface-by-surface testability notes, +- reference-comparison analysis, +- a reviewer scoring rubric, +- a single index file for navigation. + +## Files To Include + +### Core review docs +1. `docs/muon-mode-status/README.md` + - Entry point and reading order. + +2. `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` + - Point-in-time Track Detail mode functionality and replay observations. + +3. `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` + - Surface matrix across Track Detail, Visualizer Lab, and project/embed views. + +4. `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` + - Reference comparison for `fluid`, `tendrils`, and `spirit`. + +5. `docs/muon-mode-status/2026-04-28-reviewer-checklist.md` + - Third-party scoring and signoff rubric. + +6. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md` + - Implementation plan for moving non-default styles to a local iframe visualizer runtime. + +### Optional supporting source files +Include these if the reviewer needs implementation-level evidence: +7. `muon-iframe/fluid-render-graph.js` +8. `muon-iframe/tendrils-render-graph.js` +9. `muon-iframe/spirit-render-graph.js` +10. `muon-iframe/stemengine-router.js` +11. `muon-iframe/stemengine-bootstrap.js` + +## Minimal Bundle +If the recipient should receive docs only, export: +- `docs/muon-mode-status/README.md` +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` +- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` +- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md` +- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md` + +## Full Evidence Bundle +If the recipient should receive both review docs and implementation evidence, export: +- all files from the Minimal Bundle +- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md` +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` + +## Suggested Folder Layout For Sharing +```text +muon-mode-review-package-2026-04-28/ + README.md + docs/ + 2026-04-27-current-per-mode-functionality.md + 2026-04-28-export-bundle-manifest.md + 2026-04-28-surface-functionality-matrix.md + 2026-04-28-reference-comparison-third-party-review.md + 2026-04-28-reviewer-checklist.md + plans/ + 2026-04-28_local-iframe-visualizer-architecture-plan.md + source-evidence/ + fluid-render-graph.js + tendrils-render-graph.js + spirit-render-graph.js + stemengine-router.js + stemengine-bootstrap.js +``` + +## Export Notes +- This manifest freezes the intended review set as of 2026-04-28. +- The bundle is evidence-oriented, not a claim of source-equivalent parity with external references. +- Reference-comparison conclusions include inference and confidence levels; reviewers should read those caveats before scoring fidelity. + +## Reviewer Guidance +Recommended reading sequence inside the exported bundle: +1. `README.md` +2. `2026-04-28-reference-comparison-third-party-review.md` +3. `2026-04-27-current-per-mode-functionality.md` +4. `2026-04-28-surface-functionality-matrix.md` +5. `2026-04-28-reviewer-checklist.md` +6. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md` + +## Current Package Status +As of this manifest: +- `fluid` remains the main concern because of replay/FBO reliability issues. +- `tendrils` and `spirit` are documented as in-family approximations, not proven literal ports. +- Visualizer Lab and project/embed surfaces remain limited or non-testable for this exact replay matrix without additional setup. diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-reference-comparison-third-party-review.md b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-reference-comparison-third-party-review.md new file mode 100644 index 0000000..c6aa9e2 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-reference-comparison-third-party-review.md @@ -0,0 +1,141 @@ +# Muon Mode Comparison vs Reference Demos (Third-Party Review) + +Date: 2026-04-28 +Prepared for: external/third-party technical review +Scope: infer how local `fluid`, `tendrils`, and `spirit` modes compare to their likely inspiration references. + +References under review: +- https://paveldogreat.github.io/WebGL-Fluid-Simulation/ +- https://keeffeoghan.github.io/work/tendrils/ (redirects to `epok.tech` content) +- https://edankwan.com/experiments/the-spirit/ + +Local implementation inputs reviewed: +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` + +## Method Notes +This review intentionally separates: +- verified local facts (from readable source files), and +- inferred reference characteristics (from live pages + publicly visible descriptions + agent-assisted analysis). + +A `ui-designer` subagent was used to produce an independent reference summary, then cross-checked against local code. + +## Executive Summary +1. `fluid` in this fork is structurally the closest to its reference target (PavelDoGreat fluid simulation), because it implements a recognizable multi-pass fluid pipeline (splat, divergence, pressure Jacobi, gradient subtract, advection, bloom). +2. `tendrils` and `spirit` in this fork are stylistic approximations, not faithful reproductions of the original interactive systems. They are compact shader-feedback variants optimized for integration into Muon’s transport/audio stack. +3. The local `fluid` path currently has a practical stability/replay issue in your environment (`FBO incomplete: 36054` warnings and cycle-2 non-reactivity), which materially affects parity with the expected “real thing” feel. + +## Comparison Matrix + +| Mode | Local Implementation (verified) | Reference “Real Thing” (inferred) | Similarity Assessment | +|---|---|---|---| +| fluid | Full-screen multipass FBO fluid pipeline with pressure solve, advection, bloom, and audio splat injection | Interactive Navier-Stokes style GPU fluid sim with user splats and polished post FX | High structural similarity; medium experiential similarity due to current stability issues and integration constraints | +| tendrils | Feedback texture + procedural flow/noise line structures in one update shader, audio-driven decay/flow | Emergent particle-field ecosystem with richer control surface, presets, and interaction channels (keyboard/camera/mic) | Medium visual similarity; low-to-medium system similarity | +| spirit | Layered atmospheric feedback shader with fbm/noise, halo/strand/pulse composition, audio-driven fade | Highly polished ethereal particle/trail experience with strong cinematic motion language | Medium stylistic similarity; low implementation-parity confidence | + +## Per-Mode Detailed Assessment + +### A) `fluid` +Local verified facts: +- Uses explicit fluid-solver style passes: + - curl, divergence, pressure Jacobi, gradient subtract, velocity advection, dye advection. +- Uses ping-pong FBOs for velocity/dye/pressure and optional bloom chain by quality tier. +- Audio coupling includes bass/sub/treble/rms/beat/beatPhase/drumIntensity mapped to splat radius/force/color. +- Quality tier controls pressure iterations and bloom enablement. + +Evidence (local code): +- `muon-iframe/fluid-render-graph.js` (pipeline and FBO management) +- `muon-iframe/stemengine-router.js` (style lifecycle + audio feature push) + +Reference inferred characteristics: +- Canonical GPU fluid simulation style with strong interactive splat behavior and stable continuous flow. + +Assessment: +- Architecture match is strong. +- Main gap is operational reliability in your current stack: observed `FBO incomplete: 36054` warnings and replay-cycle failures reduce real-world parity. + +Confidence: +- Local implementation confidence: High +- Reference internals confidence: Medium +- Similarity conclusion confidence: Medium-High + +### B) `tendrils` +Local verified facts: +- Two-program design (`update` + `blit`) with framebuffer feedback loop. +- Motion built from angle/radius flow fields + hash noise + trail decay. +- Audio mapping primarily modulates flow rate, turbulence, color injection, and decay. +- No explicit particle-buffer architecture exposed; behavior emerges from texture feedback math. + +Evidence (local code): +- `muon-iframe/tendrils-render-graph.js` + +Reference inferred characteristics: +- Public text describes interacting particle system with advection-like behavior, optional optical flow/camera/mic interactions, rich controls and presets. + +Assessment: +- The local mode captures the “organic tendril motion” aesthetic. +- It appears materially simplified versus the broader interaction/control ecosystem and potentially deeper particle mechanics of the reference. + +Confidence: +- Local implementation confidence: High +- Reference internals confidence: Medium-Low +- Similarity conclusion confidence: Medium + +### C) `spirit` +Local verified facts: +- Two-pass feedback renderer with atmospheric layering: + - fbm-like noise term, + - radial halo decay, + - strand/pulse functions, + - additive glow and nonlinear output shaping. +- Audio mapping influences drift, fade persistence, pulse intensity, and color blend. + +Evidence (local code): +- `muon-iframe/spirit-render-graph.js` + +Reference inferred characteristics: +- Highly polished ethereal motion piece with strong artistic direction and likely richer scene dynamics. + +Assessment: +- The local mode appears designed as a stylistic homage (ethereal layered glow) rather than a strict algorithmic clone. +- Without source-level access to reference internals, parity can only be judged at visual-language level. + +Confidence: +- Local implementation confidence: High +- Reference internals confidence: Low-Medium +- Similarity conclusion confidence: Medium-Low + +## Integration Context (Important for Reviewers) +The local renderers are not standalone art demos; they are embedded into Muon transport and controlled by host commands. + +Observed integration traits: +- Router-driven style switching and lifecycle disposal/instantiation. +- Host-driven audio feature push throttled to iframe RAF cadence. +- Host-driven pause/play/seek/rate semantics through injected audio engine. +- Baseline WebGL extension checks in bootstrap layer. + +Implication: +- Even when visual math resembles a reference demo, behavior can diverge due to transport timing, iframe lifecycle, and browser/driver extension availability. + +## Known Gaps and Risks +1. Reference internals are partly opaque (minified/demo-only), so comparison is inference-based, not source-equivalence. +2. `fluid` currently shows environment-specific fragility (`FBO incomplete: 36054` warnings observed in session), impacting practical parity. +3. `tendrils`/`spirit` appear intentionally compact; they likely trade breadth/depth of original interactivity for predictable embedding in Muon. + +## Third-Party Review Conclusion +- If the review criterion is “same algorithm/system as originals”: not demonstrated for `tendrils` or `spirit`; only partial for `fluid` (closest). +- If criterion is “same visual family integrated into Muon with audio transport”: yes, with `fluid` currently hindered by reliability caveats. +- Recommended interpretation: these are reference-inspired implementations with varying structural closeness, not literal ports. + +## Suggested Evidence Bundle for Auditors +Include these files in review packets: +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-reviewer-checklist.md b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-reviewer-checklist.md new file mode 100644 index 0000000..4eaa326 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-reviewer-checklist.md @@ -0,0 +1,152 @@ +# Muon Reviewer Checklist + +Date: 2026-04-28 +Purpose: quick third-party parity and functionality review for Muon `fluid`, `tendrils`, and `spirit` modes. + +Related documents: +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` +- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md` +- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md` + +## How To Use +For each mode, mark: +- PASS: clearly meets the criterion +- PARTIAL: present but degraded, unstable, or notably simplified +- FAIL: absent, broken, or materially unlike expected behavior +- N/A: not exposed on the reviewed surface + +Recommended review surfaces: +- Track Detail preview +- Visualizer Lab +- Any project/embed surface exposing Muon controls + +## Reviewer Metadata +- Reviewer: +- Date: +- Browser / version: +- GPU / driver: +- Surface tested: +- Audio source used: +- Quality tier tested: + +## Section 1: Basic Functionality + +### Mode availability +- `fluid` is selectable +- `tendrils` is selectable +- `spirit` is selectable +- Mode switch does not visibly break iframe/canvas + +### Playback / transport +- First play starts visual motion +- Pause stops or meaningfully freezes progression +- Replay after pause resumes visual motion +- Seek while playing preserves visual responsiveness +- Mode remains reactive for at least two play cycles + +### Stability +- No blank canvas on mode switch +- No persistent frozen frame after replay +- No repeated fatal errors in console +- No WebGL context loss during normal use + +## Section 2: Visual Parity vs Reference + +### `fluid` +Expected reference family: +- Fluid-like motion with continuous advection +- Visible diffusion / swirling / dye transport feel +- Stronger impulse on beats or major peaks +- Cohesive full-frame flow rather than isolated sprite effects + +Review marks: +- Flow behavior resembles fluid simulation +- Beat/bass energy visibly changes motion intensity +- Color field behaves like transported dye, not only flashing overlays +- Motion remains continuous across replay cycles +- Presentation feels structurally close to the PavelDoGreat reference + +Overall `fluid` score: PASS / PARTIAL / FAIL +Notes: + +### `tendrils` +Expected reference family: +- Organic line or filament structures +- Emergent motion with feedback / trace persistence +- Sense of interconnected or fibrous forms +- More than a simple radial pulse or stock waveform + +Review marks: +- Visual output forms tendril/filament-like structures +- Trails persist and evolve over time +- Motion feels emergent rather than strictly canned +- Audio affects density, color, or movement in a noticeable way +- Presentation feels recognizably related to the Tendrils reference family + +Overall `tendrils` score: PASS / PARTIAL / FAIL +Notes: + +### `spirit` +Expected reference family: +- Ethereal / atmospheric glow language +- Layered soft motion rather than sharp geometric tracing +- Strong sense of aura, halo, strand, smoke, or apparition-like motion +- Audio-linked pulse or drift without collapsing into a basic particle burst + +Review marks: +- Visual output feels atmospheric / spectral / ethereal +- Glow and layering read clearly on screen +- Audio changes drift, pulse, or luminosity in a visible way +- Motion remains smooth across play / pause / replay +- Presentation feels recognizably related to the Spirit reference family + +Overall `spirit` score: PASS / PARTIAL / FAIL +Notes: + +## Section 3: Surface Matrix Check +Mark each surface: + +### Track Detail +- `fluid`: PASS / PARTIAL / FAIL / N/A +- `tendrils`: PASS / PARTIAL / FAIL / N/A +- `spirit`: PASS / PARTIAL / FAIL / N/A +Notes: + +### Visualizer Lab +- `fluid`: PASS / PARTIAL / FAIL / N/A +- `tendrils`: PASS / PARTIAL / FAIL / N/A +- `spirit`: PASS / PARTIAL / FAIL / N/A +Notes: + +### Project / embed surface +- `fluid`: PASS / PARTIAL / FAIL / N/A +- `tendrils`: PASS / PARTIAL / FAIL / N/A +- `spirit`: PASS / PARTIAL / FAIL / N/A +Notes: + +## Section 4: Red Flags +Check any that apply: +- Replay fails only on some modes +- Mode works on first play but freezes on second play +- Console shows framebuffer / FBO warnings +- Visuals appear materially simpler than reference expectations +- Surface exposes controls but is not actually testable +- Behavior differs substantially between surfaces + +## Section 5: Final Verdict +Use this rubric: +- PASS: mode is stable and recognizably in the same reference family +- PARTIAL: mode works but has obvious reliability, fidelity, or surface gaps +- FAIL: mode is broken or not meaningfully comparable to its reference family + +Final verdict by mode: +- `fluid`: +- `tendrils`: +- `spirit`: + +Overall release recommendation: +- APPROVE for review +- APPROVE WITH CAVEATS +- DO NOT APPROVE + +Summary comments: diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-surface-functionality-matrix.md b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-surface-functionality-matrix.md new file mode 100644 index 0000000..42b5be3 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/2026-04-28-surface-functionality-matrix.md @@ -0,0 +1,65 @@ +# Muon Surface Functionality Matrix + +Date: 2026-04-28 + +Surfaces covered: +- Track Detail preview (`/tracks/:trackId`) +- Visualizer Lab (`/visualizer-lab`) +- Project page (`/projects/:projectId` on webui host) + +## Legend +- PASS: verified reactive in current session +- FAIL: reproducible non-reactive behavior in current session +- BLOCKED: surface/control exists but cannot execute mode replay test without additional input/state +- N/A: control path for this Muon mode test is not exposed on this surface + +## Matrix (Current) + +| Surface | default | fluid | tendrils | spirit | Notes | +|---|---|---|---|---|---| +| Track Detail preview | PASS | FAIL | PASS | PASS | `fluid` fails on cycle 2 replay (non-reactive), others reactive in same run | +| Visualizer Lab | BLOCKED | BLOCKED | BLOCKED | BLOCKED | Style combobox is present, but song is not loaded (`Song: None selected`), play/reset disabled | +| Project page embed | N/A | N/A | N/A | N/A | No Muon replay controls or mode selector exposed in this project surface snapshot | + +## Evidence Snapshot + +### 1) Track Detail (`http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb`) +Single sweep results (cycle1/cycle2): +- default: reactive / reactive +- fluid: reactive / non-reactive +- tendrils: reactive / reactive +- spirit: reactive / reactive + +Representative diffs: +- default: c1=20141, c2=23362 +- fluid: c1=1552, c2=0 +- tendrils: c1=43459, c2=45607 +- spirit: c1=9952, c2=9313 + +Known caveat in session: +- intermittent stream/network noise still appears in browser events (`ERR_ABORTED`, occasional `502`), but does not explain the mode-specific fluid replay failure by itself. + +### 2) Visualizer Lab (`http://work:3001/visualizer-lab`) +Observed: +- Muon iframe present +- Mode style options are available via `Style` combobox (`default`, `fluid`, `tendrils`, `spirit`) +- `Song: None selected` +- `Play preview` is disabled + +Implication: +- per-mode replay reactivity cannot be validated until an audio source is loaded. + +### 3) Project page (`http://work:5174/projects/e98f2b8a-0cad-4b4e-b33e-1ca2c8c87a0a`) +Observed in current snapshot: +- No Muon iframe preview controls for replay testing +- No mode buttons (`default`, `fluid`, `tendrils`, `spirit`) exposed +- No preview play/pause controls exposed for this Muon test path + +Implication: +- Mode-by-mode replay matrix is N/A on this surface as currently rendered. + +## Delta vs Prior Note +See prior Track Detail-focused note for earlier run context: +- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md` + +This file extends that status into a cross-surface matrix and explicitly marks where tests are blocked or not applicable. diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md new file mode 100644 index 0000000..f598e4f --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md @@ -0,0 +1,346 @@ +# Local Iframe Visualizer Architecture Plan + +Date: 2026-04-28 +Status: Draft implementation plan +Owner: StemEngine frontend / visualizer pipeline + +## Problem Statement +The current attempt to approximate `fluid`, `tendrils`, and `spirit` inside the existing Muon iframe path did not deliver a reliable result. + +Observed issues: +- `default` mode is the only mode currently behaving close to the intended repeat-play iframe flow. +- `fluid` shows reproducible replay instability and framebuffer warnings. +- `tendrils` and `spirit` may look visually in-family, but are still bound to Muon-specific lifecycle assumptions that make them hard to validate and stabilize. + +## Goal +Use the same host-to-iframe transport model for every visualizer style, but move non-default styles onto a local iframe runtime that behaves like Muon from the host’s point of view. + +Target outcome: +- the host speaks one iframe transport contract, +- each style runs in an iframe-managed renderer lifecycle, +- replay / pause / seek / rate / resize are validated the same way for all styles, +- non-default styles are no longer dependent on Muon internals that were designed for the default renderer. + +## Decision Summary +Recommended architecture: +- Keep the existing Muon iframe provider for `default` for now. +- Build a local iframe visualizer runtime for `fluid`, `tendrils`, and `spirit`. +- Standardize the transport contract so the host treats both as iframe visualizer providers. + +Preferred shape: +- One shared local iframe shell with pluggable style engines. + +Fallback shape: +- One local iframe app per style. + +Recommendation: +- Use one shared local iframe shell unless style-specific boot constraints prove otherwise. + +## Why This Direction +Benefits: +- separates transport concerns from render math, +- removes style replay behavior from Muon private/default renderer assumptions, +- reduces duplicate host logic, +- gives one lifecycle model for all local styles, +- makes validation and third-party review clearer. + +## Scope +In scope: +- iframe transport contract standardization, +- host visualizer provider abstraction, +- local iframe shell runtime, +- local renderer plugin API, +- migration of `fluid`, `tendrils`, and `spirit` to local iframe shell, +- validation matrix across Track Detail, Visualizer Lab, and other exposed surfaces. + +Out of scope for first pass: +- replacing default Muon renderer, +- perfect fidelity to third-party reference demos, +- broad preset-authoring UX, +- camera/mic/advanced interaction parity with original external demos. + +## Current File Touch Points +### Host-side frontend +Likely primary touch points: +- `frontend/src/hooks/useMuonIframeTransport.ts` +- `frontend/src/components/VisualizerPreviewWidget.tsx` +- `frontend/src/components/LyricsLivePreview.tsx` +- `frontend/src/pages/VisualizerLabPage.tsx` +- `frontend/src/pages/TrackDetail.tsx` + +Likely new host-side additions: +- `frontend/src/visualizers/providers/*` +- `frontend/src/hooks/useIframeVisualizerTransport.ts` +- `frontend/src/types/iframeVisualizer.ts` + +### Existing Muon iframe runtime +Reference implementation / source of reusable logic: +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` +- `muon-iframe/stemengine-audio-engine.js` +- `muon-iframe/stemengine-feature-extractor.js` + +### Existing style renderers to migrate or adapt +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` + +### Likely new local iframe app area +Suggested location: +- `frontend/local-visualizer-iframe/` or +- `local-visualizer-iframe/` + +Recommendation: +- keep it separate from `muon-iframe/` so it is not build-time injected into Muon. + +## Target Architecture +### 1. Host visualizer provider layer +The host should target a provider abstraction, not Muon directly. + +Proposed provider model: +- provider id +- iframe URL +- supported styles +- capabilities +- command protocol version +- fallback behavior + +Example providers: +- `muon-default` +- `local-visualizer` + +### 2. Shared iframe transport contract +Commands from host to iframe: +- `init` +- `setStyle` +- `setPreset` +- `setQuality` +- `setPaused` +- `setViewport` +- `loadAudio` +- `transportPlay` +- `transportPause` +- `transportSeek` +- `transportRate` +- `pushAudioFeatures` +- `dispose` + +Events from iframe to host: +- `ready` +- `loaded` +- `capabilities` +- `state` +- `warning` +- `error` + +Required rules: +- host owns transport intent and source audio selection, +- iframe owns rendering lifecycle, +- seek while playing must not leave renderer visually paused, +- replay after pause must be deterministic, +- viewport updates must not require reload. + +### 3. Local iframe shell +Responsibilities: +- boot and emit `ready`, +- set up audio engine, +- set up feature extractor, +- receive commands from host, +- instantiate exactly one renderer at a time, +- dispose renderer cleanly on style switch or teardown, +- expose warnings/errors back to host. + +Shell should be style-agnostic. + +### 4. Renderer plugin API +Each style renderer should conform to the same API. + +Proposed renderer interface: +- `init(context)` +- `setViewport(viewport)` +- `setQuality(quality)` +- `setPreset(preset)` +- `setPaused(paused)` +- `pushFeatures(features)` +- `tick(time)` +- `dispose()` + +Renderer should not own: +- transport protocol, +- source audio loading, +- play/pause/seek intent, +- message routing. + +### 5. Migration model +Near-term routing: +- `default` -> existing Muon iframe provider +- `fluid` -> local iframe shell +- `tendrils` -> local iframe shell +- `spirit` -> local iframe shell + +Future option: +- move `default` under the same provider system later if beneficial. + +## Implementation Phases +## Phase 1: Freeze the contract +Deliverables: +- one transport spec document, +- one provider abstraction design, +- explicit replay/seek/resize semantics. + +Tasks: +- document current useful Muon messages, +- remove Muon-specific assumptions from the conceptual transport model, +- define one versioned protocol for all iframe visualizers. + +Exit criteria: +- host and iframe responsibilities are explicitly defined, +- no new style-specific transport behavior is added ad hoc. + +## Phase 2: Introduce provider abstraction on host +Deliverables: +- provider registry, +- host hook abstracted away from Muon naming, +- widgets/pages able to select a provider. + +Tasks: +- refactor `useMuonIframeTransport.ts` into provider-neutral transport hook, +- add provider config for default Muon and local iframe runtime, +- update consumers in preview surfaces. + +Exit criteria: +- Track Detail and Visualizer Lab can load an iframe provider without hard-coding Muon-only assumptions. + +## Phase 3: Build local iframe shell +Deliverables: +- local iframe app booting independently, +- postMessage router, +- audio engine and feature extractor wiring, +- renderer lifecycle manager. + +Tasks: +- clone only the reusable runtime concepts from current Muon integration, +- avoid build-time injection into third-party Muon source, +- add explicit renderer registration and disposal. + +Exit criteria: +- shell loads, emits ready, can accept audio load/play/pause/seek/viewport commands even with a placeholder renderer. + +## Phase 4: Migrate `fluid` +Why first: +- it is the closest structural match to its intended reference, +- it is currently the least reliable mode. + +Tasks: +- adapt current fluid renderer to the new renderer API, +- enforce init-time capability checks, +- surface explicit degrade/failure states when FBO support is incomplete, +- validate replay and resize in the new shell. + +Exit criteria: +- `fluid` passes two-cycle replay test in Track Detail, +- no silent frozen-canvas failure path, +- FBO failure produces explicit host-visible warning/error. + +## Phase 5: Migrate `tendrils` and `spirit` +Tasks: +- adapt both renderers to shared shell lifecycle, +- remove transport ownership assumptions from render graphs, +- validate style switch, replay, resize, and quality transitions. + +Exit criteria: +- both renderers pass same iframe lifecycle tests as `fluid`, +- no per-style transport branching on host. + +## Phase 6: Cross-surface validation and rollout +Tasks: +- validate Track Detail, +- validate Visualizer Lab with a loaded song path, +- validate any project/embed surface that exposes iframe preview, +- update docs and reviewer package with post-migration results. + +Exit criteria: +- the same style behaves consistently across supported surfaces, +- unsupported surfaces are explicitly documented. + +## Validation Matrix +Each migrated style must pass: +- iframe boot / ready +- audio load +- play cycle 1 +- pause +- replay cycle 2 +- seek while playing +- seek while paused +- rate change +- viewport resize +- quality change +- style switch away and back +- teardown / remount + +Required test surfaces: +- Track Detail +- Visualizer Lab +- any active project/embed preview surface + +## Key Risks +1. Host abstraction drift +Risk: +- partial refactor leaves old Muon-only codepaths mixed with provider abstraction. +Mitigation: +- centralize provider selection in one registry and one transport hook. + +2. Fluid hardware variability +Risk: +- local fluid renderer still fails on certain WebGL / extension combinations. +Mitigation: +- explicit capability detection and degrade behavior, not silent failure. + +3. Renderer lifecycle leaks +Risk: +- RAF loops, textures, or framebuffers survive style switches. +Mitigation: +- strict renderer API and disposable lifecycle manager in shell. + +4. Visualizer Lab remains blocked +Risk: +- shell exists but lab still cannot load audio cleanly. +Mitigation: +- define one supported sample-input path early in Phase 3. + +5. Fidelity pressure before lifecycle stability +Risk: +- development time is spent tuning visuals before replay and resize are correct. +Mitigation: +- block fidelity work until lifecycle acceptance criteria pass. + +## Deliverables Checklist +- transport spec doc +- provider registry +- provider-neutral iframe transport hook +- local iframe shell app +- renderer plugin API +- migrated `fluid` +- migrated `tendrils` +- migrated `spirit` +- updated validation docs +- updated review package + +## Recommended Execution Order +1. Freeze transport spec +2. Refactor host to provider model +3. Build local iframe shell with placeholder renderer +4. Migrate `fluid` +5. Stabilize replay / resize / seek +6. Migrate `tendrils` +7. Migrate `spirit` +8. Re-run cross-surface validation +9. Update external review package + +## Success Definition +This plan succeeds when: +- non-default styles no longer rely on Muon’s default-renderer internals, +- host uses one iframe transport model across providers, +- replay behavior is deterministic for all supported styles, +- failures are surfaced explicitly instead of appearing as frozen visuals, +- reviewers can evaluate styles through a consistent iframe lifecycle rather than special-case paths. diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/fluid-render-graph.js b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/fluid-render-graph.js new file mode 100644 index 0000000..9ed9b06 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/fluid-render-graph.js @@ -0,0 +1,1118 @@ +/* StemEngine Fluid Render Graph (Muon fork overlay) + * + * Phase 2 implementation: WebGL fluid simulation with audio reactivity. + * Injected at build time; compatible with existing Muon visualizer. + * + * Features: + * - Ping-pong FBO infrastructure (velocity, dye, divergence, pressure, curl) + * - Dynamic resize handling + * - Bloom chain (balanced + high tiers) + * - Audio-reactive splatting (bass → force, beat → impulse, treble → turbulence) + * - Three quality tiers (low, balanced, high) + */ + +;(function stemengineFluidRenderGraph() { + 'use strict' + + // ========================================================================= + // CONSTANTS & CONFIGURATION + // ========================================================================= + + /** Quality tier iteration counts */ + var QUALITY_CONFIG = { + low: { pressure_iterations: 10, bloom: false }, + balanced: { pressure_iterations: 20, bloom: true }, + high: { pressure_iterations: 40, bloom: true }, + } + + /** Default texture format for float data */ + var TEXTURE_FORMAT = 0x1908 // gl.RGBA + var TEXTURE_TYPE = 0x1406 // gl.UNSIGNED_BYTE + var FLOAT_TEXTURE_TYPE = 0x1406 // Will be overridden if EXT_color_buffer_float available + + /** Splat radius (in pixels) for audio-reactive splatting */ + var SPLAT_RADIUS_BASE = 15 + var SPLAT_RADIUS_MAX = 50 + + function clamp01(v) { + var n = Number(v) + if (!isFinite(n)) n = 0 + return Math.max(0, Math.min(1, n)) + } + + // Match Muon's snappier feel by accenting peaks while keeping low-end detail. + function shaped(v, power) { + var c = clamp01(v) + return Math.pow(c, power) + } + + function getBand(audio, key, fallback) { + var bands = audio && audio.frequencyBands ? audio.frequencyBands : null + var candidate = bands && isFinite(Number(bands[key])) ? Number(bands[key]) : fallback + return clamp01(candidate) + } + + // ========================================================================= + // UTILITY: WebGL Extensions & Capabilities + // ========================================================================= + + function detectCapabilities(gl) { + var caps = { + floatTextures: false, + floatLinear: false, + drawBuffers: false, + } + + var extColorBuffer = gl.getExtension('EXT_color_buffer_float') + var extTextureFloat = gl.getExtension('OES_texture_float') + var extTextureHalfFloat = gl.getExtension('OES_texture_half_float') + var extFloatLinear = gl.getExtension('OES_texture_float_linear') + var extDrawBuffers = gl.getExtension('WEBGL_draw_buffers') + + if (extColorBuffer && extTextureFloat && extFloatLinear) { + caps.floatTextures = true + caps.floatLinear = true + FLOAT_TEXTURE_TYPE = 0x1406 // gl.FLOAT + TEXTURE_TYPE = 0x1406 // gl.FLOAT + } else if (extTextureHalfFloat && extColorBuffer) { + caps.floatTextures = true + FLOAT_TEXTURE_TYPE = 0x140b // gl.HALF_FLOAT_OES + TEXTURE_TYPE = 0x140b + } + + if (extDrawBuffers) { + caps.drawBuffers = true + } + + return caps + } + + // ========================================================================= + // SHADER COMPILATION & LINKING + // ========================================================================= + + function compileShader(gl, source, type) { + var shader = gl.createShader(type) + gl.shaderSource(shader, source) + gl.compileShader(shader) + + if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) { + var info = gl.getShaderInfoLog(shader) + console.error('[StemEngine][Fluid] Shader compile error:', info) + window.__stemengineEmitError__('SHADER_COMPILE', info) + gl.deleteShader(shader) + return null + } + + return shader + } + + function linkProgram(gl, vsSrc, fsSrc) { + var vs = compileShader(gl, vsSrc, gl.VERTEX_SHADER) + var fs = compileShader(gl, fsSrc, gl.FRAGMENT_SHADER) + + if (!vs || !fs) return null + + var prog = gl.createProgram() + gl.attachShader(prog, vs) + gl.attachShader(prog, fs) + gl.linkProgram(prog) + + if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) { + var info = gl.getProgramInfoLog(prog) + console.error('[StemEngine][Fluid] Program link error:', info) + window.__stemengineEmitError__('PROGRAM_LINK', info) + gl.deleteProgram(prog) + return null + } + + gl.deleteShader(vs) + gl.deleteShader(fs) + + return prog + } + + // ========================================================================= + // FULLSCREEN QUAD VERTEX SHADER (shared by all passes) + // ========================================================================= + + var QUAD_VS = ` + attribute vec2 aPos; + varying vec2 vUv; + + void main() { + vUv = aPos * 0.5 + 0.5; + gl_Position = vec4(aPos, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: SPLAT + // ========================================================================= + + var SPLAT_FS = ` + precision highp float; + uniform sampler2D uVelocity; + uniform vec2 uSplatPos; + uniform float uSplatRadius; + uniform float uSplatForce; + uniform bool uSplatIsBeat; + uniform float uTurbulenceAmplitude; + varying vec2 vUv; + + void main() { + vec2 pos = gl_FragCoord.xy; + float dist = length(pos - uSplatPos); + + // Gaussian kernel + float kernel = exp(-dist * dist / (uSplatRadius * uSplatRadius)); + + vec2 splatVel; + if (uSplatIsBeat) { + // Concentrated impulse for beat + splatVel = normalize(pos - uSplatPos) * uSplatForce * kernel; + } else { + // Diffuse spread for bass + splatVel = (pos - uSplatPos) * uSplatForce * 0.1 * kernel; + } + + // Apply turbulence rotation (treble-driven vorticity) + float angle = uTurbulenceAmplitude * sin(length(pos) * 0.01) * 3.14159; + splatVel = vec2( + splatVel.x * cos(angle) - splatVel.y * sin(angle), + splatVel.x * sin(angle) + splatVel.y * cos(angle) + ); + + vec2 vel = texture2D(uVelocity, vUv).xy; + gl_FragColor = vec4(vel + splatVel, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: CURL (vorticity) + // ========================================================================= + + var CURL_FS = ` + precision highp float; + uniform sampler2D uVelocity; + uniform vec2 uInvSize; + varying vec2 vUv; + + void main() { + float n = texture2D(uVelocity, vUv + vec2(0.0, uInvSize.y)).y; + float s = texture2D(uVelocity, vUv - vec2(0.0, uInvSize.y)).y; + float e = texture2D(uVelocity, vUv + vec2(uInvSize.x, 0.0)).x; + float w = texture2D(uVelocity, vUv - vec2(uInvSize.x, 0.0)).x; + + float curl = (e - w) - (n - s); + gl_FragColor = vec4(curl, 0.0, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: DIVERGENCE + // ========================================================================= + + var DIVERGENCE_FS = ` + precision highp float; + uniform sampler2D uVelocity; + uniform vec2 uInvSize; + varying vec2 vUv; + + void main() { + float n = texture2D(uVelocity, vUv + vec2(0.0, uInvSize.y)).y; + float s = texture2D(uVelocity, vUv - vec2(0.0, uInvSize.y)).y; + float e = texture2D(uVelocity, vUv + vec2(uInvSize.x, 0.0)).x; + float w = texture2D(uVelocity, vUv - vec2(uInvSize.x, 0.0)).x; + + float div = 0.5 * ((e - w) + (n - s)); + gl_FragColor = vec4(div, 0.0, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: PRESSURE JACOBI + // ========================================================================= + + var PRESSURE_JACOBI_FS = ` + precision highp float; + uniform sampler2D uPressure; + uniform sampler2D uDivergence; + uniform float uAlpha; + uniform float uRBeta; + uniform vec2 uInvSize; + varying vec2 vUv; + + void main() { + float n = texture2D(uPressure, vUv + vec2(0.0, uInvSize.y)).x; + float s = texture2D(uPressure, vUv - vec2(0.0, uInvSize.y)).x; + float e = texture2D(uPressure, vUv + vec2(uInvSize.x, 0.0)).x; + float w = texture2D(uPressure, vUv - vec2(uInvSize.x, 0.0)).x; + + float div = texture2D(uDivergence, vUv).x; + float p = (n + s + e + w + uAlpha * div) * uRBeta; + gl_FragColor = vec4(p, 0.0, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: GRADIENT SUBTRACT + // ========================================================================= + + var GRADIENT_SUBTRACT_FS = ` + precision highp float; + uniform sampler2D uVelocity; + uniform sampler2D uPressure; + uniform vec2 uInvSize; + uniform float uGradientScale; + varying vec2 vUv; + + void main() { + float n = texture2D(uPressure, vUv + vec2(0.0, uInvSize.y)).x; + float s = texture2D(uPressure, vUv - vec2(0.0, uInvSize.y)).x; + float e = texture2D(uPressure, vUv + vec2(uInvSize.x, 0.0)).x; + float w = texture2D(uPressure, vUv - vec2(uInvSize.x, 0.0)).x; + + vec2 grad = uGradientScale * vec2(e - w, n - s); + vec2 vel = texture2D(uVelocity, vUv).xy; + gl_FragColor = vec4(vel - grad, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: ADVECT VELOCITY + // ========================================================================= + + var ADVECT_VELOCITY_FS = ` + precision highp float; + uniform sampler2D uVelocity; + uniform vec2 uInvSize; + uniform float uDt; + uniform float uDecay; + varying vec2 vUv; + + void main() { + vec2 vel = texture2D(uVelocity, vUv).xy; + vec2 backPos = vUv - uDt * vel * uInvSize; + + // Clamp to valid range + backPos = clamp(backPos, uInvSize, 1.0 - uInvSize); + + vec2 advectedVel = texture2D(uVelocity, backPos).xy * uDecay; + gl_FragColor = vec4(advectedVel, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: ADVECT DYE + // ========================================================================= + + var ADVECT_DYE_FS = ` + precision highp float; + uniform sampler2D uDye; + uniform sampler2D uVelocity; + uniform vec2 uInvSize; + uniform float uDt; + uniform float uDecay; + uniform vec3 uAudioColor; + varying vec2 vUv; + + void main() { + vec2 vel = texture2D(uVelocity, vUv).xy; + vec2 backPos = vUv - uDt * vel * uInvSize; + backPos = clamp(backPos, uInvSize, 1.0 - uInvSize); + + vec4 dye = texture2D(uDye, backPos) * uDecay; + + // Inject audio-driven color + float mixFactor = 0.15; + dye.rgb += uAudioColor * mixFactor; + dye.rgb = clamp(dye.rgb, 0.0, 1.0); + + gl_FragColor = dye; + } + ` + + // ========================================================================= + // SHADER SOURCE: BLOOM DOWNSAMPLE + // ========================================================================= + + var BLOOM_DOWNSAMPLE_FS = ` + precision highp float; + uniform sampler2D uSource; + uniform vec2 uInvSize; + varying vec2 vUv; + + void main() { + vec2 uv0 = vUv - uInvSize * 0.5; + vec2 uv1 = vUv + uInvSize * 0.5; + + vec4 s0 = texture2D(uSource, vec2(uv0.x, uv0.y)); + vec4 s1 = texture2D(uSource, vec2(uv1.x, uv0.y)); + vec4 s2 = texture2D(uSource, vec2(uv0.x, uv1.y)); + vec4 s3 = texture2D(uSource, vec2(uv1.x, uv1.y)); + + gl_FragColor = (s0 + s1 + s2 + s3) * 0.25; + } + ` + + // ========================================================================= + // SHADER SOURCE: BLOOM COMPOSITE + // ========================================================================= + + var BLOOM_COMPOSITE_FS = ` + precision highp float; + uniform sampler2D uFullRes; + uniform sampler2D uBloomUp; + uniform float uBloomIntensity; + varying vec2 vUv; + + void main() { + vec4 full = texture2D(uFullRes, vUv); + vec4 bloom = texture2D(uBloomUp, vUv); + + gl_FragColor = full + bloom * uBloomIntensity; + gl_FragColor.rgb = clamp(gl_FragColor.rgb, 0.0, 1.0); + } + ` + + // ========================================================================= + // SHADER SOURCE: COMPOSITE (final render to canvas) + // ========================================================================= + + var COMPOSITE_FS = ` + precision highp float; + uniform sampler2D uDye; + uniform sampler2D uCurl; + uniform sampler2D uVelocity; + uniform bool uShowVelocity; + uniform bool uShowCurl; + varying vec2 vUv; + + void main() { + if (uShowCurl) { + float c = texture2D(uCurl, vUv).x; + float col = abs(c) * 10.0; + gl_FragColor = vec4(vec3(col), 1.0); + } else if (uShowVelocity) { + vec2 vel = texture2D(uVelocity, vUv).xy; + float speed = length(vel) * 0.1; + gl_FragColor = vec4(vec3(speed), 1.0); + } else { + vec4 col = texture2D(uDye, vUv); + gl_FragColor = vec4(col.rgb, 1.0); + } + } + ` + + // ========================================================================= + // FRAMEBUFFER OBJECT (FBO) MANAGEMENT + // ========================================================================= + + function createFBO(gl, width, height, format, type) { + var fbo = gl.createFramebuffer() + var texture = gl.createTexture() + + gl.bindFramebuffer(gl.FRAMEBUFFER, fbo) + gl.bindTexture(gl.TEXTURE_2D, texture) + gl.texImage2D(gl.TEXTURE_2D, 0, format, width, height, 0, format, type, null) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR) + + gl.framebufferTexture2D( + gl.FRAMEBUFFER, + gl.COLOR_ATTACHMENT0, + gl.TEXTURE_2D, + texture, + 0 + ) + + var status = gl.checkFramebufferStatus(gl.FRAMEBUFFER) + if (status !== gl.FRAMEBUFFER_COMPLETE) { + console.warn('[StemEngine][Fluid] FBO incomplete:', status) + } + + gl.bindFramebuffer(gl.FRAMEBUFFER, null) + + return { fbo: fbo, texture: texture, width: width, height: height } + } + + function deleteFBO(gl, fbo) { + if (fbo) { + gl.deleteFramebuffer(fbo.fbo) + gl.deleteTexture(fbo.texture) + } + } + + // ========================================================================= + // FLUID SIMULATOR STATE + // ========================================================================= + + function pickCanvas() { + return document.querySelector('canvas') || null + } + + function getCanvasSize(canvas) { + if (!canvas) return { width: 512, height: 512 } + var w = canvas.clientWidth || canvas.width || 512 + var h = canvas.clientHeight || canvas.height || 512 + return { + width: Math.max(1, w | 0), + height: Math.max(1, h | 0), + } + } + + function FluidSimulator(gl, width, height, quality) { + this.canvas = pickCanvas() + this.gl = + gl || + (this.canvas && + (this.canvas.getContext('webgl2', { alpha: false, antialias: false }) || + this.canvas.getContext('webgl', { alpha: false, antialias: false }))) + + if (!this.gl) { + window.__stemengineEmitError__('WEBGL_CONTEXT', 'Unable to acquire WebGL context for FluidSimulator') + throw new Error('[StemEngine][Fluid] Unable to acquire WebGL context') + } + + var initialSize = getCanvasSize(this.canvas) + this.width = Math.max(1, Number(width) || initialSize.width) + this.height = Math.max(1, Number(height) || initialSize.height) + this.quality = String(quality || window.__STEMENGINE_MUON_QUALITY__ || 'balanced') + + var config = QUALITY_CONFIG[this.quality] || QUALITY_CONFIG.balanced + + this.pressureIterations = config.pressure_iterations + this.bloomEnabled = config.bloom + + // Capabilities + this.capabilities = detectCapabilities(this.gl) + + // FBO pairs (ping-pong) + this.velocity = null + this.velocityBuffer = null + this.dye = null + this.dyeBuffer = null + this.divergence = null + this.pressure = null + this.pressureBuffer = null + this.curl = null + + // Bloom chain (if enabled) + this.bloom = null + + // Shader programs + this.programs = {} + + // State + this.paused = false + this.currentStyle = 'default' + this.rafId = null + this.running = false + + // Fullscreen quad VAO + this.quadVBO = null + + // Initialize + this.initializeQuad() + this.compileShaders() + this.allocateFBOs() + this.start() + } + + FluidSimulator.prototype.setQuality = function (quality) { + var q = String(quality || 'balanced') + if (!QUALITY_CONFIG[q] || q === this.quality) return + this.quality = q + var config = QUALITY_CONFIG[this.quality] + this.pressureIterations = config.pressure_iterations + this.bloomEnabled = config.bloom + this.allocateFBOs() + } + + FluidSimulator.prototype.setViewport = function (width, height) { + var w = Math.max(1, Number(width) || this.width) + var h = Math.max(1, Number(height) || this.height) + if (w === this.width && h === this.height) return + this.width = w + this.height = h + if (this.canvas) { + this.canvas.width = w + this.canvas.height = h + } + this.allocateFBOs() + } + + FluidSimulator.prototype.start = function () { + if (this.running) return + this.running = true + this.tick() + } + + FluidSimulator.prototype.stop = function () { + this.running = false + if (this.rafId != null) { + cancelAnimationFrame(this.rafId) + this.rafId = null + } + } + + FluidSimulator.prototype.tick = function () { + if (!this.running) return + + // Apply host-driven runtime state. + this.paused = Boolean(window.__STEMENGINE_MUON_PAUSED__) + this.setQuality(window.__STEMENGINE_MUON_QUALITY__) + + var viewport = window.__STEMENGINE_MUON_VIEWPORT__ + if (viewport && typeof viewport === 'object') { + this.setViewport(viewport.width, viewport.height) + } + + var features = window.__STEMENGINE_MUON_AUDIO__ || { + bass: 0, + treble: 0, + rms: 0, + beat: false, + } + + this.step(features) + + var self = this + this.rafId = requestAnimationFrame(function () { + self.tick() + }) + } + + FluidSimulator.prototype.initializeQuad = function () { + var gl = this.gl + var positions = new Float32Array([-1, -1, 1, -1, 1, 1, -1, 1]) + + this.quadVBO = gl.createBuffer() + gl.bindBuffer(gl.ARRAY_BUFFER, this.quadVBO) + gl.bufferData(gl.ARRAY_BUFFER, positions, gl.STATIC_DRAW) + } + + FluidSimulator.prototype.compileShaders = function () { + var gl = this.gl + + this.programs.splat = linkProgram(gl, QUAD_VS, SPLAT_FS) + this.programs.curl = linkProgram(gl, QUAD_VS, CURL_FS) + this.programs.divergence = linkProgram(gl, QUAD_VS, DIVERGENCE_FS) + this.programs.pressureJacobi = linkProgram(gl, QUAD_VS, PRESSURE_JACOBI_FS) + this.programs.gradientSubtract = linkProgram(gl, QUAD_VS, GRADIENT_SUBTRACT_FS) + this.programs.advectVelocity = linkProgram(gl, QUAD_VS, ADVECT_VELOCITY_FS) + this.programs.advectDye = linkProgram(gl, QUAD_VS, ADVECT_DYE_FS) + this.programs.bloomDownsample = linkProgram(gl, QUAD_VS, BLOOM_DOWNSAMPLE_FS) + this.programs.bloomComposite = linkProgram(gl, QUAD_VS, BLOOM_COMPOSITE_FS) + this.programs.composite = linkProgram(gl, QUAD_VS, COMPOSITE_FS) + + console.log('[StemEngine][Fluid] Shaders compiled successfully') + } + + FluidSimulator.prototype.renderFullscreenQuad = function (program, fbo) { + var gl = this.gl + + if (!program) return + + gl.useProgram(program) + gl.bindFramebuffer(gl.FRAMEBUFFER, fbo ? fbo.fbo : null) + + if (fbo) { + gl.viewport(0, 0, fbo.width, fbo.height) + } else { + gl.viewport(0, 0, this.width, this.height) + } + + gl.bindBuffer(gl.ARRAY_BUFFER, this.quadVBO) + var posLoc = gl.getAttribLocation(program, 'aPos') + if (posLoc >= 0) { + gl.enableVertexAttribArray(posLoc) + gl.vertexAttribPointer(posLoc, 2, gl.FLOAT, false, 0, 0) + } + + gl.drawArrays(gl.TRIANGLE_FAN, 0, 4) + } + + FluidSimulator.prototype.allocateFBOs = function () { + var gl = this.gl + var w = this.width + var h = this.height + var halfW = Math.max(1, Math.floor(w / 2)) + var halfH = Math.max(1, Math.floor(h / 2)) + + // Free old FBOs + this.releaseFBOs() + + // Velocity field (float texture, 2-component or RGBA) + this.velocity = createFBO(gl, w, h, TEXTURE_FORMAT, FLOAT_TEXTURE_TYPE) + this.velocityBuffer = createFBO(gl, w, h, TEXTURE_FORMAT, FLOAT_TEXTURE_TYPE) + + // Dye field (color, usually RGBA) + this.dye = createFBO(gl, w, h, gl.RGBA, gl.UNSIGNED_BYTE) + this.dyeBuffer = createFBO(gl, w, h, gl.RGBA, gl.UNSIGNED_BYTE) + + // Divergence (scalar or 1-component) + this.divergence = createFBO(gl, w, h, TEXTURE_FORMAT, FLOAT_TEXTURE_TYPE) + + // Pressure field (scalar) + this.pressure = createFBO(gl, w, h, TEXTURE_FORMAT, FLOAT_TEXTURE_TYPE) + this.pressureBuffer = createFBO(gl, w, h, TEXTURE_FORMAT, FLOAT_TEXTURE_TYPE) + + // Curl (vorticity) + this.curl = createFBO(gl, w, h, TEXTURE_FORMAT, FLOAT_TEXTURE_TYPE) + + // Bloom chain (downsamples 1, 2, 4x) + if (this.bloomEnabled) { + this.bloom = { + down1: createFBO(gl, halfW, halfH, gl.RGBA, gl.UNSIGNED_BYTE), + down2: createFBO(gl, Math.max(1, Math.floor(halfW / 2)), Math.max(1, Math.floor(halfH / 2)), gl.RGBA, gl.UNSIGNED_BYTE), + up1: createFBO(gl, halfW, halfH, gl.RGBA, gl.UNSIGNED_BYTE), + } + } + + console.log('[StemEngine][Fluid] FBOs allocated:', { w: w, h: h, quality: this.quality }) + } + + FluidSimulator.prototype.releaseFBOs = function () { + var gl = this.gl + deleteFBO(gl, this.velocity) + deleteFBO(gl, this.velocityBuffer) + deleteFBO(gl, this.dye) + deleteFBO(gl, this.dyeBuffer) + deleteFBO(gl, this.divergence) + deleteFBO(gl, this.pressure) + deleteFBO(gl, this.pressureBuffer) + deleteFBO(gl, this.curl) + + if (this.bloom) { + deleteFBO(gl, this.bloom.down1) + deleteFBO(gl, this.bloom.down2) + deleteFBO(gl, this.bloom.up1) + } + } + + // ========================================================================= + // RENDER PASSES (IMPLEMENTATIONS) + // ========================================================================= + + /** + * Splat pass: inject audio-driven forces + */ + FluidSimulator.prototype.splat = function (x, y, force, color, radius) { + var gl = this.gl + var prog = this.programs.splat + if (!prog) return + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, this.velocityBuffer.fbo) + gl.viewport(0, 0, this.velocityBuffer.width, this.velocityBuffer.height) + + // Bind input velocity + gl.uniform1i(gl.getUniformLocation(prog, 'uVelocity'), 0) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.velocity.texture) + + // Set uniforms + gl.uniform2f(gl.getUniformLocation(prog, 'uSplatPos'), x, y) + gl.uniform1f(gl.getUniformLocation(prog, 'uSplatRadius'), radius || 15) + gl.uniform1f(gl.getUniformLocation(prog, 'uSplatForce'), force || 10) + gl.uniform1i(gl.getUniformLocation(prog, 'uSplatIsBeat'), force > 50 ? 1 : 0) + gl.uniform1f(gl.getUniformLocation(prog, 'uTurbulenceAmplitude'), color.g || 0) + + this.renderFullscreenQuad(prog, this.velocityBuffer) + + // Swap + var tmp = this.velocity + this.velocity = this.velocityBuffer + this.velocityBuffer = tmp + } + + /** + * Curl pass: compute vorticity + */ + FluidSimulator.prototype.computeCurl = function () { + var gl = this.gl + var prog = this.programs.curl + if (!prog) return + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, this.curl.fbo) + gl.viewport(0, 0, this.curl.width, this.curl.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uVelocity'), 0) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.velocity.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.velocity.width, 1.0 / this.velocity.height) + + this.renderFullscreenQuad(prog, this.curl) + } + + /** + * Divergence pass: compute incompressibility metric + */ + FluidSimulator.prototype.computeDivergence = function () { + var gl = this.gl + var prog = this.programs.divergence + if (!prog) return + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, this.divergence.fbo) + gl.viewport(0, 0, this.divergence.width, this.divergence.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uVelocity'), 0) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.velocity.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.velocity.width, 1.0 / this.velocity.height) + + this.renderFullscreenQuad(prog, this.divergence) + } + + /** + * Pressure Jacobi iteration + */ + FluidSimulator.prototype.pressureJacobi = function () { + var gl = this.gl + var prog = this.programs.pressureJacobi + if (!prog) return + + gl.useProgram(prog) + + var alpha = -1.0 + var rBeta = 0.25 + + for (var i = 0; i < this.pressureIterations; i++) { + gl.bindFramebuffer(gl.FRAMEBUFFER, this.pressureBuffer.fbo) + gl.viewport(0, 0, this.pressureBuffer.width, this.pressureBuffer.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uPressure'), 0) + gl.uniform1i(gl.getUniformLocation(prog, 'uDivergence'), 1) + + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.pressure.texture) + + gl.activeTexture(gl.TEXTURE1) + gl.bindTexture(gl.TEXTURE_2D, this.divergence.texture) + + gl.uniform1f(gl.getUniformLocation(prog, 'uAlpha'), alpha) + gl.uniform1f(gl.getUniformLocation(prog, 'uRBeta'), rBeta) + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.pressure.width, 1.0 / this.pressure.height) + + this.renderFullscreenQuad(prog, this.pressureBuffer) + + // Swap + var tmp = this.pressure + this.pressure = this.pressureBuffer + this.pressureBuffer = tmp + } + } + + /** + * Gradient subtract pass: Remove pressure from velocity + */ + FluidSimulator.prototype.gradientSubtract = function () { + var gl = this.gl + var prog = this.programs.gradientSubtract + if (!prog) return + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, this.velocityBuffer.fbo) + gl.viewport(0, 0, this.velocityBuffer.width, this.velocityBuffer.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uVelocity'), 0) + gl.uniform1i(gl.getUniformLocation(prog, 'uPressure'), 1) + + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.velocity.texture) + + gl.activeTexture(gl.TEXTURE1) + gl.bindTexture(gl.TEXTURE_2D, this.pressure.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.velocity.width, 1.0 / this.velocity.height) + gl.uniform1f(gl.getUniformLocation(prog, 'uGradientScale'), 1.0) + + this.renderFullscreenQuad(prog, this.velocityBuffer) + + // Swap + var tmp = this.velocity + this.velocity = this.velocityBuffer + this.velocityBuffer = tmp + } + + /** + * Advection pass for velocity + */ + FluidSimulator.prototype.advectVelocity = function () { + var gl = this.gl + var prog = this.programs.advectVelocity + if (!prog) return + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, this.velocityBuffer.fbo) + gl.viewport(0, 0, this.velocityBuffer.width, this.velocityBuffer.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uVelocity'), 0) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.velocity.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.velocity.width, 1.0 / this.velocity.height) + gl.uniform1f(gl.getUniformLocation(prog, 'uDt'), 0.01) + gl.uniform1f(gl.getUniformLocation(prog, 'uDecay'), 0.99) + + this.renderFullscreenQuad(prog, this.velocityBuffer) + + // Swap + var tmp = this.velocity + this.velocity = this.velocityBuffer + this.velocityBuffer = tmp + } + + /** + * Advection pass for dye + */ + FluidSimulator.prototype.advectDye = function (audioFeatures) { + var gl = this.gl + var prog = this.programs.advectDye + if (!prog) return + + var audio = audioFeatures || {} + var sub = getBand(audio, 'sub', audio.sub || 0) + var bass = getBand(audio, 'bass', audio.bass || 0) + var highMid = getBand(audio, 'highMid', audio.treble || 0) + var brilliance = getBand(audio, 'brilliance', audio.treble || 0) + var rms = clamp01(audio.rms || 0) + var audioColor = [ + shaped(0.55 * bass + 0.45 * sub, 0.62), + shaped(rms, 0.75), + shaped(0.6 * highMid + 0.4 * brilliance, 0.65), + ] + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, this.dyeBuffer.fbo) + gl.viewport(0, 0, this.dyeBuffer.width, this.dyeBuffer.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uDye'), 0) + gl.uniform1i(gl.getUniformLocation(prog, 'uVelocity'), 1) + + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.dye.texture) + + gl.activeTexture(gl.TEXTURE1) + gl.bindTexture(gl.TEXTURE_2D, this.velocity.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.dye.width, 1.0 / this.dye.height) + gl.uniform1f(gl.getUniformLocation(prog, 'uDt'), 0.01) + gl.uniform1f(gl.getUniformLocation(prog, 'uDecay'), 0.98) + gl.uniform3f(gl.getUniformLocation(prog, 'uAudioColor'), audioColor[0], audioColor[1], audioColor[2]) + + this.renderFullscreenQuad(prog, this.dyeBuffer) + + // Swap + var tmp = this.dye + this.dye = this.dyeBuffer + this.dyeBuffer = tmp + } + + /** + * Bloom downsample pass + */ + FluidSimulator.prototype.bloomDownsample = function () { + if (!this.bloomEnabled || !this.bloom) return + + var gl = this.gl + var prog = this.programs.bloomDownsample + if (!prog) return + + gl.useProgram(prog) + + // Down to 1/2 resolution + gl.bindFramebuffer(gl.FRAMEBUFFER, this.bloom.down1.fbo) + gl.viewport(0, 0, this.bloom.down1.width, this.bloom.down1.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uSource'), 0) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.dye.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.dye.width, 1.0 / this.dye.height) + + this.renderFullscreenQuad(prog, this.bloom.down1) + + // Down to 1/4 resolution + gl.bindFramebuffer(gl.FRAMEBUFFER, this.bloom.down2.fbo) + gl.viewport(0, 0, this.bloom.down2.width, this.bloom.down2.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uSource'), 0) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.bloom.down1.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.bloom.down1.width, 1.0 / this.bloom.down1.height) + + this.renderFullscreenQuad(prog, this.bloom.down2) + + // Upsample back to 1/2 resolution + gl.bindFramebuffer(gl.FRAMEBUFFER, this.bloom.up1.fbo) + gl.viewport(0, 0, this.bloom.up1.width, this.bloom.up1.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uSource'), 0) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.bloom.down2.texture) + + gl.uniform2f(gl.getUniformLocation(prog, 'uInvSize'), 1.0 / this.bloom.down2.width, 1.0 / this.bloom.down2.height) + + this.renderFullscreenQuad(prog, this.bloom.up1) + } + + /** + * Bloom upsample + composite pass + */ + FluidSimulator.prototype.bloomComposite = function (audioFeatures) { + if (!this.bloomEnabled || !this.bloom) return + + var gl = this.gl + var prog = this.programs.bloomComposite + if (!prog) return + + var audio = audioFeatures || {} + var rms = clamp01(audio.rms || 0) + var sparkle = getBand(audio, 'brilliance', audio.treble || 0) + var bloomIntensity = 0.45 + shaped(rms, 0.8) * 1.35 + shaped(sparkle, 0.65) * 0.45 + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, this.dye.fbo) + gl.viewport(0, 0, this.dye.width, this.dye.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uFullRes'), 0) + gl.uniform1i(gl.getUniformLocation(prog, 'uBloomUp'), 1) + + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.dye.texture) + + gl.activeTexture(gl.TEXTURE1) + gl.bindTexture(gl.TEXTURE_2D, this.bloom.up1.texture) + + gl.uniform1f(gl.getUniformLocation(prog, 'uBloomIntensity'), bloomIntensity) + + this.renderFullscreenQuad(prog, this.dye) + } + + /** + * Final composite: render dye to screen + */ + FluidSimulator.prototype.composite = function () { + var gl = this.gl + var prog = this.programs.composite + if (!prog) return + + gl.useProgram(prog) + gl.bindFramebuffer(gl.FRAMEBUFFER, null) + gl.viewport(0, 0, this.width, this.height) + + gl.uniform1i(gl.getUniformLocation(prog, 'uDye'), 0) + gl.uniform1i(gl.getUniformLocation(prog, 'uCurl'), 1) + gl.uniform1i(gl.getUniformLocation(prog, 'uVelocity'), 2) + + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.dye.texture) + + gl.activeTexture(gl.TEXTURE1) + gl.bindTexture(gl.TEXTURE_2D, this.curl.texture) + + gl.activeTexture(gl.TEXTURE2) + gl.bindTexture(gl.TEXTURE_2D, this.velocity.texture) + + gl.uniform1i(gl.getUniformLocation(prog, 'uShowVelocity'), 0) + gl.uniform1i(gl.getUniformLocation(prog, 'uShowCurl'), 0) + + this.renderFullscreenQuad(prog, null) + } + + // ========================================================================= + // MAIN LOOP & DISPATCH + // ========================================================================= + + FluidSimulator.prototype.step = function (audioFeatures) { + if (this.paused || !this.velocity) return + + var gl = this.gl + + // Richer band-aware signal shaping for better dynamic range. + var bassBand = getBand(audioFeatures, 'bass', audioFeatures?.bass || 0) + var subBand = getBand(audioFeatures, 'sub', audioFeatures?.sub || 0) + var lowMidBand = getBand(audioFeatures, 'lowMid', audioFeatures?.bass || 0) + var highMidBand = getBand(audioFeatures, 'highMid', audioFeatures?.treble || 0) + var brillianceBand = getBand(audioFeatures, 'brilliance', audioFeatures?.treble || 0) + + var bass = shaped(0.55 * bassBand + 0.45 * subBand, 0.62) + var beat = Boolean(audioFeatures?.beat) + var treble = shaped(0.45 * highMidBand + 0.55 * brillianceBand, 0.68) + var rms = shaped(clamp01(audioFeatures?.rms || 0), 0.78) + var beatPhase = clamp01(audioFeatures?.beatPhase || 0) + var drumInput = audioFeatures?.drumIntensity != null ? Number(audioFeatures.drumIntensity) : (0.7 * bassBand + 0.3 * lowMidBand) + var drumIntensity = shaped(clamp01(drumInput), 0.72) + + try { + // --- PIPELINE --- + // 1. Splat pass (inject audio-driven forces) + var phaseBoost = 0.65 + 0.35 * Math.sin(beatPhase * Math.PI * 2) + var splatRadius = SPLAT_RADIUS_BASE + (drumIntensity * phaseBoost * (SPLAT_RADIUS_MAX - SPLAT_RADIUS_BASE)) + var splatForce = drumIntensity * (40 + phaseBoost * 20) + this.splat(this.width / 2, this.height / 2, splatForce, { r: bass, g: treble, b: rms, a: 1 }, splatRadius) + + if (beat) { + // Impulse on beat detection + var beatImpulse = 90 + drumIntensity * 30 + this.splat(this.width / 2, this.height / 2, beatImpulse, { r: 1, g: 1, b: 1, a: 1 }, 30) + } + + // 2. Vorticity confinement + this.computeCurl() + + // 3. Pressure solve + this.computeDivergence() + this.pressureJacobi() + this.gradientSubtract() + + // 4. Advection + this.advectVelocity() + this.advectDye(audioFeatures) + + // 5. Bloom (optional) + if (this.bloomEnabled) { + this.bloomDownsample() + this.bloomComposite(audioFeatures) + } + + // 6. Composite to screen + this.composite() + } catch (e) { + console.error('[StemEngine][Fluid] Error in step():', e) + window.__stemengineEmitError__('STEP_ERROR', String(e)) + } + } + + FluidSimulator.prototype.dispose = function () { + this.stop() + if (this.quadVBO) { + this.gl.deleteBuffer(this.quadVBO) + this.quadVBO = null + } + this.releaseFBOs() + for (var key in this.programs) { + if (this.programs[key]) { + this.gl.deleteProgram(this.programs[key]) + } + } + } + + // ========================================================================= + // GLOBAL EXPORT & REGISTRATION + // ========================================================================= + + window.__StemEngineFluidRenderGraph__ = FluidSimulator + + console.log('[StemEngine][Fluid] Render graph module loaded') +})() diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/spirit-render-graph.js b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/spirit-render-graph.js new file mode 100644 index 0000000..7335316 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/spirit-render-graph.js @@ -0,0 +1,273 @@ +/* StemEngine Spirit Render Graph (Muon fork overlay) + * Atmospheric layered glow visualization with audio reactivity. + */ + +;(function stemengineSpiritRenderGraph() { + 'use strict' + + var QUALITY_SCALE = { low: 0.8, balanced: 1.0, high: 1.25 } + + function clamp01(v) { + var n = Number(v) + if (!isFinite(n)) n = 0 + return Math.max(0, Math.min(1, n)) + } + + function shaped(v, power) { + return Math.pow(clamp01(v), power) + } + + function getBand(audio, key, fallback) { + var bands = audio && audio.frequencyBands ? audio.frequencyBands : null + var value = bands && isFinite(Number(bands[key])) ? Number(bands[key]) : fallback + return clamp01(value) + } + + function compileProgram(gl, vsSrc, fsSrc) { + function c(type, src) { + var s = gl.createShader(type) + gl.shaderSource(s, src) + gl.compileShader(s) + if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) { + var e = gl.getShaderInfoLog(s) || 'shader compile failed' + window.__stemengineEmitError__('SPIRIT_SHADER', e) + gl.deleteShader(s) + return null + } + return s + } + var vs = c(gl.VERTEX_SHADER, vsSrc) + var fs = c(gl.FRAGMENT_SHADER, fsSrc) + if (!vs || !fs) return null + var p = gl.createProgram() + gl.attachShader(p, vs) + gl.attachShader(p, fs) + gl.linkProgram(p) + gl.deleteShader(vs) + gl.deleteShader(fs) + if (!gl.getProgramParameter(p, gl.LINK_STATUS)) { + var e2 = gl.getProgramInfoLog(p) || 'program link failed' + window.__stemengineEmitError__('SPIRIT_PROGRAM', e2) + gl.deleteProgram(p) + return null + } + return p + } + + function mkTarget(gl, w, h) { + var tex = gl.createTexture() + gl.bindTexture(gl.TEXTURE_2D, tex) + gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, w, h, 0, gl.RGBA, gl.UNSIGNED_BYTE, null) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE) + var fbo = gl.createFramebuffer() + gl.bindFramebuffer(gl.FRAMEBUFFER, fbo) + gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0) + gl.bindFramebuffer(gl.FRAMEBUFFER, null) + return { tex: tex, fbo: fbo, width: w, height: h } + } + + function SpiritSimulator(canvas, gl, width, height, quality) { + this.canvas = canvas || document.querySelector('canvas') + this.gl = gl || (this.canvas && (this.canvas.getContext('webgl2', { alpha: false, antialias: false }) || this.canvas.getContext('webgl', { alpha: false, antialias: false }))) + if (!this.gl) throw new Error('[StemEngine][Spirit] Unable to acquire WebGL context') + + this.width = Math.max(1, Number(width) || this.canvas.clientWidth || this.canvas.width || 512) + this.height = Math.max(1, Number(height) || this.canvas.clientHeight || this.canvas.height || 512) + this.quality = String(quality || window.__STEMENGINE_MUON_QUALITY__ || 'balanced') + this.paused = false + this.running = false + this.rafId = null + + var gl = this.gl + var vs = 'attribute vec2 aPos; varying vec2 vUv; void main(){ vUv=aPos*0.5+0.5; gl_Position=vec4(aPos,0.0,1.0);} ' + var updateFs = [ + 'precision highp float;', + 'uniform sampler2D uPrev;', + 'uniform vec2 uRes;', + 'uniform float uTime;', + 'uniform float uBass;', + 'uniform float uTreble;', + 'uniform float uRms;', + 'uniform float uBeat;', + 'uniform float uFade;', + 'varying vec2 vUv;', + 'float hash(vec2 p){ return fract(sin(dot(p, vec2(12.9898,78.233))) * 43758.5453); }', + 'float fbm(vec2 p){', + ' float v=0.0; float a=0.5;', + ' for(int i=0;i<4;i++){', + ' v += (hash(floor(p)) * 2.0 - 1.0) * a;', + ' p = p*2.03 + 13.1;', + ' a *= 0.5;', + ' }', + ' return v*0.5 + 0.5;', + '}', + 'void main(){', + ' vec2 uv=vUv;', + ' vec2 c=uv-0.5;', + ' float r=length(c);', + ' float a=atan(c.y,c.x);', + ' vec2 drift = vec2(cos(a + uTime*(0.3+uBass*1.8)), sin(a - uTime*(0.22+uTreble*1.4))) * (0.001 + 0.003*uBass);', + ' vec3 prev=texture2D(uPrev, clamp(uv-drift,0.0,1.0)).rgb * uFade;', + ' float n = fbm(uv*5.0 + vec2(uTime*0.12, -uTime*0.08));', + ' float halo = exp(-r*(3.0 + uTreble*3.0));', + ' float strand = smoothstep(0.0, 0.08 + uTreble*0.06, abs(sin(a*10.0 + n*9.0 + uTime*(1.8+uRms*4.0))) * (0.55-r));', + ' float pulse = uBeat * exp(-r*2.8) + uRms*0.08;', + ' vec3 cool = vec3(0.2 + uTreble*0.6, 0.35 + uRms*0.45, 0.8);', + ' vec3 warm = vec3(0.95, 0.45 + uBass*0.35, 0.2 + uRms*0.25);', + ' vec3 glow = mix(cool, warm, clamp(n + uBass*0.35, 0.0, 1.0)) * (halo*0.35 + strand*0.65 + pulse*0.5);', + ' gl_FragColor = vec4(clamp(prev + glow, 0.0, 1.0), 1.0);', + '}' + ].join('\n') + var blitFs = 'precision mediump float; uniform sampler2D uTex; varying vec2 vUv; void main(){ vec3 c=texture2D(uTex,vUv).rgb; gl_FragColor=vec4(c*c*0.15 + c,1.0);} ' + + this.updateProgram = compileProgram(gl, vs, updateFs) + this.blitProgram = compileProgram(gl, vs, blitFs) + + var quad = new Float32Array([-1,-1, 1,-1, -1,1, -1,1, 1,-1, 1,1]) + this.quad = gl.createBuffer() + gl.bindBuffer(gl.ARRAY_BUFFER, this.quad) + gl.bufferData(gl.ARRAY_BUFFER, quad, gl.STATIC_DRAW) + + this.read = null + this.write = null + this.allocateTargets() + this.start() + } + + SpiritSimulator.prototype.allocateTargets = function () { + var gl = this.gl + if (this.read) { + gl.deleteTexture(this.read.tex) + gl.deleteFramebuffer(this.read.fbo) + } + if (this.write) { + gl.deleteTexture(this.write.tex) + gl.deleteFramebuffer(this.write.fbo) + } + var scale = QUALITY_SCALE[this.quality] || QUALITY_SCALE.balanced + var w = Math.max(64, Math.floor(this.width * scale)) + var h = Math.max(64, Math.floor(this.height * scale)) + this.read = mkTarget(gl, w, h) + this.write = mkTarget(gl, w, h) + } + + SpiritSimulator.prototype.setQuality = function (q) { + var next = String(q || 'balanced') + if (next === this.quality) return + this.quality = next + this.allocateTargets() + } + + SpiritSimulator.prototype.setViewport = function (w, h) { + var nw = Math.max(1, Number(w) || this.width) + var nh = Math.max(1, Number(h) || this.height) + if (nw === this.width && nh === this.height) return + this.width = nw + this.height = nh + if (this.canvas) { + this.canvas.width = nw + this.canvas.height = nh + } + this.allocateTargets() + } + + SpiritSimulator.prototype.draw = function (program) { + var gl = this.gl + gl.useProgram(program) + gl.bindBuffer(gl.ARRAY_BUFFER, this.quad) + var loc = gl.getAttribLocation(program, 'aPos') + gl.enableVertexAttribArray(loc) + gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0) + gl.drawArrays(gl.TRIANGLES, 0, 6) + } + + SpiritSimulator.prototype.step = function () { + if (this.paused || !this.read || !this.write) return + var gl = this.gl + var t = performance.now() * 0.001 + var audio = window.__STEMENGINE_MUON_AUDIO__ || {} + var bass = shaped(0.5 * getBand(audio, 'sub', audio.sub || 0) + 0.5 * getBand(audio, 'bass', audio.bass || 0), 0.64) + var treble = shaped(0.55 * getBand(audio, 'highMid', audio.treble || 0) + 0.45 * getBand(audio, 'brilliance', audio.treble || 0), 0.66) + var rms = shaped(clamp01(audio.rms || 0), 0.8) + var beat = audio.beat ? 1 : 0 + var buildBoost = audio.isBuildUp ? 0.02 : 0 + var dropBoost = audio.isBeatDrop ? 0.02 : 0 + + gl.bindFramebuffer(gl.FRAMEBUFFER, this.write.fbo) + gl.viewport(0, 0, this.write.width, this.write.height) + gl.useProgram(this.updateProgram) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.read.tex) + gl.uniform1i(gl.getUniformLocation(this.updateProgram, 'uPrev'), 0) + gl.uniform2f(gl.getUniformLocation(this.updateProgram, 'uRes'), this.write.width, this.write.height) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTime'), t) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBass'), bass) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTreble'), treble) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uRms'), rms) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBeat'), beat) + gl.uniform1f( + gl.getUniformLocation(this.updateProgram, 'uFade'), + (this.quality === 'high' ? 0.965 : this.quality === 'low' ? 0.92 : 0.945) - buildBoost + dropBoost, + ) + this.draw(this.updateProgram) + + gl.bindFramebuffer(gl.FRAMEBUFFER, null) + gl.viewport(0, 0, this.width, this.height) + gl.useProgram(this.blitProgram) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.write.tex) + gl.uniform1i(gl.getUniformLocation(this.blitProgram, 'uTex'), 0) + this.draw(this.blitProgram) + + var tmp = this.read + this.read = this.write + this.write = tmp + } + + SpiritSimulator.prototype.tick = function () { + if (!this.running) return + this.paused = Boolean(window.__STEMENGINE_MUON_PAUSED__) + this.setQuality(window.__STEMENGINE_MUON_QUALITY__) + var vp = window.__STEMENGINE_MUON_VIEWPORT__ + if (vp) this.setViewport(vp.width, vp.height) + this.step() + var self = this + this.rafId = requestAnimationFrame(function () { self.tick() }) + } + + SpiritSimulator.prototype.start = function () { + if (this.running) return + this.running = true + this.tick() + } + + SpiritSimulator.prototype.stop = function () { + this.running = false + if (this.rafId != null) { + cancelAnimationFrame(this.rafId) + this.rafId = null + } + } + + SpiritSimulator.prototype.dispose = function () { + var gl = this.gl + this.stop() + if (this.quad) gl.deleteBuffer(this.quad) + if (this.updateProgram) gl.deleteProgram(this.updateProgram) + if (this.blitProgram) gl.deleteProgram(this.blitProgram) + if (this.read) { + gl.deleteTexture(this.read.tex) + gl.deleteFramebuffer(this.read.fbo) + } + if (this.write) { + gl.deleteTexture(this.write.tex) + gl.deleteFramebuffer(this.write.fbo) + } + } + + window.__StemEngineSpiritRenderGraph__ = SpiritSimulator + console.log('[StemEngine][Spirit] Render graph module loaded') +})() diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/stemengine-bootstrap.js b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/stemengine-bootstrap.js new file mode 100644 index 0000000..5bfdc68 --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/stemengine-bootstrap.js @@ -0,0 +1,96 @@ +/* StemEngine runtime bootstrap injected into Muon build at image build-time. */ + +(function stemengineBootstrap() { + const REQUIRED_EXTENSIONS = ['OES_texture_float_linear', 'EXT_color_buffer_float'] + + function installContextCaptureHook() { + const originalGetContext = HTMLCanvasElement.prototype.getContext + if (typeof originalGetContext !== 'function') return + if (HTMLCanvasElement.prototype.__stemengineContextHookInstalled__) return + + HTMLCanvasElement.prototype.getContext = function patchedGetContext(type, attrs) { + const ctx = originalGetContext.call(this, type, attrs) + if ((type === 'webgl2' || type === 'webgl') && ctx && this.isConnected) { + if (!window.__STEMENGINE_MUON_GL_CTX__) { + window.__STEMENGINE_MUON_GL_CTX__ = ctx + } + if (!window.__STEMENGINE_MUON_CANVAS__) { + window.__STEMENGINE_MUON_CANVAS__ = this + } + } + return ctx + } + + Object.defineProperty(HTMLCanvasElement.prototype, '__stemengineContextHookInstalled__', { + value: true, + configurable: true, + writable: false, + enumerable: false, + }) + } + + function checkBaseline() { + const canvas = document.createElement('canvas') + const gl = canvas.getContext('webgl2') || canvas.getContext('webgl') + if (!gl) { + console.warn('[StemEngine][Muon] WebGL unavailable; fallback mode should be used.') + return { + ok: false, + missing: ['webgl'], + } + } + + const missing = REQUIRED_EXTENSIONS.filter((name) => !gl.getExtension(name)) + if (missing.length > 0) { + console.warn('[StemEngine][Muon] Missing required WebGL extensions:', missing.join(', ')) + return { ok: false, missing } + } + + return { ok: true, missing: [] } + } + + function installContextLifecycleHooks() { + let retries = 0 + const maxRetries = 120 + + const attach = () => { + const canvas = document.querySelector('canvas.webgl') || document.querySelector('canvas') + if (!canvas) { + retries += 1 + if (retries <= maxRetries) requestAnimationFrame(attach) + return + } + + canvas.addEventListener('webglcontextlost', (event) => { + event.preventDefault() + console.warn('[StemEngine][Muon] webglcontextlost') + window.__STEMENGINE_MUON_CONTEXT_LOST__ = true + }) + + canvas.addEventListener('webglcontextrestored', () => { + console.info('[StemEngine][Muon] webglcontextrestored') + window.__STEMENGINE_MUON_CONTEXT_LOST__ = false + }) + } + + attach() + } + + if (typeof window.__STEMENGINE_MUON_STOP_FN__ !== 'function') { + window.__STEMENGINE_MUON_STOP_FN__ = function stemengineNoopStop() {} + } + if (typeof window.__STEMENGINE_MUON_RESUME_FN__ !== 'function') { + window.__STEMENGINE_MUON_RESUME_FN__ = function stemengineNoopResume() {} + } + + installContextCaptureHook() + + const baseline = checkBaseline() + window.__STEMENGINE_MUON_WEBGL_BASELINE__ = baseline + + if (!baseline.ok) { + document.documentElement.classList.add('stemengine-muon-degraded') + } + + installContextLifecycleHooks() +})() diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/stemengine-router.js b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/stemengine-router.js new file mode 100644 index 0000000..9d6ccbf --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/stemengine-router.js @@ -0,0 +1,356 @@ +/* StemEngine — style router + postMessage command listener. + * Injected into the Muon build at image build-time via Dockerfile. + * + * Project policy: zero access control restrictions on this path. Origin validation + * is intentionally a no-op. See docs/plans/2026-04-27_Muon_Default_Mode_Reactivity/ + * README.md section 8 for the rationale. + */ +;(function stemengineRouter() { + function isAllowedOrigin(_origin) { + return true + } + + function postToParent(message) { + // Always emit with wildcard and rely on host-side origin checks. + // This avoids stale single-origin target mismatches during mixed-origin dev workflows. + parent.postMessage(message, '*') + } + + // ------------------------------------------------------------------------- + // Helpers + // ------------------------------------------------------------------------- + + function getStyleParam() { + try { + return new URLSearchParams(location.search).get('style') || 'default' + } catch (_) { + return 'default' + } + } + + /** Clamp a value to [lo, hi]; coerce to number, NaN → 0. */ + function clamp(v, lo, hi) { + var n = Number(v) + if (!isFinite(n)) n = 0 + return Math.max(lo, Math.min(hi, n)) + } + + var KNOWN_COMMANDS = { + setStyle: true, + setPreset: true, + setQuality: true, + setPaused: true, + setViewport: true, + pushAudioFeatures: true, + loadAudio: true, + transportPlay: true, + transportPause: true, + transportSeek: true, + transportRate: true, + } + + // ------------------------------------------------------------------------- + // Audio feature state — host-push model. + // The host sends pushAudioFeatures at its RAF cadence; we rate-limit to + // one application per our own RAF tick so we never block the render loop. + // ------------------------------------------------------------------------- + + var rafPending = false + var pendingFeatures = null + + function captureFeatures(payload) { + if (!payload || typeof payload !== 'object') return + pendingFeatures = { + bass: clamp(payload.bass, 0, 1), + treble: clamp(payload.treble, 0, 1), + rms: clamp(payload.rms, 0, 1), + energy: clamp(payload.energy, 0, 1), + beat: Boolean(payload.beat), + drumIntensity: clamp(payload.drumIntensity, 0, 1), + sub: clamp(payload.sub, 0, 1), + beatPhase: clamp(payload.beatPhase, 0, 1), + bpm: Number.isFinite(Number(payload.bpm)) ? Number(payload.bpm) : 0, + beatConfidence: clamp(payload.beatConfidence, 0, 1), + isBeatDrop: Boolean(payload.isBeatDrop), + isBuildUp: Boolean(payload.isBuildUp), + isSilence: Boolean(payload.isSilence), + energyDelta: Number.isFinite(Number(payload.energyDelta)) ? Number(payload.energyDelta) : 0, + spectralCentroid: clamp(payload.spectralCentroid, 0, 1), + spectralFlatness: clamp(payload.spectralFlatness, 0, 1), + frequencyBands: payload.frequencyBands || null, + } + } + + function flushFeatures() { + if (!pendingFeatures) return + window.__STEMENGINE_MUON_AUDIO__ = pendingFeatures + pendingFeatures = null + } + + // ------------------------------------------------------------------------- + // Render graph lifecycle + // ------------------------------------------------------------------------- + + var currentRenderGraph = null + var lastStyleValue = null + + function updateRenderGraph(style) { + // Only instantiate when style actually changes or on first call. + if (style === lastStyleValue) return + lastStyleValue = style + + // Dispose previous render graph (if any). + if (currentRenderGraph && typeof currentRenderGraph.dispose === 'function') { + try { + currentRenderGraph.dispose() + } catch (e) { + console.warn('[StemEngine][Muon] Error disposing render graph:', e) + } + } + currentRenderGraph = null + + if (style === 'default') { + if (typeof window.__STEMENGINE_MUON_RESUME_FN__ === 'function') { + try { + window.__STEMENGINE_MUON_RESUME_FN__() + } catch (e) { + console.warn('[StemEngine][Muon] Error resuming default renderer:', e) + } + } + return + } + + if (typeof window.__STEMENGINE_MUON_STOP_FN__ === 'function') { + try { + window.__STEMENGINE_MUON_STOP_FN__() + } catch (e) { + console.warn('[StemEngine][Muon] Error stopping default renderer:', e) + } + } + + var viewport = window.__STEMENGINE_MUON_VIEWPORT__ || {} + var width = Number(viewport.width) || undefined + var height = Number(viewport.height) || undefined + var quality = window.__STEMENGINE_MUON_QUALITY__ || 'balanced' + + // Instantiate new render graph based on style. + if (style === 'fluid' && typeof window.__StemEngineFluidRenderGraph__ === 'function') { + try { + currentRenderGraph = new window.__StemEngineFluidRenderGraph__(window.__STEMENGINE_MUON_GL_CTX__, width, height, quality) + console.log('[StemEngine][Muon] Instantiated Fluid render graph') + } catch (e) { + console.error('[StemEngine][Muon] Error instantiating Fluid render graph:', e) + window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e)) + } + } else if (style === 'tendrils' && typeof window.__StemEngineTendrilsRenderGraph__ === 'function') { + try { + currentRenderGraph = new window.__StemEngineTendrilsRenderGraph__(window.__STEMENGINE_MUON_CANVAS__, window.__STEMENGINE_MUON_GL_CTX__, width, height, quality) + console.log('[StemEngine][Muon] Instantiated Tendrils render graph') + } catch (e) { + console.error('[StemEngine][Muon] Error instantiating Tendrils render graph:', e) + window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e)) + } + } else if (style === 'spirit' && typeof window.__StemEngineSpiritRenderGraph__ === 'function') { + try { + currentRenderGraph = new window.__StemEngineSpiritRenderGraph__(window.__STEMENGINE_MUON_CANVAS__, window.__STEMENGINE_MUON_GL_CTX__, width, height, quality) + console.log('[StemEngine][Muon] Instantiated Spirit render graph') + } catch (e) { + console.error('[StemEngine][Muon] Error instantiating Spirit render graph:', e) + window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e)) + } + } + } + + // ------------------------------------------------------------------------- + // Command dispatch + // ------------------------------------------------------------------------- + + function handleCommand(type, payload) { + var p = payload || {} + switch (type) { + case 'setStyle': + window.__STEMENGINE_MUON_STYLE__ = String(p.style || 'default') + updateRenderGraph(window.__STEMENGINE_MUON_STYLE__) + break + case 'setQuality': + window.__STEMENGINE_MUON_QUALITY__ = String(p.quality || 'balanced') + break + case 'setPreset': + window.__STEMENGINE_MUON_PRESET__ = String(p.preset || '') + break + case 'setPaused': + window.__STEMENGINE_MUON_PAUSED__ = Boolean(p.paused) + break + case 'setViewport': + var cssWidth = clamp(p.cssWidth != null ? p.cssWidth : p.width, 1, 7680) + var cssHeight = clamp(p.cssHeight != null ? p.cssHeight : p.height, 1, 4320) + var dprRaw = Number(p.dpr) + var dpr = isFinite(dprRaw) && dprRaw > 0 ? clamp(dprRaw, 0.25, 8) : 1 + var pixelWidth = Math.max(1, Math.round(cssWidth * dpr)) + var pixelHeight = Math.max(1, Math.round(cssHeight * dpr)) + window.__STEMENGINE_MUON_VIEWPORT__ = { + cssWidth: cssWidth, + cssHeight: cssHeight, + dpr: dpr, + width: pixelWidth, + height: pixelHeight, + } + break + case 'pushAudioFeatures': + captureFeatures(p) + if (!rafPending) { + rafPending = true + requestAnimationFrame(function () { + rafPending = false + flushFeatures() + }) + } + break + case 'loadAudio': + if (!window.__StemEngineAudioEngine__ || typeof window.__StemEngineAudioEngine__.loadFromUrl !== 'function') break + window.__StemEngineAudioEngine__.loadFromUrl(String(p.url || '')) + .then(function (info) { + if (window.__StemEngineFeatureExtractor__ && typeof window.__StemEngineFeatureExtractor__.start === 'function') { + window.__StemEngineFeatureExtractor__.start() + } + postToParent({ + version: 1, + id: 'loaded-' + Date.now(), + type: 'loaded', + payload: { + duration: info && Number.isFinite(Number(info.duration)) ? Number(info.duration) : 0, + }, + }) + }) + .catch(function (err) { + window.__stemengineEmitError__('AUDIO_LOAD', String(err && err.message ? err.message : err)) + }) + break + case 'transportPlay': + if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.play === 'function') { + window.__StemEngineAudioEngine__.play({ + time: Number(p.time) || 0, + rate: Number(p.rate) || 1, + }) + } + break + case 'transportPause': + if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.pause === 'function') { + window.__StemEngineAudioEngine__.pause() + } + break + case 'transportSeek': + if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.seek === 'function') { + var nextTime = Number(p.time) || 0 + window.__StemEngineAudioEngine__.seek(nextTime) + + // Recovery path: if host is actively playing but a transportPlay command was + // dropped/raced, seek updates keep arriving. Auto-resume once from seek. + if ( + !window.__STEMENGINE_MUON_PAUSED__ + && typeof window.__StemEngineAudioEngine__.isPlaying === 'function' + && !window.__StemEngineAudioEngine__.isPlaying() + && typeof window.__StemEngineAudioEngine__.play === 'function' + ) { + var resumeRate = Number(p.rate) + if (!Number.isFinite(resumeRate) || resumeRate <= 0) { + resumeRate = typeof window.__StemEngineAudioEngine__.getRate === 'function' + ? Number(window.__StemEngineAudioEngine__.getRate()) || 1 + : 1 + } + window.__StemEngineAudioEngine__.play({ + time: nextTime, + rate: resumeRate, + }) + } + } + break + case 'transportRate': + if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.setRate === 'function') { + window.__StemEngineAudioEngine__.setRate(Number(p.rate) || 1) + } + break + } + } + + // ------------------------------------------------------------------------- + // postMessage listener + // ------------------------------------------------------------------------- + + window.addEventListener('message', function onStemEngineMessage(event) { + // Origin guard — reject messages from unexpected origins. + if (!isAllowedOrigin(event.origin)) return + + var msg = event.data + if (!msg || typeof msg !== 'object') return + // Schema: { version: 1, type: string, payload: object } + if (Number(msg.version) !== 1) return + var type = String(msg.type || '') + if (!KNOWN_COMMANDS[type]) return + + handleCommand(type, msg.payload) + }) + + // ------------------------------------------------------------------------- + // Emit ready to host + // ------------------------------------------------------------------------- + + function emitReady() { + var baseline = window.__STEMENGINE_MUON_WEBGL_BASELINE__ || { ok: false, missing: [] } + var payload = { + capabilities: { + webgl: Boolean(baseline.ok), + missingExtensions: baseline.missing || [], + audioMode: 'hybrid', + }, + styles: ['fluid', 'tendrils', 'spirit', 'default'], + } + try { + postToParent({ version: 1, id: 'ready-' + Date.now(), type: 'ready', payload: payload }) + } catch (e) { + console.warn('[StemEngine][Muon] Could not emit ready:', e) + } + } + + // ------------------------------------------------------------------------- + // Emit error to host (exported for other modules to call) + // ------------------------------------------------------------------------- + + window.__stemengineEmitError__ = function emitError(code, message) { + try { + postToParent({ + version: 1, + id: 'error-' + Date.now(), + type: 'error', + payload: { code: String(code || 'UNKNOWN'), message: String(message || '') }, + }) + } catch (_) {} + } + + // ------------------------------------------------------------------------- + // Bootstrap + // ------------------------------------------------------------------------- + + // Initialize globals from query params so the render graph can read them + // on its first frame even before any postMessage arrives. + window.__STEMENGINE_MUON_STYLE__ = getStyleParam() + window.__STEMENGINE_MUON_QUALITY__ = 'balanced' + window.__STEMENGINE_MUON_PAUSED__ = false + window.__STEMENGINE_MUON_AUDIO__ = null + window.__STEMENGINE_MUON_VIEWPORT__ = null + + function onReady() { + if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.initialize === 'function') { + window.__StemEngineAudioEngine__.initialize() + } + emitReady() + // Instantiate the initial render graph based on the style query param. + updateRenderGraph(window.__STEMENGINE_MUON_STYLE__) + } + + if (document.readyState === 'loading') { + document.addEventListener('DOMContentLoaded', onReady) + } else { + onReady() + } +})() diff --git a/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/tendrils-render-graph.js b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/tendrils-render-graph.js new file mode 100644 index 0000000..acf0b8a --- /dev/null +++ b/docs/muon-mode-status/export/muon-mode-review-package-2026-04-28/source-evidence/tendrils-render-graph.js @@ -0,0 +1,265 @@ +/* StemEngine Tendrils Render Graph (Muon fork overlay) + * Particle trail / feedback visualization with audio reactivity. + */ + +;(function stemengineTendrilsRenderGraph() { + 'use strict' + + var QUALITY_SCALE = { low: 0.75, balanced: 1.0, high: 1.35 } + + function clamp01(v) { + var n = Number(v) + if (!isFinite(n)) n = 0 + return Math.max(0, Math.min(1, n)) + } + + function shaped(v, power) { + return Math.pow(clamp01(v), power) + } + + function getBand(audio, key, fallback) { + var bands = audio && audio.frequencyBands ? audio.frequencyBands : null + var value = bands && isFinite(Number(bands[key])) ? Number(bands[key]) : fallback + return clamp01(value) + } + + function createProgram(gl, vsSrc, fsSrc) { + function compile(type, src) { + var s = gl.createShader(type) + gl.shaderSource(s, src) + gl.compileShader(s) + if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) { + var err = gl.getShaderInfoLog(s) || 'shader compile failed' + window.__stemengineEmitError__('TENDRILS_SHADER', err) + gl.deleteShader(s) + return null + } + return s + } + var vs = compile(gl.VERTEX_SHADER, vsSrc) + var fs = compile(gl.FRAGMENT_SHADER, fsSrc) + if (!vs || !fs) return null + var p = gl.createProgram() + gl.attachShader(p, vs) + gl.attachShader(p, fs) + gl.linkProgram(p) + gl.deleteShader(vs) + gl.deleteShader(fs) + if (!gl.getProgramParameter(p, gl.LINK_STATUS)) { + var err2 = gl.getProgramInfoLog(p) || 'program link failed' + window.__stemengineEmitError__('TENDRILS_PROGRAM', err2) + gl.deleteProgram(p) + return null + } + return p + } + + function createTarget(gl, w, h) { + var tex = gl.createTexture() + gl.bindTexture(gl.TEXTURE_2D, tex) + gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, w, h, 0, gl.RGBA, gl.UNSIGNED_BYTE, null) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE) + gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE) + + var fbo = gl.createFramebuffer() + gl.bindFramebuffer(gl.FRAMEBUFFER, fbo) + gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0) + gl.bindFramebuffer(gl.FRAMEBUFFER, null) + return { tex: tex, fbo: fbo, width: w, height: h } + } + + function TendrilsSimulator(canvas, gl, width, height, quality) { + this.canvas = canvas || document.querySelector('canvas') + this.gl = gl || (this.canvas && (this.canvas.getContext('webgl2', { alpha: false, antialias: false }) || this.canvas.getContext('webgl', { alpha: false, antialias: false }))) + if (!this.gl) throw new Error('[StemEngine][Tendrils] Unable to acquire WebGL context') + + this.width = Math.max(1, Number(width) || this.canvas.clientWidth || this.canvas.width || 512) + this.height = Math.max(1, Number(height) || this.canvas.clientHeight || this.canvas.height || 512) + this.quality = String(quality || window.__STEMENGINE_MUON_QUALITY__ || 'balanced') + this.paused = false + this.rafId = null + this.running = false + + var gl = this.gl + var vs = 'attribute vec2 aPos; varying vec2 vUv; void main(){ vUv=aPos*0.5+0.5; gl_Position=vec4(aPos,0.0,1.0);} ' + var updateFs = [ + 'precision highp float;', + 'uniform sampler2D uPrev;', + 'uniform vec2 uRes;', + 'uniform float uTime;', + 'uniform float uBass;', + 'uniform float uTreble;', + 'uniform float uRms;', + 'uniform float uBeat;', + 'uniform float uTrailDecay;', + 'varying vec2 vUv;', + 'float hash(vec2 p){ return fract(sin(dot(p, vec2(127.1,311.7))) * 43758.5453); }', + 'void main(){', + ' vec2 uv=vUv;', + ' vec2 c=uv-0.5;', + ' float r=max(length(c), 1e-4);', + ' float a=atan(c.y,c.x);', + ' float flow=sin(a*8.0 + uTime*(1.6+uBass*4.0)) + cos(r*45.0 + uTime*(1.1+uTreble*5.0));', + ' vec2 dir=normalize(vec2(cos(a+flow*0.45), sin(a+flow*0.45)));', + ' vec2 offset=dir*(0.0015 + 0.0035*uBass) + vec2(hash(uv+uTime)-0.5, hash(uv+uTime*1.7)-0.5)*(0.002*uTreble);', + ' vec3 prev=texture2D(uPrev, clamp(uv-offset, 0.0, 1.0)).rgb * uTrailDecay;', + ' float line=exp(-abs(sin((a*12.0 + r*120.0) + uTime*6.0*(1.0+uTreble)))*26.0);', + ' float beatBurst=uBeat * exp(-r*4.0);', + ' vec3 colorA=vec3(0.2+uBass*0.8, 0.2+uRms*0.7, 0.4+uTreble*0.6);', + ' vec3 colorB=vec3(0.9,0.6+uTreble*0.35,0.2+uBass*0.4);', + ' vec3 inject=mix(colorA, colorB, clamp(r*1.5, 0.0, 1.0))*(line*0.3 + beatBurst*0.8 + uRms*0.05);', + ' gl_FragColor=vec4(clamp(prev + inject, 0.0, 1.0), 1.0);', + '}' + ].join('\n') + var blitFs = 'precision mediump float; uniform sampler2D uTex; varying vec2 vUv; void main(){ gl_FragColor = texture2D(uTex, vUv); }' + + this.updateProgram = createProgram(gl, vs, updateFs) + this.blitProgram = createProgram(gl, vs, blitFs) + + var quad = new Float32Array([-1,-1, 1,-1, -1,1, -1,1, 1,-1, 1,1]) + this.quad = gl.createBuffer() + gl.bindBuffer(gl.ARRAY_BUFFER, this.quad) + gl.bufferData(gl.ARRAY_BUFFER, quad, gl.STATIC_DRAW) + + this.read = null + this.write = null + this.allocateTargets() + this.start() + } + + TendrilsSimulator.prototype.allocateTargets = function () { + var gl = this.gl + if (this.read) { + gl.deleteTexture(this.read.tex) + gl.deleteFramebuffer(this.read.fbo) + } + if (this.write) { + gl.deleteTexture(this.write.tex) + gl.deleteFramebuffer(this.write.fbo) + } + var scale = QUALITY_SCALE[this.quality] || QUALITY_SCALE.balanced + var w = Math.max(64, Math.floor(this.width * scale)) + var h = Math.max(64, Math.floor(this.height * scale)) + this.read = createTarget(gl, w, h) + this.write = createTarget(gl, w, h) + } + + TendrilsSimulator.prototype.setQuality = function (q) { + var next = String(q || 'balanced') + if (next === this.quality) return + this.quality = next + this.allocateTargets() + } + + TendrilsSimulator.prototype.setViewport = function (w, h) { + var nw = Math.max(1, Number(w) || this.width) + var nh = Math.max(1, Number(h) || this.height) + if (nw === this.width && nh === this.height) return + this.width = nw + this.height = nh + if (this.canvas) { + this.canvas.width = nw + this.canvas.height = nh + } + this.allocateTargets() + } + + TendrilsSimulator.prototype.drawQuad = function (program) { + var gl = this.gl + gl.useProgram(program) + gl.bindBuffer(gl.ARRAY_BUFFER, this.quad) + var loc = gl.getAttribLocation(program, 'aPos') + gl.enableVertexAttribArray(loc) + gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0) + gl.drawArrays(gl.TRIANGLES, 0, 6) + } + + TendrilsSimulator.prototype.step = function () { + if (this.paused || !this.read || !this.write) return + var gl = this.gl + var t = performance.now() * 0.001 + var audio = window.__STEMENGINE_MUON_AUDIO__ || {} + var bass = shaped(0.6 * getBand(audio, 'bass', audio.bass || 0) + 0.4 * getBand(audio, 'sub', audio.sub || 0), 0.62) + var treble = shaped(0.55 * getBand(audio, 'presence', audio.treble || 0) + 0.45 * getBand(audio, 'brilliance', audio.treble || 0), 0.66) + var rms = shaped(clamp01(audio.rms || 0), 0.8) + var beat = audio.beat ? 1 : 0 + var buildBoost = audio.isBuildUp ? 0.12 : 0 + var dropBoost = audio.isBeatDrop ? 0.2 : 0 + + gl.bindFramebuffer(gl.FRAMEBUFFER, this.write.fbo) + gl.viewport(0, 0, this.write.width, this.write.height) + gl.useProgram(this.updateProgram) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.read.tex) + gl.uniform1i(gl.getUniformLocation(this.updateProgram, 'uPrev'), 0) + gl.uniform2f(gl.getUniformLocation(this.updateProgram, 'uRes'), this.write.width, this.write.height) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTime'), t) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBass'), bass) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTreble'), treble) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uRms'), rms) + gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBeat'), beat) + gl.uniform1f( + gl.getUniformLocation(this.updateProgram, 'uTrailDecay'), + (0.90 + (this.quality === 'high' ? 0.08 : this.quality === 'low' ? 0.04 : 0.06)) - buildBoost + dropBoost, + ) + this.drawQuad(this.updateProgram) + + gl.bindFramebuffer(gl.FRAMEBUFFER, null) + gl.viewport(0, 0, this.width, this.height) + gl.useProgram(this.blitProgram) + gl.activeTexture(gl.TEXTURE0) + gl.bindTexture(gl.TEXTURE_2D, this.write.tex) + gl.uniform1i(gl.getUniformLocation(this.blitProgram, 'uTex'), 0) + this.drawQuad(this.blitProgram) + + var tmp = this.read + this.read = this.write + this.write = tmp + } + + TendrilsSimulator.prototype.tick = function () { + if (!this.running) return + this.paused = Boolean(window.__STEMENGINE_MUON_PAUSED__) + this.setQuality(window.__STEMENGINE_MUON_QUALITY__) + var vp = window.__STEMENGINE_MUON_VIEWPORT__ + if (vp) this.setViewport(vp.width, vp.height) + this.step() + var self = this + this.rafId = requestAnimationFrame(function () { self.tick() }) + } + + TendrilsSimulator.prototype.start = function () { + if (this.running) return + this.running = true + this.tick() + } + + TendrilsSimulator.prototype.stop = function () { + this.running = false + if (this.rafId != null) { + cancelAnimationFrame(this.rafId) + this.rafId = null + } + } + + TendrilsSimulator.prototype.dispose = function () { + var gl = this.gl + this.stop() + if (this.quad) gl.deleteBuffer(this.quad) + if (this.updateProgram) gl.deleteProgram(this.updateProgram) + if (this.blitProgram) gl.deleteProgram(this.blitProgram) + if (this.read) { + gl.deleteTexture(this.read.tex) + gl.deleteFramebuffer(this.read.fbo) + } + if (this.write) { + gl.deleteTexture(this.write.tex) + gl.deleteFramebuffer(this.write.fbo) + } + } + + window.__StemEngineTendrilsRenderGraph__ = TendrilsSimulator + console.log('[StemEngine][Tendrils] Render graph module loaded') +})() diff --git a/docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md b/docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md new file mode 100644 index 0000000..f598e4f --- /dev/null +++ b/docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md @@ -0,0 +1,346 @@ +# Local Iframe Visualizer Architecture Plan + +Date: 2026-04-28 +Status: Draft implementation plan +Owner: StemEngine frontend / visualizer pipeline + +## Problem Statement +The current attempt to approximate `fluid`, `tendrils`, and `spirit` inside the existing Muon iframe path did not deliver a reliable result. + +Observed issues: +- `default` mode is the only mode currently behaving close to the intended repeat-play iframe flow. +- `fluid` shows reproducible replay instability and framebuffer warnings. +- `tendrils` and `spirit` may look visually in-family, but are still bound to Muon-specific lifecycle assumptions that make them hard to validate and stabilize. + +## Goal +Use the same host-to-iframe transport model for every visualizer style, but move non-default styles onto a local iframe runtime that behaves like Muon from the host’s point of view. + +Target outcome: +- the host speaks one iframe transport contract, +- each style runs in an iframe-managed renderer lifecycle, +- replay / pause / seek / rate / resize are validated the same way for all styles, +- non-default styles are no longer dependent on Muon internals that were designed for the default renderer. + +## Decision Summary +Recommended architecture: +- Keep the existing Muon iframe provider for `default` for now. +- Build a local iframe visualizer runtime for `fluid`, `tendrils`, and `spirit`. +- Standardize the transport contract so the host treats both as iframe visualizer providers. + +Preferred shape: +- One shared local iframe shell with pluggable style engines. + +Fallback shape: +- One local iframe app per style. + +Recommendation: +- Use one shared local iframe shell unless style-specific boot constraints prove otherwise. + +## Why This Direction +Benefits: +- separates transport concerns from render math, +- removes style replay behavior from Muon private/default renderer assumptions, +- reduces duplicate host logic, +- gives one lifecycle model for all local styles, +- makes validation and third-party review clearer. + +## Scope +In scope: +- iframe transport contract standardization, +- host visualizer provider abstraction, +- local iframe shell runtime, +- local renderer plugin API, +- migration of `fluid`, `tendrils`, and `spirit` to local iframe shell, +- validation matrix across Track Detail, Visualizer Lab, and other exposed surfaces. + +Out of scope for first pass: +- replacing default Muon renderer, +- perfect fidelity to third-party reference demos, +- broad preset-authoring UX, +- camera/mic/advanced interaction parity with original external demos. + +## Current File Touch Points +### Host-side frontend +Likely primary touch points: +- `frontend/src/hooks/useMuonIframeTransport.ts` +- `frontend/src/components/VisualizerPreviewWidget.tsx` +- `frontend/src/components/LyricsLivePreview.tsx` +- `frontend/src/pages/VisualizerLabPage.tsx` +- `frontend/src/pages/TrackDetail.tsx` + +Likely new host-side additions: +- `frontend/src/visualizers/providers/*` +- `frontend/src/hooks/useIframeVisualizerTransport.ts` +- `frontend/src/types/iframeVisualizer.ts` + +### Existing Muon iframe runtime +Reference implementation / source of reusable logic: +- `muon-iframe/stemengine-router.js` +- `muon-iframe/stemengine-bootstrap.js` +- `muon-iframe/stemengine-audio-engine.js` +- `muon-iframe/stemengine-feature-extractor.js` + +### Existing style renderers to migrate or adapt +- `muon-iframe/fluid-render-graph.js` +- `muon-iframe/tendrils-render-graph.js` +- `muon-iframe/spirit-render-graph.js` + +### Likely new local iframe app area +Suggested location: +- `frontend/local-visualizer-iframe/` or +- `local-visualizer-iframe/` + +Recommendation: +- keep it separate from `muon-iframe/` so it is not build-time injected into Muon. + +## Target Architecture +### 1. Host visualizer provider layer +The host should target a provider abstraction, not Muon directly. + +Proposed provider model: +- provider id +- iframe URL +- supported styles +- capabilities +- command protocol version +- fallback behavior + +Example providers: +- `muon-default` +- `local-visualizer` + +### 2. Shared iframe transport contract +Commands from host to iframe: +- `init` +- `setStyle` +- `setPreset` +- `setQuality` +- `setPaused` +- `setViewport` +- `loadAudio` +- `transportPlay` +- `transportPause` +- `transportSeek` +- `transportRate` +- `pushAudioFeatures` +- `dispose` + +Events from iframe to host: +- `ready` +- `loaded` +- `capabilities` +- `state` +- `warning` +- `error` + +Required rules: +- host owns transport intent and source audio selection, +- iframe owns rendering lifecycle, +- seek while playing must not leave renderer visually paused, +- replay after pause must be deterministic, +- viewport updates must not require reload. + +### 3. Local iframe shell +Responsibilities: +- boot and emit `ready`, +- set up audio engine, +- set up feature extractor, +- receive commands from host, +- instantiate exactly one renderer at a time, +- dispose renderer cleanly on style switch or teardown, +- expose warnings/errors back to host. + +Shell should be style-agnostic. + +### 4. Renderer plugin API +Each style renderer should conform to the same API. + +Proposed renderer interface: +- `init(context)` +- `setViewport(viewport)` +- `setQuality(quality)` +- `setPreset(preset)` +- `setPaused(paused)` +- `pushFeatures(features)` +- `tick(time)` +- `dispose()` + +Renderer should not own: +- transport protocol, +- source audio loading, +- play/pause/seek intent, +- message routing. + +### 5. Migration model +Near-term routing: +- `default` -> existing Muon iframe provider +- `fluid` -> local iframe shell +- `tendrils` -> local iframe shell +- `spirit` -> local iframe shell + +Future option: +- move `default` under the same provider system later if beneficial. + +## Implementation Phases +## Phase 1: Freeze the contract +Deliverables: +- one transport spec document, +- one provider abstraction design, +- explicit replay/seek/resize semantics. + +Tasks: +- document current useful Muon messages, +- remove Muon-specific assumptions from the conceptual transport model, +- define one versioned protocol for all iframe visualizers. + +Exit criteria: +- host and iframe responsibilities are explicitly defined, +- no new style-specific transport behavior is added ad hoc. + +## Phase 2: Introduce provider abstraction on host +Deliverables: +- provider registry, +- host hook abstracted away from Muon naming, +- widgets/pages able to select a provider. + +Tasks: +- refactor `useMuonIframeTransport.ts` into provider-neutral transport hook, +- add provider config for default Muon and local iframe runtime, +- update consumers in preview surfaces. + +Exit criteria: +- Track Detail and Visualizer Lab can load an iframe provider without hard-coding Muon-only assumptions. + +## Phase 3: Build local iframe shell +Deliverables: +- local iframe app booting independently, +- postMessage router, +- audio engine and feature extractor wiring, +- renderer lifecycle manager. + +Tasks: +- clone only the reusable runtime concepts from current Muon integration, +- avoid build-time injection into third-party Muon source, +- add explicit renderer registration and disposal. + +Exit criteria: +- shell loads, emits ready, can accept audio load/play/pause/seek/viewport commands even with a placeholder renderer. + +## Phase 4: Migrate `fluid` +Why first: +- it is the closest structural match to its intended reference, +- it is currently the least reliable mode. + +Tasks: +- adapt current fluid renderer to the new renderer API, +- enforce init-time capability checks, +- surface explicit degrade/failure states when FBO support is incomplete, +- validate replay and resize in the new shell. + +Exit criteria: +- `fluid` passes two-cycle replay test in Track Detail, +- no silent frozen-canvas failure path, +- FBO failure produces explicit host-visible warning/error. + +## Phase 5: Migrate `tendrils` and `spirit` +Tasks: +- adapt both renderers to shared shell lifecycle, +- remove transport ownership assumptions from render graphs, +- validate style switch, replay, resize, and quality transitions. + +Exit criteria: +- both renderers pass same iframe lifecycle tests as `fluid`, +- no per-style transport branching on host. + +## Phase 6: Cross-surface validation and rollout +Tasks: +- validate Track Detail, +- validate Visualizer Lab with a loaded song path, +- validate any project/embed surface that exposes iframe preview, +- update docs and reviewer package with post-migration results. + +Exit criteria: +- the same style behaves consistently across supported surfaces, +- unsupported surfaces are explicitly documented. + +## Validation Matrix +Each migrated style must pass: +- iframe boot / ready +- audio load +- play cycle 1 +- pause +- replay cycle 2 +- seek while playing +- seek while paused +- rate change +- viewport resize +- quality change +- style switch away and back +- teardown / remount + +Required test surfaces: +- Track Detail +- Visualizer Lab +- any active project/embed preview surface + +## Key Risks +1. Host abstraction drift +Risk: +- partial refactor leaves old Muon-only codepaths mixed with provider abstraction. +Mitigation: +- centralize provider selection in one registry and one transport hook. + +2. Fluid hardware variability +Risk: +- local fluid renderer still fails on certain WebGL / extension combinations. +Mitigation: +- explicit capability detection and degrade behavior, not silent failure. + +3. Renderer lifecycle leaks +Risk: +- RAF loops, textures, or framebuffers survive style switches. +Mitigation: +- strict renderer API and disposable lifecycle manager in shell. + +4. Visualizer Lab remains blocked +Risk: +- shell exists but lab still cannot load audio cleanly. +Mitigation: +- define one supported sample-input path early in Phase 3. + +5. Fidelity pressure before lifecycle stability +Risk: +- development time is spent tuning visuals before replay and resize are correct. +Mitigation: +- block fidelity work until lifecycle acceptance criteria pass. + +## Deliverables Checklist +- transport spec doc +- provider registry +- provider-neutral iframe transport hook +- local iframe shell app +- renderer plugin API +- migrated `fluid` +- migrated `tendrils` +- migrated `spirit` +- updated validation docs +- updated review package + +## Recommended Execution Order +1. Freeze transport spec +2. Refactor host to provider model +3. Build local iframe shell with placeholder renderer +4. Migrate `fluid` +5. Stabilize replay / resize / seek +6. Migrate `tendrils` +7. Migrate `spirit` +8. Re-run cross-surface validation +9. Update external review package + +## Success Definition +This plan succeeds when: +- non-default styles no longer rely on Muon’s default-renderer internals, +- host uses one iframe transport model across providers, +- replay behavior is deterministic for all supported styles, +- failures are surfaced explicitly instead of appearing as frozen visuals, +- reviewers can evaluate styles through a consistent iframe lifecycle rather than special-case paths.