Myriad changes.

This commit is contained in:
translator committed 2026-04-28 09:48:37 +09:00
1 parent 519b2c8dcd
commit 1bb503f735
19 files changed
+4058

No files matched your search

@@ -0,0 +1,78 @@
# Muon Visualizer Current Per-Mode Functionality
Date: 2026-04-27
Environment: StemEngine track preview (http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb)
Mode set under test: `default`, `fluid`, `tendrils`, `spirit`
## Scope
This document captures the CURRENT observed behavior for Muon style modes in Track Detail preview, with focus on repeat-play reactivity.
Reactivity definition used here:
- cycle 1 reactive: visual output changes after first Play preview
- cycle 2 reactive: visual output still changes after Pause -> Play replay
## Summary
- `default`: functional with caveats
- `fluid`: reproducible bug on replay (cycle 2 non-reactive)
- `tendrils`: functional in current sweep, but keep under watch (user-reported instability exists)
- `spirit`: functional in current sweep, but keep under watch (user-reported instability exists)
## Per-Mode Status
### default
Status: Functional with caveats
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: reactive (3/3 runs)
Caveats:
- Intermittent stream-layer noise still appears in session logs/events (`ERR_ABORTED`, occasional `502`), but did not break reactivity in this sweep.
### fluid
Status: Buggy (reproducible)
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: NOT reactive (3/3 runs)
Evidence:
- Motion diff for cycle 2 repeatedly dropped to `0`
- Console repeatedly logs: `[StemEngine][Fluid] FBO incomplete: 36054`
Impact:
- Replay reliability is broken for fluid mode under current build/runtime combination.
### tendrils
Status: Functional in current session (watchlist)
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: reactive (3/3 runs)
Caveat:
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
### spirit
Status: Functional in current session (watchlist)
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: reactive (3/3 runs)
Caveat:
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
## Raw Sweep Snapshot (Automated)
Run 1:
- default: c1 reactive, c2 reactive
- fluid: c1 reactive, c2 non-reactive
- tendrils: c1 reactive, c2 reactive
- spirit: c1 reactive, c2 reactive
Run 2:
- default: c1 reactive, c2 reactive
- fluid: c1 reactive, c2 non-reactive
- tendrils: c1 reactive, c2 reactive
- spirit: c1 reactive, c2 reactive
Run 3:
- default: c1 reactive, c2 reactive
- fluid: c1 reactive, c2 non-reactive
- tendrils: c1 reactive, c2 reactive
- spirit: c1 reactive, c2 reactive
## Notes
- This report is intentionally point-in-time. Re-run after renderer/transport changes to keep this file current.
- If you want, add a companion matrix file for Visualizer Lab page and project-page embed to compare behavior across surfaces.
@@ -0,0 +1,95 @@
# Muon Review Export Bundle Manifest
Date: 2026-04-28
Purpose: frozen manifest for external sharing of the Muon mode review package.
## Bundle Name
Suggested bundle label:
- `muon-mode-review-package-2026-04-28`
## Bundle Scope
This export bundle is intended for third-party reviewers who need:
- current operational status by mode,
- surface-by-surface testability notes,
- reference-comparison analysis,
- a reviewer scoring rubric,
- a single index file for navigation.
## Files To Include
### Core review docs
1. `docs/muon-mode-status/README.md`
- Entry point and reading order.
2. `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- Point-in-time Track Detail mode functionality and replay observations.
3. `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
- Surface matrix across Track Detail, Visualizer Lab, and project/embed views.
4. `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
- Reference comparison for `fluid`, `tendrils`, and `spirit`.
5. `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
- Third-party scoring and signoff rubric.
### Optional supporting source files
Include these if the reviewer needs implementation-level evidence:
6. `muon-iframe/fluid-render-graph.js`
7. `muon-iframe/tendrils-render-graph.js`
8. `muon-iframe/spirit-render-graph.js`
9. `muon-iframe/stemengine-router.js`
10. `muon-iframe/stemengine-bootstrap.js`
## Minimal Bundle
If the recipient should receive docs only, export:
- `docs/muon-mode-status/README.md`
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
## Full Evidence Bundle
If the recipient should receive both review docs and implementation evidence, export:
- all files from the Minimal Bundle
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
## Suggested Folder Layout For Sharing
```text
muon-mode-review-package-2026-04-28/
README.md
docs/
2026-04-27-current-per-mode-functionality.md
2026-04-28-surface-functionality-matrix.md
2026-04-28-reference-comparison-third-party-review.md
2026-04-28-reviewer-checklist.md
source-evidence/
fluid-render-graph.js
tendrils-render-graph.js
spirit-render-graph.js
stemengine-router.js
stemengine-bootstrap.js
```
## Export Notes
- This manifest freezes the intended review set as of 2026-04-28.
- The bundle is evidence-oriented, not a claim of source-equivalent parity with external references.
- Reference-comparison conclusions include inference and confidence levels; reviewers should read those caveats before scoring fidelity.
## Reviewer Guidance
Recommended reading sequence inside the exported bundle:
1. `README.md`
2. `2026-04-28-reference-comparison-third-party-review.md`
3. `2026-04-27-current-per-mode-functionality.md`
4. `2026-04-28-surface-functionality-matrix.md`
5. `2026-04-28-reviewer-checklist.md`
## Current Package Status
As of this manifest:
- `fluid` remains the main concern because of replay/FBO reliability issues.
- `tendrils` and `spirit` are documented as in-family approximations, not proven literal ports.
- Visualizer Lab and project/embed surfaces remain limited or non-testable for this exact replay matrix without additional setup.
@@ -0,0 +1,141 @@
# Muon Mode Comparison vs Reference Demos (Third-Party Review)
Date: 2026-04-28
Prepared for: external/third-party technical review
Scope: infer how local `fluid`, `tendrils`, and `spirit` modes compare to their likely inspiration references.
References under review:
- https://paveldogreat.github.io/WebGL-Fluid-Simulation/
- https://keeffeoghan.github.io/work/tendrils/ (redirects to `epok.tech` content)
- https://edankwan.com/experiments/the-spirit/
Local implementation inputs reviewed:
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
## Method Notes
This review intentionally separates:
- verified local facts (from readable source files), and
- inferred reference characteristics (from live pages + publicly visible descriptions + agent-assisted analysis).
A `ui-designer` subagent was used to produce an independent reference summary, then cross-checked against local code.
## Executive Summary
1. `fluid` in this fork is structurally the closest to its reference target (PavelDoGreat fluid simulation), because it implements a recognizable multi-pass fluid pipeline (splat, divergence, pressure Jacobi, gradient subtract, advection, bloom).
2. `tendrils` and `spirit` in this fork are stylistic approximations, not faithful reproductions of the original interactive systems. They are compact shader-feedback variants optimized for integration into Muon’s transport/audio stack.
3. The local `fluid` path currently has a practical stability/replay issue in your environment (`FBO incomplete: 36054` warnings and cycle-2 non-reactivity), which materially affects parity with the expected “real thing” feel.
## Comparison Matrix
| Mode | Local Implementation (verified) | Reference “Real Thing” (inferred) | Similarity Assessment |
|---|---|---|---|
| fluid | Full-screen multipass FBO fluid pipeline with pressure solve, advection, bloom, and audio splat injection | Interactive Navier-Stokes style GPU fluid sim with user splats and polished post FX | High structural similarity; medium experiential similarity due to current stability issues and integration constraints |
| tendrils | Feedback texture + procedural flow/noise line structures in one update shader, audio-driven decay/flow | Emergent particle-field ecosystem with richer control surface, presets, and interaction channels (keyboard/camera/mic) | Medium visual similarity; low-to-medium system similarity |
| spirit | Layered atmospheric feedback shader with fbm/noise, halo/strand/pulse composition, audio-driven fade | Highly polished ethereal particle/trail experience with strong cinematic motion language | Medium stylistic similarity; low implementation-parity confidence |
## Per-Mode Detailed Assessment
### A) `fluid`
Local verified facts:
- Uses explicit fluid-solver style passes:
- curl, divergence, pressure Jacobi, gradient subtract, velocity advection, dye advection.
- Uses ping-pong FBOs for velocity/dye/pressure and optional bloom chain by quality tier.
- Audio coupling includes bass/sub/treble/rms/beat/beatPhase/drumIntensity mapped to splat radius/force/color.
- Quality tier controls pressure iterations and bloom enablement.
Evidence (local code):
- `muon-iframe/fluid-render-graph.js` (pipeline and FBO management)
- `muon-iframe/stemengine-router.js` (style lifecycle + audio feature push)
Reference inferred characteristics:
- Canonical GPU fluid simulation style with strong interactive splat behavior and stable continuous flow.
Assessment:
- Architecture match is strong.
- Main gap is operational reliability in your current stack: observed `FBO incomplete: 36054` warnings and replay-cycle failures reduce real-world parity.
Confidence:
- Local implementation confidence: High
- Reference internals confidence: Medium
- Similarity conclusion confidence: Medium-High
### B) `tendrils`
Local verified facts:
- Two-program design (`update` + `blit`) with framebuffer feedback loop.
- Motion built from angle/radius flow fields + hash noise + trail decay.
- Audio mapping primarily modulates flow rate, turbulence, color injection, and decay.
- No explicit particle-buffer architecture exposed; behavior emerges from texture feedback math.
Evidence (local code):
- `muon-iframe/tendrils-render-graph.js`
Reference inferred characteristics:
- Public text describes interacting particle system with advection-like behavior, optional optical flow/camera/mic interactions, rich controls and presets.
Assessment:
- The local mode captures the “organic tendril motion” aesthetic.
- It appears materially simplified versus the broader interaction/control ecosystem and potentially deeper particle mechanics of the reference.
Confidence:
- Local implementation confidence: High
- Reference internals confidence: Medium-Low
- Similarity conclusion confidence: Medium
### C) `spirit`
Local verified facts:
- Two-pass feedback renderer with atmospheric layering:
- fbm-like noise term,
- radial halo decay,
- strand/pulse functions,
- additive glow and nonlinear output shaping.
- Audio mapping influences drift, fade persistence, pulse intensity, and color blend.
Evidence (local code):
- `muon-iframe/spirit-render-graph.js`
Reference inferred characteristics:
- Highly polished ethereal motion piece with strong artistic direction and likely richer scene dynamics.
Assessment:
- The local mode appears designed as a stylistic homage (ethereal layered glow) rather than a strict algorithmic clone.
- Without source-level access to reference internals, parity can only be judged at visual-language level.
Confidence:
- Local implementation confidence: High
- Reference internals confidence: Low-Medium
- Similarity conclusion confidence: Medium-Low
## Integration Context (Important for Reviewers)
The local renderers are not standalone art demos; they are embedded into Muon transport and controlled by host commands.
Observed integration traits:
- Router-driven style switching and lifecycle disposal/instantiation.
- Host-driven audio feature push throttled to iframe RAF cadence.
- Host-driven pause/play/seek/rate semantics through injected audio engine.
- Baseline WebGL extension checks in bootstrap layer.
Implication:
- Even when visual math resembles a reference demo, behavior can diverge due to transport timing, iframe lifecycle, and browser/driver extension availability.
## Known Gaps and Risks
1. Reference internals are partly opaque (minified/demo-only), so comparison is inference-based, not source-equivalence.
2. `fluid` currently shows environment-specific fragility (`FBO incomplete: 36054` warnings observed in session), impacting practical parity.
3. `tendrils`/`spirit` appear intentionally compact; they likely trade breadth/depth of original interactivity for predictable embedding in Muon.
## Third-Party Review Conclusion
- If the review criterion is “same algorithm/system as originals”: not demonstrated for `tendrils` or `spirit`; only partial for `fluid` (closest).
- If criterion is “same visual family integrated into Muon with audio transport”: yes, with `fluid` currently hindered by reliability caveats.
- Recommended interpretation: these are reference-inspired implementations with varying structural closeness, not literal ports.
## Suggested Evidence Bundle for Auditors
Include these files in review packets:
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
@@ -0,0 +1,152 @@
# Muon Reviewer Checklist
Date: 2026-04-28
Purpose: quick third-party parity and functionality review for Muon `fluid`, `tendrils`, and `spirit` modes.
Related documents:
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
## How To Use
For each mode, mark:
- PASS: clearly meets the criterion
- PARTIAL: present but degraded, unstable, or notably simplified
- FAIL: absent, broken, or materially unlike expected behavior
- N/A: not exposed on the reviewed surface
Recommended review surfaces:
- Track Detail preview
- Visualizer Lab
- Any project/embed surface exposing Muon controls
## Reviewer Metadata
- Reviewer:
- Date:
- Browser / version:
- GPU / driver:
- Surface tested:
- Audio source used:
- Quality tier tested:
## Section 1: Basic Functionality
### Mode availability
- `fluid` is selectable
- `tendrils` is selectable
- `spirit` is selectable
- Mode switch does not visibly break iframe/canvas
### Playback / transport
- First play starts visual motion
- Pause stops or meaningfully freezes progression
- Replay after pause resumes visual motion
- Seek while playing preserves visual responsiveness
- Mode remains reactive for at least two play cycles
### Stability
- No blank canvas on mode switch
- No persistent frozen frame after replay
- No repeated fatal errors in console
- No WebGL context loss during normal use
## Section 2: Visual Parity vs Reference
### `fluid`
Expected reference family:
- Fluid-like motion with continuous advection
- Visible diffusion / swirling / dye transport feel
- Stronger impulse on beats or major peaks
- Cohesive full-frame flow rather than isolated sprite effects
Review marks:
- Flow behavior resembles fluid simulation
- Beat/bass energy visibly changes motion intensity
- Color field behaves like transported dye, not only flashing overlays
- Motion remains continuous across replay cycles
- Presentation feels structurally close to the PavelDoGreat reference
Overall `fluid` score: PASS / PARTIAL / FAIL
Notes:
### `tendrils`
Expected reference family:
- Organic line or filament structures
- Emergent motion with feedback / trace persistence
- Sense of interconnected or fibrous forms
- More than a simple radial pulse or stock waveform
Review marks:
- Visual output forms tendril/filament-like structures
- Trails persist and evolve over time
- Motion feels emergent rather than strictly canned
- Audio affects density, color, or movement in a noticeable way
- Presentation feels recognizably related to the Tendrils reference family
Overall `tendrils` score: PASS / PARTIAL / FAIL
Notes:
### `spirit`
Expected reference family:
- Ethereal / atmospheric glow language
- Layered soft motion rather than sharp geometric tracing
- Strong sense of aura, halo, strand, smoke, or apparition-like motion
- Audio-linked pulse or drift without collapsing into a basic particle burst
Review marks:
- Visual output feels atmospheric / spectral / ethereal
- Glow and layering read clearly on screen
- Audio changes drift, pulse, or luminosity in a visible way
- Motion remains smooth across play / pause / replay
- Presentation feels recognizably related to the Spirit reference family
Overall `spirit` score: PASS / PARTIAL / FAIL
Notes:
## Section 3: Surface Matrix Check
Mark each surface:
### Track Detail
- `fluid`: PASS / PARTIAL / FAIL / N/A
- `tendrils`: PASS / PARTIAL / FAIL / N/A
- `spirit`: PASS / PARTIAL / FAIL / N/A
Notes:
### Visualizer Lab
- `fluid`: PASS / PARTIAL / FAIL / N/A
- `tendrils`: PASS / PARTIAL / FAIL / N/A
- `spirit`: PASS / PARTIAL / FAIL / N/A
Notes:
### Project / embed surface
- `fluid`: PASS / PARTIAL / FAIL / N/A
- `tendrils`: PASS / PARTIAL / FAIL / N/A
- `spirit`: PASS / PARTIAL / FAIL / N/A
Notes:
## Section 4: Red Flags
Check any that apply:
- Replay fails only on some modes
- Mode works on first play but freezes on second play
- Console shows framebuffer / FBO warnings
- Visuals appear materially simpler than reference expectations
- Surface exposes controls but is not actually testable
- Behavior differs substantially between surfaces
## Section 5: Final Verdict
Use this rubric:
- PASS: mode is stable and recognizably in the same reference family
- PARTIAL: mode works but has obvious reliability, fidelity, or surface gaps
- FAIL: mode is broken or not meaningfully comparable to its reference family
Final verdict by mode:
- `fluid`:
- `tendrils`:
- `spirit`:
Overall release recommendation:
- APPROVE for review
- APPROVE WITH CAVEATS
- DO NOT APPROVE
Summary comments:
@@ -0,0 +1,65 @@
# Muon Surface Functionality Matrix
Date: 2026-04-28
Surfaces covered:
- Track Detail preview (`/tracks/:trackId`)
- Visualizer Lab (`/visualizer-lab`)
- Project page (`/projects/:projectId` on webui host)
## Legend
- PASS: verified reactive in current session
- FAIL: reproducible non-reactive behavior in current session
- BLOCKED: surface/control exists but cannot execute mode replay test without additional input/state
- N/A: control path for this Muon mode test is not exposed on this surface
## Matrix (Current)
| Surface | default | fluid | tendrils | spirit | Notes |
|---|---|---|---|---|---|
| Track Detail preview | PASS | FAIL | PASS | PASS | `fluid` fails on cycle 2 replay (non-reactive), others reactive in same run |
| Visualizer Lab | BLOCKED | BLOCKED | BLOCKED | BLOCKED | Style combobox is present, but song is not loaded (`Song: None selected`), play/reset disabled |
| Project page embed | N/A | N/A | N/A | N/A | No Muon replay controls or mode selector exposed in this project surface snapshot |
## Evidence Snapshot
### 1) Track Detail (`http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb`)
Single sweep results (cycle1/cycle2):
- default: reactive / reactive
- fluid: reactive / non-reactive
- tendrils: reactive / reactive
- spirit: reactive / reactive
Representative diffs:
- default: c1=20141, c2=23362
- fluid: c1=1552, c2=0
- tendrils: c1=43459, c2=45607
- spirit: c1=9952, c2=9313
Known caveat in session:
- intermittent stream/network noise still appears in browser events (`ERR_ABORTED`, occasional `502`), but does not explain the mode-specific fluid replay failure by itself.
### 2) Visualizer Lab (`http://work:3001/visualizer-lab`)
Observed:
- Muon iframe present
- Mode style options are available via `Style` combobox (`default`, `fluid`, `tendrils`, `spirit`)
- `Song: None selected`
- `Play preview` is disabled
Implication:
- per-mode replay reactivity cannot be validated until an audio source is loaded.
### 3) Project page (`http://work:5174/projects/e98f2b8a-0cad-4b4e-b33e-1ca2c8c87a0a`)
Observed in current snapshot:
- No Muon iframe preview controls for replay testing
- No mode buttons (`default`, `fluid`, `tendrils`, `spirit`) exposed
- No preview play/pause controls exposed for this Muon test path
Implication:
- Mode-by-mode replay matrix is N/A on this surface as currently rendered.
## Delta vs Prior Note
See prior Track Detail-focused note for earlier run context:
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
This file extends that status into a cross-surface matrix and explicitly marks where tests are blocked or not applicable.
+87
View File
@@ -0,0 +1,87 @@
# Muon Mode Status Review Index
Date: 2026-04-28
Purpose: single entry point for current Muon mode functionality, surface coverage, reference comparison, and third-party review.
## Document Set
### 1. Current Per-Mode Functionality
File:
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
Use this first for:
- Track Detail preview mode-by-mode status
- Current replay behavior by mode
- Quick snapshot of what is working vs buggy
Current headline:
- `default`: functional with caveats
- `fluid`: reproducible replay bug
- `tendrils`: functional in tested session, watchlist
- `spirit`: functional in tested session, watchlist
### 2. Cross-Surface Matrix
File:
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
Use this for:
- Comparing Track Detail, Visualizer Lab, and project/embed surfaces
- Seeing which surfaces are testable, blocked, or N/A
- Understanding where mode controls are or are not exposed
Current headline:
- Track Detail is testable and currently shows `fluid` replay failure
- Visualizer Lab is blocked without a loaded song
- Project page does not currently expose this Muon replay test path
### 3. Reference Comparison for Third-Party Review
File:
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
Use this for:
- Evaluating how local `fluid`, `tendrils`, and `spirit` compare to reference demos
- Distinguishing verified local facts from inferred reference traits
- Reviewing confidence levels and known uncertainty
Current headline:
- `fluid` is the closest structural match to its reference
- `tendrils` and `spirit` are better described as reference-inspired approximations than literal ports
### 4. Reviewer Checklist
File:
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
Use this for:
- PASS / PARTIAL / FAIL / N/A scoring
- Consistent third-party review rubric
- Capturing final approval / caveat decisions
## Recommended Reading Order
1. `2026-04-27-current-per-mode-functionality.md`
2. `2026-04-28-surface-functionality-matrix.md`
3. `2026-04-28-reference-comparison-third-party-review.md`
4. `2026-04-28-reviewer-checklist.md`
## Reviewer Short Summary
If a reviewer reads only one page first, start with:
- `2026-04-28-reference-comparison-third-party-review.md`
If a reviewer wants only operational status first, start with:
- `2026-04-27-current-per-mode-functionality.md`
If a reviewer is validating reproducibility across UI surfaces, start with:
- `2026-04-28-surface-functionality-matrix.md`
## Current Overall Assessment
- `fluid`: closest to reference in structure, but currently weakened by replay/FBO reliability issues
- `tendrils`: visually in-family, but likely a simplified embedded variant rather than full reference parity
- `spirit`: visually in-family, but should be treated as stylistic approximation unless stronger source-level evidence emerges
## Evidence Scope Notice
This folder contains a mix of:
- direct source-code review,
- live browser observation,
- runtime probing,
- inference from public reference pages.
It should be treated as a technical review package, not a claim of source-equivalent implementation.
@@ -0,0 +1,100 @@
# Muon Mode Status Review Index
Date: 2026-04-28
Purpose: single entry point for current Muon mode functionality, surface coverage, reference comparison, and third-party review.
## Document Set
### 1. Current Per-Mode Functionality
File:
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
Use this first for:
- Track Detail preview mode-by-mode status
- Current replay behavior by mode
- Quick snapshot of what is working vs buggy
Current headline:
- `default`: functional with caveats
- `fluid`: reproducible replay bug
- `tendrils`: functional in tested session, watchlist
- `spirit`: functional in tested session, watchlist
### 2. Cross-Surface Matrix
File:
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
Use this for:
- Comparing Track Detail, Visualizer Lab, and project/embed surfaces
- Seeing which surfaces are testable, blocked, or N/A
- Understanding where mode controls are or are not exposed
Current headline:
- Track Detail is testable and currently shows `fluid` replay failure
- Visualizer Lab is blocked without a loaded song
- Project page does not currently expose this Muon replay test path
### 3. Reference Comparison for Third-Party Review
File:
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
Use this for:
- Evaluating how local `fluid`, `tendrils`, and `spirit` compare to reference demos
- Distinguishing verified local facts from inferred reference traits
- Reviewing confidence levels and known uncertainty
Current headline:
- `fluid` is the closest structural match to its reference
- `tendrils` and `spirit` are better described as reference-inspired approximations than literal ports
### 4. Reviewer Checklist
File:
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
Use this for:
- PASS / PARTIAL / FAIL / N/A scoring
- Consistent third-party review rubric
- Capturing final approval / caveat decisions
### 5. Local Iframe Architecture Plan
File:
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
Use this for:
- The implementation path for moving non-default styles onto a local Muon-like iframe flow
- Phase-by-phase execution guidance
- Host/iframe touch points, risks, and validation gates
## Recommended Reading Order
1. `2026-04-27-current-per-mode-functionality.md`
2. `2026-04-28-surface-functionality-matrix.md`
3. `2026-04-28-reference-comparison-third-party-review.md`
4. `2026-04-28-reviewer-checklist.md`
5. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
## Reviewer Short Summary
If a reviewer reads only one page first, start with:
- `2026-04-28-reference-comparison-third-party-review.md`
If a reviewer wants only operational status first, start with:
- `2026-04-27-current-per-mode-functionality.md`
If a reviewer is validating reproducibility across UI surfaces, start with:
- `2026-04-28-surface-functionality-matrix.md`
If a reviewer is planning implementation work from these findings, continue with:
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
## Current Overall Assessment
- `fluid`: closest to reference in structure, but currently weakened by replay/FBO reliability issues
- `tendrils`: visually in-family, but likely a simplified embedded variant rather than full reference parity
- `spirit`: visually in-family, but should be treated as stylistic approximation unless stronger source-level evidence emerges
## Evidence Scope Notice
This folder contains a mix of:
- direct source-code review,
- live browser observation,
- runtime probing,
- inference from public reference pages.
It should be treated as a technical review package, not a claim of source-equivalent implementation.
@@ -0,0 +1,78 @@
# Muon Visualizer Current Per-Mode Functionality
Date: 2026-04-27
Environment: StemEngine track preview (http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb)
Mode set under test: `default`, `fluid`, `tendrils`, `spirit`
## Scope
This document captures the CURRENT observed behavior for Muon style modes in Track Detail preview, with focus on repeat-play reactivity.
Reactivity definition used here:
- cycle 1 reactive: visual output changes after first Play preview
- cycle 2 reactive: visual output still changes after Pause -> Play replay
## Summary
- `default`: functional with caveats
- `fluid`: reproducible bug on replay (cycle 2 non-reactive)
- `tendrils`: functional in current sweep, but keep under watch (user-reported instability exists)
- `spirit`: functional in current sweep, but keep under watch (user-reported instability exists)
## Per-Mode Status
### default
Status: Functional with caveats
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: reactive (3/3 runs)
Caveats:
- Intermittent stream-layer noise still appears in session logs/events (`ERR_ABORTED`, occasional `502`), but did not break reactivity in this sweep.
### fluid
Status: Buggy (reproducible)
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: NOT reactive (3/3 runs)
Evidence:
- Motion diff for cycle 2 repeatedly dropped to `0`
- Console repeatedly logs: `[StemEngine][Fluid] FBO incomplete: 36054`
Impact:
- Replay reliability is broken for fluid mode under current build/runtime combination.
### tendrils
Status: Functional in current session (watchlist)
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: reactive (3/3 runs)
Caveat:
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
### spirit
Status: Functional in current session (watchlist)
Observed behavior:
- Cycle 1: reactive (3/3 runs)
- Cycle 2: reactive (3/3 runs)
Caveat:
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
## Raw Sweep Snapshot (Automated)
Run 1:
- default: c1 reactive, c2 reactive
- fluid: c1 reactive, c2 non-reactive
- tendrils: c1 reactive, c2 reactive
- spirit: c1 reactive, c2 reactive
Run 2:
- default: c1 reactive, c2 reactive
- fluid: c1 reactive, c2 non-reactive
- tendrils: c1 reactive, c2 reactive
- spirit: c1 reactive, c2 reactive
Run 3:
- default: c1 reactive, c2 reactive
- fluid: c1 reactive, c2 non-reactive
- tendrils: c1 reactive, c2 reactive
- spirit: c1 reactive, c2 reactive
## Notes
- This report is intentionally point-in-time. Re-run after renderer/transport changes to keep this file current.
- If you want, add a companion matrix file for Visualizer Lab page and project-page embed to compare behavior across surfaces.
@@ -0,0 +1,104 @@
# Muon Review Export Bundle Manifest
Date: 2026-04-28
Purpose: frozen manifest for external sharing of the Muon mode review package.
## Bundle Name
Suggested bundle label:
- `muon-mode-review-package-2026-04-28`
## Bundle Scope
This export bundle is intended for third-party reviewers who need:
- current operational status by mode,
- surface-by-surface testability notes,
- reference-comparison analysis,
- a reviewer scoring rubric,
- a single index file for navigation.
## Files To Include
### Core review docs
1. `docs/muon-mode-status/README.md`
- Entry point and reading order.
2. `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- Point-in-time Track Detail mode functionality and replay observations.
3. `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
- Surface matrix across Track Detail, Visualizer Lab, and project/embed views.
4. `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
- Reference comparison for `fluid`, `tendrils`, and `spirit`.
5. `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
- Third-party scoring and signoff rubric.
6. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
- Implementation plan for moving non-default styles to a local iframe visualizer runtime.
### Optional supporting source files
Include these if the reviewer needs implementation-level evidence:
7. `muon-iframe/fluid-render-graph.js`
8. `muon-iframe/tendrils-render-graph.js`
9. `muon-iframe/spirit-render-graph.js`
10. `muon-iframe/stemengine-router.js`
11. `muon-iframe/stemengine-bootstrap.js`
## Minimal Bundle
If the recipient should receive docs only, export:
- `docs/muon-mode-status/README.md`
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
## Full Evidence Bundle
If the recipient should receive both review docs and implementation evidence, export:
- all files from the Minimal Bundle
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
## Suggested Folder Layout For Sharing
```text
muon-mode-review-package-2026-04-28/
README.md
docs/
2026-04-27-current-per-mode-functionality.md
2026-04-28-export-bundle-manifest.md
2026-04-28-surface-functionality-matrix.md
2026-04-28-reference-comparison-third-party-review.md
2026-04-28-reviewer-checklist.md
plans/
2026-04-28_local-iframe-visualizer-architecture-plan.md
source-evidence/
fluid-render-graph.js
tendrils-render-graph.js
spirit-render-graph.js
stemengine-router.js
stemengine-bootstrap.js
```
## Export Notes
- This manifest freezes the intended review set as of 2026-04-28.
- The bundle is evidence-oriented, not a claim of source-equivalent parity with external references.
- Reference-comparison conclusions include inference and confidence levels; reviewers should read those caveats before scoring fidelity.
## Reviewer Guidance
Recommended reading sequence inside the exported bundle:
1. `README.md`
2. `2026-04-28-reference-comparison-third-party-review.md`
3. `2026-04-27-current-per-mode-functionality.md`
4. `2026-04-28-surface-functionality-matrix.md`
5. `2026-04-28-reviewer-checklist.md`
6. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
## Current Package Status
As of this manifest:
- `fluid` remains the main concern because of replay/FBO reliability issues.
- `tendrils` and `spirit` are documented as in-family approximations, not proven literal ports.
- Visualizer Lab and project/embed surfaces remain limited or non-testable for this exact replay matrix without additional setup.
@@ -0,0 +1,141 @@
# Muon Mode Comparison vs Reference Demos (Third-Party Review)
Date: 2026-04-28
Prepared for: external/third-party technical review
Scope: infer how local `fluid`, `tendrils`, and `spirit` modes compare to their likely inspiration references.
References under review:
- https://paveldogreat.github.io/WebGL-Fluid-Simulation/
- https://keeffeoghan.github.io/work/tendrils/ (redirects to `epok.tech` content)
- https://edankwan.com/experiments/the-spirit/
Local implementation inputs reviewed:
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
## Method Notes
This review intentionally separates:
- verified local facts (from readable source files), and
- inferred reference characteristics (from live pages + publicly visible descriptions + agent-assisted analysis).
A `ui-designer` subagent was used to produce an independent reference summary, then cross-checked against local code.
## Executive Summary
1. `fluid` in this fork is structurally the closest to its reference target (PavelDoGreat fluid simulation), because it implements a recognizable multi-pass fluid pipeline (splat, divergence, pressure Jacobi, gradient subtract, advection, bloom).
2. `tendrils` and `spirit` in this fork are stylistic approximations, not faithful reproductions of the original interactive systems. They are compact shader-feedback variants optimized for integration into Muon’s transport/audio stack.
3. The local `fluid` path currently has a practical stability/replay issue in your environment (`FBO incomplete: 36054` warnings and cycle-2 non-reactivity), which materially affects parity with the expected “real thing” feel.
## Comparison Matrix
| Mode | Local Implementation (verified) | Reference “Real Thing” (inferred) | Similarity Assessment |
|---|---|---|---|
| fluid | Full-screen multipass FBO fluid pipeline with pressure solve, advection, bloom, and audio splat injection | Interactive Navier-Stokes style GPU fluid sim with user splats and polished post FX | High structural similarity; medium experiential similarity due to current stability issues and integration constraints |
| tendrils | Feedback texture + procedural flow/noise line structures in one update shader, audio-driven decay/flow | Emergent particle-field ecosystem with richer control surface, presets, and interaction channels (keyboard/camera/mic) | Medium visual similarity; low-to-medium system similarity |
| spirit | Layered atmospheric feedback shader with fbm/noise, halo/strand/pulse composition, audio-driven fade | Highly polished ethereal particle/trail experience with strong cinematic motion language | Medium stylistic similarity; low implementation-parity confidence |
## Per-Mode Detailed Assessment
### A) `fluid`
Local verified facts:
- Uses explicit fluid-solver style passes:
- curl, divergence, pressure Jacobi, gradient subtract, velocity advection, dye advection.
- Uses ping-pong FBOs for velocity/dye/pressure and optional bloom chain by quality tier.
- Audio coupling includes bass/sub/treble/rms/beat/beatPhase/drumIntensity mapped to splat radius/force/color.
- Quality tier controls pressure iterations and bloom enablement.
Evidence (local code):
- `muon-iframe/fluid-render-graph.js` (pipeline and FBO management)
- `muon-iframe/stemengine-router.js` (style lifecycle + audio feature push)
Reference inferred characteristics:
- Canonical GPU fluid simulation style with strong interactive splat behavior and stable continuous flow.
Assessment:
- Architecture match is strong.
- Main gap is operational reliability in your current stack: observed `FBO incomplete: 36054` warnings and replay-cycle failures reduce real-world parity.
Confidence:
- Local implementation confidence: High
- Reference internals confidence: Medium
- Similarity conclusion confidence: Medium-High
### B) `tendrils`
Local verified facts:
- Two-program design (`update` + `blit`) with framebuffer feedback loop.
- Motion built from angle/radius flow fields + hash noise + trail decay.
- Audio mapping primarily modulates flow rate, turbulence, color injection, and decay.
- No explicit particle-buffer architecture exposed; behavior emerges from texture feedback math.
Evidence (local code):
- `muon-iframe/tendrils-render-graph.js`
Reference inferred characteristics:
- Public text describes interacting particle system with advection-like behavior, optional optical flow/camera/mic interactions, rich controls and presets.
Assessment:
- The local mode captures the “organic tendril motion” aesthetic.
- It appears materially simplified versus the broader interaction/control ecosystem and potentially deeper particle mechanics of the reference.
Confidence:
- Local implementation confidence: High
- Reference internals confidence: Medium-Low
- Similarity conclusion confidence: Medium
### C) `spirit`
Local verified facts:
- Two-pass feedback renderer with atmospheric layering:
- fbm-like noise term,
- radial halo decay,
- strand/pulse functions,
- additive glow and nonlinear output shaping.
- Audio mapping influences drift, fade persistence, pulse intensity, and color blend.
Evidence (local code):
- `muon-iframe/spirit-render-graph.js`
Reference inferred characteristics:
- Highly polished ethereal motion piece with strong artistic direction and likely richer scene dynamics.
Assessment:
- The local mode appears designed as a stylistic homage (ethereal layered glow) rather than a strict algorithmic clone.
- Without source-level access to reference internals, parity can only be judged at visual-language level.
Confidence:
- Local implementation confidence: High
- Reference internals confidence: Low-Medium
- Similarity conclusion confidence: Medium-Low
## Integration Context (Important for Reviewers)
The local renderers are not standalone art demos; they are embedded into Muon transport and controlled by host commands.
Observed integration traits:
- Router-driven style switching and lifecycle disposal/instantiation.
- Host-driven audio feature push throttled to iframe RAF cadence.
- Host-driven pause/play/seek/rate semantics through injected audio engine.
- Baseline WebGL extension checks in bootstrap layer.
Implication:
- Even when visual math resembles a reference demo, behavior can diverge due to transport timing, iframe lifecycle, and browser/driver extension availability.
## Known Gaps and Risks
1. Reference internals are partly opaque (minified/demo-only), so comparison is inference-based, not source-equivalence.
2. `fluid` currently shows environment-specific fragility (`FBO incomplete: 36054` warnings observed in session), impacting practical parity.
3. `tendrils`/`spirit` appear intentionally compact; they likely trade breadth/depth of original interactivity for predictable embedding in Muon.
## Third-Party Review Conclusion
- If the review criterion is “same algorithm/system as originals”: not demonstrated for `tendrils` or `spirit`; only partial for `fluid` (closest).
- If criterion is “same visual family integrated into Muon with audio transport”: yes, with `fluid` currently hindered by reliability caveats.
- Recommended interpretation: these are reference-inspired implementations with varying structural closeness, not literal ports.
## Suggested Evidence Bundle for Auditors
Include these files in review packets:
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
@@ -0,0 +1,152 @@
# Muon Reviewer Checklist
Date: 2026-04-28
Purpose: quick third-party parity and functionality review for Muon `fluid`, `tendrils`, and `spirit` modes.
Related documents:
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
## How To Use
For each mode, mark:
- PASS: clearly meets the criterion
- PARTIAL: present but degraded, unstable, or notably simplified
- FAIL: absent, broken, or materially unlike expected behavior
- N/A: not exposed on the reviewed surface
Recommended review surfaces:
- Track Detail preview
- Visualizer Lab
- Any project/embed surface exposing Muon controls
## Reviewer Metadata
- Reviewer:
- Date:
- Browser / version:
- GPU / driver:
- Surface tested:
- Audio source used:
- Quality tier tested:
## Section 1: Basic Functionality
### Mode availability
- `fluid` is selectable
- `tendrils` is selectable
- `spirit` is selectable
- Mode switch does not visibly break iframe/canvas
### Playback / transport
- First play starts visual motion
- Pause stops or meaningfully freezes progression
- Replay after pause resumes visual motion
- Seek while playing preserves visual responsiveness
- Mode remains reactive for at least two play cycles
### Stability
- No blank canvas on mode switch
- No persistent frozen frame after replay
- No repeated fatal errors in console
- No WebGL context loss during normal use
## Section 2: Visual Parity vs Reference
### `fluid`
Expected reference family:
- Fluid-like motion with continuous advection
- Visible diffusion / swirling / dye transport feel
- Stronger impulse on beats or major peaks
- Cohesive full-frame flow rather than isolated sprite effects
Review marks:
- Flow behavior resembles fluid simulation
- Beat/bass energy visibly changes motion intensity
- Color field behaves like transported dye, not only flashing overlays
- Motion remains continuous across replay cycles
- Presentation feels structurally close to the PavelDoGreat reference
Overall `fluid` score: PASS / PARTIAL / FAIL
Notes:
### `tendrils`
Expected reference family:
- Organic line or filament structures
- Emergent motion with feedback / trace persistence
- Sense of interconnected or fibrous forms
- More than a simple radial pulse or stock waveform
Review marks:
- Visual output forms tendril/filament-like structures
- Trails persist and evolve over time
- Motion feels emergent rather than strictly canned
- Audio affects density, color, or movement in a noticeable way
- Presentation feels recognizably related to the Tendrils reference family
Overall `tendrils` score: PASS / PARTIAL / FAIL
Notes:
### `spirit`
Expected reference family:
- Ethereal / atmospheric glow language
- Layered soft motion rather than sharp geometric tracing
- Strong sense of aura, halo, strand, smoke, or apparition-like motion
- Audio-linked pulse or drift without collapsing into a basic particle burst
Review marks:
- Visual output feels atmospheric / spectral / ethereal
- Glow and layering read clearly on screen
- Audio changes drift, pulse, or luminosity in a visible way
- Motion remains smooth across play / pause / replay
- Presentation feels recognizably related to the Spirit reference family
Overall `spirit` score: PASS / PARTIAL / FAIL
Notes:
## Section 3: Surface Matrix Check
Mark each surface:
### Track Detail
- `fluid`: PASS / PARTIAL / FAIL / N/A
- `tendrils`: PASS / PARTIAL / FAIL / N/A
- `spirit`: PASS / PARTIAL / FAIL / N/A
Notes:
### Visualizer Lab
- `fluid`: PASS / PARTIAL / FAIL / N/A
- `tendrils`: PASS / PARTIAL / FAIL / N/A
- `spirit`: PASS / PARTIAL / FAIL / N/A
Notes:
### Project / embed surface
- `fluid`: PASS / PARTIAL / FAIL / N/A
- `tendrils`: PASS / PARTIAL / FAIL / N/A
- `spirit`: PASS / PARTIAL / FAIL / N/A
Notes:
## Section 4: Red Flags
Check any that apply:
- Replay fails only on some modes
- Mode works on first play but freezes on second play
- Console shows framebuffer / FBO warnings
- Visuals appear materially simpler than reference expectations
- Surface exposes controls but is not actually testable
- Behavior differs substantially between surfaces
## Section 5: Final Verdict
Use this rubric:
- PASS: mode is stable and recognizably in the same reference family
- PARTIAL: mode works but has obvious reliability, fidelity, or surface gaps
- FAIL: mode is broken or not meaningfully comparable to its reference family
Final verdict by mode:
- `fluid`:
- `tendrils`:
- `spirit`:
Overall release recommendation:
- APPROVE for review
- APPROVE WITH CAVEATS
- DO NOT APPROVE
Summary comments:
@@ -0,0 +1,65 @@
# Muon Surface Functionality Matrix
Date: 2026-04-28
Surfaces covered:
- Track Detail preview (`/tracks/:trackId`)
- Visualizer Lab (`/visualizer-lab`)
- Project page (`/projects/:projectId` on webui host)
## Legend
- PASS: verified reactive in current session
- FAIL: reproducible non-reactive behavior in current session
- BLOCKED: surface/control exists but cannot execute mode replay test without additional input/state
- N/A: control path for this Muon mode test is not exposed on this surface
## Matrix (Current)
| Surface | default | fluid | tendrils | spirit | Notes |
|---|---|---|---|---|---|
| Track Detail preview | PASS | FAIL | PASS | PASS | `fluid` fails on cycle 2 replay (non-reactive), others reactive in same run |
| Visualizer Lab | BLOCKED | BLOCKED | BLOCKED | BLOCKED | Style combobox is present, but song is not loaded (`Song: None selected`), play/reset disabled |
| Project page embed | N/A | N/A | N/A | N/A | No Muon replay controls or mode selector exposed in this project surface snapshot |
## Evidence Snapshot
### 1) Track Detail (`http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb`)
Single sweep results (cycle1/cycle2):
- default: reactive / reactive
- fluid: reactive / non-reactive
- tendrils: reactive / reactive
- spirit: reactive / reactive
Representative diffs:
- default: c1=20141, c2=23362
- fluid: c1=1552, c2=0
- tendrils: c1=43459, c2=45607
- spirit: c1=9952, c2=9313
Known caveat in session:
- intermittent stream/network noise still appears in browser events (`ERR_ABORTED`, occasional `502`), but does not explain the mode-specific fluid replay failure by itself.
### 2) Visualizer Lab (`http://work:3001/visualizer-lab`)
Observed:
- Muon iframe present
- Mode style options are available via `Style` combobox (`default`, `fluid`, `tendrils`, `spirit`)
- `Song: None selected`
- `Play preview` is disabled
Implication:
- per-mode replay reactivity cannot be validated until an audio source is loaded.
### 3) Project page (`http://work:5174/projects/e98f2b8a-0cad-4b4e-b33e-1ca2c8c87a0a`)
Observed in current snapshot:
- No Muon iframe preview controls for replay testing
- No mode buttons (`default`, `fluid`, `tendrils`, `spirit`) exposed
- No preview play/pause controls exposed for this Muon test path
Implication:
- Mode-by-mode replay matrix is N/A on this surface as currently rendered.
## Delta vs Prior Note
See prior Track Detail-focused note for earlier run context:
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
This file extends that status into a cross-surface matrix and explicitly marks where tests are blocked or not applicable.
@@ -0,0 +1,346 @@
# Local Iframe Visualizer Architecture Plan
Date: 2026-04-28
Status: Draft implementation plan
Owner: StemEngine frontend / visualizer pipeline
## Problem Statement
The current attempt to approximate `fluid`, `tendrils`, and `spirit` inside the existing Muon iframe path did not deliver a reliable result.
Observed issues:
- `default` mode is the only mode currently behaving close to the intended repeat-play iframe flow.
- `fluid` shows reproducible replay instability and framebuffer warnings.
- `tendrils` and `spirit` may look visually in-family, but are still bound to Muon-specific lifecycle assumptions that make them hard to validate and stabilize.
## Goal
Use the same host-to-iframe transport model for every visualizer style, but move non-default styles onto a local iframe runtime that behaves like Muon from the host’s point of view.
Target outcome:
- the host speaks one iframe transport contract,
- each style runs in an iframe-managed renderer lifecycle,
- replay / pause / seek / rate / resize are validated the same way for all styles,
- non-default styles are no longer dependent on Muon internals that were designed for the default renderer.
## Decision Summary
Recommended architecture:
- Keep the existing Muon iframe provider for `default` for now.
- Build a local iframe visualizer runtime for `fluid`, `tendrils`, and `spirit`.
- Standardize the transport contract so the host treats both as iframe visualizer providers.
Preferred shape:
- One shared local iframe shell with pluggable style engines.
Fallback shape:
- One local iframe app per style.
Recommendation:
- Use one shared local iframe shell unless style-specific boot constraints prove otherwise.
## Why This Direction
Benefits:
- separates transport concerns from render math,
- removes style replay behavior from Muon private/default renderer assumptions,
- reduces duplicate host logic,
- gives one lifecycle model for all local styles,
- makes validation and third-party review clearer.
## Scope
In scope:
- iframe transport contract standardization,
- host visualizer provider abstraction,
- local iframe shell runtime,
- local renderer plugin API,
- migration of `fluid`, `tendrils`, and `spirit` to local iframe shell,
- validation matrix across Track Detail, Visualizer Lab, and other exposed surfaces.
Out of scope for first pass:
- replacing default Muon renderer,
- perfect fidelity to third-party reference demos,
- broad preset-authoring UX,
- camera/mic/advanced interaction parity with original external demos.
## Current File Touch Points
### Host-side frontend
Likely primary touch points:
- `frontend/src/hooks/useMuonIframeTransport.ts`
- `frontend/src/components/VisualizerPreviewWidget.tsx`
- `frontend/src/components/LyricsLivePreview.tsx`
- `frontend/src/pages/VisualizerLabPage.tsx`
- `frontend/src/pages/TrackDetail.tsx`
Likely new host-side additions:
- `frontend/src/visualizers/providers/*`
- `frontend/src/hooks/useIframeVisualizerTransport.ts`
- `frontend/src/types/iframeVisualizer.ts`
### Existing Muon iframe runtime
Reference implementation / source of reusable logic:
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
- `muon-iframe/stemengine-audio-engine.js`
- `muon-iframe/stemengine-feature-extractor.js`
### Existing style renderers to migrate or adapt
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
### Likely new local iframe app area
Suggested location:
- `frontend/local-visualizer-iframe/` or
- `local-visualizer-iframe/`
Recommendation:
- keep it separate from `muon-iframe/` so it is not build-time injected into Muon.
## Target Architecture
### 1. Host visualizer provider layer
The host should target a provider abstraction, not Muon directly.
Proposed provider model:
- provider id
- iframe URL
- supported styles
- capabilities
- command protocol version
- fallback behavior
Example providers:
- `muon-default`
- `local-visualizer`
### 2. Shared iframe transport contract
Commands from host to iframe:
- `init`
- `setStyle`
- `setPreset`
- `setQuality`
- `setPaused`
- `setViewport`
- `loadAudio`
- `transportPlay`
- `transportPause`
- `transportSeek`
- `transportRate`
- `pushAudioFeatures`
- `dispose`
Events from iframe to host:
- `ready`
- `loaded`
- `capabilities`
- `state`
- `warning`
- `error`
Required rules:
- host owns transport intent and source audio selection,
- iframe owns rendering lifecycle,
- seek while playing must not leave renderer visually paused,
- replay after pause must be deterministic,
- viewport updates must not require reload.
### 3. Local iframe shell
Responsibilities:
- boot and emit `ready`,
- set up audio engine,
- set up feature extractor,
- receive commands from host,
- instantiate exactly one renderer at a time,
- dispose renderer cleanly on style switch or teardown,
- expose warnings/errors back to host.
Shell should be style-agnostic.
### 4. Renderer plugin API
Each style renderer should conform to the same API.
Proposed renderer interface:
- `init(context)`
- `setViewport(viewport)`
- `setQuality(quality)`
- `setPreset(preset)`
- `setPaused(paused)`
- `pushFeatures(features)`
- `tick(time)`
- `dispose()`
Renderer should not own:
- transport protocol,
- source audio loading,
- play/pause/seek intent,
- message routing.
### 5. Migration model
Near-term routing:
- `default` -> existing Muon iframe provider
- `fluid` -> local iframe shell
- `tendrils` -> local iframe shell
- `spirit` -> local iframe shell
Future option:
- move `default` under the same provider system later if beneficial.
## Implementation Phases
## Phase 1: Freeze the contract
Deliverables:
- one transport spec document,
- one provider abstraction design,
- explicit replay/seek/resize semantics.
Tasks:
- document current useful Muon messages,
- remove Muon-specific assumptions from the conceptual transport model,
- define one versioned protocol for all iframe visualizers.
Exit criteria:
- host and iframe responsibilities are explicitly defined,
- no new style-specific transport behavior is added ad hoc.
## Phase 2: Introduce provider abstraction on host
Deliverables:
- provider registry,
- host hook abstracted away from Muon naming,
- widgets/pages able to select a provider.
Tasks:
- refactor `useMuonIframeTransport.ts` into provider-neutral transport hook,
- add provider config for default Muon and local iframe runtime,
- update consumers in preview surfaces.
Exit criteria:
- Track Detail and Visualizer Lab can load an iframe provider without hard-coding Muon-only assumptions.
## Phase 3: Build local iframe shell
Deliverables:
- local iframe app booting independently,
- postMessage router,
- audio engine and feature extractor wiring,
- renderer lifecycle manager.
Tasks:
- clone only the reusable runtime concepts from current Muon integration,
- avoid build-time injection into third-party Muon source,
- add explicit renderer registration and disposal.
Exit criteria:
- shell loads, emits ready, can accept audio load/play/pause/seek/viewport commands even with a placeholder renderer.
## Phase 4: Migrate `fluid`
Why first:
- it is the closest structural match to its intended reference,
- it is currently the least reliable mode.
Tasks:
- adapt current fluid renderer to the new renderer API,
- enforce init-time capability checks,
- surface explicit degrade/failure states when FBO support is incomplete,
- validate replay and resize in the new shell.
Exit criteria:
- `fluid` passes two-cycle replay test in Track Detail,
- no silent frozen-canvas failure path,
- FBO failure produces explicit host-visible warning/error.
## Phase 5: Migrate `tendrils` and `spirit`
Tasks:
- adapt both renderers to shared shell lifecycle,
- remove transport ownership assumptions from render graphs,
- validate style switch, replay, resize, and quality transitions.
Exit criteria:
- both renderers pass same iframe lifecycle tests as `fluid`,
- no per-style transport branching on host.
## Phase 6: Cross-surface validation and rollout
Tasks:
- validate Track Detail,
- validate Visualizer Lab with a loaded song path,
- validate any project/embed surface that exposes iframe preview,
- update docs and reviewer package with post-migration results.
Exit criteria:
- the same style behaves consistently across supported surfaces,
- unsupported surfaces are explicitly documented.
## Validation Matrix
Each migrated style must pass:
- iframe boot / ready
- audio load
- play cycle 1
- pause
- replay cycle 2
- seek while playing
- seek while paused
- rate change
- viewport resize
- quality change
- style switch away and back
- teardown / remount
Required test surfaces:
- Track Detail
- Visualizer Lab
- any active project/embed preview surface
## Key Risks
1. Host abstraction drift
Risk:
- partial refactor leaves old Muon-only codepaths mixed with provider abstraction.
Mitigation:
- centralize provider selection in one registry and one transport hook.
2. Fluid hardware variability
Risk:
- local fluid renderer still fails on certain WebGL / extension combinations.
Mitigation:
- explicit capability detection and degrade behavior, not silent failure.
3. Renderer lifecycle leaks
Risk:
- RAF loops, textures, or framebuffers survive style switches.
Mitigation:
- strict renderer API and disposable lifecycle manager in shell.
4. Visualizer Lab remains blocked
Risk:
- shell exists but lab still cannot load audio cleanly.
Mitigation:
- define one supported sample-input path early in Phase 3.
5. Fidelity pressure before lifecycle stability
Risk:
- development time is spent tuning visuals before replay and resize are correct.
Mitigation:
- block fidelity work until lifecycle acceptance criteria pass.
## Deliverables Checklist
- transport spec doc
- provider registry
- provider-neutral iframe transport hook
- local iframe shell app
- renderer plugin API
- migrated `fluid`
- migrated `tendrils`
- migrated `spirit`
- updated validation docs
- updated review package
## Recommended Execution Order
1. Freeze transport spec
2. Refactor host to provider model
3. Build local iframe shell with placeholder renderer
4. Migrate `fluid`
5. Stabilize replay / resize / seek
6. Migrate `tendrils`
7. Migrate `spirit`
8. Re-run cross-surface validation
9. Update external review package
## Success Definition
This plan succeeds when:
- non-default styles no longer rely on Muon’s default-renderer internals,
- host uses one iframe transport model across providers,
- replay behavior is deterministic for all supported styles,
- failures are surfaced explicitly instead of appearing as frozen visuals,
- reviewers can evaluate styles through a consistent iframe lifecycle rather than special-case paths.
@@ -0,0 +1,273 @@
/* StemEngine Spirit Render Graph (Muon fork overlay)
* Atmospheric layered glow visualization with audio reactivity.
*/
;(function stemengineSpiritRenderGraph() {
'use strict'
var QUALITY_SCALE = { low: 0.8, balanced: 1.0, high: 1.25 }
function clamp01(v) {
var n = Number(v)
if (!isFinite(n)) n = 0
return Math.max(0, Math.min(1, n))
}
function shaped(v, power) {
return Math.pow(clamp01(v), power)
}
function getBand(audio, key, fallback) {
var bands = audio && audio.frequencyBands ? audio.frequencyBands : null
var value = bands && isFinite(Number(bands[key])) ? Number(bands[key]) : fallback
return clamp01(value)
}
function compileProgram(gl, vsSrc, fsSrc) {
function c(type, src) {
var s = gl.createShader(type)
gl.shaderSource(s, src)
gl.compileShader(s)
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) {
var e = gl.getShaderInfoLog(s) || 'shader compile failed'
window.__stemengineEmitError__('SPIRIT_SHADER', e)
gl.deleteShader(s)
return null
}
return s
}
var vs = c(gl.VERTEX_SHADER, vsSrc)
var fs = c(gl.FRAGMENT_SHADER, fsSrc)
if (!vs || !fs) return null
var p = gl.createProgram()
gl.attachShader(p, vs)
gl.attachShader(p, fs)
gl.linkProgram(p)
gl.deleteShader(vs)
gl.deleteShader(fs)
if (!gl.getProgramParameter(p, gl.LINK_STATUS)) {
var e2 = gl.getProgramInfoLog(p) || 'program link failed'
window.__stemengineEmitError__('SPIRIT_PROGRAM', e2)
gl.deleteProgram(p)
return null
}
return p
}
function mkTarget(gl, w, h) {
var tex = gl.createTexture()
gl.bindTexture(gl.TEXTURE_2D, tex)
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, w, h, 0, gl.RGBA, gl.UNSIGNED_BYTE, null)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE)
var fbo = gl.createFramebuffer()
gl.bindFramebuffer(gl.FRAMEBUFFER, fbo)
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0)
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
return { tex: tex, fbo: fbo, width: w, height: h }
}
function SpiritSimulator(canvas, gl, width, height, quality) {
this.canvas = canvas || document.querySelector('canvas')
this.gl = gl || (this.canvas && (this.canvas.getContext('webgl2', { alpha: false, antialias: false }) || this.canvas.getContext('webgl', { alpha: false, antialias: false })))
if (!this.gl) throw new Error('[StemEngine][Spirit] Unable to acquire WebGL context')
this.width = Math.max(1, Number(width) || this.canvas.clientWidth || this.canvas.width || 512)
this.height = Math.max(1, Number(height) || this.canvas.clientHeight || this.canvas.height || 512)
this.quality = String(quality || window.__STEMENGINE_MUON_QUALITY__ || 'balanced')
this.paused = false
this.running = false
this.rafId = null
var gl = this.gl
var vs = 'attribute vec2 aPos; varying vec2 vUv; void main(){ vUv=aPos*0.5+0.5; gl_Position=vec4(aPos,0.0,1.0);} '
var updateFs = [
'precision highp float;',
'uniform sampler2D uPrev;',
'uniform vec2 uRes;',
'uniform float uTime;',
'uniform float uBass;',
'uniform float uTreble;',
'uniform float uRms;',
'uniform float uBeat;',
'uniform float uFade;',
'varying vec2 vUv;',
'float hash(vec2 p){ return fract(sin(dot(p, vec2(12.9898,78.233))) * 43758.5453); }',
'float fbm(vec2 p){',
' float v=0.0; float a=0.5;',
' for(int i=0;i<4;i++){',
' v += (hash(floor(p)) * 2.0 - 1.0) * a;',
' p = p*2.03 + 13.1;',
' a *= 0.5;',
' }',
' return v*0.5 + 0.5;',
'}',
'void main(){',
' vec2 uv=vUv;',
' vec2 c=uv-0.5;',
' float r=length(c);',
' float a=atan(c.y,c.x);',
' vec2 drift = vec2(cos(a + uTime*(0.3+uBass*1.8)), sin(a - uTime*(0.22+uTreble*1.4))) * (0.001 + 0.003*uBass);',
' vec3 prev=texture2D(uPrev, clamp(uv-drift,0.0,1.0)).rgb * uFade;',
' float n = fbm(uv*5.0 + vec2(uTime*0.12, -uTime*0.08));',
' float halo = exp(-r*(3.0 + uTreble*3.0));',
' float strand = smoothstep(0.0, 0.08 + uTreble*0.06, abs(sin(a*10.0 + n*9.0 + uTime*(1.8+uRms*4.0))) * (0.55-r));',
' float pulse = uBeat * exp(-r*2.8) + uRms*0.08;',
' vec3 cool = vec3(0.2 + uTreble*0.6, 0.35 + uRms*0.45, 0.8);',
' vec3 warm = vec3(0.95, 0.45 + uBass*0.35, 0.2 + uRms*0.25);',
' vec3 glow = mix(cool, warm, clamp(n + uBass*0.35, 0.0, 1.0)) * (halo*0.35 + strand*0.65 + pulse*0.5);',
' gl_FragColor = vec4(clamp(prev + glow, 0.0, 1.0), 1.0);',
'}'
].join('\n')
var blitFs = 'precision mediump float; uniform sampler2D uTex; varying vec2 vUv; void main(){ vec3 c=texture2D(uTex,vUv).rgb; gl_FragColor=vec4(c*c*0.15 + c,1.0);} '
this.updateProgram = compileProgram(gl, vs, updateFs)
this.blitProgram = compileProgram(gl, vs, blitFs)
var quad = new Float32Array([-1,-1, 1,-1, -1,1, -1,1, 1,-1, 1,1])
this.quad = gl.createBuffer()
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
gl.bufferData(gl.ARRAY_BUFFER, quad, gl.STATIC_DRAW)
this.read = null
this.write = null
this.allocateTargets()
this.start()
}
SpiritSimulator.prototype.allocateTargets = function () {
var gl = this.gl
if (this.read) {
gl.deleteTexture(this.read.tex)
gl.deleteFramebuffer(this.read.fbo)
}
if (this.write) {
gl.deleteTexture(this.write.tex)
gl.deleteFramebuffer(this.write.fbo)
}
var scale = QUALITY_SCALE[this.quality] || QUALITY_SCALE.balanced
var w = Math.max(64, Math.floor(this.width * scale))
var h = Math.max(64, Math.floor(this.height * scale))
this.read = mkTarget(gl, w, h)
this.write = mkTarget(gl, w, h)
}
SpiritSimulator.prototype.setQuality = function (q) {
var next = String(q || 'balanced')
if (next === this.quality) return
this.quality = next
this.allocateTargets()
}
SpiritSimulator.prototype.setViewport = function (w, h) {
var nw = Math.max(1, Number(w) || this.width)
var nh = Math.max(1, Number(h) || this.height)
if (nw === this.width && nh === this.height) return
this.width = nw
this.height = nh
if (this.canvas) {
this.canvas.width = nw
this.canvas.height = nh
}
this.allocateTargets()
}
SpiritSimulator.prototype.draw = function (program) {
var gl = this.gl
gl.useProgram(program)
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
var loc = gl.getAttribLocation(program, 'aPos')
gl.enableVertexAttribArray(loc)
gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0)
gl.drawArrays(gl.TRIANGLES, 0, 6)
}
SpiritSimulator.prototype.step = function () {
if (this.paused || !this.read || !this.write) return
var gl = this.gl
var t = performance.now() * 0.001
var audio = window.__STEMENGINE_MUON_AUDIO__ || {}
var bass = shaped(0.5 * getBand(audio, 'sub', audio.sub || 0) + 0.5 * getBand(audio, 'bass', audio.bass || 0), 0.64)
var treble = shaped(0.55 * getBand(audio, 'highMid', audio.treble || 0) + 0.45 * getBand(audio, 'brilliance', audio.treble || 0), 0.66)
var rms = shaped(clamp01(audio.rms || 0), 0.8)
var beat = audio.beat ? 1 : 0
var buildBoost = audio.isBuildUp ? 0.02 : 0
var dropBoost = audio.isBeatDrop ? 0.02 : 0
gl.bindFramebuffer(gl.FRAMEBUFFER, this.write.fbo)
gl.viewport(0, 0, this.write.width, this.write.height)
gl.useProgram(this.updateProgram)
gl.activeTexture(gl.TEXTURE0)
gl.bindTexture(gl.TEXTURE_2D, this.read.tex)
gl.uniform1i(gl.getUniformLocation(this.updateProgram, 'uPrev'), 0)
gl.uniform2f(gl.getUniformLocation(this.updateProgram, 'uRes'), this.write.width, this.write.height)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTime'), t)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBass'), bass)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTreble'), treble)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uRms'), rms)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBeat'), beat)
gl.uniform1f(
gl.getUniformLocation(this.updateProgram, 'uFade'),
(this.quality === 'high' ? 0.965 : this.quality === 'low' ? 0.92 : 0.945) - buildBoost + dropBoost,
)
this.draw(this.updateProgram)
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
gl.viewport(0, 0, this.width, this.height)
gl.useProgram(this.blitProgram)
gl.activeTexture(gl.TEXTURE0)
gl.bindTexture(gl.TEXTURE_2D, this.write.tex)
gl.uniform1i(gl.getUniformLocation(this.blitProgram, 'uTex'), 0)
this.draw(this.blitProgram)
var tmp = this.read
this.read = this.write
this.write = tmp
}
SpiritSimulator.prototype.tick = function () {
if (!this.running) return
this.paused = Boolean(window.__STEMENGINE_MUON_PAUSED__)
this.setQuality(window.__STEMENGINE_MUON_QUALITY__)
var vp = window.__STEMENGINE_MUON_VIEWPORT__
if (vp) this.setViewport(vp.width, vp.height)
this.step()
var self = this
this.rafId = requestAnimationFrame(function () { self.tick() })
}
SpiritSimulator.prototype.start = function () {
if (this.running) return
this.running = true
this.tick()
}
SpiritSimulator.prototype.stop = function () {
this.running = false
if (this.rafId != null) {
cancelAnimationFrame(this.rafId)
this.rafId = null
}
}
SpiritSimulator.prototype.dispose = function () {
var gl = this.gl
this.stop()
if (this.quad) gl.deleteBuffer(this.quad)
if (this.updateProgram) gl.deleteProgram(this.updateProgram)
if (this.blitProgram) gl.deleteProgram(this.blitProgram)
if (this.read) {
gl.deleteTexture(this.read.tex)
gl.deleteFramebuffer(this.read.fbo)
}
if (this.write) {
gl.deleteTexture(this.write.tex)
gl.deleteFramebuffer(this.write.fbo)
}
}
window.__StemEngineSpiritRenderGraph__ = SpiritSimulator
console.log('[StemEngine][Spirit] Render graph module loaded')
})()
@@ -0,0 +1,96 @@
/* StemEngine runtime bootstrap injected into Muon build at image build-time. */
(function stemengineBootstrap() {
const REQUIRED_EXTENSIONS = ['OES_texture_float_linear', 'EXT_color_buffer_float']
function installContextCaptureHook() {
const originalGetContext = HTMLCanvasElement.prototype.getContext
if (typeof originalGetContext !== 'function') return
if (HTMLCanvasElement.prototype.__stemengineContextHookInstalled__) return
HTMLCanvasElement.prototype.getContext = function patchedGetContext(type, attrs) {
const ctx = originalGetContext.call(this, type, attrs)
if ((type === 'webgl2' || type === 'webgl') && ctx && this.isConnected) {
if (!window.__STEMENGINE_MUON_GL_CTX__) {
window.__STEMENGINE_MUON_GL_CTX__ = ctx
}
if (!window.__STEMENGINE_MUON_CANVAS__) {
window.__STEMENGINE_MUON_CANVAS__ = this
}
}
return ctx
}
Object.defineProperty(HTMLCanvasElement.prototype, '__stemengineContextHookInstalled__', {
value: true,
configurable: true,
writable: false,
enumerable: false,
})
}
function checkBaseline() {
const canvas = document.createElement('canvas')
const gl = canvas.getContext('webgl2') || canvas.getContext('webgl')
if (!gl) {
console.warn('[StemEngine][Muon] WebGL unavailable; fallback mode should be used.')
return {
ok: false,
missing: ['webgl'],
}
}
const missing = REQUIRED_EXTENSIONS.filter((name) => !gl.getExtension(name))
if (missing.length > 0) {
console.warn('[StemEngine][Muon] Missing required WebGL extensions:', missing.join(', '))
return { ok: false, missing }
}
return { ok: true, missing: [] }
}
function installContextLifecycleHooks() {
let retries = 0
const maxRetries = 120
const attach = () => {
const canvas = document.querySelector('canvas.webgl') || document.querySelector('canvas')
if (!canvas) {
retries += 1
if (retries <= maxRetries) requestAnimationFrame(attach)
return
}
canvas.addEventListener('webglcontextlost', (event) => {
event.preventDefault()
console.warn('[StemEngine][Muon] webglcontextlost')
window.__STEMENGINE_MUON_CONTEXT_LOST__ = true
})
canvas.addEventListener('webglcontextrestored', () => {
console.info('[StemEngine][Muon] webglcontextrestored')
window.__STEMENGINE_MUON_CONTEXT_LOST__ = false
})
}
attach()
}
if (typeof window.__STEMENGINE_MUON_STOP_FN__ !== 'function') {
window.__STEMENGINE_MUON_STOP_FN__ = function stemengineNoopStop() {}
}
if (typeof window.__STEMENGINE_MUON_RESUME_FN__ !== 'function') {
window.__STEMENGINE_MUON_RESUME_FN__ = function stemengineNoopResume() {}
}
installContextCaptureHook()
const baseline = checkBaseline()
window.__STEMENGINE_MUON_WEBGL_BASELINE__ = baseline
if (!baseline.ok) {
document.documentElement.classList.add('stemengine-muon-degraded')
}
installContextLifecycleHooks()
})()
@@ -0,0 +1,356 @@
/* StemEngine — style router + postMessage command listener.
* Injected into the Muon build at image build-time via Dockerfile.
*
* Project policy: zero access control restrictions on this path. Origin validation
* is intentionally a no-op. See docs/plans/2026-04-27_Muon_Default_Mode_Reactivity/
* README.md section 8 for the rationale.
*/
;(function stemengineRouter() {
function isAllowedOrigin(_origin) {
return true
}
function postToParent(message) {
// Always emit with wildcard and rely on host-side origin checks.
// This avoids stale single-origin target mismatches during mixed-origin dev workflows.
parent.postMessage(message, '*')
}
// -------------------------------------------------------------------------
// Helpers
// -------------------------------------------------------------------------
function getStyleParam() {
try {
return new URLSearchParams(location.search).get('style') || 'default'
} catch (_) {
return 'default'
}
}
/** Clamp a value to [lo, hi]; coerce to number, NaN → 0. */
function clamp(v, lo, hi) {
var n = Number(v)
if (!isFinite(n)) n = 0
return Math.max(lo, Math.min(hi, n))
}
var KNOWN_COMMANDS = {
setStyle: true,
setPreset: true,
setQuality: true,
setPaused: true,
setViewport: true,
pushAudioFeatures: true,
loadAudio: true,
transportPlay: true,
transportPause: true,
transportSeek: true,
transportRate: true,
}
// -------------------------------------------------------------------------
// Audio feature state — host-push model.
// The host sends pushAudioFeatures at its RAF cadence; we rate-limit to
// one application per our own RAF tick so we never block the render loop.
// -------------------------------------------------------------------------
var rafPending = false
var pendingFeatures = null
function captureFeatures(payload) {
if (!payload || typeof payload !== 'object') return
pendingFeatures = {
bass: clamp(payload.bass, 0, 1),
treble: clamp(payload.treble, 0, 1),
rms: clamp(payload.rms, 0, 1),
energy: clamp(payload.energy, 0, 1),
beat: Boolean(payload.beat),
drumIntensity: clamp(payload.drumIntensity, 0, 1),
sub: clamp(payload.sub, 0, 1),
beatPhase: clamp(payload.beatPhase, 0, 1),
bpm: Number.isFinite(Number(payload.bpm)) ? Number(payload.bpm) : 0,
beatConfidence: clamp(payload.beatConfidence, 0, 1),
isBeatDrop: Boolean(payload.isBeatDrop),
isBuildUp: Boolean(payload.isBuildUp),
isSilence: Boolean(payload.isSilence),
energyDelta: Number.isFinite(Number(payload.energyDelta)) ? Number(payload.energyDelta) : 0,
spectralCentroid: clamp(payload.spectralCentroid, 0, 1),
spectralFlatness: clamp(payload.spectralFlatness, 0, 1),
frequencyBands: payload.frequencyBands || null,
}
}
function flushFeatures() {
if (!pendingFeatures) return
window.__STEMENGINE_MUON_AUDIO__ = pendingFeatures
pendingFeatures = null
}
// -------------------------------------------------------------------------
// Render graph lifecycle
// -------------------------------------------------------------------------
var currentRenderGraph = null
var lastStyleValue = null
function updateRenderGraph(style) {
// Only instantiate when style actually changes or on first call.
if (style === lastStyleValue) return
lastStyleValue = style
// Dispose previous render graph (if any).
if (currentRenderGraph && typeof currentRenderGraph.dispose === 'function') {
try {
currentRenderGraph.dispose()
} catch (e) {
console.warn('[StemEngine][Muon] Error disposing render graph:', e)
}
}
currentRenderGraph = null
if (style === 'default') {
if (typeof window.__STEMENGINE_MUON_RESUME_FN__ === 'function') {
try {
window.__STEMENGINE_MUON_RESUME_FN__()
} catch (e) {
console.warn('[StemEngine][Muon] Error resuming default renderer:', e)
}
}
return
}
if (typeof window.__STEMENGINE_MUON_STOP_FN__ === 'function') {
try {
window.__STEMENGINE_MUON_STOP_FN__()
} catch (e) {
console.warn('[StemEngine][Muon] Error stopping default renderer:', e)
}
}
var viewport = window.__STEMENGINE_MUON_VIEWPORT__ || {}
var width = Number(viewport.width) || undefined
var height = Number(viewport.height) || undefined
var quality = window.__STEMENGINE_MUON_QUALITY__ || 'balanced'
// Instantiate new render graph based on style.
if (style === 'fluid' && typeof window.__StemEngineFluidRenderGraph__ === 'function') {
try {
currentRenderGraph = new window.__StemEngineFluidRenderGraph__(window.__STEMENGINE_MUON_GL_CTX__, width, height, quality)
console.log('[StemEngine][Muon] Instantiated Fluid render graph')
} catch (e) {
console.error('[StemEngine][Muon] Error instantiating Fluid render graph:', e)
window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e))
}
} else if (style === 'tendrils' && typeof window.__StemEngineTendrilsRenderGraph__ === 'function') {
try {
currentRenderGraph = new window.__StemEngineTendrilsRenderGraph__(window.__STEMENGINE_MUON_CANVAS__, window.__STEMENGINE_MUON_GL_CTX__, width, height, quality)
console.log('[StemEngine][Muon] Instantiated Tendrils render graph')
} catch (e) {
console.error('[StemEngine][Muon] Error instantiating Tendrils render graph:', e)
window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e))
}
} else if (style === 'spirit' && typeof window.__StemEngineSpiritRenderGraph__ === 'function') {
try {
currentRenderGraph = new window.__StemEngineSpiritRenderGraph__(window.__STEMENGINE_MUON_CANVAS__, window.__STEMENGINE_MUON_GL_CTX__, width, height, quality)
console.log('[StemEngine][Muon] Instantiated Spirit render graph')
} catch (e) {
console.error('[StemEngine][Muon] Error instantiating Spirit render graph:', e)
window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e))
}
}
}
// -------------------------------------------------------------------------
// Command dispatch
// -------------------------------------------------------------------------
function handleCommand(type, payload) {
var p = payload || {}
switch (type) {
case 'setStyle':
window.__STEMENGINE_MUON_STYLE__ = String(p.style || 'default')
updateRenderGraph(window.__STEMENGINE_MUON_STYLE__)
break
case 'setQuality':
window.__STEMENGINE_MUON_QUALITY__ = String(p.quality || 'balanced')
break
case 'setPreset':
window.__STEMENGINE_MUON_PRESET__ = String(p.preset || '')
break
case 'setPaused':
window.__STEMENGINE_MUON_PAUSED__ = Boolean(p.paused)
break
case 'setViewport':
var cssWidth = clamp(p.cssWidth != null ? p.cssWidth : p.width, 1, 7680)
var cssHeight = clamp(p.cssHeight != null ? p.cssHeight : p.height, 1, 4320)
var dprRaw = Number(p.dpr)
var dpr = isFinite(dprRaw) && dprRaw > 0 ? clamp(dprRaw, 0.25, 8) : 1
var pixelWidth = Math.max(1, Math.round(cssWidth * dpr))
var pixelHeight = Math.max(1, Math.round(cssHeight * dpr))
window.__STEMENGINE_MUON_VIEWPORT__ = {
cssWidth: cssWidth,
cssHeight: cssHeight,
dpr: dpr,
width: pixelWidth,
height: pixelHeight,
}
break
case 'pushAudioFeatures':
captureFeatures(p)
if (!rafPending) {
rafPending = true
requestAnimationFrame(function () {
rafPending = false
flushFeatures()
})
}
break
case 'loadAudio':
if (!window.__StemEngineAudioEngine__ || typeof window.__StemEngineAudioEngine__.loadFromUrl !== 'function') break
window.__StemEngineAudioEngine__.loadFromUrl(String(p.url || ''))
.then(function (info) {
if (window.__StemEngineFeatureExtractor__ && typeof window.__StemEngineFeatureExtractor__.start === 'function') {
window.__StemEngineFeatureExtractor__.start()
}
postToParent({
version: 1,
id: 'loaded-' + Date.now(),
type: 'loaded',
payload: {
duration: info && Number.isFinite(Number(info.duration)) ? Number(info.duration) : 0,
},
})
})
.catch(function (err) {
window.__stemengineEmitError__('AUDIO_LOAD', String(err && err.message ? err.message : err))
})
break
case 'transportPlay':
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.play === 'function') {
window.__StemEngineAudioEngine__.play({
time: Number(p.time) || 0,
rate: Number(p.rate) || 1,
})
}
break
case 'transportPause':
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.pause === 'function') {
window.__StemEngineAudioEngine__.pause()
}
break
case 'transportSeek':
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.seek === 'function') {
var nextTime = Number(p.time) || 0
window.__StemEngineAudioEngine__.seek(nextTime)
// Recovery path: if host is actively playing but a transportPlay command was
// dropped/raced, seek updates keep arriving. Auto-resume once from seek.
if (
!window.__STEMENGINE_MUON_PAUSED__
&& typeof window.__StemEngineAudioEngine__.isPlaying === 'function'
&& !window.__StemEngineAudioEngine__.isPlaying()
&& typeof window.__StemEngineAudioEngine__.play === 'function'
) {
var resumeRate = Number(p.rate)
if (!Number.isFinite(resumeRate) || resumeRate <= 0) {
resumeRate = typeof window.__StemEngineAudioEngine__.getRate === 'function'
? Number(window.__StemEngineAudioEngine__.getRate()) || 1
: 1
}
window.__StemEngineAudioEngine__.play({
time: nextTime,
rate: resumeRate,
})
}
}
break
case 'transportRate':
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.setRate === 'function') {
window.__StemEngineAudioEngine__.setRate(Number(p.rate) || 1)
}
break
}
}
// -------------------------------------------------------------------------
// postMessage listener
// -------------------------------------------------------------------------
window.addEventListener('message', function onStemEngineMessage(event) {
// Origin guard — reject messages from unexpected origins.
if (!isAllowedOrigin(event.origin)) return
var msg = event.data
if (!msg || typeof msg !== 'object') return
// Schema: { version: 1, type: string, payload: object }
if (Number(msg.version) !== 1) return
var type = String(msg.type || '')
if (!KNOWN_COMMANDS[type]) return
handleCommand(type, msg.payload)
})
// -------------------------------------------------------------------------
// Emit ready to host
// -------------------------------------------------------------------------
function emitReady() {
var baseline = window.__STEMENGINE_MUON_WEBGL_BASELINE__ || { ok: false, missing: [] }
var payload = {
capabilities: {
webgl: Boolean(baseline.ok),
missingExtensions: baseline.missing || [],
audioMode: 'hybrid',
},
styles: ['fluid', 'tendrils', 'spirit', 'default'],
}
try {
postToParent({ version: 1, id: 'ready-' + Date.now(), type: 'ready', payload: payload })
} catch (e) {
console.warn('[StemEngine][Muon] Could not emit ready:', e)
}
}
// -------------------------------------------------------------------------
// Emit error to host (exported for other modules to call)
// -------------------------------------------------------------------------
window.__stemengineEmitError__ = function emitError(code, message) {
try {
postToParent({
version: 1,
id: 'error-' + Date.now(),
type: 'error',
payload: { code: String(code || 'UNKNOWN'), message: String(message || '') },
})
} catch (_) {}
}
// -------------------------------------------------------------------------
// Bootstrap
// -------------------------------------------------------------------------
// Initialize globals from query params so the render graph can read them
// on its first frame even before any postMessage arrives.
window.__STEMENGINE_MUON_STYLE__ = getStyleParam()
window.__STEMENGINE_MUON_QUALITY__ = 'balanced'
window.__STEMENGINE_MUON_PAUSED__ = false
window.__STEMENGINE_MUON_AUDIO__ = null
window.__STEMENGINE_MUON_VIEWPORT__ = null
function onReady() {
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.initialize === 'function') {
window.__StemEngineAudioEngine__.initialize()
}
emitReady()
// Instantiate the initial render graph based on the style query param.
updateRenderGraph(window.__STEMENGINE_MUON_STYLE__)
}
if (document.readyState === 'loading') {
document.addEventListener('DOMContentLoaded', onReady)
} else {
onReady()
}
})()
@@ -0,0 +1,265 @@
/* StemEngine Tendrils Render Graph (Muon fork overlay)
* Particle trail / feedback visualization with audio reactivity.
*/
;(function stemengineTendrilsRenderGraph() {
'use strict'
var QUALITY_SCALE = { low: 0.75, balanced: 1.0, high: 1.35 }
function clamp01(v) {
var n = Number(v)
if (!isFinite(n)) n = 0
return Math.max(0, Math.min(1, n))
}
function shaped(v, power) {
return Math.pow(clamp01(v), power)
}
function getBand(audio, key, fallback) {
var bands = audio && audio.frequencyBands ? audio.frequencyBands : null
var value = bands && isFinite(Number(bands[key])) ? Number(bands[key]) : fallback
return clamp01(value)
}
function createProgram(gl, vsSrc, fsSrc) {
function compile(type, src) {
var s = gl.createShader(type)
gl.shaderSource(s, src)
gl.compileShader(s)
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) {
var err = gl.getShaderInfoLog(s) || 'shader compile failed'
window.__stemengineEmitError__('TENDRILS_SHADER', err)
gl.deleteShader(s)
return null
}
return s
}
var vs = compile(gl.VERTEX_SHADER, vsSrc)
var fs = compile(gl.FRAGMENT_SHADER, fsSrc)
if (!vs || !fs) return null
var p = gl.createProgram()
gl.attachShader(p, vs)
gl.attachShader(p, fs)
gl.linkProgram(p)
gl.deleteShader(vs)
gl.deleteShader(fs)
if (!gl.getProgramParameter(p, gl.LINK_STATUS)) {
var err2 = gl.getProgramInfoLog(p) || 'program link failed'
window.__stemengineEmitError__('TENDRILS_PROGRAM', err2)
gl.deleteProgram(p)
return null
}
return p
}
function createTarget(gl, w, h) {
var tex = gl.createTexture()
gl.bindTexture(gl.TEXTURE_2D, tex)
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, w, h, 0, gl.RGBA, gl.UNSIGNED_BYTE, null)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE)
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE)
var fbo = gl.createFramebuffer()
gl.bindFramebuffer(gl.FRAMEBUFFER, fbo)
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0)
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
return { tex: tex, fbo: fbo, width: w, height: h }
}
function TendrilsSimulator(canvas, gl, width, height, quality) {
this.canvas = canvas || document.querySelector('canvas')
this.gl = gl || (this.canvas && (this.canvas.getContext('webgl2', { alpha: false, antialias: false }) || this.canvas.getContext('webgl', { alpha: false, antialias: false })))
if (!this.gl) throw new Error('[StemEngine][Tendrils] Unable to acquire WebGL context')
this.width = Math.max(1, Number(width) || this.canvas.clientWidth || this.canvas.width || 512)
this.height = Math.max(1, Number(height) || this.canvas.clientHeight || this.canvas.height || 512)
this.quality = String(quality || window.__STEMENGINE_MUON_QUALITY__ || 'balanced')
this.paused = false
this.rafId = null
this.running = false
var gl = this.gl
var vs = 'attribute vec2 aPos; varying vec2 vUv; void main(){ vUv=aPos*0.5+0.5; gl_Position=vec4(aPos,0.0,1.0);} '
var updateFs = [
'precision highp float;',
'uniform sampler2D uPrev;',
'uniform vec2 uRes;',
'uniform float uTime;',
'uniform float uBass;',
'uniform float uTreble;',
'uniform float uRms;',
'uniform float uBeat;',
'uniform float uTrailDecay;',
'varying vec2 vUv;',
'float hash(vec2 p){ return fract(sin(dot(p, vec2(127.1,311.7))) * 43758.5453); }',
'void main(){',
' vec2 uv=vUv;',
' vec2 c=uv-0.5;',
' float r=max(length(c), 1e-4);',
' float a=atan(c.y,c.x);',
' float flow=sin(a*8.0 + uTime*(1.6+uBass*4.0)) + cos(r*45.0 + uTime*(1.1+uTreble*5.0));',
' vec2 dir=normalize(vec2(cos(a+flow*0.45), sin(a+flow*0.45)));',
' vec2 offset=dir*(0.0015 + 0.0035*uBass) + vec2(hash(uv+uTime)-0.5, hash(uv+uTime*1.7)-0.5)*(0.002*uTreble);',
' vec3 prev=texture2D(uPrev, clamp(uv-offset, 0.0, 1.0)).rgb * uTrailDecay;',
' float line=exp(-abs(sin((a*12.0 + r*120.0) + uTime*6.0*(1.0+uTreble)))*26.0);',
' float beatBurst=uBeat * exp(-r*4.0);',
' vec3 colorA=vec3(0.2+uBass*0.8, 0.2+uRms*0.7, 0.4+uTreble*0.6);',
' vec3 colorB=vec3(0.9,0.6+uTreble*0.35,0.2+uBass*0.4);',
' vec3 inject=mix(colorA, colorB, clamp(r*1.5, 0.0, 1.0))*(line*0.3 + beatBurst*0.8 + uRms*0.05);',
' gl_FragColor=vec4(clamp(prev + inject, 0.0, 1.0), 1.0);',
'}'
].join('\n')
var blitFs = 'precision mediump float; uniform sampler2D uTex; varying vec2 vUv; void main(){ gl_FragColor = texture2D(uTex, vUv); }'
this.updateProgram = createProgram(gl, vs, updateFs)
this.blitProgram = createProgram(gl, vs, blitFs)
var quad = new Float32Array([-1,-1, 1,-1, -1,1, -1,1, 1,-1, 1,1])
this.quad = gl.createBuffer()
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
gl.bufferData(gl.ARRAY_BUFFER, quad, gl.STATIC_DRAW)
this.read = null
this.write = null
this.allocateTargets()
this.start()
}
TendrilsSimulator.prototype.allocateTargets = function () {
var gl = this.gl
if (this.read) {
gl.deleteTexture(this.read.tex)
gl.deleteFramebuffer(this.read.fbo)
}
if (this.write) {
gl.deleteTexture(this.write.tex)
gl.deleteFramebuffer(this.write.fbo)
}
var scale = QUALITY_SCALE[this.quality] || QUALITY_SCALE.balanced
var w = Math.max(64, Math.floor(this.width * scale))
var h = Math.max(64, Math.floor(this.height * scale))
this.read = createTarget(gl, w, h)
this.write = createTarget(gl, w, h)
}
TendrilsSimulator.prototype.setQuality = function (q) {
var next = String(q || 'balanced')
if (next === this.quality) return
this.quality = next
this.allocateTargets()
}
TendrilsSimulator.prototype.setViewport = function (w, h) {
var nw = Math.max(1, Number(w) || this.width)
var nh = Math.max(1, Number(h) || this.height)
if (nw === this.width && nh === this.height) return
this.width = nw
this.height = nh
if (this.canvas) {
this.canvas.width = nw
this.canvas.height = nh
}
this.allocateTargets()
}
TendrilsSimulator.prototype.drawQuad = function (program) {
var gl = this.gl
gl.useProgram(program)
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
var loc = gl.getAttribLocation(program, 'aPos')
gl.enableVertexAttribArray(loc)
gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0)
gl.drawArrays(gl.TRIANGLES, 0, 6)
}
TendrilsSimulator.prototype.step = function () {
if (this.paused || !this.read || !this.write) return
var gl = this.gl
var t = performance.now() * 0.001
var audio = window.__STEMENGINE_MUON_AUDIO__ || {}
var bass = shaped(0.6 * getBand(audio, 'bass', audio.bass || 0) + 0.4 * getBand(audio, 'sub', audio.sub || 0), 0.62)
var treble = shaped(0.55 * getBand(audio, 'presence', audio.treble || 0) + 0.45 * getBand(audio, 'brilliance', audio.treble || 0), 0.66)
var rms = shaped(clamp01(audio.rms || 0), 0.8)
var beat = audio.beat ? 1 : 0
var buildBoost = audio.isBuildUp ? 0.12 : 0
var dropBoost = audio.isBeatDrop ? 0.2 : 0
gl.bindFramebuffer(gl.FRAMEBUFFER, this.write.fbo)
gl.viewport(0, 0, this.write.width, this.write.height)
gl.useProgram(this.updateProgram)
gl.activeTexture(gl.TEXTURE0)
gl.bindTexture(gl.TEXTURE_2D, this.read.tex)
gl.uniform1i(gl.getUniformLocation(this.updateProgram, 'uPrev'), 0)
gl.uniform2f(gl.getUniformLocation(this.updateProgram, 'uRes'), this.write.width, this.write.height)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTime'), t)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBass'), bass)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTreble'), treble)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uRms'), rms)
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBeat'), beat)
gl.uniform1f(
gl.getUniformLocation(this.updateProgram, 'uTrailDecay'),
(0.90 + (this.quality === 'high' ? 0.08 : this.quality === 'low' ? 0.04 : 0.06)) - buildBoost + dropBoost,
)
this.drawQuad(this.updateProgram)
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
gl.viewport(0, 0, this.width, this.height)
gl.useProgram(this.blitProgram)
gl.activeTexture(gl.TEXTURE0)
gl.bindTexture(gl.TEXTURE_2D, this.write.tex)
gl.uniform1i(gl.getUniformLocation(this.blitProgram, 'uTex'), 0)
this.drawQuad(this.blitProgram)
var tmp = this.read
this.read = this.write
this.write = tmp
}
TendrilsSimulator.prototype.tick = function () {
if (!this.running) return
this.paused = Boolean(window.__STEMENGINE_MUON_PAUSED__)
this.setQuality(window.__STEMENGINE_MUON_QUALITY__)
var vp = window.__STEMENGINE_MUON_VIEWPORT__
if (vp) this.setViewport(vp.width, vp.height)
this.step()
var self = this
this.rafId = requestAnimationFrame(function () { self.tick() })
}
TendrilsSimulator.prototype.start = function () {
if (this.running) return
this.running = true
this.tick()
}
TendrilsSimulator.prototype.stop = function () {
this.running = false
if (this.rafId != null) {
cancelAnimationFrame(this.rafId)
this.rafId = null
}
}
TendrilsSimulator.prototype.dispose = function () {
var gl = this.gl
this.stop()
if (this.quad) gl.deleteBuffer(this.quad)
if (this.updateProgram) gl.deleteProgram(this.updateProgram)
if (this.blitProgram) gl.deleteProgram(this.blitProgram)
if (this.read) {
gl.deleteTexture(this.read.tex)
gl.deleteFramebuffer(this.read.fbo)
}
if (this.write) {
gl.deleteTexture(this.write.tex)
gl.deleteFramebuffer(this.write.fbo)
}
}
window.__StemEngineTendrilsRenderGraph__ = TendrilsSimulator
console.log('[StemEngine][Tendrils] Render graph module loaded')
})()
@@ -0,0 +1,346 @@
# Local Iframe Visualizer Architecture Plan
Date: 2026-04-28
Status: Draft implementation plan
Owner: StemEngine frontend / visualizer pipeline
## Problem Statement
The current attempt to approximate `fluid`, `tendrils`, and `spirit` inside the existing Muon iframe path did not deliver a reliable result.
Observed issues:
- `default` mode is the only mode currently behaving close to the intended repeat-play iframe flow.
- `fluid` shows reproducible replay instability and framebuffer warnings.
- `tendrils` and `spirit` may look visually in-family, but are still bound to Muon-specific lifecycle assumptions that make them hard to validate and stabilize.
## Goal
Use the same host-to-iframe transport model for every visualizer style, but move non-default styles onto a local iframe runtime that behaves like Muon from the host’s point of view.
Target outcome:
- the host speaks one iframe transport contract,
- each style runs in an iframe-managed renderer lifecycle,
- replay / pause / seek / rate / resize are validated the same way for all styles,
- non-default styles are no longer dependent on Muon internals that were designed for the default renderer.
## Decision Summary
Recommended architecture:
- Keep the existing Muon iframe provider for `default` for now.
- Build a local iframe visualizer runtime for `fluid`, `tendrils`, and `spirit`.
- Standardize the transport contract so the host treats both as iframe visualizer providers.
Preferred shape:
- One shared local iframe shell with pluggable style engines.
Fallback shape:
- One local iframe app per style.
Recommendation:
- Use one shared local iframe shell unless style-specific boot constraints prove otherwise.
## Why This Direction
Benefits:
- separates transport concerns from render math,
- removes style replay behavior from Muon private/default renderer assumptions,
- reduces duplicate host logic,
- gives one lifecycle model for all local styles,
- makes validation and third-party review clearer.
## Scope
In scope:
- iframe transport contract standardization,
- host visualizer provider abstraction,
- local iframe shell runtime,
- local renderer plugin API,
- migration of `fluid`, `tendrils`, and `spirit` to local iframe shell,
- validation matrix across Track Detail, Visualizer Lab, and other exposed surfaces.
Out of scope for first pass:
- replacing default Muon renderer,
- perfect fidelity to third-party reference demos,
- broad preset-authoring UX,
- camera/mic/advanced interaction parity with original external demos.
## Current File Touch Points
### Host-side frontend
Likely primary touch points:
- `frontend/src/hooks/useMuonIframeTransport.ts`
- `frontend/src/components/VisualizerPreviewWidget.tsx`
- `frontend/src/components/LyricsLivePreview.tsx`
- `frontend/src/pages/VisualizerLabPage.tsx`
- `frontend/src/pages/TrackDetail.tsx`
Likely new host-side additions:
- `frontend/src/visualizers/providers/*`
- `frontend/src/hooks/useIframeVisualizerTransport.ts`
- `frontend/src/types/iframeVisualizer.ts`
### Existing Muon iframe runtime
Reference implementation / source of reusable logic:
- `muon-iframe/stemengine-router.js`
- `muon-iframe/stemengine-bootstrap.js`
- `muon-iframe/stemengine-audio-engine.js`
- `muon-iframe/stemengine-feature-extractor.js`
### Existing style renderers to migrate or adapt
- `muon-iframe/fluid-render-graph.js`
- `muon-iframe/tendrils-render-graph.js`
- `muon-iframe/spirit-render-graph.js`
### Likely new local iframe app area
Suggested location:
- `frontend/local-visualizer-iframe/` or
- `local-visualizer-iframe/`
Recommendation:
- keep it separate from `muon-iframe/` so it is not build-time injected into Muon.
## Target Architecture
### 1. Host visualizer provider layer
The host should target a provider abstraction, not Muon directly.
Proposed provider model:
- provider id
- iframe URL
- supported styles
- capabilities
- command protocol version
- fallback behavior
Example providers:
- `muon-default`
- `local-visualizer`
### 2. Shared iframe transport contract
Commands from host to iframe:
- `init`
- `setStyle`
- `setPreset`
- `setQuality`
- `setPaused`
- `setViewport`
- `loadAudio`
- `transportPlay`
- `transportPause`
- `transportSeek`
- `transportRate`
- `pushAudioFeatures`
- `dispose`
Events from iframe to host:
- `ready`
- `loaded`
- `capabilities`
- `state`
- `warning`
- `error`
Required rules:
- host owns transport intent and source audio selection,
- iframe owns rendering lifecycle,
- seek while playing must not leave renderer visually paused,
- replay after pause must be deterministic,
- viewport updates must not require reload.
### 3. Local iframe shell
Responsibilities:
- boot and emit `ready`,
- set up audio engine,
- set up feature extractor,
- receive commands from host,
- instantiate exactly one renderer at a time,
- dispose renderer cleanly on style switch or teardown,
- expose warnings/errors back to host.
Shell should be style-agnostic.
### 4. Renderer plugin API
Each style renderer should conform to the same API.
Proposed renderer interface:
- `init(context)`
- `setViewport(viewport)`
- `setQuality(quality)`
- `setPreset(preset)`
- `setPaused(paused)`
- `pushFeatures(features)`
- `tick(time)`
- `dispose()`
Renderer should not own:
- transport protocol,
- source audio loading,
- play/pause/seek intent,
- message routing.
### 5. Migration model
Near-term routing:
- `default` -> existing Muon iframe provider
- `fluid` -> local iframe shell
- `tendrils` -> local iframe shell
- `spirit` -> local iframe shell
Future option:
- move `default` under the same provider system later if beneficial.
## Implementation Phases
## Phase 1: Freeze the contract
Deliverables:
- one transport spec document,
- one provider abstraction design,
- explicit replay/seek/resize semantics.
Tasks:
- document current useful Muon messages,
- remove Muon-specific assumptions from the conceptual transport model,
- define one versioned protocol for all iframe visualizers.
Exit criteria:
- host and iframe responsibilities are explicitly defined,
- no new style-specific transport behavior is added ad hoc.
## Phase 2: Introduce provider abstraction on host
Deliverables:
- provider registry,
- host hook abstracted away from Muon naming,
- widgets/pages able to select a provider.
Tasks:
- refactor `useMuonIframeTransport.ts` into provider-neutral transport hook,
- add provider config for default Muon and local iframe runtime,
- update consumers in preview surfaces.
Exit criteria:
- Track Detail and Visualizer Lab can load an iframe provider without hard-coding Muon-only assumptions.
## Phase 3: Build local iframe shell
Deliverables:
- local iframe app booting independently,
- postMessage router,
- audio engine and feature extractor wiring,
- renderer lifecycle manager.
Tasks:
- clone only the reusable runtime concepts from current Muon integration,
- avoid build-time injection into third-party Muon source,
- add explicit renderer registration and disposal.
Exit criteria:
- shell loads, emits ready, can accept audio load/play/pause/seek/viewport commands even with a placeholder renderer.
## Phase 4: Migrate `fluid`
Why first:
- it is the closest structural match to its intended reference,
- it is currently the least reliable mode.
Tasks:
- adapt current fluid renderer to the new renderer API,
- enforce init-time capability checks,
- surface explicit degrade/failure states when FBO support is incomplete,
- validate replay and resize in the new shell.
Exit criteria:
- `fluid` passes two-cycle replay test in Track Detail,
- no silent frozen-canvas failure path,
- FBO failure produces explicit host-visible warning/error.
## Phase 5: Migrate `tendrils` and `spirit`
Tasks:
- adapt both renderers to shared shell lifecycle,
- remove transport ownership assumptions from render graphs,
- validate style switch, replay, resize, and quality transitions.
Exit criteria:
- both renderers pass same iframe lifecycle tests as `fluid`,
- no per-style transport branching on host.
## Phase 6: Cross-surface validation and rollout
Tasks:
- validate Track Detail,
- validate Visualizer Lab with a loaded song path,
- validate any project/embed surface that exposes iframe preview,
- update docs and reviewer package with post-migration results.
Exit criteria:
- the same style behaves consistently across supported surfaces,
- unsupported surfaces are explicitly documented.
## Validation Matrix
Each migrated style must pass:
- iframe boot / ready
- audio load
- play cycle 1
- pause
- replay cycle 2
- seek while playing
- seek while paused
- rate change
- viewport resize
- quality change
- style switch away and back
- teardown / remount
Required test surfaces:
- Track Detail
- Visualizer Lab
- any active project/embed preview surface
## Key Risks
1. Host abstraction drift
Risk:
- partial refactor leaves old Muon-only codepaths mixed with provider abstraction.
Mitigation:
- centralize provider selection in one registry and one transport hook.
2. Fluid hardware variability
Risk:
- local fluid renderer still fails on certain WebGL / extension combinations.
Mitigation:
- explicit capability detection and degrade behavior, not silent failure.
3. Renderer lifecycle leaks
Risk:
- RAF loops, textures, or framebuffers survive style switches.
Mitigation:
- strict renderer API and disposable lifecycle manager in shell.
4. Visualizer Lab remains blocked
Risk:
- shell exists but lab still cannot load audio cleanly.
Mitigation:
- define one supported sample-input path early in Phase 3.
5. Fidelity pressure before lifecycle stability
Risk:
- development time is spent tuning visuals before replay and resize are correct.
Mitigation:
- block fidelity work until lifecycle acceptance criteria pass.
## Deliverables Checklist
- transport spec doc
- provider registry
- provider-neutral iframe transport hook
- local iframe shell app
- renderer plugin API
- migrated `fluid`
- migrated `tendrils`
- migrated `spirit`
- updated validation docs
- updated review package
## Recommended Execution Order
1. Freeze transport spec
2. Refactor host to provider model
3. Build local iframe shell with placeholder renderer
4. Migrate `fluid`
5. Stabilize replay / resize / seek
6. Migrate `tendrils`
7. Migrate `spirit`
8. Re-run cross-surface validation
9. Update external review package
## Success Definition
This plan succeeds when:
- non-default styles no longer rely on Muon’s default-renderer internals,
- host uses one iframe transport model across providers,
- replay behavior is deterministic for all supported styles,
- failures are surfaced explicitly instead of appearing as frozen visuals,
- reviewers can evaluate styles through a consistent iframe lifecycle rather than special-case paths.