Myriad changes.
This commit is contained in:
1 parent
519b2c8dcd
commit
1bb503f735
19 files changed
+4058
No files matched your search
@@ -0,0 +1,78 @@
|
||||
# Muon Visualizer Current Per-Mode Functionality
|
||||
|
||||
Date: 2026-04-27
|
||||
Environment: StemEngine track preview (http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb)
|
||||
Mode set under test: `default`, `fluid`, `tendrils`, `spirit`
|
||||
|
||||
## Scope
|
||||
This document captures the CURRENT observed behavior for Muon style modes in Track Detail preview, with focus on repeat-play reactivity.
|
||||
|
||||
Reactivity definition used here:
|
||||
- cycle 1 reactive: visual output changes after first Play preview
|
||||
- cycle 2 reactive: visual output still changes after Pause -> Play replay
|
||||
|
||||
## Summary
|
||||
- `default`: functional with caveats
|
||||
- `fluid`: reproducible bug on replay (cycle 2 non-reactive)
|
||||
- `tendrils`: functional in current sweep, but keep under watch (user-reported instability exists)
|
||||
- `spirit`: functional in current sweep, but keep under watch (user-reported instability exists)
|
||||
|
||||
## Per-Mode Status
|
||||
|
||||
### default
|
||||
Status: Functional with caveats
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: reactive (3/3 runs)
|
||||
Caveats:
|
||||
- Intermittent stream-layer noise still appears in session logs/events (`ERR_ABORTED`, occasional `502`), but did not break reactivity in this sweep.
|
||||
|
||||
### fluid
|
||||
Status: Buggy (reproducible)
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: NOT reactive (3/3 runs)
|
||||
Evidence:
|
||||
- Motion diff for cycle 2 repeatedly dropped to `0`
|
||||
- Console repeatedly logs: `[StemEngine][Fluid] FBO incomplete: 36054`
|
||||
Impact:
|
||||
- Replay reliability is broken for fluid mode under current build/runtime combination.
|
||||
|
||||
### tendrils
|
||||
Status: Functional in current session (watchlist)
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: reactive (3/3 runs)
|
||||
Caveat:
|
||||
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
|
||||
|
||||
### spirit
|
||||
Status: Functional in current session (watchlist)
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: reactive (3/3 runs)
|
||||
Caveat:
|
||||
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
|
||||
|
||||
## Raw Sweep Snapshot (Automated)
|
||||
Run 1:
|
||||
- default: c1 reactive, c2 reactive
|
||||
- fluid: c1 reactive, c2 non-reactive
|
||||
- tendrils: c1 reactive, c2 reactive
|
||||
- spirit: c1 reactive, c2 reactive
|
||||
|
||||
Run 2:
|
||||
- default: c1 reactive, c2 reactive
|
||||
- fluid: c1 reactive, c2 non-reactive
|
||||
- tendrils: c1 reactive, c2 reactive
|
||||
- spirit: c1 reactive, c2 reactive
|
||||
|
||||
Run 3:
|
||||
- default: c1 reactive, c2 reactive
|
||||
- fluid: c1 reactive, c2 non-reactive
|
||||
- tendrils: c1 reactive, c2 reactive
|
||||
- spirit: c1 reactive, c2 reactive
|
||||
|
||||
## Notes
|
||||
- This report is intentionally point-in-time. Re-run after renderer/transport changes to keep this file current.
|
||||
- If you want, add a companion matrix file for Visualizer Lab page and project-page embed to compare behavior across surfaces.
|
||||
@@ -0,0 +1,95 @@
|
||||
# Muon Review Export Bundle Manifest
|
||||
|
||||
Date: 2026-04-28
|
||||
Purpose: frozen manifest for external sharing of the Muon mode review package.
|
||||
|
||||
## Bundle Name
|
||||
Suggested bundle label:
|
||||
- `muon-mode-review-package-2026-04-28`
|
||||
|
||||
## Bundle Scope
|
||||
This export bundle is intended for third-party reviewers who need:
|
||||
- current operational status by mode,
|
||||
- surface-by-surface testability notes,
|
||||
- reference-comparison analysis,
|
||||
- a reviewer scoring rubric,
|
||||
- a single index file for navigation.
|
||||
|
||||
## Files To Include
|
||||
|
||||
### Core review docs
|
||||
1. `docs/muon-mode-status/README.md`
|
||||
- Entry point and reading order.
|
||||
|
||||
2. `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- Point-in-time Track Detail mode functionality and replay observations.
|
||||
|
||||
3. `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
- Surface matrix across Track Detail, Visualizer Lab, and project/embed views.
|
||||
|
||||
4. `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
- Reference comparison for `fluid`, `tendrils`, and `spirit`.
|
||||
|
||||
5. `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
|
||||
- Third-party scoring and signoff rubric.
|
||||
|
||||
### Optional supporting source files
|
||||
Include these if the reviewer needs implementation-level evidence:
|
||||
6. `muon-iframe/fluid-render-graph.js`
|
||||
7. `muon-iframe/tendrils-render-graph.js`
|
||||
8. `muon-iframe/spirit-render-graph.js`
|
||||
9. `muon-iframe/stemengine-router.js`
|
||||
10. `muon-iframe/stemengine-bootstrap.js`
|
||||
|
||||
## Minimal Bundle
|
||||
If the recipient should receive docs only, export:
|
||||
- `docs/muon-mode-status/README.md`
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
|
||||
|
||||
## Full Evidence Bundle
|
||||
If the recipient should receive both review docs and implementation evidence, export:
|
||||
- all files from the Minimal Bundle
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
|
||||
## Suggested Folder Layout For Sharing
|
||||
```text
|
||||
muon-mode-review-package-2026-04-28/
|
||||
README.md
|
||||
docs/
|
||||
2026-04-27-current-per-mode-functionality.md
|
||||
2026-04-28-surface-functionality-matrix.md
|
||||
2026-04-28-reference-comparison-third-party-review.md
|
||||
2026-04-28-reviewer-checklist.md
|
||||
source-evidence/
|
||||
fluid-render-graph.js
|
||||
tendrils-render-graph.js
|
||||
spirit-render-graph.js
|
||||
stemengine-router.js
|
||||
stemengine-bootstrap.js
|
||||
```
|
||||
|
||||
## Export Notes
|
||||
- This manifest freezes the intended review set as of 2026-04-28.
|
||||
- The bundle is evidence-oriented, not a claim of source-equivalent parity with external references.
|
||||
- Reference-comparison conclusions include inference and confidence levels; reviewers should read those caveats before scoring fidelity.
|
||||
|
||||
## Reviewer Guidance
|
||||
Recommended reading sequence inside the exported bundle:
|
||||
1. `README.md`
|
||||
2. `2026-04-28-reference-comparison-third-party-review.md`
|
||||
3. `2026-04-27-current-per-mode-functionality.md`
|
||||
4. `2026-04-28-surface-functionality-matrix.md`
|
||||
5. `2026-04-28-reviewer-checklist.md`
|
||||
|
||||
## Current Package Status
|
||||
As of this manifest:
|
||||
- `fluid` remains the main concern because of replay/FBO reliability issues.
|
||||
- `tendrils` and `spirit` are documented as in-family approximations, not proven literal ports.
|
||||
- Visualizer Lab and project/embed surfaces remain limited or non-testable for this exact replay matrix without additional setup.
|
||||
@@ -0,0 +1,141 @@
|
||||
# Muon Mode Comparison vs Reference Demos (Third-Party Review)
|
||||
|
||||
Date: 2026-04-28
|
||||
Prepared for: external/third-party technical review
|
||||
Scope: infer how local `fluid`, `tendrils`, and `spirit` modes compare to their likely inspiration references.
|
||||
|
||||
References under review:
|
||||
- https://paveldogreat.github.io/WebGL-Fluid-Simulation/
|
||||
- https://keeffeoghan.github.io/work/tendrils/ (redirects to `epok.tech` content)
|
||||
- https://edankwan.com/experiments/the-spirit/
|
||||
|
||||
Local implementation inputs reviewed:
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
|
||||
## Method Notes
|
||||
This review intentionally separates:
|
||||
- verified local facts (from readable source files), and
|
||||
- inferred reference characteristics (from live pages + publicly visible descriptions + agent-assisted analysis).
|
||||
|
||||
A `ui-designer` subagent was used to produce an independent reference summary, then cross-checked against local code.
|
||||
|
||||
## Executive Summary
|
||||
1. `fluid` in this fork is structurally the closest to its reference target (PavelDoGreat fluid simulation), because it implements a recognizable multi-pass fluid pipeline (splat, divergence, pressure Jacobi, gradient subtract, advection, bloom).
|
||||
2. `tendrils` and `spirit` in this fork are stylistic approximations, not faithful reproductions of the original interactive systems. They are compact shader-feedback variants optimized for integration into Muon’s transport/audio stack.
|
||||
3. The local `fluid` path currently has a practical stability/replay issue in your environment (`FBO incomplete: 36054` warnings and cycle-2 non-reactivity), which materially affects parity with the expected “real thing” feel.
|
||||
|
||||
## Comparison Matrix
|
||||
|
||||
| Mode | Local Implementation (verified) | Reference “Real Thing” (inferred) | Similarity Assessment |
|
||||
|---|---|---|---|
|
||||
| fluid | Full-screen multipass FBO fluid pipeline with pressure solve, advection, bloom, and audio splat injection | Interactive Navier-Stokes style GPU fluid sim with user splats and polished post FX | High structural similarity; medium experiential similarity due to current stability issues and integration constraints |
|
||||
| tendrils | Feedback texture + procedural flow/noise line structures in one update shader, audio-driven decay/flow | Emergent particle-field ecosystem with richer control surface, presets, and interaction channels (keyboard/camera/mic) | Medium visual similarity; low-to-medium system similarity |
|
||||
| spirit | Layered atmospheric feedback shader with fbm/noise, halo/strand/pulse composition, audio-driven fade | Highly polished ethereal particle/trail experience with strong cinematic motion language | Medium stylistic similarity; low implementation-parity confidence |
|
||||
|
||||
## Per-Mode Detailed Assessment
|
||||
|
||||
### A) `fluid`
|
||||
Local verified facts:
|
||||
- Uses explicit fluid-solver style passes:
|
||||
- curl, divergence, pressure Jacobi, gradient subtract, velocity advection, dye advection.
|
||||
- Uses ping-pong FBOs for velocity/dye/pressure and optional bloom chain by quality tier.
|
||||
- Audio coupling includes bass/sub/treble/rms/beat/beatPhase/drumIntensity mapped to splat radius/force/color.
|
||||
- Quality tier controls pressure iterations and bloom enablement.
|
||||
|
||||
Evidence (local code):
|
||||
- `muon-iframe/fluid-render-graph.js` (pipeline and FBO management)
|
||||
- `muon-iframe/stemengine-router.js` (style lifecycle + audio feature push)
|
||||
|
||||
Reference inferred characteristics:
|
||||
- Canonical GPU fluid simulation style with strong interactive splat behavior and stable continuous flow.
|
||||
|
||||
Assessment:
|
||||
- Architecture match is strong.
|
||||
- Main gap is operational reliability in your current stack: observed `FBO incomplete: 36054` warnings and replay-cycle failures reduce real-world parity.
|
||||
|
||||
Confidence:
|
||||
- Local implementation confidence: High
|
||||
- Reference internals confidence: Medium
|
||||
- Similarity conclusion confidence: Medium-High
|
||||
|
||||
### B) `tendrils`
|
||||
Local verified facts:
|
||||
- Two-program design (`update` + `blit`) with framebuffer feedback loop.
|
||||
- Motion built from angle/radius flow fields + hash noise + trail decay.
|
||||
- Audio mapping primarily modulates flow rate, turbulence, color injection, and decay.
|
||||
- No explicit particle-buffer architecture exposed; behavior emerges from texture feedback math.
|
||||
|
||||
Evidence (local code):
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
|
||||
Reference inferred characteristics:
|
||||
- Public text describes interacting particle system with advection-like behavior, optional optical flow/camera/mic interactions, rich controls and presets.
|
||||
|
||||
Assessment:
|
||||
- The local mode captures the “organic tendril motion” aesthetic.
|
||||
- It appears materially simplified versus the broader interaction/control ecosystem and potentially deeper particle mechanics of the reference.
|
||||
|
||||
Confidence:
|
||||
- Local implementation confidence: High
|
||||
- Reference internals confidence: Medium-Low
|
||||
- Similarity conclusion confidence: Medium
|
||||
|
||||
### C) `spirit`
|
||||
Local verified facts:
|
||||
- Two-pass feedback renderer with atmospheric layering:
|
||||
- fbm-like noise term,
|
||||
- radial halo decay,
|
||||
- strand/pulse functions,
|
||||
- additive glow and nonlinear output shaping.
|
||||
- Audio mapping influences drift, fade persistence, pulse intensity, and color blend.
|
||||
|
||||
Evidence (local code):
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
|
||||
Reference inferred characteristics:
|
||||
- Highly polished ethereal motion piece with strong artistic direction and likely richer scene dynamics.
|
||||
|
||||
Assessment:
|
||||
- The local mode appears designed as a stylistic homage (ethereal layered glow) rather than a strict algorithmic clone.
|
||||
- Without source-level access to reference internals, parity can only be judged at visual-language level.
|
||||
|
||||
Confidence:
|
||||
- Local implementation confidence: High
|
||||
- Reference internals confidence: Low-Medium
|
||||
- Similarity conclusion confidence: Medium-Low
|
||||
|
||||
## Integration Context (Important for Reviewers)
|
||||
The local renderers are not standalone art demos; they are embedded into Muon transport and controlled by host commands.
|
||||
|
||||
Observed integration traits:
|
||||
- Router-driven style switching and lifecycle disposal/instantiation.
|
||||
- Host-driven audio feature push throttled to iframe RAF cadence.
|
||||
- Host-driven pause/play/seek/rate semantics through injected audio engine.
|
||||
- Baseline WebGL extension checks in bootstrap layer.
|
||||
|
||||
Implication:
|
||||
- Even when visual math resembles a reference demo, behavior can diverge due to transport timing, iframe lifecycle, and browser/driver extension availability.
|
||||
|
||||
## Known Gaps and Risks
|
||||
1. Reference internals are partly opaque (minified/demo-only), so comparison is inference-based, not source-equivalence.
|
||||
2. `fluid` currently shows environment-specific fragility (`FBO incomplete: 36054` warnings observed in session), impacting practical parity.
|
||||
3. `tendrils`/`spirit` appear intentionally compact; they likely trade breadth/depth of original interactivity for predictable embedding in Muon.
|
||||
|
||||
## Third-Party Review Conclusion
|
||||
- If the review criterion is “same algorithm/system as originals”: not demonstrated for `tendrils` or `spirit`; only partial for `fluid` (closest).
|
||||
- If criterion is “same visual family integrated into Muon with audio transport”: yes, with `fluid` currently hindered by reliability caveats.
|
||||
- Recommended interpretation: these are reference-inspired implementations with varying structural closeness, not literal ports.
|
||||
|
||||
## Suggested Evidence Bundle for Auditors
|
||||
Include these files in review packets:
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
@@ -0,0 +1,152 @@
|
||||
# Muon Reviewer Checklist
|
||||
|
||||
Date: 2026-04-28
|
||||
Purpose: quick third-party parity and functionality review for Muon `fluid`, `tendrils`, and `spirit` modes.
|
||||
|
||||
Related documents:
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
|
||||
## How To Use
|
||||
For each mode, mark:
|
||||
- PASS: clearly meets the criterion
|
||||
- PARTIAL: present but degraded, unstable, or notably simplified
|
||||
- FAIL: absent, broken, or materially unlike expected behavior
|
||||
- N/A: not exposed on the reviewed surface
|
||||
|
||||
Recommended review surfaces:
|
||||
- Track Detail preview
|
||||
- Visualizer Lab
|
||||
- Any project/embed surface exposing Muon controls
|
||||
|
||||
## Reviewer Metadata
|
||||
- Reviewer:
|
||||
- Date:
|
||||
- Browser / version:
|
||||
- GPU / driver:
|
||||
- Surface tested:
|
||||
- Audio source used:
|
||||
- Quality tier tested:
|
||||
|
||||
## Section 1: Basic Functionality
|
||||
|
||||
### Mode availability
|
||||
- `fluid` is selectable
|
||||
- `tendrils` is selectable
|
||||
- `spirit` is selectable
|
||||
- Mode switch does not visibly break iframe/canvas
|
||||
|
||||
### Playback / transport
|
||||
- First play starts visual motion
|
||||
- Pause stops or meaningfully freezes progression
|
||||
- Replay after pause resumes visual motion
|
||||
- Seek while playing preserves visual responsiveness
|
||||
- Mode remains reactive for at least two play cycles
|
||||
|
||||
### Stability
|
||||
- No blank canvas on mode switch
|
||||
- No persistent frozen frame after replay
|
||||
- No repeated fatal errors in console
|
||||
- No WebGL context loss during normal use
|
||||
|
||||
## Section 2: Visual Parity vs Reference
|
||||
|
||||
### `fluid`
|
||||
Expected reference family:
|
||||
- Fluid-like motion with continuous advection
|
||||
- Visible diffusion / swirling / dye transport feel
|
||||
- Stronger impulse on beats or major peaks
|
||||
- Cohesive full-frame flow rather than isolated sprite effects
|
||||
|
||||
Review marks:
|
||||
- Flow behavior resembles fluid simulation
|
||||
- Beat/bass energy visibly changes motion intensity
|
||||
- Color field behaves like transported dye, not only flashing overlays
|
||||
- Motion remains continuous across replay cycles
|
||||
- Presentation feels structurally close to the PavelDoGreat reference
|
||||
|
||||
Overall `fluid` score: PASS / PARTIAL / FAIL
|
||||
Notes:
|
||||
|
||||
### `tendrils`
|
||||
Expected reference family:
|
||||
- Organic line or filament structures
|
||||
- Emergent motion with feedback / trace persistence
|
||||
- Sense of interconnected or fibrous forms
|
||||
- More than a simple radial pulse or stock waveform
|
||||
|
||||
Review marks:
|
||||
- Visual output forms tendril/filament-like structures
|
||||
- Trails persist and evolve over time
|
||||
- Motion feels emergent rather than strictly canned
|
||||
- Audio affects density, color, or movement in a noticeable way
|
||||
- Presentation feels recognizably related to the Tendrils reference family
|
||||
|
||||
Overall `tendrils` score: PASS / PARTIAL / FAIL
|
||||
Notes:
|
||||
|
||||
### `spirit`
|
||||
Expected reference family:
|
||||
- Ethereal / atmospheric glow language
|
||||
- Layered soft motion rather than sharp geometric tracing
|
||||
- Strong sense of aura, halo, strand, smoke, or apparition-like motion
|
||||
- Audio-linked pulse or drift without collapsing into a basic particle burst
|
||||
|
||||
Review marks:
|
||||
- Visual output feels atmospheric / spectral / ethereal
|
||||
- Glow and layering read clearly on screen
|
||||
- Audio changes drift, pulse, or luminosity in a visible way
|
||||
- Motion remains smooth across play / pause / replay
|
||||
- Presentation feels recognizably related to the Spirit reference family
|
||||
|
||||
Overall `spirit` score: PASS / PARTIAL / FAIL
|
||||
Notes:
|
||||
|
||||
## Section 3: Surface Matrix Check
|
||||
Mark each surface:
|
||||
|
||||
### Track Detail
|
||||
- `fluid`: PASS / PARTIAL / FAIL / N/A
|
||||
- `tendrils`: PASS / PARTIAL / FAIL / N/A
|
||||
- `spirit`: PASS / PARTIAL / FAIL / N/A
|
||||
Notes:
|
||||
|
||||
### Visualizer Lab
|
||||
- `fluid`: PASS / PARTIAL / FAIL / N/A
|
||||
- `tendrils`: PASS / PARTIAL / FAIL / N/A
|
||||
- `spirit`: PASS / PARTIAL / FAIL / N/A
|
||||
Notes:
|
||||
|
||||
### Project / embed surface
|
||||
- `fluid`: PASS / PARTIAL / FAIL / N/A
|
||||
- `tendrils`: PASS / PARTIAL / FAIL / N/A
|
||||
- `spirit`: PASS / PARTIAL / FAIL / N/A
|
||||
Notes:
|
||||
|
||||
## Section 4: Red Flags
|
||||
Check any that apply:
|
||||
- Replay fails only on some modes
|
||||
- Mode works on first play but freezes on second play
|
||||
- Console shows framebuffer / FBO warnings
|
||||
- Visuals appear materially simpler than reference expectations
|
||||
- Surface exposes controls but is not actually testable
|
||||
- Behavior differs substantially between surfaces
|
||||
|
||||
## Section 5: Final Verdict
|
||||
Use this rubric:
|
||||
- PASS: mode is stable and recognizably in the same reference family
|
||||
- PARTIAL: mode works but has obvious reliability, fidelity, or surface gaps
|
||||
- FAIL: mode is broken or not meaningfully comparable to its reference family
|
||||
|
||||
Final verdict by mode:
|
||||
- `fluid`:
|
||||
- `tendrils`:
|
||||
- `spirit`:
|
||||
|
||||
Overall release recommendation:
|
||||
- APPROVE for review
|
||||
- APPROVE WITH CAVEATS
|
||||
- DO NOT APPROVE
|
||||
|
||||
Summary comments:
|
||||
@@ -0,0 +1,65 @@
|
||||
# Muon Surface Functionality Matrix
|
||||
|
||||
Date: 2026-04-28
|
||||
|
||||
Surfaces covered:
|
||||
- Track Detail preview (`/tracks/:trackId`)
|
||||
- Visualizer Lab (`/visualizer-lab`)
|
||||
- Project page (`/projects/:projectId` on webui host)
|
||||
|
||||
## Legend
|
||||
- PASS: verified reactive in current session
|
||||
- FAIL: reproducible non-reactive behavior in current session
|
||||
- BLOCKED: surface/control exists but cannot execute mode replay test without additional input/state
|
||||
- N/A: control path for this Muon mode test is not exposed on this surface
|
||||
|
||||
## Matrix (Current)
|
||||
|
||||
| Surface | default | fluid | tendrils | spirit | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| Track Detail preview | PASS | FAIL | PASS | PASS | `fluid` fails on cycle 2 replay (non-reactive), others reactive in same run |
|
||||
| Visualizer Lab | BLOCKED | BLOCKED | BLOCKED | BLOCKED | Style combobox is present, but song is not loaded (`Song: None selected`), play/reset disabled |
|
||||
| Project page embed | N/A | N/A | N/A | N/A | No Muon replay controls or mode selector exposed in this project surface snapshot |
|
||||
|
||||
## Evidence Snapshot
|
||||
|
||||
### 1) Track Detail (`http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb`)
|
||||
Single sweep results (cycle1/cycle2):
|
||||
- default: reactive / reactive
|
||||
- fluid: reactive / non-reactive
|
||||
- tendrils: reactive / reactive
|
||||
- spirit: reactive / reactive
|
||||
|
||||
Representative diffs:
|
||||
- default: c1=20141, c2=23362
|
||||
- fluid: c1=1552, c2=0
|
||||
- tendrils: c1=43459, c2=45607
|
||||
- spirit: c1=9952, c2=9313
|
||||
|
||||
Known caveat in session:
|
||||
- intermittent stream/network noise still appears in browser events (`ERR_ABORTED`, occasional `502`), but does not explain the mode-specific fluid replay failure by itself.
|
||||
|
||||
### 2) Visualizer Lab (`http://work:3001/visualizer-lab`)
|
||||
Observed:
|
||||
- Muon iframe present
|
||||
- Mode style options are available via `Style` combobox (`default`, `fluid`, `tendrils`, `spirit`)
|
||||
- `Song: None selected`
|
||||
- `Play preview` is disabled
|
||||
|
||||
Implication:
|
||||
- per-mode replay reactivity cannot be validated until an audio source is loaded.
|
||||
|
||||
### 3) Project page (`http://work:5174/projects/e98f2b8a-0cad-4b4e-b33e-1ca2c8c87a0a`)
|
||||
Observed in current snapshot:
|
||||
- No Muon iframe preview controls for replay testing
|
||||
- No mode buttons (`default`, `fluid`, `tendrils`, `spirit`) exposed
|
||||
- No preview play/pause controls exposed for this Muon test path
|
||||
|
||||
Implication:
|
||||
- Mode-by-mode replay matrix is N/A on this surface as currently rendered.
|
||||
|
||||
## Delta vs Prior Note
|
||||
See prior Track Detail-focused note for earlier run context:
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
|
||||
This file extends that status into a cross-surface matrix and explicitly marks where tests are blocked or not applicable.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Muon Mode Status Review Index
|
||||
|
||||
Date: 2026-04-28
|
||||
Purpose: single entry point for current Muon mode functionality, surface coverage, reference comparison, and third-party review.
|
||||
|
||||
## Document Set
|
||||
|
||||
### 1. Current Per-Mode Functionality
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
|
||||
Use this first for:
|
||||
- Track Detail preview mode-by-mode status
|
||||
- Current replay behavior by mode
|
||||
- Quick snapshot of what is working vs buggy
|
||||
|
||||
Current headline:
|
||||
- `default`: functional with caveats
|
||||
- `fluid`: reproducible replay bug
|
||||
- `tendrils`: functional in tested session, watchlist
|
||||
- `spirit`: functional in tested session, watchlist
|
||||
|
||||
### 2. Cross-Surface Matrix
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
|
||||
Use this for:
|
||||
- Comparing Track Detail, Visualizer Lab, and project/embed surfaces
|
||||
- Seeing which surfaces are testable, blocked, or N/A
|
||||
- Understanding where mode controls are or are not exposed
|
||||
|
||||
Current headline:
|
||||
- Track Detail is testable and currently shows `fluid` replay failure
|
||||
- Visualizer Lab is blocked without a loaded song
|
||||
- Project page does not currently expose this Muon replay test path
|
||||
|
||||
### 3. Reference Comparison for Third-Party Review
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
|
||||
Use this for:
|
||||
- Evaluating how local `fluid`, `tendrils`, and `spirit` compare to reference demos
|
||||
- Distinguishing verified local facts from inferred reference traits
|
||||
- Reviewing confidence levels and known uncertainty
|
||||
|
||||
Current headline:
|
||||
- `fluid` is the closest structural match to its reference
|
||||
- `tendrils` and `spirit` are better described as reference-inspired approximations than literal ports
|
||||
|
||||
### 4. Reviewer Checklist
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
|
||||
|
||||
Use this for:
|
||||
- PASS / PARTIAL / FAIL / N/A scoring
|
||||
- Consistent third-party review rubric
|
||||
- Capturing final approval / caveat decisions
|
||||
|
||||
## Recommended Reading Order
|
||||
1. `2026-04-27-current-per-mode-functionality.md`
|
||||
2. `2026-04-28-surface-functionality-matrix.md`
|
||||
3. `2026-04-28-reference-comparison-third-party-review.md`
|
||||
4. `2026-04-28-reviewer-checklist.md`
|
||||
|
||||
## Reviewer Short Summary
|
||||
If a reviewer reads only one page first, start with:
|
||||
- `2026-04-28-reference-comparison-third-party-review.md`
|
||||
|
||||
If a reviewer wants only operational status first, start with:
|
||||
- `2026-04-27-current-per-mode-functionality.md`
|
||||
|
||||
If a reviewer is validating reproducibility across UI surfaces, start with:
|
||||
- `2026-04-28-surface-functionality-matrix.md`
|
||||
|
||||
## Current Overall Assessment
|
||||
- `fluid`: closest to reference in structure, but currently weakened by replay/FBO reliability issues
|
||||
- `tendrils`: visually in-family, but likely a simplified embedded variant rather than full reference parity
|
||||
- `spirit`: visually in-family, but should be treated as stylistic approximation unless stronger source-level evidence emerges
|
||||
|
||||
## Evidence Scope Notice
|
||||
This folder contains a mix of:
|
||||
- direct source-code review,
|
||||
- live browser observation,
|
||||
- runtime probing,
|
||||
- inference from public reference pages.
|
||||
|
||||
It should be treated as a technical review package, not a claim of source-equivalent implementation.
|
||||
@@ -0,0 +1,100 @@
|
||||
# Muon Mode Status Review Index
|
||||
|
||||
Date: 2026-04-28
|
||||
Purpose: single entry point for current Muon mode functionality, surface coverage, reference comparison, and third-party review.
|
||||
|
||||
## Document Set
|
||||
|
||||
### 1. Current Per-Mode Functionality
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
|
||||
Use this first for:
|
||||
- Track Detail preview mode-by-mode status
|
||||
- Current replay behavior by mode
|
||||
- Quick snapshot of what is working vs buggy
|
||||
|
||||
Current headline:
|
||||
- `default`: functional with caveats
|
||||
- `fluid`: reproducible replay bug
|
||||
- `tendrils`: functional in tested session, watchlist
|
||||
- `spirit`: functional in tested session, watchlist
|
||||
|
||||
### 2. Cross-Surface Matrix
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
|
||||
Use this for:
|
||||
- Comparing Track Detail, Visualizer Lab, and project/embed surfaces
|
||||
- Seeing which surfaces are testable, blocked, or N/A
|
||||
- Understanding where mode controls are or are not exposed
|
||||
|
||||
Current headline:
|
||||
- Track Detail is testable and currently shows `fluid` replay failure
|
||||
- Visualizer Lab is blocked without a loaded song
|
||||
- Project page does not currently expose this Muon replay test path
|
||||
|
||||
### 3. Reference Comparison for Third-Party Review
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
|
||||
Use this for:
|
||||
- Evaluating how local `fluid`, `tendrils`, and `spirit` compare to reference demos
|
||||
- Distinguishing verified local facts from inferred reference traits
|
||||
- Reviewing confidence levels and known uncertainty
|
||||
|
||||
Current headline:
|
||||
- `fluid` is the closest structural match to its reference
|
||||
- `tendrils` and `spirit` are better described as reference-inspired approximations than literal ports
|
||||
|
||||
### 4. Reviewer Checklist
|
||||
File:
|
||||
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
|
||||
|
||||
Use this for:
|
||||
- PASS / PARTIAL / FAIL / N/A scoring
|
||||
- Consistent third-party review rubric
|
||||
- Capturing final approval / caveat decisions
|
||||
|
||||
### 5. Local Iframe Architecture Plan
|
||||
File:
|
||||
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
|
||||
|
||||
Use this for:
|
||||
- The implementation path for moving non-default styles onto a local Muon-like iframe flow
|
||||
- Phase-by-phase execution guidance
|
||||
- Host/iframe touch points, risks, and validation gates
|
||||
|
||||
## Recommended Reading Order
|
||||
1. `2026-04-27-current-per-mode-functionality.md`
|
||||
2. `2026-04-28-surface-functionality-matrix.md`
|
||||
3. `2026-04-28-reference-comparison-third-party-review.md`
|
||||
4. `2026-04-28-reviewer-checklist.md`
|
||||
5. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
|
||||
|
||||
## Reviewer Short Summary
|
||||
If a reviewer reads only one page first, start with:
|
||||
- `2026-04-28-reference-comparison-third-party-review.md`
|
||||
|
||||
If a reviewer wants only operational status first, start with:
|
||||
- `2026-04-27-current-per-mode-functionality.md`
|
||||
|
||||
If a reviewer is validating reproducibility across UI surfaces, start with:
|
||||
- `2026-04-28-surface-functionality-matrix.md`
|
||||
|
||||
If a reviewer is planning implementation work from these findings, continue with:
|
||||
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
|
||||
|
||||
## Current Overall Assessment
|
||||
- `fluid`: closest to reference in structure, but currently weakened by replay/FBO reliability issues
|
||||
- `tendrils`: visually in-family, but likely a simplified embedded variant rather than full reference parity
|
||||
- `spirit`: visually in-family, but should be treated as stylistic approximation unless stronger source-level evidence emerges
|
||||
|
||||
## Evidence Scope Notice
|
||||
This folder contains a mix of:
|
||||
- direct source-code review,
|
||||
- live browser observation,
|
||||
- runtime probing,
|
||||
- inference from public reference pages.
|
||||
|
||||
It should be treated as a technical review package, not a claim of source-equivalent implementation.
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
# Muon Visualizer Current Per-Mode Functionality
|
||||
|
||||
Date: 2026-04-27
|
||||
Environment: StemEngine track preview (http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb)
|
||||
Mode set under test: `default`, `fluid`, `tendrils`, `spirit`
|
||||
|
||||
## Scope
|
||||
This document captures the CURRENT observed behavior for Muon style modes in Track Detail preview, with focus on repeat-play reactivity.
|
||||
|
||||
Reactivity definition used here:
|
||||
- cycle 1 reactive: visual output changes after first Play preview
|
||||
- cycle 2 reactive: visual output still changes after Pause -> Play replay
|
||||
|
||||
## Summary
|
||||
- `default`: functional with caveats
|
||||
- `fluid`: reproducible bug on replay (cycle 2 non-reactive)
|
||||
- `tendrils`: functional in current sweep, but keep under watch (user-reported instability exists)
|
||||
- `spirit`: functional in current sweep, but keep under watch (user-reported instability exists)
|
||||
|
||||
## Per-Mode Status
|
||||
|
||||
### default
|
||||
Status: Functional with caveats
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: reactive (3/3 runs)
|
||||
Caveats:
|
||||
- Intermittent stream-layer noise still appears in session logs/events (`ERR_ABORTED`, occasional `502`), but did not break reactivity in this sweep.
|
||||
|
||||
### fluid
|
||||
Status: Buggy (reproducible)
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: NOT reactive (3/3 runs)
|
||||
Evidence:
|
||||
- Motion diff for cycle 2 repeatedly dropped to `0`
|
||||
- Console repeatedly logs: `[StemEngine][Fluid] FBO incomplete: 36054`
|
||||
Impact:
|
||||
- Replay reliability is broken for fluid mode under current build/runtime combination.
|
||||
|
||||
### tendrils
|
||||
Status: Functional in current session (watchlist)
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: reactive (3/3 runs)
|
||||
Caveat:
|
||||
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
|
||||
|
||||
### spirit
|
||||
Status: Functional in current session (watchlist)
|
||||
Observed behavior:
|
||||
- Cycle 1: reactive (3/3 runs)
|
||||
- Cycle 2: reactive (3/3 runs)
|
||||
Caveat:
|
||||
- User has reported this class of non-default mode as unstable; no failure reproduced in this specific sweep.
|
||||
|
||||
## Raw Sweep Snapshot (Automated)
|
||||
Run 1:
|
||||
- default: c1 reactive, c2 reactive
|
||||
- fluid: c1 reactive, c2 non-reactive
|
||||
- tendrils: c1 reactive, c2 reactive
|
||||
- spirit: c1 reactive, c2 reactive
|
||||
|
||||
Run 2:
|
||||
- default: c1 reactive, c2 reactive
|
||||
- fluid: c1 reactive, c2 non-reactive
|
||||
- tendrils: c1 reactive, c2 reactive
|
||||
- spirit: c1 reactive, c2 reactive
|
||||
|
||||
Run 3:
|
||||
- default: c1 reactive, c2 reactive
|
||||
- fluid: c1 reactive, c2 non-reactive
|
||||
- tendrils: c1 reactive, c2 reactive
|
||||
- spirit: c1 reactive, c2 reactive
|
||||
|
||||
## Notes
|
||||
- This report is intentionally point-in-time. Re-run after renderer/transport changes to keep this file current.
|
||||
- If you want, add a companion matrix file for Visualizer Lab page and project-page embed to compare behavior across surfaces.
|
||||
+104
@@ -0,0 +1,104 @@
|
||||
# Muon Review Export Bundle Manifest
|
||||
|
||||
Date: 2026-04-28
|
||||
Purpose: frozen manifest for external sharing of the Muon mode review package.
|
||||
|
||||
## Bundle Name
|
||||
Suggested bundle label:
|
||||
- `muon-mode-review-package-2026-04-28`
|
||||
|
||||
## Bundle Scope
|
||||
This export bundle is intended for third-party reviewers who need:
|
||||
- current operational status by mode,
|
||||
- surface-by-surface testability notes,
|
||||
- reference-comparison analysis,
|
||||
- a reviewer scoring rubric,
|
||||
- a single index file for navigation.
|
||||
|
||||
## Files To Include
|
||||
|
||||
### Core review docs
|
||||
1. `docs/muon-mode-status/README.md`
|
||||
- Entry point and reading order.
|
||||
|
||||
2. `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- Point-in-time Track Detail mode functionality and replay observations.
|
||||
|
||||
3. `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
- Surface matrix across Track Detail, Visualizer Lab, and project/embed views.
|
||||
|
||||
4. `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
- Reference comparison for `fluid`, `tendrils`, and `spirit`.
|
||||
|
||||
5. `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
|
||||
- Third-party scoring and signoff rubric.
|
||||
|
||||
6. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
|
||||
- Implementation plan for moving non-default styles to a local iframe visualizer runtime.
|
||||
|
||||
### Optional supporting source files
|
||||
Include these if the reviewer needs implementation-level evidence:
|
||||
7. `muon-iframe/fluid-render-graph.js`
|
||||
8. `muon-iframe/tendrils-render-graph.js`
|
||||
9. `muon-iframe/spirit-render-graph.js`
|
||||
10. `muon-iframe/stemengine-router.js`
|
||||
11. `muon-iframe/stemengine-bootstrap.js`
|
||||
|
||||
## Minimal Bundle
|
||||
If the recipient should receive docs only, export:
|
||||
- `docs/muon-mode-status/README.md`
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
- `docs/muon-mode-status/2026-04-28-reviewer-checklist.md`
|
||||
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
|
||||
|
||||
## Full Evidence Bundle
|
||||
If the recipient should receive both review docs and implementation evidence, export:
|
||||
- all files from the Minimal Bundle
|
||||
- `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
|
||||
## Suggested Folder Layout For Sharing
|
||||
```text
|
||||
muon-mode-review-package-2026-04-28/
|
||||
README.md
|
||||
docs/
|
||||
2026-04-27-current-per-mode-functionality.md
|
||||
2026-04-28-export-bundle-manifest.md
|
||||
2026-04-28-surface-functionality-matrix.md
|
||||
2026-04-28-reference-comparison-third-party-review.md
|
||||
2026-04-28-reviewer-checklist.md
|
||||
plans/
|
||||
2026-04-28_local-iframe-visualizer-architecture-plan.md
|
||||
source-evidence/
|
||||
fluid-render-graph.js
|
||||
tendrils-render-graph.js
|
||||
spirit-render-graph.js
|
||||
stemengine-router.js
|
||||
stemengine-bootstrap.js
|
||||
```
|
||||
|
||||
## Export Notes
|
||||
- This manifest freezes the intended review set as of 2026-04-28.
|
||||
- The bundle is evidence-oriented, not a claim of source-equivalent parity with external references.
|
||||
- Reference-comparison conclusions include inference and confidence levels; reviewers should read those caveats before scoring fidelity.
|
||||
|
||||
## Reviewer Guidance
|
||||
Recommended reading sequence inside the exported bundle:
|
||||
1. `README.md`
|
||||
2. `2026-04-28-reference-comparison-third-party-review.md`
|
||||
3. `2026-04-27-current-per-mode-functionality.md`
|
||||
4. `2026-04-28-surface-functionality-matrix.md`
|
||||
5. `2026-04-28-reviewer-checklist.md`
|
||||
6. `docs/plans/2026-04-28_local-iframe-visualizer-architecture-plan.md`
|
||||
|
||||
## Current Package Status
|
||||
As of this manifest:
|
||||
- `fluid` remains the main concern because of replay/FBO reliability issues.
|
||||
- `tendrils` and `spirit` are documented as in-family approximations, not proven literal ports.
|
||||
- Visualizer Lab and project/embed surfaces remain limited or non-testable for this exact replay matrix without additional setup.
|
||||
+141
@@ -0,0 +1,141 @@
|
||||
# Muon Mode Comparison vs Reference Demos (Third-Party Review)
|
||||
|
||||
Date: 2026-04-28
|
||||
Prepared for: external/third-party technical review
|
||||
Scope: infer how local `fluid`, `tendrils`, and `spirit` modes compare to their likely inspiration references.
|
||||
|
||||
References under review:
|
||||
- https://paveldogreat.github.io/WebGL-Fluid-Simulation/
|
||||
- https://keeffeoghan.github.io/work/tendrils/ (redirects to `epok.tech` content)
|
||||
- https://edankwan.com/experiments/the-spirit/
|
||||
|
||||
Local implementation inputs reviewed:
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
|
||||
## Method Notes
|
||||
This review intentionally separates:
|
||||
- verified local facts (from readable source files), and
|
||||
- inferred reference characteristics (from live pages + publicly visible descriptions + agent-assisted analysis).
|
||||
|
||||
A `ui-designer` subagent was used to produce an independent reference summary, then cross-checked against local code.
|
||||
|
||||
## Executive Summary
|
||||
1. `fluid` in this fork is structurally the closest to its reference target (PavelDoGreat fluid simulation), because it implements a recognizable multi-pass fluid pipeline (splat, divergence, pressure Jacobi, gradient subtract, advection, bloom).
|
||||
2. `tendrils` and `spirit` in this fork are stylistic approximations, not faithful reproductions of the original interactive systems. They are compact shader-feedback variants optimized for integration into Muon’s transport/audio stack.
|
||||
3. The local `fluid` path currently has a practical stability/replay issue in your environment (`FBO incomplete: 36054` warnings and cycle-2 non-reactivity), which materially affects parity with the expected “real thing” feel.
|
||||
|
||||
## Comparison Matrix
|
||||
|
||||
| Mode | Local Implementation (verified) | Reference “Real Thing” (inferred) | Similarity Assessment |
|
||||
|---|---|---|---|
|
||||
| fluid | Full-screen multipass FBO fluid pipeline with pressure solve, advection, bloom, and audio splat injection | Interactive Navier-Stokes style GPU fluid sim with user splats and polished post FX | High structural similarity; medium experiential similarity due to current stability issues and integration constraints |
|
||||
| tendrils | Feedback texture + procedural flow/noise line structures in one update shader, audio-driven decay/flow | Emergent particle-field ecosystem with richer control surface, presets, and interaction channels (keyboard/camera/mic) | Medium visual similarity; low-to-medium system similarity |
|
||||
| spirit | Layered atmospheric feedback shader with fbm/noise, halo/strand/pulse composition, audio-driven fade | Highly polished ethereal particle/trail experience with strong cinematic motion language | Medium stylistic similarity; low implementation-parity confidence |
|
||||
|
||||
## Per-Mode Detailed Assessment
|
||||
|
||||
### A) `fluid`
|
||||
Local verified facts:
|
||||
- Uses explicit fluid-solver style passes:
|
||||
- curl, divergence, pressure Jacobi, gradient subtract, velocity advection, dye advection.
|
||||
- Uses ping-pong FBOs for velocity/dye/pressure and optional bloom chain by quality tier.
|
||||
- Audio coupling includes bass/sub/treble/rms/beat/beatPhase/drumIntensity mapped to splat radius/force/color.
|
||||
- Quality tier controls pressure iterations and bloom enablement.
|
||||
|
||||
Evidence (local code):
|
||||
- `muon-iframe/fluid-render-graph.js` (pipeline and FBO management)
|
||||
- `muon-iframe/stemengine-router.js` (style lifecycle + audio feature push)
|
||||
|
||||
Reference inferred characteristics:
|
||||
- Canonical GPU fluid simulation style with strong interactive splat behavior and stable continuous flow.
|
||||
|
||||
Assessment:
|
||||
- Architecture match is strong.
|
||||
- Main gap is operational reliability in your current stack: observed `FBO incomplete: 36054` warnings and replay-cycle failures reduce real-world parity.
|
||||
|
||||
Confidence:
|
||||
- Local implementation confidence: High
|
||||
- Reference internals confidence: Medium
|
||||
- Similarity conclusion confidence: Medium-High
|
||||
|
||||
### B) `tendrils`
|
||||
Local verified facts:
|
||||
- Two-program design (`update` + `blit`) with framebuffer feedback loop.
|
||||
- Motion built from angle/radius flow fields + hash noise + trail decay.
|
||||
- Audio mapping primarily modulates flow rate, turbulence, color injection, and decay.
|
||||
- No explicit particle-buffer architecture exposed; behavior emerges from texture feedback math.
|
||||
|
||||
Evidence (local code):
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
|
||||
Reference inferred characteristics:
|
||||
- Public text describes interacting particle system with advection-like behavior, optional optical flow/camera/mic interactions, rich controls and presets.
|
||||
|
||||
Assessment:
|
||||
- The local mode captures the “organic tendril motion” aesthetic.
|
||||
- It appears materially simplified versus the broader interaction/control ecosystem and potentially deeper particle mechanics of the reference.
|
||||
|
||||
Confidence:
|
||||
- Local implementation confidence: High
|
||||
- Reference internals confidence: Medium-Low
|
||||
- Similarity conclusion confidence: Medium
|
||||
|
||||
### C) `spirit`
|
||||
Local verified facts:
|
||||
- Two-pass feedback renderer with atmospheric layering:
|
||||
- fbm-like noise term,
|
||||
- radial halo decay,
|
||||
- strand/pulse functions,
|
||||
- additive glow and nonlinear output shaping.
|
||||
- Audio mapping influences drift, fade persistence, pulse intensity, and color blend.
|
||||
|
||||
Evidence (local code):
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
|
||||
Reference inferred characteristics:
|
||||
- Highly polished ethereal motion piece with strong artistic direction and likely richer scene dynamics.
|
||||
|
||||
Assessment:
|
||||
- The local mode appears designed as a stylistic homage (ethereal layered glow) rather than a strict algorithmic clone.
|
||||
- Without source-level access to reference internals, parity can only be judged at visual-language level.
|
||||
|
||||
Confidence:
|
||||
- Local implementation confidence: High
|
||||
- Reference internals confidence: Low-Medium
|
||||
- Similarity conclusion confidence: Medium-Low
|
||||
|
||||
## Integration Context (Important for Reviewers)
|
||||
The local renderers are not standalone art demos; they are embedded into Muon transport and controlled by host commands.
|
||||
|
||||
Observed integration traits:
|
||||
- Router-driven style switching and lifecycle disposal/instantiation.
|
||||
- Host-driven audio feature push throttled to iframe RAF cadence.
|
||||
- Host-driven pause/play/seek/rate semantics through injected audio engine.
|
||||
- Baseline WebGL extension checks in bootstrap layer.
|
||||
|
||||
Implication:
|
||||
- Even when visual math resembles a reference demo, behavior can diverge due to transport timing, iframe lifecycle, and browser/driver extension availability.
|
||||
|
||||
## Known Gaps and Risks
|
||||
1. Reference internals are partly opaque (minified/demo-only), so comparison is inference-based, not source-equivalence.
|
||||
2. `fluid` currently shows environment-specific fragility (`FBO incomplete: 36054` warnings observed in session), impacting practical parity.
|
||||
3. `tendrils`/`spirit` appear intentionally compact; they likely trade breadth/depth of original interactivity for predictable embedding in Muon.
|
||||
|
||||
## Third-Party Review Conclusion
|
||||
- If the review criterion is “same algorithm/system as originals”: not demonstrated for `tendrils` or `spirit`; only partial for `fluid` (closest).
|
||||
- If criterion is “same visual family integrated into Muon with audio transport”: yes, with `fluid` currently hindered by reliability caveats.
|
||||
- Recommended interpretation: these are reference-inspired implementations with varying structural closeness, not literal ports.
|
||||
|
||||
## Suggested Evidence Bundle for Auditors
|
||||
Include these files in review packets:
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
+152
@@ -0,0 +1,152 @@
|
||||
# Muon Reviewer Checklist
|
||||
|
||||
Date: 2026-04-28
|
||||
Purpose: quick third-party parity and functionality review for Muon `fluid`, `tendrils`, and `spirit` modes.
|
||||
|
||||
Related documents:
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
- `docs/muon-mode-status/2026-04-28-surface-functionality-matrix.md`
|
||||
- `docs/muon-mode-status/2026-04-28-reference-comparison-third-party-review.md`
|
||||
|
||||
## How To Use
|
||||
For each mode, mark:
|
||||
- PASS: clearly meets the criterion
|
||||
- PARTIAL: present but degraded, unstable, or notably simplified
|
||||
- FAIL: absent, broken, or materially unlike expected behavior
|
||||
- N/A: not exposed on the reviewed surface
|
||||
|
||||
Recommended review surfaces:
|
||||
- Track Detail preview
|
||||
- Visualizer Lab
|
||||
- Any project/embed surface exposing Muon controls
|
||||
|
||||
## Reviewer Metadata
|
||||
- Reviewer:
|
||||
- Date:
|
||||
- Browser / version:
|
||||
- GPU / driver:
|
||||
- Surface tested:
|
||||
- Audio source used:
|
||||
- Quality tier tested:
|
||||
|
||||
## Section 1: Basic Functionality
|
||||
|
||||
### Mode availability
|
||||
- `fluid` is selectable
|
||||
- `tendrils` is selectable
|
||||
- `spirit` is selectable
|
||||
- Mode switch does not visibly break iframe/canvas
|
||||
|
||||
### Playback / transport
|
||||
- First play starts visual motion
|
||||
- Pause stops or meaningfully freezes progression
|
||||
- Replay after pause resumes visual motion
|
||||
- Seek while playing preserves visual responsiveness
|
||||
- Mode remains reactive for at least two play cycles
|
||||
|
||||
### Stability
|
||||
- No blank canvas on mode switch
|
||||
- No persistent frozen frame after replay
|
||||
- No repeated fatal errors in console
|
||||
- No WebGL context loss during normal use
|
||||
|
||||
## Section 2: Visual Parity vs Reference
|
||||
|
||||
### `fluid`
|
||||
Expected reference family:
|
||||
- Fluid-like motion with continuous advection
|
||||
- Visible diffusion / swirling / dye transport feel
|
||||
- Stronger impulse on beats or major peaks
|
||||
- Cohesive full-frame flow rather than isolated sprite effects
|
||||
|
||||
Review marks:
|
||||
- Flow behavior resembles fluid simulation
|
||||
- Beat/bass energy visibly changes motion intensity
|
||||
- Color field behaves like transported dye, not only flashing overlays
|
||||
- Motion remains continuous across replay cycles
|
||||
- Presentation feels structurally close to the PavelDoGreat reference
|
||||
|
||||
Overall `fluid` score: PASS / PARTIAL / FAIL
|
||||
Notes:
|
||||
|
||||
### `tendrils`
|
||||
Expected reference family:
|
||||
- Organic line or filament structures
|
||||
- Emergent motion with feedback / trace persistence
|
||||
- Sense of interconnected or fibrous forms
|
||||
- More than a simple radial pulse or stock waveform
|
||||
|
||||
Review marks:
|
||||
- Visual output forms tendril/filament-like structures
|
||||
- Trails persist and evolve over time
|
||||
- Motion feels emergent rather than strictly canned
|
||||
- Audio affects density, color, or movement in a noticeable way
|
||||
- Presentation feels recognizably related to the Tendrils reference family
|
||||
|
||||
Overall `tendrils` score: PASS / PARTIAL / FAIL
|
||||
Notes:
|
||||
|
||||
### `spirit`
|
||||
Expected reference family:
|
||||
- Ethereal / atmospheric glow language
|
||||
- Layered soft motion rather than sharp geometric tracing
|
||||
- Strong sense of aura, halo, strand, smoke, or apparition-like motion
|
||||
- Audio-linked pulse or drift without collapsing into a basic particle burst
|
||||
|
||||
Review marks:
|
||||
- Visual output feels atmospheric / spectral / ethereal
|
||||
- Glow and layering read clearly on screen
|
||||
- Audio changes drift, pulse, or luminosity in a visible way
|
||||
- Motion remains smooth across play / pause / replay
|
||||
- Presentation feels recognizably related to the Spirit reference family
|
||||
|
||||
Overall `spirit` score: PASS / PARTIAL / FAIL
|
||||
Notes:
|
||||
|
||||
## Section 3: Surface Matrix Check
|
||||
Mark each surface:
|
||||
|
||||
### Track Detail
|
||||
- `fluid`: PASS / PARTIAL / FAIL / N/A
|
||||
- `tendrils`: PASS / PARTIAL / FAIL / N/A
|
||||
- `spirit`: PASS / PARTIAL / FAIL / N/A
|
||||
Notes:
|
||||
|
||||
### Visualizer Lab
|
||||
- `fluid`: PASS / PARTIAL / FAIL / N/A
|
||||
- `tendrils`: PASS / PARTIAL / FAIL / N/A
|
||||
- `spirit`: PASS / PARTIAL / FAIL / N/A
|
||||
Notes:
|
||||
|
||||
### Project / embed surface
|
||||
- `fluid`: PASS / PARTIAL / FAIL / N/A
|
||||
- `tendrils`: PASS / PARTIAL / FAIL / N/A
|
||||
- `spirit`: PASS / PARTIAL / FAIL / N/A
|
||||
Notes:
|
||||
|
||||
## Section 4: Red Flags
|
||||
Check any that apply:
|
||||
- Replay fails only on some modes
|
||||
- Mode works on first play but freezes on second play
|
||||
- Console shows framebuffer / FBO warnings
|
||||
- Visuals appear materially simpler than reference expectations
|
||||
- Surface exposes controls but is not actually testable
|
||||
- Behavior differs substantially between surfaces
|
||||
|
||||
## Section 5: Final Verdict
|
||||
Use this rubric:
|
||||
- PASS: mode is stable and recognizably in the same reference family
|
||||
- PARTIAL: mode works but has obvious reliability, fidelity, or surface gaps
|
||||
- FAIL: mode is broken or not meaningfully comparable to its reference family
|
||||
|
||||
Final verdict by mode:
|
||||
- `fluid`:
|
||||
- `tendrils`:
|
||||
- `spirit`:
|
||||
|
||||
Overall release recommendation:
|
||||
- APPROVE for review
|
||||
- APPROVE WITH CAVEATS
|
||||
- DO NOT APPROVE
|
||||
|
||||
Summary comments:
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
# Muon Surface Functionality Matrix
|
||||
|
||||
Date: 2026-04-28
|
||||
|
||||
Surfaces covered:
|
||||
- Track Detail preview (`/tracks/:trackId`)
|
||||
- Visualizer Lab (`/visualizer-lab`)
|
||||
- Project page (`/projects/:projectId` on webui host)
|
||||
|
||||
## Legend
|
||||
- PASS: verified reactive in current session
|
||||
- FAIL: reproducible non-reactive behavior in current session
|
||||
- BLOCKED: surface/control exists but cannot execute mode replay test without additional input/state
|
||||
- N/A: control path for this Muon mode test is not exposed on this surface
|
||||
|
||||
## Matrix (Current)
|
||||
|
||||
| Surface | default | fluid | tendrils | spirit | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| Track Detail preview | PASS | FAIL | PASS | PASS | `fluid` fails on cycle 2 replay (non-reactive), others reactive in same run |
|
||||
| Visualizer Lab | BLOCKED | BLOCKED | BLOCKED | BLOCKED | Style combobox is present, but song is not loaded (`Song: None selected`), play/reset disabled |
|
||||
| Project page embed | N/A | N/A | N/A | N/A | No Muon replay controls or mode selector exposed in this project surface snapshot |
|
||||
|
||||
## Evidence Snapshot
|
||||
|
||||
### 1) Track Detail (`http://work:3001/tracks/a5f76485ba8f7e5c41aea2c6a41ca0eb`)
|
||||
Single sweep results (cycle1/cycle2):
|
||||
- default: reactive / reactive
|
||||
- fluid: reactive / non-reactive
|
||||
- tendrils: reactive / reactive
|
||||
- spirit: reactive / reactive
|
||||
|
||||
Representative diffs:
|
||||
- default: c1=20141, c2=23362
|
||||
- fluid: c1=1552, c2=0
|
||||
- tendrils: c1=43459, c2=45607
|
||||
- spirit: c1=9952, c2=9313
|
||||
|
||||
Known caveat in session:
|
||||
- intermittent stream/network noise still appears in browser events (`ERR_ABORTED`, occasional `502`), but does not explain the mode-specific fluid replay failure by itself.
|
||||
|
||||
### 2) Visualizer Lab (`http://work:3001/visualizer-lab`)
|
||||
Observed:
|
||||
- Muon iframe present
|
||||
- Mode style options are available via `Style` combobox (`default`, `fluid`, `tendrils`, `spirit`)
|
||||
- `Song: None selected`
|
||||
- `Play preview` is disabled
|
||||
|
||||
Implication:
|
||||
- per-mode replay reactivity cannot be validated until an audio source is loaded.
|
||||
|
||||
### 3) Project page (`http://work:5174/projects/e98f2b8a-0cad-4b4e-b33e-1ca2c8c87a0a`)
|
||||
Observed in current snapshot:
|
||||
- No Muon iframe preview controls for replay testing
|
||||
- No mode buttons (`default`, `fluid`, `tendrils`, `spirit`) exposed
|
||||
- No preview play/pause controls exposed for this Muon test path
|
||||
|
||||
Implication:
|
||||
- Mode-by-mode replay matrix is N/A on this surface as currently rendered.
|
||||
|
||||
## Delta vs Prior Note
|
||||
See prior Track Detail-focused note for earlier run context:
|
||||
- `docs/muon-mode-status/2026-04-27-current-per-mode-functionality.md`
|
||||
|
||||
This file extends that status into a cross-surface matrix and explicitly marks where tests are blocked or not applicable.
|
||||
+346
@@ -0,0 +1,346 @@
|
||||
# Local Iframe Visualizer Architecture Plan
|
||||
|
||||
Date: 2026-04-28
|
||||
Status: Draft implementation plan
|
||||
Owner: StemEngine frontend / visualizer pipeline
|
||||
|
||||
## Problem Statement
|
||||
The current attempt to approximate `fluid`, `tendrils`, and `spirit` inside the existing Muon iframe path did not deliver a reliable result.
|
||||
|
||||
Observed issues:
|
||||
- `default` mode is the only mode currently behaving close to the intended repeat-play iframe flow.
|
||||
- `fluid` shows reproducible replay instability and framebuffer warnings.
|
||||
- `tendrils` and `spirit` may look visually in-family, but are still bound to Muon-specific lifecycle assumptions that make them hard to validate and stabilize.
|
||||
|
||||
## Goal
|
||||
Use the same host-to-iframe transport model for every visualizer style, but move non-default styles onto a local iframe runtime that behaves like Muon from the host’s point of view.
|
||||
|
||||
Target outcome:
|
||||
- the host speaks one iframe transport contract,
|
||||
- each style runs in an iframe-managed renderer lifecycle,
|
||||
- replay / pause / seek / rate / resize are validated the same way for all styles,
|
||||
- non-default styles are no longer dependent on Muon internals that were designed for the default renderer.
|
||||
|
||||
## Decision Summary
|
||||
Recommended architecture:
|
||||
- Keep the existing Muon iframe provider for `default` for now.
|
||||
- Build a local iframe visualizer runtime for `fluid`, `tendrils`, and `spirit`.
|
||||
- Standardize the transport contract so the host treats both as iframe visualizer providers.
|
||||
|
||||
Preferred shape:
|
||||
- One shared local iframe shell with pluggable style engines.
|
||||
|
||||
Fallback shape:
|
||||
- One local iframe app per style.
|
||||
|
||||
Recommendation:
|
||||
- Use one shared local iframe shell unless style-specific boot constraints prove otherwise.
|
||||
|
||||
## Why This Direction
|
||||
Benefits:
|
||||
- separates transport concerns from render math,
|
||||
- removes style replay behavior from Muon private/default renderer assumptions,
|
||||
- reduces duplicate host logic,
|
||||
- gives one lifecycle model for all local styles,
|
||||
- makes validation and third-party review clearer.
|
||||
|
||||
## Scope
|
||||
In scope:
|
||||
- iframe transport contract standardization,
|
||||
- host visualizer provider abstraction,
|
||||
- local iframe shell runtime,
|
||||
- local renderer plugin API,
|
||||
- migration of `fluid`, `tendrils`, and `spirit` to local iframe shell,
|
||||
- validation matrix across Track Detail, Visualizer Lab, and other exposed surfaces.
|
||||
|
||||
Out of scope for first pass:
|
||||
- replacing default Muon renderer,
|
||||
- perfect fidelity to third-party reference demos,
|
||||
- broad preset-authoring UX,
|
||||
- camera/mic/advanced interaction parity with original external demos.
|
||||
|
||||
## Current File Touch Points
|
||||
### Host-side frontend
|
||||
Likely primary touch points:
|
||||
- `frontend/src/hooks/useMuonIframeTransport.ts`
|
||||
- `frontend/src/components/VisualizerPreviewWidget.tsx`
|
||||
- `frontend/src/components/LyricsLivePreview.tsx`
|
||||
- `frontend/src/pages/VisualizerLabPage.tsx`
|
||||
- `frontend/src/pages/TrackDetail.tsx`
|
||||
|
||||
Likely new host-side additions:
|
||||
- `frontend/src/visualizers/providers/*`
|
||||
- `frontend/src/hooks/useIframeVisualizerTransport.ts`
|
||||
- `frontend/src/types/iframeVisualizer.ts`
|
||||
|
||||
### Existing Muon iframe runtime
|
||||
Reference implementation / source of reusable logic:
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
- `muon-iframe/stemengine-audio-engine.js`
|
||||
- `muon-iframe/stemengine-feature-extractor.js`
|
||||
|
||||
### Existing style renderers to migrate or adapt
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
|
||||
### Likely new local iframe app area
|
||||
Suggested location:
|
||||
- `frontend/local-visualizer-iframe/` or
|
||||
- `local-visualizer-iframe/`
|
||||
|
||||
Recommendation:
|
||||
- keep it separate from `muon-iframe/` so it is not build-time injected into Muon.
|
||||
|
||||
## Target Architecture
|
||||
### 1. Host visualizer provider layer
|
||||
The host should target a provider abstraction, not Muon directly.
|
||||
|
||||
Proposed provider model:
|
||||
- provider id
|
||||
- iframe URL
|
||||
- supported styles
|
||||
- capabilities
|
||||
- command protocol version
|
||||
- fallback behavior
|
||||
|
||||
Example providers:
|
||||
- `muon-default`
|
||||
- `local-visualizer`
|
||||
|
||||
### 2. Shared iframe transport contract
|
||||
Commands from host to iframe:
|
||||
- `init`
|
||||
- `setStyle`
|
||||
- `setPreset`
|
||||
- `setQuality`
|
||||
- `setPaused`
|
||||
- `setViewport`
|
||||
- `loadAudio`
|
||||
- `transportPlay`
|
||||
- `transportPause`
|
||||
- `transportSeek`
|
||||
- `transportRate`
|
||||
- `pushAudioFeatures`
|
||||
- `dispose`
|
||||
|
||||
Events from iframe to host:
|
||||
- `ready`
|
||||
- `loaded`
|
||||
- `capabilities`
|
||||
- `state`
|
||||
- `warning`
|
||||
- `error`
|
||||
|
||||
Required rules:
|
||||
- host owns transport intent and source audio selection,
|
||||
- iframe owns rendering lifecycle,
|
||||
- seek while playing must not leave renderer visually paused,
|
||||
- replay after pause must be deterministic,
|
||||
- viewport updates must not require reload.
|
||||
|
||||
### 3. Local iframe shell
|
||||
Responsibilities:
|
||||
- boot and emit `ready`,
|
||||
- set up audio engine,
|
||||
- set up feature extractor,
|
||||
- receive commands from host,
|
||||
- instantiate exactly one renderer at a time,
|
||||
- dispose renderer cleanly on style switch or teardown,
|
||||
- expose warnings/errors back to host.
|
||||
|
||||
Shell should be style-agnostic.
|
||||
|
||||
### 4. Renderer plugin API
|
||||
Each style renderer should conform to the same API.
|
||||
|
||||
Proposed renderer interface:
|
||||
- `init(context)`
|
||||
- `setViewport(viewport)`
|
||||
- `setQuality(quality)`
|
||||
- `setPreset(preset)`
|
||||
- `setPaused(paused)`
|
||||
- `pushFeatures(features)`
|
||||
- `tick(time)`
|
||||
- `dispose()`
|
||||
|
||||
Renderer should not own:
|
||||
- transport protocol,
|
||||
- source audio loading,
|
||||
- play/pause/seek intent,
|
||||
- message routing.
|
||||
|
||||
### 5. Migration model
|
||||
Near-term routing:
|
||||
- `default` -> existing Muon iframe provider
|
||||
- `fluid` -> local iframe shell
|
||||
- `tendrils` -> local iframe shell
|
||||
- `spirit` -> local iframe shell
|
||||
|
||||
Future option:
|
||||
- move `default` under the same provider system later if beneficial.
|
||||
|
||||
## Implementation Phases
|
||||
## Phase 1: Freeze the contract
|
||||
Deliverables:
|
||||
- one transport spec document,
|
||||
- one provider abstraction design,
|
||||
- explicit replay/seek/resize semantics.
|
||||
|
||||
Tasks:
|
||||
- document current useful Muon messages,
|
||||
- remove Muon-specific assumptions from the conceptual transport model,
|
||||
- define one versioned protocol for all iframe visualizers.
|
||||
|
||||
Exit criteria:
|
||||
- host and iframe responsibilities are explicitly defined,
|
||||
- no new style-specific transport behavior is added ad hoc.
|
||||
|
||||
## Phase 2: Introduce provider abstraction on host
|
||||
Deliverables:
|
||||
- provider registry,
|
||||
- host hook abstracted away from Muon naming,
|
||||
- widgets/pages able to select a provider.
|
||||
|
||||
Tasks:
|
||||
- refactor `useMuonIframeTransport.ts` into provider-neutral transport hook,
|
||||
- add provider config for default Muon and local iframe runtime,
|
||||
- update consumers in preview surfaces.
|
||||
|
||||
Exit criteria:
|
||||
- Track Detail and Visualizer Lab can load an iframe provider without hard-coding Muon-only assumptions.
|
||||
|
||||
## Phase 3: Build local iframe shell
|
||||
Deliverables:
|
||||
- local iframe app booting independently,
|
||||
- postMessage router,
|
||||
- audio engine and feature extractor wiring,
|
||||
- renderer lifecycle manager.
|
||||
|
||||
Tasks:
|
||||
- clone only the reusable runtime concepts from current Muon integration,
|
||||
- avoid build-time injection into third-party Muon source,
|
||||
- add explicit renderer registration and disposal.
|
||||
|
||||
Exit criteria:
|
||||
- shell loads, emits ready, can accept audio load/play/pause/seek/viewport commands even with a placeholder renderer.
|
||||
|
||||
## Phase 4: Migrate `fluid`
|
||||
Why first:
|
||||
- it is the closest structural match to its intended reference,
|
||||
- it is currently the least reliable mode.
|
||||
|
||||
Tasks:
|
||||
- adapt current fluid renderer to the new renderer API,
|
||||
- enforce init-time capability checks,
|
||||
- surface explicit degrade/failure states when FBO support is incomplete,
|
||||
- validate replay and resize in the new shell.
|
||||
|
||||
Exit criteria:
|
||||
- `fluid` passes two-cycle replay test in Track Detail,
|
||||
- no silent frozen-canvas failure path,
|
||||
- FBO failure produces explicit host-visible warning/error.
|
||||
|
||||
## Phase 5: Migrate `tendrils` and `spirit`
|
||||
Tasks:
|
||||
- adapt both renderers to shared shell lifecycle,
|
||||
- remove transport ownership assumptions from render graphs,
|
||||
- validate style switch, replay, resize, and quality transitions.
|
||||
|
||||
Exit criteria:
|
||||
- both renderers pass same iframe lifecycle tests as `fluid`,
|
||||
- no per-style transport branching on host.
|
||||
|
||||
## Phase 6: Cross-surface validation and rollout
|
||||
Tasks:
|
||||
- validate Track Detail,
|
||||
- validate Visualizer Lab with a loaded song path,
|
||||
- validate any project/embed surface that exposes iframe preview,
|
||||
- update docs and reviewer package with post-migration results.
|
||||
|
||||
Exit criteria:
|
||||
- the same style behaves consistently across supported surfaces,
|
||||
- unsupported surfaces are explicitly documented.
|
||||
|
||||
## Validation Matrix
|
||||
Each migrated style must pass:
|
||||
- iframe boot / ready
|
||||
- audio load
|
||||
- play cycle 1
|
||||
- pause
|
||||
- replay cycle 2
|
||||
- seek while playing
|
||||
- seek while paused
|
||||
- rate change
|
||||
- viewport resize
|
||||
- quality change
|
||||
- style switch away and back
|
||||
- teardown / remount
|
||||
|
||||
Required test surfaces:
|
||||
- Track Detail
|
||||
- Visualizer Lab
|
||||
- any active project/embed preview surface
|
||||
|
||||
## Key Risks
|
||||
1. Host abstraction drift
|
||||
Risk:
|
||||
- partial refactor leaves old Muon-only codepaths mixed with provider abstraction.
|
||||
Mitigation:
|
||||
- centralize provider selection in one registry and one transport hook.
|
||||
|
||||
2. Fluid hardware variability
|
||||
Risk:
|
||||
- local fluid renderer still fails on certain WebGL / extension combinations.
|
||||
Mitigation:
|
||||
- explicit capability detection and degrade behavior, not silent failure.
|
||||
|
||||
3. Renderer lifecycle leaks
|
||||
Risk:
|
||||
- RAF loops, textures, or framebuffers survive style switches.
|
||||
Mitigation:
|
||||
- strict renderer API and disposable lifecycle manager in shell.
|
||||
|
||||
4. Visualizer Lab remains blocked
|
||||
Risk:
|
||||
- shell exists but lab still cannot load audio cleanly.
|
||||
Mitigation:
|
||||
- define one supported sample-input path early in Phase 3.
|
||||
|
||||
5. Fidelity pressure before lifecycle stability
|
||||
Risk:
|
||||
- development time is spent tuning visuals before replay and resize are correct.
|
||||
Mitigation:
|
||||
- block fidelity work until lifecycle acceptance criteria pass.
|
||||
|
||||
## Deliverables Checklist
|
||||
- transport spec doc
|
||||
- provider registry
|
||||
- provider-neutral iframe transport hook
|
||||
- local iframe shell app
|
||||
- renderer plugin API
|
||||
- migrated `fluid`
|
||||
- migrated `tendrils`
|
||||
- migrated `spirit`
|
||||
- updated validation docs
|
||||
- updated review package
|
||||
|
||||
## Recommended Execution Order
|
||||
1. Freeze transport spec
|
||||
2. Refactor host to provider model
|
||||
3. Build local iframe shell with placeholder renderer
|
||||
4. Migrate `fluid`
|
||||
5. Stabilize replay / resize / seek
|
||||
6. Migrate `tendrils`
|
||||
7. Migrate `spirit`
|
||||
8. Re-run cross-surface validation
|
||||
9. Update external review package
|
||||
|
||||
## Success Definition
|
||||
This plan succeeds when:
|
||||
- non-default styles no longer rely on Muon’s default-renderer internals,
|
||||
- host uses one iframe transport model across providers,
|
||||
- replay behavior is deterministic for all supported styles,
|
||||
- failures are surfaced explicitly instead of appearing as frozen visuals,
|
||||
- reviewers can evaluate styles through a consistent iframe lifecycle rather than special-case paths.
|
||||
+1118
File diff suppressed because it is too large.
Load diff
+273
@@ -0,0 +1,273 @@
|
||||
/* StemEngine Spirit Render Graph (Muon fork overlay)
|
||||
* Atmospheric layered glow visualization with audio reactivity.
|
||||
*/
|
||||
|
||||
;(function stemengineSpiritRenderGraph() {
|
||||
'use strict'
|
||||
|
||||
var QUALITY_SCALE = { low: 0.8, balanced: 1.0, high: 1.25 }
|
||||
|
||||
function clamp01(v) {
|
||||
var n = Number(v)
|
||||
if (!isFinite(n)) n = 0
|
||||
return Math.max(0, Math.min(1, n))
|
||||
}
|
||||
|
||||
function shaped(v, power) {
|
||||
return Math.pow(clamp01(v), power)
|
||||
}
|
||||
|
||||
function getBand(audio, key, fallback) {
|
||||
var bands = audio && audio.frequencyBands ? audio.frequencyBands : null
|
||||
var value = bands && isFinite(Number(bands[key])) ? Number(bands[key]) : fallback
|
||||
return clamp01(value)
|
||||
}
|
||||
|
||||
function compileProgram(gl, vsSrc, fsSrc) {
|
||||
function c(type, src) {
|
||||
var s = gl.createShader(type)
|
||||
gl.shaderSource(s, src)
|
||||
gl.compileShader(s)
|
||||
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) {
|
||||
var e = gl.getShaderInfoLog(s) || 'shader compile failed'
|
||||
window.__stemengineEmitError__('SPIRIT_SHADER', e)
|
||||
gl.deleteShader(s)
|
||||
return null
|
||||
}
|
||||
return s
|
||||
}
|
||||
var vs = c(gl.VERTEX_SHADER, vsSrc)
|
||||
var fs = c(gl.FRAGMENT_SHADER, fsSrc)
|
||||
if (!vs || !fs) return null
|
||||
var p = gl.createProgram()
|
||||
gl.attachShader(p, vs)
|
||||
gl.attachShader(p, fs)
|
||||
gl.linkProgram(p)
|
||||
gl.deleteShader(vs)
|
||||
gl.deleteShader(fs)
|
||||
if (!gl.getProgramParameter(p, gl.LINK_STATUS)) {
|
||||
var e2 = gl.getProgramInfoLog(p) || 'program link failed'
|
||||
window.__stemengineEmitError__('SPIRIT_PROGRAM', e2)
|
||||
gl.deleteProgram(p)
|
||||
return null
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
function mkTarget(gl, w, h) {
|
||||
var tex = gl.createTexture()
|
||||
gl.bindTexture(gl.TEXTURE_2D, tex)
|
||||
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, w, h, 0, gl.RGBA, gl.UNSIGNED_BYTE, null)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE)
|
||||
var fbo = gl.createFramebuffer()
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, fbo)
|
||||
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0)
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
|
||||
return { tex: tex, fbo: fbo, width: w, height: h }
|
||||
}
|
||||
|
||||
function SpiritSimulator(canvas, gl, width, height, quality) {
|
||||
this.canvas = canvas || document.querySelector('canvas')
|
||||
this.gl = gl || (this.canvas && (this.canvas.getContext('webgl2', { alpha: false, antialias: false }) || this.canvas.getContext('webgl', { alpha: false, antialias: false })))
|
||||
if (!this.gl) throw new Error('[StemEngine][Spirit] Unable to acquire WebGL context')
|
||||
|
||||
this.width = Math.max(1, Number(width) || this.canvas.clientWidth || this.canvas.width || 512)
|
||||
this.height = Math.max(1, Number(height) || this.canvas.clientHeight || this.canvas.height || 512)
|
||||
this.quality = String(quality || window.__STEMENGINE_MUON_QUALITY__ || 'balanced')
|
||||
this.paused = false
|
||||
this.running = false
|
||||
this.rafId = null
|
||||
|
||||
var gl = this.gl
|
||||
var vs = 'attribute vec2 aPos; varying vec2 vUv; void main(){ vUv=aPos*0.5+0.5; gl_Position=vec4(aPos,0.0,1.0);} '
|
||||
var updateFs = [
|
||||
'precision highp float;',
|
||||
'uniform sampler2D uPrev;',
|
||||
'uniform vec2 uRes;',
|
||||
'uniform float uTime;',
|
||||
'uniform float uBass;',
|
||||
'uniform float uTreble;',
|
||||
'uniform float uRms;',
|
||||
'uniform float uBeat;',
|
||||
'uniform float uFade;',
|
||||
'varying vec2 vUv;',
|
||||
'float hash(vec2 p){ return fract(sin(dot(p, vec2(12.9898,78.233))) * 43758.5453); }',
|
||||
'float fbm(vec2 p){',
|
||||
' float v=0.0; float a=0.5;',
|
||||
' for(int i=0;i<4;i++){',
|
||||
' v += (hash(floor(p)) * 2.0 - 1.0) * a;',
|
||||
' p = p*2.03 + 13.1;',
|
||||
' a *= 0.5;',
|
||||
' }',
|
||||
' return v*0.5 + 0.5;',
|
||||
'}',
|
||||
'void main(){',
|
||||
' vec2 uv=vUv;',
|
||||
' vec2 c=uv-0.5;',
|
||||
' float r=length(c);',
|
||||
' float a=atan(c.y,c.x);',
|
||||
' vec2 drift = vec2(cos(a + uTime*(0.3+uBass*1.8)), sin(a - uTime*(0.22+uTreble*1.4))) * (0.001 + 0.003*uBass);',
|
||||
' vec3 prev=texture2D(uPrev, clamp(uv-drift,0.0,1.0)).rgb * uFade;',
|
||||
' float n = fbm(uv*5.0 + vec2(uTime*0.12, -uTime*0.08));',
|
||||
' float halo = exp(-r*(3.0 + uTreble*3.0));',
|
||||
' float strand = smoothstep(0.0, 0.08 + uTreble*0.06, abs(sin(a*10.0 + n*9.0 + uTime*(1.8+uRms*4.0))) * (0.55-r));',
|
||||
' float pulse = uBeat * exp(-r*2.8) + uRms*0.08;',
|
||||
' vec3 cool = vec3(0.2 + uTreble*0.6, 0.35 + uRms*0.45, 0.8);',
|
||||
' vec3 warm = vec3(0.95, 0.45 + uBass*0.35, 0.2 + uRms*0.25);',
|
||||
' vec3 glow = mix(cool, warm, clamp(n + uBass*0.35, 0.0, 1.0)) * (halo*0.35 + strand*0.65 + pulse*0.5);',
|
||||
' gl_FragColor = vec4(clamp(prev + glow, 0.0, 1.0), 1.0);',
|
||||
'}'
|
||||
].join('\n')
|
||||
var blitFs = 'precision mediump float; uniform sampler2D uTex; varying vec2 vUv; void main(){ vec3 c=texture2D(uTex,vUv).rgb; gl_FragColor=vec4(c*c*0.15 + c,1.0);} '
|
||||
|
||||
this.updateProgram = compileProgram(gl, vs, updateFs)
|
||||
this.blitProgram = compileProgram(gl, vs, blitFs)
|
||||
|
||||
var quad = new Float32Array([-1,-1, 1,-1, -1,1, -1,1, 1,-1, 1,1])
|
||||
this.quad = gl.createBuffer()
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
|
||||
gl.bufferData(gl.ARRAY_BUFFER, quad, gl.STATIC_DRAW)
|
||||
|
||||
this.read = null
|
||||
this.write = null
|
||||
this.allocateTargets()
|
||||
this.start()
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.allocateTargets = function () {
|
||||
var gl = this.gl
|
||||
if (this.read) {
|
||||
gl.deleteTexture(this.read.tex)
|
||||
gl.deleteFramebuffer(this.read.fbo)
|
||||
}
|
||||
if (this.write) {
|
||||
gl.deleteTexture(this.write.tex)
|
||||
gl.deleteFramebuffer(this.write.fbo)
|
||||
}
|
||||
var scale = QUALITY_SCALE[this.quality] || QUALITY_SCALE.balanced
|
||||
var w = Math.max(64, Math.floor(this.width * scale))
|
||||
var h = Math.max(64, Math.floor(this.height * scale))
|
||||
this.read = mkTarget(gl, w, h)
|
||||
this.write = mkTarget(gl, w, h)
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.setQuality = function (q) {
|
||||
var next = String(q || 'balanced')
|
||||
if (next === this.quality) return
|
||||
this.quality = next
|
||||
this.allocateTargets()
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.setViewport = function (w, h) {
|
||||
var nw = Math.max(1, Number(w) || this.width)
|
||||
var nh = Math.max(1, Number(h) || this.height)
|
||||
if (nw === this.width && nh === this.height) return
|
||||
this.width = nw
|
||||
this.height = nh
|
||||
if (this.canvas) {
|
||||
this.canvas.width = nw
|
||||
this.canvas.height = nh
|
||||
}
|
||||
this.allocateTargets()
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.draw = function (program) {
|
||||
var gl = this.gl
|
||||
gl.useProgram(program)
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
|
||||
var loc = gl.getAttribLocation(program, 'aPos')
|
||||
gl.enableVertexAttribArray(loc)
|
||||
gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0)
|
||||
gl.drawArrays(gl.TRIANGLES, 0, 6)
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.step = function () {
|
||||
if (this.paused || !this.read || !this.write) return
|
||||
var gl = this.gl
|
||||
var t = performance.now() * 0.001
|
||||
var audio = window.__STEMENGINE_MUON_AUDIO__ || {}
|
||||
var bass = shaped(0.5 * getBand(audio, 'sub', audio.sub || 0) + 0.5 * getBand(audio, 'bass', audio.bass || 0), 0.64)
|
||||
var treble = shaped(0.55 * getBand(audio, 'highMid', audio.treble || 0) + 0.45 * getBand(audio, 'brilliance', audio.treble || 0), 0.66)
|
||||
var rms = shaped(clamp01(audio.rms || 0), 0.8)
|
||||
var beat = audio.beat ? 1 : 0
|
||||
var buildBoost = audio.isBuildUp ? 0.02 : 0
|
||||
var dropBoost = audio.isBeatDrop ? 0.02 : 0
|
||||
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, this.write.fbo)
|
||||
gl.viewport(0, 0, this.write.width, this.write.height)
|
||||
gl.useProgram(this.updateProgram)
|
||||
gl.activeTexture(gl.TEXTURE0)
|
||||
gl.bindTexture(gl.TEXTURE_2D, this.read.tex)
|
||||
gl.uniform1i(gl.getUniformLocation(this.updateProgram, 'uPrev'), 0)
|
||||
gl.uniform2f(gl.getUniformLocation(this.updateProgram, 'uRes'), this.write.width, this.write.height)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTime'), t)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBass'), bass)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTreble'), treble)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uRms'), rms)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBeat'), beat)
|
||||
gl.uniform1f(
|
||||
gl.getUniformLocation(this.updateProgram, 'uFade'),
|
||||
(this.quality === 'high' ? 0.965 : this.quality === 'low' ? 0.92 : 0.945) - buildBoost + dropBoost,
|
||||
)
|
||||
this.draw(this.updateProgram)
|
||||
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
|
||||
gl.viewport(0, 0, this.width, this.height)
|
||||
gl.useProgram(this.blitProgram)
|
||||
gl.activeTexture(gl.TEXTURE0)
|
||||
gl.bindTexture(gl.TEXTURE_2D, this.write.tex)
|
||||
gl.uniform1i(gl.getUniformLocation(this.blitProgram, 'uTex'), 0)
|
||||
this.draw(this.blitProgram)
|
||||
|
||||
var tmp = this.read
|
||||
this.read = this.write
|
||||
this.write = tmp
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.tick = function () {
|
||||
if (!this.running) return
|
||||
this.paused = Boolean(window.__STEMENGINE_MUON_PAUSED__)
|
||||
this.setQuality(window.__STEMENGINE_MUON_QUALITY__)
|
||||
var vp = window.__STEMENGINE_MUON_VIEWPORT__
|
||||
if (vp) this.setViewport(vp.width, vp.height)
|
||||
this.step()
|
||||
var self = this
|
||||
this.rafId = requestAnimationFrame(function () { self.tick() })
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.start = function () {
|
||||
if (this.running) return
|
||||
this.running = true
|
||||
this.tick()
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.stop = function () {
|
||||
this.running = false
|
||||
if (this.rafId != null) {
|
||||
cancelAnimationFrame(this.rafId)
|
||||
this.rafId = null
|
||||
}
|
||||
}
|
||||
|
||||
SpiritSimulator.prototype.dispose = function () {
|
||||
var gl = this.gl
|
||||
this.stop()
|
||||
if (this.quad) gl.deleteBuffer(this.quad)
|
||||
if (this.updateProgram) gl.deleteProgram(this.updateProgram)
|
||||
if (this.blitProgram) gl.deleteProgram(this.blitProgram)
|
||||
if (this.read) {
|
||||
gl.deleteTexture(this.read.tex)
|
||||
gl.deleteFramebuffer(this.read.fbo)
|
||||
}
|
||||
if (this.write) {
|
||||
gl.deleteTexture(this.write.tex)
|
||||
gl.deleteFramebuffer(this.write.fbo)
|
||||
}
|
||||
}
|
||||
|
||||
window.__StemEngineSpiritRenderGraph__ = SpiritSimulator
|
||||
console.log('[StemEngine][Spirit] Render graph module loaded')
|
||||
})()
|
||||
+96
@@ -0,0 +1,96 @@
|
||||
/* StemEngine runtime bootstrap injected into Muon build at image build-time. */
|
||||
|
||||
(function stemengineBootstrap() {
|
||||
const REQUIRED_EXTENSIONS = ['OES_texture_float_linear', 'EXT_color_buffer_float']
|
||||
|
||||
function installContextCaptureHook() {
|
||||
const originalGetContext = HTMLCanvasElement.prototype.getContext
|
||||
if (typeof originalGetContext !== 'function') return
|
||||
if (HTMLCanvasElement.prototype.__stemengineContextHookInstalled__) return
|
||||
|
||||
HTMLCanvasElement.prototype.getContext = function patchedGetContext(type, attrs) {
|
||||
const ctx = originalGetContext.call(this, type, attrs)
|
||||
if ((type === 'webgl2' || type === 'webgl') && ctx && this.isConnected) {
|
||||
if (!window.__STEMENGINE_MUON_GL_CTX__) {
|
||||
window.__STEMENGINE_MUON_GL_CTX__ = ctx
|
||||
}
|
||||
if (!window.__STEMENGINE_MUON_CANVAS__) {
|
||||
window.__STEMENGINE_MUON_CANVAS__ = this
|
||||
}
|
||||
}
|
||||
return ctx
|
||||
}
|
||||
|
||||
Object.defineProperty(HTMLCanvasElement.prototype, '__stemengineContextHookInstalled__', {
|
||||
value: true,
|
||||
configurable: true,
|
||||
writable: false,
|
||||
enumerable: false,
|
||||
})
|
||||
}
|
||||
|
||||
function checkBaseline() {
|
||||
const canvas = document.createElement('canvas')
|
||||
const gl = canvas.getContext('webgl2') || canvas.getContext('webgl')
|
||||
if (!gl) {
|
||||
console.warn('[StemEngine][Muon] WebGL unavailable; fallback mode should be used.')
|
||||
return {
|
||||
ok: false,
|
||||
missing: ['webgl'],
|
||||
}
|
||||
}
|
||||
|
||||
const missing = REQUIRED_EXTENSIONS.filter((name) => !gl.getExtension(name))
|
||||
if (missing.length > 0) {
|
||||
console.warn('[StemEngine][Muon] Missing required WebGL extensions:', missing.join(', '))
|
||||
return { ok: false, missing }
|
||||
}
|
||||
|
||||
return { ok: true, missing: [] }
|
||||
}
|
||||
|
||||
function installContextLifecycleHooks() {
|
||||
let retries = 0
|
||||
const maxRetries = 120
|
||||
|
||||
const attach = () => {
|
||||
const canvas = document.querySelector('canvas.webgl') || document.querySelector('canvas')
|
||||
if (!canvas) {
|
||||
retries += 1
|
||||
if (retries <= maxRetries) requestAnimationFrame(attach)
|
||||
return
|
||||
}
|
||||
|
||||
canvas.addEventListener('webglcontextlost', (event) => {
|
||||
event.preventDefault()
|
||||
console.warn('[StemEngine][Muon] webglcontextlost')
|
||||
window.__STEMENGINE_MUON_CONTEXT_LOST__ = true
|
||||
})
|
||||
|
||||
canvas.addEventListener('webglcontextrestored', () => {
|
||||
console.info('[StemEngine][Muon] webglcontextrestored')
|
||||
window.__STEMENGINE_MUON_CONTEXT_LOST__ = false
|
||||
})
|
||||
}
|
||||
|
||||
attach()
|
||||
}
|
||||
|
||||
if (typeof window.__STEMENGINE_MUON_STOP_FN__ !== 'function') {
|
||||
window.__STEMENGINE_MUON_STOP_FN__ = function stemengineNoopStop() {}
|
||||
}
|
||||
if (typeof window.__STEMENGINE_MUON_RESUME_FN__ !== 'function') {
|
||||
window.__STEMENGINE_MUON_RESUME_FN__ = function stemengineNoopResume() {}
|
||||
}
|
||||
|
||||
installContextCaptureHook()
|
||||
|
||||
const baseline = checkBaseline()
|
||||
window.__STEMENGINE_MUON_WEBGL_BASELINE__ = baseline
|
||||
|
||||
if (!baseline.ok) {
|
||||
document.documentElement.classList.add('stemengine-muon-degraded')
|
||||
}
|
||||
|
||||
installContextLifecycleHooks()
|
||||
})()
|
||||
+356
@@ -0,0 +1,356 @@
|
||||
/* StemEngine — style router + postMessage command listener.
|
||||
* Injected into the Muon build at image build-time via Dockerfile.
|
||||
*
|
||||
* Project policy: zero access control restrictions on this path. Origin validation
|
||||
* is intentionally a no-op. See docs/plans/2026-04-27_Muon_Default_Mode_Reactivity/
|
||||
* README.md section 8 for the rationale.
|
||||
*/
|
||||
;(function stemengineRouter() {
|
||||
function isAllowedOrigin(_origin) {
|
||||
return true
|
||||
}
|
||||
|
||||
function postToParent(message) {
|
||||
// Always emit with wildcard and rely on host-side origin checks.
|
||||
// This avoids stale single-origin target mismatches during mixed-origin dev workflows.
|
||||
parent.postMessage(message, '*')
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
function getStyleParam() {
|
||||
try {
|
||||
return new URLSearchParams(location.search).get('style') || 'default'
|
||||
} catch (_) {
|
||||
return 'default'
|
||||
}
|
||||
}
|
||||
|
||||
/** Clamp a value to [lo, hi]; coerce to number, NaN → 0. */
|
||||
function clamp(v, lo, hi) {
|
||||
var n = Number(v)
|
||||
if (!isFinite(n)) n = 0
|
||||
return Math.max(lo, Math.min(hi, n))
|
||||
}
|
||||
|
||||
var KNOWN_COMMANDS = {
|
||||
setStyle: true,
|
||||
setPreset: true,
|
||||
setQuality: true,
|
||||
setPaused: true,
|
||||
setViewport: true,
|
||||
pushAudioFeatures: true,
|
||||
loadAudio: true,
|
||||
transportPlay: true,
|
||||
transportPause: true,
|
||||
transportSeek: true,
|
||||
transportRate: true,
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Audio feature state — host-push model.
|
||||
// The host sends pushAudioFeatures at its RAF cadence; we rate-limit to
|
||||
// one application per our own RAF tick so we never block the render loop.
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
var rafPending = false
|
||||
var pendingFeatures = null
|
||||
|
||||
function captureFeatures(payload) {
|
||||
if (!payload || typeof payload !== 'object') return
|
||||
pendingFeatures = {
|
||||
bass: clamp(payload.bass, 0, 1),
|
||||
treble: clamp(payload.treble, 0, 1),
|
||||
rms: clamp(payload.rms, 0, 1),
|
||||
energy: clamp(payload.energy, 0, 1),
|
||||
beat: Boolean(payload.beat),
|
||||
drumIntensity: clamp(payload.drumIntensity, 0, 1),
|
||||
sub: clamp(payload.sub, 0, 1),
|
||||
beatPhase: clamp(payload.beatPhase, 0, 1),
|
||||
bpm: Number.isFinite(Number(payload.bpm)) ? Number(payload.bpm) : 0,
|
||||
beatConfidence: clamp(payload.beatConfidence, 0, 1),
|
||||
isBeatDrop: Boolean(payload.isBeatDrop),
|
||||
isBuildUp: Boolean(payload.isBuildUp),
|
||||
isSilence: Boolean(payload.isSilence),
|
||||
energyDelta: Number.isFinite(Number(payload.energyDelta)) ? Number(payload.energyDelta) : 0,
|
||||
spectralCentroid: clamp(payload.spectralCentroid, 0, 1),
|
||||
spectralFlatness: clamp(payload.spectralFlatness, 0, 1),
|
||||
frequencyBands: payload.frequencyBands || null,
|
||||
}
|
||||
}
|
||||
|
||||
function flushFeatures() {
|
||||
if (!pendingFeatures) return
|
||||
window.__STEMENGINE_MUON_AUDIO__ = pendingFeatures
|
||||
pendingFeatures = null
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Render graph lifecycle
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
var currentRenderGraph = null
|
||||
var lastStyleValue = null
|
||||
|
||||
function updateRenderGraph(style) {
|
||||
// Only instantiate when style actually changes or on first call.
|
||||
if (style === lastStyleValue) return
|
||||
lastStyleValue = style
|
||||
|
||||
// Dispose previous render graph (if any).
|
||||
if (currentRenderGraph && typeof currentRenderGraph.dispose === 'function') {
|
||||
try {
|
||||
currentRenderGraph.dispose()
|
||||
} catch (e) {
|
||||
console.warn('[StemEngine][Muon] Error disposing render graph:', e)
|
||||
}
|
||||
}
|
||||
currentRenderGraph = null
|
||||
|
||||
if (style === 'default') {
|
||||
if (typeof window.__STEMENGINE_MUON_RESUME_FN__ === 'function') {
|
||||
try {
|
||||
window.__STEMENGINE_MUON_RESUME_FN__()
|
||||
} catch (e) {
|
||||
console.warn('[StemEngine][Muon] Error resuming default renderer:', e)
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if (typeof window.__STEMENGINE_MUON_STOP_FN__ === 'function') {
|
||||
try {
|
||||
window.__STEMENGINE_MUON_STOP_FN__()
|
||||
} catch (e) {
|
||||
console.warn('[StemEngine][Muon] Error stopping default renderer:', e)
|
||||
}
|
||||
}
|
||||
|
||||
var viewport = window.__STEMENGINE_MUON_VIEWPORT__ || {}
|
||||
var width = Number(viewport.width) || undefined
|
||||
var height = Number(viewport.height) || undefined
|
||||
var quality = window.__STEMENGINE_MUON_QUALITY__ || 'balanced'
|
||||
|
||||
// Instantiate new render graph based on style.
|
||||
if (style === 'fluid' && typeof window.__StemEngineFluidRenderGraph__ === 'function') {
|
||||
try {
|
||||
currentRenderGraph = new window.__StemEngineFluidRenderGraph__(window.__STEMENGINE_MUON_GL_CTX__, width, height, quality)
|
||||
console.log('[StemEngine][Muon] Instantiated Fluid render graph')
|
||||
} catch (e) {
|
||||
console.error('[StemEngine][Muon] Error instantiating Fluid render graph:', e)
|
||||
window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e))
|
||||
}
|
||||
} else if (style === 'tendrils' && typeof window.__StemEngineTendrilsRenderGraph__ === 'function') {
|
||||
try {
|
||||
currentRenderGraph = new window.__StemEngineTendrilsRenderGraph__(window.__STEMENGINE_MUON_CANVAS__, window.__STEMENGINE_MUON_GL_CTX__, width, height, quality)
|
||||
console.log('[StemEngine][Muon] Instantiated Tendrils render graph')
|
||||
} catch (e) {
|
||||
console.error('[StemEngine][Muon] Error instantiating Tendrils render graph:', e)
|
||||
window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e))
|
||||
}
|
||||
} else if (style === 'spirit' && typeof window.__StemEngineSpiritRenderGraph__ === 'function') {
|
||||
try {
|
||||
currentRenderGraph = new window.__StemEngineSpiritRenderGraph__(window.__STEMENGINE_MUON_CANVAS__, window.__STEMENGINE_MUON_GL_CTX__, width, height, quality)
|
||||
console.log('[StemEngine][Muon] Instantiated Spirit render graph')
|
||||
} catch (e) {
|
||||
console.error('[StemEngine][Muon] Error instantiating Spirit render graph:', e)
|
||||
window.__stemengineEmitError__('RENDER_GRAPH_INIT', String(e))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Command dispatch
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
function handleCommand(type, payload) {
|
||||
var p = payload || {}
|
||||
switch (type) {
|
||||
case 'setStyle':
|
||||
window.__STEMENGINE_MUON_STYLE__ = String(p.style || 'default')
|
||||
updateRenderGraph(window.__STEMENGINE_MUON_STYLE__)
|
||||
break
|
||||
case 'setQuality':
|
||||
window.__STEMENGINE_MUON_QUALITY__ = String(p.quality || 'balanced')
|
||||
break
|
||||
case 'setPreset':
|
||||
window.__STEMENGINE_MUON_PRESET__ = String(p.preset || '')
|
||||
break
|
||||
case 'setPaused':
|
||||
window.__STEMENGINE_MUON_PAUSED__ = Boolean(p.paused)
|
||||
break
|
||||
case 'setViewport':
|
||||
var cssWidth = clamp(p.cssWidth != null ? p.cssWidth : p.width, 1, 7680)
|
||||
var cssHeight = clamp(p.cssHeight != null ? p.cssHeight : p.height, 1, 4320)
|
||||
var dprRaw = Number(p.dpr)
|
||||
var dpr = isFinite(dprRaw) && dprRaw > 0 ? clamp(dprRaw, 0.25, 8) : 1
|
||||
var pixelWidth = Math.max(1, Math.round(cssWidth * dpr))
|
||||
var pixelHeight = Math.max(1, Math.round(cssHeight * dpr))
|
||||
window.__STEMENGINE_MUON_VIEWPORT__ = {
|
||||
cssWidth: cssWidth,
|
||||
cssHeight: cssHeight,
|
||||
dpr: dpr,
|
||||
width: pixelWidth,
|
||||
height: pixelHeight,
|
||||
}
|
||||
break
|
||||
case 'pushAudioFeatures':
|
||||
captureFeatures(p)
|
||||
if (!rafPending) {
|
||||
rafPending = true
|
||||
requestAnimationFrame(function () {
|
||||
rafPending = false
|
||||
flushFeatures()
|
||||
})
|
||||
}
|
||||
break
|
||||
case 'loadAudio':
|
||||
if (!window.__StemEngineAudioEngine__ || typeof window.__StemEngineAudioEngine__.loadFromUrl !== 'function') break
|
||||
window.__StemEngineAudioEngine__.loadFromUrl(String(p.url || ''))
|
||||
.then(function (info) {
|
||||
if (window.__StemEngineFeatureExtractor__ && typeof window.__StemEngineFeatureExtractor__.start === 'function') {
|
||||
window.__StemEngineFeatureExtractor__.start()
|
||||
}
|
||||
postToParent({
|
||||
version: 1,
|
||||
id: 'loaded-' + Date.now(),
|
||||
type: 'loaded',
|
||||
payload: {
|
||||
duration: info && Number.isFinite(Number(info.duration)) ? Number(info.duration) : 0,
|
||||
},
|
||||
})
|
||||
})
|
||||
.catch(function (err) {
|
||||
window.__stemengineEmitError__('AUDIO_LOAD', String(err && err.message ? err.message : err))
|
||||
})
|
||||
break
|
||||
case 'transportPlay':
|
||||
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.play === 'function') {
|
||||
window.__StemEngineAudioEngine__.play({
|
||||
time: Number(p.time) || 0,
|
||||
rate: Number(p.rate) || 1,
|
||||
})
|
||||
}
|
||||
break
|
||||
case 'transportPause':
|
||||
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.pause === 'function') {
|
||||
window.__StemEngineAudioEngine__.pause()
|
||||
}
|
||||
break
|
||||
case 'transportSeek':
|
||||
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.seek === 'function') {
|
||||
var nextTime = Number(p.time) || 0
|
||||
window.__StemEngineAudioEngine__.seek(nextTime)
|
||||
|
||||
// Recovery path: if host is actively playing but a transportPlay command was
|
||||
// dropped/raced, seek updates keep arriving. Auto-resume once from seek.
|
||||
if (
|
||||
!window.__STEMENGINE_MUON_PAUSED__
|
||||
&& typeof window.__StemEngineAudioEngine__.isPlaying === 'function'
|
||||
&& !window.__StemEngineAudioEngine__.isPlaying()
|
||||
&& typeof window.__StemEngineAudioEngine__.play === 'function'
|
||||
) {
|
||||
var resumeRate = Number(p.rate)
|
||||
if (!Number.isFinite(resumeRate) || resumeRate <= 0) {
|
||||
resumeRate = typeof window.__StemEngineAudioEngine__.getRate === 'function'
|
||||
? Number(window.__StemEngineAudioEngine__.getRate()) || 1
|
||||
: 1
|
||||
}
|
||||
window.__StemEngineAudioEngine__.play({
|
||||
time: nextTime,
|
||||
rate: resumeRate,
|
||||
})
|
||||
}
|
||||
}
|
||||
break
|
||||
case 'transportRate':
|
||||
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.setRate === 'function') {
|
||||
window.__StemEngineAudioEngine__.setRate(Number(p.rate) || 1)
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// postMessage listener
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
window.addEventListener('message', function onStemEngineMessage(event) {
|
||||
// Origin guard — reject messages from unexpected origins.
|
||||
if (!isAllowedOrigin(event.origin)) return
|
||||
|
||||
var msg = event.data
|
||||
if (!msg || typeof msg !== 'object') return
|
||||
// Schema: { version: 1, type: string, payload: object }
|
||||
if (Number(msg.version) !== 1) return
|
||||
var type = String(msg.type || '')
|
||||
if (!KNOWN_COMMANDS[type]) return
|
||||
|
||||
handleCommand(type, msg.payload)
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Emit ready to host
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
function emitReady() {
|
||||
var baseline = window.__STEMENGINE_MUON_WEBGL_BASELINE__ || { ok: false, missing: [] }
|
||||
var payload = {
|
||||
capabilities: {
|
||||
webgl: Boolean(baseline.ok),
|
||||
missingExtensions: baseline.missing || [],
|
||||
audioMode: 'hybrid',
|
||||
},
|
||||
styles: ['fluid', 'tendrils', 'spirit', 'default'],
|
||||
}
|
||||
try {
|
||||
postToParent({ version: 1, id: 'ready-' + Date.now(), type: 'ready', payload: payload })
|
||||
} catch (e) {
|
||||
console.warn('[StemEngine][Muon] Could not emit ready:', e)
|
||||
}
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Emit error to host (exported for other modules to call)
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
window.__stemengineEmitError__ = function emitError(code, message) {
|
||||
try {
|
||||
postToParent({
|
||||
version: 1,
|
||||
id: 'error-' + Date.now(),
|
||||
type: 'error',
|
||||
payload: { code: String(code || 'UNKNOWN'), message: String(message || '') },
|
||||
})
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Bootstrap
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
// Initialize globals from query params so the render graph can read them
|
||||
// on its first frame even before any postMessage arrives.
|
||||
window.__STEMENGINE_MUON_STYLE__ = getStyleParam()
|
||||
window.__STEMENGINE_MUON_QUALITY__ = 'balanced'
|
||||
window.__STEMENGINE_MUON_PAUSED__ = false
|
||||
window.__STEMENGINE_MUON_AUDIO__ = null
|
||||
window.__STEMENGINE_MUON_VIEWPORT__ = null
|
||||
|
||||
function onReady() {
|
||||
if (window.__StemEngineAudioEngine__ && typeof window.__StemEngineAudioEngine__.initialize === 'function') {
|
||||
window.__StemEngineAudioEngine__.initialize()
|
||||
}
|
||||
emitReady()
|
||||
// Instantiate the initial render graph based on the style query param.
|
||||
updateRenderGraph(window.__STEMENGINE_MUON_STYLE__)
|
||||
}
|
||||
|
||||
if (document.readyState === 'loading') {
|
||||
document.addEventListener('DOMContentLoaded', onReady)
|
||||
} else {
|
||||
onReady()
|
||||
}
|
||||
})()
|
||||
+265
@@ -0,0 +1,265 @@
|
||||
/* StemEngine Tendrils Render Graph (Muon fork overlay)
|
||||
* Particle trail / feedback visualization with audio reactivity.
|
||||
*/
|
||||
|
||||
;(function stemengineTendrilsRenderGraph() {
|
||||
'use strict'
|
||||
|
||||
var QUALITY_SCALE = { low: 0.75, balanced: 1.0, high: 1.35 }
|
||||
|
||||
function clamp01(v) {
|
||||
var n = Number(v)
|
||||
if (!isFinite(n)) n = 0
|
||||
return Math.max(0, Math.min(1, n))
|
||||
}
|
||||
|
||||
function shaped(v, power) {
|
||||
return Math.pow(clamp01(v), power)
|
||||
}
|
||||
|
||||
function getBand(audio, key, fallback) {
|
||||
var bands = audio && audio.frequencyBands ? audio.frequencyBands : null
|
||||
var value = bands && isFinite(Number(bands[key])) ? Number(bands[key]) : fallback
|
||||
return clamp01(value)
|
||||
}
|
||||
|
||||
function createProgram(gl, vsSrc, fsSrc) {
|
||||
function compile(type, src) {
|
||||
var s = gl.createShader(type)
|
||||
gl.shaderSource(s, src)
|
||||
gl.compileShader(s)
|
||||
if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) {
|
||||
var err = gl.getShaderInfoLog(s) || 'shader compile failed'
|
||||
window.__stemengineEmitError__('TENDRILS_SHADER', err)
|
||||
gl.deleteShader(s)
|
||||
return null
|
||||
}
|
||||
return s
|
||||
}
|
||||
var vs = compile(gl.VERTEX_SHADER, vsSrc)
|
||||
var fs = compile(gl.FRAGMENT_SHADER, fsSrc)
|
||||
if (!vs || !fs) return null
|
||||
var p = gl.createProgram()
|
||||
gl.attachShader(p, vs)
|
||||
gl.attachShader(p, fs)
|
||||
gl.linkProgram(p)
|
||||
gl.deleteShader(vs)
|
||||
gl.deleteShader(fs)
|
||||
if (!gl.getProgramParameter(p, gl.LINK_STATUS)) {
|
||||
var err2 = gl.getProgramInfoLog(p) || 'program link failed'
|
||||
window.__stemengineEmitError__('TENDRILS_PROGRAM', err2)
|
||||
gl.deleteProgram(p)
|
||||
return null
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
function createTarget(gl, w, h) {
|
||||
var tex = gl.createTexture()
|
||||
gl.bindTexture(gl.TEXTURE_2D, tex)
|
||||
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, w, h, 0, gl.RGBA, gl.UNSIGNED_BYTE, null)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE)
|
||||
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE)
|
||||
|
||||
var fbo = gl.createFramebuffer()
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, fbo)
|
||||
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, tex, 0)
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
|
||||
return { tex: tex, fbo: fbo, width: w, height: h }
|
||||
}
|
||||
|
||||
function TendrilsSimulator(canvas, gl, width, height, quality) {
|
||||
this.canvas = canvas || document.querySelector('canvas')
|
||||
this.gl = gl || (this.canvas && (this.canvas.getContext('webgl2', { alpha: false, antialias: false }) || this.canvas.getContext('webgl', { alpha: false, antialias: false })))
|
||||
if (!this.gl) throw new Error('[StemEngine][Tendrils] Unable to acquire WebGL context')
|
||||
|
||||
this.width = Math.max(1, Number(width) || this.canvas.clientWidth || this.canvas.width || 512)
|
||||
this.height = Math.max(1, Number(height) || this.canvas.clientHeight || this.canvas.height || 512)
|
||||
this.quality = String(quality || window.__STEMENGINE_MUON_QUALITY__ || 'balanced')
|
||||
this.paused = false
|
||||
this.rafId = null
|
||||
this.running = false
|
||||
|
||||
var gl = this.gl
|
||||
var vs = 'attribute vec2 aPos; varying vec2 vUv; void main(){ vUv=aPos*0.5+0.5; gl_Position=vec4(aPos,0.0,1.0);} '
|
||||
var updateFs = [
|
||||
'precision highp float;',
|
||||
'uniform sampler2D uPrev;',
|
||||
'uniform vec2 uRes;',
|
||||
'uniform float uTime;',
|
||||
'uniform float uBass;',
|
||||
'uniform float uTreble;',
|
||||
'uniform float uRms;',
|
||||
'uniform float uBeat;',
|
||||
'uniform float uTrailDecay;',
|
||||
'varying vec2 vUv;',
|
||||
'float hash(vec2 p){ return fract(sin(dot(p, vec2(127.1,311.7))) * 43758.5453); }',
|
||||
'void main(){',
|
||||
' vec2 uv=vUv;',
|
||||
' vec2 c=uv-0.5;',
|
||||
' float r=max(length(c), 1e-4);',
|
||||
' float a=atan(c.y,c.x);',
|
||||
' float flow=sin(a*8.0 + uTime*(1.6+uBass*4.0)) + cos(r*45.0 + uTime*(1.1+uTreble*5.0));',
|
||||
' vec2 dir=normalize(vec2(cos(a+flow*0.45), sin(a+flow*0.45)));',
|
||||
' vec2 offset=dir*(0.0015 + 0.0035*uBass) + vec2(hash(uv+uTime)-0.5, hash(uv+uTime*1.7)-0.5)*(0.002*uTreble);',
|
||||
' vec3 prev=texture2D(uPrev, clamp(uv-offset, 0.0, 1.0)).rgb * uTrailDecay;',
|
||||
' float line=exp(-abs(sin((a*12.0 + r*120.0) + uTime*6.0*(1.0+uTreble)))*26.0);',
|
||||
' float beatBurst=uBeat * exp(-r*4.0);',
|
||||
' vec3 colorA=vec3(0.2+uBass*0.8, 0.2+uRms*0.7, 0.4+uTreble*0.6);',
|
||||
' vec3 colorB=vec3(0.9,0.6+uTreble*0.35,0.2+uBass*0.4);',
|
||||
' vec3 inject=mix(colorA, colorB, clamp(r*1.5, 0.0, 1.0))*(line*0.3 + beatBurst*0.8 + uRms*0.05);',
|
||||
' gl_FragColor=vec4(clamp(prev + inject, 0.0, 1.0), 1.0);',
|
||||
'}'
|
||||
].join('\n')
|
||||
var blitFs = 'precision mediump float; uniform sampler2D uTex; varying vec2 vUv; void main(){ gl_FragColor = texture2D(uTex, vUv); }'
|
||||
|
||||
this.updateProgram = createProgram(gl, vs, updateFs)
|
||||
this.blitProgram = createProgram(gl, vs, blitFs)
|
||||
|
||||
var quad = new Float32Array([-1,-1, 1,-1, -1,1, -1,1, 1,-1, 1,1])
|
||||
this.quad = gl.createBuffer()
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
|
||||
gl.bufferData(gl.ARRAY_BUFFER, quad, gl.STATIC_DRAW)
|
||||
|
||||
this.read = null
|
||||
this.write = null
|
||||
this.allocateTargets()
|
||||
this.start()
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.allocateTargets = function () {
|
||||
var gl = this.gl
|
||||
if (this.read) {
|
||||
gl.deleteTexture(this.read.tex)
|
||||
gl.deleteFramebuffer(this.read.fbo)
|
||||
}
|
||||
if (this.write) {
|
||||
gl.deleteTexture(this.write.tex)
|
||||
gl.deleteFramebuffer(this.write.fbo)
|
||||
}
|
||||
var scale = QUALITY_SCALE[this.quality] || QUALITY_SCALE.balanced
|
||||
var w = Math.max(64, Math.floor(this.width * scale))
|
||||
var h = Math.max(64, Math.floor(this.height * scale))
|
||||
this.read = createTarget(gl, w, h)
|
||||
this.write = createTarget(gl, w, h)
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.setQuality = function (q) {
|
||||
var next = String(q || 'balanced')
|
||||
if (next === this.quality) return
|
||||
this.quality = next
|
||||
this.allocateTargets()
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.setViewport = function (w, h) {
|
||||
var nw = Math.max(1, Number(w) || this.width)
|
||||
var nh = Math.max(1, Number(h) || this.height)
|
||||
if (nw === this.width && nh === this.height) return
|
||||
this.width = nw
|
||||
this.height = nh
|
||||
if (this.canvas) {
|
||||
this.canvas.width = nw
|
||||
this.canvas.height = nh
|
||||
}
|
||||
this.allocateTargets()
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.drawQuad = function (program) {
|
||||
var gl = this.gl
|
||||
gl.useProgram(program)
|
||||
gl.bindBuffer(gl.ARRAY_BUFFER, this.quad)
|
||||
var loc = gl.getAttribLocation(program, 'aPos')
|
||||
gl.enableVertexAttribArray(loc)
|
||||
gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0)
|
||||
gl.drawArrays(gl.TRIANGLES, 0, 6)
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.step = function () {
|
||||
if (this.paused || !this.read || !this.write) return
|
||||
var gl = this.gl
|
||||
var t = performance.now() * 0.001
|
||||
var audio = window.__STEMENGINE_MUON_AUDIO__ || {}
|
||||
var bass = shaped(0.6 * getBand(audio, 'bass', audio.bass || 0) + 0.4 * getBand(audio, 'sub', audio.sub || 0), 0.62)
|
||||
var treble = shaped(0.55 * getBand(audio, 'presence', audio.treble || 0) + 0.45 * getBand(audio, 'brilliance', audio.treble || 0), 0.66)
|
||||
var rms = shaped(clamp01(audio.rms || 0), 0.8)
|
||||
var beat = audio.beat ? 1 : 0
|
||||
var buildBoost = audio.isBuildUp ? 0.12 : 0
|
||||
var dropBoost = audio.isBeatDrop ? 0.2 : 0
|
||||
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, this.write.fbo)
|
||||
gl.viewport(0, 0, this.write.width, this.write.height)
|
||||
gl.useProgram(this.updateProgram)
|
||||
gl.activeTexture(gl.TEXTURE0)
|
||||
gl.bindTexture(gl.TEXTURE_2D, this.read.tex)
|
||||
gl.uniform1i(gl.getUniformLocation(this.updateProgram, 'uPrev'), 0)
|
||||
gl.uniform2f(gl.getUniformLocation(this.updateProgram, 'uRes'), this.write.width, this.write.height)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTime'), t)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBass'), bass)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uTreble'), treble)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uRms'), rms)
|
||||
gl.uniform1f(gl.getUniformLocation(this.updateProgram, 'uBeat'), beat)
|
||||
gl.uniform1f(
|
||||
gl.getUniformLocation(this.updateProgram, 'uTrailDecay'),
|
||||
(0.90 + (this.quality === 'high' ? 0.08 : this.quality === 'low' ? 0.04 : 0.06)) - buildBoost + dropBoost,
|
||||
)
|
||||
this.drawQuad(this.updateProgram)
|
||||
|
||||
gl.bindFramebuffer(gl.FRAMEBUFFER, null)
|
||||
gl.viewport(0, 0, this.width, this.height)
|
||||
gl.useProgram(this.blitProgram)
|
||||
gl.activeTexture(gl.TEXTURE0)
|
||||
gl.bindTexture(gl.TEXTURE_2D, this.write.tex)
|
||||
gl.uniform1i(gl.getUniformLocation(this.blitProgram, 'uTex'), 0)
|
||||
this.drawQuad(this.blitProgram)
|
||||
|
||||
var tmp = this.read
|
||||
this.read = this.write
|
||||
this.write = tmp
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.tick = function () {
|
||||
if (!this.running) return
|
||||
this.paused = Boolean(window.__STEMENGINE_MUON_PAUSED__)
|
||||
this.setQuality(window.__STEMENGINE_MUON_QUALITY__)
|
||||
var vp = window.__STEMENGINE_MUON_VIEWPORT__
|
||||
if (vp) this.setViewport(vp.width, vp.height)
|
||||
this.step()
|
||||
var self = this
|
||||
this.rafId = requestAnimationFrame(function () { self.tick() })
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.start = function () {
|
||||
if (this.running) return
|
||||
this.running = true
|
||||
this.tick()
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.stop = function () {
|
||||
this.running = false
|
||||
if (this.rafId != null) {
|
||||
cancelAnimationFrame(this.rafId)
|
||||
this.rafId = null
|
||||
}
|
||||
}
|
||||
|
||||
TendrilsSimulator.prototype.dispose = function () {
|
||||
var gl = this.gl
|
||||
this.stop()
|
||||
if (this.quad) gl.deleteBuffer(this.quad)
|
||||
if (this.updateProgram) gl.deleteProgram(this.updateProgram)
|
||||
if (this.blitProgram) gl.deleteProgram(this.blitProgram)
|
||||
if (this.read) {
|
||||
gl.deleteTexture(this.read.tex)
|
||||
gl.deleteFramebuffer(this.read.fbo)
|
||||
}
|
||||
if (this.write) {
|
||||
gl.deleteTexture(this.write.tex)
|
||||
gl.deleteFramebuffer(this.write.fbo)
|
||||
}
|
||||
}
|
||||
|
||||
window.__StemEngineTendrilsRenderGraph__ = TendrilsSimulator
|
||||
console.log('[StemEngine][Tendrils] Render graph module loaded')
|
||||
})()
|
||||
@@ -0,0 +1,346 @@
|
||||
# Local Iframe Visualizer Architecture Plan
|
||||
|
||||
Date: 2026-04-28
|
||||
Status: Draft implementation plan
|
||||
Owner: StemEngine frontend / visualizer pipeline
|
||||
|
||||
## Problem Statement
|
||||
The current attempt to approximate `fluid`, `tendrils`, and `spirit` inside the existing Muon iframe path did not deliver a reliable result.
|
||||
|
||||
Observed issues:
|
||||
- `default` mode is the only mode currently behaving close to the intended repeat-play iframe flow.
|
||||
- `fluid` shows reproducible replay instability and framebuffer warnings.
|
||||
- `tendrils` and `spirit` may look visually in-family, but are still bound to Muon-specific lifecycle assumptions that make them hard to validate and stabilize.
|
||||
|
||||
## Goal
|
||||
Use the same host-to-iframe transport model for every visualizer style, but move non-default styles onto a local iframe runtime that behaves like Muon from the host’s point of view.
|
||||
|
||||
Target outcome:
|
||||
- the host speaks one iframe transport contract,
|
||||
- each style runs in an iframe-managed renderer lifecycle,
|
||||
- replay / pause / seek / rate / resize are validated the same way for all styles,
|
||||
- non-default styles are no longer dependent on Muon internals that were designed for the default renderer.
|
||||
|
||||
## Decision Summary
|
||||
Recommended architecture:
|
||||
- Keep the existing Muon iframe provider for `default` for now.
|
||||
- Build a local iframe visualizer runtime for `fluid`, `tendrils`, and `spirit`.
|
||||
- Standardize the transport contract so the host treats both as iframe visualizer providers.
|
||||
|
||||
Preferred shape:
|
||||
- One shared local iframe shell with pluggable style engines.
|
||||
|
||||
Fallback shape:
|
||||
- One local iframe app per style.
|
||||
|
||||
Recommendation:
|
||||
- Use one shared local iframe shell unless style-specific boot constraints prove otherwise.
|
||||
|
||||
## Why This Direction
|
||||
Benefits:
|
||||
- separates transport concerns from render math,
|
||||
- removes style replay behavior from Muon private/default renderer assumptions,
|
||||
- reduces duplicate host logic,
|
||||
- gives one lifecycle model for all local styles,
|
||||
- makes validation and third-party review clearer.
|
||||
|
||||
## Scope
|
||||
In scope:
|
||||
- iframe transport contract standardization,
|
||||
- host visualizer provider abstraction,
|
||||
- local iframe shell runtime,
|
||||
- local renderer plugin API,
|
||||
- migration of `fluid`, `tendrils`, and `spirit` to local iframe shell,
|
||||
- validation matrix across Track Detail, Visualizer Lab, and other exposed surfaces.
|
||||
|
||||
Out of scope for first pass:
|
||||
- replacing default Muon renderer,
|
||||
- perfect fidelity to third-party reference demos,
|
||||
- broad preset-authoring UX,
|
||||
- camera/mic/advanced interaction parity with original external demos.
|
||||
|
||||
## Current File Touch Points
|
||||
### Host-side frontend
|
||||
Likely primary touch points:
|
||||
- `frontend/src/hooks/useMuonIframeTransport.ts`
|
||||
- `frontend/src/components/VisualizerPreviewWidget.tsx`
|
||||
- `frontend/src/components/LyricsLivePreview.tsx`
|
||||
- `frontend/src/pages/VisualizerLabPage.tsx`
|
||||
- `frontend/src/pages/TrackDetail.tsx`
|
||||
|
||||
Likely new host-side additions:
|
||||
- `frontend/src/visualizers/providers/*`
|
||||
- `frontend/src/hooks/useIframeVisualizerTransport.ts`
|
||||
- `frontend/src/types/iframeVisualizer.ts`
|
||||
|
||||
### Existing Muon iframe runtime
|
||||
Reference implementation / source of reusable logic:
|
||||
- `muon-iframe/stemengine-router.js`
|
||||
- `muon-iframe/stemengine-bootstrap.js`
|
||||
- `muon-iframe/stemengine-audio-engine.js`
|
||||
- `muon-iframe/stemengine-feature-extractor.js`
|
||||
|
||||
### Existing style renderers to migrate or adapt
|
||||
- `muon-iframe/fluid-render-graph.js`
|
||||
- `muon-iframe/tendrils-render-graph.js`
|
||||
- `muon-iframe/spirit-render-graph.js`
|
||||
|
||||
### Likely new local iframe app area
|
||||
Suggested location:
|
||||
- `frontend/local-visualizer-iframe/` or
|
||||
- `local-visualizer-iframe/`
|
||||
|
||||
Recommendation:
|
||||
- keep it separate from `muon-iframe/` so it is not build-time injected into Muon.
|
||||
|
||||
## Target Architecture
|
||||
### 1. Host visualizer provider layer
|
||||
The host should target a provider abstraction, not Muon directly.
|
||||
|
||||
Proposed provider model:
|
||||
- provider id
|
||||
- iframe URL
|
||||
- supported styles
|
||||
- capabilities
|
||||
- command protocol version
|
||||
- fallback behavior
|
||||
|
||||
Example providers:
|
||||
- `muon-default`
|
||||
- `local-visualizer`
|
||||
|
||||
### 2. Shared iframe transport contract
|
||||
Commands from host to iframe:
|
||||
- `init`
|
||||
- `setStyle`
|
||||
- `setPreset`
|
||||
- `setQuality`
|
||||
- `setPaused`
|
||||
- `setViewport`
|
||||
- `loadAudio`
|
||||
- `transportPlay`
|
||||
- `transportPause`
|
||||
- `transportSeek`
|
||||
- `transportRate`
|
||||
- `pushAudioFeatures`
|
||||
- `dispose`
|
||||
|
||||
Events from iframe to host:
|
||||
- `ready`
|
||||
- `loaded`
|
||||
- `capabilities`
|
||||
- `state`
|
||||
- `warning`
|
||||
- `error`
|
||||
|
||||
Required rules:
|
||||
- host owns transport intent and source audio selection,
|
||||
- iframe owns rendering lifecycle,
|
||||
- seek while playing must not leave renderer visually paused,
|
||||
- replay after pause must be deterministic,
|
||||
- viewport updates must not require reload.
|
||||
|
||||
### 3. Local iframe shell
|
||||
Responsibilities:
|
||||
- boot and emit `ready`,
|
||||
- set up audio engine,
|
||||
- set up feature extractor,
|
||||
- receive commands from host,
|
||||
- instantiate exactly one renderer at a time,
|
||||
- dispose renderer cleanly on style switch or teardown,
|
||||
- expose warnings/errors back to host.
|
||||
|
||||
Shell should be style-agnostic.
|
||||
|
||||
### 4. Renderer plugin API
|
||||
Each style renderer should conform to the same API.
|
||||
|
||||
Proposed renderer interface:
|
||||
- `init(context)`
|
||||
- `setViewport(viewport)`
|
||||
- `setQuality(quality)`
|
||||
- `setPreset(preset)`
|
||||
- `setPaused(paused)`
|
||||
- `pushFeatures(features)`
|
||||
- `tick(time)`
|
||||
- `dispose()`
|
||||
|
||||
Renderer should not own:
|
||||
- transport protocol,
|
||||
- source audio loading,
|
||||
- play/pause/seek intent,
|
||||
- message routing.
|
||||
|
||||
### 5. Migration model
|
||||
Near-term routing:
|
||||
- `default` -> existing Muon iframe provider
|
||||
- `fluid` -> local iframe shell
|
||||
- `tendrils` -> local iframe shell
|
||||
- `spirit` -> local iframe shell
|
||||
|
||||
Future option:
|
||||
- move `default` under the same provider system later if beneficial.
|
||||
|
||||
## Implementation Phases
|
||||
## Phase 1: Freeze the contract
|
||||
Deliverables:
|
||||
- one transport spec document,
|
||||
- one provider abstraction design,
|
||||
- explicit replay/seek/resize semantics.
|
||||
|
||||
Tasks:
|
||||
- document current useful Muon messages,
|
||||
- remove Muon-specific assumptions from the conceptual transport model,
|
||||
- define one versioned protocol for all iframe visualizers.
|
||||
|
||||
Exit criteria:
|
||||
- host and iframe responsibilities are explicitly defined,
|
||||
- no new style-specific transport behavior is added ad hoc.
|
||||
|
||||
## Phase 2: Introduce provider abstraction on host
|
||||
Deliverables:
|
||||
- provider registry,
|
||||
- host hook abstracted away from Muon naming,
|
||||
- widgets/pages able to select a provider.
|
||||
|
||||
Tasks:
|
||||
- refactor `useMuonIframeTransport.ts` into provider-neutral transport hook,
|
||||
- add provider config for default Muon and local iframe runtime,
|
||||
- update consumers in preview surfaces.
|
||||
|
||||
Exit criteria:
|
||||
- Track Detail and Visualizer Lab can load an iframe provider without hard-coding Muon-only assumptions.
|
||||
|
||||
## Phase 3: Build local iframe shell
|
||||
Deliverables:
|
||||
- local iframe app booting independently,
|
||||
- postMessage router,
|
||||
- audio engine and feature extractor wiring,
|
||||
- renderer lifecycle manager.
|
||||
|
||||
Tasks:
|
||||
- clone only the reusable runtime concepts from current Muon integration,
|
||||
- avoid build-time injection into third-party Muon source,
|
||||
- add explicit renderer registration and disposal.
|
||||
|
||||
Exit criteria:
|
||||
- shell loads, emits ready, can accept audio load/play/pause/seek/viewport commands even with a placeholder renderer.
|
||||
|
||||
## Phase 4: Migrate `fluid`
|
||||
Why first:
|
||||
- it is the closest structural match to its intended reference,
|
||||
- it is currently the least reliable mode.
|
||||
|
||||
Tasks:
|
||||
- adapt current fluid renderer to the new renderer API,
|
||||
- enforce init-time capability checks,
|
||||
- surface explicit degrade/failure states when FBO support is incomplete,
|
||||
- validate replay and resize in the new shell.
|
||||
|
||||
Exit criteria:
|
||||
- `fluid` passes two-cycle replay test in Track Detail,
|
||||
- no silent frozen-canvas failure path,
|
||||
- FBO failure produces explicit host-visible warning/error.
|
||||
|
||||
## Phase 5: Migrate `tendrils` and `spirit`
|
||||
Tasks:
|
||||
- adapt both renderers to shared shell lifecycle,
|
||||
- remove transport ownership assumptions from render graphs,
|
||||
- validate style switch, replay, resize, and quality transitions.
|
||||
|
||||
Exit criteria:
|
||||
- both renderers pass same iframe lifecycle tests as `fluid`,
|
||||
- no per-style transport branching on host.
|
||||
|
||||
## Phase 6: Cross-surface validation and rollout
|
||||
Tasks:
|
||||
- validate Track Detail,
|
||||
- validate Visualizer Lab with a loaded song path,
|
||||
- validate any project/embed surface that exposes iframe preview,
|
||||
- update docs and reviewer package with post-migration results.
|
||||
|
||||
Exit criteria:
|
||||
- the same style behaves consistently across supported surfaces,
|
||||
- unsupported surfaces are explicitly documented.
|
||||
|
||||
## Validation Matrix
|
||||
Each migrated style must pass:
|
||||
- iframe boot / ready
|
||||
- audio load
|
||||
- play cycle 1
|
||||
- pause
|
||||
- replay cycle 2
|
||||
- seek while playing
|
||||
- seek while paused
|
||||
- rate change
|
||||
- viewport resize
|
||||
- quality change
|
||||
- style switch away and back
|
||||
- teardown / remount
|
||||
|
||||
Required test surfaces:
|
||||
- Track Detail
|
||||
- Visualizer Lab
|
||||
- any active project/embed preview surface
|
||||
|
||||
## Key Risks
|
||||
1. Host abstraction drift
|
||||
Risk:
|
||||
- partial refactor leaves old Muon-only codepaths mixed with provider abstraction.
|
||||
Mitigation:
|
||||
- centralize provider selection in one registry and one transport hook.
|
||||
|
||||
2. Fluid hardware variability
|
||||
Risk:
|
||||
- local fluid renderer still fails on certain WebGL / extension combinations.
|
||||
Mitigation:
|
||||
- explicit capability detection and degrade behavior, not silent failure.
|
||||
|
||||
3. Renderer lifecycle leaks
|
||||
Risk:
|
||||
- RAF loops, textures, or framebuffers survive style switches.
|
||||
Mitigation:
|
||||
- strict renderer API and disposable lifecycle manager in shell.
|
||||
|
||||
4. Visualizer Lab remains blocked
|
||||
Risk:
|
||||
- shell exists but lab still cannot load audio cleanly.
|
||||
Mitigation:
|
||||
- define one supported sample-input path early in Phase 3.
|
||||
|
||||
5. Fidelity pressure before lifecycle stability
|
||||
Risk:
|
||||
- development time is spent tuning visuals before replay and resize are correct.
|
||||
Mitigation:
|
||||
- block fidelity work until lifecycle acceptance criteria pass.
|
||||
|
||||
## Deliverables Checklist
|
||||
- transport spec doc
|
||||
- provider registry
|
||||
- provider-neutral iframe transport hook
|
||||
- local iframe shell app
|
||||
- renderer plugin API
|
||||
- migrated `fluid`
|
||||
- migrated `tendrils`
|
||||
- migrated `spirit`
|
||||
- updated validation docs
|
||||
- updated review package
|
||||
|
||||
## Recommended Execution Order
|
||||
1. Freeze transport spec
|
||||
2. Refactor host to provider model
|
||||
3. Build local iframe shell with placeholder renderer
|
||||
4. Migrate `fluid`
|
||||
5. Stabilize replay / resize / seek
|
||||
6. Migrate `tendrils`
|
||||
7. Migrate `spirit`
|
||||
8. Re-run cross-surface validation
|
||||
9. Update external review package
|
||||
|
||||
## Success Definition
|
||||
This plan succeeds when:
|
||||
- non-default styles no longer rely on Muon’s default-renderer internals,
|
||||
- host uses one iframe transport model across providers,
|
||||
- replay behavior is deterministic for all supported styles,
|
||||
- failures are surfaced explicitly instead of appearing as frozen visuals,
|
||||
- reviewers can evaluate styles through a consistent iframe lifecycle rather than special-case paths.
|
||||
Reference in new issue
Block a user