Muon iframe — Runtime Baseline Notes
This directory contains the self-hosted Muon iframe packaging context used by StemEngine.
Build pipeline and release process (Phase 0-A)
Build sources:
- Docker compose service build args (deployment stack):
MUON_GIT_REFin /home/translator/docker/stemengine.yml
- CI workflow (this repository):
Expected release flow:
- Pin a tested upstream Muon ref (
MUON_GIT_REF=<commit-or-tag>). - Run
muon-iframe-build-publishworkflow manually (or push amuon-iframe-v*tag). - Publish image tags (
latest, ref,sha) to GHCR. - Update deployment env with pinned
MUON_GIT_REFif needed. - Rebuild/redeploy
stemengine-muonservice.
Rollback:
- Revert
MUON_GIT_REFto previous known-good ref and redeploystemengine-muon. - Keep previous image tags available to accelerate rollback.
WebGL capability baseline (Phase 0)
Required extensions for Fluid/Tendrils/Spirit fork work:
OES_texture_float_linearEXT_color_buffer_float
Recommended additional capability checks in fork startup:
OES_texture_float- non-zero
MAX_VERTEX_TEXTURE_IMAGE_UNITS
Fallback policy (when required extensions are absent)
If the required extensions are unavailable:
- disable advanced fluid/tendrils/spirit renderer paths,
- fall back to existing/default Muon rendering mode,
- preserve playback and host command handling,
- expose non-fatal status to host UI (
degradedstate in later phases).
webglcontextlost / webglcontextrestored strategy
Planned behavior in the Muon fork:
- register listeners on the visualizer canvas for
webglcontextlostandwebglcontextrestored, - call
event.preventDefault()on context loss to enable restoration, - stop animation loop and suppress expensive updates while context is lost,
- rebuild render targets/programs on restore,
- rehydrate current style/preset/quality from host-provided state,
- if restore fails repeatedly, fall back to default mode and notify host.
Origin and embedding notes
- Host postMessage command origin must be StemEngine-owned.
- Third-party demo origins are not valid command targets.
- iframe embedding should use:
sandbox="allow-scripts allow-same-origin"referrerpolicy="no-referrer"
postMessage command contract (Phase 1)
The iframe accepts versioned commands via postMessage from STEMENGINE_ALLOWED_ORIGIN only.
Messages from any other origin are silently ignored.
Message envelope
{ "version": 1, "id": "<unique-string>", "type": "<command>", "payload": { ... } }
Commands (host → iframe)
| Command | Payload | Notes |
|---|---|---|
setStyle |
{ style: "fluid" | "tendrils" | "spirit" | "default" } |
Switch render graph |
setQuality |
{ quality: "low" | "balanced" | "high" } |
Adjusts iteration counts/bloom |
setPreset |
{ preset: string } |
Named preset (future use) |
setPaused |
{ paused: boolean } |
Pause/resume render loop |
setViewport |
{ width: number, height: number } |
Update canvas size; values clamped to [1, 7680/4320] |
pushAudioFeatures |
{ bass, treble, rms, energy, beat, ...optionalV2 } |
Audio data; rate-limited to one-per-RAF |
Optional v2 fields currently accepted (all additive, v1-compatible):
drumIntensity(0..1)sub(0..1)beatPhase(0..1)bpm(number)beatConfidence(0..1)isBeatDrop(boolean)isBuildUp(boolean)isSilence(boolean)energyDelta(number)spectralCentroid(normalized 0..1)spectralFlatness(0..1)
Events (iframe → host)
| Event type | Payload | Notes |
|---|---|---|
ready |
{ capabilities: { webgl, missingExtensions }, styles: string[] } |
Emitted on DOMContentLoaded |
error |
{ code: string, message: string } |
Unrecoverable failures only |
Audio feature bridge policy (Phase 1.3)
Decision: host push.
StemEngine already runs a full audio analysis pipeline (MuonFeatureExtractor) on the main
track and optional stem tracks. Rather than duplicating an AnalyserNode inside the iframe,
the host pushes normalized pushAudioFeatures commands each animation frame.
Rationale:
- No duplicated audio graph or additional CPU overhead in the iframe.
- StemEngine can blend stem-channel features before sending (e.g., drive visuals from the drums stem only).
- The iframe stays a pure rendering surface with no audio I/O permissions required.
Rate-limiting:
- The host sends
pushAudioFeaturesno more often than its own RAF cadence. - The iframe further rate-limits application to one-per-RAF via its own
requestAnimationFramegate so the render loop is never blocked by message bursts. - Pushes where all feature deltas are below a threshold may be skipped in a future optimization.
STEMENGINE_ALLOWED_ORIGIN build arg:
- Set in compose
stemengine.ymlviaSTEMENGINE_ALLOWED_ORIGINenv var (defaults tohttp://localhost:5173for development). - Must match the StemEngine frontend origin exactly (scheme + host + port).