Files
stemengine_old/muon-iframe
2026-04-23 18:08:53 +09:00
..
2026-04-21 16:24:11 +09:00
2026-04-23 14:59:57 +09:00
2026-04-21 16:24:11 +09:00
2026-04-21 16:24:11 +09:00
2026-04-21 16:24:11 +09:00
2026-04-23 14:59:57 +09:00
2026-04-21 16:24:11 +09:00
2026-04-21 16:24:11 +09:00
2026-04-23 18:08:53 +09:00
2026-04-21 16:24:11 +09:00
2026-04-21 16:24:11 +09:00

Muon iframe — Runtime Baseline Notes

This directory contains the self-hosted Muon iframe packaging context used by StemEngine.

Build pipeline and release process (Phase 0-A)

Build sources:

Expected release flow:

  1. Pin a tested upstream Muon ref (MUON_GIT_REF=<commit-or-tag>).
  2. Run muon-iframe-build-publish workflow manually (or push a muon-iframe-v* tag).
  3. Publish image tags (latest, ref, sha) to GHCR.
  4. Update deployment env with pinned MUON_GIT_REF if needed.
  5. Rebuild/redeploy stemengine-muon service.

Rollback:

  • Revert MUON_GIT_REF to previous known-good ref and redeploy stemengine-muon.
  • Keep previous image tags available to accelerate rollback.

WebGL capability baseline (Phase 0)

Required extensions for Fluid/Tendrils/Spirit fork work:

  • OES_texture_float_linear
  • EXT_color_buffer_float

Recommended additional capability checks in fork startup:

  • OES_texture_float
  • non-zero MAX_VERTEX_TEXTURE_IMAGE_UNITS

Fallback policy (when required extensions are absent)

If the required extensions are unavailable:

  1. disable advanced fluid/tendrils/spirit renderer paths,
  2. fall back to existing/default Muon rendering mode,
  3. preserve playback and host command handling,
  4. expose non-fatal status to host UI (degraded state in later phases).

webglcontextlost / webglcontextrestored strategy

Planned behavior in the Muon fork:

  • register listeners on the visualizer canvas for webglcontextlost and webglcontextrestored,
  • call event.preventDefault() on context loss to enable restoration,
  • stop animation loop and suppress expensive updates while context is lost,
  • rebuild render targets/programs on restore,
  • rehydrate current style/preset/quality from host-provided state,
  • if restore fails repeatedly, fall back to default mode and notify host.

Origin and embedding notes

  • Host postMessage command origin must be StemEngine-owned.
  • Third-party demo origins are not valid command targets.
  • iframe embedding should use:
    • sandbox="allow-scripts allow-same-origin"
    • referrerpolicy="no-referrer"

postMessage command contract (Phase 1)

The iframe accepts versioned commands via postMessage from STEMENGINE_ALLOWED_ORIGIN only. Messages from any other origin are silently ignored.

Message envelope

{ "version": 1, "id": "<unique-string>", "type": "<command>", "payload": { ... } }

Commands (host → iframe)

Command Payload Notes
setStyle { style: "fluid" | "tendrils" | "spirit" | "default" } Switch render graph
setQuality { quality: "low" | "balanced" | "high" } Adjusts iteration counts/bloom
setPreset { preset: string } Named preset (future use)
setPaused { paused: boolean } Pause/resume render loop
setViewport { width: number, height: number } Update canvas size; values clamped to [1, 7680/4320]
pushAudioFeatures { bass, treble, rms, energy, beat, ...optionalV2 } Audio data; rate-limited to one-per-RAF

Optional v2 fields currently accepted (all additive, v1-compatible):

  • drumIntensity (0..1)
  • sub (0..1)
  • beatPhase (0..1)
  • bpm (number)
  • beatConfidence (0..1)
  • isBeatDrop (boolean)
  • isBuildUp (boolean)
  • isSilence (boolean)
  • energyDelta (number)
  • spectralCentroid (normalized 0..1)
  • spectralFlatness (0..1)

Events (iframe → host)

Event type Payload Notes
ready { capabilities: { webgl, missingExtensions }, styles: string[] } Emitted on DOMContentLoaded
error { code: string, message: string } Unrecoverable failures only

Audio feature bridge policy (Phase 1.3)

Decision: host push.

StemEngine already runs a full audio analysis pipeline (MuonFeatureExtractor) on the main track and optional stem tracks. Rather than duplicating an AnalyserNode inside the iframe, the host pushes normalized pushAudioFeatures commands each animation frame.

Rationale:

  • No duplicated audio graph or additional CPU overhead in the iframe.
  • StemEngine can blend stem-channel features before sending (e.g., drive visuals from the drums stem only).
  • The iframe stays a pure rendering surface with no audio I/O permissions required.

Rate-limiting:

  • The host sends pushAudioFeatures no more often than its own RAF cadence.
  • The iframe further rate-limits application to one-per-RAF via its own requestAnimationFrame gate so the render loop is never blocked by message bursts.
  • Pushes where all feature deltas are below a threshold may be skipped in a future optimization.

STEMENGINE_ALLOWED_ORIGIN build arg:

  • Set in compose stemengine.yml via STEMENGINE_ALLOWED_ORIGIN env var (defaults to http://localhost:5173 for development).
  • Must match the StemEngine frontend origin exactly (scheme + host + port).