# Phase 2: Fluid Render Graph โ€” Shader Implementation Guide **Status**: In Progress (Infrastructure โœ…, Shaders ๐Ÿšง) **Date Started**: 2026-04-17 **Last Updated**: 2026-04-17 ## Overview The `fluid-render-graph.js` overlay provides FBO infrastructure, quality tier configuration, and audio mapping logic. This guide documents the **10 GLSL shader programs** that must be implemented to complete Phase 2 rendering. ## Architecture Review ### Execution Order Each frame, the `FluidSimulator.step(audioFeatures)` method executes render passes in this sequence: 1. **Splat** โ€” Inject audio-driven forces into velocity field 2. **Curl** โ€” Compute vorticity from velocity 3. **Divergence** โ€” Compute divergence of velocity field 4. **Pressure Jacobi** (loop N times per quality tier) 5. **Gradient Subtract** โ€” Remove pressure gradient from velocity 6. **Advect Velocity** โ€” Self-advection of velocity field 7. **Advect Dye** โ€” Carry colors downstream 8. **(Optional) Bloom Downsample** โ€” 2-tap downsampling (balanced/high only) 9. **(Optional) Bloom Composite** โ€” Upsample + accumulate (balanced/high only) 10. **Composite** โ€” Final render to canvas ### FBO Layout ``` Ping-Pong Pairs (swapped each frame): - velocity[0], velocity[1] [vec4: vx, vy, divx, divy] - dye[0], dye[1] [vec4: r, g, b, a] - divergence[0], divergence[1] [float: div] - pressure[0], pressure[1] [float: p] - curl[0], curl[1] [float: ฯ‰_z (2D vorticity)] Bloom Chain (balanced/high only): - bloomDown1, bloomDown2 [quarter-res downsamples] - bloomUp1 [upsampled for composite] ``` ## Shader Programs ### 1. Splat Shader **Purpose**: Inject audio-driven forces into velocity field. **Input Audio Mapping**: - `bass` โ†’ Force magnitude & radius (low frequencies = diffuse spread) - `beat` โ†’ Impulse splat (percussion pulse = concentrated force) - `treble` โ†’ Turbulence injection (high frequencies = chaotic rotation) **Uniforms**: ```glsl uniform sampler2D u_velocity; // Ping-pong read (velocity[i-1]) uniform vec2 u_splatPos; // Canvas center or random offset uniform float u_splatRadius; // Gaussian kernel radius [0.5..2.0] * canvas_scale uniform float u_splatForce; // Force magnitude [0..10] uniform bool u_splatIsBeat; // Beat-driven impulse? (concentrated) uniform float u_turbulenceAmplitude; // Treble-driven vorticity [0..1] ``` **Algorithm**: ```glsl vec2 pos = gl_FragCoord.xy; float dist = length(pos - u_splatPos); float kernel = exp(-dist * dist / (u_splatRadius * u_splatRadius)); vec2 splatVel; if (u_splatIsBeat) { // Concentrated impulse for percussion splatVel = normalize(u_splatPos - pos) * u_splatForce * kernel; } else { // Diffuse spread for bass splatVel = (u_splatPos - pos) * u_splatForce * kernel * 0.1; } // Rotate splat velocity by treble-driven angle float angle = u_turbulenceAmplitude * sin(length(pos) * 0.01) * 3.14159; splatVel = vec2( splatVel.x * cos(angle) - splatVel.y * sin(angle), splatVel.x * sin(angle) + splatVel.y * cos(angle) ); vec2 vel = texture2D(u_velocity, uv).xy; gl_FragColor = vec4(vel + splatVel, 0.0, 1.0); ``` **Call Site** (in `FluidSimulator.splat()`): ```javascript const audioFeatures = window.__STEMENGINE_MUON_AUDIO__ const bass = audioFeatures.bass // [0..1] const beat = audioFeatures.beat // boolean const treble = audioFeatures.treble // [0..1] const splatRadius = 50 + bass * 100 // Spread with bass: [50..150] px const splatForce = 10 + bass * 20 // Force with bass: [10..30] const turbulence = treble * 1.5 // Amplitude: [0..1.5] // Render splat pass with these uniforms ``` ### 2. Curl Shader **Purpose**: Compute 2D vorticity (rotation) from velocity field. **Formula**: `ฯ‰_z = โˆ‚v_y/โˆ‚x - โˆ‚v_x/โˆ‚y` (central differences) **Uniforms**: ```glsl uniform sampler2D u_velocity; uniform vec2 u_invSize; // 1.0 / vec2(texW, texH) ``` **Algorithm**: ```glsl float n = texture2D(u_velocity, uv + vec2(0.0, u_invSize.y)).y; float s = texture2D(u_velocity, uv - vec2(0.0, u_invSize.y)).y; float e = texture2D(u_velocity, uv + vec2(u_invSize.x, 0.0)).x; float w = texture2D(u_velocity, uv - vec2(u_invSize.x, 0.0)).x; float curl = (e - w) - (n - s); gl_FragColor = vec4(curl, 0.0, 0.0, 1.0); ``` ### 3. Divergence Shader **Purpose**: Compute divergence of velocity field (โˆ‡ยทv). **Formula**: `div = โˆ‚v_x/โˆ‚x + โˆ‚v_y/โˆ‚y` (central differences) **Uniforms**: ```glsl uniform sampler2D u_velocity; uniform vec2 u_invSize; ``` **Algorithm**: ```glsl float n = texture2D(u_velocity, uv + vec2(0.0, u_invSize.y)).y; float s = texture2D(u_velocity, uv - vec2(0.0, u_invSize.y)).y; float e = texture2D(u_velocity, uv + vec2(u_invSize.x, 0.0)).x; float w = texture2D(u_velocity, uv - vec2(u_invSize.x, 0.0)).x; float div = 0.5 * ((e - w) + (n - s)); gl_FragColor = vec4(div, 0.0, 0.0, 1.0); ``` ### 4. Pressure Jacobi Shader **Purpose**: Iterative pressure solver using Jacobi method. **Converges to**: โˆ‡ยฒp = -div(v) **Iterations per tier**: Low=10, Balanced=20, High=40 **Uniforms**: ```glsl uniform sampler2D u_pressure; // Ping-pong read uniform sampler2D u_divergence; uniform float u_alpha; // -1.0 / 4.0 uniform float u_rBeta; // 0.25 (1/4) uniform vec2 u_invSize; ``` **Algorithm**: ```glsl float n = texture2D(u_pressure, uv + vec2(0.0, u_invSize.y)).x; float s = texture2D(u_pressure, uv - vec2(0.0, u_invSize.y)).x; float e = texture2D(u_pressure, uv + vec2(u_invSize.x, 0.0)).x; float w = texture2D(u_pressure, uv - vec2(u_invSize.x, 0.0)).x; float div = texture2D(u_divergence, uv).x; float p = (n + s + e + w + u_alpha * div) * u_rBeta; gl_FragColor = vec4(p, 0.0, 0.0, 1.0); ``` ### 5. Gradient Subtract Shader **Purpose**: Subtract pressure gradient from velocity (enforces incompressibility). **Formula**: `v_new = v - โˆ‡p` (central difference gradient) **Uniforms**: ```glsl uniform sampler2D u_velocity; uniform sampler2D u_pressure; uniform vec2 u_invSize; uniform float u_gradientScale; // 1.0 (or scaled for stability) ``` **Algorithm**: ```glsl float n = texture2D(u_pressure, uv + vec2(0.0, u_invSize.y)).x; float s = texture2D(u_pressure, uv - vec2(0.0, u_invSize.y)).x; float e = texture2D(u_pressure, uv + vec2(u_invSize.x, 0.0)).x; float w = texture2D(u_pressure, uv - vec2(u_invSize.x, 0.0)).x; vec2 grad = u_gradientScale * vec2(e - w, n - s); vec2 vel = texture2D(u_velocity, uv).xy; gl_FragColor = vec4(vel - grad, 0.0, 1.0); ``` ### 6. Advect Velocity Shader **Purpose**: Self-advect velocity field along its own streamlines. **Formula**: `v(t+dt) = v(t + dt*โˆ‡x(t))` (semi-Lagrangian backtracking) **Uniforms**: ```glsl uniform sampler2D u_velocity; uniform vec2 u_invSize; uniform float u_dt; // Time step [0.016 / 1000] โ‰ˆ 0.00001..0.01 uniform float u_decay; // [0.99..0.995] for damping ``` **Algorithm**: ```glsl vec2 vel = texture2D(u_velocity, uv).xy; vec2 backPos = uv - u_dt * vel * u_invSize; // Clamp to valid texture range [0, 1] backPos = clamp(backPos, u_invSize, 1.0 - u_invSize); vec2 advectedVel = texture2D(u_velocity, backPos).xy * u_decay; gl_FragColor = vec4(advectedVel, 0.0, 1.0); ``` ### 7. Advect Dye Shader **Purpose**: Carry color/dye downstream along velocity streamlines. **Maps audio features to dye color**: - `bass` โ†’ Red channel (low frequencies = warm) - `treble` โ†’ Blue channel (high frequencies = cool) - `rms` โ†’ Green channel (total energy = brightness) **Uniforms**: ```glsl uniform sampler2D u_dye; uniform sampler2D u_velocity; uniform vec2 u_invSize; uniform float u_dt; uniform float u_decay; // [0.98..0.99] for dye persistence uniform vec3 u_audioColor; // (r, g, b) from bass/treble/rms ``` **Algorithm**: ```glsl vec2 vel = texture2D(u_velocity, uv).xy; vec2 backPos = uv - u_dt * vel * u_invSize; backPos = clamp(backPos, u_invSize, 1.0 - u_invSize); vec4 dye = texture2D(u_dye, backPos) * u_decay; // Inject audio-driven color into this fragment float mixFactor = 0.15; // Injection strength dye.rgb += u_audioColor * mixFactor; dye.rgb = clamp(dye.rgb, 0.0, 1.0); gl_FragColor = dye; ``` **Call Site** (in `FluidSimulator.advectDye()`): ```javascript const audio = window.__STEMENGINE_MUON_AUDIO__ const audioColor = [ audio.bass, // Red channel audio.rms, // Green channel audio.treble, // Blue channel ] // Pass audioColor as u_audioColor uniform ``` ### 8. Bloom Downsample Shader **Purpose**: 2-tap downsampling for bloom chain. **Only rendered on balanced/high quality tiers**. **Uniforms**: ```glsl uniform sampler2D u_source; // Full-res dye texture uniform vec2 u_invSize; ``` **Algorithm**: ```glsl vec2 uv0 = uv - u_invSize * 0.5; vec2 uv1 = uv + u_invSize * 0.5; vec4 s0 = texture2D(u_source, vec2(uv0.x, uv0.y)); vec4 s1 = texture2D(u_source, vec2(uv1.x, uv0.y)); vec4 s2 = texture2D(u_source, vec2(uv0.x, uv1.y)); vec4 s3 = texture2D(u_source, vec2(uv1.x, uv1.y)); gl_FragColor = (s0 + s1 + s2 + s3) * 0.25; ``` ### 9. Bloom Composite Shader **Purpose**: Upsample and composite bloom chain back into full-res dye. **Creates soft glow/halo effect from bright regions**. **Uniforms**: ```glsl uniform sampler2D u_fullRes; // Full-res dye uniform sampler2D u_bloomUp; // Upsampled bloom (from downโ†’up) uniform float u_bloomIntensity; // [0..2], maps from rms audio ``` **Algorithm**: ```glsl vec4 full = texture2D(u_fullRes, uv); vec4 bloom = texture2D(u_bloomUp, uv); // Additive composite with intensity control gl_FragColor = full + bloom * u_bloomIntensity; gl_FragColor.rgb = clamp(gl_FragColor.rgb, 0.0, 1.0); ``` **Call Site** (in `FluidSimulator.bloomComposite()`): ```javascript const audio = window.__STEMENGINE_MUON_AUDIO__ const bloomIntensity = 0.5 + audio.rms * 1.5 // [0.5..2.0] // Pass as u_bloomIntensity uniform ``` ### 10. Composite Shader **Purpose**: Final render to canvas. **Outputs**: Velocity curl as hue, magnitude as value (HSV colorspace for artistic effect). **Uniforms**: ```glsl uniform sampler2D u_dye; // Advected color field uniform sampler2D u_curl; // Vorticity field uniform sampler2D u_velocity; // Velocity field uniform bool u_showVelocity; // Debug mode flag (default: false) uniform bool u_showCurl; // Debug mode flag (default: false) ``` **Algorithm (Production)**: ```glsl // Normal mode: show dye vec4 col = texture2D(u_dye, uv); gl_FragColor = vec4(col.rgb, 1.0); ``` **Algorithm (Debug)**: ```glsl // Show curl (vorticity) as grayscale float c = texture2D(u_curl, uv).x; float col = abs(c) * 10.0; // Amplify for visibility gl_FragColor = vec4(vec3(col), 1.0); ``` ## Implementation Checklist - [ ] Splat shader compiles, uniforms bind, output visible - [ ] Curl shader computes correctly (validation: compare against CPU reference) - [ ] Divergence shader computes correctly - [ ] Pressure Jacobi shader converges (test: divergence should decrease after N iterations) - [ ] Gradient Subtract shader enforces incompressibility - [ ] Advect Velocity shader prevents fluid explosion (decay working) - [ ] Advect Dye shader shows audio-reactive color injection - [ ] Bloom Downsample shader outputs quarter-res texture - [ ] Bloom Composite shader produces visible glow on balanced/high tiers - [ ] Composite shader renders final image to canvas - [ ] All 10 shaders linked into one WebGL program or individual programs - [ ] FBOs allocated, ping-pong swapping works - [ ] Quality tier iteration counts applied (low=10, balanced=20, high=40) - [ ] Audio features (bass, beat, treble, rms) correctly injected into splat/advect - [ ] Live performance: 60 FPS on desktop, graceful degration on mobile - [ ] Style router instantiates FluidSimulator when style='fluid' ## Integration Points ### In `fluid-render-graph.js`: Each method stub in `FluidSimulator.prototype` must: 1. Create or reuse a WebGL program with the appropriate shader pair (vertex + fragment) 2. Bind input textures to texture units 3. Set uniforms from `this.width`, `this.height`, `this.quality`, audio features 4. Bind output FBO 5. Render fullscreen quad (use existing VAO from constructor) 6. Ensure ping-pong swapping happens (read from [i-1], write to [i]) ### In `stemengine-router.js`: `handleCommand('setStyle', {style: 'fluid'})` โ†’ `updateRenderGraph()` โ†’ `new window.__StemEngineFluidRenderGraph__()` - Constructor detects WebGL context from canvas - Allocates FBOs and compiles all 10 shader programs - Hooks into animation loop via `requestAnimationFrame` ### In Dockerfile: ```dockerfile COPY fluid-render-graph.js /tmp/ RUN cp /tmp/fluid-render-graph.js /src/app/src/fluid-render-graph.js RUN grep -q "fluid-render-graph" /src/app/src/index.js || sed -i '1i import "./fluid-render-graph";' /src/app/src/index.js ``` ## Testing Strategy ### Unit Tests (manual): 1. **Shader compilation**: Verify each shader compiles without errors 2. **Uniform binding**: Check all uniforms receive correct values 3. **Texture unit management**: Verify no texture conflicts (max 8 units, carefully manage) 4. **FBO attachments**: Test each FBO binds correctly and renders to its texture ### Integration Tests: 1. **Full loop**: Run step() for 60 frames, verify no WebGL errors 2. **Audio injection**: Verify bass/beat/treble/rms values propagate to splat 3. **Bloom effectiveness**: Compare balanced/high to low tier visually (bloom should appear on balanced+) 4. **Quality tier switching**: Jacobi iteration counts should change on quality change ### Performance Tests: 1. **Low tier**: 256x256 resolution, 10 pressure iterations โ†’ expect 144+ FPS 2. **Balanced tier**: 512x512 resolution, 20 pressure iterations + bloom โ†’ expect 60 FPS 3. **High tier**: 1024x1024 resolution, 40 pressure iterations + bloom โ†’ expect 30+ FPS ## Debugging Tips - Use `u_showVelocity` and `u_showCurl` flags in composite shader to visualize intermediate fields - Check WebGL error log after each render pass: `gl.getError()` - Verify texture coordinates are in [0, 1] range (invSize calculations often source of bugs) - Use small splat radius during development (easier to see what's happening) - Rasterize shader output to canvas temporarily during debugging to inspect intermediate results