Skip to content

65-visuals: fluid shader backgrounds (M2) + smoke/ink fluid layer and particle flow fields (M6) - #438

Merged
AlexZ005 merged 8 commits into
feat/65from
feat/65-visuals
Oct 23, 2026
Merged

AlexZ005 merged 8 commits into
feat/65from
feat/65-visuals

Conversation

@AlexZ005

Copy link
Copy Markdown
Collaborator

Roadmap 65 M2 fluid shader backgrounds + M6 2D VFX (lane 65-visuals), into feat/65. Packs half: theprototype-app/packs#20.

What it adds

  • Background shader domain. A third domain beside surface and post. It has a Background output terminal and 10 new nodes: Pointer, Scroll and Aspect-correct UV (background only); Fractal noise (fBm), Domain warp, Mesh gradient, Metaballs, Smooth min, Film grain and Rotate (any pixel stage). Bayer and Resolution now also work in backgrounds. compileBackgroundGraphToIR builds the whole fragment shader. preludeDeps puts a shared GLSL helper in once, and TP_OCTAVE_CAP is the budget knob.
  • Background layer (backgroundLayer.js + pure backgroundLayerCore.js):
    • It draws full screen behind the scene, or on a picked object's surface. In XR it becomes a sphere re-centred per eye, because a full-screen quad would be head-locked.
    • The document is shaderGraphs['background'], so it rides the existing shader-graph sync, undo and the four save paths. No new wire message.
    • Pointer and scroll inputs are local to each viewer.
    • The tier drops to low (octave cap 2) in XR, on a phone, or once the governor gives up the scene look.
    • UI: Configure Scene ▸ Background, and a Background tab in the shader editor.
  • Preset loader (backgroundPresets.js): reads PACKS_BASE/backgrounds/index.json (packs#20). Applying a preset COPIES its graph into the scene, so peers never need the CDN. One built-in preset (Soft gradient) means an offline build is never empty.
  • Smoke / ink fluid layer (fluidSim.js, fluidLayer.js, pure fluidLayerCore.js):
    • The solver is our own stable-fluids code: curl, vorticity confinement, divergence, decayed-pressure Jacobi, gradient subtract, advection, on half-float ping-pong targets. WebGL-Fluid-Simulation (MIT) was a reference for which passes to run; no code was copied.
    • On screens it is a scene-look kind Smoke / ink (Stylize). Its Effect steps the solver in update() and composites the dye: smoke adds light, ink mixes pigment in.
    • In XR it is a world panel stirred by the controller rays.
    • Budget tiers: 128/512/20 on desktop; 96/384/14 at dpr ≤ 0.72; 64/256/8 on a phone or in XR; off in XR once particles are capped.
    • It stops stepping 500 ms after nothing composites it, and frees its targets after 10 s.
    • The params are scene data; the smoke itself is each viewer's own.
  • Particle flow fields (particleShader/Runtime/Presets, ParticleMotionRows): the curl of an analytic potential (divergence-free) integrated in 4 steps, or 2 in VR or under the particle cap. New config flow/flowScale/flowSpeed, presets Flow field and Swirl ribbon, and three Inspector rows.
  • Latent bug fixed: shaderGraphsSnapshot({pruneMissing}) dropped every non-object key, so autosave and session saves silently lost every post graph (post:<id>). Pruning now applies to surface-domain keys only, and buildSelectionPayload keeps surface documents only, so a prefab carries no look or background.

Shared-file touch points (small, additive)

  • Scene.svelte: 2 imports, 2 useTask lines, 1 <T.Group name="background-root">.
  • Inspector.svelte: the section.
  • debugHooks.js: 3 rows.
  • shaderGraph.js: registerGraphDomain, the background domain in normalize, the prune fix.
  • sessions.js: the selection filter.
  • postGraphs.js: the domain.
  • ShaderEditor.svelte: the tab.
  • particleRuntime.js: particleEntries reports flow.
  • shaderCatalog/shaderCompile, particleShader/Presets, ParticleMotionRows.svelte.

Proof

  • FPS rows (tests/e2e/visuals-perf.cjs under e2e-slot --exclusive, Chromium uncapped: --disable-frame-rate-limit --disable-gpu-vsync; Deck GPU, RADV):

    Profile Row p50 ms p95 ms calls tris
    desktop empty scene (baseline) 2.9 6.4 1* –
    desktop each of the 6 backgrounds 1.3–2.8 8.6–9.6 1* –
    desktop smoke layer (128/512/20) 3.0 8.8 1* –
    desktop smoke + metaballs 3.2 9.5 1* –
    desktop 8 Flow field emitters / same with flow 0 0.8 / 0.6 10.8 / 9.7 1* –
    quest (IWER, low) empty scene (baseline) 1.4 2.3 36 18 080
    quest background sphere, cap 2 (5 presets) 1.9–2.2 3.1–4.3 38 22 496
    quest smoke panel (64/256/8) 1.8 2.9 38 18 084
    quest smoke panel + metaballs sphere 1.9 2.8 40 22 500

    * On the desktop the composer's last fullscreen pass is what renderer.info reports (the 37-fx note). Uncapped desktop frames are CPU-bound, so the rows sit inside the baseline's own spread. The Quest rows render directly: the sphere costs +0.5–0.8 ms and +4.4k triangles, and the panel +0.4 ms. Everything is far inside the Quest budget (≤ 150 calls, ≤ 300k tris, ≤ 13.9 ms). These are the Deck's GPU running the headset workload, a proxy rather than a Quest measurement.

  • Unit: tests/unit/backgroundGraph, backgroundLayer, fluidLayer: 41 pass.

  • Node: background-presets (every packs file loads, compiles and passes glslangValidator at both tiers), shader-compile, shader-node-docs (the docs-table half SKIPS here: no docs checkout). The 19 fluid shaders also pass glslangValidator.

  • New e2e:

    Suite Checks What it covers
    background-layer 58/58 Every preset paints the view; the held-clock control; the pointer moves it; plane mode with delete/undo; saves keep the background; the override; the section UI
    fluid-layer 26/26 Steps and composites; the pointer puts dye where it went; ink; the medium tier; idles when the look is given up or switched off; the particle flow field and its capped step count
    visuals-xr-iwer 19/19 Quest 3 on IWER: the background sphere at low / octave cap 2; the smoke panel on the 64/256/8 grid; the controller moves the pointer and stirs the panel; particles capped in XR means no smoke; exiting restores both
    visuals-two-peer 17/17 The background reaches the other peer and a late joiner; edits and removal follow; the smoke entry is shared while the dye and the background pointer stay local
  • Regression battery: shader-graph 67, shader-persist 19, shader-post-domain 48, shader-editor 66, shader-window 40, shader-sync 20 (two-peer), post-backends 21, particles 17, particles-fx 20, save-as-formats 39, session-scene-data 27. All pass.

  • A/B against v1.41.0, same remote browser:

    • The empty scene and a scene with a box differ by 174 px (0.017 %), the peer-id band.
    • Configure Scene differs only by the new Background section.
  • Checks: npm run build passes; check-ratchet 299/39 = baseline; check-tokens, check-storage and check-zindex are clean.

For the integrator

  • After packs#20 merges: retag packs format-1 and purge jsDelivr for backgrounds/*, then check each file's content. Until then the section offers only the built-in preset, with a Retry line.
  • Docs: the 11 node rows and two pages are in the lane's docs-fragment.md. shader-node-docs will compare the node table once the docs checkout is present.
  • M7 hook for 65-export: backgroundLayer.setBackgroundScroll(progress|null) drives the Scroll input from a scroll-hero page.
  • Owed on a real device: feel and frame time on a Quest. The IWER rows are the Deck's GPU running the headset workload at the low tier.

🤖 Generated with Claude Code

AlexZ005 and others added 8 commits October 23, 2026 15:37
… node, pattern nodes, compiler

Background output + Pointer / Scroll / Aspect-correct UV (background stage only), and
fBm, Domain warp, Mesh gradient, Metaballs, Smooth min, Film grain, Rotate for every stage
that has a pixel. compileBackgroundGraphToIR emits a whole fragment shader for the layer
material; preludeDeps hoists a shared helper once; TP_OCTAVE_CAP is the budget knob.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nter + scroll, budget tier, packs preset loader

shaderGraphs['background'] (domain 'background') carries the layer settings, so wire, undo,
saves and late join are inherited. Configure Scene ▸ Background + a Background tab in the
shader editor. Fixes shaderGraphsSnapshot pruning post/background documents as orphans
(a selection still carries none). e2e background-layer written, first run pending.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…Flow field / Swirl ribbon presets

Divergence-free curl of a trigonometric potential with analytic derivatives, integrated in
uFlowSteps midpoint steps (4; 2 on a headset or when the governor caps particles). Inspector
Particles: Flow field / Swirl size / Field speed. e2e owed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d yet)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…reens, a world panel in VR

- fluidLayer.js: one shared FluidSim, stirred by the local pointer (or the VR controller
  rays) and an optional automatic stirrer. Look kind `fluid` (Stylize): its Effect steps
  the solver in update() and composites the dye (smoke adds light, ink mixes pigment).
  In a headset (no composer) the same entry becomes a panel ~3 m ahead under
  background-root, stepped from the Scene useTask with xr.enabled off for the passes.
  Nothing composited for 500 ms = no steps; 10 s = targets freed.
- fluidLayerCore.js (pure): params normalize, the budget tiers (128/512/20 desktop,
  96/384/14 at dpr <= 0.72, 64/256/8 phone/XR, none in XR once particles are capped),
  palettes, the auto stirrer, pointer -> splat.
- fluidSim.js: fields declared for the checker. backgroundLayer: backgroundRootGroup().
- particleEntries() reports flow / flowSteps. debugHooks row fluidLayer.
- tests: unit fluidLayer (15), e2e fluid-layer (desktop) + visuals-xr-iwer (Quest profile).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… frames) and visuals-two-peer (scene data shared, pointer/smoke local)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- background-layer 2.10 measured the built-in Soft gradient, which is still by design (it moves
  only with the pointer); it now applies Noise flow first, with a usesClock premise
  (backgroundLayerDebug reports usesClock).
- fluid-layer 2.4: left-to-right strokes throw their own smoke rightwards (measured left 93.9 /
  right 144.8, the screenshot shows it); the strokes are vertical on the left now.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…) and 4.1 expects the preset 2.10 leaves (58/58)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AlexZ005
AlexZ005 merged commit dacb7f9 into feat/65 Oct 23, 2026
9 checks passed
@AlexZ005
AlexZ005 deleted the feat/65-visuals branch October 23, 2026 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant