A Flock Clip is a native, three-dimensional particle swarm whose behavior and appearance are defined by an executable node graph. The graph runs on the GPU, draws into the shared 3D scene (real depth against models, planes and splats), and is exported through the same scene path as preview.
The ordinary Properties panel shows promoted graph parameters; Open Nodes
shows the very same graph as the green Flock group on the unified clip canvas. Both surfaces
edit one definition (clip.flock) and one set of keyframes.
Flock follows the shared Nodes interaction model: typed cables, previews, nested groups and the compact property inspector. Reset uses each parameter's default; animated numeric controls edit the existing keyframe owner. The link icon exposes or removes a control in Properties. Connections can also be edited through the inspector, including repeated behavior inputs. Previews read existing particle samples and evaluated parameters without starting another simulation; scene images are labeled as the shared Flock scene.
The swarm is the Flocking effect (Effects → 3D & Particles → Flocking) and
works on any clip: video, image, text or an empty host clip. The clip's own
image stays visible underneath; the swarm is drawn as an extra 3D layer directly
above it with the clip's transform. Disabling or bypassing the effect hides the
swarm, removing it deletes the graph and its flock.node.* keyframes. One swarm
per clip: adding Flocking again selects the existing effect.
- In the Effects tab the Flocking entry shows the same controls as the Flock tab (preset, Open Nodes, exposed controls, Time & Quality, diagnostics).
- On the node canvas the effect node is the green Flock group with bypass.
- Add Layer → Generators → Flock: … still creates an empty host clip that carries the effect. Older projects' flock clips are migrated on load: they gain the Flocking effect entry; nothing else changes.
- Swarm nodes are filed under the shared node categories — Particles (emitter, spawn merge, simulation, flock rules, cruise, cluster, compose), Forces & Physics (attractor, vortex, turbulence, drag, wind, path, follow path, obstacle, boundary), Logic & Switch (selections), Values & Time, Math, Color & Mask (palette), Inputs (audio) and Output & Render (points, instances, links, trails, curves, glyphs, vectors, output). The workspace right-click menu lists them with every other node and adds a Flocking effect on first use; same-named nodes of other graphs name their graph, e.g. Drag (Flocking).
- AI tools:
addEffectwitheffectType: "flocking"adds a swarm to an existing clip; every flock tool accepts any clip with a Flocking effect.
See Node Workspace for graph editing and shared controls.
- Right-click an empty area of a video track → Add Layer → Generators → Flock: …. Every preset creates one selected 10 s clip on that (unlocked) track.
- The Flock tab in Properties shows the preset picker, Open Nodes, the exposed controls grouped as Population / Behavior / Guidance / Appearance / Lines, Time & Quality (step rate, warm-up, loop, precompute, cache, host capability, live runtime status) and graph diagnostics.
- Open Nodes activates the Node Workspace and shows the clip's Flock group alongside its other nodes. Add nodes from the right-click menu, connect typed ports, bypass, duplicate (Ctrl+D), group (Ctrl+G), rename, expose any parameter to the Properties panel, save group or whole-graph presets, and apply presets to other clips.
The Preset row of the Flocking card offers the effect's five presets first (Free Swarm,
Krill Cloud, Follow Path, Technical Network, Violet Filaments; FLOCKING_CARD_PRESET_IDS) and the
other swarms under More swarms. Picking one replaces the graph in one undo step; when the clip
has flock.node.* keyframes, the card asks first because they are removed. The row always names
the graph's presetId, so it follows tools, the Nodes tab and undo. A flock preset replaces the
whole graph, so it is not an EffectPreset of the effect definition. Pictures and descriptions:
Flocking.
| Preset | Content |
|---|---|
| Free Swarm | Boids + turbulence inside a soft sphere sized to the default frame, soft points |
| Krill Cloud | Aligned krill instances in drifting clusters, faint neighbor links |
| Follow Path | School following a figure-eight that fits the frame, with tapered trails |
| Technical Network | Points, neighbor links and square trail-head markers inside a reflecting box |
| Violet Filaments | 20 000 clustered particles with a positional violet palette, long curved filaments, wire-cube heads and dot tails |
| Vortex (more) | Swirling shell, speed-colored points, velocity strokes |
| Shrimp Pullback (more) | One large hero animal inside a 24 000-animal structured swarm with links, trails and ring markers |
| Terracotta / Lilac Sculpture (more) | A settling APIC sheet with pigment in a lit gallery box |
Saved clips store their whole graph, so retuned presets change only swarms built afterwards.
---|---| | Free Swarm | Boids + turbulence inside a soft sphere, soft points | | Krill Cloud | Aligned krill instances in drifting clusters, faint neighbor links | | Vortex | Swirling shell, speed-colored points, velocity strokes | | Follow Path | School following a figure-eight with tapered trails | | Technical Network | Points, neighbor links and square trail-head markers | | Shrimp Pullback (showcase) | One large hero animal inside a 24 000-animal structured swarm with links, trails and ring markers | | Violet Filaments (showcase) | 20 000 clustered particles with a positional violet palette, long curved filaments, wire-cube heads and dot tails |
FlockDefinition (src/types/flock.ts) is plain, versioned JSON: nodes with
stable ids (no . so they fit property paths), edges, exposed-parameter
bindings, reusable group definitions, layout (stored separately so moving a
node never invalidates the simulation), loop settings and cache settings.
Unknown operators and newer operator versions are retained and reported, never
dropped.
Operators are JSON descriptors in src/services/flock/operators/ (ports,
params, invalidation class, bypass contract, instance limits) and map to
SignalOperatorDescriptor for Signal IR.
| Category | Operators |
|---|---|
| Population | Emitter (sphere/shell/box/disc/point/line/grid, count, group, seed, burst/stagger births, lifetime, active fraction, heading/spread), Merge Emitters |
| Behavior | Flock Rules (cohesion, separation, alignment, radii, FOV, group interaction), Attractor, Vortex, Turbulence, Curl Flow, Home Pull, Drag, Wind, Cruise Speed, Cluster Anchors, Compose Behavior |
| Guidance | Path (circle, figure-8, helix, line, 4-point spline), Follow Path, Obstacle (box/sphere/capsule/plane), Boundary (contain/wrap/reflect/kill) |
| Selection | Group, ID Fraction, Region, Speed, Age, Combine (and/or/xor) |
| Values | Value, Math, Remap, Oscillator, Source Time, Audio Level (existing loudness analysis), Palette |
| Simulation | Simulation (step rate, pace (how far each step carries the swarm: the presets' card Speed, 0 to 4), speed/acceleration limits, turn smoothing, neighbor limit, cell candidate budget, planar mode, warm-up) |
| Render | Points, Instances (krill/fish/arrow/tetra/cube/sphere or an imported model), Neighbor Links, Trails, Curves, Endpoint Glyphs (dot/ring/square/cube/cross/diamond), Velocity Vectors |
| Output | Scene Output |
Rules enforced by validateFlockDefinition / checkFlockConnection: typed
ports, single-input occupancy (repeated inputs are evaluated in edge order),
no cycles (temporal feedback belongs only to the Simulation node), required
inputs, parameter ranges, operator versions, instance limits, group recursion
and capacity ≤ 1 048 576. Invalid drafts are saved and inspectable with
node-scoped diagnostics; preview keeps the last valid image (status stale)
and export is blocked.
The compiler (src/services/flock/compiler/) expands groups, applies
passthrough/mute bypass, lowers the graph to a bounded program and hashes it
into topology / behavior / derived / appearance classes. Layout never changes
a hash; appearance edits re-render only; behavior and keyframe edits
resimulate from the earliest affected step; topology edits restart.
Every graph parameter has the property path flock.node.<nodeId>.<param> with
.x/.y/.z components for vectors and .r/.g/.b (0–255) channels for colors.
Group instances override inner parameters as <innerNodeId>__<param> on the
group node.
Flock parameters use a source-time keyframe basis: keyframe times are
simulation source seconds, not clip-local seconds. The timeline, curve graph
and store actions convert through FlockTimeMapper, so:
- split gives both halves the full parameter history and the second half does not restart;
- trims and slips never shift flock keys;
- speed and reverse retime source-animated behavior consistently;
- keys outside the visible source window are kept, just not drawn.
Emitter counts, seeds, step rate and other structural values are not keyframeable; animate Active Fraction or Lifetime to change population.
- Fixed source-time steps (30/60/120 Hz), independent of render FPS, viewer count or wall clock. Rendering interpolates between the two adjacent steps.
- Packed 64-byte particle state (position, age, velocity, lifetime, smoothed forward, group, identity random, generation, emitter, neighbor count).
- Each step: deterministic births/deaths → spatial hash keys → bitonic sort by (cell, identity) → per-cell ranges → neighbor rules against the previous state with deterministic stride sampling in dense cells and a neighbor limit → field forces, obstacles, boundary → integration with speed/acceleration limits → trail history rings.
- A deterministic TypeScript reference solver (
src/engine/flock/cpu/) defines the semantics the WGSL mirrors and backs the unit tests. - Sessions are keyed per clip and consumer (
preview, pinnedexport,precompute) so several viewers never double-step one simulation. - Sparse restart checkpoints (1 s, thinned under a memory budget) make backward seeks restore-and-replay. Seeking back and forth reproduces bit-identical particle state and pixels on the same device.
- Buffer sizes are admitted against device limits; an oversized swarm is
reported as
unsupportedrather than silently reduced. GPU device loss rebuilds sessions from source/checkpoints. - A session encodes nothing until the driver has accepted every compute
pipeline it uses (
FlockPipelineGate,src/engine/flock/gpu/flockPipelineGate.ts). The pipelines are built synchronously but their verdict arrives later; a frame that encoded a rejected pipeline loses its whole command buffer in every browser (WebGPU spec), and shipping Safari also leaves the encoder locked, so the next pass fails withencoder state is invalid. While verdicts are pending the clip showscompiling("Preparing GPU pipelines…") for a frame and re-renders when they arrive; export, precompute and the render worker wait for them (the worker holds its frame instead of failing it), and a particle sample (sampleFlockParticles, node previews) waits too. If a required pipeline is rejected the clip turnsunsupportedwith an error diagnostic naming the pipeline and the driver's compiler error, the warning is logged once per session, no simulation work is encoded, and export, precompute, the worker and particle samples fail with that message. The workgroup particle-to-grid transfer (fluidP2GBlock) is optional: if the driver rejects it, the liquid runs the referencefluidP2Gtransfer (same fixed-point sums, slower), shows an info diagnostic, raises no skipped-feature notice, and reports one non-fatalGPU pipeline fallback usederror per page session with the condensed compiler lines, so production diagnostics (failure codegpu_pipeline_fallback) can still name the failing construct. - The renderer draws a flock frame only once every pipeline that frame can
use has a verdict (render, shadow, point-cache, compute rasterizer and path
trace pipelines,
FlockBranchRenderer.framePipelines). Until then flock draws nothing for that frame and requests a re-render when the verdicts arrive; a rejected one is then skipped by the device guard instead of invalidating the shared frame encoder. A standaloneFlockFluidGridthat owns its particle order encodes the order reset into its firstencodeinstead of submitting it from the constructor.
Time & Quality → Precompute simulates a source range in a separate session
with 0.25 s checkpoints, hands them to the preview session, and (with Persist
for reopen) writes them to IndexedDB (masterselects-flock-cache) under a key
covering solver version, program semantics, behavior keyframes and GPU adapter.
Quota errors are reported and never touch the editable definition. Clear
cache drops GPU and persisted checkpoints for the clip.
Checkpoint readback submits copies of all state and trail sections before waiting for CPU access, so concurrent cache eviction cannot invalidate persistence. Snapshots evicted within a simulation batch are released after that batch is submitted. The checkpoint owner also supports identity-ordered auxiliary state: each section stays in its own GPU buffer and is appended after canonical particle bytes in the persisted payload; import and adoption validate section sizes. Persistent precompute flushes each checkpoint interval to IndexedDB before advancing to the next interval, so older checkpoints survive GPU-cache eviction. Readback and disk writes are sequential; quota failures stop the job and preserve the successfully written prefix without claiming full completion. Cache pruning reads only record keys, avoiding deserialization of obsolete particle arrays. GPU retention uses a capacity-dependent budget: at least 16 MiB for small snapshots, up to 256 MiB for multiple snapshots, or one complete snapshot when that alone is larger. Particle, affine and trail bytes all count. Capture, import, adoption and budget reductions enforce the same limit.
SceneFlockLayer joins the shared native scene. Opaque branches draw with
depth writes after voxels; additive/alpha branches draw after transparent
planes, depth-tested against everything opaque. Branches read the GPU state
buffers directly (no per-particle CPU copies):
- points and glyph quads in screen or world size with distance fade;
- instanced meshes oriented along velocity with phase-shifted swimming deformation; imported OBJ/FBX/glTF/GLB models are flattened, centered, scaled to the procedural body length and decimated above 20 000 triangles (no skeletal animation);
- screen-space line quads with near-plane clipping for links, vectors, curves and wire-cube glyphs;
- neighbor links are rebuilt from the simulation's spatial index, capped per particle and globally, deduplicated, and faded by distance; link distance is limited to twice the simulation neighborhood (reported, never by changing the simulation);
- trails are Catmull-Rom ribbons over stable identity subsets with taper, tail fade and breaks at rebirths or wraps;
- palettes map group, identity, position noise, speed or age to four colors;
- Points, Instances and Vectors offer an Image color mode ("data pigments"): the branch samples a project image (Pigment Image) at each particle's birth coordinate. Grid emitters map one image pixel per particle (row-major, top row first), so the picture stays readable while the particles move; other emitter shapes sample a stable random pixel.
Terracotta Sculpture and Lilac Sculpture are reusable starting presets for a slowly settling sculpture in a square gallery box. They combine 524,288 grid particles, sixteen requested render sub-particles, weak downward gravity (Y = -8), APIC, slow Curl Flow, restoring Home Pull, Drag, spatial color regions and lit opaque grains. Six local Curl Flow fields overlap a slower global flow. Each has its own noise offset, frequency and evolution rate; their forces add in shared regions so larger folds emerge from interacting flows instead of timed crest triggers. Curl Flow supports Noise Offset, Field Center and a smoothly fading Field Radius (zero retains the existing global behavior). The dense rest sheet starts inside the box, behind the frame. Home Pull and Drag limit dispersal. Motion follows source time and repeats on seek/export; emerging crests have no fixed two-second cadence. The simulation domain extends beyond the gallery opening so folds can project in front of the frame. They start without simulation pre-roll, so the first frame is calm and folds develop during playback. Exposed controls include flow, fold frequency, damping, pre-roll, grain size, pigment colors and lighting.
Preview shows a preparation bar while active Flock clips catch up: the percentage comes from completed simulation steps versus the requested step. Initial scene preparation has an indeterminate bar until that work is measurable. The overlay waits briefly to avoid flashing on fast frames and disappears when ready.
These are particle-surface approximations: Curl Flow applies a force, Drag damps velocity, and APIC transfers velocity through the pressure grid. They do not implement a prescribed velocity field, a viscosity solve, FLIP or a reconstructed continuous fluid surface. Dense folded sculpture references remain a visual target, not a claim of equivalence or real-time performance.
A Grid emitter lays its particles out as a flat canvas whose columns and rows follow the emitter size aspect. Curl Flow is a divergence-free curl-noise force (strength, frequency, evolution over source time, optional finer detail octave), so neighboring particles move together and the canvas folds into sheets instead of clumping. Home Pull springs every particle back to its grid cell (other emitters: the emitter center), so the canvas breathes around its rest pose; combine it with Drag for damping. Grid cells carry a stable per-particle jitter (Grid Jitter, in cells, default 0.6) so dense canvases do not moire against the pixel grid; Home Pull targets the same jittered rest position. Points can draw up to 16 render-only Sub-particles per simulated particle: on grid emitters they are placed bilinearly between a particle and its right, lower and diagonal neighbors, so they stay on the folded surface; other emitters scatter them within Sub-particle Spread. One million simulated particles with eight sub-particles can draw about 8.4 million points. The actual scene target limits sub-particles to about 1.5 points per pixel: 1,048,576 simulated particles request eight children but draw three at 1080p and eight at 4K. Screen point sizes use a 1080p reference, so the same point is twice as wide in a 4K frame. Small round sprites use one triangle; square sprites and larger points retain quads. Emitter capacity is 4,194,304 particles per clip; large scenes retain only the restart snapshots that fit their GPU budget and reload persisted checkpoints as needed.
Opaque screen points with a maximum physical diameter of two pixels use compute rasterization. Full-precision depth and a separate deterministic point-identity pass select each pixel; a fullscreen resolve shades the winning point and writes the shared scene depth, so Room, instances and other 3D layers still occlude it. Alpha/additive points, world-sized points and larger sprites use the existing triangle/quad path. A bounded scratch-buffer budget falls back to that path when device limits cannot accommodate the pixel buffers. The point cache and raster dispatch support two dimensions, including point populations above 16.7 million when device buffer limits permit them.
Parent-particle shadows also use compute for footprints up to eight shadow texels in radius; larger footprints and instance geometry keep the triangle path. The shadow resolve shares the same shadow-map depth attachment with instance casters.
getStats().flockGpu reports the actual scene viewport, requested and drawn
children, simulated particles and shadow point counts. On devices supporting
timestamp-query, it also reports GPU milliseconds for simulation (including
any neighbor-grid construction), P2G (clear and normalization included), pressure
(divergence and projection included), G2P, point caches, shadows and main passes.
Each sample includes pass counts, completion time and a truncation flag; simulation
batch totals must be divided by their pass counts to compare individual steps.
Readback uses three bounded slots and skips samples while they are busy, without
waiting on the GPU. Unsupported devices keep rendering with no timing queries.
computePoints reports the number actually routed through compute. Additional
GPU pass labels are clearPixels, pointDepth, pointWinner and shadowDepth;
lighting for compute points runs in the main resolve, without a per-point
cacheVisibility pass.
Room, light and shadows. The Room render node draws an open-front white gallery box (back wall, floor, ceiling, sides) with an optional flat frame ring around the opening, so particles can spill out past the box edge. Room also defines the key light (direction in simulation space, ambient, shadow strength). Each lit flock clip renders its point and instance branches into a 2048² orthographic shadow map framed around the room (or the emitters); point shadows use only the simulated parent particles, with a wider footprint based on the requested sub-particle count, independent of preview LOD. Points with Shading: Lit draw sphere impostors that receive the light and 3x3 PCF shadows, lit Instances and the Room walls receive them too, and the walls darken softly where they meet. Points Relief moves each point and sub-particle along the emitter normal by Pigment Image brightness. Instances keep their previous fixed-light look while no Room or lit Points branch enables the key light.
Fluid sessions keep both simulation/interpolation states in a stable spatial order,
refreshed every four steps with a GPU cell-key radix sort. Forward and inverse
identity maps preserve spawning, pigment UVs, sub-particle neighbors, boid rules,
links and trails. Workgroups combine fixed-point face contributions in 1024
workgroup slots (12 KiB, within the 16 KiB spec default) before global atomics.
Slots are claimed in a separate phase with atomicMin (the smallest face hashed
to a slot owns it); contributions whose slot belongs to another face go straight
to the global sums. Every contribution is the same integer either way, so the
result is bit-identical to fluidP2G. Shipping Safari's Metal compiler rejected
the earlier compare-exchange hash table (production 2026-10-29). That shader was
valid WGSL within the default limits; the most likely cause is WebKit's MSL
codegen for atomicCompareExchangeWeak (strong inference, the compiler line was
truncated). As a portability guard,
tests/unit/flockWgslPortability.test.ts keeps all flock compute shaders within
the default workgroup limits and free of atomicCompareExchangeWeak. Checkpoints and
diagnostic samples retain canonical identity order; checkpoint restore resets the
maps and rebuilds the spatial order. Ordinary substeps do not copy the complete
state into and out of an intermediate transfer buffer. On sort steps, only the
simulation input is reordered: the next simulation dispatch fully overwrites the
output, leaving both interpolation states in the same new order.
Sorting uses a reusable 64-byte-per-particle reorder buffer, permutation scratch
and two identity maps, included in runtime memory estimates. GPU timings separate
fluidSort, order-keys, order-reorder and order-updateMapping from P2G,
pressure and G2P. These pass samples exclude buffer-copy commands; use timestamps
around the complete command sequence when measuring total step costs.
APIC Fluid. The APIC Fluid behavior turns the particles into an incompressible liquid inside a box domain (Domain Center/Size, Cell Size, Pressure Iterations, Gravity, Affine Strength, Separation, Particle Spacing, Position Jitter). Each particle stores a tightly packed 3×3 velocity-gradient matrix (36 additional bytes), preserving local rotation and shear across transfers. Each step, after forces and advection, the GPU transfers particle velocities plus the affine contribution to a staggered MAC grid with fixed-point atomics (order independent, so resimulation stays deterministic), marks fluid cells, solves pressure with multigrid-preconditioned conjugate gradients (MGPCG), projects the grid velocity and transfers it back with trilinear velocity interpolation and its gradient, plus a position correction; the domain walls are solid. Two deterministic extrapolation layers fill missing face velocities before pressure projection, including corners whose interpolation weight is zero but derivative is nonzero. At truncated walls the gradient differentiates normalized weights, so a constant tangential velocity does not introduce artificial shear. Affine Strength defaults to 1 (APIC); 0 removes the affine contribution for more dissipative PIC behavior. Separation (default 0.15) gently moves overlapping particles apart after G2P; Particle Spacing (default 0.1) is measured in grid cells, with a maximum of 0.5. Dense neighborhoods use a stable sample of at most 64 candidates per particle. All corrections read the same particle state before being applied, are bounded, and leave velocity unchanged. Separation zero skips the extra index and neighbor passes. Position Jitter (default 0.002 grid cells) adds small, deterministic, zero-centered position noise keyed by particle identity, generation and absolute simulation step. Its amplitude scales with the square root of the timestep. These controls are intended to reduce grid-aligned bands; they do not guarantee exact volume preservation. CPU and GPU use the same rules. The GPU reuses the spatial sorter and reorder scratch, adding only two cell-range arrays; separation still incurs an additional sort and neighbor pass each active substep. The range pass also gathers positions and ages in sorted order. Neighbor queries read this contiguous snapshot instead of repeatedly following identity and slot maps. Corrections occupy the first 16 bytes per particle of reorder scratch, and the position snapshot the next 16 bytes; both full and compact layouts fit without another allocation. Candidate order, identity-based sampling and corrections are unchanged. A Worker GPU check on 262,144 particles produced bit-identical state and reduced the isolated separation median from about 10.1 to 2.7 ms on the tested AMD device (two runs each, 40-second cooldowns). This is a pass measurement, not an overall playback FPS guarantee. Fluid submissions scale their step count down with particle capacity. Preview simulation waits for its preceding submission to finish before queuing more steps, preventing a seek or playback catch-up from building an unbounded GPU work backlog. This can leave the simulation behind the playhead on slow GPUs; export still computes every required step. The separate boid hash/sort buffers are sized for the population only when a connected boid-rules node or neighbor-link render branch consumes them. Other graphs keep 32 bytes of placeholder storage for the shared shader bindings. Fluid's own stable cell sorter remains allocated. Adding/removing a neighbor consumer changes graph topology and recreates the session with the appropriate layout; animated boid weights retain the full index even when currently zero. Simulation, boid hashing/sorting/cell ranges, trails and neighbor links support two-dimensional compute dispatch. Each shader derives its linear particle index from the dispatched row width, so partial final rows preserve identity and do not revisit earlier particles. Fluid transfer and radix ordering already use two-dimensional dispatch. This removes the one-row dispatch constraint without raising the configured population ceiling or bypassing storage-binding limits. The particle branch renderer receives model and pigment resources through an injected runtime asset interface. The editor adapter owns store lookups and render requests; a device-local registry accepts uploaded bitmaps and mesh data without importing the editor host. This allows the same GPU simulation and renderer to run in a Worker with a transferred OffscreenCanvas. The editor's shared 3D path still runs on the main thread until worker-host routing, scene transport, audio evaluation and runtime diagnostics are connected.
FlockSimulationRuntime now owns the same persistent GPU sessions, source-time
mapping, keyframe invalidation, checkpoints and precompute in either execution
realm. Its injected host provides render notifications, status delivery, assets
and audio analysis access. The main-thread registry is an adapter; a worker can
use the runtime without importing timeline/media stores. This removes duplicate
simulation orchestration from the migration, but does not yet switch the editor
preview to the worker host.
The native scene's FlockPass also receives its runtime from its scene owner.
Its collection, simulation preparation and opaque/transparent drawing run in
either realm with the same device and depth attachments. The editor scene owner
still defaults to the main-thread adapter.
NativeSceneRuntime contains the shared scene graph and GPU passes. The editor's
NativeSceneRenderer supplies live host callbacks for simulation, playback state
and media fingerprints; worker owners can supply equivalent snapshots. A real
Worker check renders Flock together with native mesh geometry and verifies their
shared depth. This boundary does not yet route editor compositions to the Worker.
The existing Worker GPU frame stack accepts a versioned native-scene payload for
Flock, primitive meshes and light layers. Its target owns persistent scene and simulation state
per composition occurrence, waits for the requested source step, and composites
the scene texture with other sources using the existing exact-frame/export path.
Camera matrices, graph definitions, keyframes and clocks are admitted as bounded
plain data. The opt-in Worker host groups evaluated Flock, primitive and light layers
at the shared scene's original stack position, including nested compositions
with their own camera context. Image pigments and imported instance models use
runtime media URLs: the Worker loads and caches them on its device, replaces
changed URLs, and releases inactive resources. Images share the main renderer's
2048-pixel decode limit; full model imports are reduced to instance geometry.
Missing resources fail the exact frame instead of silently using placeholders.
Audio-driven graphs transport references to the existing loudness analysis,
with occurrence-local clip placements and explicit unavailable-analysis values.
Main and Worker share dB normalization and smoothing; analysis changes and clip
placement changes invalidate simulation state and use distinct persisted
checkpoint keys. Curve data is loaded once per resource URL using the existing
loudness payload format. Host URLs remain valid until referencing frames expire.
An audio change during precompute cancels that run before mixed analysis state
can be persisted or adopted by the preview. Export preparation also rejects a
frame if its audio input changes while awaiting checkpoints or GPU completion.
The comparison uses the audio identity captured when preparation began, even if
another request refreshes the shared export session during that wait.
Point, panel and constant-color environment lights carry evaluated transforms,
color, intensity, diameter and shadow settings into the same native scene renderer.
Environment-map resource references are rejected until their Worker resource path
is implemented. Flock's Room-node lighting remains independent of these scene lights.
Presented Worker frames return Flock simulation steps, memory, timings, checkpoints
and device limits. The active composition's inspector reads this Worker status;
nested occurrences remain separate under render-host diagnostics (flockStatus).
Changing composition or detaching a target cannot expose a stale Main status as
Worker progress. Precompute, particle sampling and cache mutation still await
Worker command routing; requests do not operate on the dormant Main backend.
getStats.flockGpu selects measurements from the active scene renderer and labels
them with source: worker or source: main. Worker frame results include their
device's latest completed GPU timing samples and point counts. capturedAt is the
snapshot time; each sample's updatedAt, sequence, pass counts and truncated
flag describe the asynchronous readback, which can precede the presented frame.
These are bounded recent per-scope samples, not whole-frame totals or FPS; no
extra wait or pixel readback is added. A missing Worker snapshot never substitutes
stale Main measurements. Worker devices request timestamp-query when supported
and the adapter's maxStorageBufferBindingSize / maxBufferSize. Unsupported
timestamps remain disabled; raised limits do not themselves allocate GPU memory.
Render-host flockStatus.frameTiming separates Worker frame age, resource setup,
simulation preparation, command encoding and submission wait in wall-clock
milliseconds. These include scheduling and waiting, unlike GPU pass timestamps.
flockStatus.preparation adds simulation step and prepareCalls counts,
CPU encodeMs, intermediate completion waitMs, and deferredSteps whose
completion is covered by the final compositor fence. Intermediate catch-up
uses a window of at most two simulation command buffers before waiting for
completion; each buffer retains its capacity-dependent step limit. This allows
backend command preparation to overlap preceding GPU work. The final simulation
block and rendering use the same GPU queue in order, avoiding an extra CPU
round trip before rendering. Exact presentation still requires the compositor's
completion fence; queued work is not reported as completed deadline progress.
Each GPU timing sample also reports spanMs from its first measured pass start
to its last measured pass end, and gapMs for time within that span outside
the measured passes. This includes GPU scheduling gaps and any intervening
unmeasured commands, not CPU time or a whole-frame GPU total. Truncated samples
describe only their measured prefix.
Presented-frame status remains current during playback even when the next frame
is already queued; partial catch-up status still requires the newest request.
If simulation catch-up exceeds a frame deadline, the Worker reports its retained
completed steps and a computing status. The host requests a fresh frame from the
current editor state; expired frame resources are never reused. A newer seek or
replaced target suppresses continuation of the old request. Errors without
completed progress stop instead of triggering an unbounded retry loop.
Other 3D layer kinds, scene effectors, gizmos and complete
cache control routing remain outstanding; this is not yet the full Worker migration.
Local development defaults to strict worker-gpu-only, including /editor
without a query parameter. Automatic main-renderer fallback is disabled, even
when worker capabilities are unavailable. Explicit ?renderHost=main is still
available for diagnostics. Production startup is unchanged.
Worker frame stacks use the current render-target size, including Half/Quarter
preview quality. The Worker resizes the transferred canvas itself, and sizes set
before canvas registration are retained. Preview simulation catches up in batches
of at most four steps, yielding at the asynchronous GPU completion fence between
batches. No fixed timer is added after the fence, avoiding extra playback backlog.
Simulation time steps and particle counts
are preserved; export keeps its unrestricted step budget.
Worker preview cache controls route Precompute, Cancel and Clear cache to the
prepared scene's simulation owner. Precompute uses a separate GPU session with
four-step submissions; the UI polls bounded progress snapshots every 250 ms.
Persistent checkpoints use IndexedDB in the Worker and can be reused after reopen.
These are restart checkpoints, not a cache of every rendered output frame.
Replacing the scene or editing simulation inputs cancels the old computation.
The host is selected before preview canvases
acquire a context, allowing the existing editor preview to register directly with
the Worker. Development URL overrides are ignored in production; reloads still
clear stale experimental host preferences. A live mode switch does not transfer
canvases that already belong to the main renderer.
The GPU fluid, cell-order and separation kernels also accept a 32-byte fluid
core (position/age, velocity/generation). Pipeline caches distinguish it
from the full 64-byte particle state, and sorting scratch uses the same stride.
The editor session and renderer still use the full format: lifecycle metadata,
render attributes and checkpoint integration must be connected before enabling
compact storage in the preview. This kernel support alone does not reduce the
current editor's particle memory.
GPU particle-to-grid transfers select power-of-two fixed-point scales per grid face from current local particle counts and conservative velocity bounds, including the affine contribution. Dense piles and fast motion therefore retain signed sums without int32 wraparound. Both direct and workgroup transfers use the same scales; ordinary low-density transfers retain the original precision. Bounds use 16 extra bytes per grid cell and reuse existing face scratch for scales.
Affine Strength replaces the former FLIP Ratio; existing fluid nodes now use APIC, and older simulation caches are invalidated. The affine matrix remains in identity order through particle sorting, is reset on respawn/restart, and is included in CPU/GPU checkpoints, persisted imports and precompute handoff (100 state bytes per particle in a GPU checkpoint, before trails). Pressure Iterations is the maximum CG iteration count (12 for new nodes); updates stop at a relative residual of 1e-5 or an absolute residual of 1e-6. The symmetric V-cycle uses Galerkin aggregates, two damped-Jacobi sweeps before and after coarse correction, and 16 coarse sweeps. It retains solid-wall and air boundaries, including odd grid sizes and thin fluid regions. CPU and GPU use the same solver. Pressure scratch is included in memory checks; changing the solver version invalidates older simulation checkpoints. Domain and cell size are topology (changing them rebuilds the grid and resimulates). Other forces still apply, so Curl Flow adds swirl; bypass Home Pull for a free liquid. Pointing Gravity into the box (for example 0, 0, -150) with a matching shallow Room makes the liquid pour against the back wall. A CPU reference solver with the same discretization backs tests and the low-count fallback. The grid is capped at 256 cells per axis and about 2 million cells. There is no particle collision. A typical graph is Grid Emitter -> Curl Flow + Home Pull + Drag -> Simulation (min speed 0) -> Points in Image color mode.
Flock clips render only in the main-thread render host (like all shared-scene 3D today). Color, masks and 2D clip effects are not offered for flock clips; use nested compositions for image-space treatment.
Audio Level reads the existing loudness envelope of a timeline audio clip's
media (audioAnalysisRefs.loudnessEnvelopeId), mapped from flock source time
through both clips' timeline placement. Missing analysis is reported as
unavailable and outputs the floor value; a completed analysis load resimulates
the clip.
With a flock clip selected, the preview draws guidance overlays for the graph: emitter, vortex, attractor, boundary and obstacle centers are draggable handles (one undo step per drag), paths and boundary volumes are shown as dashed outlines. Timeline clips show their icon by default. Settings → Performance → Simulated Flock thumbnails adds a filmstrip from a capped CPU re-simulation; it runs on the editor thread and stalls heavy scenes such as fluid sculptures.
The editor exposes atomic flock tools (src/services/aiTools/definitions/flock.ts):
| Tool | Policy |
|---|---|
listFlockOperators, getFlockClip |
read-only |
createFlockClip, applyFlockPreset, addFlockNode (optionally wired in the same step), updateFlockNode, removeFlockNodes, connectFlockPorts, disconnectFlockEdge |
medium, undoable |
exposeFlockParam, unexposeFlockParam, scheduleFlockPrecompute, cancelFlockPrecompute |
low, undoable |
sampleFlockParticles |
read-only, dev bridge/console only (not offered to chat) |
Graph edits are planned on a copy and committed atomically; an invalid wiring
or parameter leaves the definition unchanged. addKeyframe accepts
sourceTime for flock.node.* properties, and keyframe results report
timeBasis: 'source' together with the clip-local time.
Reference measurement (2026-09-13): AMD Radeon RDNA3 discrete GPU, Chrome 152, 1920×1080 scene target, Free Swarm graph (flock rules + turbulence + contain boundary + soft points), uniform sphere distribution, 60 Hz step rate. Step cost is measured end-to-end including GPU completion for 60 consecutive steps.
| Population | GPU memory (state + index) | Step cost |
|---|---|---|
| 10 000 | 2.5 MB | ~1 ms |
| 50 000 | 11.7 MB | ~8 ms |
| 100 000 | 23.3 MB | ~21.5 ms |
The spatial index (hash + bitonic sort + cell ranges) dominates at high counts; it is skipped on steps without flock rules. Resulting tiers on this hardware: interactive real-time playback at a 60 Hz step rate up to roughly 40 000 particles, and at a 30 Hz step rate up to about 100 000. Dense and fully collapsed distributions stay bounded by the candidate budget (every particle is stride-sampled; reported as sampled particles in Time & Quality). Larger populations are admitted up to the device's storage-binding limit and still export correctly, just slower than real time. Integrated and mobile GPU tiers have not been measured yet.
- Supported counts are GPU-dependent (see above); the showcase presets run 20–24 k animals/particles. Higher counts are admitted up to device buffer limits.
- Not part of this release: visual density amplification (X1), mesh/SDF obstacles, formations and cross-clip interactions (X2), skeletal/animated instances and refraction (X3), a particle expression language (X4), a general procedural platform (X5).
| Area | Location |
|---|---|
| Types | src/types/flock.ts |
| Operators, validation, compiler, evaluation, time, presets | src/services/flock/ |
| Store actions | src/stores/timeline/flockClipSlice.ts |
| CPU reference solver, GPU sessions, shaders, renderer, registry | src/engine/flock/ |
| Scene integration | src/engine/native3d/passes/FlockPass.ts |
| Properties tab | src/components/panels/properties/flock/ |
| Node Workspace Flock group | src/components/panels/nodes/flock/, src/services/nodeGraph/clipGraphFlockProjection.ts |
| Tests | tests/unit/flock*.test.ts, tests/unit/FlockTab.test.tsx |
The Wind operator shares its directional-force contract and CPU/WGSL calculation with Face Cables. Existing Flock defaults and deterministic gust modulation are retained. See Node Workspace for the common canvas.
Points offer Sprite Softness, Sprite Opacity Variance, and Sprite Motion Blur. Softness expands a Gaussian halo without a flat white center; stable per-particle opacity variation avoids flicker. Motion blur stretches the sprite along projected simulation velocity (in seconds), preserving energy, with no additional simulation or scene samples. Camera motion is not included. Defaults preserve existing presets. These are Raster sprite controls; path-traced points remain physical spheres.
The Transform panel's blend mode and opacity also apply to Flock layers. Custom modes isolate the depth-tested particle layer and use the editor's shared blend functions against the scene. Normal/full-opacity layers keep the direct fast path. Custom-composited point layers remain raster overlays when using path tracing so the layer is not rendered twice.