perf: halve export pipeline peak memory, with bit-identical output - #114
Open
Dgmtnz wants to merge 2 commits into
Open
perf: halve export pipeline peak memory, with bit-identical output#114Dgmtnz wants to merge 2 commits into
Dgmtnz wants to merge 2 commits into
Conversation
…d its check meshRepair packs an edge as `a * 4294967296 + b` (a * 2^32 + b) into a JS number. float64 holds that exactly only while it stays inside 2^53, i.e. up to a = 2^21 = 2,097,152 vertices. Above that distinct edges collide onto one key. diag-edgekey-collision.mjs (added) builds a closed torus whose manifoldness follows from its grid topology rather than from any measurement, so a counter that disagrees is wrong by construction: 1.54M vertices -> 0 non-manifold edges (below the threshold) 2.52M vertices -> 210,422 non-manifold edges (mesh is perfect) 3.74M vertices -> 819,608 non-manifold edges (mesh is perfect) In countEdgeDefects the collision is a false alarm on a good file: a 400mm sphere exported at 0.35mm / 5M triangles reported 185,146 non-manifold edges that staged measurement showed were not in the mesh at any pipeline stage. In resolveTJunctions it is a correctness bug. A genuine boundary edge (count 1) colliding with another reads as count 2, so its T-junction is silently left unrepaired, and decoding the key back with `b = k % 4294967296` yields vertex ids that were never on that edge, feeding wrong candidates to the split search. Both now key on exact Int32 pairs via a new IntPairMap in meshIndex.js, over a dense edge table. This also removes a second ceiling in countEdgeDefects, which counted in a JS Map and throws past V8's ~16.7M entry cap (~11M triangles). Below 2.1M vertices the old keys were exact, so this is a no-op there: pipeline fingerprints are unchanged across sphere, cube-with-fillets, cone, holed-plate and sliver-strip inputs at 10 model/texture/resolution combinations.
Profiling the export pipeline with live typed-array accounting (process.memoryUsage().arrayBuffers + .heapUsed sampled every 5ms; RSS overstates by ~30% because V8 does not return freed pages promptly) put the peak at 660 bytes per subdivided triangle, all of it in decimation — not the "~145 B/tri" the subdivision safety-cap comment assumes. Peak per subdivided triangle, sphere at 3.29M triangles: stage before after subdivide 178 147 displace 254 216 decimate 660 327 All of it is allocation sizing and lifetime; none of it changes geometry. - SoAHeap was sized at 3F entries, then rounded up to a power of two. Seeding pushes one entry per UNIQUE edge (1.5F by Euler), so a 4.9M-entry heap was allocated as 16.7M slots x 48B = 805MB. Capacity is only a bound in push() — nothing masks on it — so the pow2 rounding was pure waste too. - buildIndexed allocated `positions` at the corner count and returned a subarray VIEW, keeping a 6x-oversized buffer reachable for the entire run (237MB to store 39MB). Grows on demand, returns a copy. - slotFace[s] is always (s/3)|0 and faceSlot[s] only ever held s or -1; the first is gone and the second is a Uint8Array flag (-24 B/tri). - seedSeen and vertMap sizing hints were 2-6x over, each tipping its table over a power-of-two doubling; seedSeen is also freed after seeding rather than held through the collapse loop. - decimate(..., releaseInput) lets the caller hand over the input geometry, so its 72 B/tri is not held across the collapse loop. dispose() cannot do this: it frees GPU resources, not the JS typed arrays. - subdivision's splitEdges/midCache move to IntPairMap (12 B/slot vs 28). Verified with bench-pipeline.mjs: 10/10 fingerprints identical to before, across sphere, cube-with-fillets, cone, holed-plate and sliver-strip inputs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Cuts the export pipeline's peak memory roughly in half, with bit-identical output. Pure allocation sizing and lifetime — no geometry, no algorithm, no UX change.
Depends on #113 for
IntPairMap.Measurement
process.memoryUsage().arrayBuffers + .heapUsed, sampled every 5 ms. Not RSS — V8 doesn't return freed pages to the OS promptly and RSS overstates the peak by ~30 %, which is part of why the existing figure had drifted.Sphere, 3.29 M subdivided triangles, bytes per subdivided triangle:
For context,
subdivision.js's safety-cap comment assumes ~145 B/tri. The real figure was 660.Where it went
SoAHeapcapacity. Sized atfaceCount * 3, then rounded up to a power of two. Seeding pushes one entry per unique edge — 1.5 F by Euler, not the 3 F edge slots the face loop visits — so a 4.9 M-entry heap was allocated as 16.7 M slots × 48 B = 805 MB, over half the pipeline peak. Capacity is only a bound inpush(); nothing masks on it, so the pow2 rounding bought nothing either.buildIndexedpositions. Allocated at the corner countnand returned as asubarray— a view — so a 6× oversized buffer stayed reachable for the whole decimation (237 MB holding 39 MB). Now grows on demand and returns a copy.slotFace/faceSlot. Slots are assigneds = f*3+kand never renumbered, soslotFace[s]is always(s/3)|0;faceSlot[s]only ever heldsor-1, i.e. one bit. First removed, second is aUint8Arrayflag. −24 B/tri.seedSeenandvertMapwere 2–6× over, each tipping its table over a power-of-two doubling.seedSeenis also released after seeding instead of held through the collapse loop.decimate(…, releaseInput).buildIndexedis the only reader of the input geometry; when the caller discards it anyway (the export pipeline disposes it on the next line), dropping the attributes frees 72 B/tri for the whole collapse loop.dispose()can't do this — it frees GPU resources, not the JS typed arrays.splitEdges/midCachemove toIntPairMap(12 B/slot vs 28).Risk
Output is bit-identical: 10/10
bench-pipeline.mjsfingerprints unchanged across sphere, cube-with-fillets, cone, holed plate and sliver strip. Every change is allocation size or object lifetime.Caveats
bench-decim-quality.mjs/bench-tri-estimate.mjsnot run — they need model files not in the repo.