perf: read each wrapped module from disk once#265
Draft
BridgeAR wants to merge 3 commits into
Draft
Conversation
bengl
reviewed
Jun 30, 2026
BridgeAR
force-pushed
the
BridgeAR/2026-06-30-dedup-source-reads
branch
3 times, most recently
from
9eace65 to
7bcd78f
Compare
Wrapping an ESM module reads its source to lex the exports, and the wrapper it emits then does `import * as namespace from <realUrl>`, which without this made Node read the same file a second time. A leaf reached through several barrels (`export *`) was additionally read and lexed once per barrel, though its export set is intrinsic to the file and stable for the process. `esmExportsCache` now memoizes both: the first lex serves every later barrel, and the source it read is served to the wrapper's own namespace import via `takeCachedSource`, so each wrapped module is read from disk once. The memo holds source only until that one import consumes it, then drops it and keeps the export names. Source is cached solely for `module` / `module-typescript`: a wrapped CJS module's real load must return `source: undefined` (else `require()` of a "module-sync" package fails with ERR_VM_MODULE_LINK_FAILURE), and a load without a usable format has to be resolved by Node, which serving a cached entry would short-circuit into ERR_UNKNOWN_MODULE_FORMAT. The TypeScript source is cached before type-stripping because Node strips again on the real load.
A leaf reached through two barrels and imported directly exercises the export memo and the served-once source: the second barrel reaches the leaf through the memo. Importing the leaf's bindings through both barrels makes the memoized re-export set observable, so a memo that returned the wrong set for the second barrel would drop those bindings and fail here.
BridgeAR
force-pushed
the
BridgeAR/2026-06-30-dedup-source-reads
branch
from
7bcd78f to
96e400a
Compare
Wrapping an ESM module reads its source once to lex the exports; the wrapper then does `import * as namespace from <realUrl>`, and serving that import the source the export scan cached skips the loader chain's second `load()` for the real module. An upstream loader that returns different source per call — a non-idempotent transform or codegen loader, or one that keys off `context` — then has its second load silently dropped: both the executed module and the hooked namespace see the export-scan source, not the source the real load would produce. The wrapper's namespace import now falls through to the loader chain like any other import. Only the export names stay memoized, so a leaf reached through several barrels is still lexed once; the per-wrapped-module source read the cache saved comes back. On a 480-module mixed graph that costs ~7% of import time on Node 24 (sync hooks 113 -> 121 ms, n=21, best+worst dropped) and is within noise on Node 18; the export-name dedup keeps the graph faster than before the barrel memo landed.
BridgeAR
force-pushed
the
BridgeAR/2026-06-30-dedup-source-reads
branch
from
87210c7 to
77fbe68
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wrapping a module reads its source twice and re-lexes shared leaves once per
barrel. Two changes remove both, with no observable behavior change:
emits then does
import * as namespace from <realUrl>, so Node reads thesame file again to compile that import. Stash the source the export pass
fetched and serve it to the namespace import.
export *barrels is lexed once per barrel eventhough its export set is intrinsic to the file. Memoize the lexed ESM export
names by URL so the first lex serves the rest.
On a 390-module barrel-heavy graph under the synchronous loader: export lexes
588 → 396 (-33%), total module read calls 1970 → 794 (-60%).
The source cache only holds entries whose upstream load supplied a format; a
load that returns source without one relies on Node to resolve the format on
the real load, and serving such an entry would throw ERR_UNKNOWN_MODULE_FORMAT.
The export memo only caches the pure ESM result — the CommonJS path mutates
context.formatand resolves re-exports per call, and built-ins alreadymemoize via BUILT_INS.