perf(native): project-local content-keyed transform cache, lazy @babel/core#73
Open
danfry1 wants to merge 4 commits into
Open
perf(native): project-local content-keyed transform cache, lazy @babel/core#73danfry1 wants to merge 4 commits into
danfry1 wants to merge 4 commits into
Conversation
…l/core - Move the transform disk cache from os.tmpdir() to the project's node_modules/.cache/vitest-native/ (tmpdir fallback when node_modules is absent/unwritable). tmpdir is ephemeral on CI runners — every job paid the full cold Babel transform of RN's ~250-file boot graph — and macOS purges it periodically. node_modules/.cache persists across runs and is restorable by standard CI dependency-cache actions. The V8 compile cache is colocated so both caches share a lifecycle. - Key disk entries by sha1(platform + source content) instead of path + mtime + size. Content keys survive fresh installs, Docker mtime normalization, and CI cache restores, and eliminate the wrong-hit class where a same-size same-mtime file with different content (e.g. a different RN patch under a version-independent path) served stale executable code. The cache dir name now carries the @babel/core version next to the preset version, so a Babel upgrade invalidates cleanly. - Load @babel/core lazily, only on a cache miss. Requiring Babel costs ~35ms vs ~0.5ms for resolve-only version reads; under the default engine (isolate: true) every worker paid that eagerly even with a fully warm disk cache. Measured on the package's native suite (warm): aggregate worker setup -30%, wall clock -11%; scales with test-file count. The source string is already in hand at both transform call sites, so the content hash replaces a statSync rather than adding I/O. The in-memory per-thread cache keeps its mtime+size key (one stat) so the hot path skips hashing.
Babel's output is not content-pure: the preset's dev-mode JSX transform embeds the absolute filename (_jsxFileName) and varies by BABEL_ENV/NODE_ENV (dev injects source info that test/production do not). Verified empirically against @react-native/babel-preset 0.85. A pure content key would therefore share entries across identical sources at different paths — serving file A's embedded identity as file B's — and across environments. The disk key now hashes platform + path + content, and the cache directory name carries the Babel environment next to the preset and Babel versions. CI-restore validity is preserved: CI workspaces use a stable checkout path and a stable NODE_ENV, which is the sharing that matters. The cross-path dedup a pure content key would have provided is deliberately given up.
Pre-PR review finding: other test files transform into the same shared project cache directory from parallel workers, so directory-entry counts race. The assertions now compute the expected sha1(platform, path, content) key and check entry existence and non-rewrite (mtime stability) directly — which also pins the key scheme itself.
Windows absolute paths (D:\...) are rejected by Node's ESM loader as an unsupported URL scheme; pathToFileURL makes the inline script portable. Caught by the Windows leg of the full gate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Three cost sinks in the native engine's transform pipeline, all from the audit:
os.tmpdir()— ephemeral on CI runners, so every job paid the full cold Babel transform of React Native's ~250-file boot graph, and macOS purges it periodically, silently re-cooling local caches.@babel/corewas loaded eagerly before the cache lookup. Requiring Babel costs ~35ms vs ~0.5ms for resolve-only version reads; under the default engine (isolate: true) every worker paid that even with a fully warm disk cache.Change
<projectRoot>/node_modules/.cache/vitest-native/(tmpdir fallback when node_modules is absent or unwritable — strictly more environments work than before). The V8 compile cache is colocated so both share a lifecycle, and the location is restorable by standard CI cache actions (wired up in ci: cancel superseded runs, cache bun downloads and transform output, bound matrix cells #72; the two PRs land in either order).@babel/coreversion, and the Babel environment. The path and env are in the key deliberately: Babel's output is not content-pure — the preset's dev-mode JSX transform embeds the absolute filename (_jsxFileName) and differs underNODE_ENV=developmentvs test/production (verified empirically against@react-native/babel-preset0.85). A pure content key would share file A's embedded identity with file B. CI-restore validity is what matters and is preserved: CI workspaces use a stable checkout path and env. The content hash replaces astatSync(the source string is already in hand at both call sites), so the hot path gains no I/O.@babel/coreloads lazily on the first cache miss;init()only resolves paths and reads versions, and the friendly install-guidance error still fires at startup for the missing-package case.Measured
Package's own native suite (29 files), warm cache: aggregate worker setup −30% (4.4s → 3.0s), wall clock −11% (1.75s → 1.56s) — scales with test-file count since every isolated worker pays init. On CI the relocation eliminates the per-job cold transform entirely once #72's cache step is active.
Review
Independently adversarially reviewed pre-PR (no blockers). Findings incorporated: exact-key test assertions replacing racy directory-entry counts; the filename/env non-purity was caught during development and is regression-locked by the key scheme and tests. Known accepted nits: no cache eviction yet (entries are small; follow-up), and a corrupt-but-resolvable Babel install now errors at first miss rather than startup (the raw error is arguably more actionable than wrong install advice).
Full gate: mock 1297 + 9 new cache tests, native suites (stock/hot/isolation/android/navigation/soak), crosscheck 75/75, consumers from the packed tarball, lint/format/typecheck clean.