feat(goexec): support runtime embedding and reduce scheduler contention

[kdy1]

Adds the APIs needed to embed goexec in Turbopack: entered runtime contexts, borrowed caller-thread Handle::block_on, independently owned cancellation handles, worker lifecycle hooks, and shutdown that joins all workers and the monitor. Existing Runtime::block_on and timeout shutdown remain available.

Handle::block_on marks only waits between polls as blocking and reacquires an execution permit before polling again. Nested waits work with one permit. Calls inside an already marked blocking region are rejected. Hook callbacks run outside scheduler locks; a hook panic closes admission and cancels tasks.

Task publications notify registered event listeners without taking the scheduler mutex. Workers retain their stealing peer lists until the worker set grows, and retain permits across a bounded poll loop with scheduler checkpoints. Permit acquisition and FIFO priority for returning callers remain protected by the scheduler.

Blocking handoff measures the unchanged default 100µs grace period from call entry. One scan can replace multiple mature blocked workers when queued work needs them, reserving a replacement for every released permit and respecting the configured thread cap. This retains a clock read at the outermost blocking boundary. WASM uses the event's Future interface with the existing parker.

Cancellation registries reuse sharded Slab slots, with at least 16 shards even at parallelism 1 to separate external registration from completion. JoinHandle uses async-task's result storage directly; dropping it still detaches the task. User futures of up to 64 bytes share async-task's pinned allocation, while larger futures retain the previous Pin<Box<Abortable<F>>> layout. Safe pin projection and pinned drop preserve address stability and isolate poll/destructor panics. Unobserved output destructor panics are also isolated. Per-worker poll counts use a single-writer atomic store.

Verification

• Pinned nightly-2024-07-21 and Next.js nightly-2026-08-20: 51 unit/integration/doc tests and 7 Loom models passed on each, plus clippy with warnings denied.
• Formatting and browser WASM compilation passed. CI at the exact head SHA passed, including macOS, Linux and Windows goexec tests.
• Next.js: 515 Rust tests plus N-API runtime ownership; 20 dev/start/addon lifecycle tests; full bootstrap (18 tasks), explicit release native/CLI/micro builds and related clippy/fmt passed.
• Next.js browser WASM and WASI N-API compile checks passed. All 11 product feature/target dependency graphs are Tokio-free. Native Next.js execution was validated on macOS ARM64.
• Regression coverage includes nested single-permit progress, CPU parallelism, cancellation and shutdown races, worker growth and spawn failure recovery, hooks, stale cancellation handles after slot reuse, detached output panics, and exactly-once destruction/address stability for small and large non-Unpin futures across completion, cancellation and poll/drop panics.

Performance

Final goexec SHA: 19171c705febb83495f98773c604594f488b04e6.

Measured locally on Apple M3 Max (16 logical CPUs, 48 GiB), release/thin LTO/one codegen unit/TurboMalloc. Each comparison contains 12 workloads at parallelism 1/4/16, 10 alternating process pairs, then a separate 10-pair confirmation: 1,440 observations per comparison, 2,880 total. Profiling and exploratory builds are excluded. Binary hashes and identical fixtures were verified. Intervals are paired log-ratio t intervals (95%, 9 degrees of freedom), without multiplicity correction; series are not pooled. This is not a dedicated machine.

Against the immediately previous goexec ddd0fcfb, the allocation/sharding/counter changes produced these replicated confirmation results:

Workload	Parallelism	Time change	95% CI
spawn/join	1	-21.7%	-23.7% to -19.6%
spawn/join	4	-18.5%	-20.8% to -16.2%
spawn/join	16	-17.6%	-18.6% to -16.6%
yield	1	-21.1%	-21.8% to -20.4%
yield	4	-8.1%	-10.4% to -5.8%
yield	16	-4.2%	-6.8% to -1.6%

No product build/dev/HMR improvement replicated in this final follow-up. Parallelism-1 HMR increased 9.4% in confirmation (4.7% to 14.3%), but the primary series was inconclusive (-0.2%, -7.4% to 7.4%); this is not evidence of equivalence.

A fresh comparison against Tokio-based Next.js 22fffea9d4 found the following replicated product results. These include the entire migration: Tokio, PriorityRunner and inline-read execution removal, plus I/O/process/HTTP/N-API changes. They do not isolate goexec.

Product workload	Parallelism	Time change	95% CI
heavy cold build	1	+9.5%	+8.2% to +10.8%
heavy cold build	4	-3.4%	-5.0% to -1.8%
heavy cold build	16	-4.8%	-6.6% to -3.0%
dev first compile	1	+6.4%	+2.4% to +10.5%
dev first compile	16	+2.4%	+0.6% to +4.1%

Small-app, warm build and HMR had no replicated clear timing change. The heavy fixture currently enables Lodash only. Parallelism-16 cold-build peak RSS medians increased from 1563.2 to 1669.2 MiB; sampled peak process-tree thread medians increased from 162 to 188. Sampling requests 10ms intervals and can miss brief peaks or double-count shared pages.

Executor-only spawn/join beats Tokio at parallelism 1 (-10.9%) but remains slower at 4/16 (+11.1%/+22.4%). Yield is faster at all three parallelisms, with Tokio's existing Turbopack LIFO-disabled setting. Blocking 1ms batches remain slower (+827.0%/+267.1%/+72.8%): Tokio immediately delegates to its blocking pool, while goexec keeps its 100µs handoff grace period. At parallelism 1 the sampled peak thread median is 130 for Tokio versus 13 for goexec. These results do not establish general Tokio performance parity.