Skip to content

[Wasm] Use membarrier(2) to publish modified code on FreeBSD - #769

Open
samm-git wants to merge 1 commit into
oven-sh:mainfrom
samm-git:wtf-icache-barrier-nonparking
Open

samm-git wants to merge 1 commit into
oven-sh:mainfrom
samm-git:wtf-icache-barrier-nonparking

Conversation

@samm-git

@samm-git samm-git commented Oct 5, 2026 •

Copy link
Copy Markdown

[Wasm] Use membarrier(2) to publish modified code on FreeBSD

Problem

A bun build --compile executable deadlocks on FreeBSD/aarch64 (oven-sh/bun#44572).
The freeze is in the Wasm instruction-cache publication path:

  • Wasm::barrierInstructionCacheOnAllThreads() made every tracked thread re-fetch
    modified instructions through Thread::barrierInstructionCache(), which published by
    suspend()+resume()-ing the target. That takes the global ThreadSuspendLocker
    and parks the target.
  • The GC's stop-the-world path (MachineThreads::tryCopyOtherThreadStacks) and a
    Worker VM's JITWorklist::suspendAllThreads() both want the same global lock, while
    the JIT-worklist / Wasm-compiler threads sit parked on their worklists.

The result is a two-GC plus JIT-worklist stop-the-world inversion, with the Wasm
publisher holding the suspend lock. Symbolized lldb captures are in oven-sh/bun#44572.

Every signal-based variant we tried was either still deadlock-prone (holding
ThreadSuspendLocker) or lost its acknowledgment (sharing sigThreadSuspendResume
with suspend/resume, or dropping the Wasm thread-list lock and racing Thread::didExit()).

Fix

Implement the FIXME that was already here: on FreeBSD, ask the kernel to run a
context-synchronizing event (an ISB on ARM64) on every running thread via
MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE. FreeBSD implements membarrier(2) as a
clean-room port of the Linux interface, and this is the mechanism SpiderMonkey uses for
the same purpose.

This publishes the modified code with:

  • no signal handler, so nothing to interfere with the suspend/resume protocol;
  • no ThreadSuspendLocker, so it cannot participate in the GC/JITWorklist inversion;
  • no per-thread handshake that could be lost.

Registration is process-wide and done once (std::call_once). Kernels without the
syscall (or non-FreeBSD) keep the existing per-thread path as a fallback.

Testing

  • WebKit commit bun 1.4.2 pins (2e2aa2290f) on FreeBSD 15.1-RELEASE aarch64, with the
    compiled OpenCode TUI and libopentui via bun:ffi. With the equivalent change, the
    repro goes from 5/5 freezes to repeated passes: three consecutive turns all
    complete (deterministic, zero-latency mock model), three real-model conversations pass,
    and the process survives an 8× SIGUSR1 heap-snapshot-GC storm.
  • dtrace confirms the syscall (membarrier) is what now publishes the code during
    Wasm compilation.
  • This commit is the same change rebased onto main, compile-checked with the
    FreeBSD/aarch64 cross build. (Runtime validation on main directly is not possible
    yet because bun 1.4.2's src/jsc/bindings does not compile against main's headers.)

Not in this PR

There is a separate, architecture-independent FreeBSD deadlock at process exit
(after a successful conversation), in the WTF thread-suspension handshake
(Thread::suspend ↔ tryCopyOtherThreadStacks ↔ VMTraps::SignalSender). It reproduces
on amd64 as well as aarch64 and is unrelated to this change; it is being tracked
separately and will be filed on its own.

Refs: oven-sh/bun#44572.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used
📚 Code guidelines (2)
CLAUDE.md — auto-discovered
Source/JavaScriptCore/CLAUDE.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 972ec942-a89d-4cf9-87a9-d9ac24ccf174
📥 Commits

Reviewing files that changed from the base of the PR and between 19e72b4 and 1986b99.

📒 Files selected for processing (1)
  • Source/JavaScriptCore/wasm/WasmMachineThreads.cpp

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.


Walkthrough

On non-X86_64 FreeBSD, the WebAssembly instruction-cache barrier attempts private expedited sync-core membarrier. If registration or execution fails, it uses the existing locked per-thread barrier iteration. X86_64 retains its immediate return.

Changes

Instruction-cache barrier

Layer / File(s) Summary
FreeBSD membarrier with thread fallback
Source/JavaScriptCore/wasm/WasmMachineThreads.cpp
The file adds the FreeBSD membarrier header and attempts private expedited sync-core membarrier on non-X86_64. If registration or execution fails, the existing locked per-thread barrier iteration runs.

Priority: ⬆️ High

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: using membarrier to publish modified Wasm code on FreeBSD.
Description check ✅ Passed The description explains the bug, the fix, and the testing. It references the related Bun issue, but it does not include a WebKit Bugzilla bug link, a “Reviewed by” line, or the changed-file details f…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Git: CodeRabbit could not clone the repository, so clone-backed analysis was skipped and this review may be incomplete. Verify repository clone access, such as SSH credentials, before requesting another full review. If clone access is intentionally unavailable, use path_filters to narrow the review scope.


Comment @coderabbitai help to get the list of available commands.

@samm-git
samm-git marked this pull request as draft October 5, 2026 07:37
barrierInstructionCacheOnAllThreads() published modified Wasm code by suspend()ing and
resume()ing each tracked thread through Thread::barrierInstructionCache(). That takes
the global ThreadSuspendLocker and parks the target, which deadlocks against the GC's
stop-the-world path on FreeBSD (a two-GC plus JIT-worklist inversion; see
oven-sh/bun#44572).

FreeBSD implements membarrier(2) as a clean-room port of the Linux interface.
MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE asks the kernel to run a
context-synchronizing event (an ISB on ARM64) on every running thread in the process,
so the modified code is published without a signal, without ThreadSuspendLocker and
without any per-thread handshake. This is the mechanism the FIXME here already asked
for.

Registration is process-wide and done once; kernels without the syscall (or non-FreeBSD)
keep the existing path as a fallback.

Refs: oven-sh/bun#44572.
@samm-git
samm-git force-pushed the wtf-icache-barrier-nonparking branch from 19e72b4 to 1986b99 Compare October 5, 2026 10:56
@samm-git samm-git changed the title [WTF] Make Thread::barrierInstructionCache stop parking the target [Wasm] Use membarrier(2) to publish modified code on FreeBSD Oct 5, 2026
@samm-git
samm-git marked this pull request as ready for review October 5, 2026 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant