fix(core): eliminate Windows fork bomb caused by recursive shim execution

[chawyehsu]

This patch was submitted to upstream in PR volta-cli#1761 and had received code reviews, but obsoleted because of upstream becoming unmaintained. It was also applied in https://github.com/chawyehsu/volta/tree/staging

Details

Background

This PR was initially created to address a fork bomb issue observed on Windows. The root cause of this issue consists of two parts.

First, the existing RECURSION_ENV_VAR is always set to 1 and never incremented when volta/volta-shim spawning child processes, which makes a fork bomb possible. This environment variable should instead be incremented on each recursion and guarded by an upper limit. Once the limit is reached, the process should return an error and exit. For example, rustup implements a recursion guard with a limit of 20¹.

Second, based on the first condition, a fork bomb could theoretically occur on all operating systems supported by Volta, yet in practice the issue only manifests on Windows. This discrepancy comes from a flaw in Volta’s create_command function used to spawn child processes.

Child processes are created by passing the executable name into create_command. After Command is constructed, the Volta platform bin directory is appended to the PATH environment variable in the execution context. Differences in the implementation behavior of create_command lead to different outcomes across platforms.

On Unix systems, the system call resolves the actual target executable through PATH and executes it as expected. On Windows, however, executable lookup order may behave less intuitively. When the executable path is relative rather than absolute, Windows searches in the following order: current working directory (cwd), then PATH.

Because Command::new("cmd.exe") is used, even though Rust’s standard library resolves the executable from PATH, it only resolves cmd.exe itself. After the cmd.exe process is created, the system will prioritize searching for the actual target executable in the cwd of cmd.exe (which is also the directory containing the parent Volta shim). As a result, resolution points back to the Volta shim itself, ultimately triggering a fork bomb.

Why Command::new("cmd.exe") / cmd.exe /C

In the PR I submitted to the upstream repository, volta-cli#1761 (comment) @charlespierce explained in detail why Volta originally adopted cmd.exe /C: Rust's Command in std library does not reliably resolve .bat and .cmd files, which are created and used by the Node.js runtime.

The Fix

In the PR I submitted to the upstream repository, I introduced which-rs to resolve and obtain the full path of the target executable before Command is finally executed. A new Command instance is reconstructed using the resolved absolute path and executed accordingly. With this approach, I did not modify the existing create_command implementation that relies on cmd.exe /C.

That was because Volta’s Executors (such as ToolCommand) invoke create_command during ::new(), completing Command construction at executor initialization time. Without refactoring them, it is impossible to avoid a second Command::new invocation. To address this, I separated the related refactoring work into #44 for easier historical tracking.

By leveraging the third-party which-rs library, we are able to reliably resolve .bat and .cmd files, making it possible to remove the dependency on cmd.exe /C. which-rs resolves target executables using the system PATHEXT environment variable.

With the introduction of a recursion limit, together with the Executor refactor and the adoption of which-rs, the fork bomb issue is ultimately eliminated. At the same time, removing cmd.exe /C also resolves a series of other issues caused by its usage.

Fixes

• Fix Potential fork bomb volta-cli/volta#1741 (this is the original issue the PR targetting to fix)
• Most issues related to cmd.exe /C wrapper on Windows could be fixed
  • (tested) This will also fix Newlines in arguments passed to node -e ... volta-cli/volta#1791
  • (tested) Fix Windows: quoted arguments containing & are split when invoking node through Volta shim volta-cli/volta#2105
  • (untested) Windows: "'node' is not recognized as an internal or external command, operable program or batch file." in some postinstall scripts volta-cli/volta#1870
  • Close Running 'node' on Windows can show unhelpful error volta-cli/volta#577

Fun thing

• https://x.com/chawyehsu/status/1784577035064791349

Footnotes

1. https://github.com/rust-lang/rustup/blob/318c5b7072de9e33d4608213aae7cecdc99e178b/src/env_var.rs#L8 ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.53%. Comparing base (3dc4534) to head (d85f204).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #41      +/-   ##
==========================================
+ Coverage   80.44%   80.53%   +0.08%     
==========================================
  Files         105      105              
  Lines       10567    10629      +62     
==========================================
+ Hits         8501     8560      +59     
- Misses       2066     2069       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.