fix(coding-agent): persist kernel stderr to disk and bound the in-memory tail

[snimu]

When a Python kernel failed to start, the failure was often undiagnosable: its stderr lived only in an in-memory string that died with the process, and the startup error showed just the last kilobyte of it. Nothing stayed on disk to inspect after the fact — and on the other end, a long-lived chatty kernel grew that same string without bound for its whole lifetime, even though every reader only ever showed the tail.

The log is startup-shaped because the kernel's stderr is startup-shaped: the runtime dup2's fd 2 into its protocol pipe before it reports ready (repl.py _setup_fds), so the process-level stderr only ever carries pre-ready bytes — interpreter, venv, and import failures; everything after ready already flows through the protocol into the bounded per-execution stderr. The kernel now always spawns with stderr piped, and the manager's data handler is the single write path into kernel-stderr.log in the session's artifact directory: it appends raw chunks under a 5 MiB per-spawn write budget, and once the budget is spent it writes one [stderr log budget exhausted] marker, then keeps draining the pipe but discards — so a kernel that spews until the 30s ready timeout (a corrupted venv looping import warnings, multiplied by repair retries) cannot land more than the budget on disk per spawn, and a full pipe can never wedge a pre-ready kernel. The file rotates to .old when a spawn finds it over the budget, hard-capping per-session disk at roughly 2× the budget.

Startup failure messages read their 1 KiB tail from the in-memory string — bounded to an 8 KiB tail, decoded across chunk boundaries, and fed by the same pipe as the file, so it sees every logged byte plus host-side [kernel] diagnostics. On kernel exit the pipe is drained to quiescence before the stream is torn down, so the kernel's last words reach both the tail and the log even when an orphaned grandchild holds the pipe's write end open past the parent's death. Sessions without an artifact directory (or whose log open fails) keep the memory-tail behavior; the log persists across kernel restarts within the rotation cap.

Validation: three pins — a kernel that writes CR progress output and a split multi-byte character with no trailing newline before dying produces a log byte-identical to what it wrote, with that tail in the startup error; a kernel that spews 6 MiB before dying leaves a log capped at the budget plus the exhaustion marker while the startup error still shows its final words and the script runs to completion (the pipe stayed drained); and a log pre-filled past 5 MiB rotates to .old at the next spawn, with the new incarnation's stderr starting a fresh file. Kernel/ipython/repl suites, tsgo, and root npm run check green.

Linear: RES-1246 https://linear.app/primeintellect/issue/RES-1246/persist-python-kernel-stderr-to-disk-and-bound-the-in-memory-buffer

LOC

Total src: +123/−6 (net +117); tests: +244/−3 (net +241).

Note

Persist kernel stderr to disk with per-spawn budget and bound the in-memory tail

• Adds an optional stderrLogPath to KernelManagerOptions; when set, ReplKernelManager.doStart opens an append log and rotates the prior file when it exceeds the size threshold.
• Bounds the retained in-memory stderr tail to a configured character limit via appendKernelStderrText and appendKernelDiagnostic, replacing unbounded accumulation for the kernel lifetime.
• Writes raw stderr bytes to the log up to a per-spawn budget, emits a one-time exhaustion marker, and continues draining the stream through process exit so final output is not lost.
• IpythonKernelProvisioner.startKernel configures the log path under the session snapshot/artifact directory when it exists.
• shutdown and the pre-ready exit path in waitForReady now wait for stderr stream closure before finalizing, instead of discarding buffered data.
• Risk: ReplKernelManager.shutdown no longer destroys the child stderr stream immediately; any caller relying on synchronous stderr teardown during shutdown may see delayed closure until the exit-drain completes.

Changes since #1947 opened

• Wrapped stderr log file rotation in ReplKernelManager.openStderrLogFd with try/catch error handling that logs a diagnostic message and continues appending to the existing log file when rotation fails, instead of throwing an error [86de757]
• Added 'end' and 'close' event listeners to child.stderr in ReplKernelManager.start to flush buffered partial UTF-8 character data from the TextDecoder and close the stderr log file descriptor [86de757]
• Added a new test validating that stderr logging continues when rotation fails, and updated the existing stderr bytes test to verify partial UTF-8 character handling [86de757]
• Extended the mocked child.stderr type in the configuredManager test utility to include a once method implementation [86de757]
• Reworked stderr stream lifecycle management in ReplKernelManager kernel process exit handler [78a1822]
• Added tests for ReplKernelManager stderr handling edge cases [78a1822]
• Added writeFullySync utility function and replaced direct fs.writeSync calls in ReplKernelManager kernel stderr data handler with writeFullySync to handle partial writes, including writing the budget-exhausted marker as a Buffer [5698258]
• Extended node:fs mock in repl-kernel-startup.test.ts to simulate short writes and updated test assertions to verify byte fidelity under partial write conditions [5698258]
• Modified ReplKernelManager.openStderrLog to return both file descriptor and remaining write budget based on existing log file size, and implemented rotation logic that attempts to move oversized logs to .old before resetting budget [2a056a9]
• Updated ReplKernelManager.start to track stderr write budget from log file initialization and enforce capacity limits by writing a budget-exhausted marker and discarding further pre-ready stderr bytes when exceeded [2a056a9]
• Fixed ReplKernelManager.shutdown to conditionally destroy child stderr stream only when the child process has not exited, preserving buffered output for already-exited processes [2a056a9]
• Added test in repl-kernel-shutdown.test.ts verifying that kill() destroys stderr only when the child process is alive and not when it has an exitCode [2a056a9]
• Updated tests in repl-kernel-startup.test.ts to verify that rotation-failure scenarios respect capacity limits and that write budget equals remaining file capacity at MAX size [2a056a9]

^{Macroscope summarized e9a7d24.}

---

Note

Medium Risk
Changes kernel subprocess stderr lifecycle and synchronous file I/O on every spawn; behavior is heavily tested but affects startup-failure diagnostics and teardown timing.

Overview
Pre-ready Python kernel stderr is now persisted and memory-safe instead of living only in an unbounded in-memory string.

ReplKernelManager accepts optional stderrLogPath and, when set, appends raw pre-ready stderr to kernel-stderr.log with rotation to .old when the file exceeds 5 MiB, a per-spawn write budget based on remaining file capacity, and continued pipe draining after the budget is exhausted (with a one-time marker) so spew cannot wedge startup. The in-memory diagnostics tail is capped at 8 KiB with UTF-8 decoding across chunk boundaries.

Startup failures wait for stderr stream closure before building the error tail; exit/teardown avoids destroying stderr on already-exited children so last bytes are not dropped, while still destroying the pipe for live children that ignore kill.

IpythonKernelProvisioner sets the log path under the session artifact directory when a snapshot dir exists. Tests cover byte fidelity, rotation, budget caps, rotation failure, grandchild stderr noise, and shutdown stderr destroy rules.

^{Reviewed by Cursor Bugbot for commit 2a056a9. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 52ac831. Configure here.}