Skip to content

test(corral): spawn the duckdb reopen child with posix_spawn, not fork - #255

Merged
e-lo merged 1 commit into
mainfrom
fix/corral-reopen-test-posix-spawn
Oct 25, 2026
Merged

e-lo merged 1 commit into
mainfrom
fix/corral-reopen-test-posix-spawn

Conversation

@e-lo

@e-lo e-lo commented Oct 23, 2026

Copy link
Copy Markdown
Owner

Summary

test_write_duckdb_while_a_reader_holds_the_file (added in #251) failed intermittently under pytest -n auto on macOS. The child that reopens the DuckDB file exited with -11 and produced no stdout or stderr.

Cause. The failure had nothing to do with DuckDB locking or a shared instance. Every macOS crash report shows the same stack:

subprocess_fork_exec → fork → _pthread_atfork_child_handlers
  → PROJ SQLiteHandleCache::getHandle atfork lambda → lru cache clear
  → sqlite3Close → unixClose → verifyDbFile → sqlite3_log → os_log → SIGSEGV

The forked copy of the xdist worker crashes before exec. pyproj/PROJ is loaded by the geo tests that run on the same worker, and it registers an atfork-child handler that closes its cached SQLite handles. On macOS, that close path calls os_log, which is not safe to call after fork. subprocess.run with the default close_fds=True forks on macOS because the platform has no POSIX_SPAWN_CLOSEFROM.

Fix. Pass close_fds=False (the executable, sys.executable, is already an absolute path) so subprocess uses posix_spawn, which runs no atfork handlers. This is the same workaround the run_node fixture, the project-cards harness and the documented-CLI contract tests already use. The change is test-only.

Verification

Full suite, uv run --all-extras pytest packages plugins -n auto -q -m "", macOS, Python 3.13.3:

  • Before: 3 of 8 runs failed on this test (-11, empty output). All 3 have a matching crash report with the stack above.
  • After: 20 of 20 runs passed (4308 passed, 6 skipped each), with no new crash reports.
  • Checked that the call now goes through os.posix_spawn (the default call went through fork). ruff check and ruff format --check are clean.

🤖 Generated with Claude Code

test_write_duckdb_while_a_reader_holds_the_file segfaulted (-11, no output)
in ~40% of full parallel runs on macOS. Crash reports show the forked copy
of the xdist worker dying before exec: fork runs PROJ's atfork-child
handler (pyproj is loaded by the geo tests on the same worker), which
closes cached SQLite handles and hits os_log, unsafe after fork on macOS.

close_fds=False with the absolute sys.executable lets subprocess use
posix_spawn, which runs no atfork handlers -- the same workaround as the
node and documented-CLI tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the test label Oct 23, 2026
@e-lo
e-lo merged commit 20e5fdb into main Oct 25, 2026
8 checks passed
@e-lo
e-lo deleted the fix/corral-reopen-test-posix-spawn branch October 25, 2026 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant