Repository navigation
test: run the Python tests in parallel with pytest-xdist - #223
Merged
Merged
Conversation
CI's pytest steps, make test and make test-fast run with -n auto --dist worksteal (one worker per CPU, each with its own MiniStack); a plain uv run pytest stays serial for debugging. CI also prints the 15 slowest tests. The emulator fixture now waits for MiniStack after killing it. Co-authored-by: Carl Oscar Aaro <hello@carloscar.com>
kalaspuff
deleted the
fctl/332-run-the-python-tests-in-parallel-if-poss
branch
November 1, 2026 02:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Python suites now run in parallel with pytest-xdist (
-n auto --dist worksteal, one worker per CPU) in every CI pytest step and inmake test/make test-fast, as fanferro does since its #249; a plainuv run pyteststays serial so a single test is still easy to debug. Each worker starts its own MiniStack, which is safe because every test already uses unique names and emulator clients get a per-test table prefix: nothing in the suite needed changing. CI also prints the 15 slowest tests, and the emulator fixture now waits for MiniStack after killing it (fanferro hit aResourceWarningthere under load).The gain is smaller than fanferro's, since dynamoxy's suite is already fast: in the 4-CPU sandbox the full suite (989 tests, with MiniStack) went from 11 s serial to about 6 s on 4 workers, with or without coverage. On CI the pytest step went from 17β28 s to 11β18 s on Linux, 10β16 s to 8β11 s on macOS and 19β24 s to 14β18 s on Windows, and the uvloop step from 7 s to 3 s; the jobs' totals are dominated by
uv syncbuilding the extension. I ran the suite three times on 4 workers with coverage, once on 8 workers, once on uvloop and once without the emulator, and every run passed.Worth confirming
fctl session
Note
Low risk. It only changes how the test suite is run (a dev dependency, CI steps,
maketargets and docs) and adds await()after killing the test emulator; no library code changes.What it does. Runs the Python tests in parallel with pytest-xdist (
-n auto --dist worksteal) in every CI pytest step and inmake testandmake test-fast, while a plainuv run pyteststays serial. Each worker starts its own MiniStack, which the existing unique resource names and per-test table prefixes already allow. CI also prints the 15 slowest tests, and the emulator fixture now waits for MiniStack after killing it so noResourceWarningis left behind. AGENTS.md and the two skills describe the new commands and the rule that tests must not depend on each other or on tight wall-clock bounds.Parallel runs everywhere. The macOS and Windows jobs also run with
-n auto, which spawns worker processes; the description asks maintainers to confirm this, and it also has aCI_RESULTplaceholder where the CI timings should be.fctl review's first look at
d24f525Β· 8 files, +44 β11 Β· Review agent settings