Skip to content

Python optimizations - #280

Open
samuelcolvin wants to merge 5 commits into
mainfrom
python-optimizations
Open

Python optimizations#280
samuelcolvin wants to merge 5 commits into
mainfrom
python-optimizations

Conversation

@samuelcolvin

@samuelcolvin samuelcolvin commented Aug 29, 2026

Copy link
Copy Markdown
Member

Summary by cubic

Speeds up Python parsing by locking the string cache once per parse instead of per string, and enables the same release optimizations for wheel builds that the Rust benchmarks already had.

Also makes the Python benchmark script more correct: it now validates each parser's output against json.loads and shows a cross-case summary.

  • The cache lock is acquired with try_lock, so concurrent parses fall back to uncached strings instead of blocking on the mutex.
  • Re-entering the cache from a parse on the same thread panics with a clear message instead of deadlocking; other threads still block.
  • Single-character ASCII strings use CPython's interned singletons, avoiding allocation for each short string.
  • profile.release now uses lto = "fat" and codegen-units = 1 for compiled wheels.
  • Benchmark rows that don't match stdlib output are shown as invalid and excluded from the summary; parsers that raise are shown as errors and not timed.

Written for commit fc620f4. Summary will update on new commits.

Review in cubic

Comment thread crates/jiter-python/bench.py

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread crates/jiter-python/bench.py Outdated
Comment thread crates/jiter/src/py_string_cache.rs
Comment thread crates/jiter-python/bench.py Outdated
Comment thread crates/jiter/src/python.rs
Comment thread Cargo.toml
samuelcolvin and others added 4 commits August 29, 2026 19:56
…uilds

Profiling jiter-python showed ~10% of object-heavy parses spent in
pthread mutex lock/unlock, because the string cache mutex was acquired
once per string. The parser now takes the lock once in `parse()` and
passes the guard through `StringMaybeCache::get_key`/`get_value`.

The lock is acquired with `try_lock` so that on free-threaded builds a
concurrent parse falls back to creating uncached strings rather than
serialising on the mutex.

`maturin develop --release` and the CI wheel builds use `profile.release`,
which had no `lto`/`codegen-units` settings (only `profile.bench` did);
add them so wheels get the same optimisation as the Rust benchmarks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011axe7edsWuf1JqHQS88Npo
`pystring_ascii_new` called `PyUnicode_New` for every ASCII string,
including one-character ones, which the string cache deliberately skips.
CPython keeps interned singletons for all 1-byte ASCII strings and
`PyUnicode_FromStringAndSize` returns them without allocating, which is
what `json`, ujson and orjson all end up doing.

On the `array_short_arrays` benchmark (40,000 one-character strings per
parse) this removes 40,000 allocations and deallocations: 940µs -> 660µs,
taking jiter from slowest to fastest of the parsers benchmarked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011axe7edsWuf1JqHQS88Npo
Ranks packages by the geometric mean of their per-case slowdown relative
to the fastest package, and shows the worst case and number of cases won.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011axe7edsWuf1JqHQS88Npo
Each parser's output is compared against the stdlib result before timing;
mismatches are shown as "invalid" instead of a slowdown and excluded from
the summary. orjson returns lossy floats for the massive_ints_array case,
so its 4x lead there was never a like-for-like comparison.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011axe7edsWuf1JqHQS88Npo
@codecov

codecov Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.42105% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/jiter/src/py_string_cache.rs 93.05% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

Comment thread crates/jiter/src/py_string_cache.rs
@codspeed-hq

codspeed-hq Bot commented Aug 29, 2026

Copy link
Copy Markdown

Merging this PR will regress 3 benchmarks

⚡ 7 improved benchmarks
❌ 3 regressed benchmarks
✅ 51 untouched benchmarks
⏩ 71 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime python_parse_x100 137 ns 198 ns -30.81%
WallTime python_parse_other 369 ns 429 ns -13.99%
WallTime python_parse_x100_not_cached 132 ns 145 ns -8.97%
WallTime python_parse_string_array 13.3 µs 10.4 µs +28.09%
WallTime python_parse_string_array_unique 1.6 ms 1.3 ms +26.7%
WallTime python_parse_medium_response 25.4 µs 22 µs +15.82%
WallTime python_parse_true_object 26.3 µs 22.9 µs +15.11%
WallTime short_numbers_jiter_value 57.1 µs 51.5 µs +10.86%
WallTime string_array_jiter_value_owned 17.5 µs 16 µs +9.94%
WallTime python_parse_numeric 1.3 µs 1.3 µs +6.5%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing python-optimizations (fc620f4) with main (d2e138b)

Open in CodSpeed

Footnotes

  1. 71 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Holding the cache lock for the whole parse meant a Python callback that
runs while the parser allocates (GC callbacks and `__del__` on CPython
<= 3.11, the `decimal` import in decimal mode) would deadlock if it
called `jiter.cache_clear()` or `cache_usage()`. A thread-local flag now
records that the lock is held by the current thread, and those functions
panic with a clear message instead of blocking on their own lock;
other threads still block as before.

Also make the benchmark script tolerate a parser raising on a case
(shown as "error", not timed) and a parser with no valid cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011axe7edsWuf1JqHQS88Npo

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 5 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread crates/jiter/src/py_string_cache.rs
Comment thread crates/jiter-python/bench.py
Comment thread crates/jiter-python/bench.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant