perf(translate,turso): borrowed-key Arc cache + opt-in Turso handle reuse - #35
Merged
Merged
Conversation
…euse
TranslateCache: key each dialect's LRU by a borrowed &str and hand back a
shared Arc<[TranslateResult]> instead of allocating a (Dialect, String) key
per lookup and deep-cloning a Vec on every hit. One lock per dialect; no
sharding — sharding measured a net loss: an LRU get is a write (it promotes
to MRU), so concurrent hot-key hits contend on that entry's list links no
matter how many locks guard the map, while the second hash needed to pick a
shard costs more than the lookup it parallelises.
hot single-key hit, release, median of 9 x 2M iters:
before (owned String key + Vec clone) 69.8 ns/op
after (borrowed &str key + Arc clone) 23.8 ns/op 2.93x
litewire-turso: opt-in idle-connection reuse (TursoBuilder::handle_reuse).
A clean connection is parked on drop and checked back out by a later session,
keeping the engine's per-connection prepared-statement cache warm — a fresh
connect() opens with an empty cache and the first prepare of each statement
is the dominant per-session cost. Hygiene is cheap per-statement
dirty-tracking (literal PRAGMA writes, CREATE TEMP, ATTACH/DETACH — a wire
client's SET translates to a no-op, so those are the only vectors) that gates
*parking*; checkout stays free. last_insert_rowid is snapshot at checkout so
a reused handle never reports the prior session's insert id (the engine
exposes no setter).
An earlier design that validated a PRAGMA-readback fingerprint at checkout
measured 0.99x — the probe cost as much as the statement-cache win it
protected — which drove the move to park-time dirty-tracking:
file-backed session cycle (connect + 5 first-queries + drop), median:
reuse OFF 154.9 us reuse ON 53.0 us 2.92x
MySQL wire session (TCP + handshake + 5 SELECTs + close), end-to-end:
reuse OFF 1460 us reuse ON 1220 us 1.20x
Reuse is disabled by default and ignored when enable_cdc_on_connect is set.
Adds ReuseStats + Turso::reuse_stats() for observability, and reset-behaviour
tests for the temp-object, pragma-write, open-transaction and
last_insert_rowid leak vectors.
luthermonson
added a commit
that referenced
this pull request
Sep 1, 2026
…euse (#35) TranslateCache: key each dialect's LRU by a borrowed &str and hand back a shared Arc<[TranslateResult]> instead of allocating a (Dialect, String) key per lookup and deep-cloning a Vec on every hit. One lock per dialect; no sharding — sharding measured a net loss: an LRU get is a write (it promotes to MRU), so concurrent hot-key hits contend on that entry's list links no matter how many locks guard the map, while the second hash needed to pick a shard costs more than the lookup it parallelises. hot single-key hit, release, median of 9 x 2M iters: before (owned String key + Vec clone) 69.8 ns/op after (borrowed &str key + Arc clone) 23.8 ns/op 2.93x litewire-turso: opt-in idle-connection reuse (TursoBuilder::handle_reuse). A clean connection is parked on drop and checked back out by a later session, keeping the engine's per-connection prepared-statement cache warm — a fresh connect() opens with an empty cache and the first prepare of each statement is the dominant per-session cost. Hygiene is cheap per-statement dirty-tracking (literal PRAGMA writes, CREATE TEMP, ATTACH/DETACH — a wire client's SET translates to a no-op, so those are the only vectors) that gates *parking*; checkout stays free. last_insert_rowid is snapshot at checkout so a reused handle never reports the prior session's insert id (the engine exposes no setter). An earlier design that validated a PRAGMA-readback fingerprint at checkout measured 0.99x — the probe cost as much as the statement-cache win it protected — which drove the move to park-time dirty-tracking: file-backed session cycle (connect + 5 first-queries + drop), median: reuse OFF 154.9 us reuse ON 53.0 us 2.92x MySQL wire session (TCP + handshake + 5 SELECTs + close), end-to-end: reuse OFF 1460 us reuse ON 1220 us 1.20x Reuse is disabled by default and ignored when enable_cdc_on_connect is set. Adds ReuseStats + Turso::reuse_stats() for observability, and reset-behaviour tests for the temp-object, pragma-write, open-transaction and last_insert_rowid leak vectors.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two independent perf changes to the translate cache and the experimental Turso backend.
1.
TranslateCache: borrowed-key + sharedArcEach dialect gets its own LRU keyed by a borrowed
&str(Box<str>keys,Box<str>: Borrow<str>), returning a sharedArc<[TranslateResult]>. This removes, on every cache hit:(Dialect, String)key allocation that a tuple key forced on each lookup, andVec<TranslateResult>clone handed to the caller.Callers (
litewire-session,litewire-postgres) now iterate the shared slice by reference;translate_cachedreturns theArc.Sharding was measured and dropped. An LRU
getis a write (it promotes to MRU), so concurrent hits on a hot statement contend on that entry's list links regardless of how many locks guard the map — aggregate throughput did not improve — while the extra hash to select a shard cost more than the lookup, and dividing a fixed capacity across shards turned a skewed key distribution into thrashing.Stringkey +Vecclone&strkey +Arcclone2.93×.
2.
litewire-turso: opt-in idle-connection reuseTursoBuilder::handle_reuse(max_idle)(off by default). A clean connection is parked on drop and checked back out by a later session, keeping the engine's per-connection prepared-statement cache warm. A freshDatabase::connect()opens with an empty statement cache, and the firstprepareof each distinct statement on a cold connection is the dominant per-session cost — the whole prize is amortising it.Hygiene is cheap per-statement dirty-tracking that gates parking, not an expensive checkout probe:
SET ...translates to a no-op, so the only way to change connection-scoped pragma state is a literalPRAGMA name = value— caught generically.CREATE TEMP/TEMPORARY(asqlite_temp_masterobject) andATTACH/DETACHare the remaining vectors.is_autocommit, sync);last_insert_rowidis snapshot at checkout so a reused handle never reports the prior session's insert id (the engine has no setter).Checkout is a free autocommit re-confirm — no queries.
2.92×.
mysql_async, median)1.20× — reuse saves ~240 µs/session; the absolute is dominated by the per-session TCP + MySQL handshake, which reuse does not touch.
Adds
ReuseStats+Turso::reuse_stats(). Reuse is ignored (with awarn!) whenenable_cdc_on_connectis set.Tests
Reset-behaviour tests for every leak vector: temp objects, pragma writes, open transactions,
last_insert_rowid, plus staleness age-out, pool-capacity bound, and reuse-disabled-by-default.cargo test --workspace --all-featuresgreen (956); clippy-D warningsclean on default and--all-features; nightlyfmtclean.Notes
Cargo.tomlchanges. Turso backend remains experimental.