Repository navigation
1.0.5: wait for the WAL switch on a store being migrated - #21
Merged
Merged
Conversation
Opening the store for writing starts with PRAGMA journal_mode=WAL. On a file not in WAL mode yet SQLite takes the exclusive lock for that switch without calling the busy handler, so with a second writer in the file (the detached maintain) the pragma failed at once with SQLITE_BUSY and the hook exited 0 without migrating. The switch now retries inside the open's own busy budget. This is what failed s10_schema_versions on master's CI.
Owner
Author
|
recheck |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
testjob on master failed on four pushes, each time ins10_schema_versionsof the fault suite. The scenario puts an empty file where the store should be, runs SessionStart and SessionEnd, and expects the store migrated. Depending on the run it found a RED status, "no such table: meta", or a newer schema that was not refused. 1.0.3 fixed the last of the three.The other two come from the first statement a write open runs,
PRAGMA journal_mode=WAL. On a file that is not in WAL mode yet, SQLite needs an exclusive lock for the switch and takes it without calling the busy handler. SessionStart launches a detachedmaintain, and when that process was in the file, the SessionEnd hook got "database is locked" at once, exited 0 and left the store without its schema. The 700 msbusy_timeoutnever applied.Db::open_with_busynow retries the switch for as long as the samebusy_msallows, using the backoff steps the read path already has. Only theReadWriteopen changes, so the read hooks are untouched.writers_racing_to_open_an_empty_store_all_migrate_itinmuninn-coreopens an empty file from four threads at once. Before the change it failed in under 0.1 s on each of 8 runs on Windows. After it, it passed 5 of 5.The
branch-namefailure on #20 is separate: the CLA bot'scla-signaturesbranch does not match the naming rule. This PR does not change that.Checked with the change in place: fmt, clippy and the workspace tests on Windows; in a Linux container, the fault suite at 200 repetitions twice (17 of 17 both times), the boot block budget, and
perf --strictwith the CI scale of 1.5. The fault suite did not fail in that container before the change either (2 runs), so the proof that the race is closed is the new test and the next pushes to master.