Repository navigation
Preserve overflow data across rollback and add recovery model tests - #249
Merged
Merged
Conversation
Unify mutation completion and aborted-transaction handling across engine and client entry points. Preserve rollback in every WAL mode and distinguish uncertain commits from ordinary statement failures. Add crash and fsync proofs, explicit correlated bindings, view failure guards, and patched TLS dependencies. Constraint: Preserve existing disk formats and numeric wire error classes. Rejected: Treat a missing COMMIT reply as rollback | the durable outcome can be unknown. Confidence: high Scope-risk: broad Directive: Keep internal BEGIN markers even without a separate fsync; legacy replay accepts boundary-free logs. Tested: Linux instrumented workspace 2682 passed, strict Clippy/rustdoc, TS and Node suites, package smoke, audit/deny/gitleaks, Criterion execution smoke. Not-tested: Actual power loss, full cross-file DDL fault matrix, authoritative Depot performance gate; WAL-Off write overhead is explicitly accepted and documented.
A seeded mutation model exposed two recovery interactions: dropping a retired catalog truncated the replacement writer's WAL, and LSN-skipped overflow writes left reused pages allocatable. Retire the old handle without checkpointing and reserve replayed pages before the physical-write guard. Constraint: Keep existing disk formats, wire behavior, and dependencies unchanged. Rejected: Skip already-durable overflow records entirely | their allocation bookkeeping must still replay. Confidence: high Scope-risk: moderate Directive: A retired catalog must not checkpoint; overflow allocation state precedes the replay LSN guard. Tested: Linux workspace 2685 passed; 18432 modeled mutation steps; both reduced regressions failed before fixes; deliberately disabled rollback was detected; Clippy, rustdoc, format, actionlint, gitleaks. Not-tested: Physical power loss and the complete storage fault matrix; larger corpus uses disclosed tmpfs for logical-state testing.
10 of 11 tasks
Preserve the reviewed correctness fixes while carrying forward current compiler/Miri compatibility and the published-site analytics integration. Constraint: Strict branch protection requires the candidate to include current main Confidence: high Scope-risk: moderate Tested: Previously green independent branches; fresh combined CI follows this merge Not-tested: Fresh merged CI until pushed
Constraint: Validate the stack against current main without rewriting history Confidence: high Scope-risk: moderate Tested: Fresh combined CI will run after push
…ffer Relieve prior committed dirty pages before the next implicit rollback pin. Settle WAL generations before writing heap/header/index state, retain the WAL history, and leave explicit transaction admission unchanged. Constraint: Preserve statement rollback, retained history and deferred durability Rejected: Raise the budget or cap the benchmark fixture | That would hide a real long-lived autocommit failure Confidence: high Scope-risk: moderate Tested: Six query regressions after three failing-first cases; all WAL modes; prior-row preservation; deferred claims and retained history; 16 atomicity/durability tests; 257 storage unit tests; independent code review; fmt/diff checks Not-tested: Fresh authoritative Depot rerun and combined CI pending
…tack Confidence: high Scope-risk: narrow Tested: Six fresh regressions on the integrity base; combined CI follows
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #248. Base:
codex/powdb-integrity-milestone. Review only this incremental diff.Add deterministic mixed-mutation/reopen testing against an independent model. It found two additional recovery defects, fixed with 15 production-code lines:
These fixes should ship with #248 as one release set. Neither PR is merged or released by this change.
Type of change
Behavior change
Repeated overflow-write/rollback sequences preserve committed data and a usable WAL. No new disk format, wire behavior, dependency, or automatic repair.
The normal test suite now exercises four deterministic seeds across three WAL modes and optimized/generic execution. It checks full rows, live counts, real unique/nonunique B-tree entries, dependent views, and graceful reopen. Every trace covers INSERT/UPSERT/UPDATE/DELETE, late uniqueness failures, explicit commit/rollback and aborted transactions. Replay instructions are in
docs/recovery-testing.md.CI also runs for stacked PRs targeting
codex/**. The existing nightly/manual fuzz workflow runs a larger bounded model corpus and retains its operation traces for 14 days.Test plan
Limits
The larger Linux corpus uses
/dev/shmto bound runtime; it proves logical-state/graceful-reopen behavior, not physical-disk or power-loss durability. Existing process-kill and fsync-failure suites remain separate and passed in the workspace run. This is a bounded addition to recovery coverage, not a complete storage/DDL fault matrix or an arbitrary-corrupted-file repair feature. No throughput gain is claimed.Related issues
Depends on #248. After that PR merges, retarget this PR to
mainand rerun required checks before the release.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.