refactor: concentrate bulk-decode unsafe in a Utf8Writer - #37
Merged
Conversation
Replace the raw `&mut *mut u8` cursor threaded through the decode helpers with a `Utf8Writer` that owns the overshoot invariant behind `push_entry`/`push_ascii_word`/`finish`, leaving the decode functions as safe code. `Utf8Writer::for_input_len` computes the `len * 3 + 1` output capacity so callers can't pass a wrong size, and each write `debug_assert`s its slack, so Miri and the fuzzers trip on an overrun at the write site. Drop the `transmute::<&usize, &[u8; 8]>` in the word-at-a-time path for a `to_ne_bytes` memory-order passthrough (no unsafe, `to_le`/`to_be` would break big-endian). No behavior or codegen change: the hot-loop assembly is instruction- identical to master, confirmed by the differential fuzzers and Miri under Stacked + Tree Borrows on little- and big-endian (s390x). Add `tests/writer_stress.rs`, a bounded Miri-runnable differential sweep of the writer, and a `miri` CI job (via a `.#miri` nightly dev shell) gating it.
bonega
enabled auto-merge (squash)
July 4, 2026 10:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wraps the bulk decoder's hand-written
unsafein aUtf8Writerso the decode helpers become safe code and the overshoot invariant lives in one place.Changes
Utf8Writerowns the write cursor behind three primitives —push_entry(the branchless 4-byte overshoot store),push_ascii_word(word-at-a-time ASCII copy),finish. Each documents the slack the caller must guarantee.for_input_lencomputes thelen * 3 + 1output capacity, so a wrong size is unrepresentable.transmutein the aligned-ASCII path forto_ne_bytes(memory-order passthrough;to_le/to_bewould break big-endian).Not changed
Behavior and codegen. The hot-loop assembly is instruction-identical to master.
Verification
decode_byte,invariant,validate) — no findings.s390x.tests/writer_stress.rs: a bounded, deterministic differential sweep of the writer (prefix/aligned/suffix + exact-capacity tail), cheap enough for Miri — libFuzzer targets can't run under Miri, so this is what exercises theunsafethere.miriCI job (.#mirinightly dev shell inflake.nix) gates the above on every PR.