Fold first sumfold - #227
Draft
wu-s-john wants to merge 110 commits into
Draft
Conversation
Use borrowed/in-place field addition while accumulating rotated and shifted binary polynomial evaluations. These paths run once per relevant set bit across each virtual bit-op or shifted bit-slice column, so avoiding clones removes a large amount of temporary field-element copying without changing the immediate-reduction semantics.
Add a narrow delayed modular reduction path for 4-limb Montgomery fields and use it in the hot binary polynomial evaluation paths. The new `zinc_utils::delayed_reduction` module introduces: - `MontgomeryLimbs` for exposing reduced Montgomery-form field limbs. - `DelayedModularReduction` for sum-only delayed accumulation. - `BarrettReductionParams` with const `mu` computation. - A `Uint<5>` accumulator implementation for summing 4-limb field elements. - An optimized `barrett_reduce_5` path for reducing bounded 5-limb sums. - Implementations for both `MontyField<4>` and `ConstMontyField<_, 4>`. This lets binary polynomial evaluation accumulate many selected `eq(r, b)` values as raw Montgomery limbs, then perform one Barrett reduction per output coefficient instead of doing a field reduction after every conditional add. Apply the accumulator to two hot paths: - Lifted binary polynomial evaluation in the protocol layer. - Streaming shifted bit-slice evaluation in the PIOP booleanity code. The lifted binary evaluation now builds the `eq(point, *)` table once, scans the binary trace rows, conditionally adds `eq_b` into per-bit `Uint<5>` accumulators, and reduces once per bit coefficient. The shifted bit-slice streaming path uses the same delayed accumulation strategy while continuing to avoid materializing shifted bit-slice MLE buffers. Use `crypto_bigint::Uint<5>` directly as the accumulator rather than a custom wide-limb wrapper, keeping the representation aligned with the rest of the integer code. The Barrett reducer is specialized to the actual accumulator width, avoiding the unused sixth limb from the earlier 6-limb reducer shape. Also extend the relevant protocol prover/verifier bounds so the optimized paths can access Montgomery limbs, and generalize `ConstMontyField` projection support through `FromRef`.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Univariate-skip booleanity zerocheck over the instance axis: symmetric integer packing domain with Lagrange/barycentric helpers, the (G, h) Gram sufficient statistic (per-point interpolation work eliminated), and the evaluation-basis skip-round message with the gamma-weighted zerocheck, alpha challenge, fold weights, and folded booleanity residue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extract fold_projected_traces_with_weights so the fold-first path can fold with Lagrange weights, and add the fold-commutes identity test: the skip-round endpoint q(alpha) equals the booleanity residue of the theta-folded trace. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
prove/verify_fold_first_sha_sumfold run the full V2 transcript: r_ic, rho, gamma, booleanity skip round, Lagrange-weight fold, folded ideal polynomials with membership checked before sampling a, single post-fold scalarization, and the folded row sumcheck against the assembled target T' = sum lambda^f E'_f(a) + xi*B_star. Integration tests cover the zero-trace roundtrip, the algebraic completeness identity with the zero-family correction term, and tamper rejections (ideal membership, gamma zero-check, forged ideal polys). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add prove_prepared_fold_first_mixed_hyrax / verify_fold_first_linear_ ideal_fold_mixed_hyrax: booleanity skip round, Lagrange-weight fold, folded IdealCheck before sampling a, assembled row-sumcheck target, and the shared endpoint/opening tail (verify phases refactored to take explicit fields so V1 and V2 share them). Add the fold_first_full_block bench to piop neutron_nova_sumfold and fold-first rows to the combined SHA-256 instance sweep (SHA256_COMBINED_SWEEP_INCLUDE_FOLD_FIRST). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mask-packed fast path (O-10/O-12/O-15): one u128 instance mask per (source, row) with exact bitwise validity recipes for the virtual Ch/Maj expressions, per-item general-path fallback for non-binary values, and rayon row-sharded thread-local accumulators. Batched Montgomery inversion for the skip-domain Lagrange denominators (O-6) and a fold-first witness prepare that skips the V1 SumFold basis and the power-of-two requirement (O-19). Reference pass retained and differentially tested against the fast path on honest, non-binary, and invalid-virtual-relation inputs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mask construction shards over source slots with an is_zero fast test (avoids full PartialEq parameter compares); the Gram pair loop adds by reference. Reference-vs-fast differential tests unchanged and green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The fold stage (commit-and-absorb, skip round, masked Lagrange fold, folded IdealCheck, assembled target) produces a FoldFirstShaFoldOutput accumulator; the decider (row sumcheck, endpoints, PCS opening) runs later on the same transcript via decide_fold_first_mixed_hyrax. The monolithic prover delegates to both, so existing callers and the verifier are unchanged. Gram and theta-fold hot paths accumulate in unreduced 5-limb buckets honoring the reducer flush contract, with the theta-fold reusing the Gram instance masks (build-once). The combined sweep reports commit / folding-core / fold-stage / decider columns for the fold-first rows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The C' homomorphic fold is EC work the folding step never consumes; absorbing it at the start of the decider keeps the transcript sequence identical while removing ~3ms from the recurring fold stage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
build_fold_first_ideal_polys accumulates the nonzero-family coefficient vectors over row chunks with per-thread buffers (differentially tested against build_sha_ideal_values_at_point). The sweep probe now reports the whole fresh-instances phase (MSMs plus instance assembly) as the commit lane and takes best-of-three stage timings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ShaBooleanityCatalog selects which word columns get direct per-bit booleanity checks: Full (640 sources), Tier1DropChMajAux (480, drops the Ch/Maj aux columns), Tier2DropXorResults (352, also drops the XOR-result sigma columns pinned pointwise by R0..R3). The virtual Ch/Maj families and the A/E/W/MuPacked/overflow columns are never droppable. The reduction lemma, per-tier case analysis, and case-to-test table live in documentation/fold-first-sumfold-doc/booleanity-reduction-soundness.md. Covered by catalog layout unit tests, per-tier masked-vs-reference Gram differentials, dropped-column fallback checks, reduced-catalog prove and verify roundtrips, and a tamper matrix rejecting boolean, non-binary, and half-integer corruption in every dropped column. The fold-first bench gains one variant per tier. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shrink the mixed-Hyrax committed witness from 17 to 13 word columns: the four XOR-result sigma columns are reconstructed as rot-sum(parent) minus twice the overflow column, like the virtual Ch/Maj values. The prover trace still materializes them (the row-expression layout keeps its own extended source list), while the commitment, PCS lifted evals, multipoint layout, resolver, and the verifier endpoint terminal work from the committed set; derived_sigma_endpoint_word_poly rebuilds sigma at every consumed row shift from parent openings. The endpoint source count is unchanged (sigma sources out, parent shifts in), and deriving sigma closes the X^32-1 rotation freedom the Tier-2 catalog left to the ideal checks. Thread ShaBooleanityCatalog through the V1 and V2 prove/verify entry points with a transcript absorb ahead of the rho/xi challenges; V2 defaults to Tier 2 in the sweep, V1 stays on Full. Add a per-raw-phase timing layer and a SHA256_COMBINED_SWEEP_PROFILE_ONCE probe that prints every span plus commit/fold-core/decide roll-ups. Measured at N=32 (Tier 2, 3-sample medians, same machine): prove 46.7 to 38.4 ms, verify 37.8 to 30.3 ms, proof 660k to 525k raw bytes; commit MSM span 11.45 to 9.65 ms. Covered by a derived-vs-trace endpoint differential, overflow and trace-sigma tamper rejection, all-tier V2 and V1-mixed roundtrips at batch size 13, and untouched packed and generic path suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a fold-first SumFold path for batched SHA proving.
Instead of running IdealCheck and scalarization independently before folding, the new V2 flow:
The construction supports arbitrary batch sizes rather than requiring the number of instances to be a power of two.
Main changes
u128instance masks for real and virtual SHA booleanity sources.Booleanity catalogs
Add three transcript-bound SHA booleanity catalogs:
Full: 640 sources.Tier1DropChMajAux: 480 sources.Tier2DropXorResults: 352 sources.The reduced tiers remove direct checks whose values are constrained by surviving virtual booleanity checks, ideal membership, or committed parent columns.
The Tier 2 path also derives the four sigma result columns from their committed parent and overflow columns. This reduces the mixed-Hyrax committed witness from 17 to 13 word columns while closing the rotation freedom that would otherwise remain after dropping the sigma booleanity checks.
Performance
Measured at
N=32using Tier 2 with three-sample medians on the same machine:The benchmark output now separates commit, fold-core, fold-stage, and decider timings and supports profiling each booleanity tier.
Test coverage
Coverage includes:
q(α)and the folded booleanity residue.Documentation
This PR adds: