fix(rotortree): cut some cruft, improve some perf

[rymnc]

• bulk

  • n2: −71.5%
  • n4: −62.7%
  • n8: −53.5%
  • n16: −54.1%

• Instructions: −76.8%

  • cycles: −37.1%
  • L1D_store_miss: −92.2%

This pull request significantly refactors the ChunkedLevel implementation in crates/rotortree/src/chunked_level.rs to simplify chunk management, improve allocation efficiency, and remove the separate "tail" buffer concept. The changes unify chunk storage, streamline chunk extension and writing, and update related APIs. The checkpoint storage code is also updated to remove now-unnecessary tail handling.

Core data structure and API refactor:

• The ChunkedLevel struct no longer maintains a separate fixed-size "tail" buffer for partial chunks; all hashes are managed uniformly in chunks, simplifying the logic and reducing edge cases. [1] [2] [3] [4]
• Chunk extension and writing logic is unified: new methods (extend_with, write_with) allow efficient in-place initialization of chunk memory, and all appends/grows are handled through these, removing special-case tail logic.
• The chunk access API is refactored: chunk and slice access is now handled through chunk and chunk_slice methods, and iteration over runs of contiguous hashes is provided by the new runs method. [1] [2]

Memory allocation and initialization improvements:

• New allocation helpers are introduced (from_fn, new_arc_from_fn, as_uninit_mut) to allow in-place initialization of chunk memory, reducing unnecessary copying and improving efficiency. [1] [2] [3] [4]

Parallelization and utility improvements:

• The build_chunks utility allows parallel chunk allocation when the "parallel" feature is enabled, improving performance for large operations.

Checkpoint and storage API cleanup:

• All code and APIs related to "tail" checkpointing are removed, as the tail buffer is no longer used; this includes removing write_tails, read_tails, and related fields from checkpoint data structures. [1] [2]

Internal simplifications and test code updates:

• Several internal methods are simplified or removed, including chunk collection and remapping logic, which now use the unified chunk access API. [1] [2]

These changes make the chunked tree structure more robust, easier to reason about, and more efficient, especially for large-scale or parallel operations.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.