Skip to content

Add crc64c support - #138

Merged
yunzheng merged 1 commit into
fox-it:mainfrom
alan-divd:feat/crc64c
Oct 27, 2026
Merged

yunzheng merged 1 commit into
fox-it:mainfrom
alan-divd:feat/crc64c

Conversation

@alan-divd

@alan-divd alan-divd commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds CRC-64/NVME, the 64 bit counterpart to the existing crc32c, in both a
pure Python and a native Rust implementation.

The layout follows crc32c exactly, so dissect.util.hash.crc64c resolves to
the native version when the extension is available and falls back to Python
otherwise, with crc64c_native and crc64c_python available for picking one
explicitly.

Parameters are polynomial 0xAD93D23594C93659, reflected, with all-ones init
and xorout.

Test plan

  • tox run -e py3 passes (163 passed, 63 skipped)
  • Matches the official CRC-64/NVME check value,
    crc64c(b"123456789") == 0xae8b14860a799888
  • Cross-checked against an independent from-spec bitwise reference
    implementation rather than only against itself
  • Native and Python agree over 300 randomized inputs, including chunked
    updates composed via the value argument
  • The shared crc64c fixture runs every case against both
    implementations, matching how crc32c is tested
  • Closes Add crc64c support #134

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (f4ff370) to head (53a45a0).

Files with missing lines Patch % Lines
dissect/util/hash/crc64c.py 0.00% 10 Missing ⚠️
dissect/util/hash/__init__.py 0.00% 4 Missing ⚠️
dissect/util/__init__.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##            main    #138   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files         30      31    +1     
  Lines       2157    2170   +13     
=====================================
- Misses      2157    2170   +13     
Flag Coverage Δ
unittests 0.00% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed

codspeed Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 21.32%

1 warning

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
✅ 36 untouched benchmarks
🆕 1 new benchmark

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ test_benchmark_lz4_decompress_dictionary[python-large] 344.8 µs 258.5 µs +33.38%
⚡ test_benchmark_lz4_decompress[python-large] 126.9 µs 115 µs +10.36%
🆕 test_crc64c_benchmark[python] N/A 28.6 µs N/A

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing alan-divd:feat/crc64c (53a45a0) with main (f4ff370)

Open in CodSpeed

@yunzheng
yunzheng merged commit 04ef326 into fox-it:main Oct 27, 2026
34 checks passed
falklindner added a commit to falklindner/dissect.util that referenced this pull request Oct 28, 2026
The native crc64c from fox-it#138 takes its input as a Vec<u8>, which pyo3
fills from a bytes object one Python int at a time. It now takes the
same Buffer argument as crc32c. On Linux with Python 3.13, a 4 KiB
input takes 7.7 us instead of 39.6 us.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HJvjcGbjsG74UunzfT5zNE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add crc64c support

3 participants