Skip to content

Add generic Crc32/Crc64 classes and crc64_redis - #155

Merged
yunzheng merged 5 commits into
mainfrom
generic-crc-classes
Oct 29, 2026
Merged

yunzheng merged 5 commits into
mainfrom
generic-crc-classes

Conversation

@yunzheng

@yunzheng yunzheng commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Add table-driven CRC classes that take catalogue parameters (poly in normal notation, init, xorout, reflected), in both pure Python and native Rust. The native classes build their table once, borrow bytes input without copying, and release the GIL for large inputs.

Add crc64_redis (CRC-64/REDIS, Jones polynomial) on top of them, with the same module layout as crc32c/crc64c.

resolves #156

This PR was created with the assistance of an AI model (Claude Opus 5.5)

Add table-driven CRC classes that take catalogue parameters (poly in
normal notation, init, xorout, reflected), in both pure Python and
native Rust. The native classes build their table once, borrow bytes
input without copying, and release the GIL for large inputs.

Add crc64_redis (CRC-64/REDIS, Jones polynomial) on top of them, with
the same module layout as crc32c/crc64c.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 73 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (5f44da4) to head (f192d77).

Files with missing lines Patch % Lines
dissect/util/hash/crc.py 0.00% 54 Missing ⚠️
dissect/util/hash/__init__.py 0.00% 9 Missing ⚠️
dissect/util/hash/crc64_redis.py 0.00% 9 Missing ⚠️
dissect/util/__init__.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##            main    #155   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files         32      34    +2     
  Lines       2178    2249   +71     
=====================================
- Misses      2178    2249   +71     
Flag Coverage Δ
unittests 0.00% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yunzheng
yunzheng requested a review from Schamper October 28, 2026 21:50
Comment thread dissect/util/_native/hash/crc.pyi Resolved
Comment thread dissect/util/hash/__init__.py Resolved
Comment thread dissect/util/hash/crc64_redis.py Resolved
yunzheng and others added 4 commits October 29, 2026 09:31
Building the lookup table with Crc64() at import time cost ~120 µs per
process. hash/__init__.py always imports the module to expose
crc64_redis_python, so this cost was paid even when the native version is
used. Embed the table as a literal, like crc64c and crc32c do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Schamper

Copy link
Copy Markdown
Contributor

Regarding build-native (ubuntu-latest, i686) (pull_request) failing, is today the day that PyPy 3.10 finally bites the dust? I no longer see it listed on their website, and Python 3.10 dies Saturday anyway. In fact, I even see a PyPy 3.12 👀.

@Schamper Schamper left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if you're happy.

@yunzheng

yunzheng commented Oct 29, 2026 •

Copy link
Copy Markdown
Member Author

Regarding build-native (ubuntu-latest, i686) (pull_request) failing, is today the day that PyPy 3.10 finally bites the dust? I no longer see it listed on their website, and Python 3.10 dies Saturday anyway. In fact, I even see a PyPy 3.12 👀.

Time for a s/"pypy3.10"/"pypy3.12"/g in a new dissect-workflow-template PR :)

@yunzheng
yunzheng merged commit a84f8e3 into main Oct 29, 2026
29 of 31 checks passed
@yunzheng
yunzheng deleted the generic-crc-classes branch October 29, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add generic CRC32 and CRC64 classes

2 participants