Skip to content

feat(compression): Added pure rust compression codecs to avoid C-backed crates - #433

Open
LorenzoTettamanti wants to merge 4 commits into
streamnative:masterfrom
LorenzoTettamanti:issue_420
Open

LorenzoTettamanti wants to merge 4 commits into
streamnative:masterfrom
LorenzoTettamanti:issue_420

Conversation

@LorenzoTettamanti

@LorenzoTettamanti LorenzoTettamanti commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Related issue: #420

Changes

Replaces the C-backed compression crates with pure-Rust implementations, so
that consumers of pulsar-rs no longer link liblz4 / libzstd at all:

Codec Before After
lz4 (lz4) lz4-sys (C) lz4_flex 0.11 (pure Rust)
zstd (zstd) zstd-sys (C) ruzstd 0.9 (pure Rust)
zlib (flate2) already pure Rust unchanged
snappy (snap) already pure Rust unchanged
  • CompressionLz4::mode removed: lz4_flex doesn't expose a compression
    mode; the mode field had no effect anymore. LZ4 compression now always
    uses the default lz4_flex::block::compress behavior.
  • CompressionZstd::level changed from i32 to
    ruzstd::encoding::CompressionLevel. Only Fastest and Uncompressed
    are currently implemented upstream (Default/Better/Best panic in
    ruzstd 0.9);

@LorenzoTettamanti
LorenzoTettamanti marked this pull request as ready for review October 2, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant