Release v1.0.1: Rust core, Python bindings, audit fixes#3
Merged
Conversation
Implements the CSSD algorithm from Storath & Weinmann (JCGS 2023) in Rust with NumPy-friendly Python bindings via PyO3 / maturin. * crates/cssd-core: pure Rust algorithm crate (DP, FPVI/PELT, Reinsch reconstruction, optional locally-adaptive Jacobi preconditioning). * crates/cssd-py: thin PyO3 binding -> _cssd_core extension module. * python/cssd: NumPy-friendly Python wrapper (cssd, cssd_cv, PiecewisePoly). * tests_py + crates/cssd-core/tests: ~210 unit + integration tests; the MATLAB-fixture parity tests auto-skip when fixtures aren't present. * demos_py: 7 ported demos (synthetic, HeaviSine, vector-valued, geyser CV, CPU time, preconditioning comparison). * .github/workflows: CI on push/PR + maturin-action release-on-tag pipeline. * README: adds Python install + quickstart section. * Wheels are abi3-py39 -> one wheel per platform covers Python 3.9+. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PyString::from_object buffer-overread vulnerability affects every pyo3
in [0.1.0, 0.24.1). The cssd-py wrapper does not call PyString::from_object
directly, but the vulnerable code path is loaded into the Python process
whenever the extension module is imported.
Changes:
- crates/cssd-py/Cargo.toml: pyo3 0.22 → 0.24.1; numpy 0.22 → 0.24
(numpy crate version tracks pyo3 minor).
- Cargo.lock: regenerated by cargo.
- crates/cssd-py/src/lib.rs: drop the _bound suffix on numpy/pyo3 APIs that
pyo3 0.23+ deprecated once Bound<'py, T> became the default:
into_pyarray_bound → into_pyarray (13 callsites)
PyDict::new_bound → PyDict::new (2 callsites)
PyList::empty_bound → PyList::empty (2 callsites)
No algorithmic change.
Verified: cargo build --release -p cssd-py is warning-free; cssd-core
Rust tests pass; all 211 Python tests pass.
- CITATION.cff at root with Storath & Weinmann authorship and the JCGS
2024 reference (DOI 10.1080/10618600.2023.2262000).
- CHANGELOG.md with the v0.1.0 release notes (Keep a Changelog layout).
- pyproject.toml:
- Add author emails (storath@thws, weinmann@thws).
- Expand classifiers: Operating System :: OS Independent, Python 3.9–
3.13, Topic :: Mathematics.
- Add keywords list.
- Expand [project.urls]: Homepage, Repository, Changelog, Bug Tracker.
Compares cssd_core::cssd() against the MATLAB implementation invoked via the dev container's matlab shim. Three cases: piecewise-constant step inputs with FPVI and PELT pruning, plus one smooth quadratic with γ=1e8 that degenerates to the classical smoothing spline. Tolerance atol=1e-8 / rtol=1e-5 on pp.coefs (matches existing tests_py/test_matlab_parity.py); atol=1e-10 on pp.breaks; exact on discont_idx. Tightening below 1e-5 relative on coefs is empirically not feasible without algorithmic changes (LAPACK vs nalgebra/ndarray QR internals). Skipped (not failed) when the matlab shim is unavailable. Adds no runtime dependencies; uses cssd-core's existing dev-deps.
Six worked examples that double as a v0.1.0 release verification and a
first-time-user introduction:
1. Smooth-only regime (γ=∞ ≡ classical smoothing spline) at three p
values.
2. Single-discontinuity recovery (γ=∞ blurs, γ=1.0 finds the jump).
3. Heteroscedastic noise + multiple discontinuities with delta weights.
4. FPVI vs PELT pruning agreement (rtol < 1e-10 on coefs).
5. Automatic (p, γ) selection via cssd_cv 5-fold cross-validation.
Cells render inline in the committed .ipynb; viewable on GitHub without
re-execution. Run-all completes in <10 s.
Used to be a CSSD-side parallel to demos_py/'s existing fixture-based
content; this notebook is API-driven, not fixture-driven.
This commit lands the MATLAB-side fixes catalogued in PORTING_NOTES.md
during the Rust port audit. Each tagged change is named after its
PORTING_NOTES entry; cross-reference there for the rationale.
Bug fixes (output-affecting):
B1 cssd.m: PELT/FPVI rolling-CV used stale stored_R/stored_z when no
candidate improved F(rb). Fall back to first_lb's updated state
(PELT) or extrapolate from x_{rb-1} (FPVI), so rcv_score is always
a valid extrapolation residual.
B3 cssd.m: rcv_score / (N-2) divided by zero for N <= 2; install_cssd's
smoke test was hitting it. Score is now 0 in that regime; smoke
test bumped from N=2 to N=3.
B4 cssd_cv.m: inputParser was named 'p', shadowing the smoothing
parameter; renamed to 'parser'.
B5 cssd_cv.m: K=5 default applied only when both cv_type AND cv_arg
were defaulted, so cssd_cv(x, y, 'random') crashed. Default now
kicks in whenever cv_arg is empty.
B6 cssd.m: invalid pruning string silently fell through to FPVI; the
switch's 'otherwise' branch is now an explicit error and FPVI is
its own case.
B7 PcwFunReal.m: eval allocated NaN(size(xx)), correct only for
scalar pieces. Reshaped to (numel(xx), dim) so vector-valued pp
output (csaps with dim > 1) is handled. Plot helper updated to
match.
B8 cssd_cv.m: gamma_pq parametrisation gave NaN at the q=1 boundary
(0 * Inf = NaN for p=0, q=1). Clamped q upper bound to 1-eps and
max(1-q, eps) in the formula.
Audit notes (preconditions / robustness):
N1 chkxydelta.m: documented the implicit Curve Fitting Toolbox
dependency on chckxywp.
N2 linext_pp.m: forced pp.breaks to a row before concat to handle
column-shaped breaks; clearer assertion message.
N3 merge_ppcell.m: assert consecutive endpoints match (was silently
dropping the previous-end break).
N4 spline_innerenergy.m: documented dependence on natural BC.
N5 cssd.m: assertion message updated to match the (non-strict) check
on gamma.
N6 kfoldcv_split.m: assert 2 <= K <= N.
N8 cssd.m: gamma=Inf branch now uses linext + embed_pptocubic so
output.pp has the same domain convention as the DP branch.
tests/TestCSSD.m (+142 lines): regression tests for each B*/N* fix,
named test_<id>_* for cross-reference with PORTING_NOTES.md.
Output behaviour change: rcv_score values for the same (x, y, p, gamma)
input differ between this commit and HEAD~1. Consumers of rcv_score
(only diagnostic / hyperparameter selection) should regenerate any
cached scores. pp.coefs, pp.breaks, discont, discont_idx are unaffected
by these fixes.
Mirrors tests/test_matlab_parity.py in Pottslab: each test generates a
small input, calls the Rust port (cssd Python module) in-process, then
shells out to host MATLAB via the matlab shim, parses CSV outputs, and
compares pp.coefs, pp.breaks, and discont_idx.
The previous fixture-based test_matlab_parity.py loaded 594 pre-baked
.mat files generated by matlab_fixtures/dump_fixtures.m. Those fixtures
were generated against pre-audit MATLAB (HEAD~1) and are now stale for
the rcv_score field (B1, B2 fixed) — though pp.coefs / pp.breaks /
discont_idx remain valid. The live approach removes the fixture
maintenance burden and surfaces regressions in either implementation
immediately. Skipped (not failed) when the matlab shim is unavailable.
Test cases (8 total, ~50s):
- Step signal × FPVI / PELT, two (p, γ) settings: 4 cases
- Smooth quadratic with γ=∞ (degenerates to classical smoothing
spline): 1 case
- Random Gaussian signals × 3 seeds × varied (n, p, γ): 3 cases
Tolerances: atol=1e-8 / rtol=1e-5 on pp.coefs (LAPACK QR vs Givens QR
divergence is the limiter); atol=1e-10 on pp.breaks; exact on
discont_idx. Verified all 8 pass against the audit-fixed MATLAB at
HEAD.
The orphaned tests_py/fixtures/ directory (594 .mat files) and
matlab_fixtures/dump_fixtures.m are left in place; cleanup is a
separate decision.
Following the live-MATLAB-parity test rewrite (commit d3fe3f4), three
loose ends remained:
- tests_py/conftest.py defined a session-scope `fixtures_dir` fixture
pointing to tests_py/fixtures/. No test referenced it after the
rewrite. Removed; conftest.py is now an empty scaffold (one-liner
docstring).
- .gitignore had two entries (`tests_py/fixtures/` and
`matlab_fixtures/`) for now-deleted directories. Removed.
- Local copies of tests_py/fixtures/ (594 .mat files, ~2.9 MB) and
matlab_fixtures/dump_fixtures.m were untracked artifacts of the
fixture-generation workflow and were rm -rf'd in the working tree.
The 211 remaining Python unit tests still pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cssd. Continues the MATLAB version line (last MATLAB tag v1.0.0 -> v1.0.1).cssd-core) + PyO3 bindings (cssd-py) + Python wrapper.See reports/05-pypi-publication-roadmap.md for the wider context.
Test plan
cargo test --workspace --releasegreenpytest tests_pygreen (211 tests)v1.0.1-rc1tag triggers TestPyPI publish;pip install -i https://test.pypi.org/simple cssd==1.0.1rc1works