You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The FFI wrapper has no sparse query-vector setter. Upstream's C API provides one (zvec_sub_query_set_sparse_vector, zvec/src/binding/c/c_api.cc:6342), but ffi/zvec_ffi.h exposes only fp32/fp64 dense setters (ffi/zvec_ffi.h:313-314). As a result, queries against sparse vector fields silently behave as no-ops in radius terms.
Consequences (all verified empirically against the built library)
ZVecVectorQuery::setRadius() on a sparse field never filters anything: a dense payload on a sparse field is converted to a 0-index sparse vector (zvec/src/db/index/common/query.cc:80,141-157), giving IP score 0.0 with every doc — above any -radius threshold, so nothing is ever excluded. Radius queries on sparse fields are silent no-ops.
ZVec::resetRadiusThreshold() (fix: add resetRadiusThreshold() mitigation for radius threshold leak (closes #200) #209, queryVector(): radius-filtered query leaks threshold into subsequent usingRefiner() queries #200) cannot take a sparse payload either. This is cosmetic rather than a bug: sparse fields in this binding are hardcoded to HNSW (ffi/zvec_ffi.cc:420-424), and the HNSW-sparse context reset() clears the threshold every query (zvec/src/core/algorithm/hnsw_sparse/hnsw_sparse_context.h:349-354) — so sparse fields cannot leak thresholds in the first place. (Flat-sparse would leak — flat_sparse_context.h:132 no-op reset — but no Flat-sparse creation path exists in this binding, and IVF rejects sparse outright, ivf_index.cc:24-25.)
Negative sparse weights are accepted upstream (verified via setSparseVectorFp32), so sparse similarity scores can be negative — relevant once real sparse radius queries are exposed.
Implementation sketch
Add zvec_vector_query_set_sparse_vector (or wrap zvec_sub_query_set_sparse_vector) in ffi/zvec_ffi.h/.cc
PHP: sparse-aware query construction (e.g. ZVecVectorQuery::setSparseVector(array $indices, array $weights) or auto-detect index=>weight arrays), keeping BC with the current dense path
Tests: sparse radius query that actually filters; verify resetRadiusThreshold() semantics on sparse fields once real sparse queries exist
Summary
The FFI wrapper has no sparse query-vector setter. Upstream's C API provides one (
zvec_sub_query_set_sparse_vector,zvec/src/binding/c/c_api.cc:6342), butffi/zvec_ffi.hexposes only fp32/fp64 dense setters (ffi/zvec_ffi.h:313-314). As a result, queries against sparse vector fields silently behave as no-ops in radius terms.Consequences (all verified empirically against the built library)
ZVecVectorQuery::setRadius()on a sparse field never filters anything: a dense payload on a sparse field is converted to a 0-index sparse vector (zvec/src/db/index/common/query.cc:80,141-157), giving IP score 0.0 with every doc — above any-radiusthreshold, so nothing is ever excluded. Radius queries on sparse fields are silent no-ops.ZVec::resetRadiusThreshold()(fix: add resetRadiusThreshold() mitigation for radius threshold leak (closes #200) #209, queryVector(): radius-filtered query leaks threshold into subsequent usingRefiner() queries #200) cannot take a sparse payload either. This is cosmetic rather than a bug: sparse fields in this binding are hardcoded to HNSW (ffi/zvec_ffi.cc:420-424), and the HNSW-sparse contextreset()clears the threshold every query (zvec/src/core/algorithm/hnsw_sparse/hnsw_sparse_context.h:349-354) — so sparse fields cannot leak thresholds in the first place. (Flat-sparse would leak —flat_sparse_context.h:132no-op reset — but no Flat-sparse creation path exists in this binding, and IVF rejects sparse outright,ivf_index.cc:24-25.)setSparseVectorFp32), so sparse similarity scores can be negative — relevant once real sparse radius queries are exposed.Implementation sketch
zvec_vector_query_set_sparse_vector(or wrapzvec_sub_query_set_sparse_vector) inffi/zvec_ffi.h/.ccZVecVectorQuery::setSparseVector(array $indices, array $weights)or auto-detect index=>weight arrays), keeping BC with the current dense pathresetRadiusThreshold()semantics on sparse fields once real sparse queries existRelated
refs #200, #209