Skip to content

feat: expose sparse query vectors (upstream zvec_sub_query_set_sparse_vector) — sparse radius queries are currently silent no-ops #210

Description

@s2x

Summary

The FFI wrapper has no sparse query-vector setter. Upstream's C API provides one (zvec_sub_query_set_sparse_vector, zvec/src/binding/c/c_api.cc:6342), but ffi/zvec_ffi.h exposes only fp32/fp64 dense setters (ffi/zvec_ffi.h:313-314). As a result, queries against sparse vector fields silently behave as no-ops in radius terms.

Consequences (all verified empirically against the built library)

  • ZVecVectorQuery::setRadius() on a sparse field never filters anything: a dense payload on a sparse field is converted to a 0-index sparse vector (zvec/src/db/index/common/query.cc:80,141-157), giving IP score 0.0 with every doc — above any -radius threshold, so nothing is ever excluded. Radius queries on sparse fields are silent no-ops.
  • ZVec::resetRadiusThreshold() (fix: add resetRadiusThreshold() mitigation for radius threshold leak (closes #200) #209, queryVector(): radius-filtered query leaks threshold into subsequent usingRefiner() queries #200) cannot take a sparse payload either. This is cosmetic rather than a bug: sparse fields in this binding are hardcoded to HNSW (ffi/zvec_ffi.cc:420-424), and the HNSW-sparse context reset() clears the threshold every query (zvec/src/core/algorithm/hnsw_sparse/hnsw_sparse_context.h:349-354) — so sparse fields cannot leak thresholds in the first place. (Flat-sparse would leak — flat_sparse_context.h:132 no-op reset — but no Flat-sparse creation path exists in this binding, and IVF rejects sparse outright, ivf_index.cc:24-25.)
  • Negative sparse weights are accepted upstream (verified via setSparseVectorFp32), so sparse similarity scores can be negative — relevant once real sparse radius queries are exposed.

Implementation sketch

  • Add zvec_vector_query_set_sparse_vector (or wrap zvec_sub_query_set_sparse_vector) in ffi/zvec_ffi.h/.cc
  • PHP: sparse-aware query construction (e.g. ZVecVectorQuery::setSparseVector(array $indices, array $weights) or auto-detect index=>weight arrays), keeping BC with the current dense path
  • Tests: sparse radius query that actually filters; verify resetRadiusThreshold() semantics on sparse fields once real sparse queries exist

Related

refs #200, #209

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions