Fix update_nodes for Vec (multi-column) node attributes - #35
Conversation
4f33c94 to
01127bd
Compare
update_nodes could not update a Vec node attribute:
- SQLite stores a Vec across per-dimension columns (embedding_0, ...), but
update_nodes wrote the whole sequence into a single column
(`SET embedding = [1.0, 2.0, 3.0]`), which is invalid SQL. It now expands
each attribute to its column(s) via _node_attrs_to_columns and emits one
assignment per column (scalars unchanged; None handled).
- PostgreSQL stores a Vec as a single native array column; __convert_to_sql
now renders list/tuple/ndarray values as an array literal '{v0,v1,...}'
(matching the COPY insert format), and normalizes numpy scalars/arrays.
Adds a regression test parametrized over the sqlite/psql providers.
01127bd to
537d23a
Compare
|
Heads up: the failing Python ty check here is a pre-existing issue, not something this PR introduces.
So this looks like The tests and ruff checks pass; the added |
|
|
||
| def __convert_to_sql(self, x: Any) -> str: | ||
| if hasattr(x, "tolist") and not isinstance(x, (str, bytes)): | ||
| # normalize numpy scalars/arrays to native python |
There was a problem hiding this comment.
my only concern is that this is a bit fragile. Seems like a great solution for numpy scalars/arrays. Not sure if this will lead to unexpected errors given other datatypes.
also str doesn't have a "tolist" method. So the checks seem redundant.
Replace the duck-typed hasattr(x, "tolist") check (and the redundant str/bytes exclusion) with explicit isinstance checks against np.ndarray and np.generic. Only numpy values are normalized to native python; every other datatype passes through exactly as it did before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DVCWK56VDzoEoCRwbkxBcg
Problem
SQLGraphDataBase.update_nodescannot update aVec(multi-column) node attribute. For a node attr declared asVec(float, N), the SQLite backend stores it across per-dimension columns (embedding_0,embedding_1, …), butupdate_nodesbuilt a single assignment from the raw attribute name and value:This raises
sqlite3.OperationalError(surfaced asValueError: UPDATE nodes SET embedding = [...]). The insert path (write_nodes→_node_attrs_to_columns) already expandsVecattrs into their columns; onlyupdate_nodeswas missing this.Fix
Expand each attribute to its column(s) via the existing
_node_attrs_to_columnsand emit one assignment per column. Scalar attributes (one column) are unchanged;Vecattributes get one assignment per dimension, mirroring the insert path.Test
Adds
test_graph_update_vector_node_attr(parametrized over the existing SQLite/PgSQL provider fixture) that writes a node with aVec(float, 3)attribute, updates it viaupdate_nodes, and reads it back. Passes on SQLite; existing graph tests unaffected.Notes
Scoped to the shared setter-building in
update_nodes. The PostgreSQL backend storesVecas a single native array column, so it takes the unchanged single-column path; its own array-literal conversion for updates can be addressed separately if needed (it was equally unsupported before this change).Found while building a volara plugin that writes per-fragment embedding vectors onto RAG nodes for
DistanceAgglom.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com