Skip to content

Fix update_nodes for Vec (multi-column) node attributes - #35

Merged
pattonw merged 2 commits into
funkelab:mainfrom
rhoadesScholar:fix-sqlite-vector-node-attr-update
Jul 23, 2026
Merged

Fix update_nodes for Vec (multi-column) node attributes#35
pattonw merged 2 commits into
funkelab:mainfrom
rhoadesScholar:fix-sqlite-vector-node-attr-update

Conversation

@rhoadesScholar

Copy link
Copy Markdown
Contributor

Problem

SQLGraphDataBase.update_nodes cannot update a Vec (multi-column) node attribute. For a node attr declared as Vec(float, N), the SQLite backend stores it across per-dimension columns (embedding_0, embedding_1, …), but update_nodes built a single assignment from the raw attribute name and value:

setters = [f"{k} = {self.__convert_to_sql(v)}" for k, v in zip(attrs, values)]
# -> UPDATE nodes SET embedding = [1.0, 2.0, 3.0]   # invalid SQL

This raises sqlite3.OperationalError (surfaced as ValueError: UPDATE nodes SET embedding = [...]). The insert path (write_nodes_node_attrs_to_columns) already expands Vec attrs into their columns; only update_nodes was missing this.

Fix

Expand each attribute to its column(s) via the existing _node_attrs_to_columns and emit one assignment per column. Scalar attributes (one column) are unchanged; Vec attributes get one assignment per dimension, mirroring the insert path.

Test

Adds test_graph_update_vector_node_attr (parametrized over the existing SQLite/PgSQL provider fixture) that writes a node with a Vec(float, 3) attribute, updates it via update_nodes, and reads it back. Passes on SQLite; existing graph tests unaffected.

Notes

Scoped to the shared setter-building in update_nodes. The PostgreSQL backend stores Vec as a single native array column, so it takes the unchanged single-column path; its own array-literal conversion for updates can be addressed separately if needed (it was equally unsupported before this change).

Found while building a volara plugin that writes per-fragment embedding vectors onto RAG nodes for DistanceAgglom.

Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com

@rhoadesScholar
rhoadesScholar force-pushed the fix-sqlite-vector-node-attr-update branch from 4f33c94 to 01127bd Compare July 16, 2026 05:47
update_nodes could not update a Vec node attribute:

- SQLite stores a Vec across per-dimension columns (embedding_0, ...), but
  update_nodes wrote the whole sequence into a single column
  (`SET embedding = [1.0, 2.0, 3.0]`), which is invalid SQL. It now expands
  each attribute to its column(s) via _node_attrs_to_columns and emits one
  assignment per column (scalars unchanged; None handled).
- PostgreSQL stores a Vec as a single native array column; __convert_to_sql
  now renders list/tuple/ndarray values as an array literal '{v0,v1,...}'
  (matching the COPY insert format), and normalizes numpy scalars/arrays.

Adds a regression test parametrized over the sqlite/psql providers.
@rhoadesScholar
rhoadesScholar force-pushed the fix-sqlite-vector-node-attr-update branch from 01127bd to 537d23a Compare July 16, 2026 06:40
@rhoadesScholar
rhoadesScholar marked this pull request as ready for review July 16, 2026 06:41
@rhoadesScholar

Copy link
Copy Markdown
Contributor Author

Heads up: the failing Python ty check here is a pre-existing issue, not something this PR introduces.

  • On the base commit (13d91d6, current main), uv run --extra dev ty check src tests reports the same 9 diagnostics as this branch. This change adds none — they're all in unrelated code (arrays/metadata.py, arrays/array.py, arrays/ome_datasets.py, graphs/pgsql_graph_database.py:225, and the existing tests/test_graph.py:165, whose # type: ignore[call-overload] ty doesn't honor).
  • main's ty workflow last ran green on 2026-02-13; it hasn't re-run since. Because the workflow installs ty unpinned (ty>=0.0.16 via setup-uv@main), a fresh run now picks up a newer, stricter ty that flags these. A re-run on main would fail identically.

So this looks like ty/stub version drift rather than a regression from this change. Happy to help address it separately (pin ty, or fix/# ty: ignore the flagged sites) if useful — just didn't want to bundle unrelated type fixes into this focused update_nodes bugfix.

The tests and ruff checks pass; the added test_graph_update_vector_node_attr covers the fix on both the sqlite and psql providers.


def __convert_to_sql(self, x: Any) -> str:
if hasattr(x, "tolist") and not isinstance(x, (str, bytes)):
# normalize numpy scalars/arrays to native python

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my only concern is that this is a bit fragile. Seems like a great solution for numpy scalars/arrays. Not sure if this will lead to unexpected errors given other datatypes.
also str doesn't have a "tolist" method. So the checks seem redundant.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts? @pattonw

Replace the duck-typed hasattr(x, "tolist") check (and the redundant
str/bytes exclusion) with explicit isinstance checks against np.ndarray
and np.generic. Only numpy values are normalized to native python;
every other datatype passes through exactly as it did before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVCWK56VDzoEoCRwbkxBcg
@pattonw
pattonw merged commit 1745744 into funkelab:main Jul 23, 2026
1 of 2 checks passed
@rhoadesScholar
rhoadesScholar deleted the fix-sqlite-vector-node-attr-update branch July 23, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants