feat(embeddings): store knowledge bases at five vector widths and add Ollama - #7472
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThe PR expands knowledge-base embedding storage to five vector widths and adds Ollama support for knowledge-base indexing and the Embeddings block.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| packages/db/migrations/0321_multi_width_embeddings.sql | Adds width-specific vector columns, a one-vector-per-row constraint, and matching HNSW indexes, including the 3072-dimensional halfvec expression index. |
| apps/sim/lib/knowledge/vector-columns.ts | Centralizes vector-column writes and distance expressions for all supported widths. |
| apps/sim/lib/knowledge/application/search.ts | Resolves the stored embedding target, rejects incompatible multi-base searches, and routes query vectors to the matching column. |
| apps/sim/lib/embeddings/client.ts | Extends embedding execution to support Ollama and dimension-aware knowledge-base targets. |
| apps/sim/lib/embeddings/ollama-model-catalog.server.ts | Discovers locally available Ollama models and extracts embedding capabilities and vector widths. |
| apps/sim/blocks/blocks/embeddings.ts | Adds Ollama as a keyless Embeddings block provider with a live model selector. |
| packages/sim-setup/src/capability-config.ts | Adds setup options for Ollama and Gemini knowledge-base embedding configurations. |
Sequence Diagram
sequenceDiagram
participant User
participant API as Knowledge API
participant Provider as Embedding Provider
participant DB as PostgreSQL/pgvector
User->>API: Create knowledge base
API->>Provider: Resolve model and vector width
API->>DB: Store model and width
User->>API: Upload document
API->>Provider: Generate embeddings
API->>DB: Write width-specific vector column
User->>API: Search compatible knowledge bases
API->>Provider: Generate query embedding
API->>DB: Compare through matching vector expression/index
DB-->>API: Ranked chunks
API-->>User: Search results
Reviews (5): Last reviewed commit: "fix(embeddings): one rule for a configur..." | Re-trigger Greptile
icecrasher321
force-pushed
the
staging-v104
branch
from
September 4, 2026 02:32
af27f67 to
d1b7ba9
Compare
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@icecrasher321 I have started the AI code review. It will take a few minutes to complete. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
icecrasher321
force-pushed
the
staging-v104
branch
from
September 4, 2026 18:00
d1b7ba9 to
73ca382
Compare
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@icecrasher321 I have started the AI code review. It will take a few minutes to complete. |
Contributor
There was a problem hiding this comment.
12 issues found across 78 files
Confidence score: 2/5
apps/sim/lib/knowledge/embeddings.tsrecords Ollama models at a hard-coded width when lookup fails or returns an unsupported width, while the runtime returns the model’s native width; this can produce incorrect embedding-dimension handling across embeddings — preserve the native width or fail clearly.packages/deployment-config/src/env-capabilities.tsandpackages/sim-setup/src/capability-config.tscan classify or preserve incompatible model values when credentials contain whitespace or users switch providers, causing status to misreport the active provider or setup transitions to fail — normalize IDs and reset provider-specific defaults.apps/sim/lib/internal/embeddings/operations.tsturns selected-model inspection failures into a misleading “missing model” 400 and performs an uncached full-catalog scan for every Ollama request; this can obscure upstream outages and add avoidable latency — preserve upstream errors and inspect or cache only the selected model.apps/sim/blocks/blocks/embeddings.tscan carry stale keyed-provider credentials into Ollama workflows, whileapps/sim/lib/knowledge/embeddings.test.tsremoves direct billing-attribution coverage; clear both credential fields and retain tests for usage, cost, and threshold billing before relying on the provider transition.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/deployment-config/src/integrations.json">
<violation number="1" location="packages/deployment-config/src/integrations.json:7463">
P2: After adding Ollama to this catalog description, the integration page still tells users to paste an Embeddings API key. Add provider-aware auth copy (or model this entry as supporting multiple auth modes) so Ollama users are told to configure `OLLAMA_URL` instead.</violation>
</file>
<file name="apps/sim/lib/knowledge/embeddings.test.ts">
<violation number="1" location="apps/sim/lib/knowledge/embeddings.test.ts:27">
P2: This rewrite removes the only direct behavioral test for `recordSearchEmbeddingUsage` (billing attribution, cost calculation, and threshold billing), and the function is still in production unchanged. The two remaining references to it are both mocked imports, so a regression in that billing path would now pass CI undetected. Add coverage for `recordSearchEmbeddingUsage` (e.g. a separate describe block in this file) alongside the new `getConfiguredKbEmbedding` tests.</violation>
</file>
<file name="apps/docs/content/docs/platform/self-hosting/troubleshooting.mdx">
<violation number="1" location="apps/docs/content/docs/platform/self-hosting/troubleshooting.mdx:253">
P2: When `OPENROUTER_API_KEY` is the only configured embedding credential, the knowledge-base fallback can serve OpenAI-family models, but this troubleshooting list omits it and directs operators to another provider. Document the OpenRouter fallback and keep the `KB_EMBEDDING_MODEL`/`EMBEDDING_OUTPUT_DIMS` guidance alongside it.
(Based on your team's feedback about OpenRouter knowledge-base setup.)</violation>
</file>
<file name="packages/deployment-config/src/env-capabilities.ts">
<violation number="1" location="packages/deployment-config/src/env-capabilities.ts:1269">
P2: When `KB_EMBEDDING_MODEL` contains surrounding whitespace, `knowledgeEmbeddingFamily` selects Gemini or Ollama while the runtime rejects that raw ID and falls back to OpenAI’s default. Status can therefore report the non-OpenAI credentials as configured while embedding calls use the wrong model or fail; classify the raw value to match runtime routing.</violation>
</file>
<file name="apps/sim/lib/internal/embeddings/operations.ts">
<violation number="1" location="apps/sim/lib/internal/embeddings/operations.ts:95">
P2: When the selected model’s `/api/show` fails after `/api/tags` succeeds, this call makes the operation report the installed model as missing with HTTP 400. Preserve selected-model inspection failures as upstream errors (502), or query the selected model directly so transient Ollama failures are not presented as a pull request.</violation>
<violation number="2" location="apps/sim/lib/internal/embeddings/operations.ts:95">
P2: Every Ollama embedding request now performs an uncached full-catalog scan, adding O(number of installed models) network work and waiting on unrelated model probes. Query only the selected model or reuse a cached catalog with bounded refreshes so one unrelated `/api/show` cannot delay every embedding.</violation>
</file>
<file name="apps/sim/blocks/blocks/embeddings.ts">
<violation number="1" location="apps/sim/blocks/blocks/embeddings.ts:277">
P2: When a workflow switches from a keyed provider to Ollama, the generic executor preserves the stale `apiKey` and `openRouterApiKey` because this branch omits them. Clear both credential fields explicitly so they do not cross into and get serialized by the Ollama tool operation.</violation>
</file>
<file name="packages/sim-setup/src/capability-config.ts">
<violation number="1" location="packages/sim-setup/src/capability-config.ts:743">
P2: When an existing installation uses Gemini or Ollama, selecting an OpenAI-family option preserves that incompatible model value and the setup transition rejects the selection. Reset the model to an OpenAI default when switching families, or make the prompt provider-aware so the user is not required to discover and clear the stale value manually.</violation>
</file>
<file name="apps/docs/content/docs/integrations/embeddings.mdx">
<violation number="1" location="apps/docs/content/docs/integrations/embeddings.mdx:22">
P2: Recent Ollama builds accept `dimensions` for Matryoshka models; the adapter omits it for compatibility. Describe the block as not exposing this control, rather than claiming Ollama rejects it.</violation>
<violation number="2" location="apps/docs/content/docs/integrations/embeddings.mdx:22">
P2: Using Ollama avoids provider/API charges, but Sim still applies its base run charge. Replace “costs nothing” with “incurs no provider/API charge” to avoid misleading billing expectations.</violation>
</file>
<file name="apps/docs/content/docs/platform/self-hosting/environment-variables.mdx">
<violation number="1" location="apps/docs/content/docs/platform/self-hosting/environment-variables.mdx:202">
P3: When `EMBEDDING_OUTPUT_DIMS` is unset for Ollama, `getConfiguredKbEmbedding` reads `/api/show` and uses its reported width, so Sim does know the model width. Hosted unsupported values fall back to `1536`; only an explicitly mismatched Ollama width reaches document failure, so distinguish these paths here.</violation>
</file>
<file name="apps/sim/lib/knowledge/embeddings.ts">
<violation number="1" location="apps/sim/lib/knowledge/embeddings.ts:127">
P1: When Ollama width lookup fails or returns an unsupported width, this fallback records the same model at 1536. Ollama does not send a dimensions override, so fixed-width models return their native width and every embedding call then fails response validation; reject the target or choose a compatible model instead.</violation>
</file>
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@icecrasher321 I have started the AI code review. It will take a few minutes to complete. |
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@icecrasher321 I have started the AI code review. It will take a few minutes to complete. |
Collaborator
Author
|
@cubic review |
Contributor
@icecrasher321 I have started the AI code review. It will take a few minutes to complete. |
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 78 files
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
Re-trigger cubic
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 78 files
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
Re-trigger cubic
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 78 files
Requires human review: Auto-approval blocked by 15 unresolved issues from a previous review of this commit.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
Re-trigger cubic
icecrasher321
force-pushed
the
staging-v104
branch
from
September 4, 2026 19:40
8b7c524 to
cbac4a4
Compare
Collaborator
Author
… Ollama Knowledge bases were pinned to one 1536-dimension pgvector column, so only models that emit exactly that width could index one. The embedding table now carries a column per storable width — 384, 768, 1024, 1536, 3072 — and a base records which one it uses at creation, chosen with EMBEDDING_OUTPUT_DIMS. Ollama becomes an embedding provider on both paths: KB_EMBEDDING_MODEL accepts ollama/<model> for knowledge bases, and the Embeddings block offers the models installed on the configured server, with no API key and nothing billed.
…amily gating Round of review findings: - Size the shared indexing batch for the widest storable width; the aggregate guard rejects rather than splits, so a 3,072-wide base past 1,064 chunks failed - Resolve an unstated Ollama width from the server instead of defaulting to 1,536 - Substitute EMBEDDING_OUTPUT_DIMS alongside the model when evaluating the knowledge-embedding capability, so the chain is judged for the target at hand - Classify the embedding family exactly as the runtime does, so an id the runtime rejects cannot report its family as configured - Validate a capability field against the provider being configured, not the first one declaring the key, which rejected 384 in the Ollama wizard branch - Give OpenRouter the OpenAI-family model and width fields it was missing - Resolve model records by own property, so KB_EMBEDDING_MODEL=toString falls back - Narrow a knowledge base's width only for query searches, not tag-only ones - Report an unreachable Ollama as 502 rather than a missing model - Correct the sim-setup command and scope, the Ollama filtering claims, and the unstorable-width troubleshooting advice in the docs
… path Second round of review findings: - Restore apps/sim/lib/knowledge/embeddings.test.ts, whose recordSearchEmbeddingUsage suite a whole-file write had destroyed; the new configuration tests move to a sibling file so the original's vi.spyOn wiring stays intact - Refuse to create a knowledge base when an Ollama model's width cannot be established, rather than pinning it at 1536 where every document would fail - Resolve an Ollama model through /api/show for that model alone instead of scanning the catalog, so one unrelated probe cannot delay or break an embedding, and a 404 stays a 400 while anything else is a 502 - Classify the raw KB_EMBEDDING_MODEL value, since the runtime does not trim it - Clear apiKey and openRouterApiKey explicitly when switching a block to Ollama - Offer the OpenAI-family embedding model as a choice, so switching families in the wizard cannot carry a Gemini or Ollama model forward - Correct the docs on Ollama dimension support, run charges, the OpenRouter knowledge-base fallback, and what Sim can determine about a local model's width
… target Third round of review findings: - Require an explicit OLLAMA_URL before resolving a model's width, matching what the embedding client already demands, so a loopback server cannot produce a knowledge base every later call refuses to serve - Validate EMBEDDING_OUTPUT_DIMS against the selected model rather than its family, since the family pattern accepted widths the model silently overrides - Raise a validation error for a missing model or unreportable width, so the route renders 400 instead of a generic 500 - Stop the Embeddings block auto-persisting whichever model Ollama lists first - Do not offer back a configured value that the provider being set up rejects, so switching families cannot carry the previous family's model forward - Cancel the response body of a rejected Ollama probe - Make the drop-on-inspection-failure test actually exercise that path - Correct the setup provider list, the older-Ollama filtering caveat, and the Sim Cloud OLLAMA_URL claim in the docs
…rcing its width Fourth round of review findings: - Make the setup wizard's EMBEDDING_OUTPUT_DIMS optional and undefaulted for Ollama; requiring it suppressed the server-side width detection entirely, and its 768 default was wrong for a 384-wide model - Replace three different notions of 'an Ollama server is configured' with one predicate shared by the selector, the width lookup, and the embedding client: self-hosted is served by the loopback default as the chat provider already is, and only hosted must be pointed at a server - Document that an ollama/ model id is taken at face value rather than falling back, so a model absent from the server fails creation
icecrasher321
force-pushed
the
staging-v104
branch
from
September 4, 2026 19:52
cbac4a4 to
29cfcfe
Compare
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@icecrasher321 I have started the AI code review. It will take a few minutes to complete. |
Contributor
There was a problem hiding this comment.
1 existing issue remains and 3 new issues found across 78 files
Confidence score: 3/5
apps/sim/lib/embeddings/ollama-model-catalog.server.tscan classify a reachable non-Ollama endpoint returning 404 from/api/showas a missing model, obscuring the configuration or service error; preserve the response status in a typed error and only classify expected missing-model responses.apps/docs/content/docs/integrations/embeddings.mdxmay lead users on Ollama versions before 0.5 to select chat models because/api/showcan omit capabilities while still providingembedding_length; document the 0.5 cutoff and the resulting selector behavior.apps/docs/content/docs/platform/self-hosting/environment-variables.mdxdoes not clearly identify that an explicitly setEMBEDDING_OUTPUT_DIMScan allow knowledge-base creation with a missing Ollama model before failing during width discovery or document indexing; clarify the actual failure point.packages/sim-setup/src/capability-config.tsadvertises 3072 for the defaulttext-embedding-3-small, even though final validation rejects that width; make the hint list widths for the selected model.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="apps/docs/content/docs/integrations/embeddings.mdx">
<violation number="1" location="apps/docs/content/docs/integrations/embeddings.mdx:22">
P2: On Ollama versions older than 0.5, `/api/show` may omit capabilities while still reporting `embedding_length`, so the selector can include chat models without losing width labels. Name the 0.5 cutoff and tell users to verify the model or upgrade.
(Based on your team's feedback about the Ollama older-server caveat.)</violation>
</file>
<file name="packages/sim-setup/src/capability-config.ts">
<violation number="1" location="packages/sim-setup/src/capability-config.ts:777">
P3: When the operator keeps the default `text-embedding-3-small`, this hint advertises 3072 even though that model cannot emit it, so final setup validation rejects the value. Make the hint list widths per selected model.</violation>
</file>
<file name="apps/docs/content/docs/platform/self-hosting/environment-variables.mdx">
<violation number="1" location="apps/docs/content/docs/platform/self-hosting/environment-variables.mdx:193">
P2: When `EMBEDDING_OUTPUT_DIMS` is explicitly set, a missing Ollama model leaves the knowledge base created and fails later document indexing. Describe width discovery or document indexing as the failure point instead of claiming knowledge-base creation always fails.</violation>
</file>
Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
Fix all with cubic | Re-trigger cubic
| Two things worth knowing before you build on it. Vectors are only comparable when they come from the same model at the same size, so changing either means re-embedding everything you intend to compare. And input longer than the model's limit is shortened to fit rather than rejected, with a warning in the run, so chunk long documents yourself when the tail matters. | ||
|
|
||
| Sim's knowledge bases embed separately, at a fixed vector width and from a smaller set of models. This block is for embedding text yourself inside a workflow. | ||
| Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and adds no provider charge — Sim's own per-run charge still applies — and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains. The block reads it live, drops the models that report a non-embedding capability, and shows each one's vector width next to its name where Ollama reports one. A server too old to report either will list its chat models too and label none of them, so check the model you pick. The block offers no task-type or dimension control for Ollama: task conditioning has no equivalent there, and while recent Ollama builds do accept a dimension override for Matryoshka models, older ones silently ignore it, so Sim uses each model's own width rather than one that may or may not take effect. Point Sim at the server with `OLLAMA_URL`. Sim Cloud runs no Ollama of its own, so without that variable the list comes back empty rather than dialling a loopback address that cannot answer — set it to a reachable server and Cloud will use it like any other deployment. |
Contributor
There was a problem hiding this comment.
P2: On Ollama versions older than 0.5, /api/show may omit capabilities while still reporting embedding_length, so the selector can include chat models without losing width labels. Name the 0.5 cutoff and tell users to verify the model or upgrade.
(Based on your team's feedback about the Ollama older-server caveat.)
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/docs/content/docs/integrations/embeddings.mdx, line 22:
<comment>On Ollama versions older than 0.5, `/api/show` may omit capabilities while still reporting `embedding_length`, so the selector can include chat models without losing width labels. Name the 0.5 cutoff and tell users to verify the model or upgrade.
(Based on your team's feedback about the Ollama older-server caveat.) </comment>
<file context>
@@ -13,19 +13,21 @@ import { BlockInfoCard } from "@/components/ui/block-info-card"
Two things worth knowing before you build on it. Vectors are only comparable when they come from the same model at the same size, so changing either means re-embedding everything you intend to compare. And input longer than the model's limit is shortened to fit rather than rejected, with a warning in the run, so chunk long documents yourself when the tail matters.
-Sim's knowledge bases embed separately, at a fixed vector width and from a smaller set of models. This block is for embedding text yourself inside a workflow.
+Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and adds no provider charge — Sim's own per-run charge still applies — and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains. The block reads it live, drops the models that report a non-embedding capability, and shows each one's vector width next to its name where Ollama reports one. A server too old to report either will list its chat models too and label none of them, so check the model you pick. The block offers no task-type or dimension control for Ollama: task conditioning has no equivalent there, and while recent Ollama builds do accept a dimension override for Matryoshka models, older ones silently ignore it, so Sim uses each model's own width rather than one that may or may not take effect. Point Sim at the server with `OLLAMA_URL`. Sim Cloud runs no Ollama of its own, so without that variable the list comes back empty rather than dialling a loopback address that cannot answer — set it to a reachable server and Cloud will use it like any other deployment.
+
+Sim's knowledge bases embed separately: a base fixes one model and one vector width when it is created, from a smaller set of models. This block is for embedding text yourself inside a workflow.
</file context>
Suggested change
| Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and adds no provider charge — Sim's own per-run charge still applies — and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains. The block reads it live, drops the models that report a non-embedding capability, and shows each one's vector width next to its name where Ollama reports one. A server too old to report either will list its chat models too and label none of them, so check the model you pick. The block offers no task-type or dimension control for Ollama: task conditioning has no equivalent there, and while recent Ollama builds do accept a dimension override for Matryoshka models, older ones silently ignore it, so Sim uses each model's own width rather than one that may or may not take effect. Point Sim at the server with `OLLAMA_URL`. Sim Cloud runs no Ollama of its own, so without that variable the list comes back empty rather than dialling a loopback address that cannot answer — set it to a reachable server and Cloud will use it like any other deployment. | |
| Ollama is the exception to most of the above. It runs on your own deployment, so it needs no API key and adds no provider charge — Sim's own per-run charge still applies — and the model list is whatever you have pulled onto that server rather than a catalog Sim maintains. The block reads it live, drops the models that report a non-embedding capability, and shows each one's vector width next to its name where Ollama reports one. Ollama only began reporting capabilities in version 0.5; older servers may leave chat models in the list, even when they still report widths, so verify the model you pick or upgrade. The block offers no task-type or dimension control for Ollama: task conditioning has no equivalent there, and while recent Ollama builds do accept a dimension override for Matryoshka models, older ones silently ignore it, so Sim uses each model's own width rather than one that may or may not take effect. Point Sim at the server with `OLLAMA_URL`. Sim Cloud runs no Ollama of its own, so without that variable the list comes back empty rather than dialling a loopback address that cannot answer — set it to a reachable server and Cloud will use it like any other deployment. |
| | Variable | Description | | ||
| |----------|-------------| | ||
| | `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; an unsupported value falls back to the default | | ||
| | `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; use `ollama/<model>` for a model on your own Ollama. An unrecognised hosted model id falls back to the default, but an `ollama/` id is taken at face value — if that model is not on the server, knowledge-base creation fails rather than falling back | |
Contributor
There was a problem hiding this comment.
P2: When EMBEDDING_OUTPUT_DIMS is explicitly set, a missing Ollama model leaves the knowledge base created and fails later document indexing. Describe width discovery or document indexing as the failure point instead of claiming knowledge-base creation always fails.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/docs/content/docs/platform/self-hosting/environment-variables.mdx, line 193:
<comment>When `EMBEDDING_OUTPUT_DIMS` is explicitly set, a missing Ollama model leaves the knowledge base created and fails later document indexing. Describe width discovery or document indexing as the failure point instead of claiming knowledge-base creation always fails.</comment>
<file context>
@@ -190,10 +190,43 @@ See [Observability](/platform/self-hosting/observability).
| Variable | Description |
|----------|-------------|
-| `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; an unsupported value falls back to the default |
+| `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; use `ollama/<model>` for a model on your own Ollama. An unrecognised hosted model id falls back to the default, but an `ollama/` id is taken at face value — if that model is not on the server, knowledge-base creation fails rather than falling back |
+| `EMBEDDING_OUTPUT_DIMS` | Vector width new knowledge bases are stored at: `384`, `768`, `1024`, `1536` (default), or `3072`. It must be a width the chosen model can emit; anything else falls back to `1536` with a warning |
| `OPENROUTER_API_KEY` | Fallback route for the OpenAI embedding models — used when it is set and `OPENAI_API_KEY` is not the chosen path |
</file context>
Suggested change
| | `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; use `ollama/<model>` for a model on your own Ollama. An unrecognised hosted model id falls back to the default, but an `ollama/` id is taken at face value — if that model is not on the server, knowledge-base creation fails rather than falling back | | |
| | `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; use `ollama/<model>` for a model on your own Ollama. An unrecognised hosted model id falls back to the default, but an `ollama/` id is taken at face value — if that model is not on the server, width discovery or document indexing fails rather than falling back | |
| type: 'field', | ||
| key: 'EMBEDDING_OUTPUT_DIMS', | ||
| input: 'text', | ||
| hint: 'optional vector width for new knowledge bases: 768, 1024, 1536 (default), or 3072', |
Contributor
There was a problem hiding this comment.
P3: When the operator keeps the default text-embedding-3-small, this hint advertises 3072 even though that model cannot emit it, so final setup validation rejects the value. Make the hint list widths per selected model.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/sim-setup/src/capability-config.ts, line 777:
<comment>When the operator keeps the default `text-embedding-3-small`, this hint advertises 3072 even though that model cannot emit it, so final setup validation rejects the value. Make the hint list widths per selected model.</comment>
<file context>
@@ -731,6 +731,54 @@ export const KNOWLEDGE_SETUP = defineCapabilitySetup(OCR_CAPABILITY, {
+ type: 'field',
+ key: 'EMBEDDING_OUTPUT_DIMS',
+ input: 'text',
+ hint: 'optional vector width for new knowledge bases: 768, 1024, 1536 (default), or 3072',
+ validate: true,
+ },
</file context>
Suggested change
| hint: 'optional vector width for new knowledge bases: 768, 1024, 1536 (default), or 3072', | |
| hint: 'optional vector width; small: 768, 1024, or 1536 (default); large: 768, 1024, 1536, or 3072', |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
embeddingnow has a column per storable width — 384, 768, 1024, 1536, 3072 — and a base records which one it uses at creationEMBEDDING_OUTPUT_DIMSpicks that width. A value the deployment can't store, or one the configured model can't emit, warns and falls back to 1536 rather than failing knowledge-base creationKB_EMBEDDING_MODELacceptsollama/<model>, so a self-hosted deployment can index against its own Ollama with no API key and nothing billedvectorup to 2,000 dimensions, so the 3072 column is indexed through ahalfveccast. Queries have to repeat that exact cast to use the index, so the distance expression for every width comes from one placesim-setupgains Gemini and Ollama options, andsim-setup statusnow reports the one familyKB_EMBEDDING_MODELactually selects instead of every provider a key exists for. Gemini previously displayed as "OpenAI"Type of Change
Testing
Tested manually against a local Postgres (pgvector 0.8.0) and a real Ollama:
executeKnowledgeSearchin hybrid and vector mode — withollama/nomic-embed-text(768),ollama/all-minilm(384),text-embedding-3-large(3072 and 1024), andtext-embedding-3-small(1536)EXPLAINconfirms each width hits its own HNSW index, and that 3072 without thehalfveccast falls to a sequential scanembedding_width_checkrejects rows with no vector and rows with two; the migration applies from the pre-migration shape and is replay-safeapps/sim, all packages),turbo type-check,bun run lint,check:audits(45 audits),docs-manifest:check, block-registry check, andcheck:migrations origin/stagingChecklist