Updated architecture

security/architecture/data_flows.mdx

@@ -9,16 +9,15 @@
 Documents from connectors and user uploads are processed by the Onyx indexing pipeline.
 With default configurations no data ever leaves the deployment.
 
-The general processing outline goes as follows:
-- Documents, metadata, and access permissions are pulled in from connectors
-- Documents are processed into text through document parsing utilities
-- The texts are chunked and passed through deep learning (embedding) models
-- These representations are stored in the vector database
-- Optionally (default off), an LLM can be used to extract entities and relations from the documents
-  and represent them as a graph within Postgres
-
 <img className="rounded-image" src="/assets/security/OnyxIndexingFlow.png" alt="Onyx Indexing Flow"/>
 
+<Note>
+  The LLM-based contextual retrieval and the informativeness model are not enabled by default.
+</Note>
+
+Not shown in the diagram, MinIO (which runs locally)
+acts as an intermediate store for the document before the downstream processing steps like chunking.
+
 Onyx does also allow configuring the following options:
 
 <Info>
@@ -27,20 +26,21 @@
 </Info>
 
 - API based embedding model. Teams may choose to do this instead of choosing between running their
   own GPUs, using a less capable embedding model, or accepting a slower initial indexing.
-- Third party document-to-text service. Some third party services provide better processing using
-  large vision models and other approaches. This can yield better extraction of text from your documents.
-- Connecting an LLM for the generation of the knowledge graph. The knowledge graph provides an
-  additional representation of the connected knowledge and can be used to answer more abstract type questions.
+- Image captioning service. Connecting a vision-capable model lets files with embedded images, as
+  well as standalone image files, be captioned into text so they can be represented for retrieval.
+- Connecting an LLM for contextual retrieval. This uses the LLM to prepend a short, document-aware
+  summary to each chunk so it carries the surrounding context,
+  which improves retrieval accuracy for chunks that would otherwise be ambiguous on their own.
 
 ## Query Flow
 
 <img className="rounded-image" src="/assets/security/OnyxQueryFlow.png" alt="Onyx Query Flow"/>
 
 When users query Onyx, the LLM determines if the system should fetch additional context or respond to the user directly.
 
-If additional context is needed, the system can choose between available options including: ingested knowledge,
-web search (if configured), built-in actions (like code execution), or additional user configured actions.
+If additional context is needed, the system can run an internal search (outlined above),
+invoke a built-in action (such as code execution or web search), or call an action configured by your admins.
 
 By default, the system does not communicate data to any external systems outside of the admin configured LLM.
 
@@ -52,10 +52,10 @@
   It is recommended to enable these functionalities to let your users get the most of out Onyx.
 </Tip>
 
 **Web Search**: Sends search queries to a configured search provider. Supported providers include Google PSE, Serper,
 and Exa AI to get links and snippets. A crawler is used to fetch the full contents of the page,
 Onyx has a built in one and also supports Firecrawl.
 
 **Image Generation**: Sends prompts to a third party image generation endpoint like OpenAI's Dalle model.
 
 **Custom Actions**: API calls available to the LLM, configured by the Admin users of your Onyx deployment.

security/architecture/data_storage.mdx

@@ -22,7 +22,7 @@
 Stores:
 - User information, preferences, settings, etc.
 - Query History
 - Credentials to LLMs, connectors, actions (these are encrypted in Enterprise Edition)
 - Document access control information
 - Entities/Relationships extracted from documents (only if knowledge graph is turned on)
 
@@ -30,7 +30,7 @@
 
 **Access**: Protected by database user authentication with configurable credentials and optional IAM authentication
 
-## Vector Database + Search Engine - Vespa
+## Vector Database + Search Engine - OpenSearch
 
 Stores:
 - Documents and metadata
@@ -39,12 +39,12 @@
 
 **Encryption**: Relies on the disk encryption of the deployment
 
-**Access**: Protected by Vespa authentication and access controls
+**Access**: Protected by OpenSearch authentication and access controls
 
 ## Object Storage - MinIO
 
 Stores:
 - Original documents from connectors and user uploads (PDFs, Word docs, etc.)
 - Document attachments and media files
 - Temporary files during document processing
 

security/architecture/system_description.mdx

@@ -27,17 +27,25 @@
 - **API Server**: Python FastAPI web server to handle business logic and requests
 - **Background Workers**: Python workers to handle async jobs like fetching document updates, etc.
 
+### Model Servers
+
+- **Inference Model Server**: Serves the embedding and reranking models used at query time to
+  retrieve and order context
+- **Indexing Model Server**: A dedicated model server that handles embedding of documents during
+  indexing, isolating heavy indexing workloads from live query traffic
+
 ### Data Layer
 
 - **Relational Database**: Postgres database to store application data, user sessions, and system
   state
-- **Vector Database**: Vespa keyword search engine + vector store for retrieving context for queries
+- **Search Engine**: OpenSearch providing keyword (BM25) search + vector store for retrieving
+  context for queries
 - **In Memory Cache**: Redis, used to optimize performance
 - **File Store**: MinIO blob store for user uploaded files and documents pulled from connectors
 
 ### Infrastructure Layer
 
 - **Request Router**: Nginx reverse proxy for load balancing and request routing
 
 ## Component Replacements
 
@@ -53,8 +61,9 @@
 It is much more challenging and not advised to switch to a different relational DB due to language specific features and
 optimizations.
 
-**Vespa** can be replaced with multi-node Vespa deployment (typically not necessary) or Vespa Cloud.
-Vespa is tightly integrated with the context retrieval functionalities and replacing it would require significant
-development effort to maintain the same functionality.
+**OpenSearch** can be replaced with a multi-node OpenSearch cluster (typically not necessary)
+or a managed OpenSearch service such as AWS OpenSearch Service.
+OpenSearch is tightly integrated with the context retrieval functionalities and switching to a different search engine
+would require significant development effort to maintain the same functionality.
 
 **Nginx** can be removed and replaced with your own routing proxy.