Add AGENTS.md for query-container

Signed-off-by: Aman Singh <[email protected]>

Author: amansinghoriginal

query-container/AGENTS.md

@@ -0,0 +1,47 @@
+# AGENTS.md: `query-container`
+
+## 1. Purpose & Architectural Intent
+
+This directory contains the **Query Container**, a set of Rust microservices providing the core runtime environment for executing Drasi's continuous queries.
+
+**Core Intent**: To provide a scalable, stateful, and resilient host for the `drasi-core` query engine. It manages the complete lifecycle of a continuous query, from data ingestion and processing to publishing results and maintaining a queryable view.
+
+**Key Technologies**:
+-   **Rust**: For performance and reliability.
+-   **Dapr**: Used for the actor model, which provides stateful, addressable instances for each query.
+-   **Redis Streams**: The primary mechanism for ordered, reliable data ingestion into the query engine.
+-   **MongoDB**: The persistent backend for the `view-svc` to store and serve materialized query results.
+
+## 2. Component Interaction Diagram
+
+```mermaid
+graph TD
+    subgraph "Data Ingestion"
+        A[Drasi Source's Change Dispatcher] -->|HTTP POST| B["publish-api"];
+        B -->|Writes to Stream| C[Change Queue Redis Stream];
+    end
+
+    subgraph "Core Query Processing"
+        C -->|Reads from Change Queue Stream| D["query-host"];
+        D -- "Hosts Engine" --> E["drasi-core"];
+        E -- "Publishes Results" --> F[Dapr Pub/Sub];
+    end
+
+    subgraph "Result Viewing"
+        F -->|Subscribes to Topic| G["view-svc"];
+        G -- "Persists View" --> H[MongoDB];
+        I[Client Application] -->|HTTP GET| G;
+    end
+```
+
+## 3. Component Directory Guide
+
+-   **`publish-api/`**: A stateless service providing an HTTP endpoint for data ingestion. It receives data from Drasi Sources (`Change Dispatcher` component) and publishes it to a Redis Stream for consumption by the `query-host`.
+
+-   **`query-host/`**: A Dapr-enabled service that hosts and executes continuous queries. It uses the Dapr actor model to manage the lifecycle of each query as a stateful, addressable entity. It consumes data from Redis, processes it using the embedded `drasi-core` engine, and publishes results.
+
+-   **`view-svc/`**: A service that materializes and serves the results of continuous queries. It subscribes to result topics from the `query-host`, stores the data in MongoDB using a temporal model, and exposes an HTTP API for clients to query the current or historical state of a view.
+
+## 4. Build and Test
+
+The top-level `Makefile` orchestrates the build for all services in this workspace. It delegates targets (`docker-build`, `test`, `lint-check`) to the `Makefile` in each subdirectory.

query-container/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

query-container/publish-api/AGENTS.md

@@ -0,0 +1,30 @@
+# AGENTS.md
+
+Publish-API is an `axum` web-service that receives data via HTTP and publishes it to a Redis stream. It is invoked by `Change Dispatcher` component of Drasi `Source`s.
+
+-   **Configuration**: Reads `QUERY_NODE_ID`, `REDIS_BROKER`, and `PORT` from environment variables. It will exit if `QUERY_NODE_ID` is not set.
+
+-   **Publisher Initialization**: Creates a `Publisher` instance that connects to the specified Redis broker and publishes to a topic derived from the `QUERY_NODE_ID` (e.g., `{QUERY_NODE_ID}-publish`).
+
+-   **HTTP Endpoints**: 
+    -   `/change`: Handles incoming change events.
+    -   `/data`: Handles incoming data.
+
+-   **Publishing**: Publishes messages containing the incoming data, along with some tracing metadata like, to the configured Redis stream using the `XADD` command.
+
+## Mermaid Diagram
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant Publish API (axum)
+    participant Publisher
+    participant Redis
+
+    Client->>Publish API (axum): POST /change (body, headers)
+    Publish API (axum)->>Publisher: publish(body, trace_headers)
+    Publisher->>Redis: XADD topic * data <body_string> + <metadata>
+    Redis-->>Publisher: OK
+    Publisher-->>Publish API (axum): Ok(())
+    Publish API (axum)-->>Client: 200 OK
+```

query-container/publish-api/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

query-container/query-host/AGENTS.md

@@ -0,0 +1,78 @@
+# AGENTS.md: `query-host`
+
+## 1. Purpose & Architectural Intent
+
+This directory contains the `query-host` service, the central component of the `query-container`.
+
+**Core Intent**: To act as a robust, Dapr-enabled host for the `drasi-core` continuous query engine. It is responsible for managing the complete lifecycle of individual queries, making each query a stateful, addressable, and resilient entity.
+
+**Architectural Strategy**:
+-   **Dapr Actor Model**: Each continuous query is instantiated as a Dapr `QueryActor`. This pattern is used to:
+    -   Manage the state of each query independently (e.g., its configuration, status).
+    -   Provide a reliable, addressable endpoint for managing the query's lifecycle (configure, deprovision, reconcile).
+    -   Ensure resilience, as Dapr can automatically reactivate actors on failure.
+-   **Decoupled Worker Logic**: The `QueryActor` spawns a `QueryWorker` which contains the actual data processing logic. This separates the actor's lifecycle management from the core task of running the query.
+
+## 2. Core Dependencies & Data Flow
+
+-   **`drasi-core` (Git Submodule)**: This is the high-performance, embeddable Rust library that performs the actual continuous query evaluation. The `query-host` acts as a runtime environment for it.
+-   **Redis Streams**: The `QueryWorker` consumes a dedicated Redis Stream for each query, ensuring ordered, reliable processing of incoming `ChangeEvent` messages. `publish-api` service publushes the Change Events to the Redis stream.
+-   **Dapr Pub/Sub**: After processing data through `drasi-core`, the `QueryWorker` publishes the resulting diffs to a separate Dapr pub/sub topic for consumption by downstream services like the `view-svc`.
+
+```mermaid
+graph TD
+    subgraph Dapr Sidecar
+        A[Dapr Runtime]
+    end
+
+    subgraph Query Host Container
+        B[DaprHttpServer]
+        C[QueryActor]
+        D[QueryWorker]
+        E[drasi-core Engine]
+    end
+
+    F[Input Redis Stream from Publish API]
+    G[Output Dapr Pub/Sub]
+    H[Query API component of Source]
+
+    A <--> B;
+    B -- "Manages Lifecycle" --> C;
+    C -- "Spawns & Supervises" --> D;
+    D -- "Uses" --> E;
+    D -- "Consumes From" --> F;
+    E -- "Publishes To" --> G;
+    D -- "Bootstraps From" --> H;
+```
+
+```mermaid
+graph TD
+    subgraph Dapr Actor Runtime
+        A[QueryActor]
+    end
+
+    subgraph QueryWorker Thread
+        B[QueryWorker] -- Manages --> C{drasi-core Engine}
+        D[RedisChangeStream] -- Feeds --> B
+        C -- Publishes to --> E[ResultPublisher]
+        C -- Schedules --> F[FutureQueue]
+    end
+
+    G[Query API in Sources]
+    H[Redis Stream]
+    I[Pub/Sub Topic]
+
+    A -- Spawns/Manages --> B
+    B -- Bootstraps from --> G
+    D -- Reads from --> H
+    E -- Publishes to --> I
+    F -- Consumed by --> FutureConsumer -- Publishes back to --> H
+
+```
+
+## 3. Key Abstractions
+
+-   **`QueryActor`**: The Dapr actor responsible for state management and lifecycle of a single continuous query.
+-   **`QueryWorker`**: The core processing loop for a query. It bootstraps data from sources, consumes the change stream, and drives the `drasi-core` engine.
+-   **`IndexFactory`**: A factory for providing pluggable storage backends (`ElementIndex`, `ResultIndex`, `FutureQueue`) to `drasi-core`, allowing for different persistence strategies (in-memory, Redis, RocksDB).
+-   **`MiddlewareTypeRegistry`**: Enables extensible, declarative data transformation pipelines for pre-processing source data before it reaches the query engine.

query-container/query-host/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

query-container/query-host/src/change_stream/AGENTS.md

@@ -0,0 +1,3 @@
+# AGENTS.md: `query-container/query-host/src/change_stream`
+
+Implements a sequential change stream consumer on top of Redis Streams ensuring that messages are processed in order and acknowledges them only after successful processing. Goal is to provide a reliable, ordered message consumption mechanism from a Redis stream, abstracting away the complexities of Redis Streams consumer groups, pending messages, and acknowledgements.

query-container/query-host/src/change_stream/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

query-container/view-svc/AGENTS.md

@@ -0,0 +1,33 @@
+# AGENTS.md: `view-svc`
+
+## 1. Purpose & Architectural Intent
+
+View Service is responsible for creating, managing, and serving materialized views of Drasi continuous query results.
+
+**Core Intent**: To provide a durable, queryable, and time-aware view of the output from a continuous query. It acts as the primary interface for client applications to consume query results.
+
+**Architectural Strategy**:
+-   **Dapr Actor Model**: Each view is managed by a Dapr `ViewActor`. This provides a stateful, addressable endpoint for each view's lifecycle (configure, deprovision) and encapsulates its state (e.g., retention policy).
+-   **Decoupled Worker Logic**: The `ViewActor` spawns a `ViewWorker` that runs in the background, consuming the result stream for a specific query and updating the materialized view in the database. This decouples the actor's lifecycle from the data processing task.
+-   **Temporal Data Model**: The `MongoViewStore` implements a temporal data model. Each result row is stored with `validFrom` and `validTo` timestamps. This allows clients to query the view not just for its current state, but for its state at any point in time.
+-   **`RetentionPolicy`**: A key configuration that dictates how historical data is managed in the view (e.g., `Latest`, `Expire`, `All`). A background garbage collection task in the `MongoViewStore` enforces this policy.
+
+## 2. Core Dependencies & Data Flow
+
+-   **Dapr Pub/Sub**: The `ViewWorker` subscribes to a Dapr pub/sub topic to receive `ResultEvent` streams from a `query-host`.
+-   **MongoDB**: The `MongoViewStore` uses MongoDB as its persistence layer. It creates a separate collection for each view to store the temporal documents.
+-   **Axum**: An Axum-based HTTP server exposes an API for clients to query the view.
+
+```mermaid
+graph TD
+    A[query-host] -- Publishes --> B[Dapr Pub/Sub];
+    
+    subgraph view-svc
+        C[ViewWorker] -- Subscribes to --> B;
+        C -- "Writes to" --> D[MongoViewStore];
+        D -- "Uses" --> E[MongoDB];
+        F[Axum API] -- "Reads from" --> D;
+    end
+
+    G[Client Application] -- "GET /:query_id" --> F;
+```

query-container/view-svc/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md