Skip to content

Commit 94afc43

Browse files
amansinghoriginalamansinghoriginal
committed
Add AGENTS.md for drasi, sources, reactions
Signed-off-by: Aman Singh <[email protected]>
1 parent 8056f5c commit 94afc43

File tree

8 files changed

+708
-0
lines changed

8 files changed

+708
-0
lines changed

‎AGENTS.md‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# AGENTS.md: Drasi Platform
2+
3+
## 1. High-Level Overview
4+
5+
- **Purpose**: Drasi is a platform for building real-time, change-driven systems using continuous queries.
6+
- **Architecture**: Polyglot microservices system designed for Kubernetes.
7+
- **Core Technologies**: Rust (backend), Go (CLI), Dapr (inter-service communication).
8+
- **Repository**: `https://github.com/drasi-project/drasi-platform/`.
9+
- **Contribution Workflow**: Fork repo, commit to your fork and submit pull request.
10+
11+
### Architectural Flow Diagram
12+
13+
This diagram shows two primary flows: management (solid lines) and data pipeline (dashed lines).
14+
15+
```mermaid
16+
graph TD
17+
subgraph "User Interaction"
18+
User["User/Developer"] --> CLI["cli/"];
19+
end
20+
21+
subgraph "Platform Management"
22+
CLI -->|Manages Resources via API| ControlPlane["control-planes/"];
23+
ControlPlane -->|Deploys & Configures| DeployedServices;
24+
end
25+
26+
subgraph "Deployed Services (on Kubernetes)"
27+
direction LR
28+
DeployedServices("Sources, Query Containers, Reactions");
29+
end
30+
31+
subgraph "Core Data Pipeline"
32+
direction LR
33+
ExternalDataSource["External Data Source"] -.-> Sources["sources/"];
34+
Sources -.-> QueryContainer["query-container/"];
35+
QueryContainer -.-> Reactions["reactions/"];
36+
Reactions -.-> ExternalSystem["External System (e.g., API, DB)"];
37+
end
38+
```
39+
40+
## 2. Critical Development Notes
41+
42+
### **Git Submodule: `drasi-core`**
43+
44+
The `query-container/query-host` service depends on the `drasi-core` git submodule. Before building the `query-container`, the submodule **must** be initialized. Other components (`reactions`, `sources`, `control-planes`) can be built without it.
45+
46+
- **Location**: `query-container/query-host/drasi-core/`
47+
- **Source Repository**: `https://github.com/drasi-project/drasi-core`
48+
- **Initialization Command**: `git submodule update --init --recursive`
49+
50+
Failure to initialize will cause compilation errors in `query-container`. For details on the engine, see the `README.md` within the submodule directory.
51+
52+
### **Data Contracts: `typespec/`**
53+
54+
The `typespec/` directory is the **single source of truth** for all inter-service data contracts.
55+
56+
- **DO NOT** edit generated model files elsewhere.
57+
- All data model changes **must** be made in the source `.tsp` files.
58+
- After editing `.tsp` files, run the full generation workflow:
59+
1. `npm install && npm run build` (in `typespec/`).
60+
2. Run `quicktype` commands (defined in component `Makefile`s) to generate language-specific code from the updated JSON Schema.
61+
62+
Failure to follow this workflow will cause data contract mismatches.
63+
64+
## 3. Component Directory Guide
65+
66+
- **`cli/`**: Go CLI (`drasi`) for platform installation and management.
67+
- **`control-planes/`**: Rust backend. Contains the `mgmt_api` and the `kubernetes_provider`.
68+
- **`query-container/`**: Rust microservices for the continuous query execution environment (`publish-api`, `query-host`, `view-svc`).
69+
- **`reactions/`**: Rust microservices that act on query results. Contains built-in reactions and SDKs.
70+
- **`sources/`**: Rust microservices that ingest data from external systems. Contains built-in sources and SDKs.
71+
- **`typespec/`**: Source of truth for all data contracts.
72+
- **`e2e-tests/`**: Jest/TypeScript end-to-end tests.
73+
74+
## 4. Build and Test
75+
76+
The root `Makefile` provides top-level targets for the primary development workflows:
77+
- **`make docker-build`**: Builds all container images for the platform.
78+
- **`make test`**: Runs the test suites for all components.
79+
- **`make lint-check`**: Performs all code quality and lint checks.
80+
81+
For local Kubernetes workflows, `make kind-load` and `make k3d-load` are available to load built images into a cluster.
82+
83+
The build system uses a recursive `make` structure. The root `Makefile` delegates targets (e.g., `docker-build`, `test`) to Makefiles in component subdirectories (`control-planes/`, `query-container/`, etc.), which in turn delegate to their own subdirectories. This orchestrates a full repository build from the top down.
84+
85+
## 5. Core Architecture & Runtime
86+
87+
- **Pluggable Resource Providers**: The control plane is platform-agnostic. The `mgmt_api` communicates with a resource provider via a shared contract (`resource_provider_api`). The `kubernetes_provider` is the current implementation, translating Drasi resources into Kubernetes objects. This design allows for future platform providers.
88+
89+
- **Dapr Integration**: Dapr is a mandatory dependency for the backend, handling all inter-service communication (Pub/Sub) and resource lifecycle management (Dapr Actors). `drasi init` installs Dapr if needed.
90+
91+
- **Deployment Modes**:
92+
- **Kubernetes**: The standard deployment target. `drasi init` installs into the current `kubectl` context.
93+
- **Docker**: `drasi init --docker` provides a self-contained environment using a K3d (Kubernetes in Docker) cluster.
94+
95+
- **User Workflow**:
96+
1. Deploy the platform using `drasi init`.
97+
2. Define `Source`, `ContinuousQuery`, and `Reaction` resources in YAML.
98+
3. Deploy and manage resources using `drasi apply -f <file>.yaml`.

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
AGENTS.md

‎reactions/AGENTS.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# AGENTS.md: Drasi Reactions
2+
3+
## 1. Architectural Role
4+
5+
- **Position in Pipeline**: Reactions are the final stage in the Drasi data processing pipeline: **Sources -> Continuous Queries -> Reactions**.
6+
- **Core Function**: They are event-driven microservices that subscribe to the real-time stream of changes (add, update, delete) from Continuous Queries and execute actions on external systems.
7+
- **Responsibilities**:
8+
- **Integration**: Act as a bridge between Drasi and external systems (e.g., message brokers, databases, APIs).
9+
- **Action**: Execute business logic based on query result changes.
10+
- **Decoupling**: Separate the change detection logic (the "what" in a query) from the action logic (the "how" in a reaction).
11+
12+
## 2. Core Abstraction: The `ReactionProvider` Model
13+
14+
The architecture separates a reaction's *type definition* from its *instance configuration* using two resource types: `ReactionProvider` and `Reaction`.
15+
16+
### `ReactionProvider`
17+
- **Role**: Type Definition / Template.
18+
- **Purpose**: A cluster-level resource that registers a **type** of reaction with the Drasi control plane. It serves as a template and validation schema for all reactions of its kind.
19+
- **Defines**:
20+
- Container image for the reaction's implementation.
21+
- Default Dapr settings and service configurations.
22+
- A **`config_schema` (JSON Schema)** which serves as a validation contract for all `Reaction` instances of this type. The `mgmt_api` enforces this schema during creation and updates.
23+
- **Lifecycle**: Registered once per reaction type. Built-in providers are registered during `drasi init`.
24+
25+
### `Reaction`
26+
- **Role**: Instance / Configured Deployment.
27+
- **Purpose**: A specific, configured instance of a `ReactionProvider`. This is the running microservice pod.
28+
- **Defines**:
29+
- A reference (`kind`) to its `ReactionProvider`.
30+
- Specific `properties` (e.g., connection strings) that conform to the provider's `config_schema`.
31+
- The list of `queries` it subscribes to.
32+
- **Lifecycle**: Created, updated, and deleted as needed by users via the `drasi` CLI.
33+
34+
- **Key Takeaway**: This model enables a single containerized reaction implementation to be deployed multiple times, each with a unique configuration, subscription list, and lifecycle. For fully annotated YAML examples of both `ReactionProvider` and `Reaction` resources, refer to the `README.md` file in this directory.
35+
36+
## 3. Deployment Orchestration Flow
37+
38+
Applying a `Reaction` manifest via `drasi apply` initiates the following platform-agnostic orchestration flow:
39+
40+
1. **CLI to Management API**: The `drasi` CLI sends the `Reaction` manifest to the `/v1/reactions/{reaction-name}` endpoint.
41+
2. **Validation & Persistence**: The Management API validates the `spec.properties` against the `config_schema` from the corresponding `ReactionProvider` and persists the desired state.
42+
3. **Delegation to Resource Provider**: The Management API delegates the provisioning task to the active resource provider (e.g., `kubernetes_provider`) using the platform-agnostic `resource_provider_api` contract.
43+
4. **Platform-Specific Resource Generation**: The active resource provider translates the abstract `Reaction` spec into concrete resources. For the default `kubernetes_provider`, this includes:
44+
- Kubernetes `Deployment`
45+
- Kubernetes `ConfigMap` (for query configurations)
46+
- Kubernetes `Service`
47+
- Dapr `Component` (for `pubsub.redis` subscription)
48+
5. **Apply to Platform**: The resource provider applies the generated manifests to the target platform's API (e.g., Kubernetes API).
49+
50+
## 4. Configuration at Runtime
51+
52+
A running reaction container accesses its configuration via two mechanisms:
53+
54+
- **Environment Variables**: All key-value pairs from the `spec.properties` section of the `Reaction` manifest are injected as environment variables. Values from Kubernetes Secrets are securely resolved and injected.
55+
- **Volume-Mounted Files**: The `spec.queries` map is stored in a `ConfigMap` and mounted as a file volume at `/etc/queries`.
56+
57+
## 5. Reaction Runtime Data Flow
58+
59+
This diagram illustrates the data flow *inside* a running Reaction pod.
60+
61+
```mermaid
62+
graph TD
63+
A[Continuous Query] -->|Change Results| B[Dapr Pub/Sub]
64+
B -->|Event Stream| C[Reaction Pod]
65+
C -->|Deserialize| D[Change Events]
66+
D -->|Apply Query Config| E[Event Handler]
67+
E -->|Execute Action| F[External System]
68+
69+
G[ConfigMap] -->|Query Configuration| C
70+
H[Secrets] -->|Credentials| C
71+
```
72+
73+
## 6. Examples of Reactions
74+
75+
- **`PostDaprPubSub`**: Forwards query results to a Dapr Pub/Sub topic for decoupled microservice architectures.
76+
- **`SyncDaprStateStore`**: Materializes query results into a Dapr state store, creating a continuously updated data cache.
77+
- **`SignalR`**: Exposes a SignalR endpoint to broadcast query changes for real-time UIs.
78+
79+
## 7. Reaction SDKs
80+
81+
Drasi provides SDKs for multiple languages (located in `reactions/sdk/`) for the development of custom reactions.

‎reactions/CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
AGENTS.md

0 commit comments

Comments
 (0)