diff --git a/docs/services/index.md b/docs/services/index.md new file mode 100644 index 00000000..4e6428f7 --- /dev/null +++ b/docs/services/index.md @@ -0,0 +1,100 @@ +# Supporting Services (Beta) + +The pgEdge Control Plane lets you run services alongside your +databases. Services are applications that attach to a database, run on +any host in the cluster, and connect via automatically-managed +database credentials. + +## What Are Supporting Services? + +A supporting service is an application that runs alongside a database. +Each service instance runs on a single host and receives its own set of +database credentials scoped to that instance. The Control Plane supports +the following service types: + +- The [pgEdge Postgres MCP Server](mcp.md) connects AI agents and + LLM-powered applications to your database, enabling natural language + queries and AI-powered data access. +- The pgEdge RAG Server *(coming soon)* enables retrieval-augmented + generation workflows using your database as a knowledge store. +- PostgREST *(coming soon)* automatically generates a REST API from + your PostgreSQL schema, making your data accessible over HTTP without + writing backend code. + +## Service Instances + +When you add a service to a database, the Control Plane creates one +service instance per host listed in the service's `host_ids`. Each +instance runs on a single host and receives its own database +credentials. Services can run on any host in the cluster; they do not +need to be co-located with database instances. + +The following table describes the lifecycle states for service +instances: + +| State | Description | +|-------|-------------| +| `creating` | The Control Plane is provisioning the service instance. | +| `running` | The service instance is healthy and operational. | +| `failed` | The service instance exited or failed its health check. | +| `deleting` | The Control Plane is removing the service instance. | + +## Deployment Topologies + +Services are independent of your database node topology, so you can +place service instances on any host in the cluster. The following +deployment patterns are common: + +- In a co-located topology, the service runs on the same host as a + database instance, which minimizes network latency between the + service and Postgres. +- In a separate-host topology, the service runs on a dedicated host + with no database instance, which isolates the service workload from + the database. +- In a multiple-instances topology, one service instance runs per host + for redundancy or regional proximity; each instance receives its own + credentials and connects to the database independently. + +In the following example, the service runs on the same host as the +database node (`host-1`): + +```json +"nodes": [ { "name": "n1", "host_ids": ["host-1"] } ], +"services": [ { ..., "host_ids": ["host-1"] } ] +``` + +In the following example, the service runs on a dedicated host +(`host-3`) with no database instance: + +```json +"nodes": [ { "name": "n1", "host_ids": ["host-1"] }, + { "name": "n2", "host_ids": ["host-2"] } ], +"services": [ { ..., "host_ids": ["host-3"] } ] +``` + +In the following example, the service runs on each database host, +creating one instance per host for redundancy: + +```json +"nodes": [ { "name": "n1", "host_ids": ["host-1"] }, + { "name": "n2", "host_ids": ["host-2"] } ], +"services": [ { ..., "host_ids": ["host-1", "host-2"] } ] +``` + +## Database Credentials + +Each service instance is automatically provisioned with two dedicated +database users. The Control Plane manages these credentials; you do not +need to create or rotate them manually. The credentials are: + +- `svc_{service_id}_ro` is a read-only user with read access to the + database; this user is the default for most service types. +- `svc_{service_id}_rw` is a read-write user with read and write access + to the database; this user is provisioned when the service needs + read/write access. + +## Next Steps + +To add a service to a database, see [Managing Services](managing.md). +Then refer to the page for your specific service type for configuration +details. diff --git a/docs/services/managing.md b/docs/services/managing.md new file mode 100644 index 00000000..cbb1dec4 --- /dev/null +++ b/docs/services/managing.md @@ -0,0 +1,167 @@ +# Managing Services + +Services are declared as part of your database spec. You add, update, +and remove services by modifying the `services` array in a +`create_database` or `update_database` request. See the +[Services Overview](index.md) for a conceptual introduction. + +## Service Spec Fields + +Each service in the `services` array is declared using a service spec. +The following table describes the fields in a service spec: + +| Field | Type | Required | Description | +|-------|------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `service_id` | string | Yes | A unique identifier for this service within the database. Used in credential names (`svc_{service_id}_ro` / `svc_{service_id}_rw`). | +| `service_type` | string | Yes | The type of service to run. One of: `mcp`, `rag`, `postgrest`. | +| `version` | string | Yes | The service version in semver format (e.g., `1.0.0`) or the literal `latest`. | +| `host_ids` | array | Yes | The IDs of the hosts to run this service on. One instance is created per host. | +| `config` | object | Yes | Service-type-specific configuration. See the page for your service type for valid fields. | +| `port` | integer | No | Host port to publish the service on. Set to `0` to let Docker assign a random port. When omitted, the service is not reachable from outside the Docker network. | +| `cpus` | string | No | CPU limit for the service container. Accepts a decimal (e.g., `"0.5"`) or millicpu suffix (e.g., `"500m"`). Defaults to container defaults if unspecified. | +| `memory` | string | No | Memory limit for the service container in SI or IEC notation (e.g., `"512M"`, `"1GiB"`). Defaults to container defaults if unspecified. | +| `database_connection` | object | No | Optional routing configuration for how the service connects to the database. See [Database Connection Routing](#database-connection-routing). | + +## Adding a Service + +Include a `services` array in your database spec when creating or +updating a database. In the following example, a `curl` command creates +a single-node database with one MCP service instance: + +=== "curl" + + ```sh + curl -X POST http://host-1:3000/v1/databases \ + -H 'Content-Type: application/json' \ + --data '{ + "id": "example", + "spec": { + "database_name": "example", + "nodes": [ + { "name": "n1", "host_ids": ["host-1"] } + ], + "services": [ + { + "service_id": "mcp-server", + "service_type": "mcp", + "version": "latest", + "host_ids": ["host-1"], + "port": 8080, + "config": { + "llm_enabled": true, + "llm_provider": "anthropic", + "llm_model": "claude-sonnet-4-5", + "anthropic_api_key": "sk-ant-..." + } + } + ] + } + }' + ``` + +The response includes a task ID you can use to track progress. See +[Tasks & Logs](../using/tasks-logs.md) for details. + +## Updating a Service + +To update a service's configuration, submit a `POST` request to +`/v1/databases/{database_id}` with the modified service spec in the +`services` array. + +!!! important + + The `services` array in an update request is declarative; it + replaces the complete list of services for the database. To keep an + existing service running unchanged, include its current spec + alongside any new or modified entries. + +In the following example, a `curl` command updates the MCP service to +use a different model: + +=== "curl" + + ```sh + curl -X POST http://host-1:3000/v1/databases/example \ + -H 'Content-Type: application/json' \ + --data '{ + "spec": { + "database_name": "example", + "nodes": [ + { "name": "n1", "host_ids": ["host-1"] } + ], + "services": [ + { + "service_id": "mcp-server", + "service_type": "mcp", + "version": "latest", + "host_ids": ["host-1"], + "port": 8080, + "config": { + "llm_enabled": true, + "llm_provider": "anthropic", + "llm_model": "claude-opus-4-5", + "anthropic_api_key": "sk-ant-..." + } + } + ] + } + }' + ``` + +## Removing a Service + +To remove a service, submit an update request that omits the service +from the `services` array. The Control Plane stops and deletes all +service instances for that service and revokes its database credentials. + +!!! warning + + Removing a service is irreversible. The Control Plane deletes all + service instances, their configuration, and their data directories. + Database credentials for the service are revoked. Any clients + connected to the service lose access immediately. + +## Checking Service Status + +To check the current state of your service instances, retrieve the +database and inspect the `service_instances` field in the response. In +the following example, a `curl` command retrieves the database: + +=== "curl" + + ```sh + curl http://host-1:3000/v1/databases/example + ``` + +Each service instance in the response includes a `state` field. See the +[Services Overview](index.md#service-instances) for a description of +each state. + +## Database Connection Routing + +By default, the Control Plane builds a connection string that includes +all database nodes, with the local node listed first. You can override +this behavior using the `database_connection` field in the service spec. +The following table describes the `database_connection` fields: + +| Field | Type | Description | +|-------|------|-------------| +| `target_nodes` | array of strings | An ordered list of node names to include in the connection string. Nodes are tried in the order listed. | +| `target_session_attrs` | string | Overrides the libpq `target_session_attrs` parameter. Valid values: `primary`, `prefer-standby`, `standby`, `read-write`, `any`. | + +In the following example, the `database_connection` field routes the +service to the `n1` node only: + +=== "curl" + + ```sh + "database_connection": { + "target_nodes": ["n1"], + "target_session_attrs": "primary" + } + ``` + +!!! tip + + Use `database_connection` when your service needs to read from a + specific node or enforce write routing to the primary. diff --git a/docs/services/mcp.md b/docs/services/mcp.md new file mode 100644 index 00000000..6cbc783a --- /dev/null +++ b/docs/services/mcp.md @@ -0,0 +1,413 @@ +# pgEdge Postgres MCP Server + +The MCP service runs a [Model Context Protocol](https://modelcontextprotocol.io) +server alongside your database. AI agents and LLM-powered applications +use the MCP server to query and interact with your data. For more +information, see the +[pgEdge Postgres MCP](https://github.com/pgEdge/pgedge-postgres-mcp) +project. + +## Overview + +The Control Plane provisions an MCP server container on each specified +host. The server connects to the database using automatically-managed +credentials. AI agents call the server's tools to query data, inspect +schemas, run EXPLAIN plans, and perform vector similarity searches. + +See [Managing Services](managing.md) for instructions on adding, +updating, and removing services. The sections below cover MCP-specific +configuration. + +## Configuration Reference + +All configuration fields are provided in the `config` object of the +service spec. + +### LLM Proxy + +The MCP server can optionally act as an LLM proxy for the built-in web +client and direct HTTP chat. When the LLM proxy is disabled (the +default), the MCP server still exposes all tools over HTTP. AI clients +such as Claude Desktop or Cursor connect via the MCP protocol and +supply their own LLM. The following table describes the LLM proxy +configuration fields: + +| Field | Type | Default | Description | +|------------------------|---------|---------|-------------| +| `llm_enabled` | boolean | `false` | Set to `true` to enable the LLM proxy. When `false`, the fields below must not be provided. | +| `llm_provider` | string | — | The LLM provider to use. One of: `anthropic`, `openai`, `ollama`. Required when `llm_enabled` is `true`. | +| `llm_model` | string | — | The model name for the selected provider (e.g., `claude-sonnet-4-5`, `gpt-4o`, `llama3.2`). Required when `llm_enabled` is `true`. | +| `anthropic_api_key` | string | — | Your Anthropic API key. Required when `llm_provider` is `anthropic`. | +| `openai_api_key` | string | — | Your OpenAI API key. Required when `llm_provider` is `openai`. | +| `ollama_url` | string | — | The base URL of your Ollama server (e.g., `http://ollama-host:11434`). Required when `llm_provider` is `ollama`. | + +### Security + +The security fields control database access level and initial +authentication for the MCP server. The following table describes the +security configuration fields: + +| Field | Type | Default | Description | +|------------------|---------|---------|-------------| +| `allow_writes` | boolean | `false` | When `true`, the service connects using the read-write database user (`svc_{service_id}_rw`) and the `query_database` tool can execute write statements. When `false`, the read-only user (`svc_{service_id}_ro`) is used and write statements are rejected at the database level. | +| `init_token` | string | — | A bootstrap token for initial access to the MCP server. See [Bootstrapping](#bootstrapping). | +| `init_users` | array | — | Initial user accounts to create on the MCP server. See [Bootstrapping](#bootstrapping). | + +### Tools + +The MCP server exposes tools to AI agents that enable querying, schema +inspection, vector search, and other operations. All tools are enabled +by default; set the corresponding `disable_*` field to `true` to turn +off a specific tool. The following table describes the available tools: + +| Tool | Disable Flag | Description | +|-------------------------|---------------------------------|-------------| +| `query_database` | `disable_query_database` | Execute SQL queries against the database. Writes are only permitted when `allow_writes` is `true`. | +| `get_schema_info` | `disable_get_schema_info` | Inspect tables, columns, and indexes. | +| `similarity_search` | `disable_similarity_search` | Perform vector similarity search. Requires embeddings to be configured. | +| `execute_explain` | `disable_execute_explain` | Run `EXPLAIN ANALYZE` on a query. | +| `generate_embedding` | `disable_generate_embedding` | Generate a vector embedding for a given text. Requires embeddings to be configured. | +| `search_knowledgebase` | `disable_search_knowledgebase` | Search a configured knowledge base. | +| `count_rows` | `disable_count_rows` | Count rows matching a condition. | + +### Embeddings + +Embedding support enables the `similarity_search` and +`generate_embedding` tools. All embedding fields are optional, but +`embedding_model` is required when `embedding_provider` is set. The +following table describes the embedding configuration fields: + +| Field | Type | Description | +|------------------------|--------|-------------| +| `embedding_provider` | string | The embedding provider. One of: `voyage`, `openai`, `ollama`. | +| `embedding_model` | string | The embedding model name (e.g., `voyage-3`, `text-embedding-3-small`, `nomic-embed-text`). Required when `embedding_provider` is set. | +| `embedding_api_key` | string | API key for the embedding provider. Required for `voyage` and `openai` providers. | + +### LLM Tuning + +The LLM tuning fields control the behavior of the LLM proxy and are +only valid when `llm_enabled` is `true`. The following table describes +the LLM tuning fields: + +| Field | Type | Range | Description | +|--------------------|---------|------------------|-------------| +| `llm_temperature` | number | `0.0`–`2.0` | Controls randomness in LLM responses. Lower values produce more deterministic output. | +| `llm_max_tokens` | integer | Positive integer | Maximum number of tokens in the LLM response. | + +### Connection Pool + +The connection pool fields control how many database connections the +MCP server maintains. The following table describes the connection pool +configuration fields: + +| Field | Type | Description | +|--------------------|---------|-------------| +| `pool_max_conns` | integer | Maximum number of database connections the service maintains in its pool. Must be a positive integer. | + +## Bootstrapping + +You can use `init_token` and `init_users` to establish initial access +when provisioning an MCP service for the first time. + +The `init_token` field sets a bootstrap token for authenticating with +the MCP server. The bootstrap token is useful for automating initial +setup or connecting a client immediately after provisioning. + +The `init_users` field creates one or more user accounts during +provisioning. In the following example, the `init_users` field defines +two user accounts: + +```json +"init_users": [ + { "username": "alice", "password": "s3cr3t" }, + { "username": "bob", "password": "s3cr3t2" } +] +``` + +The Control Plane hashes tokens (SHA-256) and passwords (bcrypt) before +writing them to disk. The MCP server stores these files on a persistent +bind-mount volume that survives container restarts. After bootstrap, the +MCP server owns these files; you manage additional tokens and users +through the MCP server's native CLI or API. + +!!! warning + + `init_token` and `init_users` can only be set when the service is + first created. Providing either field in a subsequent update request + will be rejected. Store your bootstrap credentials before + provisioning; they cannot be retrieved or modified through the + Control Plane after the service is created. + +## Examples + +The following examples show how to configure the MCP service for common +use cases. + +### Minimal (No LLM) + +In the following example, a `curl` command provisions an MCP service +without the LLM proxy. The MCP server exposes all tools over HTTP, and +you connect via an MCP client that supplies its own LLM: + +=== "curl" + + ```sh + curl -X POST http://host-1:3000/v1/databases \ + -H 'Content-Type: application/json' \ + --data '{ + "id": "example", + "spec": { + "database_name": "example", + "nodes": [ + { "name": "n1", "host_ids": ["host-1"] } + ], + "services": [ + { + "service_id": "mcp-server", + "service_type": "mcp", + "version": "latest", + "host_ids": ["host-1"], + "port": 8080, + "config": { + "init_token": "my-bootstrap-token", + "init_users": [ + { "username": "alice", "password": "s3cr3t" } + ] + } + } + ] + } + }' + ``` + +### Anthropic (Claude) with LLM Proxy + +In the following example, a `curl` command enables the LLM proxy with +Anthropic as the provider: + +=== "curl" + + ```sh + curl -X POST http://host-1:3000/v1/databases \ + -H 'Content-Type: application/json' \ + --data '{ + "id": "example", + "spec": { + "database_name": "example", + "nodes": [ + { "name": "n1", "host_ids": ["host-1"] } + ], + "services": [ + { + "service_id": "mcp-server", + "service_type": "mcp", + "version": "latest", + "host_ids": ["host-1"], + "port": 8080, + "config": { + "llm_enabled": true, + "llm_provider": "anthropic", + "llm_model": "claude-sonnet-4-5", + "anthropic_api_key": "sk-ant-...", + "init_token": "my-bootstrap-token", + "init_users": [ + { "username": "alice", "password": "s3cr3t" } + ] + } + } + ] + } + }' + ``` + +### OpenAI with Embeddings + +In the following example, a `curl` command enables the LLM proxy with +OpenAI and configures embedding support: + +=== "curl" + + ```sh + curl -X POST http://host-1:3000/v1/databases \ + -H 'Content-Type: application/json' \ + --data '{ + "id": "example", + "spec": { + "database_name": "example", + "nodes": [ + { "name": "n1", "host_ids": ["host-1"] } + ], + "services": [ + { + "service_id": "mcp-server", + "service_type": "mcp", + "version": "latest", + "host_ids": ["host-1"], + "port": 8080, + "config": { + "llm_enabled": true, + "llm_provider": "openai", + "llm_model": "gpt-4o", + "openai_api_key": "sk-...", + "embedding_provider": "openai", + "embedding_model": "text-embedding-3-small", + "embedding_api_key": "sk-...", + "init_token": "my-bootstrap-token", + "init_users": [ + { "username": "alice", "password": "s3cr3t" } + ] + } + } + ] + } + }' + ``` + +### Ollama (Self-Hosted) + +In the following example, a `curl` command configures the MCP service +to use a self-hosted Ollama server for both the LLM and embeddings: + +=== "curl" + + ```sh + curl -X POST http://host-1:3000/v1/databases \ + -H 'Content-Type: application/json' \ + --data '{ + "id": "example", + "spec": { + "database_name": "example", + "nodes": [ + { "name": "n1", "host_ids": ["host-1"] } + ], + "services": [ + { + "service_id": "mcp-server", + "service_type": "mcp", + "version": "latest", + "host_ids": ["host-1"], + "port": 8080, + "config": { + "llm_enabled": true, + "llm_provider": "ollama", + "llm_model": "llama3.2", + "ollama_url": "http://ollama-host:11434", + "embedding_provider": "ollama", + "embedding_model": "nomic-embed-text" + } + } + ] + } + }' + ``` + +## Connecting to the MCP Server + +The MCP server accepts JSON-RPC 2.0 requests once the service instance +reaches the `running` state. Send requests to the following endpoint: + +```text +POST http://{host}:{port}/mcp/v1 +``` + +Replace `{host}` with the hostname of the host running the instance. +Replace `{port}` with the value from the `port` field of the service +spec. + +### Authenticating with an Init Token + +If you provisioned the service with an `init_token`, you can use the +token immediately as a Bearer token. In the following example, a `curl` +command calls the `get_schema_info` tool using the bootstrap token: + +=== "curl" + + ```sh + curl -sX POST http://host-1:8080/mcp/v1 \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer my-bootstrap-token" \ + -d '{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "get_schema_info", + "arguments": {} + } + }' | jq . + ``` + +### Authenticating with a User Account + +If you provisioned the service with `init_users`, authenticate using +the `authenticate_user` tool to obtain a session token. In the +following example, a `curl` command authenticates as user `alice`: + +=== "curl" + + ```sh + curl -sX POST http://host-1:8080/mcp/v1 \ + -H "Content-Type: application/json" \ + -d '{ + "jsonrpc": "2.0", + "id": 1, + "method": "tools/call", + "params": { + "name": "authenticate_user", + "arguments": { + "username": "alice", + "password": "s3cr3t" + } + } + }' | jq . + ``` + +A successful response returns a `session_token` you can use as a Bearer +token for subsequent requests: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "content": [ + { + "type": "text", + "text": "{\"expires_at\":\"...\",\"message\":\"Authentication successful\",\"session_token\":\"\",\"success\":true}" + } + ] + } +} +``` + +### Connecting with Claude Desktop + +You can connect Claude Desktop to the MCP server using `mcp-remote`. +Claude provides its own LLM; the MCP server only serves tools. This +works regardless of the `llm_enabled` setting. + +Add the following to your Claude Desktop config +(`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS): + +```json +{ + "mcpServers": { + "pgedge": { + "command": "npx", + "args": [ + "mcp-remote", + "http://{host}:{port}/mcp/v1", + "--header", + "Authorization: Bearer {token}" + ] + } + } +} +``` + +Replace `{host}` and `{port}` with the host and port of your MCP +service instance. Replace `{token}` with your `init_token` or a +session token from `authenticate_user`. + +Restart Claude Desktop to apply the configuration. The pgEdge MCP +tools will then appear in your conversations. diff --git a/mkdocs.yml b/mkdocs.yml index 7a48d08f..09e1ea52 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -78,6 +78,10 @@ nav: - Managing Database Instances: using/database-instances.md - Deleting a Database: using/delete-db.md - Migrating a Database: using/migrate-db.md + - Supporting Services (Beta): + - Overview: services/index.md + - Managing Services: services/managing.md + - pgEdge Postgres MCP Server: services/mcp.md - Troubleshooting: - Recovering a Control Plane Cluster: disaster-recovery/disaster-recovery.md - API: