Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
500 changes: 447 additions & 53 deletions Cargo.lock

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions crates/buzz-agent/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@ serde_json = { workspace = true }
serde_yaml = { workspace = true }
reqwest = { workspace = true, features = ["json", "rustls", "form"] }
rmcp = { version = "1", default-features = false, features = ["client", "transport-child-process"] }
# Maple (OpenSecret) confidential inference: attested, end-to-end encrypted
# transport to TEE-hosted models. `http`, `bytes`, and `futures-util` are for
# the SDK's request/response types and are already in its dependency tree.
opensecret = "3"
http = "1"
bytes = "1"
futures-util = { workspace = true }
arc-swap = "1"
getrandom = "0.4"
tracing = { workspace = true }
Expand Down
29 changes: 27 additions & 2 deletions crates/buzz-agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,12 @@ BUZZ_AGENT_PROVIDER=databricks \
DATABRICKS_HOST=https://dbc-...cloud.databricks.com \
DATABRICKS_MODEL=goose-claude-4-6-sonnet \
./target/release/buzz-agent

# Or Maple (OpenSecret) confidential inference, attested and end-to-end encrypted
BUZZ_AGENT_PROVIDER=maple \
MAPLE_API_KEY=... \
MAPLE_MODEL=llama3-3-70b \
./target/release/buzz-agent
```

That's the whole setup. The agent reads JSON-RPC frames from stdin, writes them to stdout, and logs to stderr.
Expand Down Expand Up @@ -135,7 +141,7 @@ Everything is environment variables. No flags, no config files. (We are a subpro

| Variable | Default | Notes |
|---|---|---|
| `BUZZ_AGENT_PROVIDER` | — | Required. `anthropic`, `openai`, `openrouter`, `databricks`, or `databricks_v2`. No implicit fallback — the agent errors at startup when this is unset. |
| `BUZZ_AGENT_PROVIDER` | — | Required. `anthropic`, `openai`, `openrouter`, `maple`, `databricks`, or `databricks_v2`. No implicit fallback — the agent errors at startup when this is unset. |
| `ANTHROPIC_API_KEY` | — | Required when provider=anthropic. |
| `ANTHROPIC_MODEL` | — | Required when provider=anthropic. |
| `ANTHROPIC_BASE_URL` | `https://api.anthropic.com` | |
Expand All @@ -147,6 +153,10 @@ Everything is environment variables. No flags, no config files. (We are a subpro
| `OPENROUTER_API_KEY` | — | Required when provider=openrouter. |
| `OPENROUTER_MODEL` | — | Required when provider=openrouter. Use OpenRouter's `vendor/model` id, e.g. `anthropic/claude-sonnet-4.5`. |
| `OPENROUTER_BASE_URL` | `https://openrouter.ai/api/v1` | |
| `MAPLE_API_KEY` | — | Required when provider=maple. |
| `MAPLE_MODEL` | — | Required when provider=maple. |
| `MAPLE_BASE_URL` | `https://enclave.trymaple.ai` | The enclave endpoint the opensecret SDK attests against. |
| `MAPLE_PCR0_ENVIRONMENT` | `production` | `production` \| `development`. Which PCR0 trust roots verify the enclave's attestation document. |
| `DATABRICKS_HOST` | — | Required when provider=databricks or provider=databricks_v2. |
| `DATABRICKS_MODEL` | — | Required when provider=databricks or provider=databricks_v2. |
| `DATABRICKS_TOKEN` | — | Optional static bearer escape hatch. If unset, Databricks uses browser OAuth + refresh cache. |
Expand Down Expand Up @@ -240,10 +250,11 @@ lifecycle hook — see [MCP_DRIVEN_HOOKS.md](../../docs/MCP_DRIVEN_HOOKS.md).
| Ollama | `openai` | `POST {base}/chat/completions` | llama3.1, qwen2.5-coder |
| Block Gateway | `openai` | `POST {base}/chat/completions` | gpt-5, claude |
| OpenRouter | `openrouter` | `POST {base}/chat/completions` | anything they route (extended-thinking replay, provider-agnostic tool calling) |
| Maple (OpenSecret) | `maple` | `POST {base}/v1/chat/completions` (attested + encrypted) | TEE-hosted open models (llama, deepseek, qwen, …) |
| Databricks | `databricks` | `POST {host}/serving-endpoints/{model}/invocations` | goose-claude-4-6-sonnet |
| Databricks AI Gateway v2 | `databricks_v2` | `POST {host}/ai-gateway/{provider}/v1/...` | databricks-gpt-5-5, databricks-claude-opus-4-7 |

If `BUZZ_AGENT_PROVIDER=anthropic` is selected without `ANTHROPIC_API_KEY`, `BUZZ_AGENT_PROVIDER=openai` is selected without `OPENAI_COMPAT_API_KEY`, or `BUZZ_AGENT_PROVIDER=openrouter` is selected without `OPENROUTER_API_KEY`, the agent returns an error — there is no implicit fallback to another provider.
If `BUZZ_AGENT_PROVIDER=anthropic` is selected without `ANTHROPIC_API_KEY`, `BUZZ_AGENT_PROVIDER=openai` is selected without `OPENAI_COMPAT_API_KEY`, `BUZZ_AGENT_PROVIDER=openrouter` is selected without `OPENROUTER_API_KEY`, or `BUZZ_AGENT_PROVIDER=maple` is selected without `MAPLE_API_KEY`, the agent returns an error — there is no implicit fallback to another provider.

`provider=openai` speaks two HTTP dialects: the [Responses API](https://platform.openai.com/docs/api-reference/responses) (`/v1/responses`, required for GPT-5 / o-series tool-calling on OpenAI's own service) and the [Chat Completions API](https://platform.openai.com/docs/api-reference/chat) (`/chat/completions`, the broadly-supported OpenAI-compatible wire format).

Expand All @@ -256,8 +267,22 @@ By default (`OPENAI_COMPAT_API=auto`) the agent picks **Responses** when `OPENAI
- `anthropic/*` models get Anthropic-style `cache_control` breakpoints injected on the system message and the last two user messages.
- Retryable statuses (429 and typed `provider_overloaded` 503) honor the documented `Retry-After` header (clamped to a small ceiling — see `RETRY_AFTER_CAP_SECS` in `llm.rs` — since the sleep happens outside `BUZZ_AGENT_LLM_TIMEOUT_SECS`); 502 and untyped 503 retry with jittered backoff instead. `401` is treated as an expired/invalid key and refreshed once, while `402` (no credits) and `403` (guardrail/moderation/permission) fail immediately without retry.

`provider=maple` speaks OpenAI's Chat Completions wire format, but every request travels through the [opensecret](https://crates.io/crates/opensecret) SDK instead of a plain HTTPS POST: the SDK verifies the enclave's AWS Nitro attestation document against pinned PCR0 trust roots, performs an X25519 key exchange with the attested enclave, and end-to-end encrypts the request and response bodies. The handshake runs lazily on the first request and re-runs transparently when the enclave session goes stale. One divergence from the shared OpenAI path: a `401`/`403` fails immediately without a refresh retry, since a static Maple API key cannot mint a different token.

`Provider` is a Rust `enum` with one `match` in `Llm::complete`. There is no trait, no `Box<dyn>`, no async-trait. Adding a provider is a `match` arm and one `body`/`parse` pair in `llm.rs`.

### Model discovery

`buzz-agent models` prints the provider's live model catalog as a JSON array of `{"id","name"}` objects on stdout. It reads the same provider env vars as the ACP server but does **not** require a model. Providers without a live agent-side catalog (Anthropic, OpenAI, OpenRouter; frontends list those over plain HTTP) print `[]`. Failures exit non-zero with the error on stderr.

For Maple, `MAPLE_API_KEY` is optional here: the enclave lists models for any attested session, so a picker can fill in before the user has a key. An enclave that still requires a credential answers a keyless call with `config: MAPLE_API_KEY required`.

```bash
BUZZ_AGENT_PROVIDER=databricks_v2 DATABRICKS_HOST=https://dbc-...cloud.databricks.com buzz-agent models
```

The per-provider dispatch lives in `catalog.rs` (`discover_models`) and also backs the `availableModels` list in the ACP `session/new` response, so a provider added there reaches every frontend, including Buzz Desktop's model picker.

## MCP Servers

The client passes MCP server specs in `session/new`. The agent spawns each one as a stdio subprocess, calls `tools/list`, and merges everything into a single tool catalog the LLM sees. Tool names are namespaced as `server__tool` (double underscore separator). Bare tool names containing `__` are rejected at registration.
Expand Down
196 changes: 187 additions & 9 deletions crates/buzz-agent/src/catalog.rs
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
//! Databricks model catalog discovery.
//! Live model-catalog discovery, per provider.
//!
//! Exposes [`discover_databricks_models`] — an async helper that lists
//! available models for the `databricks` and `databricks_v2` providers
//! without triggering a browser OAuth flow. Auth is acquired in-process via
//! [`build_token_source`](crate::llm::build_token_source):
//! [`discover_models`] is the dispatch point behind ACP `session/new` and
//! the `buzz-agent models` subcommand.
//!
//! Databricks ([`discover_databricks_models`]) lists endpoints for the
//! `databricks` and `databricks_v2` providers without opening a browser.
//! Auth comes from [`build_token_source`](crate::llm::build_token_source):
//!
//! - Static bearer (`DATABRICKS_TOKEN`): returned immediately.
//! - PKCE cache hit: returned from disk without a network round-trip.
//! - PKCE cache empty / no token: returns `Err(AgentError::LlmAuth)`.
//! - PKCE cache hit: read from disk, no network round-trip.
//! - PKCE cache empty, no token: `Err(AgentError::LlmAuth)`.
//!
//! This helper never opens a browser. Callers choose whether to reject, degrade,
//! or start a separate interactive authentication flow.
//! Maple's catalog is only reachable through the opensecret SDK's attested
//! transport, so its discovery lives here too.

use std::sync::Arc;

Expand Down Expand Up @@ -401,6 +403,113 @@ pub(crate) fn parse_v2_endpoints_page(
Ok((models, next_page_token))
}

// ---------------------------------------------------------------------------
// Provider-generic dispatch
// ---------------------------------------------------------------------------

/// Discover the live model catalog for `cfg.provider`.
///
/// `Ok(Some(models))` is a non-empty catalog. `Ok(None)` means the provider
/// has no agent-side catalog: the frontend lists its models itself over
/// plain OpenAI-compatible HTTP, or the configured model is the only option.
pub async fn discover_models(cfg: &Config) -> Result<Option<Vec<ModelEntry>>, AgentError> {
match cfg.provider {
Provider::Databricks | Provider::DatabricksV2 => {
discover_databricks_models(cfg).await.map(Some)
}
Provider::Maple => discover_maple_models(cfg).await.map(Some),
Provider::Anthropic | Provider::OpenAi | Provider::OpenRouter => Ok(None),
}
}

// ---------------------------------------------------------------------------
// Maple (OpenSecret)
// ---------------------------------------------------------------------------

/// Cap on one Maple catalog exchange, attestation handshake included. The
/// SDK's HTTP client sets no timeout, and this call sits on the `session/new`
/// path where a hang would block agent startup.
const MAPLE_DISCOVERY_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);

/// Discover the Maple (OpenSecret) model catalog: `GET /v1/models` through
/// the opensecret SDK.
///
/// The catalog is unreachable over plain HTTPS, so the desktop's picker
/// cannot probe it directly; it calls this via `buzz-agent models` or ACP
/// `session/new`.
///
/// Auth is the static `MAPLE_API_KEY`. An empty key is allowed: the enclave
/// lists models for any attested session, so the picker can fill in before
/// the user enters a key. An older enclave that still requires a credential
/// answers a keyless call with `Err(AgentError::LlmAuth)`.
async fn discover_maple_models(cfg: &Config) -> Result<Vec<ModelEntry>, AgentError> {
let client = crate::llm::build_maple_client(cfg)?.ok_or_else(|| {
AgentError::InvalidParams("discover_maple_models called for non-Maple provider".into())
})?;
let keyless = cfg.api_key.trim().is_empty();
let response = tokio::time::timeout(MAPLE_DISCOVERY_TIMEOUT, client.get_models())
.await
.map_err(|_| {
AgentError::Llm(format!(
"Maple model discovery timed out after {MAPLE_DISCOVERY_TIMEOUT:?}"
))
})?
.map_err(|error| maple_discovery_error(error, keyless))?;
let models = filter_maple_models(response.data.into_iter().map(|m| m.id));
if models.is_empty() {
return Err(AgentError::Llm(
"Maple model discovery returned no chat-capable models".into(),
));
}
Ok(models)
}

/// Maple's raw model ids as picker entries, dropping ids that cannot serve
/// chat traffic. The catalog has no display-name or task field, so the id is
/// also the label and the name is the only capability signal, as in
/// [`is_chat_capable_endpoint`].
fn filter_maple_models(ids: impl Iterator<Item = String>) -> Vec<ModelEntry> {
ids.filter(|id| is_maple_chat_model(id))
.map(|id| ModelEntry {
name: id.clone(),
id,
})
.collect()
}

fn is_maple_chat_model(id: &str) -> bool {
let lower = id.to_ascii_lowercase();
if !is_chat_capable_endpoint(id) {
return false;
}
!["embed", "whisper", "tts", "transcribe", "speech"]
.iter()
.any(|needle| lower.contains(needle))
}

/// Map an opensecret SDK discovery failure onto `AgentError`. 401/403 become
/// `LlmAuth` so `session/new` rejects with the credential error instead of
/// silently degrading to the configured model.
///
/// A keyless call rejected for auth means this enclave does not list models
/// without a credential. Word that as the standard `config: MAPLE_API_KEY
/// required` error so the frontend shows its usual key prompt.
fn maple_discovery_error(error: opensecret::Error, keyless: bool) -> AgentError {
match error {
opensecret::Error::Api {
status: status @ (401 | 403),
message,
} if keyless => AgentError::LlmAuth(format!(
"config: MAPLE_API_KEY required — this enclave does not list models without a credential (HTTP {status}: {message})"
)),
opensecret::Error::Api {
status: status @ (401 | 403),
message,
} => AgentError::LlmAuth(format!("Maple model discovery HTTP {status}: {message}")),
other => AgentError::Llm(format!("Maple model discovery failed: {other}")),
}
}

// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
Expand Down Expand Up @@ -728,4 +837,73 @@ mod tests {
assert!(!is_chat_capable_endpoint("databricks-gte-large-en"));
assert!(!is_chat_capable_endpoint("databricks-qwen3-embedding-0-6b"));
}

/// Providers without an agent-side catalog resolve to `Ok(None)`; the
/// dispatch must not error for them and must not return an empty `Some`.
#[tokio::test]
async fn discover_models_returns_none_for_providers_without_live_catalog() {
for provider in [Provider::Anthropic, Provider::OpenAi, Provider::OpenRouter] {
let cfg =
Config::for_discovery(provider, "key".into(), "https://example.invalid".into());
let discovered = discover_models(&cfg)
.await
.expect("no-catalog providers must not error");
assert!(discovered.is_none(), "{provider:?} has no live catalog");
}
}

/// A keyless listing rejected for auth is reported as the standard
/// missing-key config error so the desktop shows its "enter an API key"
/// prompt; the same rejection with a key present names the HTTP status.
#[test]
fn maple_discovery_auth_rejection_wording_depends_on_keyless() {
let rejected = || opensecret::Error::Api {
status: 401,
message: "Invalid JWT".into(),
};
match maple_discovery_error(rejected(), true) {
AgentError::LlmAuth(s) => assert!(s.contains("config: MAPLE_API_KEY required"), "{s}"),
other => panic!("expected LlmAuth, got {other:?}"),
}
match maple_discovery_error(rejected(), false) {
AgentError::LlmAuth(s) => {
assert!(s.contains("HTTP 401"), "{s}");
assert!(!s.contains("MAPLE_API_KEY required"), "{s}");
}
other => panic!("expected LlmAuth, got {other:?}"),
}
// Non-auth failures are plain errors regardless of key presence.
assert!(matches!(
maple_discovery_error(opensecret::Error::Session("stale".into()), true),
AgentError::Llm(_)
));
}

/// The catalog is dynamic, so this pins the filter rules, not a
/// snapshot: audio, TTS, and embedding ids drop; everything else stays,
/// including audio-capable chat models. Order is preserved and the id is
/// also the label, since Maple has no name field.
#[test]
fn filter_maple_models_drops_non_chat_families() {
let kept = [
"llama3-3-70b",
"deepseek-v4-flash",
"gpt-oss-120b",
"voxtral-small-24b",
"some-future-model",
];
let dropped = [
"whisper-large-v3",
"voxtral-tts",
"nomic-embed-text",
"qwen3-embedding-0-6b",
"kokoro-speech",
"parakeet-transcribe",
];
let models =
filter_maple_models(kept.iter().chain(dropped.iter()).map(|id| id.to_string()));
let ids: Vec<&str> = models.iter().map(|m| m.id.as_str()).collect();
assert_eq!(ids, kept);
assert!(models.iter().all(|m| m.id == m.name));
}
}
Loading