From f1826703d0dc6fda68a2cab1df249cd840beb86b Mon Sep 17 00:00:00 2001 From: justin-tahara Date: Wed, 1 Jul 2026 17:30:52 -0500 Subject: [PATCH 1/2] docs(admins): add Tracing setup guide for Braintrust/Langfuse --- admins/advanced_configs/tracing.mdx | 83 +++++++++++++++++++++++++++++ docs.json | 1 + 2 files changed, 84 insertions(+) create mode 100644 admins/advanced_configs/tracing.mdx diff --git a/admins/advanced_configs/tracing.mdx b/admins/advanced_configs/tracing.mdx new file mode 100644 index 00000000..936a762b --- /dev/null +++ b/admins/advanced_configs/tracing.mdx @@ -0,0 +1,83 @@ +--- +title: "Tracing" +description: "Send LLM call traces to observability platforms to monitor and evaluate your Onyx deployment" +icon: "chart-line" +--- + +Tracing forwards every LLM call Onyx makes to an external observability platform so you can +monitor latency, token usage, and cost, and evaluate response quality. Onyx supports +[Braintrust](https://www.braintrust.dev) and [Langfuse](https://langfuse.com). + + + Tracing is available on **self-hosted** deployments. It is not configurable on Onyx Cloud. + + +## Connect a provider + + + + Click your user profile icon, select **Admin Panel**, and open the **Tracing** tab under + the **Usage** section of the sidebar. + + + + You can connect either or both providers — traces are sent to every connected provider. + + + + Create an API key in the [Braintrust dashboard](https://www.braintrust.dev/app) under + **Settings → API Keys**. + + In Onyx, click **Connect** on the Braintrust card and fill in: + + - **API Key** — your Braintrust API key. + - **Project Name** *(optional)* — the Braintrust project traces are logged to. Defaults + to `Onyx`. + - **API URL** *(optional)* — only needed for self-hosted Braintrust or non-default + regions. Defaults to `https://api.braintrust.dev`. + + + + Create a key pair in your [Langfuse](https://cloud.langfuse.com) project settings under + **Setup → API Keys**. + + In Onyx, click **Connect** on the Langfuse card and fill in: + + - **Secret Key** — your Langfuse secret key (`sk-...`). + - **Public Key** — your Langfuse public key (`pk-...`). + - **API Base URL** *(optional)* — defaults to the EU region + (`https://cloud.langfuse.com`). Set this for the US region, another region, or a + self-hosted Langfuse instance. + + + + Onyx validates the credentials before saving. Once connected, the card shows **Connected** + and traces begin flowing within about 30 seconds — no restart required. + + + +## Disconnect a provider + +Open the provider card, click the disconnect button, and confirm. Onyx stops sending new traces +to that provider immediately. Traces already delivered are unaffected. + +## Configuring with environment variables + +Configuring tracing from the Admin Panel is the recommended approach, but Onyx also honors +environment variables for backwards compatibility. A provider configured through the UI takes +precedence over its environment variables. + +| Provider | Variable | Description | +| --- | --- | --- | +| Braintrust | `BRAINTRUST_API_KEY` | API key. Enables Braintrust tracing when set. | +| Braintrust | `BRAINTRUST_PROJECT` | Project name traces are logged to. Defaults to `Onyx`. | +| Braintrust | `BRAINTRUST_API_URL` | Custom API URL for self-hosted / non-default regions. | +| Langfuse | `LANGFUSE_SECRET_KEY` | Secret key. Required (with the public key) to enable Langfuse. | +| Langfuse | `LANGFUSE_PUBLIC_KEY` | Public key. Required (with the secret key) to enable Langfuse. | +| Langfuse | `LANGFUSE_HOST` | Base URL for a non-default region or self-hosted Langfuse. | + + + Providers configured through environment variables appear in the Tracing dashboard as + configured via environment. You can adopt them into UI-managed configuration by connecting the + provider from the Admin Panel. + diff --git a/docs.json b/docs.json index 78c53962..748d0341 100644 --- a/docs.json +++ b/docs.json @@ -296,6 +296,7 @@ "admins/advanced_configs/white_labeling", "admins/advanced_configs/llm_access_controls", "admins/advanced_configs/hook_extensions", + "admins/advanced_configs/tracing", "admins/getting_started/slack_bot_setup", "admins/miscellaneous/discord_bot" ] From 726d4aff026069623e49486e198d983ab0b6e71a Mon Sep 17 00:00:00 2001 From: justin-tahara Date: Wed, 1 Jul 2026 17:34:22 -0500 Subject: [PATCH 2/2] docs(admins): apply format_docs to tracing guide --- admins/advanced_configs/tracing.mdx | 42 +++++++++++++---------------- 1 file changed, 19 insertions(+), 23 deletions(-) diff --git a/admins/advanced_configs/tracing.mdx b/admins/advanced_configs/tracing.mdx index 936a762b..37f30486 100644 --- a/admins/advanced_configs/tracing.mdx +++ b/admins/advanced_configs/tracing.mdx @@ -4,9 +4,9 @@ description: "Send LLM call traces to observability platforms to monitor and eva icon: "chart-line" --- -Tracing forwards every LLM call Onyx makes to an external observability platform so you can -monitor latency, token usage, and cost, and evaluate response quality. Onyx supports -[Braintrust](https://www.braintrust.dev) and [Langfuse](https://langfuse.com). +Tracing forwards every LLM call Onyx makes to an external observability platform so you can monitor latency, +token usage, and cost, and evaluate response quality. Onyx supports [Braintrust](https://www.braintrust.dev) +and [Langfuse](https://langfuse.com). Tracing is available on **self-hosted** deployments. It is not configurable on Onyx Cloud. @@ -16,8 +16,8 @@ monitor latency, token usage, and cost, and evaluate response quality. Onyx supp - Click your user profile icon, select **Admin Panel**, and open the **Tracing** tab under - the **Usage** section of the sidebar. + Click your user profile icon, select **Admin Panel**, + and open the **Tracing** tab under the **Usage** section of the sidebar. @@ -25,47 +25,44 @@ monitor latency, token usage, and cost, and evaluate response quality. Onyx supp - Create an API key in the [Braintrust dashboard](https://www.braintrust.dev/app) under - **Settings → API Keys**. + Create an API key in the [Braintrust dashboard](https://www.braintrust.dev/app) under **Settings → API Keys**. In Onyx, click **Connect** on the Braintrust card and fill in: - **API Key** — your Braintrust API key. - **Project Name** *(optional)* — the Braintrust project traces are logged to. Defaults - to `Onyx`. + to `Onyx`. - **API URL** *(optional)* — only needed for self-hosted Braintrust or non-default - regions. Defaults to `https://api.braintrust.dev`. + regions. Defaults to `https://api.braintrust.dev`. - Create a key pair in your [Langfuse](https://cloud.langfuse.com) project settings under - **Setup → API Keys**. + Create a key pair in your [Langfuse](https://cloud.langfuse.com) project settings under **Setup → API Keys**. In Onyx, click **Connect** on the Langfuse card and fill in: - **Secret Key** — your Langfuse secret key (`sk-...`). - **Public Key** — your Langfuse public key (`pk-...`). - **API Base URL** *(optional)* — defaults to the EU region - (`https://cloud.langfuse.com`). Set this for the US region, another region, or a - self-hosted Langfuse instance. + (`https://cloud.langfuse.com`). Set this for the US region, another region, or a self-hosted Langfuse instance. - Onyx validates the credentials before saving. Once connected, the card shows **Connected** - and traces begin flowing within about 30 seconds — no restart required. + Onyx validates the credentials before saving. Once connected, + the card shows **Connected** and traces begin flowing within about 30 seconds — no restart required. ## Disconnect a provider -Open the provider card, click the disconnect button, and confirm. Onyx stops sending new traces -to that provider immediately. Traces already delivered are unaffected. +Open the provider card, click the disconnect button, and confirm. +Onyx stops sending new traces to that provider immediately. Traces already delivered are unaffected. ## Configuring with environment variables -Configuring tracing from the Admin Panel is the recommended approach, but Onyx also honors -environment variables for backwards compatibility. A provider configured through the UI takes -precedence over its environment variables. +Configuring tracing from the Admin Panel is the recommended approach, +but Onyx also honors environment variables for backwards compatibility. +A provider configured through the UI takes precedence over its environment variables. | Provider | Variable | Description | | --- | --- | --- | @@ -77,7 +74,6 @@ precedence over its environment variables. | Langfuse | `LANGFUSE_HOST` | Base URL for a non-default region or self-hosted Langfuse. | - Providers configured through environment variables appear in the Tracing dashboard as - configured via environment. You can adopt them into UI-managed configuration by connecting the - provider from the Admin Panel. + Providers configured through environment variables appear in the Tracing dashboard as configured via environment. + You can adopt them into UI-managed configuration by connecting the provider from the Admin Panel.