|
| 1 | +# Everruns + Maple Runbook |
| 2 | + |
| 3 | +This is the fastest path to use the new Everruns telemetry export now. |
| 4 | + |
| 5 | +It sends `f ai everruns` traces to: |
| 6 | + |
| 7 | +- local Maple (dev visualization) |
| 8 | +- hosted Maple (shared/history visualization) |
| 9 | + |
| 10 | +## What gets exported |
| 11 | + |
| 12 | +When enabled, Flow exports: |
| 13 | + |
| 14 | +- `everruns.tool_call` spans for each `seq_*` tool execution |
| 15 | +- runtime spans such as: |
| 16 | + - `everruns.tool_call_requested` |
| 17 | + - `everruns.output_message_completed` |
| 18 | + - `everruns.turn_failed` |
| 19 | + |
| 20 | +## Prerequisites |
| 21 | + |
| 22 | +1. `seqd` is running and reachable at your socket (`/tmp/seqd.sock` by default). |
| 23 | +2. Everruns API is reachable (`http://127.0.0.1:9300/api` by default). |
| 24 | +3. You have Maple ingest keys for local and/or hosted endpoint. |
| 25 | + |
| 26 | +## 1) Configure env (now) |
| 27 | + |
| 28 | +From `~/code/flow`, set the endpoints + keys: |
| 29 | + |
| 30 | +```bash |
| 31 | +f env set SEQ_EVERRUNS_MAPLE_LOCAL_ENDPOINT=http://ingest.maple.localhost/v1/traces |
| 32 | +f env set SEQ_EVERRUNS_MAPLE_LOCAL_INGEST_KEY=maple_pk_local_xxx |
| 33 | +f env set SEQ_EVERRUNS_MAPLE_HOSTED_ENDPOINT=https://ingest.1focus.ai/v1/traces |
| 34 | +f env set SEQ_EVERRUNS_MAPLE_HOSTED_INGEST_KEY=maple_pk_hosted_xxx |
| 35 | +``` |
| 36 | + |
| 37 | +Optional tuning: |
| 38 | + |
| 39 | +```bash |
| 40 | +f env set SEQ_EVERRUNS_MAPLE_QUEUE_CAPACITY=4096 |
| 41 | +f env set SEQ_EVERRUNS_MAPLE_MAX_BATCH_SIZE=128 |
| 42 | +f env set SEQ_EVERRUNS_MAPLE_FLUSH_INTERVAL_MS=50 |
| 43 | +f env set SEQ_EVERRUNS_MAPLE_CONNECT_TIMEOUT_MS=400 |
| 44 | +f env set SEQ_EVERRUNS_MAPLE_REQUEST_TIMEOUT_MS=800 |
| 45 | +``` |
| 46 | + |
| 47 | +For optimized mirror (remote ClickHouse + durable local spool), also set: |
| 48 | + |
| 49 | +```bash |
| 50 | +f env set SEQ_CH_MODE=mirror |
| 51 | +f env set SEQ_CH_MEM_PATH=~/repos/ClickHouse/ClickHouse/user_files/seq_mem.jsonl |
| 52 | +f env set SEQ_CH_LOG_PATH=~/repos/ClickHouse/ClickHouse/user_files/seq_trace.jsonl |
| 53 | +``` |
| 54 | + |
| 55 | +## 2) Run with env injected |
| 56 | + |
| 57 | +Use `f env run` so runtime sees configured values: |
| 58 | + |
| 59 | +```bash |
| 60 | +f env run -- f ai everruns "open Safari and take a screenshot" |
| 61 | +``` |
| 62 | + |
| 63 | +If you already export envs another way, this also works: |
| 64 | + |
| 65 | +```bash |
| 66 | +f ai everruns "open Safari and take a screenshot" |
| 67 | +``` |
| 68 | + |
| 69 | +On startup, if telemetry is enabled, Flow prints: |
| 70 | + |
| 71 | +`maple dual-ingest telemetry enabled` |
| 72 | + |
| 73 | +## 3) Verify in Maple |
| 74 | + |
| 75 | +In Maple (local and hosted), filter by: |
| 76 | + |
| 77 | +- `service.name = seq-everruns-bridge` |
| 78 | + |
| 79 | +Look for span names: |
| 80 | + |
| 81 | +- `everruns.tool_call` |
| 82 | +- `everruns.tool_call_requested` |
| 83 | +- `everruns.output_message_completed` |
| 84 | + |
| 85 | +## Troubleshooting |
| 86 | + |
| 87 | +1. Error: `invalid SEQ_EVERRUNS_MAPLE_* configuration` |
| 88 | + - You set only endpoint or only key for local/hosted pair. |
| 89 | + - Fix by setting both or removing both for that pair. |
| 90 | +2. Everruns command works but no spans in Maple |
| 91 | + - Confirm ingest endpoint includes `/v1/traces`. |
| 92 | + - Confirm ingest key is valid for that endpoint. |
| 93 | + - Confirm you ran through `f env run -- ...` (or equivalent env injection). |
| 94 | +3. Temporary Maple outage |
| 95 | + - Tool execution continues. |
| 96 | + - Export is best-effort and non-blocking. |
0 commit comments